Email Sync

Configuration

Preparation

  • Keep sure table Setting is up to date.

  • Recommended: install the PHP mailparse extension:

    sudo apt-get install php-mailparse

    Without it, sender/recipient addresses in the RFC 5322 address (Display Name) form - e.g. Unix cron mails From: root@host (Cron Daemon) - are not parsed and the sender column stays empty. With the extension QFQ parses all address formats robustly. Reload PHP and restart the emailSync service after installing.

Create a new Typo3 page for IMAP configuration and add a tt-content with following content::

file=_emailSync

Use this page to manage all your IMAP accounts and folders.

QFQ Config

  • Extension > QFQ > Debug

    • imapLogMode: debug, verbose, error, none

    • imapLog: fileadmin/protected/qfqProject/log/imap.log

Account settings

Each IMAP account is one row in table Setting (type = credentials), managed via the _emailSync report form:

  • Host: IMAP server (incoming). A plain host name uses port 993 with implicit TLS. Append a port for other setups: imap.example.com:143 connects plain and upgrades via STARTTLS. Any other explicit port keeps implicit TLS. For Outlook/Office365 use outlook.office365.com.

  • Username: the login name the IMAP server expects - for OAuth2 accounts the full e-mail address / UPN of the mailbox.

  • Password: the account password, encrypted (AES-256) in Setting.view. For OAuth2 accounts (see below) enter oauth2token:<path to token file> instead of a password.

  • Folder: comma separated list of IMAP folders to sync, e.g. INBOX or INBOX,Sent.

OAuth2 accounts (Outlook / Office365)

Microsoft (and Google) disabled basic authentication (username/password) for IMAP - those accounts require OAuth2 (SASL XOAUTH2). QFQ authenticates them via a token file.

Register your own Azure app first. QFQ ships no built-in OAuth2 client id. Create an Azure AD app registration (public client with the device code flow enabled; API permissions IMAP.AccessAsUser.All, SMTP.Send and offline_access, consented by the tenant admin) and put its Application (client) id into imapOAuth2ClientId (Extension > QFQ > IMAP). Without a configured client id, OAuth2 accounts cannot authorize. A per-account client_id in the token file overrides the global setting.

  1. In the account form set the password to:

    oauth2token:/path/outside/webroot/<account>.token

    The directory must exist and be writable for the webserver user.

  2. Authorize once (browser login incl. MFA): in the _emailSync report click the key button of the account. QFQ runs an OAuth2 device-code flow, shows a login URL (https://microsoft.com/devicelogin) plus a one-time code; log in, then click Check login. The token file is created automatically (mode 0600, owned by the webserver user).

  3. Refresh is automatic: OAuth2TokenManager renews the access token via the OAuth2 refresh token when it expires (single POST, no user interaction) and rewrites the token file in place. Regular syncing keeps the refresh token rolling; re-authorization is only needed after ~90 days of inactivity or if the tenant revokes consent.

Note

The Azure app behind imapOAuth2ClientId must be consented by the tenant admin. Gmail is not supported via the button - use a Google app password and configure the account classically. See Documentation-develop/IMAP-EMAILSYNC.md for the full reference (own Azure app setup) and troubleshooting.

Shared mailboxes / functional addresses: two options. Either authorize (browser) as a user who has FullAccess to the shared mailbox and set the shared mailbox as username (delegated), or use an app-only account (own Azure app with IMAP.AccessAsApp, no personal login, not bound to a person) - see Documentation-develop/IMAP-EMAILSYNC.md → App-only accounts.

Running

The php script typo3conf/ext/qfq/Classes/External/email-sync.php syncs all mails from the configured accounts and folders into table EmailSync. Called via CLI without arguments it runs as a polling daemon; the following flags select a one-shot mode:

email-sync.php        # daemon: poll & sync forever (default on CLI)
email-sync.php -s     # sync once and exit
email-sync.php -l     # list all folders per account
email-sync.php -L     # list all folders and their mails per account

  • If a configured folder does not exist, all available folders are reported to imap.log.

  • A failing or transient error on one account/folder is logged and skipped - the other accounts keep syncing.

  • The sync is incremental and chunked (header-only walk in pages, full fetch only for mails not yet stored, deduplicated by messageId), so even very large mailboxes sync without hitting server/read limits.

  • Attachments (opt-in, ext_conf checkbox imapSyncAttachments, default off): files are stored under fileadmin/protected/qfqProject/emailSync/, their metadata as JSON in EmailSync.attachment - see Documentation-develop/IMAP-EMAILSYNC.md → Attachments.

Deployment (systemd)

Run email-sync.php as a service under a watchdog. QFQ ships both files in typo3conf/ext/qfq/Resources/Public/script/:

  • emailSync.service - systemd unit

  • emailSync.sh - watchdog that starts one daemon per instance, restarts on failure and alerts

Setup:

  1. Copy both to the host, e.g. /etc/scripts/emailSync.sh and /etc/systemd/system/emailSync.service (make the script executable).

  2. In emailSync.sh set SCRIPTS to the absolute path(s) of email-sync.php and adjust ADMIN_EMAIL / SERVICE_USER if needed.

  3. Enable it:

    systemctl daemon-reload && systemctl enable --now emailSync

Important

Run the service as the webserver user (User=www-data), not root. OAuth2 token files are created/refreshed by the web UI (mode 0600, owned by the webserver user) and the daemon must read/write the same files.

The _emailSync report shows the systemd state and the number of running daemon processes, and offers a Start / restart service button (needs a sudoers rule for systemctl restart emailSync - see the button’s error hint).

Note

Restart the daemon after changing the account configuration - credentials are read once at startup.