Perfecting the Self-Hosted Email Server

oemb1905selfhosted / sysadminLeave a comment
  • Perfecting the Self-Hosted Email Server
  • Jonathan Haack
  • Haack’s Networking
  • webmaster@haacksnetworking.org 

//mailserver//
//roundcube//

Latest Updates: https://wiki.haacksnetworking.org/doku.php?id=computing:roundcube
Latest Updates: https://wiki.haacksnetworking.org/doku.php?id=computing:mailserver

It’s been 5 years since I switched full time to my own email server. When I first built it, I was overtaken by the enormity of migrating all the emails, creating all the new accounts, and building two more servers for the other two domains I used for personal and family matters. It was a lot of work. As email started to come in to the new system, I saw evidence in pflogsumm of certain emails not arriving, and began to identify reasons. The primary reason was that certain companies were emailing me but were not punctuating my handle correctly. Without permission, they were capitalizing part or all of the handle, and those emails were failing. The largest secondary reason was because PTR record checking was failing, and many or these failures were mistakes postfix was making either because it incorrectly assessed PTR health from mails going through a relay, or because the upstream DNS simply failed at querying (yes, this happens). I immediately relaxed as many rules as possible, removed spam controls, removed reject/discard rules, etc., and ensured that as many emails as possible made it through the gates. Until I could refine the setup, I relied on email clients to do the spam filtering and training, which worked very well. Over time, the collection of all the problems together began to take its toll:

  • PTR fails wrongly on relays, pulling 127.0.0.1 instead of the fqdn, postfix rejects and user never sees the email
  • DNS fails and therefore PTR fails, and postfix rejects and user never sees the email
  • Senders ignore RFC 5321 and change your email’s punctuation, meaning the emails both don’t arrive to the user and the company gets a bounceback that’s their fault
  • The lack of a web interface and the frequency with which I setup new client workstations for myself, my family, and/or for clients began to be cumbersome

So, I split these up into chunkable tasks and tackled them one by one from November of 2024 through March of 2025. The first task I addressed was the fact that companies were ignoring RFC 5321. There was no way I could call the local State Farm rep and have them call corporate to fix their email tooling to not capitalize my email. Sure, I could try that … but we all know that won’t happen. Also, in researching the problem, it turns out the violation of this RFC has a long history and there are reasons for doing this. Imagine my Grandmother in the late 90s sending me an email and inadvertently capitalizing my name in Jonathan@aol.com, for example. Do we really want the end user to get bouncebacks for all these? What about impersonating people or phishing/social engineering? It began to make sense that we should not expect the wilds to honor this RFC, but rather should handle these “good faith mistakes” and/or “potential social engineering” of the spec on the server’s side. So, here’s what I located and did after a bit of research:

sudo nano /etc/postfix/main.cf<virtual_alias_maps = regexp:/etc/postfix/virtual_alias>

Then, add some regular expressions and aliases as follows:

sudo nano /etc/postfix/virtual_alias</^[Ss][Uu][Pp][Pp][Oo][Rr][Tt]@haacksnetworking.org/     support>

It’s imperative to add the domain at the end, otherwise any emails that begin with support that are sent from this server, for example support@google.com, will arrive in the alias’d inbox. I found this out the hard way. Good times. The fix is easy though. Just add a rule like this for each and every user and this regex will allow any punctuation of your name to arrive in your inbox, whether Support, suPPort, or SUPPORT, etc. Testing this was difficult as many email clients change your spelling on your behalf. Web-based Gmail worked well for sending and honoring the capitalization, while Thunderbird proved best for rendering the addresses as they came instead of altering them like Evolution did. A few weeks later, eTrade and State Farm emails that originated from their auto-mailers started arriving again. The next task was to figure out why my pflogsumm reports were empty at times. If I ran them manually, I could get the information I needed, but the cronjobs and rules I had in place to send me reports daily, were often empty due to some misconfiguration. So, I dug into the matter and it turned out to be due to the deprecation of rsyslog. Since rsyslog is no longer default in Bookworm, I was installing it manually and when doing so, it includes a log rotation rule in /etc/logrotate.d/rsyslog that was setup differently than the other log rotation rule I had setup on earlier and non-Bookworm instances. Having not realized this for some time, I kept focusing on my own log rotation rule/syntax, thinking wrongly that something was setup erroneously, despite it having worked for years prior. Here’s what to check and what to put in place to ensure timely and meaningul reports:

sudo apt install pflogsummsudo apt install rsyslogsudo nano /etc/logrotate.d/rsyslog<#/var/log/mail.log>sudo nano /etc/logrotate.d/postfix-log

In that file, enter the following rule:

/var/log/mail.log {
  missingok
  daily
  rotate 7
  create
  compress
  start 0
}

Now, we just need to craft a script that will unzip and rezip the rotated and historical logs that matches the naming convention we’ve specified here:

#!/bin/sh 
#/usr/sbin/logrotate -f /etc/logrotate.d/postfix-log 
gunzip /var/log/mail.log.0.gz 
/usr/sbin/pflogsumm /var/log/mail.log.0 --problems-first --rej-add-from --verbose-msg-detail -q | mail -s "[pflog-lastlog]-$(hostname -f)-$(date)" email@email.com 
gzip /var/log/mail.log.0 
sleep 2s 
systemctl restart rsyslog 
systemctl restart postfix 
systemctl restart dovecot 
exit 0

(I include the second and commented line just to remind myself how to rotate logs manually for a specific rule.) So now that the conflicting log rotation is removed and the script syntax matches the log naming convention, no reports will be empty. I then setup a cronjob as follows:

sudo crontab -e30 12 * * * /bin/bash /usr/local/bin/pflog-run.sh >> /home/logs/pflog-run.log

Now, each day at 30 past noon I get my email log reports from the servers I manage and can see what was delivered, what bounced, who was rejected etc., and not have to secure shell in, locate the correct log.gz and run the report manually. This is super helpful! On or around this time, we had a local GLUG meeting and my colleague was giving me shit for not having server-side spam rules in functioning order and instead relying on Evolution running 24-7 to handle spam for me. Evolution inspects the emails itself, and has some built in x-spam-flag and other rules whereby it makes a determination. It also has a database and responds to training. It worked very well, but ultimately, I agreed with him. It was time to reconfigure my spam assassin rules, resolve all prior issues and get everything working correctly … and making sure that no emails are ever rejected, but rather just sent to the spam folder. I can always white or black list as needed for individual cases if/when needed. But, I needed tooling that was server-side that would do the lion’s share of separating ham from spam. In addition to not wanting client email lost, I also personally had an email dating from 2004ish that simply got a lot of spam. It was not practical to change it as it was in widespread use for nearly every online account I have and need for daily life. So, this was also personal, and I knew it would be a source of pride if I could manage the spam myself instead of relying on Evolution. I had done it before, just needed to suck it up and do it better this time. Here’s what I did:

sudo apt install dovecot-sieve dovecot-managesieved spamassassin spamc spamass-milter postfix-pcre
sudo nano /etc/dovecot/dovecot.conf
<protocols = imap lmtp sieve>
sudo nano /etc/dovecot/conf.d/15-lda.conf

In that file, enter the following:

protocol lda {
  mail_plugins = $mail_plugins sieve
}

Then, in /etc/dovecot/conf.d/20-lmtp.conf, enter the following

protocol lmtp {
  mail_plugins = quota sieve
}

Let’s make sure postfix (not just spam assassin) can also inspect the headers and email body if we so choose. 

sudo nano /etc/postfix/main.cf
<header_checks = pcre:/etc/postfix/header_checks>
<body_checks = pcre:/etc/postfix/body_checks>
sudo nano /etc/postfix/header_checks
</free mortgage quote/    REJECT>
sudo nano /etc/postfix/body_checks
</free mortgage quote/    DISCARD>
sudo postmap /etc/postfix/body_checks
sudo postmap /etc/postfix/header_checks

Adjust the files as you see fit. Do note that this is postfix doing the work, not spam assassin and that whatever you discard or reject will never make it to spam assassin or dovecot. For that reason, I keep these as empty for now, but if/when I identify a regular and predictable header or body that I can with certainty reject or discard, I will be able to do so. For now, however, I choose not to use it and instead make sure spam assassin does all the inspecting and routing to the spam folder. Make sure spam assassin is added as a milter in your main.cf file. I use opendkim and opendmarc so mine looks like follows:

sudo nano /etc/postfix/main.cf
<smtpd_milters = local:opendkim/opendkim.sock,local:opendmarc/opendmarc.sock,local:spamass/spamass.sock>

Next, take a look at this file:

sudo nano /etc/default/spamass-milter

Once again, I do not prefer reject/discard behavior, so unlike most tutorials recommend, I leave this file untouched and/or make sure that the #Reject emails with spamassassin scores > 15 is commented out. After verifying that, I now set up a spam sieve rule to run before email arrives to dovecot. This means the server receives the incoming email, hands it over to spam assassin via lda/lmtp, and then spamassassin sends it over to dovecot where it lands in your inbox. So, to do this in this order, we need to alert dovecot that we want sieve and/or spamassassin to inspect the emails first. 

sudo nano /etc/dovecot/conf.d/90-sieve.conf
<sieve_before = /var/mail/SpamToJunk.sieve>
sudo nano /var/mail/SpamToJunk.sieve

In the sieve file that we created, enter something like what I have below. Warning: make sure the folder you route spam to is set to autocreate = yes in your dovecot configuration and/or exists, otherwise this sieve rule will fail.

require "fileinto";
if header :contains "X-Spam-Flag" "YES"
{
  fileinto "Junk";
  stop;
}

This rule does something simple. That is, spam assassin inspects the email according to the rules specified in sudo nano /etc/spamassassin/local.cf and applies headers to the email, culminating in a final determination of “yes or no” that’s applied to the X-Spam-Flag header. Thus, all that matters in determining whether something is spam or not, is whether it says yes or no at the end of the day. Now, whether the determinations being made are correct and/or valid is entirely different. That’s of course also important and that hinges upon creating healthy scores and rules for spamassassin to leverage. The first thing to note is that spam assassin won’t be able to query RBLs unless you run your own recursive DNS server. Secondly, it’s also important to note that spam assassin is nowadays configured out of the box to query a meta RBL list all on its own, so ignore the tutorials that provide complex configs to add these – those are no longer current or needed. First, I edit the basic information, and then I setup DNS and scoring.

sudo nano /etc/spamassassin/local.cf
<report_contact webmaster@domain.com>
<required_score 5.0> [or your preference, 5 is good start though]
<#rewrite_header Subject **Possible Spam**> [I don't like, comment out]
<report_safe 0> [Set to 0, terrible for false positives otherwise]

After that, let’s install a simple recursive DNS server, called unbound with sudo apt install unbound and then sudo nano /etc/unbound/unbound.conf and add something like:

server:
    interface: 127.0.0.1
    cache-max-ttl: 14400
    cache-min-ttl: 1200
    num-threads: 4 
    msg-cache-slabs: 8
    rrset-cache-slabs: 8
    infra-cache-slabs: 8
    key-cache-slabs: 8
    rrset-cache-size: 256m
    msg-cache-size: 128m
    #prefetch: yes
    harden-dnssec-stripped: yes
    use-syslog: yes
    aggressive-nsec: yes
    hide-identity: yes
    hide-version: yes
    use-caps-for-id: yes
    do-tcp: yes
    do-udp: yes

Make sure to add the configuration after the include-toplevel: /etc/unbound/unbound.conf.d/*.conf line. Now, let’s make sure that spam assassin knows which DNS server to use to conduct its queries and enter some common symbolic headers and scores:

sudo apt install unbound
sudo nano /etc/spamassassin/local.cf
<dns_server 127.0.0.1>
<score MISSING_FROM                      5.0>
<score MISSING_DATE                      5.0>
<score MISSING_HEADERS                   3.0>
<score PDS_FROM_2_EMAILS                 3.0>
<score FREEMAIL_FORGED_REPLYTO           3.5>
<score DKIM_ADSP_NXDOMAIN                5.0>
<score FORGED_GMAIL_RCVD                 2.5>
<score FREEMAIL_FORGED_FROMDOMAIN        3.0>
<score HEADER_FROM_DIFFERENT_DOMAINS     3.0>
<score FREEMAIL_FROM                     3.0>
<score ACCT_PHISHING                     3.0>
<score AD_PREFS                          3.0>
<score ADMAIL                            3.0>
<score ADMITS_SPAM                       3.0>
<score CONFIRMED_FORGED                  3.0>
<score FROM_PAYPAL_SPOOF                 3.0>
<score SPF_SOFTFAIL                      2.0>
<score SPF_FAIL                          5.0>
<score DMARC_MISSING                     2.0>
<score DMARC_NONE                        2.0>
<score RDNS_NONE                         5.0>
<score RDNS_DYNAMIC                      2.0>
<score RCVD_IN_SBL_CSS                   5.0>
<score RCVD_IN_MSPIKE_H2                 5.0>
<score RCVD_IN_PBL                       5.0>
<score PDS_DBL_URL_LINKBAIT              5.0>
<score URIBL_DBL_SPAM                    5.0>
<score URIBL_BLACK                       5.0>
<score RCVD_IN_VALIDITY_RPBL             5.0>
<score URIBL_ABUSE_SURBL                 5.0>
<score URIBL_SBL_A                       5.0>
<score RCVD_IN_DNSWL_NONE                5.0>
<score KHOP_HELO_FCRDNS                  2.0>
<score RCVD_IN_SBL                       5.0>
<score RCVD_IN_BL_SPAMCOP_NET            5.0>
<score URIBL_PH_SURBL                    5.0>
<score RCVD_IN_MSPIKE_L5                 5.0>
<score RCVD_IN_MSPIKE_BL                 5.0>
<score URIBL_GREY                        2.0>
<score URIBL_CSS_A                       5.0>

Make sure to add this to the bottom of the file. The first line above specifies to use the newly installed unbound DNS server, which was installed directly on the host. Do make sure that port 53 and/or port 5335 are not publicly exposed and/or you will be providing recursive DNS for the world. So long as those rules are in place, that simple line instructs spam assassin to use unbound for its queries, and now any querying it does of the email against the RBLs will function. Also, the example scores I have above were obtained from online research, but/and also from inspecting email headers after spam assassin is functioning. Let it run for a week or so, catch some spam, and then you can inspect the full headers and copy/paste the spam assassin symbolic header that need score tweaking, etc. In addition to fine tuning spam assassin’s scoring mechanism, you can also white and black list emails as follows in the same config file:

whitelist_from example@example.com
whitelist_from *@example.com
blacklist_from badperson@baddomain.com
blacklist_from *@baddomain.com

You don’t want to play wack-a-mole, so it’s imperative that you only use whitelisting for mission-critical emails that must arrive. If at all practical or possible, don’t whitelist them at first. If they wind up in spam, look at the headers and see why. There might be DNS health related issues causing this, which can be fixed. In my case, I learned I had forgotten to setup dmarc for an entire domain. The health was otherwise correct, so these emails never failed to arrive, but as soon as I turned on the filter, they landed in spam and alerted me to the DNS oversight. So, this tool, with reject/discard turned off, serves as an amazing email health auditing tool. In addition to limiting whitelist use, you also want to limit blacklist use for practical reasons. That is, the goal is not shell in to your server and black spam one by one, but rather to let sieve and spam assassin do the work for you according to the scores and values you’ve created. That’s the whole point. Only use the blacklist feature for advanced marketing scams and/or high level spammers who pass spf, pass dmarc, pass dkim, etc. Yes, those spammers require a black list because they’ve gone to a great extent to make their spam look like ham. Now that this was all in place, I needed to set up a simple Roundcube instance for these servers. Although its relatively easy for me to setup Thunderbird, it proves harder for the family (1) and is a non-starter for clients (2), both of which I began to build these for as soon as the core recipe was functional. So, this also had become somewhat urgent. After looking online and at the Roundcube resources, I whipped up the following:

cd /var/www
wget https://github.com/roundcube/roundcubemail/releases/download/1.6.1/roundcubemail-1.6.1-complete.tar.gz
tar xvf roundcubemail-1.6.1-complete.tar.gz
ln -s roundcubemail-1.6.1/ roundcube
chown root:root -R roundcube
cd roundcube    
sudo chown www-data:www-data temp/ logs/ -R
sudo apt install software-properties-common php-net-ldap2 php-net-ldap3 php-imagick php8.2-fpm php8.2-common php8.2-gd php8.2-imap php8.2-mysql php8.2-curl php8.2-zip php8.2-xml php8.2-mbstring php8.2-bz2 php8.2-intl php8.2-gmp php8.2-redis

This setups the web root, makes it a symlink to the latest release which is convenient for upgrades as all you do is delete the link, make a new link and copy over the config and set perms and you are upgraded! Next, set up the database:

sudo mysql -u root
CREATE DATABASE roundcube DEFAULT CHARACTER SET utf8 COLLATE utf8_general_ci;
CREATE USER roundcube@localhost IDENTIFIED BY 'password';
GRANT ALL PRIVILEGES ON roundcube.* TO roundcube@localhost;
FLUSH PRIVILEGES;
EXIT;

I would make sure that this pass is 20 characters’ish and also make sure your permissions are correct. So long as you keep apache defaults and don’t mess with any of the rewrites, things should be plenty secure. Once the database is done, import the tables that Roundcube expects to see:

sudo mysql -u root -p roundcube < /var/www/roundcube/SQL/mysql.initial.sql

After that, it’s time to make sure you have both an http and https virtual host configuration in place. Obviously, this presumes you already have a LAMP and/or equivalent stack setup, and are familiar with how to cut and generate Let’s Encrypt keys. If so, I advise using a simple virtual host with no reverse proxy in place to cut the keys. Once that’s done, go edit the virtual hosts and replace the default contents with something like this for http:

sudo nano /etc/apache2/sites-enabled/mail.domain.com.conf
<VirtualHost *:80>
      ServerName mail.domain.com
      ServerAdmin email@email.com
      DocumentRoot /var/www/roundcube/
      ErrorLog ${APACHE_LOG_DIR}/roundcube_error.log
      CustomLog ${APACHE_LOG_DIR}/roundcube_access.log combined
      <Directory />
              Options FollowSymLinks
              AllowOverride All
      </Directory>
      <Directory /var/www/roundcube/>
              Options FollowSymLinks MultiViews
              AllowOverride All
              Order allow,deny
              Allow from all
      </Directory>
      <FilesMatch ".+\.ph(ar|p|tml)$">
              SetHandler "proxy:unix:/run/php/php8.2-fpm.sock|fcgi://localhost"
      </FilesMatch>
      RewriteEngine on
      RewriteCond %{SERVER_NAME} =mail.domain.com
      RewriteRule ^ https://%{SERVER_NAME}%{REQUEST_URI} [END,NE,R=permanent]
</VirtualHost>

For your https virtual host, something like this:

sudo nano /etc/apache2/sites-enabled/mail.domain.com-ssl.conf
<IfModule mod_ssl.c>
<VirtualHost *:443>
      ServerName mail.domain.com
      ServerAdmin email@email.com
      DocumentRoot /var/www/roundcube/
      ErrorLog ${APACHE_LOG_DIR}/roundcube_error.log
      CustomLog ${APACHE_LOG_DIR}/roundcube_access.log combined
      <Directory />
              Options FollowSymLinks
              AllowOverride All
      </Directory>
      <Directory /var/www/roundcube/>
              Options FollowSymLinks MultiViews
              AllowOverride All
              Order allow,deny
              Allow from all
      </Directory>
      <FilesMatch ".+\.ph(ar|p|tml)$">
              SetHandler "proxy:unix:/run/php/php8.2-fpm.sock|fcgi://localhost"
      </FilesMatch>
      SSLCertificateFile /etc/letsencrypt/live/mail.domain.com/fullchain.pem
      SSLCertificateKeyFile /etc/letsencrypt/live/mail.domain.com/privkey.pem
</VirtualHost>
</IfModule>

So long as the keys were already cut on the default virtual host prior, this makes things very smooth and there’s no need to tinker around with cutting them as stand-alone first, etc. I find it to be a quick work around for anything requiring reverse proxy virtual hosts. Before enabling the virtual hosts and restarting apache, let’s configure Roundcube:

cd /var/www/roundcube/config/
sudo cp config.inc.php.sample config.inc.php
sudo nano config.inc.php
<$config['db_dsnw'] = 'mysql://roundcube:pass@localhost/roundcube';>
<$config['des_key'] = 'rcmail-!24ByteDESkey*Str';? 
[retain des_key string length and make it unique]
<$config['imap_host'] = 'startls://domain.com:143';>
<$config['smtp_host'] = 'tls://mail.domain.com:587';>
<$config['enable_spellcheck'] = true;>
<$config['plugins'] = [
  'archive',
  'zipdownload',
  'acl',
  'additional_message_headers',
  'attachment_reminder',
  'autologon',
  'debug_logger',
  'emoticons',
  //'enigma',//
  'filesystem_attachments',
  'help',
  'hide_blockquote',
  'http_authentication',
  'identicon',
  'identity_select',
  'jqueryui',
  'krb_authentication',
  'managesieve',
  'markasjunk',
  'new_user_dialog',
  'new_user_identity',
  'newmail_notifier',
  'password',
  'reconnect',
  'redundant_attachments',
  'show_additional_headers',
  'squirrelmail_usercopy',
  'subscriptions_option',
  'userinfo',
  'vcard_attachments',
  'virtuser_file',
  'virtuser_query'
];>

After configuring Roundcube, let’s enable the virtual hosts with a2ensite mail.domain.com.conf and a2ensite mail.domain.com-ssl.conf and then systemctl restart apache2. After this, double check permissions, reboot, and you should now have access in your web browser at the specified fqdn. Again, all one has to do to upgrade, is download the latest release, remove the old symlink and replace it with the new one, copy over the config file from the old one, set permissions, restart services and you are all set.

The last 6 months was a fun and busy time, with tons of work to do at all three of my jobs, but taking side time to bang these things out was super helpful. The few vendors whose email never arrived was now sorted. Emails never arriving in error due to rejects was now sorted. Spam rules that get executed on the server and that make sense and help with auditing DNS health are now in place. Unbreakable reports and emails from pflogsumm to verify things works as expected and continues to alert me to issues and misconfigurations so that I can fix them. The addition of a simple web GUI with simple back end UNIX user names, provides easy end user access both for family and clients.

Good times, but my bromance with email servers has worn on me. It’s now time to just let them work … at least, until Trixie goes live and breaks everything.

Kindly, oemb1905

UPDATE: Forgot to add that all reject rules for PTR, etc., in the smtpd sender (incoming) postfix block were removed. Only reject rules for SASL and/or unauthenticated users (trying to send through my server) were retained. This means that there’s no possibility that postfix will reject an email erroneously. Here’s what I changed the main.cf block too:

smtpd_sender_restrictions =
 permit_mynetworks,
 permit_sasl_authenticated,
 #reject_unknown_reverse_client_hostname,
 #reject_unknown_client_hostname,
 #reject_unknown_sender_domain,
 reject_unauthenticated_sender_login_mismatch,
 reject_sender_login_mismatch,
 permit

Leave a Reply

Your email address will not be published. Required fields are marked *

Close