Your Email, Your Rules: Self-Hosting Simplified

Your Email, Your Rules: Self-Hosting Simplified
Jonathan Haack
gnulinux.studio
webmaster@haacksnetworking.org

//mailserver//

Latest Updates: https://wiki.haacksnetworking.org/doku.php?id=computing:mailserver

Contrary to popular belief, it’s entirely possible to self-host email servers. This tutorial is for Debian users who want to give this a try. In this tutorial, I used an 8-core VM with 8GB of RAM on a SuperMicro host with 48 physical cores. I chose postfix for delivery, dovecot for IMAP/LMTP, and a LAMP stack for Roundcude serving. The setup includes SPF, DKIM, DMARC, SpamAssassin, and log monitoring. It’s designed for small teams (20-100 users) who want reliable and custom domain branded email. Using simple UNIX users names is a design choice, not a limitation or a bug. Before we begin, make sure you know how to set up a basic LAMP stack; feel free to use my Apache Survival guide.

NOTE: You need to have PTR access.

Once you got a sufficiently hardened VM/VPS up for which you have reverse DNS and/or PTR access setup, let’s get started:

sudo apt update && sudo apt upgrade -y
sudo apt install mailutils postfix ufw fail2ban nginx apache2 php8.4-fpm php8.4-mysql php8.4-curl php8.4-gd php8.4-mbstring php8.4-xml php8.4-zip dovecot-core dovecot-imapd dovecot-lmtpd

Next, we edit the hosts file:

sudo nano /etc/hosts
127.0.1.1 yourdomain.com yourdomain
127.0.0.1 mail.yourdomain.com localhost

Install Postfix and Mailutils:
sudo apt-get install mailutils postfix -y

Pick “Internet Site” and set your domain to yourdomain.com. Install UFW and open ports:
sudo apt install ufw sudo ufw allow 22/tcp sudo ufw allow 25/tcp sudo ufw allow 587/tcp sudo ufw allow 143/tcp sudo ufw allow 465/tcp sudo ufw allow 993/tcp sudo ufw allow 80 sudo ufw allow 443 sudo ufw enable

Increase message size:
sudo postconf -e message_size_limit=52428800

Edit /etc/postfix/main.cf:
sudo nano /etc/postfix/main.cf
myhostname = mail.yourdomain.com myorigin = /etc/mailname mydestination = yourdomain.com, $myhostname, localhost.$mydomain, localhost mailbox_size_limit = 0 inet_protocols = ipv4 message_size_limit = 52428800

Edit /etc/aliases:
sudo nano /etc/aliases
postmaster: root root: yourusername
sudo newaliases

Set up Nginx for certs:
sudo nano /etc/nginx/conf.d/mail.yourdomain.com.conf
server {
listen 80; server_name mail.yourdomain.com; root /usr/share/nginx/html/;
location ~ /.well-known/acme-challenge {
allow all;
}
}
sudo systemctl reload nginx

Install Certbot:
sudo apt install certbot letsencrypt python3-certbot-apache sudo certbot --authenticator standalone --installer apache -d domain.com --pre-hook "systemctl stop apache2" --post-hook "systemctl start apache2"

Edit /etc/postfix/master.cf for TLS/SASL:
sudo nano /etc/postfix/master.cf
submission inet n - y - - smtpd -o syslog_name=postfix/submission -o smtpd_tls_security_level=encrypt -o smtpd_tls_wrappermode=no -o smtpd_sasl_auth_enable=yes -o smtpd_relay_restrictions=permit_sasl_authenticated,reject -o smtpd_recipient_restrictions=permit_mynetworks,permit_sasl_authenticated,reject -o smtpd_sasl_type=dovecot -o smtpd_sasl_path=private/auth
smtps inet n - y - - smtpd -o syslog_name=postfix/smtps -o smtpd_tls_wrappermode=yes -o smtpd_sasl_auth_enable=yes -o smtpd_relay_restrictions=permit_sasl_authenticated,reject -o smtpd_recipient_restrictions=permit_mynetworks,permit_sasl_authenticated,reject -o smtpd_sasl_type=dovecot -o smtpd_sasl_path=private/auth

Add to /etc/postfix/main.cf for TLS:
sudo nano /etc/postfix/main.cf
smtpd_tls_cert_file=/etc/letsencrypt/live/mail.yourdomain.com/fullchain.pem smtpd_tls_key_file=/etc/letsencrypt/live/mail.yourdomain.com/privkey.pem smtpd_tls_security_level=may smtpd_tls_loglevel = 1 smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache smtp_tls_security_level = may smtp_tls_loglevel = 1 smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtpd_tls_mandatory_protocols = !SSLv2, !SSLv3, !TLSv1, !TLSv1.1 smtpd_tls_protocols = !SSLv2, !SSLv3, !TLSv1, !TLSv1.1 smtp_tls_mandatory_protocols = !SSLv2, !SSLv3, !TLSv1, !TLSv1.1 smtp_tls_protocols = !SSLv2, !SSLv3, !TLSv1, !TLSv1.1 smtpd_sasl_auth_enable = yes smtpd_sasl_type = dovecot smtpd_sasl_path = private/auth smtpd_sasl_security_options = noanonymous

Install Dovecot:
sudo apt install dovecot-core dovecot-imapd dovecot-lmtpd

Edit /etc/dovecot/dovecot.conf:
sudo nano /etc/dovecot/dovecot.conf
protocols = imap lmtp

Edit /etc/dovecot/conf.d/10-mail.conf:
sudo nano /etc/dovecot/conf.d/10-mail.conf
mail_location = maildir:~/Maildir

Add groups:
sudo adduser dovecot mail
sudo adduser yourusername mail

Edit /etc/dovecot/conf.d/10-master.conf for LMTP:
sudo nano /etc/dovecot/conf.d/10-master.conf

service lmtp {
  unix_listener /var/spool/postfix/private/dovecot-lmtp {
    mode = 0600
    user = postfix
    group = postfix
  }
}

Add to /etc/postfix/main.cf for LMTP:
sudo nano /etc/postfix/main.cf
mailbox_transport = lmtp:unix:private/dovecot-lmtp
smtputf8_enable = no

Edit /etc/dovecot/conf.d/10-auth.conf:
sudo nano /etc/dovecot/conf.d/10-auth.conf
disable_plaintext_auth = yes auth_username_format = %n auth_mechanisms = plain login

Edit /etc/dovecot/conf.d/10-ssl.conf:
sudo nano /etc/dovecot/conf.d/10-ssl.conf
ssl = required ssl_cert = </etc/letsencrypt/live/mail.yourdomain.com/fullchain.pem ssl_key = </etc/letsencrypt/live/mail.yourdomain.com/privkey.pem ssl_prefer_server_ciphers = yes ssl_min_protocol = TLSv1.2

Add SASL listener in /etc/dovecot/conf.d/10-master.conf:
sudo nano /etc/dovecot/conf.d/10-master.conf

service auth {
  unix_listener /var/spool/postfix/private/auth {
    mode = 0660
    user = postfix
    group = postfix
  }
}

Test handshakes:
openssl s_client -connect mail.yourdomain.com:465
openssl s_client -starttls smtp -connect mail.yourdomain.com:25

Install SPF policy:
sudo apt install postfix-policyd-spf-python

Edit /etc/postfix/master.cf:
sudo nano /etc/postfix/master.cf
policyd-spf unix - n n - 0 spawn
user=policyd-spf argv=/usr/bin/policyd-spf

Add to /etc/postfix/main.cf:
sudo nano /etc/postfix/main.cf
policyd-spf_time_limit = 3600
smtpd_recipient_restrictions = permit_mynetworks, permit_sasl_authenticated, reject_unauth_destination, check_policy_service unix:private/policyd-spf

Edit /etc/postfix-policyd-spf-python/policyd-spf.conf:
nano /etc/postfix-policyd-spf-python/policyd-spf.conf
HELO_reject = False
Mail_From_reject = False

Install DKIM:
sudo apt install opendkim opendkim-tools
sudo adduser postfix opendkim

Edit /etc/opendkim.conf:
sudo nano /etc/opendkim.conf

Canonicalization relaxed/simple
Mode sv
SubDomains no
Nameservers 8.8.8.8,1.1.1.1
KeyTable refile:/etc/opendkim/key.table
SigningTable refile:/etc/opendkim/signing.table
ExternalIgnoreList /etc/opendkim/trusted.hosts
InternalHosts /etc/opendkim/trusted.hosts

Create keys:
sudo mkdir -p /etc/opendkim/keys sudo chown -R opendkim:opendkim /etc/opendkim sudo chmod 711 /etc/opendkim/keys sudo mkdir /etc/opendkim/keys/yourdomain.com sudo opendkim-genkey -b 2048 -d yourdomain.com -D /etc/opendkim/keys/yourdomain.com -s default -v sudo chown opendkim:opendkim /etc/opendkim/keys/yourdomain.com/default.private sudo chmod 600 /etc/opendkim/keys/yourdomain.com/default.private

Edit /etc/opendkim/signing.table:
sudo nano /etc/opendkim/signing.table
'*@yourdomain.com default._domainkey.yourdomain.com'
'*@*.yourdomain.com default._domainkey.yourdomain.com'

Edit /etc/opendkim/key.table:
sudo nano /etc/opendkim/key.table
default._domainkey.yourdomain.com yourdomain.com:default:/etc/opendkim/keys/yourdomain.com/default.private

Edit /etc/opendkim/trusted.hosts:
sudo nano /etc/opendkim/trusted.hosts
127.0.0.1 localhost .yourdomain.com

Test:
sudo opendkim-testkey -d yourdomain.com -s default -vvv

Add TXT record from:
sudo cat /etc/opendkim/keys/yourdomain.com/default.txt

Socket:
sudo mkdir /var/spool/postfix/opendkim sudo chown opendkim:postfix /var/spool/postfix/opendkim

In /etc/opendkim.conf:
sudo nano /etc/opendkim.conf

Socket local:/var/spool/postfix/opendkim/opendkim.sock

In /etc/default/opendkim:
sudo nano /etc/default/opendkim
SOCKET="local:/var/spool/postfix/opendkim/opendkim.sock"

In /etc/postfix/main.cf:
sudo nano /etc/postfix/main.cf
milter_default_action = accept milter_protocol = 6 smtpd_milters = local:opendkim/opendkim.sock non_smtpd_milters = $smtpd_milters

Install DMARC:
sudo apt install opendmarc

Edit /etc/opendmarc.conf:
sudo nano /etc/opendmarc.conf
AuthservID OpenDMARC TrustedAuthservIDs mail.yourdomain.com RejectFailures false IgnoreAuthenticatedClients true RequireHeaders true SPFSelfValidate true Socket local:/var/spool/postfix/opendmarc/opendmarc.sock

Socket:
sudo mkdir -p /var/spool/postfix/opendmarc sudo chown opendmarc:opendmarc /var/spool/postfix/opendmarc -R sudo chmod 750 /var/spool/postfix/opendmarc/ -R sudo adduser postfix opendmarc sudo systemctl restart opendmarc

In /etc/postfix/main.cf:
sudo nano /etc/postfix/main.cf

milter_default_action = accept
milter_protocol = 6
smtpd_milters = local:opendkim/opendkim.sock,local:opendmarc/opendmarc.sock
non_smtpd_milters = $smtpd_milters

Auto-folders in /etc/dovecot/conf.d/15-mailboxes.conf:
sudo nano /etc/dovecot/conf.d/15-mailboxes.conf

mailbox Drafts {
  auto = create
  special_use = \Drafts
}

Aliases in /etc/postfix/main.cf:
sudo nano /etc/postfix/main.cf
virtual_alias_maps = regexp:/etc/postfix/virtual_alias

In /etc/postfix/virtual_alias:
sudo nano /etc/postfix/virtual_alias
^[Jj][Oo][Nn][Aa][Tt][Hh][Aa][Nn]@yourdomain.com/ jonathan ^[Ww][Ee][Bb][Mm][Aa][Ss][Tt][Ee][Rr]@yourdomain.com/ webmaster
postmap /etc/postfix/virtual_alias

Rejects in /etc/postfix/main.cf:
sudo nano /etc/postfix/main.cf
smtpd_sender_restrictions = permit_mynetworks, permit_sasl_authenticated, reject_sender_login_mismatch permit

For patterns:
sudo apt install postfix-pcre

In /etc/postfix/main.cf:
sudo nano /etc/postfix/main.cf
header_checks = pcre:/etc/postfix/header_checks body_checks = pcre:/etc/postfix/body_checks

In /etc/postfix/header_checks and /etc/postfix/body_checks:
sudo nano /etc/postfix/header_checks sudo nano /etc/postfix/body_checks
/free mortgage quote/ REJECT /free mortgage quote/ DISCARD
sudo postmap /etc/postfix/header_checks sudo postmap /etc/postfix/body_checks

SpamAssassin:
sudo apt install dovecot-sieve dovecot-managesieved spamassassin spamc spamass-milter

In /etc/dovecot/dovecot.conf:
sudo nano /etc/dovecot/dovecot.conf
protocols = imap lmtp sieve

In /etc/dovecot/conf.d/15-lda.conf:
sudo nano /etc/dovecot/conf.d/15-lda.conf

protocol lda {
  mail_plugins = $mail_plugins sieve
}

In /etc/dovecot/conf.d/20-lmtp.conf:
sudo nano /etc/dovecot/conf.d/20-lmtp.conf

protocol lmtp {
  mail_plugins = quota sieve
}

In /etc/postfix/main.cf:
sudo nano /etc/postfix/main.cf
milter_default_action = accept milter_protocol = 6 smtpd_milters = local:opendkim/opendkim.sock,local:opendmarc/opendmarc.sock,local:spamass/spamass.sock non_smtpd_milters = $smtpd_milters

In /etc/default/spamass-milter:
sudo nano /etc/default/spamass-milter
OPTIONS="-u spamass-milter -i 127.0.0.1"

Global sieve in /etc/dovecot/conf.d/90-sieve.conf:
sudo nano /etc/dovecot/conf.d/90-sieve.conf
sieve_before = /var/mail/SpamToJunk.sieve

Create /var/mail/SpamToJunk.sieve:
sudo nano /var/mail/SpamToJunk.sieve

require "fileinto";
if header :contains "X-Spam-Flag" "YES" {
  fileinto "Junk";
  stop;
}

sudo sievec /var/mail/SpamToJunk.sieve

In /etc/spamassassin/local.cf:
sudo nano /etc/spamassassin/local.cf
report_contact webmaster@yourdomain.com required_score 5.0 report_safe 0 add_header all Spam-Flag _YESNO_ add_header all Score _SCORE_ add_header all Report _REPORT_ add_header all Level _STARS_ add_header all Status "_YESNO_, score=_SCORE_ required=_REQD_ tests=_TESTS_ autolearn=_AUTOLEARN_ version=_VERSION_" add_header all Checker-Version "SpamAssassin _VERSION_ (_DATE_) on _HOSTNAME_" dns_server 127.0.0.1 score MISSING_FROM 5.0 score MISSING_DATE 5.0 score MISSING_HEADERS 3.0 score PDS_FROM_2_EMAILS 3.0 score FREEMAIL_FORGED_REPLYTO 3.5 score DKIM_ADSP_NXDOMAIN 5.0 score FORGED_GMAIL_RCVD 2.5 score FREEMAIL_FORGED_FROMDOMAIN 3.0 score HEADER_FROM_DIFFERENT_DOMAINS 3.0 score FREEMAIL_FROM 3.0 score ACCT_PHISHING 3.0 score AD_PREFS 3.0 score ADMAIL 3.0 score ADMITS_SPAM 3.0 score CONFIRMED_FORGED 3.0 score FROM_PAYPAL_SPOOF 3.0 score SPF_SOFTFAIL 2.0 score SPF_FAIL 5.0 whitelist_from *@statefarm.com blacklist_from *@email.freethinkerdaily.com

Unbound for DNS:
sudo apt install unbound

Install logging:
sudo apt install pflogsumm rsyslog

Remove mail.log from /etc/logrotate.d/rsyslog. Create /etc/logrotate.d/postfix-log:
sudo nano /etc/logrotate.d/postfix-log

/var/log/mail.log {
  missingok
  daily
  rotate 7
  create
  compress
  start 0
}

Script /usr/local/bin/pflog-run.sh:
sudo nano /usr/local/bin/pflog-run.sh

#!/bin/sh
gunzip /var/log/mail.log.0.gz
/usr/sbin/pflogsumm /var/log/mail.log.0 --problems-first --rej-add-from --verbose-msg-detail -q | mail -s "[pflog-lastlog]-$(hostname -f)-$(date)" your@email.com
gzip /var/log/mail.log.0
sleep 2s
systemctl restart rsyslog
systemctl restart postfix
systemctl restart dovecot
exit 0

Cron:
sudo crontab -e
30 12 * * * /bin/bash /usr/local/bin/pflog-run.sh >> /home/logs/pflog-run.log

Next Steps

Next, configure Roundcube with a custom PHP script for password changes, ensuring it’s secure and user-friendly. Automate this entire build with shell scripts and Debian preseed.cfg files for quick deployments.

Next Steps

Leave a Reply Cancel reply