Skip to content

Hosting

Email Hosting

Email hosting means running the servers that send, receive and store mail for your domains. It is one of the hardest things to self-host: the protocols are old and unforgiving, and the wider internet aggressively filters mail, so a small misconfiguration sends your messages straight to spam. This page covers the mail stack, the protocols and ports, the DNS that makes mail deliverable, and the honest trade-offs of running it yourself.

Tested on

AlmaLinux 9 / RHEL 9 with Postfix and Dovecot from the base repos. Debian/Ubuntu ship the same software; package and config paths differ (e.g. /etc/postfix/main.cf exists on both, Dovecot config lives under /etc/dovecot/).

The mail stack

A full mail server is several cooperating daemons. Think of it as two jobs: moving mail between servers and letting users read their mailbox.

  • MTA — Mail Transfer Agent (Postfix). Accepts mail from other servers and from your own users, and relays outbound mail to recipients' servers. Postfix is the de-facto standard MTA on Linux (Exim and Sendmail are alternatives).
  • MDA / IMAP-POP3 server (Dovecot). Stores delivered mail and serves it to mail clients so users can read, search and organize it. Dovecot is the standard.
  • Submission. Authenticated mail your users send (from Thunderbird, a phone, webmail) comes in on the submission port and is then relayed by Postfix. This is separate from the port-25 server-to-server traffic so that authentication and policy can differ.
  • Webmail (Roundcube). A browser front-end that talks IMAP/SMTP to Dovecot/Postfix so users can read mail without a desktop client.
  • Spam filtering (Rspamd or SpamAssassin). Scores incoming mail and rejects or tags spam; Rspamd also helps sign and check DKIM.
Internet --25--> [ Postfix MTA ] --LMTP/local--> [ Dovecot ] <--993/995-- Mail client
   ^                  ^                                 ^
   |               filtering                         IMAP/POP3
 outbound         (Rspamd)                          mailbox access
   |
Your users --587/465 (submission, authenticated)--> Postfix --> Internet

Protocols and ports

Protocol Port Encryption Purpose
SMTP 25 Opportunistic STARTTLS Server-to-server mail transfer (MTA to MTA). Not for client submission.
Submission 587 STARTTLS Authenticated outbound from users' mail clients (preferred).
SMTPS 465 Implicit TLS Authenticated submission over TLS from the start.
IMAP 143 STARTTLS Mailbox access, mail stays on the server (multi-device).
IMAPS 993 Implicit TLS IMAP over TLS (preferred).
POP3 110 STARTTLS Mailbox access, typically downloads then deletes.
POP3S 995 Implicit TLS POP3 over TLS.

Use 587/993 with TLS

Modern clients should use 587 (submission) for sending and 993 (IMAPS) for reading, both with TLS. Plain 110/143/25 without encryption should not carry credentials.

DNS for deliverability — the part that matters most

A working mail server that receives mail needs only an MX record. A mail server whose outbound mail actually lands in the inbox needs a full set of authentication records. Missing or wrong records are the number-one reason legitimate mail is marked as spam. See Domains and DNS for record syntax basics.

MX — where mail is delivered

The MX record tells other servers which host receives mail for your domain. Lower priority number = preferred.

example.com.   IN  MX  10  mail.example.com.
mail.example.com.  IN  A   203.0.113.25

SPF — who is allowed to send

An SPF record (a TXT record) lists the servers permitted to send mail for your domain. Receivers reject or downgrade mail from servers not listed.

example.com.  IN  TXT  "v=spf1 mx a:mail.example.com -all"

-all means "anything not listed should be rejected." Use exactly one SPF TXT record per domain.

DKIM — cryptographic signature

DKIM signs outbound mail with a private key; the matching public key is published in DNS. Receivers verify the signature proves the message was authorized by your domain and not altered in transit. The selector (default here) is chosen when you generate the key.

default._domainkey.example.com.  IN  TXT  "v=DKIM1; k=rsa; p=MIGfMA0GCSqGSIb3DQ...AQAB"

DMARC — policy and reporting

DMARC tells receivers what to do when SPF/DKIM fail and where to send aggregate reports. It ties SPF/DKIM to the visible From address (alignment).

_dmarc.example.com.  IN  TXT  "v=DMARC1; p=quarantine; rua=mailto:[email protected]; adkim=s; aspf=s"

Start with p=none (monitor only), then move to quarantine/reject once reports confirm your mail passes.

Reverse DNS / PTR — the IP's name

The PTR record maps your sending IP back to a hostname, and that hostname should match the server's HELO/EHLO name and have a matching forward A record. Many large receivers (notably Gmail/Microsoft) reject mail from IPs with no PTR or a generic ISP PTR. PTR is set by whoever controls the IP block — your VPS/hosting provider — not in your own zone file. Ask them to set it to mail.example.com.

# Confirm the PTR for your sending IP
dig -x 203.0.113.25 +short

Why missing records mean spam

No SPF/DKIM → receivers cannot verify you authorized the mail → spam or rejection. No DMARC → spammers can spoof your domain and you have no visibility. No/generic PTR → many providers refuse the connection outright. All four together are effectively required for the inbox.

Webmail and spam filtering

  • Roundcube gives users a browser inbox. It connects to Dovecot over IMAP and submits through Postfix; put it behind nginx/Apache with HTTPS (see HTTPS with Let's Encrypt).
  • Rspamd (modern, fast) or SpamAssassin scores inbound mail using content rules, DNS blocklists, SPF/DKIM/DMARC results and greylisting, then rejects, tags or quarantines. Rspamd also handles DKIM signing of outbound mail.

Self-hosting email is hard — be honest about it

Beyond installing the software, the real difficulty is reputation:

  • IP reputation. A brand-new VPS IP has no sending history; large providers throttle or junk it until it earns trust. IPs in ranges flagged as "residential/dynamic" are often blocked outright.
  • Blocklists (RBLs). If your IP (or a noisy neighbour in the same /24) gets listed on Spamhaus and similar, your mail is rejected until you request delisting.
  • Port 25 is frequently blocked. Most cloud and residential providers block outbound port 25 by default to fight spam; you must request it be opened, and many never allow it. Inbound 25 must be open to receive mail, so the firewall must allow it (see the server hardening checklist). Submission ports 587/465 should be open for your users; 143/993/110/995 only as needed.
# Allow the mail ports on firewalld (receive on 25, submission on 587/465, IMAPS 993)
sudo firewall-cmd --permanent --add-service=smtp           # 25
sudo firewall-cmd --permanent --add-service=smtp-submission # 587
sudo firewall-cmd --permanent --add-service=smtps           # 465
sudo firewall-cmd --permanent --add-service=imaps           # 993
sudo firewall-cmd --reload

Because of all this, many organizations do not self-host. They use managed email — Google Workspace or Microsoft 365 — or a dedicated mail platform (Mailcow, Mailu, iRedMail, or a relay/SMTP service like Amazon SES, Postmark or SendGrid for outbound). These outsource IP reputation and deliverability, which is usually the right call unless running your own mail is itself the goal.

Common troubleshooting

  • Read the mail log first. Postfix and Dovecot log every transaction; rejections, TLS errors and auth failures all show up there. See Analyzing log files.

    # Follow the mail log (RHEL/AlmaLinux uses /var/log/maillog; Debian uses /var/log/mail.log)
    sudo journalctl -u postfix -u dovecot -f
    sudo tail -f /var/log/maillog
    
  • Confirm your MX and DNS records resolve.

    dig MX example.com +short
    dig TXT example.com +short            # SPF
    dig TXT default._domainkey.example.com +short   # DKIM
    dig TXT _dmarc.example.com +short     # DMARC
    dig -x 203.0.113.25 +short            # PTR
    
  • Test deliverability end to end. Send a message to a tool like mail-tester.com (gives a spam score and flags missing SPF/DKIM/DMARC/PTR), or use the Google Postmaster Tools and MXToolbox to check blocklists and record syntax.

  • Test SMTP by hand with swaks --to [email protected] --server mail.example.com --tls to see the full SMTP conversation and any rejection reason.

Verify your work

# Mail ports are listening
sudo ss -tlnp | grep -E ':(25|465|587|143|993|995)'

# Postfix and Dovecot are running
systemctl status postfix dovecot

# DNS is in place (run from any machine)
dig MX example.com +short
dig TXT example.com +short
dig -x 203.0.113.25 +short
  • Send a test message to mail-tester.com and aim for 10/10 — it will name any missing SPF/DKIM/DMARC/PTR.
  • Send to a Gmail/Outlook account and confirm it lands in the inbox, not spam; view the message headers to see spf=pass, dkim=pass, dmarc=pass.
  • Confirm a webmail login works over HTTPS.

Summary

  • A mail server combines an MTA (Postfix) for transfer/relay, a Dovecot IMAP/POP3 server for mailbox access, and a separate authenticated submission path for users; webmail (Roundcube) and spam filtering (Rspamd/SpamAssassin) round it out.
  • Know the ports: SMTP 25, submission 587/465, IMAP 143/993, POP3 110/995 — prefer 587 and 993 with TLS.
  • Deliverability lives in DNS: MX (where mail goes), SPF (who may send), DKIM (signature), DMARC (policy), and PTR/reverse DNS (the IP's name). Missing any of these sends you to spam.
  • Self-hosting is genuinely hard because of IP reputation, blocklists and blocked port 25 — many use managed email (Google Workspace/M365) or a dedicated mail platform instead.
  • Troubleshoot by reading the mail log, checking records with dig, and using deliverability test tools.

Test yourself