Very interesting.. a few questions if I may?
On 11/08/2013 08:07 AM, Timo Sirainen wrote:
Hi all,
I've never really wanted to create my own MTA, because I like Postfix quite a
lot. And I always thought it would require a horribly lot of time to be able to
create something that was anywhere even close to having Postfix's features. (I
would shudder to even think about recreating Dovecot from scratch nowadays.)
But slowly over time I've also been thinking of ways how things could be done a
bit better, and I think I have enough ideas to start thinking about Dovecot MTA
more seriously in a few more months (after my current busy schedule calms down
a bit). And (unlike Dovecot!) I'm not planning on taking over the world with
the MTA (or at least not very quickly), but it would definitely be useful for
many installations I know of.
My main design goals for the MTA are:
* In normal load don't queue mails, just continue delivering the mail through
different processes/services until it succeeds or fails, and only after that
return ok/failure to the SMTP client. So there's no (forced) post-queue
filtering, everything would normally happen pre-queue. This is required because
in Germany (and EU in general?) you aren't allowed to just drop spams after
SMTP server has responsed OK to the client, even if you’re 100% sure it’s a
spam. So this would also mean that the SMTP DATA replies will come more slowly,
which means that the SMTP server must be able to handle a lot more concurrent
SMTP connections, which means that in large installations the smtpd process
must be able to asynchronously handle multiple SMTP client connections.
This is basically what I normally do with exim, and I believe it can be
achieved with postfix, so basically your point is a single asynchronous
smtpd for multiple connections?
My experience has been that the real problem with SMTP-time decision
making is the concurrency of the extremely heavy (e.g.) spamassassin
processes, heavy in both memory and CPU, and I/O if you use bayes which
you should.
* In some cases you can't really avoid placing mails into a queue. This could
be because of temporary failures or maybe because of an abnormal load spike. A
mail queue in local disk isn't very nice though, because if the local disk
dies, the queued mails are lost. Dovecot MTA will allow the queue to be in
object storage and it will also likely support replication (similar to current
dsync replication). In both of these cases if a server dies, another server can
quickly take over its queue and continue handling it.
Yes that would be nice. Another thing regarding multiple servers that
I'd build in is a much more powerful way to manage scanning backends,
keep track of dead ones (like freeradius zombie/dead tracking).
* Dovecot MTA is a new product, which means we can add some requirements to how
it's being used, especially related to securely sending emails between servers.
It could do a bunch of checks at startup and fail to even start if everything
isn't correct. Here are some things I had in mind - not sure if all of these
are good ideas or not:
They are all good ideas as long as these requirements can be turned off
per site :-)
- Require DKIM configuration. All outgoing mails will be DKIM signed.
- Require the domain’s DNS to contain _submission._tcp SRV record (and actually
might as well require _imap._tcp too)
- Require SSL certificates to be configured and always allow remote to use
STARTTLS
- Require DANE TLSA record to exist and match the server's configured SSL cert
- Have very good (and strict?) DNSSEC support. If we know a remote server is
supposed to have valid DNSSEC entries, but doesn't, fail to deliver mail
entirely?
- Add a new DNS record that advertises this is a Dovecot MTA (or compatible).
If such entry is found (especially when correctness is guaranteed by DNSSEC),
the email sender can assume that certain features exist and work correctly. If
they don't, it could indicate an attack and the mail sending should be retried
later. This DNS record would of course be good to try to standardize.
* Configuration: It would take years to implement all of the settings that
Postfix has, but I think it's not going to be necessary. In fact I think the
number of new settings to dovecot.conf that Dovecot MTA requires would be very
minimal. Instead nearly all of the configuration could be done using Sieve
scripts. We'd need to implement some new MTA-specific Sieve extensions and a
few core features/configurations/databases that the scripts can use, but after
that there wouldn't be really any limits to what could be done with them.
It comes to mind that you would want a separate master process for this
in case one would run it on the same box with dovecot imap. Or at least
a way to restart/reconfigure it separately.
* Try to implement as many existing interfaces as possible (e.g. Milter and
various Postfix APIs like policy servers) so that it wouldn’t be necessary to
reimplement all the tools and filters.
So perhaps something like this could be done in time for Dovecot v2.4. Any
thoughts/ideas/suggestions?
