Re: Make smtpd/Postscreen compatible with load balancers
Non-production release postfix-2.10-20120617-nonprod has support for up-stream proxy agents in postscreen(8) and smtpd(8). To enable, specify one of: postscreen_upstream_proxy_protocol = haproxy smtpd_upstream_proxy_protocol = haproxy haproxy is not the only proxy agent that works with Postfix. Support for nginx with proxied SASL authentication is available in Postfix 2.9 smtpd(8). This uses the XCLIENT protocol. After another week of testing, this is now released as a regular development release postfix-2.10-20120625. Wietse
Re: Make smtpd/Postscreen compatible with load balancers
Non-production release postfix-2.10-20120617-nonprod has support for up-stream proxy agents in postscreen(8) and smtpd(8). To enable, specify one of: postscreen_upstream_proxy_protocol = haproxy smtpd_upstream_proxy_protocol = haproxy haproxy is not the only proxy agent that works with Postfix. Support for nginx with proxied SASL authentication is available in Postfix 2.9 smtpd(8). This uses the XCLIENT protocol. Wietse
Re: Make smtpd/Postscreen compatible with load balancers
Hi Wietse, On Sun, Jun 17, 2012 at 08:25:12PM -0400, Wietse Venema wrote: Non-production release postfix-2.10-20120617-nonprod has support for up-stream proxy agents in postscreen(8) and smtpd(8). To enable, specify one of: postscreen_upstream_proxy_protocol = haproxy smtpd_upstream_proxy_protocol = haproxy haproxy is not the only proxy agent that works with Postfix. Support for nginx with proxied SASL authentication is available in Postfix 2.9 smtpd(8). This uses the XCLIENT protocol. This is awesome work, it opens a wider interoperability between our components which are commonly placed close to each other in a number of infrastructures. I hope David (who originally asked for the feature) will quickly test and provide us with some feedback. I'll forward your mail to the haproxy mailing list, since I know there are a number of postfix users there too. Best regards, Willy
Re: Make smtpd/Postscreen compatible with load balancers
Hi Wietse,
[ just subscribed to the list, I realized that our past conversation
was dropped since I was not subscribed, never mind ]
On Thu, Jun 07, 2012 at 08:16:53PM -0400, Wietse Venema wrote:
Willy Tarreau:
Regardless of command format details, if the proxy prepends a command
to the client's SMTP stream, then postscreen must use unbuffered
I/O to read that command. If buffering were turned on, the buffering
layer could read past the proxy'sCRLF and eat up part of the
client input kind-of like CVE-2011-0411.
Precisely on this point there is an easier way, it consists in using
recv(MSG_PEEK). The big advantage is that you don't need to store the
temporary bytes you've read since they remain in the kernel's buffers.
So it more or less looks like this :
len = recv(fd, trash, sizeof(trash), MSG_PEEK);
if (len == -1 errno == EAGAIN)
return;
lf = memchr(trash, '\n', len);
if (lf == NULL) {
if (len trash) /* Huh?? */
return;
/* else abort the connection */
}
In an event-driven program such as postscreen, this code breaks
when the proxy line arrives as multiple fragments.
If the program does not drain proxy line fragments from the kernel
buffer, then the socket remains readable and the program will go
into a read-notification loop until the entire line is received.
Indeed this is totally true. I would say that given the short size
of the message the risk of this occurring is barely 0 explaining
whit this has probably never hit anybody, but it would be sufficient
that someone implements the protocol using a series of
write(fd, buf++, 1) and you'd be spinning (even worse if it dies in
the middle).
I've seen implementations doing recv(MSG_PEEK) and reject connections
from incomplete messages (again, almost zero risk but YMMV).
In haproxy, I'm using the input buffer associated to the fd, so I
don't have this problem. I'm basically doing a recv() on the buffer.
I think it is equivalent to the VSTREAM you're using.
In postscreen you don't want to do this since you don't want to
consume any possible incoming data (btw you probably drop the
connection if you get any data at this point). That said, if you
have a buffer associated to the connection, then you can perform
the first MSG_PEEK to check the pending data size and then a real
recv() to only consume up to the end of line.
But then doing so indeed invalidates the following suggestion.
This implies that the following suggestion is not valid for an
event-driven program such as postscreen:
On the one hand, if it is as trivial to make smtpd parse the PROXY
line as it was for postscreen, it can solve the problem by having
postscreen not consume the first line, which makes sense in that
postscreen remains the first layer analyser which doesn't mangle
data on the connection.
Either you need to update the protocol spec (require non-fragmented
proxy lines)
I have mixed opinions on this. On the one hand, we can't really impose
lower layers segmentation behaviour, so from a layering perspective, it
is not correct. On the other hand, the use cases for the protocol are
very specific. We're the very first segment over the connection so we
are always allowed to send at least one MSS. Nobody should sanely use
this proxy line on connections with an MSS lower than the 116 bytes a
max line may be for long IPv6 addresses and ports.
So indeed, I'm tempted to follow your suggestion, it will ease processing
for everyone and ensure that nobody tries sending fragmented lines. We'd
rely on a sane lower layer and declare other cases out of scope.
or provide a code example that doesn't go into a
read-notification loop when the proxy line arrives as multiple
fragments.
With a buffer this problem does not happen, but it's the first case I'm
facing this need with fd passing, which makes me scratch my head a lot.
I really like the way you're plugging postscreen in front of smtpd, and
I'd like to ensure we don't make it complex to keep this nice model.
That's why I think that adding a sane requirement in the spec should be
the most adequate solution. If in the mean time you get a smarter idea,
do not hesitate to share it :-)
I'll keep thinking about it a bit before updating it. I think I will also
propose some generic code in the spec for both sides.
Best regards,
Willy
Re: Make smtpd/Postscreen compatible with load balancers
Willy Tarreau: Either you need to update the protocol spec (require non-fragmented proxy lines) I have mixed opinions on this. On the one hand, we can't really impose lower layers segmentation behaviour, so from a layering perspective, it is not correct. On the other hand, the use cases for the protocol are very specific. We're the very first segment over the connection so we are always allowed to send at least one MSS. Nobody should sanely use this proxy line on connections with an MSS lower than the 116 bytes a max line may be for long IPv6 addresses and ports. So indeed, I'm tempted to follow your suggestion, it will ease processing for everyone and ensure that nobody tries sending fragmented lines. We'd rely on a sane lower layer and declare other cases out of scope. It is quite usual that the first deployments of a protocol expose some unexpected pain points. Here, a minor protocol tweak (no proxy line fragmentation) makes it possible to use MSG_PEEK lookahead without going into a busy-wait loop. Fragmentation makes no difference for a postscreen implementation that reads the proxy line one character at a time until it reads CRLF, before it hands off the file descriptor to a real SMTP server. Wietse
Re: Make smtpd/Postscreen compatible with load balancers
Willy Tarreau:
Regardless of command format details, if the proxy prepends a command
to the client's SMTP stream, then postscreen must use unbuffered
I/O to read that command. If buffering were turned on, the buffering
layer could read past the proxy'sCRLF and eat up part of the
client input kind-of like CVE-2011-0411.
Precisely on this point there is an easier way, it consists in using
recv(MSG_PEEK). The big advantage is that you don't need to store the
temporary bytes you've read since they remain in the kernel's buffers.
So it more or less looks like this :
len = recv(fd, trash, sizeof(trash), MSG_PEEK);
if (len == -1 errno == EAGAIN)
return;
lf = memchr(trash, '\n', len);
if (lf == NULL) {
if (len trash) /* Huh?? */
return;
/* else abort the connection */
}
In an event-driven program such as postscreen, this code breaks
when the proxy line arrives as multiple fragments.
If the program does not drain proxy line fragments from the kernel
buffer, then the socket remains readable and the program will go
into a read-notification loop until the entire line is received.
This implies that the following suggestion is not valid for an
event-driven program such as postscreen:
On the one hand, if it is as trivial to make smtpd parse the PROXY
line as it was for postscreen, it can solve the problem by having
postscreen not consume the first line, which makes sense in that
postscreen remains the first layer analyser which doesn't mangle
data on the connection.
Either you need to update the protocol spec (require non-fragmented
proxy lines) or provide a code example that doesn't go into a
read-notification loop when the proxy line arrives as multiple
fragments.
Wietse
Re: Make smtpd/Postscreen compatible with load balancers
Willy Tarreau: I'm totally in sync with you on this. On the one hand, if it is as trivial to make smtpd parse the PROXY line as it was for postscreen, it can solve the problem by having postscreen not consume the first line, which makes sense in that postscreen remains the first layer analyser which doesn't mangle data on the connection. But I can also understand that you're not necessarily interested in supporting a new method of passing connection information. Sorry, that might work for haproxy, but could complicate attempts to add support for other protocols. If the proxy talks to postscreen then that is where its protocol will terminate. Wietse
Re: Make smtpd/Postscreen compatible with load balancers
Adding a haproxy-to-postscreen adapter turns out to be pretty trivial. However, a major code rewrite would be needed in the way that postscreen(8) talks to smtpd(8). Begin background: Postscreen is optimized for the following case: a client connects, postscreen looks up the client IP address in its temporary cache, and when the client is OK, postscreen sends the connection's file descriptor to a Postfix SMTP server process and gets out of the loop. There is no other communication between postscreen and SMTP server processes. The file descriptor carries all information that an SMTP server process needs. In fact, the file descriptor is indistinguishable from a file descriptor that an SMTP server gets when it is configured in master.cf to listen directly on the SMTP port. End background. To make postscreen work with before-postscreen proxies, it either has to become a proxy itself (over my dead body) or Postfix needs a small protocol to send (attributes plus a file descriptor) from postscreen to smtpd. It's relatively simple to pass a few attributes from haproxy to postscreen with a simple hard-coded non-reusable protocol. On the other hand, Postfix support for (file descriptor + arbitrary attribute passing) will have to be reusable (*), so that the same infrastructure can be used later to improve Postfix. For example, to hand off a connection mid-session from postscreen to smtpd, something that is currently not possible. Wietse (*) The client decides what attributes to send and passes the attributes + file descriptor to the low-level sender infrastructure; the low-level receiver infrastructure first reads the attributes into a hash and then passes the attributes and file descriptor up to the application, in an application-specified order, and deals with missing attributes and other problems.
Re: Make smtpd/Postscreen compatible with load balancers
Willy Tarreau: Regardless of command format details, if the proxy prepends a command to the client's SMTP stream, then postscreen must use unbuffered I/O to read that command. If buffering were turned on, the buffering layer could read past the proxy'sCRLF and eat up part of the client input kind-of like CVE-2011-0411. Precisely on this point there is an easier way, it consists in using recv(MSG_PEEK). The big advantage is that you don't need to store the temporary bytes you've read since they remain in the kernel's buffers. So it more or less looks like this : First, just like SMTP and HTTP protocol documentation, HAPROXY documentation states nowhere that any particular information must be sent (or received) in exactly one TCP segment. If this atomicity is an essential requirement of the HAPROXY protocol, then that had better be made explicit in the documentation. Second, it makes little sense to re-invent all the error and time-limit handling that Postfix already has. I prefer to reuse the line reading routine that postscreen already has, instead of reaching for the lowest-level kernel API. Wietse
Re: Make smtpd/Postscreen compatible with load balancers
Willy Tarreau: Hi Wietse, On Tue, May 29, 2012 at 08:18:35AM -0400, Wietse Venema wrote: Willy Tarreau: Regardless of command format details, if the proxy prepends a command to the client's SMTP stream, then postscreen must use unbuffered I/O to read that command. If buffering were turned on, the buffering layer could read past the proxy'sCRLF and eat up part of the client input kind-of like CVE-2011-0411. Precisely on this point there is an easier way, it consists in using recv(MSG_PEEK). The big advantage is that you don't need to store the temporary bytes you've read since they remain in the kernel's buffers. So it more or less looks like this : First, just like SMTP and HTTP protocol documentation, HAPROXY documentation states nowhere that any particular information must be sent (or received) in exactly one TCP segment. No, there is no such requirement, as this can never be guaranteed anywhere. That's why in my example, there was a return on incomplete lines, waiting for the next event to try to complete the line. Postscreen, faced with the same non-problem in SMTP, does exactly the same thing. The line read routine would need to be moved out of the dummy SMTP engine so that it can be reused to read proxy data, whether from haproxy, xclient or something else. Wietse
Re: Make smtpd/Postscreen compatible with load balancers
Dear I have played with DNS and MX but the problem is DNS did not care about servers load and if servers have a huge queue. It just balance to next MX only if the server did not respond. Load-balancing is a cool feature that take care on the server health. Especially if you add services on the server such as Spamassassin, backup mails, MDA... Wietse, i copy the founder of HaProxy (Willy Tarreau) in this mail in order to introduce him and see if he can help you implementing such protocol in PostScreen/smtpd. Best regards. Le 27/05/2012 15:25, Wietse Venema a écrit : David Touzeau: Dear We are facing an problem that we cannot resolve... Our main goal is to implement a load-balancer in front of Postfix (HaProxy). We have made a discuss with HaProxy founder in order to implement the XCLIENT protcol but this is difficult for him to implement such protocol. FYI nginx implements XCLIENT (and more). But it does not matter, since postscreen has no proxy support. In other way it seems that PostScreen is not really compatible with the proxy protocol. First, there needs to be a configuration parameter that tells postscreen what kind of proxy command will be prepended to the client's SMTP stream. /etc/postfix/main.cf: # Is there a better name than proxy protocol? postscreen_proxy_protocol = whatever With this, postscreen will drop connections that fail to conform to the configured protocol. Regardless of command format details, if the proxy prepends a command to the client's SMTP stream, then postscreen must use unbuffered I/O to read that command. If buffering were turned on, the buffering layer could read past the proxy'sCRLF and eat up part of the client input kind-of like CVE-2011-0411. A rough estimate of what this requires: - A new main.cf parameter and documentation. - A way to turn off buffering on VSTREAMs. Alternative: use low-level read system calls and re-invent VSTREAM timeout/etc. error handling. Either this may take a dozen or so lines of code. - A new postscreen code module with generic hook to prepend proxies. - An event-driven loop that reads the proxy command one character at a time untilCRLF, length exceeded, EOF, time limit, or other error. Another dozen lines. - A proxy command parser that does all the necessary sanity checks (valid address syntax, numerical TCP ports in the range 1..65535), no missing or extra fields. Another dozen lines. - Reuse the existing postscreen data structures that are now filled with endpoint information from getpeername() and getsockname(). Wietse
Re: Make smtpd/Postscreen compatible with load balancers
On May 28, 2012, at 10:26, David Touzeau wrote: I have played with DNS and MX but the problem is DNS did not care about servers load and if servers have a huge queue. It just balance to next MX only if the server did not respond. Load-balancing is a cool feature that take care on the server health. Especially if you add services on the server such as Spamassassin, backup mails, MDA... Did you read (and understand) Victor's example? Did you test with multiple servers that had the same preference in their MX record? If they do not have the same preference, the server with the lowest preference value will always be preferred, and failover to the others will indeed only happen when that server is down. -- If you operate at scale, you build out your SMTP infrastructure not just in width, but also in depth; 1) Mail exchangers. Your first 'line of defense'. These show up in your MX records, and run postscreen to fend off bot traffic and the like. They reject everything you know you're not going to accept, BEFORE the queue. How much depends on your profile, but in our case at more than 70% of connections never makes it to the queue. Everything that is accepted is passed on immediately. These do not have any local user mailboxes, and do not do any significant amount of content filtering. They should never have significant amounts of mail in the queue. 2) Mail routing. Where does this message go? Does it need to be archived, copied, rerouted? 3) Content filtering. This is where Spamassassin runs, for example. Spam is marked, everything is passed on. Nothing is bounced, ever. 4) Mailbox server. Accepts the message and sorts them into local folders, where they are stored on disk. This is where POP3/IMAP daemons run to serve users accessing their mailboxes. 5) Submission server. This is what your clients talk to to send their outgoing mail. Requires authentication, runs on port 587, and does not need postscreen. May do some basic checks, but passes messages on as quickly as possible. Should never have much of a queue. 6) Relay servers. These are the servers that talk to the rest of the internet, for outgoing mail. Since you have little control over the availability and load of the servers you are communicating with, mail may be in the queue here for a while, up to several days. -- In most setups, several of these roles can be combined; MX and routing generally go together, and can take care of outgoing relay as well. Content filtering and submission can run on the mailbox servers. But really, find the real bottleneck. You should not need to load balance postscreen, and it's quite likely that it is not the right place to start optimizing. Where are you maxing out? CPU, memory, disk? Is it a single customer that can be split off to a seperate box, perhaps? Cya, Jona -- Le 27/05/2012 15:25, Wietse Venema a écrit : David Touzeau: Dear We are facing an problem that we cannot resolve... Our main goal is to implement a load-balancer in front of Postfix (HaProxy). We have made a discuss with HaProxy founder in order to implement the XCLIENT protcol but this is difficult for him to implement such protocol. FYI nginx implements XCLIENT (and more). But it does not matter, since postscreen has no proxy support. In other way it seems that PostScreen is not really compatible with the proxy protocol. First, there needs to be a configuration parameter that tells postscreen what kind of proxy command will be prepended to the client's SMTP stream. /etc/postfix/main.cf: # Is there a better name than proxy protocol? postscreen_proxy_protocol = whatever With this, postscreen will drop connections that fail to conform to the configured protocol. Regardless of command format details, if the proxy prepends a command to the client's SMTP stream, then postscreen must use unbuffered I/O to read that command. If buffering were turned on, the buffering layer could read past the proxy'sCRLF and eat up part of the client input kind-of like CVE-2011-0411. A rough estimate of what this requires: - A new main.cf parameter and documentation. - A way to turn off buffering on VSTREAMs. Alternative: use low-level read system calls and re-invent VSTREAM timeout/etc. error handling. Either this may take a dozen or so lines of code. - A new postscreen code module with generic hook to prepend proxies. - An event-driven loop that reads the proxy command one character at a time untilCRLF, length exceeded, EOF, time limit, or other error. Another dozen lines. - A proxy command parser that does all the necessary sanity checks (valid address syntax, numerical TCP ports in the range 1..65535), no missing or extra fields. Another dozen lines. - Reuse the existing postscreen data structures that are now filled with endpoint information from getpeername() and getsockname(). Wietse
Make smtpd/Postscreen compatible with load balancers
Dear We are facing an problem that we cannot resolve... Our main goal is to implement a load-balancer in front of Postfix (HaProxy). We have made a discuss with HaProxy founder in order to implement the XCLIENT protcol but this is difficult for him to implement such protocol. In other way it seems that PostScreen is not really compatible with the proxy protocol. src/postscreen/postscreen.c : /* BUGS /* The \fBpostscreen\fR(8) built-in SMTP protocol engine /* currently does not announce support for AUTH, XCLIENT or /* XFORWARD. /* Support for AUTH may be added in the future. In fact we need to Postfix be compatible with Proxy protocol in order to store the IP address at each ends of the socket in the VSTREAM descriptor. With this way, an other handler, before PostScreen should get the IP/Port. We asking if there is a way to implement a load-balancer such as HaProxy or Crossroads without loosing IP addresses sources in order to make PostScreen available or any blacklisting feature.. Best regards.
Re: Make smtpd/Postscreen compatible with load balancers
On May 27, 2012, at 10:25, David Touzeau wrote: We are facing an problem that we cannot resolve... Our main goal is to implement a load-balancer in front of Postfix (HaProxy). We have made a discuss with HaProxy founder in order to implement the XCLIENT protcol but this is difficult for him to implement such protocol. In other way it seems that PostScreen is not really compatible with the proxy protocol. src/postscreen/postscreen.c : /* BUGS /* The \fBpostscreen\fR(8) built-in SMTP protocol engine /* currently does not announce support for AUTH, XCLIENT or /* XFORWARD. /* Support for AUTH may be added in the future. In fact we need to Postfix be compatible with Proxy protocol in order to store the IP address at each ends of the socket in the VSTREAM descriptor. With this way, an other handler, before PostScreen should get the IP/Port. We asking if there is a way to implement a load-balancer such as HaProxy or Crossroads without loosing IP addresses sources in order to make PostScreen available or any blacklisting feature.. I am a tad confused; why would your load balancer need to implement the XCLIENT protocol, when load balancing SMTP generally happens at one of the layers below that? Typically at layer 3 or 4, if I am not mistaken? Also, why would you need a proxy? Unlike HTTP, SMTP does not benefit from front-end caching and the like. HTTP is a stateless, SMTP a stateful protocol? It is easily made highly available, just based on the MX records, and you can implement basic load balancing by having several MX records with the same priority, IIRC. In other words, what is the core problem you are trying to solve? Cya, Jona
Re: Make smtpd/Postscreen compatible with load balancers
David Touzeau: Dear We are facing an problem that we cannot resolve... Our main goal is to implement a load-balancer in front of Postfix (HaProxy). We have made a discuss with HaProxy founder in order to implement the XCLIENT protcol but this is difficult for him to implement such protocol. FYI nginx implements XCLIENT (and more). But it does not matter, since postscreen has no proxy support. In other way it seems that PostScreen is not really compatible with the proxy protocol. First, there needs to be a configuration parameter that tells postscreen what kind of proxy command will be prepended to the client's SMTP stream. /etc/postfix/main.cf: # Is there a better name than proxy protocol? postscreen_proxy_protocol = whatever With this, postscreen will drop connections that fail to conform to the configured protocol. Regardless of command format details, if the proxy prepends a command to the client's SMTP stream, then postscreen must use unbuffered I/O to read that command. If buffering were turned on, the buffering layer could read past the proxy's CRLF and eat up part of the client input kind-of like CVE-2011-0411. A rough estimate of what this requires: - A new main.cf parameter and documentation. - A way to turn off buffering on VSTREAMs. Alternative: use low-level read system calls and re-invent VSTREAM timeout/etc. error handling. Either this may take a dozen or so lines of code. - A new postscreen code module with generic hook to prepend proxies. - An event-driven loop that reads the proxy command one character at a time until CRLF, length exceeded, EOF, time limit, or other error. Another dozen lines. - A proxy command parser that does all the necessary sanity checks (valid address syntax, numerical TCP ports in the range 1..65535), no missing or extra fields. Another dozen lines. - Reuse the existing postscreen data structures that are now filled with endpoint information from getpeername() and getsockname(). Wietse
Re: Make smtpd/Postscreen compatible with load balancers
On Sun, May 27, 2012 at 12:21:49PM +0200, DTNX Postmaster wrote: Also, why would you need a proxy? Unlike HTTP, SMTP does not benefit from front-end caching and the like. HTTP is a stateless, SMTP a stateful protocol? It is easily made highly available, just based on the MX records, and you can implement basic load balancing by having several MX records with the same priority, IIRC. I concur. Postscreen is only needed in front of public MX hosts to reduce hogging of the SMTP service by ratware. On public MX hosts, you can use a proxy that sits in the network path between the outside world and the MX hosts, in which case the proxy will not rewrite the source IP and no XCLIENT is required. Or you can use a proxy like F5 that can implement XCLIENT (the F5 can perform an programmable initial chat-script before handing the client stream to the server). Finally, no postscreen is needed in front of submission servers. So you can choose proxies that don't rewrite the layer 3 IP address, proxies that do, but can do XCLIENT. Or multiple MX hosts with no proxies at all: example.com. IN MX 0 mx1.example.com. example.com. IN MX 0 mx2.example.com. example.com. IN MX 0 mx3.example.com. example.com. IN MX 0 mx4.example.com. ; mx1.example.com. IN A 192.0.2.1 mx1.example.com. IN A 192.0.2.2 mx1.example.com. IN A 192.0.2.3 mx1.example.com. IN A 192.0.2.4 ; mx2.example.com. IN A 192.0.2.5 mx2.example.com. IN A 192.0.2.6 mx2.example.com. IN A 192.0.2.7 mx2.example.com. IN A 192.0.2.8 ; mx3.example.com. IN A 192.0.2.9 mx3.example.com. IN A 192.0.2.10 mx3.example.com. IN A 192.0.2.11 mx3.example.com. IN A 192.0.2.12 ; mx4.example.com. IN A 192.0.2.13 mx4.example.com. IN A 192.0.2.14 mx4.example.com. IN A 192.0.2.15 mx4.example.com. IN A 192.0.2.16 The above gets you 16 MX hosts with no load balancers required. You only need load balancers when you start to get to the size of Google, Hotmail, ... and they use DNS load-balancers, that return geo-proximate IPs for the MX host or any-cast IPs. There is likely a second layer of load-balancing below the DNS layer at that scale, but very few sites need either. -- Viktor.
Re: Make smtpd/Postscreen compatible with load balancers
Viktor Dukhovni: On Sun, May 27, 2012 at 12:21:49PM +0200, DTNX Postmaster wrote: Also, why would you need a proxy? Unlike HTTP, SMTP does not benefit from front-end caching and the like. HTTP is a stateless, SMTP a stateful protocol? It is easily made highly available, just based on the MX records, and you can implement basic load balancing by having several MX records with the same priority, IIRC. I concur. Postscreen is only needed in front of public MX hosts to reduce hogging of the SMTP service by ratware. On public MX hosts, you can use a proxy that sits in the network path between the outside world and the MX hosts, in which case the proxy will not rewrite the source IP and no XCLIENT is required. This depends on the proxy. - If it's a packet-level proxy then it just forwards unmodified packets that belong to the same session to the same back-end SMTP server. - If it's a circuit-level proxy like nginx, then it needs to forward session info in-band, with XCLIENT or equivalent. Circuit-level proxies can do a few things that are difficult with packet-level proxies, such as sitting in a remote network, or that are impossible such as off-loading TLS or AUTH. [fan-out with four MX records and four a records per MX name] The above gets you 16 MX hosts with no load balancers required. You only need load balancers when you start to get to the size of Google, Hotmail, ... and they use DNS load-balancers, that return geo-proximate IPs for the MX host or any-cast IPs. There is likely a second layer of load-balancing below the DNS layer at that scale, but very few sites need either. Some people don't understand the difference between browser-to-server HTTP, and MTA-to-MTA SMTP. Wietse
Re: Make smtpd/Postscreen compatible with load balancers
Wietse Venema: David Touzeau: Dear We are facing an problem that we cannot resolve... Our main goal is to implement a load-balancer in front of Postfix (HaProxy). We have made a discuss with HaProxy founder in order to implement the XCLIENT protcol but this is difficult for him to implement such protocol. FYI nginx implements XCLIENT (and more). But it does not matter, since postscreen has no proxy support. If smtpd is sufficient for you, then nginx (and other XCLIENT capable proxies, see Victor's post) will do the job. Wietse
