Re: [exim-dev] XCLIENT patch to Exim; Cambridge
On 16/01/15 10:57, Vsevolod Stakhov wrote: BTW, I'd like also to ask the same question about native rspamd support, which could be found at the following address: https://github.com/vstakhov/rspamd/blob/master/contrib/exim/patch-exim-src_spam.c.diff So far, I have several requests from rspamd users who run exim as MTA. Rspamd went in at c5f280e20a8e - thanks to Andrew Lewis. That's since 4.85 and it's still mutating gently, but it's there to play with if you build from source. -- Cheers, Jeremy -- ## List details at https://lists.exim.org/mailman/listinfo/exim-dev Exim details at http://www.exim.org/ ##
Re: [exim-dev] XCLIENT patch to Exim; Cambridge
On Thu, Jan 15, 2015 at 7:52 PM, Jeremy Harris j...@wizmail.org wrote: I think we'd want to know what features are actually used by existing proxy front-ends. Is that Postfix page the sole definition of the ESMTP option? I note it doesn't specify the format of an IP address (there is an example of an IPv4 one). Swaks supports it, and mentions another (undocumented!) attribute: REVERSE_NAME ... ( http://www.jetmore.org/john/code/vmail/latest/doc/ref.txt ) Speaking of standardization and swaks supporting it, I got a trouble report related to swaks and XCLIENT last night. The person reporting it was using Message Systems' Momentum MTA (formerly ecelerity). In particular, it advertised XCLIENT during EHLO like so: 250-XCLIENT which violates the postfix doc by not listing support attributes, and on acknowledgement of successful XCLIENT request it responds with 250 2.0.0 XCLIENT handled Despite the postfix doc listing 220 as the success code. None of this is really Exim's problem, but as more tools support this, it would be really nice to point to something other than a postfix readme as the implementation standard. --John -- ## List details at https://lists.exim.org/mailman/listinfo/exim-dev Exim details at http://www.exim.org/ ##
Re: [exim-dev] XCLIENT patch to Exim; Cambridge
On Fri, Jan 16, 2015 at 10:34 AM, Phil Pennock p...@exim.org wrote: [ I'm hoping you're setting yourself up for both roles... ] I can do a security review and one-time merge, but can't commit to more than that; I haven't even found time to look at the DANE work. :( I wanted to take a moment and thank Viktor for taking a few mornings via Skype and stepping through the Exim DNS lookup/tracking and starting into how Exim uses the DANE library he has written. We were going to have one last session deep diving into how Exim uses his DANE library, but work commitments have altered my availability. (I didn't CC Viktor/Wietse on this email because it contained some of Phil's private comments that were not part of the full CC list.) Update: I have officially switched jobs at this point, and my time availability for anything outside of work and family has fallen to an all-time low. I lurk in IRC, and when I notice it, I'm there for quick collaboration and giving feedback. In the middle of switching jobs I was cutting RC's and cut the last release. I anticipate I can continue to make time to cut these releases. But for now I have stalled on the two main Exim development projects I had: 1) Adding milter interface (thanks also to Viktor and Wietse for a few directional emails). 2) integration of Python - haven't done anything for at least a year now (2.x worked, 3.x posed ALL KINDS OF PROBLEMS and didn't work). There's also hummus/tahini stuff to do first, on my plate for when I have Exim time. Same here for when I have time, which doesn't look to be too much over the next few months, at least not until my current largest job project (system migration into AWS) gets completed. Billable hours have precedence as they pay the bills. To complicate things more, my job capacity no longer interfaces with email. ...Todd -- The total budget at all receivers for solving senders' problems is $0. If you want them to accept your mail and manage it the way you want, send it the way the spec says to. --John Levine -- ## List details at https://lists.exim.org/mailman/listinfo/exim-dev Exim details at http://www.exim.org/ ##
Re: [exim-dev] XCLIENT patch to Exim; Cambridge
Viktor Dukhovni: On Fri, Jan 16, 2015 at 12:52:46AM +, Jeremy Harris wrote: Most of the attributes look ok for us to support. I'm dubious about the LOGIN one though; this feels like a protocol level violation. In Postfix this allows proxying of SASL logins. A proxy might handle SASL auth in front of the MTA. The MTA receives the SASL login name as determined by the proxy and applies access control decisions accordingly. Specifically, this was added for nginx. Is that Postfix page the sole definition of the ESMTP option? There is an IETF draft for XFORWARD but that solves a different problem (logging instead of impersonation). I am not aware of a similar effort with respect to XCLIENT. Either way, if there is an IETF spec then I'll update Postfix where needed. Postfix accepts XCLIENT without EHLO. That is OK because XCLIENT is meant for settings where ESMTP negotiation is unnecessary. Some in-the-field implementations may rely on this optimization. Wietse I think so, that's why it is perhaps time for an IETF draft, the feature seems to have caught on far beyond Postfix. Not sure whether the I-D should be informational or standards-track. XCLIENT is a local matter between various load-balancer systems and the MTA behind them, and is not generally a concern across indepedently operated systems. So informational might be less work, and still get the job done. I note it doesn't specify the format of an IP address (there is an example of an IPv4 one). Quoting XCLIENT_README: The ADDR attribute specifies an SMTP client numerical IPv4 network address, an IPv6 address prefixed with IPV6:, or [UNAVAILABLE] when the address information is unavailable. Address information is not enclosed with []. Thus one of the three forms: 192.0.2.1 IPv6:2001:db8::1 [UNAVAILABLE] Swaks supports it, and mentions another (undocumented!) attribute: REVERSE_NAME ... ( http://www.jetmore.org/john/code/vmail/latest/doc/ref.txt ) It does looks like XCLIENT_README omits that attribute which was added more recently. The full list is: #define XCLIENT_NAMENAME /* client name */ #define XCLIENT_REVERSE_NAMEREVERSE_NAME /* reverse client name */ #define XCLIENT_ADDRADDR /* client address */ #define XCLIENT_PORTPORT /* client port */ #define XCLIENT_PROTO PROTO /* client protocol */ #define XCLIENT_HELOHELO /* client helo */ #define XCLIENT_LOGIN LOGIN /* SASL login name */ The REVERSE_NAME is the unverified result of of a PTR lookup. While NAME is the FCrDNS. These are of course not always available to the proxy. -- Viktor. -- ## List details at https://lists.exim.org/mailman/listinfo/exim-dev Exim details at http://www.exim.org/ ##
Re: [exim-dev] XCLIENT patch to Exim; Cambridge
-BEGIN PGP SIGNED MESSAGE- Hash: SHA512 On 15/01/15 19:00, Phil Pennock wrote: FreeBSD is carrying a local patch to Exim, adding XCLIENT support. The ticket requesting its addition is at: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=133891 and there's no indication given there about why this was aimed directly at one OS's packaging, rather than at upstream. The feature documentation is: http://www.postfix.org/XCLIENT_README.html The patch can be found at: https://github.com/freebsd/freebsd-ports/blob/master/mail/exim/files/extra-patch-xclient This should probably be considered as a parallel to the proxy protocol support which we have. Aside from security review, the biggest issue is likely to be that the patch wasn't given to us and is currently a standalone work without a license statement, so we'd need to chase down the original author and ask about permission to include as part of Exim, under GPL. I have no objections about applying GPL terms to this patch, actually, I'm fine with distributing it under the CC0 license. In 2008 there was discussion on exim-users, subject XCLIENT supported by exim?; Nigel summarized the state as zero previous discussion, no patches, so no apparent interest. The tone of the response I see was generally oh that's a Postfix thing, we just connect Exim directly to the Internet without anything in front of it. One of the last posts referenced an existing patch by Vsevolod Stakhov: http://cebka.pp.ru/blog/2007/12/xclient-exim.html http://cebka.pp.ru/blog/patch-exim-xclient but there's no longer any DNS for that host; however, the initial report in the FreeBSD PR #133891 referenced http://cebka.pp.ru/blog/2009/01/-eximxclient.html so it seems that the history of this patch in FreeBSD traces back to then, even though the FreeBSD patch has been maintained as it's patched for more recent Exim releases. In 2013, I was contacted by Tony Finch who asked me to review the patch. I have fixed several issues and added XTEXT encoding for the original patch and sent it back. It would be really nice if this feature will be included in the upstream, because acting as mail/exim maintainer in FreeBSD I have to check all extra-patch for applicability to the recent exim sources. I think that the biggest problem is that most postmaster folks back then didn't see the benefit of siting an Exim behind a front-end proxy, especially since this was presented as a security proxy adding features, where all the features _could_ be done in Exim already. Since then, with the widening spread of protocol-generic front-end loadbalancers, we've seen the haproxy Proxy Protocol take off, the approach of setting normally-from-getsockopt vars based upon remote data _if_ the connecting host passes an ACL has been validated and seen not to be a security issue (well, unless someone allows the extension from the open Internet, instead of just from the local trusted proxies) and I think that this is _much_ less controversial. It looks like the Vsevolod Stakhov from the original report is probably the gentleman by that name now at the University of Cambridge (oh, it's the same guy who did the cool libucl config library stuff, that's why the name was familiar :) ). On this basis, I'm going to explicitly CC Tony, also at UoC who could perhaps chat with Vsevolod, and the address found on https://github.com/vstakhov. Guys, okay to pull this patch into Exim? BTW, I'd like also to ask the same question about native rspamd support, which could be found at the following address: https://github.com/vstakhov/rspamd/blob/master/contrib/exim/patch-exim-src_spam.c.diff So far, I have several requests from rspamd users who run exim as MTA. - -- Vsevolod Stakhov -BEGIN PGP SIGNATURE- Version: GnuPG v1 iQIcBAEBCgAGBQJUuO6RAAoJEHnvd0hTzOjBicgP/15jB1WYQ58JY7w8BlQ2o9LB TgDIRfCyCctbouKDh5/zqYR3fMTzie/4Ca7M0M07k2gaaztZ/M5TDZJqEPZzhFQL cK7jSKTgVGekGkg1vgClCbpCKDYZaHO4ye5wrjR7epUAZq4HXMrXw4LfiFcxG6ri HlkMq5Lpy+veBhscWnDtAyGa8+NDlBAfoBViY1caqnfRKbYu331SoP3xsIMyaVv2 YK6VcR6t2TLmsCAuCxgdhFi2U7K8WiGkGYcKZObWKVE2Pr8LU9+Jf3hPvsh7u0i2 aKhEQsi1IO0WqY5nMJJ6+BdRhknUI1ZkQXFeevCqB07uhrHGXHLXYtC666TvXf0d dW/D2wZ+XKCs8zaKEq9MQvbHQXoj/iXJSKO5hgRCEuAFj2bG3j91v3MFZrizKhf5 urhMsyjRWLcz4CdtnGgesidA/b0bnPTXES1mlvFlLPpqVqn28u/LO8Z3Q7ysTKvX wp0dXC2f8PJ01WyRO/xHj6i580MRMYbRKmXUjleD4i31nyfwxHr+fjOFWNlRNr5l Jx/fL50rxf3scuim5lXDnEJvoSWBTWTgKpOzGjkGSWD/KdSXTAbZTXPNlG3N9dXO Nfn7CytQLsU5etRqqTKZgHOciXKuXUlJeF96LRabiQ3Z7N3L99El65cDDCnTaHAd LPcgVE0OwbtubNGPPinI =9wSF -END PGP SIGNATURE- -- ## List details at https://lists.exim.org/mailman/listinfo/exim-dev Exim details at http://www.exim.org/ ##
Re: [exim-dev] XCLIENT patch to Exim; Cambridge
[ Wietse is not on the exim-dev list, forwarding with a Bcc to him... ] From: wie...@porcupine.org (Wietse Venema) Viktor Dukhovni: On Fri, Jan 16, 2015 at 12:52:46AM +, Jeremy Harris wrote: Most of the attributes look ok for us to support. I'm dubious about the LOGIN one though; this feels like a protocol level violation. In Postfix this allows proxying of SASL logins. A proxy might handle SASL auth in front of the MTA. The MTA receives the SASL login name as determined by the proxy and applies access control decisions accordingly. Specifically, this was added for nginx. Is that Postfix page the sole definition of the ESMTP option? There is an IETF draft for XFORWARD but that solves a different problem (logging instead of impersonation). I am not aware of a similar effort with respect to XCLIENT. Either way, if there is an IETF spec then I'll update Postfix where needed. Postfix accepts XCLIENT without EHLO. That is OK because XCLIENT is meant for settings where ESMTP negotiation is unnecessary. Some in-the-field implementations may rely on this optimization. Wietse [ Deleting the rest of the quoted text of my message on which Wietse did not comment. ] - End of forwarded message from Wietse Venema - -- ## List details at https://lists.exim.org/mailman/listinfo/exim-dev Exim details at http://www.exim.org/ ##
Re: [exim-dev] XCLIENT patch to Exim; Cambridge
On 2015-01-16 at 00:52 +, Jeremy Harris wrote: We need, I think: - project sponsor If Tony, at Cam, can handle it, that would be ideal; otherwise, I seem to have already done more work than we usually do for any feature being merged from a submitted patch. ;) (Of course, we also need more committers ...) -- delivery positioning (experimental?) Yes. At least until we have tests, which means figuring out how to test, a test framework, etc etc. None of which is trivial, so this is the sort of thing which goes in as EXPERIMENTAL_XCLIENT as a this code exists, we might break it still bucket. -- legal chasing We appear to have approval to include, so done. - coder/architect -- builds, testcases, documentation -- security review (coding, operational constraints, logging) -- feature-incompatibility (proxy-protocol? TLS? X509 certs?) -- coding standards -- feature spinoffs (xcode string expansions?) This should just be incompatible with proxy protocol, but with a note that there's no _useful_ interaction with TLS, as you're only verifying the connection from the loadbalancer, not from the end-client, and XCLIENT does not support passing on attributes of the TLS session, not even that there is one. So this limits authentication restriction to TLS and makes it impossible for gsasl users to set up channel binding information (which, currently, is not a loss since the current channel binding data turns out to be a security hole resulting from TLS problems). We have coding standards? I mean, that's cool, but that's also new. We should have them. What's the proposal? Are we also looking at using a decent code-review tool? [ I'm hoping you're setting yourself up for both roles... ] I can do a security review and one-time merge, but can't commit to more than that; I haven't even found time to look at the DANE work. :( There's also hummus/tahini stuff to do first, on my plate for when I have Exim time. I mostly just blinked at seeing XCLIENT in the list of build options on FreeBSD for a system where I stick to Ports, after the recent work added to the build-options so I got reprompted during a poudriere run. -Phil -- ## List details at https://lists.exim.org/mailman/listinfo/exim-dev Exim details at http://www.exim.org/ ##
Re: [exim-dev] XCLIENT patch to Exim; Cambridge
On Fri, Jan 16, 2015 at 06:34:33PM +, Phil Pennock wrote: - coder/architect -- builds, testcases, documentation -- security review (coding, operational constraints, logging) -- feature-incompatibility (proxy-protocol? TLS? X509 certs?) -- coding standards -- feature spinoffs (xcode string expansions?) This should just be incompatible with proxy protocol, but with a note that there's no _useful_ interaction with TLS, as you're only verifying the connection from the loadbalancer, not from the end-client, and XCLIENT does not support passing on attributes of the TLS session, not even that there is one. So this limits authentication restriction to TLS and makes it impossible for gsasl users to set up channel binding information (which, currently, is not a loss since the current channel binding data turns out to be a security hole resulting from TLS problems). XCLIENT is not incompatible with end-to-end TLS. Many proxies are just layer 4 load-balancers, and only engage in XCLIENT at the start of the connection (while SMTP is still doing cleartext before STARTTLS), allowing the original client to complete a TLS handshake with the ultimate back-end server. Some proxies may do TLS and even SASL auth with the client, in which case the client authenticates and perhaps channel-binds the proxy. The proxy can choose to open a second TLS channel to the backend server, and perhaps authenticate it by some means. If the TLS tunnel is the proxies, but SASL is with the backend server, that can break GSSAPI channel binding. Which SMTP clients support GSSAPI with TLS channel binding? I've not had time to contemplate that for Postfix just yet. -- Viktor. -- ## List details at https://lists.exim.org/mailman/listinfo/exim-dev Exim details at http://www.exim.org/ ##
[exim-dev] XCLIENT patch to Exim; Cambridge
FreeBSD is carrying a local patch to Exim, adding XCLIENT support. The ticket requesting its addition is at: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=133891 and there's no indication given there about why this was aimed directly at one OS's packaging, rather than at upstream. The feature documentation is: http://www.postfix.org/XCLIENT_README.html The patch can be found at: https://github.com/freebsd/freebsd-ports/blob/master/mail/exim/files/extra-patch-xclient This should probably be considered as a parallel to the proxy protocol support which we have. Aside from security review, the biggest issue is likely to be that the patch wasn't given to us and is currently a standalone work without a license statement, so we'd need to chase down the original author and ask about permission to include as part of Exim, under GPL. In 2008 there was discussion on exim-users, subject XCLIENT supported by exim?; Nigel summarized the state as zero previous discussion, no patches, so no apparent interest. The tone of the response I see was generally oh that's a Postfix thing, we just connect Exim directly to the Internet without anything in front of it. One of the last posts referenced an existing patch by Vsevolod Stakhov: http://cebka.pp.ru/blog/2007/12/xclient-exim.html http://cebka.pp.ru/blog/patch-exim-xclient but there's no longer any DNS for that host; however, the initial report in the FreeBSD PR #133891 referenced http://cebka.pp.ru/blog/2009/01/-eximxclient.html so it seems that the history of this patch in FreeBSD traces back to then, even though the FreeBSD patch has been maintained as it's patched for more recent Exim releases. I think that the biggest problem is that most postmaster folks back then didn't see the benefit of siting an Exim behind a front-end proxy, especially since this was presented as a security proxy adding features, where all the features _could_ be done in Exim already. Since then, with the widening spread of protocol-generic front-end loadbalancers, we've seen the haproxy Proxy Protocol take off, the approach of setting normally-from-getsockopt vars based upon remote data _if_ the connecting host passes an ACL has been validated and seen not to be a security issue (well, unless someone allows the extension from the open Internet, instead of just from the local trusted proxies) and I think that this is _much_ less controversial. It looks like the Vsevolod Stakhov from the original report is probably the gentleman by that name now at the University of Cambridge (oh, it's the same guy who did the cool libucl config library stuff, that's why the name was familiar :) ). On this basis, I'm going to explicitly CC Tony, also at UoC who could perhaps chat with Vsevolod, and the address found on https://github.com/vstakhov. Guys, okay to pull this patch into Exim? -Phil -- ## List details at https://lists.exim.org/mailman/listinfo/exim-dev Exim details at http://www.exim.org/ ##
Re: [exim-dev] XCLIENT patch to Exim; Cambridge
On Thu, Jan 15, 2015 at 07:00:48PM +, Phil Pennock wrote: FreeBSD is carrying a local patch to Exim, adding XCLIENT support. If you adopt this, at some point it might make sense to write up an informational I-D describing an experimental SMTP protocol extension. This makes it less Postfix-specific. It by now likely implemented in various other SMTP cleint and/or server stacks, and perhaps we could skip directly to a standards-track I-D. -- Viktor. -- ## List details at https://lists.exim.org/mailman/listinfo/exim-dev Exim details at http://www.exim.org/ ##
Re: [exim-dev] XCLIENT patch to Exim; Cambridge
On 15/01/15 19:00, Phil Pennock wrote: FreeBSD is carrying a local patch to Exim, adding XCLIENT support. [...] Guys, okay to pull this patch into Exim? If you think there's enough value / interest, yup. The three justifications for it on the Postfix page are, I think, irrelevant to use with Exim, leaving only your loadbalancer case. We need, I think: - project sponsor -- delivery positioning (experimental?) -- legal chasing - coder/architect -- builds, testcases, documentation -- security review (coding, operational constraints, logging) -- feature-incompatibility (proxy-protocol? TLS? X509 certs?) -- coding standards -- feature spinoffs (xcode string expansions?) [ I'm hoping you're setting yourself up for both roles... ] Most of the attributes look ok for us to support. I'm dubious about the LOGIN one though; this feels like a protocol level violation. OTOH, nginx (according to https://www.ruby-forum.com/topic/205086 ) does indeed violate in that way. Echhh. If we did support it I would argue strong for a separate configuration control on it. I think we'd want to know what features are actually used by existing proxy front-ends. Is that Postfix page the sole definition of the ESMTP option? I note it doesn't specify the format of an IP address (there is an example of an IPv4 one). Swaks supports it, and mentions another (undocumented!) attribute: REVERSE_NAME ... ( http://www.jetmore.org/john/code/vmail/latest/doc/ref.txt ) -- Cheers, Jeremy -- ## List details at https://lists.exim.org/mailman/listinfo/exim-dev Exim details at http://www.exim.org/ ##
Re: [exim-dev] XCLIENT patch to Exim; Cambridge
On Fri, Jan 16, 2015 at 12:52:46AM +, Jeremy Harris wrote: Most of the attributes look ok for us to support. I'm dubious about the LOGIN one though; this feels like a protocol level violation. In Postfix this allows proxying of SASL logins. A proxy might handle SASL auth in front of the MTA. The MTA receives the SASL login name as determined by the proxy and applies access control decisions accordingly. Is that Postfix page the sole definition of the ESMTP option? I think so, that's why it is perhaps time for an IETF draft, the feature seems to have caught on far beyond Postfix. Not sure whether the I-D should be informational or standards-track. XCLIENT is a local matter between various load-balancer systems and the MTA behind them, and is not generally a concern across indepedently operated systems. So informational might be less work, and still get the job done. I note it doesn't specify the format of an IP address (there is an example of an IPv4 one). Quoting XCLIENT_README: The ADDR attribute specifies an SMTP client numerical IPv4 network address, an IPv6 address prefixed with IPV6:, or [UNAVAILABLE] when the address information is unavailable. Address information is not enclosed with []. Thus one of the three forms: 192.0.2.1 IPv6:2001:db8::1 [UNAVAILABLE] Swaks supports it, and mentions another (undocumented!) attribute: REVERSE_NAME ... ( http://www.jetmore.org/john/code/vmail/latest/doc/ref.txt ) It does looks like XCLIENT_README omits that attribute which was added more recently. The full list is: #define XCLIENT_NAMENAME /* client name */ #define XCLIENT_REVERSE_NAMEREVERSE_NAME /* reverse client name */ #define XCLIENT_ADDRADDR /* client address */ #define XCLIENT_PORTPORT /* client port */ #define XCLIENT_PROTO PROTO /* client protocol */ #define XCLIENT_HELOHELO /* client helo */ #define XCLIENT_LOGIN LOGIN /* SASL login name */ The REVERSE_NAME is the unverified result of of a PTR lookup. While NAME is the FCrDNS. These are of course not always available to the proxy. -- Viktor. -- ## List details at https://lists.exim.org/mailman/listinfo/exim-dev Exim details at http://www.exim.org/ ##
