Re: postfix milter body chunk length
> Postfix with default milter body chunk size 65535: > > mail processing time 1m30.154298259s > > Postfix with milter body chunk size 1048576: > > mail processing time 17.52360866s it looks to me like postfix is able to feed a milter very quickly i just did a couple of quick tests here, an ~83 MiB message only takes a second or two to pass through two milters (debug log shows 1360 64K fragments and one smaller tail fragment)
Re: SSL communication between MTAs
On Fri, Aug 16, 2019 at 04:53:23PM +1000, Viktor Dukhovni wrote: > Bottom line, only trust local resolvers you deploy, configure > *correctly* and test. Well, it doesn't _have_ to be local. You could, for instance, be connected to a resolver that you know you can trust (FSVO "know" and "trust") over IPsec. I believe that was the use case originally for the AD bit, which otherwise is more or less useless for all the reasons you outline. (Your general point, of course, still stands.) A -- Andrew Sullivan a...@anvilwalrusden.com
Re: postfix milter body chunk length
Matthias Schneider: > Any chance to get a config for MILTER_CHUNK_SIZE? It would be great so > have an easy and fast switch. I may have different settings on different > mail systems (because of 3rd party milters), its hard to manage > different customized postfix versions in my distribution repositories. Why? It's a compile-time constant in sendmail/libmilter. In Postfix, MILTER_CHUNK_SIZE is not the only constant to limit packet sizes. There is also the XXX_MAX_DATA sanity limit for responses from the Milters, that will trigger a panic if the size is exceeded (that should probably be just a protocol error). All that stuff would need to be updated to make the packet size configurable without introducing unstable edge cases. Wietse > > Thanks, > > Matthias Schneider > > Am 16.08.19 um 15:32 schrieb Matthias Schneider: > > Any chance to get a config for MILTER_CHUNK_SIZE? It would be great so > > have an easy and fast switch. I may have different settings on > > different mail systems (because of 3rd party milters), its hard to > > manage different customized postfix versions in my distribution > > repositories. > > > > > > Thanks, > > > > Matthias Schneider > > > > > > > > Am 16.08.19 um 14:49 schrieb Viktor Dukhovni: > >>> On Aug 16, 2019, at 10:33 PM, Matthias Schneider > >>> wrote: > >>> > >>> I was wondering why the transfer of a 100mb mail to my milter > >>> application was slow, i found the bottleneck in the body chunk > >>> transfer. > >>> > >>> The maximum packet length seems to be fixed to 64k, it would be > >>> great if we could make that configurable in postfix (uint32 is > >>> possible). > >> * milter communication fails if a single header is larger than 64K. > >> > >> ?? If a single header is larger than 64KB (which is not possible in the > >> ?? default configuration) then it cannot be transferred in one block to > >> ?? libmilter and hence the communication fails.? This can be avoided by > >> ?? increasing the constant MILTER_CHUNK_SIZE in > >> ?? include/libmilter/mfdef.h and recompiling sendmail, libmilter, and > >> ?? all (statically linked) milters (or by using undocumented compile > >> ?? time options: _FFR_MAXDATASIZE/_FFR_MDS_NEGOTIATE; you have to > >> ?? read the source code in order to use these properly). > >> > Wietse
Re: postfix milter body chunk length
Matthias Schneider: > my tests are little bit frustrating: > > > Postfix with default milter body chunk size 65535: > > mail processing time 1m30.154298259s > > Postfix with milter body chunk size 1048576: 16x larger packet size. > mail processing time 17.52360866s 5x faster processing. You can get 4x with a much smaller packet size. But who gives a * about memory these days. Wietse > (same mail of 107421kb size / some time is added in body() but it should > be comparable) > > > Best regards > > Matthias Schneider > > > > Am 16.08.19 um 15:50 schrieb Wietse Venema: > > > > On the Postfix side, edit src/milter/milter8.c and update its > > MILTER_CHUNK_SIZE definition accordingly. It's only compile-time > > configurable in Postfix, because it's only compile-time configurable > > in libmilter. > > > > Wietse >
Re: postfix milter body chunk length
Am 16.08.19 um 16:03 schrieb Viktor Dukhovni: You could raise the limit, but it may be more productive to figure out why your milter application is "slow" with the default chunk size. AFAIK, other milter applications are doing fine with the default size. I am using the golang implementation: https://github.com/mschneider82/milter/blob/master/session.go#L288 I think the problem is that every chunk needs to be accepted by the milter. 109.999.104 bytes / 65535 = 1679 accepts I have the same speed issue in the java library too. Best regards Matthias Schneider
Re: postfix milter body chunk length
my tests are little bit frustrating: Postfix with default milter body chunk size 65535: mail processing time 1m30.154298259s Postfix with milter body chunk size 1048576: mail processing time 17.52360866s (same mail of 107421kb size / some time is added in body() but it should be comparable) Best regards Matthias Schneider Am 16.08.19 um 15:50 schrieb Wietse Venema: On the Postfix side, edit src/milter/milter8.c and update its MILTER_CHUNK_SIZE definition accordingly. It's only compile-time configurable in Postfix, because it's only compile-time configurable in libmilter. Wietse
Re: postfix milter body chunk length
On Fri, Aug 16, 2019 at 03:51:05PM +0200, Matthias Schneider wrote: > Any chance to get a config for MILTER_CHUNK_SIZE? It would be great so > have an easy and fast switch. I may have different settings on different > mail systems (because of 3rd party milters), its hard to manage > different customized postfix versions in my distribution repositories. You could raise the limit, but it may be more productive to figure out why your milter application is "slow" with the default chunk size. AFAIK, other milter applications are doing fine with the default size. -- Viktor.
Re: postfix milter body chunk length
Any chance to get a config for MILTER_CHUNK_SIZE? It would be great so have an easy and fast switch. I may have different settings on different mail systems (because of 3rd party milters), its hard to manage different customized postfix versions in my distribution repositories. Thanks, Matthias Schneider Am 16.08.19 um 15:32 schrieb Matthias Schneider: Any chance to get a config for MILTER_CHUNK_SIZE? It would be great so have an easy and fast switch. I may have different settings on different mail systems (because of 3rd party milters), its hard to manage different customized postfix versions in my distribution repositories. Thanks, Matthias Schneider Am 16.08.19 um 14:49 schrieb Viktor Dukhovni: On Aug 16, 2019, at 10:33 PM, Matthias Schneider wrote: I was wondering why the transfer of a 100mb mail to my milter application was slow, i found the bottleneck in the body chunk transfer. The maximum packet length seems to be fixed to 64k, it would be great if we could make that configurable in postfix (uint32 is possible). * milter communication fails if a single header is larger than 64K. If a single header is larger than 64KB (which is not possible in the default configuration) then it cannot be transferred in one block to libmilter and hence the communication fails. This can be avoided by increasing the constant MILTER_CHUNK_SIZE in include/libmilter/mfdef.h and recompiling sendmail, libmilter, and all (statically linked) milters (or by using undocumented compile time options: _FFR_MAXDATASIZE/_FFR_MDS_NEGOTIATE; you have to read the source code in order to use these properly).
Re: postfix milter body chunk length
Viktor Dukhovni: > > On Aug 16, 2019, at 10:33 PM, Matthias Schneider > > wrote: > > > > I was wondering why the transfer of a 100mb mail to my milter application > > was slow, i found the bottleneck in the body chunk transfer. > > > > The maximum packet length seems to be fixed to 64k, it would be great if we > > could make that configurable in postfix (uint32 is possible). > > https://github.com/avar/sendmail-pmilter/blob/master/doc/milter-protocol.txt#L182-L213 > > http://ftp.sendmail.org/KNOWNBUGS > > * milter communication fails if a single header is larger than 64K. > > If a single header is larger than 64KB (which is not possible in the > default configuration) then it cannot be transferred in one block to > libmilter and hence the communication fails. This can be avoided by > increasing the constant MILTER_CHUNK_SIZE in > include/libmilter/mfdef.h and recompiling sendmail, libmilter, and > all (statically linked) milters (or by using undocumented compile > time options: _FFR_MAXDATASIZE/_FFR_MDS_NEGOTIATE; you have to > read the source code in order to use these properly). On the Postfix side, edit src/milter/milter8.c and update its MILTER_CHUNK_SIZE definition accordingly. It's only compile-time configurable in Postfix, because it's only compile-time configurable in libmilter. Wietse
Re: postfix milter body chunk length
> On Aug 16, 2019, at 10:33 PM, Matthias Schneider > wrote: > > I was wondering why the transfer of a 100mb mail to my milter application was > slow, i found the bottleneck in the body chunk transfer. > > The maximum packet length seems to be fixed to 64k, it would be great if we > could make that configurable in postfix (uint32 is possible). https://github.com/avar/sendmail-pmilter/blob/master/doc/milter-protocol.txt#L182-L213 http://ftp.sendmail.org/KNOWNBUGS * milter communication fails if a single header is larger than 64K. If a single header is larger than 64KB (which is not possible in the default configuration) then it cannot be transferred in one block to libmilter and hence the communication fails. This can be avoided by increasing the constant MILTER_CHUNK_SIZE in include/libmilter/mfdef.h and recompiling sendmail, libmilter, and all (statically linked) milters (or by using undocumented compile time options: _FFR_MAXDATASIZE/_FFR_MDS_NEGOTIATE; you have to read the source code in order to use these properly). -- Viktor.
postfix milter body chunk length
Hi, I was wondering why the transfer of a 100mb mail to my milter application was slow, i found the bottleneck in the body chunk transfer. The maximum packet length seems to be fixed to 64k, it would be great if we could make that configurable in postfix (uint32 is possible). best regards, Matthias Schneider
Re: SSL communication between MTAs
> On Aug 16, 2019, at 1:29 AM, Viktor Dukhovni > wrote: > > enable DANE outbound: > > http://www.postfix.org/TLS_README.html#client_tls_dane > > main.cf: > smtp_dns_support_level = dnssec > smtp_tls_security_level = dane > > /etc/resolv.conf > # A validating *local* resolver > nameserver 127.0.0.1 I got an off-list suggestion to stress the importance of the validating resolver being *local* to the Postfix server. In addition to improved performance when the DNS cache is local, this avoids potential MiTM attacks that "forge" the AD bit or data of a DNS response. The Postfix DANE code fully trusts answers from the configured resolvers, and only provides meaningful resistance to active attacks when traffic between the validating resolver and Postfix is not vulnerable to modification in transit. And with distant validating resolvers you have no control over the timing and reliability of potential changes in their validation logic. For example, 8.8.8.8 and 8.8.4.4 returned incorrect AD bits for some domains for a few days this past week (now believed resolved). Bottom line, only trust local resolvers you deploy, configure *correctly* and test. -- Viktor.
