I put exactly: $Con{$fh}->{mailfrom} =~ /\@gmail\.com/ && $Con{$fh}->{SIZE} > 1024000 into debugCode
I've looks both in the base installation folder and in the logs folder, but I don't see any .dbg file. Any pointers on where else I should look? On Thu, Aug 4, 2016 at 9:54 PM, K Post <nntp.p...@gmail.com> wrote: > Now I'm in a position where the powers that be have requested that TLS be > disabled because of inbound problems from gmail. Apparently, gmail users > who send 25mb+ files have been getting this error more frequently than I > thought. > > This is an automatically generated Delivery Status Notification > > THIS IS A WARNING MESSAGE ONLY. > > YOU DO NOT NEED TO RESEND YOUR MESSAGE. > > Delivery to the following recipient has been delayed: > > ouru...@ourcharity.org > > Message will be retried for 1 more day(s) > > Technical details of temporary failure: > Missed upload deadline (899.99s) (state SENT_MESSAGE) > > One of the major donors got this today, which raised the flag with the > directors. Makes testing really tough.... > > I might be able to test this for a little bit after hours this weekend. > > > > > On Thu, Aug 4, 2016 at 3:32 AM, Thomas Eckardt <thomas.ecka...@thockar.com > > wrote: > >> debug such a connection >> >> set debugCode to: >> >> $Con{$fh}->{mailfrom} =~ /\@gmail\.com/ && $Con{$fh}->{SIZE} > 1024000 >> >> 1024000 can be larger >> >> Thomas >> >> >> >> >> >> Von: K Post <nntp.p...@gmail.com> >> An: ASSP development mailing list <assp-test@lists.sourceforge.net> >> Datum: 03.08.2016 19:08 >> Betreff: Re: [Assp-test] Inbound TLS from gmail.com addresses / >> servers >> >> >> >> watching the SMTP Connections GUI, it looks like google starts out pretty >> fast for the first 2mb or so, but then really slows down. Might there be >> something with memory handling on my end? >> >> after x seconds: total bytes transferred >> 10 seconds: 1,400,000 bytes >> 30 seconds: 2,600,000 bytes >> 55 seconds: 3,800,000 bytes >> 90 seconds: 5,300,000 bytes >> 160 seconds: 7,500,000 bytes >> >> Hit 1.4mb in the first 10 seconds, but then slows to a rate of about 2mb a >> minute, sometimes slower. Does this help you figure out what might be >> going on with gmail? >> >> >> >> >> On Tue, Aug 2, 2016 at 10:40 PM, K Post <nntp.p...@gmail.com> wrote: >> >> > activestate just published net::ssleay 1.77 in their repository. Doesn't >> > seem to make any difference in terms of speed. Capping out at about 2mb >> a >> > minute with TLS. >> > >> > the ssleay.dll that is in c:\perl\site\lib\auto\Net\SSLeay appears to >> have >> > been updated by the ppm. ASSP in infostats still says: >> > OpenSSL 1.0.2h >> > OpenSSL-lib 1.0.2g Mar 2016 >> > >> > Is that first line my c:\openssl installation from Shining Light (I >> don't >> > know anywhere else that 1.0.2h is installed)? >> > and OpenSSL-lib is the ssleay.dll that is seen in the >> > c:\perl\sit\lib\auto\net\ssleay folder? >> > >> > Does it matter that there's also a ssleay.dll in c:\openssl that is >> surely >> > 1.0.2h? >> > >> > Still, I ask all these questions, but it's only gmail that's giving me a >> > headache. Other senders all seem fine so far, no nearly as fast as >> without >> > TLS. For example, I just sent the same 11mb file that google takes >> about 7 >> > minutes to send via Outlook.com and it only took 35 seconds. >> > >> > thanks again >> > >> > >> > >> > >> > >> > On Tue, Aug 2, 2016 at 9:44 PM, K Post <nntp.p...@gmail.com> wrote: >> > >> >> scratch that Bob. I'm still closer to 1.5-2mb per minute despite the >> >> tweaks. >> >> >> >> On Tue, Aug 2, 2016 at 9:36 PM, K Post <nntp.p...@gmail.com> wrote: >> >> >> >>> Thanks Thomas, but what OpenSSL should I be using? I really don't >> think >> >>> this is the problem, but I might as well eliminate it. I've got >> >>> activestate's perl 5.20 installed and net::ssleay from the activestate >> >>> ppm. However,the OpenSSL binaries that I have (I'm talking about the >> FULL >> >>> openssl installation in c:\openssl) not the dll files that net::ssleay >> >>> >might< have, is 1.0.2h from Shiining LIght ( >> >>> slproweb.com/products/Win32OpenSSL.html) >> >>> >> >>> ASSP says net::ssleay is OpenSSL 1.0.2g - apparently it hasn't been >> >>> compiled using 1.0.2h yet. That the readme from net::ssleay talks >> >>> specifically about shining light and that it's best to roll your own >> >>> worries me. >> >>> >> >>> And Bob, >> >>> Thanks for testing this out. 3MB in 25 seconds is about what I'm >> >>> generally seeing now that I've tweaked the performance settings of >> ASSP, >> >>> but without TLS, we can receive a 10mb attachment in just a few >> seconds >> >>> thanks to a fast line. Curious, if you disable TLS temporarily and >> send >> >>> yourself that same 3mb attachment from gmail, how long does it take? >> >>> >> >>> >> >>> >> >>> On Tue, Aug 2, 2016 at 2:04 PM, Thomas Eckardt < >> >>> thomas.ecka...@thockar.com> wrote: >> >>> >> >>>> >Having looked through the Net:SSLEAY readme, there's a bunch that >> >>>> suggests >> >>>> >that it's best to compile your own net:ssleay and OpenSSL on the >> same >> >>>> >machine with the same settings. >> >>>> >> >>>> This will be the case, if you use the PPM from ActiveState. Perl and >> all >> >>>> modules are compiled with the same compiler and header files. >> >>>> Net::SSLeay >> >>>> is compiled static, means it contains all required openssl code. >> >>>> >> >>>> >I'd love to find the time to give this a go, >> >>>> You'll find something better to do, than to compile this module on >> >>>> windows. >> >>>> >> >>>> >> >>>> Thomas >> >>>> >> >>>> >> >>>> >> >>>> >> >>>> Von: K Post <nntp.p...@gmail.com> >> >>>> An: ASSP development mailing list >> <assp-test@lists.sourceforge.net> >> >>>> Datum: 02.08.2016 19:42 >> >>>> Betreff: Re: [Assp-test] Inbound TLS from gmail.com addresses >> / >> >>>> servers >> >>>> >> >>>> >> >>>> >> >>>> Having looked through the Net:SSLEAY readme, there's a bunch that >> >>>> suggests >> >>>> that it's best to compile your own net:ssleay and OpenSSL on the same >> >>>> machine with the same settings. I've not done that, and never have >> (nor >> >>>> do >> >>>> I have the skillset to do much more than run a simple make command). >> >>>> I'd >> >>>> love to find the time to give this a go, but what do you all think - >> >>>> could >> >>>> this be it? Why would gmail.com always be bad, but others not (that >> >>>> I've >> >>>> seen)? >> >>>> >> >>>> On Tue, Aug 2, 2016 at 1:22 PM, Thomas Eckardt >> >>>> <thomas.ecka...@thockar.com> >> >>>> wrote: >> >>>> >> >>>> > >How do you know the type of encryption that gmail is using? >> >>>> > >> >>>> > You'll find it in the Received header line written by assp. >> >>>> > >> >>>> > >I have SSLDebug set to level 3, >> >>>> > >> >>>> > This helps not much. Most of the SSL-debug output goes to NUL. >> >>>> > But if there were errors in SSL - you would see them in the >> maillog. >> >>>> > >> >>>> > >I changed EnableHighPerformace to "very high," >> >>>> > I don't recommend to do this. This cuts the cycle time (poll/select >> >>>> wait >> >>>> > time) in the workers to a minmum. Even if assp is idle - if this is >> >>>> set, >> >>>> > it will permanently poll the sockets and will produce unwanted CPU >> >>>> > workload. I know 'EnableHighPerformace' sounds magic, but it is >> more >> >>>> > implemented to tweak exceptional environments. >> >>>> > How ever, if your host accepts this workload - it is fine. >> >>>> > >> >>>> > >Anything else I should try tweaking? >> >>>> > >> >>>> > Don't try to much. Tweak (if) one by one step. Use the >> >>>> > 'notes/confighistory.txt' - the old and new values are recoded >> there. >> >>>> > >> >>>> > I have an idea about the gmail problem. It may be the case, that >> they >> >>>> > request SSL rehandshakes more or less often depending on the used >> >>>> > certificate and/or cipher to raise the security of the connection. >> >>>> Such >> >>>> a >> >>>> > behavior would slow down the SSL speed - BUT, now the bad news, >> this >> >>>> is >> >>>> a >> >>>> > client request (made my gmail). Perl's Net::SSLeay has no easy way >> to >> >>>> > ignore these requests. The only way would be to pipe all SSL >> packest >> >>>> > through an assp routine (this is possible), which would drop the >> >>>> > renegotiation requests. Such a code will slow down ALL SSL traffic >> >>>> > dramaticaly, if written in pure perl. >> >>>> > >> >>>> > >We are using a 2048bit certificate. It's a wildcard (*. >> >>>> ourcharity.org) >> >>>> > >cert, but I don't think that has anything to do with it. >> >>>> > >> >>>> > Who knows? But to exclude this, you may use an innocent selfcert >> >>>> > certificate and key - create it with openssl - for a while. >> >>>> > BTW. assp will create such certificate and keys, if the >> 'assp/certs' >> >>>> > folder is empty at startup. :):) >> >>>> > >> >>>> > Thomas >> >>>> > >> >>>> > >> >>>> > >> >>>> > >> >>>> > Von: K Post <nntp.p...@gmail.com> >> >>>> > An: ASSP development mailing list < >> >>>> assp-test@lists.sourceforge.net> >> >>>> > Datum: 02.08.2016 18:34 >> >>>> > Betreff: Re: [Assp-test] Inbound TLS from gmail.com >> addresses >> >>>> / >> >>>> > servers >> >>>> > >> >>>> > >> >>>> > >> >>>> > Thanks for chiming in Thomas with such a detailed response. >> >>>> > >> >>>> > First, when Google gives up, it gives a message like: >> >>>> > >> >>>> > Technical details of temporary failure: >> >>>> > >> >>>> > Missed upload deadline (899.97s) (state SENT_MESSAGE) >> >>>> > >> >>>> > So it's 15 minutes that it'll try to send a file for. At under 2mb >> a >> >>>> > minute, anything over about 25megs (considering overhead) will >> >>>> ultimately >> >>>> > fail. No good since lots of gmail users send us large files. >> >>>> > >> >>>> > >> >>>> > We're on a 100mbit line, both directions, but I'd happily take a >> 9.1 >> >>>> mb >> >>>> > attachment sent over TLS taking 2 minutes. I suspect when i find >> out >> >>>> what >> >>>> > the problem is that it'll be MUCh faster than that. >> >>>> > >> >>>> > We are using a 2048bit certificate. It's a wildcard (*. >> >>>> ourcharity.org) >> >>>> > cert, but I don't think that has anything to do with it. >> >>>> > >> >>>> > We're using local storage on the Hypver-V host, RAID 10 with 4 >> 7200rpm >> >>>> SAS >> >>>> > drives. It's not the fasted disk array, but it seems fine. I >> can't >> >>>> see >> >>>> > slow disks impacting TLS like this if non-TLS connections fly. >> >>>> > >> >>>> > The hyper-v host is a dual processor, 2.6ghz, 6 core each, 12mb >> cache. >> >>>> > I've got a total of 10 cores assigned to the ASSP guest. >> >>>> > >> >>>> > I have SSLDebug set to level 3, but I don't see anything in the >> >>>> maillog. >> >>>> > How do you know the type of encryption that gmail is using? It >> would >> >>>> be >> >>>> > nice to compare how gmail is connecting vs outlook.com which seems >> >>>> much >> >>>> > faster (though not super fast) >> >>>> > >> >>>> > I've got SSL_Version set to >> >>>> > SSLv23:!SSLv3:!SSLv2 >> >>>> > >> >>>> > and >> >>>> > SSL_Cipher_List set to >> >>>> > >> >>>> > >> >>>> >> >>>> >> kEECDH+ECDSA:kEECDH:kEDH:HIGH:+SHA:+RC4:RC4:!aNULL:!eNULL:!L >> OW:!3DES:!MD5:!EXP:!DSS:!PSK:!SRP:!kECDH:!CAMELLIA128:!IDEA:!SEED >> >>>> > >> >>>> > my unscientific test of changing the cipher list to the default >> >>>> doesn't >> >>>> > seem to make a difference. >> >>>> > >> >>>> > MinPollTime is 1, I think it always has been. >> >>>> > I changed EnableHighPerformace to "very high," changed thread cycle >> >>>> time >> >>>> > to >> >>>> > 1000, maintenance thread cycle time to 2000, and >> >>>> rebuildthreadcycletime >> >>>> to >> >>>> > 15. That definitely made a difference in the rebuild time, almost >> >>>> halving >> >>>> > it (not that I really care about that though). >> >>>> > >> >>>> > Anything else I should try tweaking? I don't care if there's high >> CPU >> >>>> > usage, we have reasonable processing power to spare. >> >>>> > >> >>>> > Thank you >> >>>> > >> >>>> > On Tue, Aug 2, 2016 at 12:02 PM, Thomas Eckardt >> >>>> > <thomas.ecka...@thockar.com> >> >>>> > wrote: >> >>>> > >> >>>> > > I just made simlar tests with my gmail account. I can't reproduce >> >>>> this >> >>>> > > behavior related to gmail.com. >> >>>> > > >> >>>> > > I've sent a 9.1MB attachment in 133 seconds. Gmail used >> >>>> SMTPS(TLSv1_2 >> >>>> > > ECDHE-RSA-AES256-GCM-SHA384)- which is commonly used by many >> >>>> > > clients/servers. >> >>>> > > Sender was mail-qt0-f181.google.com ([209.85.216.181] >> >>>> > > helo=mail-qt0-f181.google.com) >> >>>> > > My line speed is 16MB/s inbound and 4MB/s outbound. >> >>>> > > >> >>>> > > I saw many faster SMTPS connections but also many slower - this >> may >> >>>> > depend >> >>>> > > on the usage of my ISP connection. >> >>>> > > >> >>>> > > 133 seconds for such a mail is acceptable (I think). >> >>>> > > >> >>>> > > SSLv2/3:!SSLv3:!SSLv2 >> >>>> > > DEFAULT:!aNULL:!RC4:!MD5 >> >>>> > > >> >>>> > > are my SSL settings - not very strong - I know :):) >> >>>> > > >> >>>> > > the privat key used is 2048 Bit long >> >>>> > > >> >>>> > > In front of assp is the ISP-router and a pfsense 2.3.2 with snort >> >>>> > 3.2.9.1 >> >>>> > > . Snort is configured the very hard way, except the SMTP rules >> are a >> >>>> bit >> >>>> > > more weak, because I need some spam. >> >>>> > > ASSP is running on a 4 Core 6GB W2K3 enterprise with an absolute >> >>>> > uptodate >> >>>> > > ActivePerl 5.16.3 - using all Plugins, features and a replicated >> >>>> MySQL >> >>>> > > 5.6. >> >>>> > > Domain based mail routing (in- and out-bound) is done by >> hmailserver >> >>>> > > 5.6.4-B2283. >> >>>> > > All components are configured to use SSL/TLS when ever this is >> >>>> possible. >> >>>> > > For testing purposes I use a FreeBSD 10.2 with Perl 5.20 and ASSP >> - >> >>>> it >> >>>> > > runs the same way stable like the production system. >> >>>> > > >> >>>> > > You see - nothing magic, but maintenained (except the nice old >> W2K3 >> >>>> - >> >>>> > but >> >>>> > > it works like a swiss made watch with an ETA 7750). >> >>>> > > >> >>>> > > I really don't know what I can do to fix up the SSL/TLS problems. >> >>>> > > >> >>>> > > Only to be complete: >> >>>> > > Backend for the mail environment and LDAP stuff is a Domino >> >>>> 9.0.1FP6. >> >>>> > > All the stuff above (and very much more) is running on a single >> >>>> VMWare >> >>>> > > vSphere 5.5 ( 8x 2.66GHz 48GB / x3650M2). >> >>>> > > Backups are done with EMC-Networker + EBR + DataDomain-VE, stored >> >>>> at a >> >>>> > > QNAP 419P+ >> >>>> > > >> >>>> > > Thomas >> >>>> > > >> >>>> > > >> >>>> > > >> >>>> > > >> >>>> > > Von: K Post <nntp.p...@gmail.com> >> >>>> > > An: ASSP development mailing list >> >>>> <assp-test@lists.sourceforge.net> >> >>>> > > Datum: 02.08.2016 00:07 >> >>>> > > Betreff: [Assp-test] Inbound TLS from gmail.com addresses >> / >> >>>> > servers >> >>>> > > >> >>>> > > >> >>>> > > >> >>>> > > I originally thought that we had a problem with all TLS inbound >> >>>> email. >> >>>> > As >> >>>> > > it turns out, my conclusion appears to have been wrong. >> >>>> > > >> >>>> > > >> >>>> > > - There are some SLOW servers outside that are just plain slow >> >>>> > (nothing >> >>>> > > I can do there), >> >>>> > > >> >>>> > > - TLS seems to work reasonably fast with most inbound mail, >> >>>> though >> >>>> > > significantly slower than without TLS (5 seconds for an 11mb >> >>>> file >> >>>> > > without >> >>>> > > tls, vs 45 seconds with TLS on) >> >>>> > > >> >>>> > > - GMAIL.com inbound TLS emails are SLOW, no matter what >> settings >> >>>> I >> >>>> > > tweak >> >>>> > > >> >>>> > > >> >>>> > > With inbound gmail.com message. if I have TLS off, an 11mb >> >>>> attachment >> >>>> is >> >>>> > > delivered through ASSP in under 5 seconds. With TLS on it takes >> >>>> close >> >>>> > to >> >>>> > > 10 minutes, which gets close to gmail's limit. >> >>>> > > >> >>>> > > I've tested with Outlook.com and that same 11mb attachment comes >> in >> >>>> > > through >> >>>> > > ASSP with TLS on in about 45 seconds. >> >>>> > > >> >>>> > > Sending a 30mb attachment from gmail FAILS because it takes too >> >>>> long. >> >>>> > > gmail >> >>>> > > will try for I believe 10 minutes to send a message, then it >> quits >> >>>> and >> >>>> > > retries. After a couple tries, it sends an NDR. >> >>>> > > >> >>>> > > This is a Windows 2012 R2 server, latest ASSP dev, OpenSSL 1.0.2h >> >>>> > > installed >> >>>> > > from slproweb.com/products/Win32OpenSSL.html (though I've also >> >>>> tried >> >>>> > with >> >>>> > > the OpenSSL I downloaded a while back from the ASSP sourceforge >> >>>> site. >> >>>> > > net::ssleay 1.74 (openssl 1.0.2g). I'm almost certain that the >> >>>> OpenSSL >> >>>> > > installation is not used by ASSP, but I've not been able to get >> >>>> > > confirmation of that here. >> >>>> > > >> >>>> > > Just updated IO::Socket::SSL to 2.033. >> >>>> > > Net::SMTP:SSL 1.02. >> >>>> > > >> >>>> > > CPU usage as reported by assp is 4.78%. It's not on the fastest >> >>>> machine >> >>>> > > in >> >>>> > > the world (it's a hypver-v guest on a decent machine), but it >> seems >> >>>> > speedy >> >>>> > > enough. 24gb ram. We've got similar physical hosts running >> >>>> Exchange >> >>>> as >> >>>> > a >> >>>> > > guest without any speed issues whatsoever. >> >>>> > > >> >>>> > > Any other info I can provide to help figure this out? >> >>>> > > >> >>>> > > Disabling TLS for any gmail inbound mail isn't a feasible option, >> >>>> plus >> >>>> I >> >>>> > > don't know if it really is just google, or just the way that >> google >> >>>> > > connects which others might too... >> >>>> > > >> >>>> > > Thank you all. >> >>>> > > >> >>>> > > >> >>>> > >> >>>> > >> >>>> >> >>>> >> ------------------------------------------------------------ >> ------------------ >> >>>> > > _______________________________________________ >> >>>> > > Assp-test mailing list >> >>>> > > Assp-test@lists.sourceforge.net >> >>>> > > https://lists.sourceforge.net/lists/listinfo/assp-test >> >>>> > > >> >>>> > > >> >>>> > > >> >>>> > > >> >>>> > > DISCLAIMER: >> >>>> > > ******************************************************* >> >>>> > > This email and any files transmitted with it may be confidential, >> >>>> > legally >> >>>> > > privileged and protected in law and are intended solely for the >> use >> >>>> of >> >>>> > the >> >>>> > > >> >>>> > > individual to whom it is addressed. >> >>>> > > This email was multiple times scanned for viruses. There should >> be >> >>>> no >> >>>> > > known virus in this email! >> >>>> > > ******************************************************* >> >>>> > > >> >>>> > > >> >>>> > > >> >>>> > > >> >>>> > >> >>>> > >> >>>> >> >>>> >> ------------------------------------------------------------ >> ------------------ >> >>>> > > >> >>>> > > _______________________________________________ >> >>>> > > Assp-test mailing list >> >>>> > > Assp-test@lists.sourceforge.net >> >>>> > > https://lists.sourceforge.net/lists/listinfo/assp-test >> >>>> > > >> >>>> > > >> >>>> > >> >>>> > >> >>>> >> >>>> >> ------------------------------------------------------------ >> ------------------ >> >>>> > _______________________________________________ >> >>>> > Assp-test mailing list >> >>>> > Assp-test@lists.sourceforge.net >> >>>> > https://lists.sourceforge.net/lists/listinfo/assp-test >> >>>> > >> >>>> > >> >>>> > >> >>>> > >> >>>> > DISCLAIMER: >> >>>> > ******************************************************* >> >>>> > This email and any files transmitted with it may be confidential, >> >>>> legally >> >>>> > privileged and protected in law and are intended solely for the use >> of >> >>>> the >> >>>> > >> >>>> > individual to whom it is addressed. >> >>>> > This email was multiple times scanned for viruses. There should be >> no >> >>>> > known virus in this email! >> >>>> > ******************************************************* >> >>>> > >> >>>> > >> >>>> > >> >>>> > >> >>>> >> >>>> >> ------------------------------------------------------------ >> ------------------ >> >>>> > >> >>>> > _______________________________________________ >> >>>> > Assp-test mailing list >> >>>> > Assp-test@lists.sourceforge.net >> >>>> > https://lists.sourceforge.net/lists/listinfo/assp-test >> >>>> > >> >>>> > >> >>>> >> >>>> >> ------------------------------------------------------------ >> ------------------ >> >>>> _______________________________________________ >> >>>> Assp-test mailing list >> >>>> Assp-test@lists.sourceforge.net >> >>>> https://lists.sourceforge.net/lists/listinfo/assp-test >> >>>> >> >>>> >> >>>> >> >>>> >> >>>> DISCLAIMER: >> >>>> ******************************************************* >> >>>> This email and any files transmitted with it may be confidential, >> >>>> legally >> >>>> privileged and protected in law and are intended solely for the use >> of >> >>>> the >> >>>> >> >>>> individual to whom it is addressed. >> >>>> This email was multiple times scanned for viruses. There should be no >> >>>> known virus in this email! >> >>>> ******************************************************* >> >>>> >> >>>> >> >>>> >> >>>> >> ------------------------------------------------------------ >> ------------------ >> >>>> >> >>>> _______________________________________________ >> >>>> Assp-test mailing list >> >>>> Assp-test@lists.sourceforge.net >> >>>> https://lists.sourceforge.net/lists/listinfo/assp-test >> >>>> >> >>>> >> >>> >> >> >> > >> ------------------------------------------------------------ >> ------------------ >> _______________________________________________ >> Assp-test mailing list >> Assp-test@lists.sourceforge.net >> https://lists.sourceforge.net/lists/listinfo/assp-test >> >> >> >> >> DISCLAIMER: >> ******************************************************* >> This email and any files transmitted with it may be confidential, legally >> privileged and protected in law and are intended solely for the use of the >> >> individual to whom it is addressed. >> This email was multiple times scanned for viruses. There should be no >> known virus in this email! >> ******************************************************* >> >> >> ------------------------------------------------------------ >> ------------------ >> >> _______________________________________________ >> Assp-test mailing list >> Assp-test@lists.sourceforge.net >> https://lists.sourceforge.net/lists/listinfo/assp-test >> >> >
------------------------------------------------------------------------------
_______________________________________________ Assp-test mailing list Assp-test@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/assp-test