Re: [qmailtoaster] Very slow mail submission
My apologies, everyone… tcpserver status code 256 apparently means that the client did not gracefully terminate the connection? So something is timing it out after 60 seconds. I tried a different client and a test message went right through, so it appears to be a client-side issue (Apple Mail on OS 11.1). Sorry for spamming the list, but maybe this will help a future googler. :/ - Steve > On Jan 30, 2021, at 2:53 PM, Eric Broch wrote: > > Try a different dns server > > On 1/30/21 12:25 PM, Steve Linberg wrote: >> Greetings all. I’ve been running qmail on my servers and vms for about 20 >> years and it’s mostly worked great, but I’m having a thumper of a slowdown >> problem on my current setup and I can’t get to the bottom of it… any help >> appreciated. >> >> The environment is a very lightly-used private VM running centos7 and the >> following packages: >> >> [root@radbug ~]# yum list installed | grep qmail >> qmail.x86_641.03-2.2.1.qt.el7 @qmt-current >> qmailadmin.x86_64 1.2.16-3.2.qt.el7 @qmt-current >> qmailmrtg.x86_644.2-3.qt.el7 @qmt-current >> >> I’ve narrowed it down to a delay shown in /var/log/qmail/submission/current, >> when I send a test message to myself at another address, using my vm as my >> mail server. The first lines hit immediately when I click “send” in the mail >> client, but then it takes a full minute between the tcpserver “end” message >> and whatever preceded it, during which time my mail client hangs at >> “sending…”: (annotations [*** like this ***] inline are mine, obviously, and >> addresses/servers replaced with caps but the IPs are unchanged): >> >> 2021-01-30 13:42:52.255716500 tcpserver: status: 0/100 >> [*** message sent. ***] >> 2021-01-30 13:47:33.912724500 tcpserver: status: 1/100 >> 2021-01-30 13:47:33.912947500 tcpserver: pid 6009 from 24.62.203.29 >> 2021-01-30 13:47:33.912948500 tcpserver: ok 6009 MYHOST:104.236.46.99:587 >> :24.62.203.29::55124 >> [*** waiting... one minute 4 seconds pass ***] >> 2021-01-30 13:48:37.899374500 tcpserver: end 6009 status 256 >> 2021-01-30 13:48:37.899376500 tcpserver: status: 0/100 >> 2021-01-30 13:48:37.930988500 tcpserver: status: 1/100 >> 2021-01-30 13:48:37.931122500 tcpserver: pid 6021 from 24.62.203.29 >> 2021-01-30 13:48:37.931198500 tcpserver: ok 6021 MYHOST:104.236.46.99:587 >> :24.62.203.29::55129 >> 2021-01-30 13:48:38.263537500 CHKUSER accepted sender: from >> mailto:m...@myserver.com>:m...@myserver.com >> <mailto:m...@myserver.com>:> remote <[10.0.98.178]:unknown:24.62.203.29> >> rcpt <> : sender accepted >> 2021-01-30 13:48:38.327973500 CHKUSER relaying rcpt: from > <mailto:m...@myserver.com>:m...@myserver.com <mailto:m...@myserver.com>:> >> remote <[10.0.98.178]:unknown:24.62.203.29> rcpt > <mailto:m...@otherserver.com>> : client allowed to relay >> 2021-01-30 13:48:38.327975500 policy_check: local m...@myserver.com >> <mailto:m...@myserver.com> -> remote m...@otherserver.com >> <mailto:m...@otherserver.com> (AUTHENTICATED SENDER) >> 2021-01-30 13:48:38.327976500 policy_check: policy allows transmission >> [*** waiting... (message received) then one more minute passes ***] >> 2021-01-30 13:49:38.403478500 tcpserver: end 6021 status 0 >> 2021-01-30 13:49:38.403479500 tcpserver: status: 0/100 >> >> It’s very consistent in this timing, and loads on the server are basically >> 0, nothing in /var/log/messages or anywhere else that I can see. Vpopmail is >> using mysql, but it isn’t logging any slow queries, and this is more or less >> the only thing happening on the box. It’s been doing it for a while now, >> maybe a month or two? I don’t send much mail so I didn’t lean into it too >> hard when it started happening, but now it’s really annoying me and I want >> to get to the bottom of it. (Inbound mail TO the server from other locations >> (like the auto response to the “subscribe” message I sent to the list just >> now) lands instantly.) >> >> My /etc/tcprules.d/tcp.smtp: (I thought it might have been something about >> domain keys, so I removed that but it hasn’t made any difference) >> 127.:allow,BADMIMETYPE="",BADLOADERTYPE="M",CHKUSER_RCPTLIMIT="50",CHKUSER_WRONGRCPTLIMIT="10",NOP0FCHECK="1",QMAILQUEUE="/var/qmail/bin/simscan" >> >> My /var/qmail/supervise/submission/run: >> #!/bin/sh
Re: [qmailtoaster] Very slow mail submission
Thanks Bill… I’m not running a local DNS. And regular nslookups from the command-line are instantaneous, if that’s significant, using either 8.8.8.8 or 1.1.1.1, so I’m not sure that DNS is the issue here unless a different modality would explain it; as I mentioned to Eric, I tried 1.1.1.1 and it gave the same delays as when I was using 8.8.8.8. > On Jan 30, 2021, at 3:09 PM, Bill Silverstein wrote: > > One problem I ran into is that I was using djbdns as a forward online > caching relay to 1.1.1.1, but had problems with SpamAssassin. Some of the > SpamAssassin failed do to the black list lookups seeing it come from and > overused address (1.1.1.1). > > To work around this, I set the resolv.conf to 1.1.1.1, but added 127.0.0.1 > (not SpamAssassin by adding to the local.cf: > dns_available yes > dns_server 127.0.0.1 > > This way I get the speed of 1.1.1.1, but still can have the lookups. > > > > On Sat, January 30, 2021 11:53 am, Eric Broch wrote: >> Try a different dns server >> >> On 1/30/21 12:25 PM, Steve Linberg wrote: >>> Greetings all. I’ve been running qmail on my servers and vms for about >>> 20 years and it’s mostly worked great, but I’m having a thumper of a >>> slowdown problem on my current setup and I can’t get to the bottom of >>> it… any help appreciated. >>> >>> The environment is a very lightly-used private VM running centos7 and >>> the following packages: >>> >>> [root@radbug ~]# yum list installed | grep qmail >>> qmail.x86_64 1.03-2.2.1.qt.el7 >>> @qmt-current >>> qmailadmin.x86_64 1.2.16-3.2.qt.el7 >>> >>> @qmt-current >>> qmailmrtg.x86_64 4.2-3.qt.el7 >>> @qmt-current >>> >>> I’ve narrowed it down to a delay shown in >>> /var/log/qmail/submission/current, when I send a test message to >>> myself at another address, using my vm as my mail server. The first >>> lines hit immediately when I click “send” in the mail client, but >>> then >>> it takes a full minute between the tcpserver “end” message and >>> whatever preceded it, during which time my mail client hangs at >>> “sending…”: (annotations [*** like this ***] inline are mine, >>> obviously, and addresses/servers replaced with caps but the IPs are >>> unchanged): >>> >>> 2021-01-30 13:42:52.255716500 tcpserver: status: 0/100 >>> [*** message sent. ***] >>> 2021-01-30 13:47:33.912724500 tcpserver: status: 1/100 >>> 2021-01-30 13:47:33.912947500 tcpserver: pid 6009 from 24.62.203.29 >>> 2021-01-30 13:47:33.912948500 tcpserver: ok 6009 >>> MYHOST:104.236.46.99:587 :24.62.203.29::55124 >>> [*** waiting... one minute 4 seconds pass ***] >>> 2021-01-30 13:48:37.899374500 tcpserver: end 6009 status 256 >>> 2021-01-30 13:48:37.899376500 tcpserver: status: 0/100 >>> 2021-01-30 13:48:37.930988500 tcpserver: status: 1/100 >>> 2021-01-30 13:48:37.931122500 tcpserver: pid 6021 from 24.62.203.29 >>> 2021-01-30 13:48:37.931198500 tcpserver: ok 6021 >>> MYHOST:104.236.46.99:587 :24.62.203.29::55129 >>> 2021-01-30 13:48:38.263537500 CHKUSER accepted sender: from >>> mailto:m...@myserver.com> <mailto:m...@myserver.com >>> <mailto:m...@myserver.com>>:m...@myserver.com <mailto:m...@myserver.com> >>> <mailto:m...@myserver.com <mailto:m...@myserver.com>>:> remote >>> <[10.0.98.178]:unknown:24.62.203.29> >>> rcpt <> : sender accepted >>> 2021-01-30 13:48:38.327973500 CHKUSER relaying rcpt: from >>> mailto:m...@myserver.com> <mailto:m...@myserver.com >>> <mailto:m...@myserver.com>>:m...@myserver.com <mailto:m...@myserver.com> >>> <mailto:m...@myserver.com <mailto:m...@myserver.com>>:> remote >>> <[10.0.98.178]:unknown:24.62.203.29> >>> rcpt mailto:m...@otherserver.com> >>> <mailto:m...@otherserver.com <mailto:m...@otherserver.com>>> : client >>> allowed >>> to relay >>> 2021-01-30 13:48:38.327975500 policy_check: local m...@myserver.com >>> <mailto:m...@myserver.com> >>> <mailto:m...@myserver.com <mailto:m...@myserver.com>> ->remote >>> m...@otherserver.com <mailto:m...@otherserver.com> >>> <mailto:m...@otherserver.com <mailto:m...@otherserver.com>> (AUTHENTICATED >>> SENDER) >>> 2021-01-30 13:48:38.327976500 policy_check: policy allows transmission >>> [*** wai
Re: [qmailtoaster] Very slow mail submission
Thanks Eric, I changed from 8.8.8.8 to 1.1.1.1, but no difference… still a one-minute delay in the same place: 2021-01-30 15:45:02.015947500 tcpserver: pid 2233 from 24.62.203.29 2021-01-30 15:45:02.016782500 tcpserver: ok 2233 MYHOST:104.236.46.99:587 :24.62.203.29::56627 2021-01-30 15:46:02.080429500 tcpserver: end 2233 status 256 2021-01-30 15:46:02.080430500 tcpserver: status: 0/100 Any other ideas? > On Jan 30, 2021, at 2:53 PM, Eric Broch wrote: > > Try a different dns server > > On 1/30/21 12:25 PM, Steve Linberg wrote: >> Greetings all. I’ve been running qmail on my servers and vms for about 20 >> years and it’s mostly worked great, but I’m having a thumper of a slowdown >> problem on my current setup and I can’t get to the bottom of it… any help >> appreciated. >> >> The environment is a very lightly-used private VM running centos7 and the >> following packages: >> >> [root@radbug ~]# yum list installed | grep qmail >> qmail.x86_641.03-2.2.1.qt.el7 @qmt-current >> qmailadmin.x86_64 1.2.16-3.2.qt.el7 @qmt-current >> qmailmrtg.x86_644.2-3.qt.el7 @qmt-current >> >> I’ve narrowed it down to a delay shown in /var/log/qmail/submission/current, >> when I send a test message to myself at another address, using my vm as my >> mail server. The first lines hit immediately when I click “send” in the mail >> client, but then it takes a full minute between the tcpserver “end” message >> and whatever preceded it, during which time my mail client hangs at >> “sending…”: (annotations [*** like this ***] inline are mine, obviously, and >> addresses/servers replaced with caps but the IPs are unchanged): >> >> 2021-01-30 13:42:52.255716500 tcpserver: status: 0/100 >> [*** message sent. ***] >> 2021-01-30 13:47:33.912724500 tcpserver: status: 1/100 >> 2021-01-30 13:47:33.912947500 tcpserver: pid 6009 from 24.62.203.29 >> 2021-01-30 13:47:33.912948500 tcpserver: ok 6009 MYHOST:104.236.46.99:587 >> :24.62.203.29::55124 >> [*** waiting... one minute 4 seconds pass ***] >> 2021-01-30 13:48:37.899374500 tcpserver: end 6009 status 256 >> 2021-01-30 13:48:37.899376500 tcpserver: status: 0/100 >> 2021-01-30 13:48:37.930988500 tcpserver: status: 1/100 >> 2021-01-30 13:48:37.931122500 tcpserver: pid 6021 from 24.62.203.29 >> 2021-01-30 13:48:37.931198500 tcpserver: ok 6021 MYHOST:104.236.46.99:587 >> :24.62.203.29::55129 >> 2021-01-30 13:48:38.263537500 CHKUSER accepted sender: from >> mailto:m...@myserver.com>:m...@myserver.com >> <mailto:m...@myserver.com>:> remote <[10.0.98.178]:unknown:24.62.203.29> >> rcpt <> : sender accepted >> 2021-01-30 13:48:38.327973500 CHKUSER relaying rcpt: from > <mailto:m...@myserver.com>:m...@myserver.com <mailto:m...@myserver.com>:> >> remote <[10.0.98.178]:unknown:24.62.203.29> rcpt > <mailto:m...@otherserver.com>> : client allowed to relay >> 2021-01-30 13:48:38.327975500 policy_check: local m...@myserver.com >> <mailto:m...@myserver.com> -> remote m...@otherserver.com >> <mailto:m...@otherserver.com> (AUTHENTICATED SENDER) >> 2021-01-30 13:48:38.327976500 policy_check: policy allows transmission >> [*** waiting... (message received) then one more minute passes ***] >> 2021-01-30 13:49:38.403478500 tcpserver: end 6021 status 0 >> 2021-01-30 13:49:38.403479500 tcpserver: status: 0/100 >> >> It’s very consistent in this timing, and loads on the server are basically >> 0, nothing in /var/log/messages or anywhere else that I can see. Vpopmail is >> using mysql, but it isn’t logging any slow queries, and this is more or less >> the only thing happening on the box. It’s been doing it for a while now, >> maybe a month or two? I don’t send much mail so I didn’t lean into it too >> hard when it started happening, but now it’s really annoying me and I want >> to get to the bottom of it. (Inbound mail TO the server from other locations >> (like the auto response to the “subscribe” message I sent to the list just >> now) lands instantly.) >> >> My /etc/tcprules.d/tcp.smtp: (I thought it might have been something about >> domain keys, so I removed that but it hasn’t made any difference) >> 127.:allow,BADMIMETYPE="",BADLOADERTYPE="M",CHKUSER_RCPTLIMIT="50",CHKUSER_WRONGRCPTLIMIT="10",NOP0FCHECK="1",QMAILQUEUE="/var/qmail/bin/simscan" >> >> My /var/qmail/supervise/submission/run: >> #!/bin/sh
[qmailtoaster] Very slow mail submission
Greetings all. I’ve been running qmail on my servers and vms for about 20 years and it’s mostly worked great, but I’m having a thumper of a slowdown problem on my current setup and I can’t get to the bottom of it… any help appreciated. The environment is a very lightly-used private VM running centos7 and the following packages: [root@radbug ~]# yum list installed | grep qmail qmail.x86_641.03-2.2.1.qt.el7 @qmt-current qmailadmin.x86_64 1.2.16-3.2.qt.el7 @qmt-current qmailmrtg.x86_644.2-3.qt.el7 @qmt-current I’ve narrowed it down to a delay shown in /var/log/qmail/submission/current, when I send a test message to myself at another address, using my vm as my mail server. The first lines hit immediately when I click “send” in the mail client, but then it takes a full minute between the tcpserver “end” message and whatever preceded it, during which time my mail client hangs at “sending…”: (annotations [*** like this ***] inline are mine, obviously, and addresses/servers replaced with caps but the IPs are unchanged): 2021-01-30 13:42:52.255716500 tcpserver: status: 0/100 [*** message sent. ***] 2021-01-30 13:47:33.912724500 tcpserver: status: 1/100 2021-01-30 13:47:33.912947500 tcpserver: pid 6009 from 24.62.203.29 2021-01-30 13:47:33.912948500 tcpserver: ok 6009 MYHOST:104.236.46.99:587 :24.62.203.29::55124 [*** waiting... one minute 4 seconds pass ***] 2021-01-30 13:48:37.899374500 tcpserver: end 6009 status 256 2021-01-30 13:48:37.899376500 tcpserver: status: 0/100 2021-01-30 13:48:37.930988500 tcpserver: status: 1/100 2021-01-30 13:48:37.931122500 tcpserver: pid 6021 from 24.62.203.29 2021-01-30 13:48:37.931198500 tcpserver: ok 6021 MYHOST:104.236.46.99:587 :24.62.203.29::55129 2021-01-30 13:48:38.263537500 CHKUSER accepted sender: from remote <[10.0.98.178]:unknown:24.62.203.29> rcpt <> : sender accepted 2021-01-30 13:48:38.327973500 CHKUSER relaying rcpt: from remote <[10.0.98.178]:unknown:24.62.203.29> rcpt : client allowed to relay 2021-01-30 13:48:38.327975500 policy_check: local m...@myserver.com -> remote m...@otherserver.com (AUTHENTICATED SENDER) 2021-01-30 13:48:38.327976500 policy_check: policy allows transmission [*** waiting... (message received) then one more minute passes ***] 2021-01-30 13:49:38.403478500 tcpserver: end 6021 status 0 2021-01-30 13:49:38.403479500 tcpserver: status: 0/100 It’s very consistent in this timing, and loads on the server are basically 0, nothing in /var/log/messages or anywhere else that I can see. Vpopmail is using mysql, but it isn’t logging any slow queries, and this is more or less the only thing happening on the box. It’s been doing it for a while now, maybe a month or two? I don’t send much mail so I didn’t lean into it too hard when it started happening, but now it’s really annoying me and I want to get to the bottom of it. (Inbound mail TO the server from other locations (like the auto response to the “subscribe” message I sent to the list just now) lands instantly.) My /etc/tcprules.d/tcp.smtp: (I thought it might have been something about domain keys, so I removed that but it hasn’t made any difference) 127.:allow,BADMIMETYPE="",BADLOADERTYPE="M",CHKUSER_RCPTLIMIT="50",CHKUSER_WRONGRCPTLIMIT="10",NOP0FCHECK="1",QMAILQUEUE="/var/qmail/bin/simscan" My /var/qmail/supervise/submission/run: #!/bin/sh QMAILDUID=`id -u vpopmail` NOFILESGID=`id -g vpopmail` MAXSMTPD=`cat /var/qmail/control/concurrencyincoming` SMTPD="/var/qmail/bin/qmail-smtpd" TCP_CDB="/etc/tcprules.d/tcp.smtp.cdb" HOSTNAME=`hostname` VCHKPW="/home/vpopmail/bin/vchkpw" export REQUIRE_AUTH=1 exec /usr/bin/softlimit -m 12800 \ /usr/bin/tcpserver -v -R -H -l $HOSTNAME -x $TCP_CDB -c "$MAXSMTPD" \ -u "$QMAILDUID" -g "$NOFILESGID" 0 587 \ $SMTPD $VCHKPW /bin/true 2>&1 In case it’s relevant, my /var/qmail/supervise/smtp/run: #!/bin/sh QMAILDUID=`id -u vpopmail` NOFILESGID=`id -g vpopmail` MAXSMTPD=`cat /var/qmail/control/concurrencyincoming` SPAMDYKE="/usr/bin/spamdyke" SPAMDYKE_CONF="/etc/spamdyke/spamdyke.conf" SMTPD="/var/qmail/bin/qmail-smtpd" TCP_CDB="/etc/tcprules.d/tcp.smtp.cdb" HOSTNAME=`hostname` VCHKPW="/home/vpopmail/bin/vchkpw" REQUIRE_AUTH=0 exec /usr/bin/softlimit -m 6400 \ /usr/bin/tcpserver -v -R -H -l $HOSTNAME -x $TCP_CDB -c "$MAXSMTPD" \ -u "$QMAILDUID" -g "$NOFILESGID" 0 smtp \ $SPAMDYKE --config-file $SPAMDYKE_CONF \ $SMTPD $VCHKPW /bin/true 2>&1 I tried removing spamdyke from this just to see if it made a difference
[qmailtoaster] segmentation fault in ezmlm-send
I run an ezmlm mailing list for a friend, which has always worked fine. The list is moderated and today the moderator tried to send a message to the list, but the moderator confirmation message didn’t reach her because her mail server blocked it (I checked and am not on any RBL lists, so have asked the ISP to investigate). Meanwhile, I tried to push the message awaiting moderation via the instructions from the ezmlm FAQ. ezmlm-accept dies with a segfault. When I try to run ezmlm-send directly, which ezmlm-accept does, again the segfault: segfault at 0 ip 004093d0 sp 7ffe72c0e2e8 error 4 in ezmlm-send[40+e000] The command line is (with LISTDIR replacing the name of the vpopmail directory): /usr/bin/ezmlm-send LISTDIR < LISTDIR/mod/pending/1501468655.5150 I’ve also done it with fully-qualified paths for LISTDIR and it didn’t make any difference. This is on centos 7, all packages and kernel/glibc/etc up to date. Everything else is working normally, no other signs of problems that I can see, nothing in /var/log/messages except the segfault line above. I reinstalled the ezmlm yum package to make sure no binaries were corrupted. I haven’t a clue what to do, but I really need to get this message out. Once something hits a segfault, I’m pretty stuck. I’ve rebooted the server twice, also, which I hate doing; didn’t help. Any ideas for either a fix or a workaround so I can get this message out? Thanks. -- Steve Linberg, Chief Goblin Silicon Goblin Technologies http://silicongoblin.com Be kind. Remember, everyone you meet is fighting a hard battle.
Re: [qmailtoaster] Uptick in spam / sa-learn
Got it, all correct. Thanks for the gentle dope-slap and threading the maze. :) I still have the problem of all messages being tagged with USER_IN_BLACKLIST, but I’ll look for answers on a SA forum (unless anyone here has an easy answer!) - Steve > On Feb 7, 2017, at 12:29 AM, Eric Broch <ebr...@whitehorsetc.com> wrote: > > Hi Steve, > > Look at the line in '/var/qmail/supervise/smtp/run' where tcpserver initiates > the rules of the compiled database (cdb), > > '/usr/bin/tcpserver -v -R -H -l $HOSTNAME -x $TCP_CDB -c "$MAXSMTPD"' > > specifically, > > $TCP_CDB > > which translates to, > > '/etc/tcprules.d/tcp.smtp.cdb' > > which you edit as... > > '/etc/tcprules.d/tcp.smtp'. > > Dump this file, > > # cat /etc/tcprules.d/tcp.smtp > > and you should see an entry of the following form (where the QMAILQUEUE > variable is set to simscan): > > QMAILQUEUE="/var/qmail/bin/simscan". > > then > > # cat /var/qmail/control/simcontrol > > and you should see the following lines which indicate that simscan calls > clamav and spamd: > > :clam=yes,spam=yes,spam_hits=12,attach=.mp3:.src:.bat:.pif > > Eric > > > > > > > > On 2/6/2017 9:47 PM, Steve Linberg wrote: >> Hi Eric. >> >> I checked my /var/qmail/supervise/smtp/run and this is what it contains. No >> mention of clamav / spamassassin / simscan: >> >> #!/bin/sh >> >> >> QMAILDUID=`id -u vpopmail` >> NOFILESGID=`id -g vpopmail` >> MAXSMTPD=`cat /var/qmail/control/concurrencyincoming` >> SPAMDYKE="/usr/bin/spamdyke" >> SPAMDYKE_CONF="/etc/spamdyke/spamdyke.conf" >> SMTPD="/var/qmail/bin/qmail-smtpd" >> TCP_CDB="/etc/tcprules.d/tcp.smtp.cdb" >> HOSTNAME=`hostname` >> VCHKPW="/home/vpopmail/bin/vchkpw" >> REQUIRE_AUTH=0 >> >> exec /usr/bin/softlimit -m 12800 \ >> /usr/bin/tcpserver -v -R -H -l $HOSTNAME -x $TCP_CDB -c "$MAXSMTPD" \ >> -u "$QMAILDUID" -g "$NOFILESGID" 0 smtp \ >> $SPAMDYKE --config-file $SPAMDYKE_CONF \ >> $SMTPD $VCHKPW /bin/true 2>&1 >> >> I haven’t modified it from the stock install. >> >> I don’t have any .qmail-default file anywhere that mentions dspam, but the >> process is running, and when I change /etc/spamassassin/local.cf and >> systemctl restart dspam, it reacts to the changes. >> >> However, right now I”m in a quasi-panic because at some point after running >> sa-learn against 1500+ spams and 2000+ hams, I started having massive >> deliverability problems with almost email being rejected with the >> USER_IN_BLACKLIST tag applied, even when I removed the (relatively few) >> blacklist_from lines in my local.cf and restarted everything, even rebooted. >> There was no explicit blacklist set anywhere, and for the life of me I have >> no idea why that tag is being applied. There’s no way that I can find so far >> to get spamassassin to show the blacklist, or indicate where it can be >> found. The only way I’ve been able to restore deliverability so far is to >> set the score for USER_IN_BLACKLIST to 0 in local.cf. >> >> Ack? >> >> - Steve >> >> >>> On Jan 30, 2017, at 11:59 AM, Eric Broch <ebr...@whitehorsetc.com >>> <mailto:ebr...@whitehorsetc.com>> wrote: >>> >>> Steve, >>> >>> I think you mean simscan. >>> >>> Basically, the email flow is as follows for QMT (look in >>> '/var/qmail/supervise/smtp/run'): >>> 1) softlimit >>> >>> 2) tcpserver (utilizes the config file /etc/tcprules.d/tcp.smtp where >>> QMAILQUEUE is configured as simscan) >>> 3) simscan (utilizes /var/qmail/control/simcontrol where clamav and >>> spamassassin are invoked) >>> 4) clamav & spamassassin >>> >>> 5) spamdyke >>> >>> 6) qmail-smtpd ... >>> >>> >>> Dspam on the other hand is called later during delivery. I use it in >>> .qmail-default. >>> >>> >>> >>> >>> >>> On 1/30/2017 9:09 AM, Steve Linberg wrote: >>>> Thanks Eric. It’s very likely that I”m confused about this - I am using >>>> dspam. My understanding is that dspam is the daemon that launches >>>
Re: [qmailtoaster] Uptick in spam / sa-learn
Hi Eric. I checked my /var/qmail/supervise/smtp/run and this is what it contains. No mention of clamav / spamassassin / simscan: #!/bin/sh QMAILDUID=`id -u vpopmail` NOFILESGID=`id -g vpopmail` MAXSMTPD=`cat /var/qmail/control/concurrencyincoming` SPAMDYKE="/usr/bin/spamdyke" SPAMDYKE_CONF="/etc/spamdyke/spamdyke.conf" SMTPD="/var/qmail/bin/qmail-smtpd" TCP_CDB="/etc/tcprules.d/tcp.smtp.cdb" HOSTNAME=`hostname` VCHKPW="/home/vpopmail/bin/vchkpw" REQUIRE_AUTH=0 exec /usr/bin/softlimit -m 12800 \ /usr/bin/tcpserver -v -R -H -l $HOSTNAME -x $TCP_CDB -c "$MAXSMTPD" \ -u "$QMAILDUID" -g "$NOFILESGID" 0 smtp \ $SPAMDYKE --config-file $SPAMDYKE_CONF \ $SMTPD $VCHKPW /bin/true 2>&1 I haven’t modified it from the stock install. I don’t have any .qmail-default file anywhere that mentions dspam, but the process is running, and when I change /etc/spamassassin/local.cf and systemctl restart dspam, it reacts to the changes. However, right now I”m in a quasi-panic because at some point after running sa-learn against 1500+ spams and 2000+ hams, I started having massive deliverability problems with almost email being rejected with the USER_IN_BLACKLIST tag applied, even when I removed the (relatively few) blacklist_from lines in my local.cf and restarted everything, even rebooted. There was no explicit blacklist set anywhere, and for the life of me I have no idea why that tag is being applied. There’s no way that I can find so far to get spamassassin to show the blacklist, or indicate where it can be found. The only way I’ve been able to restore deliverability so far is to set the score for USER_IN_BLACKLIST to 0 in local.cf. Ack? - Steve > On Jan 30, 2017, at 11:59 AM, Eric Broch <ebr...@whitehorsetc.com> wrote: > > Steve, > > I think you mean simscan. > > Basically, the email flow is as follows for QMT (look in > '/var/qmail/supervise/smtp/run'): > 1) softlimit > > 2) tcpserver (utilizes the config file /etc/tcprules.d/tcp.smtp where > QMAILQUEUE is configured as simscan) > 3) simscan (utilizes /var/qmail/control/simcontrol where clamav and > spamassassin are invoked) > 4) clamav & spamassassin > > 5) spamdyke > > 6) qmail-smtpd ... > > > Dspam on the other hand is called later during delivery. I use it in > .qmail-default. > > > > > > On 1/30/2017 9:09 AM, Steve Linberg wrote: >> Thanks Eric. It’s very likely that I”m confused about this - I am using >> dspam. My understanding is that dspam is the daemon that launches >> SpamAssassin and connects it to qmail, but I admit I’m fuzzy on the details. >> The soup of spam-related things I know I am running are: >> >> SpamAssassin: evaluate individual messages and decide whether something is >> spam or not >> pyzor: additional information used by SpamAssassin to decide if something is >> spammy or not >> dspam: daemon wrapper around SpamAssassin >> spamdyke: erm…. something else that connects to SpamAssassin somehow >> >> But again, I’m not completely clear on how it all fits together, and it’s >> entirely possible I’ve got something miswired? On my older setup before your >> modernization of it, spamc was a part of the process, but that seems to be >> removed now? >> >> Whenever I make changes to /etc/spamassassin/local.cf, I activate them with >> “systemctl restart spamd”. I don’t restart qmail. >> >> Any dope-slaps would be most welcome :) >> >> - Steve >> >> >>> On Jan 30, 2017, at 10:59 AM, Eric Broch <ebr...@whitehorsetc.com >>> <mailto:ebr...@whitehorsetc.com>> wrote: >>> >>> Steve, >>> >>> You might try Dspam. I still use it on my home server with a couple >>> domains. If you need any help let me know. >>> >>> Eric >>> >>> On 1/30/2017 8:39 AM, Steve Linberg wrote: >>>> Thanks for the suggestions, everyone. This is a small-scale system that’s >>>> mostly for personal use, so unfortunately it doesn’t warrant the expense >>>> of a standalone appliance for spam, so I’m pretty much limited to what I >>>> can roll up on the box itself. I’ve used SA+qmail in various permutations >>>> for about 15 years, and for the most part it’s worked reliably and well, >>>> but every once in a while spammers change tactics and figure out a way >>>> through and there’s an upsurge for a bit. >>>>
Re: [qmailtoaster] Uptick in spam / sa-learn
Ah! Thanks, I’ll review all this. Much appreciated. - Steve > On Jan 30, 2017, at 11:59 AM, Eric Broch <ebr...@whitehorsetc.com> wrote: > > Steve, > > I think you mean simscan. > > Basically, the email flow is as follows for QMT (look in > '/var/qmail/supervise/smtp/run'): > 1) softlimit > > 2) tcpserver (utilizes the config file /etc/tcprules.d/tcp.smtp where > QMAILQUEUE is configured as simscan) > 3) simscan (utilizes /var/qmail/control/simcontrol where clamav and > spamassassin are invoked) > 4) clamav & spamassassin > > 5) spamdyke > > 6) qmail-smtpd ... > > > Dspam on the other hand is called later during delivery. I use it in > .qmail-default. > > > > > > On 1/30/2017 9:09 AM, Steve Linberg wrote: >> Thanks Eric. It’s very likely that I”m confused about this - I am using >> dspam. My understanding is that dspam is the daemon that launches >> SpamAssassin and connects it to qmail, but I admit I’m fuzzy on the details. >> The soup of spam-related things I know I am running are: >> >> SpamAssassin: evaluate individual messages and decide whether something is >> spam or not >> pyzor: additional information used by SpamAssassin to decide if something is >> spammy or not >> dspam: daemon wrapper around SpamAssassin >> spamdyke: erm…. something else that connects to SpamAssassin somehow >> >> But again, I’m not completely clear on how it all fits together, and it’s >> entirely possible I’ve got something miswired? On my older setup before your >> modernization of it, spamc was a part of the process, but that seems to be >> removed now? >> >> Whenever I make changes to /etc/spamassassin/local.cf, I activate them with >> “systemctl restart spamd”. I don’t restart qmail. >> >> Any dope-slaps would be most welcome :) >> >> - Steve >> >> >>> On Jan 30, 2017, at 10:59 AM, Eric Broch <ebr...@whitehorsetc.com >>> <mailto:ebr...@whitehorsetc.com>> wrote: >>> >>> Steve, >>> >>> You might try Dspam. I still use it on my home server with a couple >>> domains. If you need any help let me know. >>> >>> Eric >>> >>> On 1/30/2017 8:39 AM, Steve Linberg wrote: >>>> Thanks for the suggestions, everyone. This is a small-scale system that’s >>>> mostly for personal use, so unfortunately it doesn’t warrant the expense >>>> of a standalone appliance for spam, so I’m pretty much limited to what I >>>> can roll up on the box itself. I’ve used SA+qmail in various permutations >>>> for about 15 years, and for the most part it’s worked reliably and well, >>>> but every once in a while spammers change tactics and figure out a way >>>> through and there’s an upsurge for a bit. >>>> >>>> I’ll give sa-learn a shot and see if it improves anything. >>>> >>>> Cheers, >>>> >>>> Steve >>>> >>>> -- >>>> Steve Linberg, Chief Goblin >>>> Silicon Goblin Technologies >>>> http://silicongoblin.com <http://silicongoblin.com/> >>>> Be kind. Remember, everyone you meet is fighting a hard battle. >>>> >>> >>> -- >>> Eric Broch, IMSO, DAM, NGOO, DITH, URTS >>> White Horse Technical Consulting (WHTC) >>> 406.214.6802 >> >> -- >> Steve Linberg, Chief Goblin >> Silicon Goblin Technologies >> http://silicongoblin.com <http://silicongoblin.com/> >> Be kind. Remember, everyone you meet is fighting a hard battle. >> > > -- > Eric Broch, IMSO, DAM, NGOO, DITH, URTS > White Horse Technical Consulting (WHTC) > 406.214.6802 -- Steve Linberg, Chief Goblin Silicon Goblin Technologies http://silicongoblin.com Be kind. Remember, everyone you meet is fighting a hard battle.
Re: [qmailtoaster] Uptick in spam / sa-learn
Thanks Eric. It’s very likely that I”m confused about this - I am using dspam. My understanding is that dspam is the daemon that launches SpamAssassin and connects it to qmail, but I admit I’m fuzzy on the details. The soup of spam-related things I know I am running are: SpamAssassin: evaluate individual messages and decide whether something is spam or not pyzor: additional information used by SpamAssassin to decide if something is spammy or not dspam: daemon wrapper around SpamAssassin spamdyke: erm…. something else that connects to SpamAssassin somehow But again, I’m not completely clear on how it all fits together, and it’s entirely possible I’ve got something miswired? On my older setup before your modernization of it, spamc was a part of the process, but that seems to be removed now? Whenever I make changes to /etc/spamassassin/local.cf, I activate them with “systemctl restart spamd”. I don’t restart qmail. Any dope-slaps would be most welcome :) - Steve > On Jan 30, 2017, at 10:59 AM, Eric Broch <ebr...@whitehorsetc.com> wrote: > > Steve, > > You might try Dspam. I still use it on my home server with a couple domains. > If you need any help let me know. > > Eric > > On 1/30/2017 8:39 AM, Steve Linberg wrote: >> Thanks for the suggestions, everyone. This is a small-scale system that’s >> mostly for personal use, so unfortunately it doesn’t warrant the expense of >> a standalone appliance for spam, so I’m pretty much limited to what I can >> roll up on the box itself. I’ve used SA+qmail in various permutations for >> about 15 years, and for the most part it’s worked reliably and well, but >> every once in a while spammers change tactics and figure out a way through >> and there’s an upsurge for a bit. >> >> I’ll give sa-learn a shot and see if it improves anything. >> >> Cheers, >> >> Steve >> >> -- >> Steve Linberg, Chief Goblin >> Silicon Goblin Technologies >> http://silicongoblin.com <http://silicongoblin.com/> >> Be kind. Remember, everyone you meet is fighting a hard battle. >> > > -- > Eric Broch, IMSO, DAM, NGOO, DITH, URTS > White Horse Technical Consulting (WHTC) > 406.214.6802 -- Steve Linberg, Chief Goblin Silicon Goblin Technologies http://silicongoblin.com Be kind. Remember, everyone you meet is fighting a hard battle.
Re: [qmailtoaster] Uptick in spam / sa-learn
Thanks for the suggestions, everyone. This is a small-scale system that’s mostly for personal use, so unfortunately it doesn’t warrant the expense of a standalone appliance for spam, so I’m pretty much limited to what I can roll up on the box itself. I’ve used SA+qmail in various permutations for about 15 years, and for the most part it’s worked reliably and well, but every once in a while spammers change tactics and figure out a way through and there’s an upsurge for a bit. I’ll give sa-learn a shot and see if it improves anything. Cheers, Steve -- Steve Linberg, Chief Goblin Silicon Goblin Technologies http://silicongoblin.com Be kind. Remember, everyone you meet is fighting a hard battle.
[qmailtoaster] Uptick in spam / sa-learn
For a while, my Centos7 build was doing a pretty good job; once I enabled Pyzor and blocked a number of suspect TLDs, I was getting spam rejection rates averaging around 90%, but over the last month or so it’s dropped to more like 65%. I can’t see any obvious reasons for this; nothing unexpected in logs as far as I can tell. I’ve been saving spam that got through, and I’ve got a body of about 1500 spam messages now. I’m reading conflicting advice regarding the use of sa-learn. If I run it against this body to teach it spam, do I also need to do a “ham” run against OK mail? And are these numbers high enough, or should I wait until I have more? Or is it risky to do that at all? Any advice for tuning this thing up? The spam problem seems to be getting worse lately. Thanks for any ideas. -- Steve Linberg, Chief Goblin Silicon Goblin Technologies http://silicongoblin.com Be kind. Remember, everyone you meet is fighting a hard battle.
Re: [qmailtoaster] Where are Log Files?
> On Aug 3, 2016, at 5:52 PM, Roxanne Sandesara <roxie.sil...@gmail.com> wrote: > > I cannot find the log files on the new qmail toaster installation I’ve built > on this CentOS 7 box. The locations mentioned on the wiki don’t even exist. > Where should I be looking? /var/log/qmail is the standard starting location for most logs. -- Steve Linberg, Chief Goblin Silicon Goblin Technologies http://silicongoblin.com Be kind. Remember, everyone you meet is fighting a hard battle.
[qmailtoaster] qmailadmin under Nginx?
Now that the dust is settling on my various other toaster issues and things seem to be working well, I’m thinking about getting the toaster working with Nginx, which I use instead of Apache; I'd like to get qmailadmin working under Nginx for some of my users to whom I’m not giving shell access. Does anybody have a working config for nginx that’s the equivalent of the /etc/httpd/conf/toaster.conf apache config file installed with the toaster? I have a CGI wrapper script for nginx already in place that I’m using successfully with some old CGI code, so that part’s sorted out, but I haven’t managed to get the various binaries working yet, and before I go bananas on it, I just wanted to see if anyone else has already gotten it working. :) -- Steve Linberg, Chief Goblin Silicon Goblin Technologies http://silicongoblin.com Be kind. Remember, everyone you meet is fighting a hard battle.
Re: [qmailtoaster] vchkpw segfaults and spamdyke errors
Happy to report no segfaults since I increased the softlimit in /var/qmail/supervise/smtp/run from 64 megs to 128 megs. I also dropped the softlimit in /var/qmail/supervise/submission/run back to 128 megs. So far so good. Phew. -- Steve Linberg, Chief Goblin Silicon Goblin Technologies http://silicongoblin.com Be kind. Remember, everyone you meet is fighting a hard battle.
Re: [qmailtoaster] vchkpw segfaults and spamdyke errors
> On Jun 10, 2016, at 1:09 PM, Eric <ebr...@whitehorsetc.com> wrote: > > Mine is twice that: 12800 That’s what I set mine to just now; I also set the submission softlimit back to that. Now I know better how to track the segfaults, if they occur, so we’ll see how they both go over the next few days. Thanks again. -- Steve Linberg, Chief Goblin Silicon Goblin Technologies http://silicongoblin.com Be kind. Remember, everyone you meet is fighting a hard battle.
Re: [qmailtoaster] vchkpw segfaults and spamdyke errors
> On Jun 10, 2016, at 12:49 PM, Eric <ebr...@whitehorsetc.com> wrote: > > Steve, > > I think 19152 of vchkpw[19152] is the process id. Can you look through the > log file in /var/log/qmail/smtp, /var/log/qmail/send, and > /var/log/qmail/submission for such a process? Ah - I should have thought of that. I thought the pid was long-gone, but of course it’s logged. And it turned up in the smtp log, not the submission log. So I’ve been softlimiting the wrong process! The softlimit in /var/qmail/supervise/smtp/run is 6400 still. I’ll pump that up and see how it goes, but I’m pretty confident now. Thanks very much for the clues. Will report back after the weekend, but I’ve got a good feeling now. :) - Steve -- Steve Linberg, Chief Goblin Silicon Goblin Technologies http://silicongoblin.com Be kind. Remember, everyone you meet is fighting a hard battle.
Re: [qmailtoaster] vchkpw segfaults and spamdyke errors
> On Jun 10, 2016, at 11:27 AM, Eric <ebr...@whitehorsetc.com> wrote: > > I guess I'm confused. Which log file are you getting segfault in? Jeez, sorry, I referenced it incorrectly before. They’re in /var/log/messages, NOT /var/log/maillog. :( /var/log/messages:Jun 10 09:51:15 xxx kernel: vchkpw[19152]: segfault at 0 ip 7f386f954ad6 sp 7ffea3855d68 error 4 in libc-2.17.so[7f386f822000+1b7000] Again, though, for what it’s worth, I grepped the date/time of the segfault in all of the other files in /var/log and didn’t find any matches. I know the qmail logfiles timestamp in a different format until piped through tai64nlocal, but I really looked for any other time-based clues in all of the other system logfiles and couldn’t find anything happening at or near the same time. Very puzzling. So I’m still not clear whether there’s any other way to find out what process invoked vchkpw in the above instance (they’re all like that, always in vchkpw). I don’t know for sure if I’m softlimiting the right process in qmail submission, or if it could be something else, or how to tell. -- Steve Linberg, Chief Goblin Silicon Goblin Technologies http://silicongoblin.com Be kind. Remember, everyone you meet is fighting a hard battle.
Re: [qmailtoaster] vchkpw segfaults and spamdyke errors
> On Jun 10, 2016, at 11:14 AM, Eric <ebr...@whitehorsetc.com> wrote: > > Look in /var/log/maillog and compare times with the occurrences in > /var/log/qmail/submission/'any log'. Use tai64nlocal to convert time stamps* > in the qmail log. > > *cat /var/log/qmail/submission/'any log' | tai64nlocal | grep segfault > > newfile.txt Yes, there’s nothing in the submission, smtp or send logs that are near the times of the segfaults, and no match of ‘segfault’ at all in any of the qmail logs. The entries in /var/log/maillog are the only clue I have that anything’s even happening… I can’t find any further evidence anywhere, or observe any consequences or results of the segfaults. Is there any other process that could write to /var/log/maillog? Or anything else that calls vckhpw and whose results might end up in maillog somehow, even if not somehow directly invoked by the qmail submission process chain? -- Steve Linberg, Chief Goblin Silicon Goblin Technologies http://silicongoblin.com Be kind. Remember, everyone you meet is fighting a hard battle.
Re: [qmailtoaster] vchkpw segfaults and spamdyke errors
Still working on the segfault problem; got dozens of them overnight when I definitely wasn’t using any services. softlimit in /var/qmail/supervise/submission/run is at 500 megs and rising, but I’m still having trouble believing it needs to be that high or higher. While I continue to push that: is there any way to know what process chain is invoking the vchkpw process that’s segfaulting? Don’t a number of different processes use it? I don’t know for a fact that submission is the one that’s causing it. I can’t find any other clues in my logs, like events happening at the same time as the segfaults, but are there any other possible culprits that might invoke vchkpw without enough RAM to do whatever it’s trying to do? -- Steve Linberg, Chief Goblin Silicon Goblin Technologies http://silicongoblin.com Be kind. Remember, everyone you meet is fighting a hard battle.
Re: [qmailtoaster] vchkpw segfaults and spamdyke errors
> On Jun 9, 2016, at 1:53 PM, Eric <ebr...@whitehorsetc.com> wrote: > > Domain users usually use port 587 (submission) to authenticate for relay > purposes. How many users do you have? And, which ones are using the > submission port to authenticate in order to send email? So far, it’s just me. I do have accounts in a half-dozen or so domains, all lightly used. I’ll be adding a half-dozen heavier users soon, once things stabilize. What’s odd is: 1. The segfaults I’m finding are not occurring around the sending of mail; some of them are in the middle of the night, which makes me wonder if they’re fake auth attempts from someone / somewhere else, brute-force crack attempts, or maybe attempts to use invalid characters or somehow trigger other bugs in the auth process; 2. With these segfaults, I’m not actually noticing any consequences so far - I find them in the logs, but that’s all. At an earlier point in the build process, I started segfaulting on every attempt to send mail when I changed from the self-signed cert created automatically to the LetsEncrypt cert I got for the domain, and until I raised the softlimit, they didn’t stop. (I think I was rebooting the server between changes at that point, which did the hard stop necessary for the submission/run change to take effect, since the build was brand new and I was the only one using it, and VM reboots are fast… just me being lazy.) Anyway, though, as far as I can tell, mail is working fine for me… but the segfaults are still popping up in the logs at weird times and intervals. I’ve sent a few messages (to the list) since I made the last change, and I’m tailing -f /var/log/messages in a term window, and no segfaults so far since I bumped the submission/run softlimit to 200 megs coming up on an hour ago now, but some of the windows between segfaults have been longer than that, so I’m still holding my breath. - Steve -- Steve Linberg, Chief Goblin Silicon Goblin Technologies http://silicongoblin.com Be kind. Remember, everyone you meet is fighting a hard battle.
Re: [qmailtoaster] vchkpw segfaults and spamdyke errors
> On Jun 9, 2016, at 1:43 PM, Eric <ebr...@whitehorsetc.com> wrote: > > Steve, > > You should be able to tell immediately if it's working or not. The qmailctl start / stop process, compared to qmailctl restart, definitely worked, as far as resetting the process goes. I dropped the softlimit back to 100 megs (which still feels like an awful lot) and got immediate segfaults. I bumped it up to 200 megs about 30 minutes ago, and no segfaults so far, but sometimes they’re a few hours apart. I don’t know exactly what’s triggering them, so in part, it’s kind of a waiting game. - Steve -- Steve Linberg, Chief Goblin Silicon Goblin Technologies http://silicongoblin.com Be kind. Remember, everyone you meet is fighting a hard battle.
Re: [qmailtoaster] vchkpw segfaults and spamdyke errors
> On Jun 9, 2016, at 12:51 PM, Helmut Fritz <hel...@fritz.us.com> wrote: > > Mine is at 4800. That seems much more reasonable... > I do not think imap uses the submission run file. Should it not use the > imap4 run file? Sorry, I misspoke - I meant smtp authentication, not imap authentication. > https://bugzilla.redhat.com/show_bug.cgi?id=986427 > <https://bugzilla.redhat.com/show_bug.cgi?id=986427> > > you might look along those lines, make sure you glibc is not old, etc. are > you using a hosts.deny or allow file? Might fall in line with the above then > even though it is somewhat old. this does not look like it will be easy to > resolve unfortunately. I do have /etc/hosts.allow and /etc/hosts.deny, but they’re both empty. Libc is 2.17, the current version for CentOS 7.2. I wasn’t paying enough attention to the output of “qmailctl restart”, and as Eric pointed out, it doesn’t cause the submission process to restart. I just tested it with stop / start and it did restart submission, so I’m starting testing over again. I dropped it back to 1 again (100 megs), and will go up from there over the next couple of days and see if it changes anything. Thanks again to everyone for the help - hopefully we’ll get to the bottom of this, and hopefully it’ll help some other people too. - Steve -- Steve Linberg, Chief Goblin Silicon Goblin Technologies http://silicongoblin.com Be kind. Remember, everyone you meet is fighting a hard battle.
Re: [qmailtoaster] vchkpw segfaults and spamdyke errors
> On Jun 9, 2016, at 1:00 PM, Jaime Lerner <jaimeler...@geekgoddess.com> wrote: > > Are your users doing any other kind of login/authentication other than via > the submission port (587)? If so, I'd try upping the softlimit on the other > run files. I only did submission because that's the only port I'm using. No, and it’s really just me authenticating at this point. There will be a few more soon, but so far it’s very small-scale. I’m going to do another round of incremental testing with qmailctl stop / qmailctl start rather than qmailctl restart, and see if that makes any difference. -- Steve Linberg, Chief Goblin Silicon Goblin Technologies http://silicongoblin.com Be kind. Remember, everyone you meet is fighting a hard battle.
Re: [qmailtoaster] vchkpw segfaults and spamdyke errors
> On Jun 9, 2016, at 12:57 PM, Eric <ebr...@whitehorsetc.com> wrote: > > Steve, > > qmailctl restart doesn't restart submission > > try... > qmailctl stop > qmailctl start > > and let me know. Wow - that’s huge. Maybe it should? :) OK - I presumed “restart” was synonymous with stop and start. This invalidates all of my testing, actually, so I’ll roll it back to lower levels and do the stop/start process instead of restart, and see how it goes. Thanks. - Steve -- Steve Linberg, Chief Goblin Silicon Goblin Technologies http://silicongoblin.com Be kind. Remember, everyone you meet is fighting a hard battle.
Re: [qmailtoaster] vchkpw segfaults and spamdyke errors
I’ve been incrementally raising the softlimit in /var/qmail/supervise/submission/run over the past few days in an effort to stop the vchkpw segfaults, and I’ve taken it all the way up to 12: #!/bin/sh QMAILDUID=`id -u vpopmail` NOFILESGID=`id -g vpopmail` MAXSMTPD=`cat /var/qmail/control/concurrencyincoming` SMTPD="/var/qmail/bin/qmail-smtpd" TCP_CDB="/etc/tcprules.d/tcp.smtp.cdb" HOSTNAME=`hostname` VCHKPW="/home/vpopmail/bin/vchkpw" export REQUIRE_AUTH=1 exec /usr/bin/softlimit -m 12 \ /usr/bin/tcpserver -v -R -H -l $HOSTNAME -x $TCP_CDB -c "$MAXSMTPD" \ -u "$QMAILDUID" -g "$NOFILESGID" 0 587 \ $SMTPD $VCHKPW /bin/true 2>&1 Unfortunately, it doesn’t seem to have made any difference at all in the number or frequency of segfaults. Jun 9 11:56:44 xxx kernel: vchkpw[29989]: segfault at 0 ip 7f687b728ad6 sp 7ffc7d038908 error 4 in libc-2.17.so[7f687b5f6000+1b7000] I just can’t believe that 1.2 BILLION BYTES isn’t enough to handle qmail-smtp or an imap login or whatever it’s doing here. My previous qmailtoaster build runs on a VM with 512mb total RAM, also running Apache and Mailman, and has never segfaulted. There must be something else going on. Is “qmailctl restart” the right way to activate changes to /var/qmail/supervise/submission/run? That’s what I’ve been doing. Anybody have any other ideas or theories? Again, this is a clean CentOS 7.2 VM at DigitalOcean, with 2gb RAM. I’ve disabled ipv6 and selinux. -- Steve Linberg, Chief Goblin Silicon Goblin Technologies http://silicongoblin.com Be kind. Remember, everyone you meet is fighting a hard battle.
Re: [qmailtoaster] Changing the max spam/clam scan message size
> On Jun 7, 2016, at 12:03 PM, Eric <ebr...@whitehorsetc.com> wrote: > > Steve, > > Is this what you're looking for? > > /var/qmail/control/databytes > > Eric Hi Eric. I believe that’s the setting that controls the maximum message size that the server will accept for delivery. I’m looking for the setting that’s the threshold for SpamAssassin (at least, possibly also ClamAV) to skip spam scanning on an incoming message to avoid tying up the CPU too long. It’s set at 500k as a default, so messages over 500k don’t get scanned for spam/virus and just pass through and get delivered. Some spammers send spam in the form of graphics that are just over 500k, so no scanning takes place. I want to increase that limit to a meg or so. In Bill Shupp’s version of the toaster, this was a setting in /etc/mail/spamassassin/spamc.conf, via a “-s” flag, as in “-s 100”. But there’s no spamc in the current toaster build, as far as I can see, and if there is, I can’t find any config files, or any other component that controls it. Something must, though, because messages over 500k are passing through without being checked. - Steve -- Steve Linberg, Chief Goblin Silicon Goblin Technologies http://silicongoblin.com Be kind. Remember, everyone you meet is fighting a hard battle.
[qmailtoaster] Changing the max spam/clam scan message size
Thanks in advance for all the assistance with the current toaster iteration, it’s monstrously helpful. The scanner won’t scan emails larger than 500k by default, and some spammers have figured out that by getting attachments just over 500k, they can bypass scanning on systems left at the default. In the previous iteration of the toaster, this was managed by spamc, but spamc doesn’t seem to be part of the current build - or, if it is, I can’t find it, and for the life of me I can’t figure out where to change this setting. Previously I bumped it up to around 1mb, and that seemed to do the trick pretty well. Anyone know where it is? -- Steve Linberg, Chief Goblin Silicon Goblin Technologies http://silicongoblin.com Be kind. Remember, everyone you meet is fighting a hard battle.
Re: [qmailtoaster] vchkpw segfaults and spamdyke errors
> On Jun 7, 2016, at 11:38 AM, Eric <ebr...@whitehorsetc.com> wrote: > > And, "reasonable" is what the program, or library, needs. If 16000 KB is > not sufficient you must bump it up. Either qmail-smtp or libc or both are > trying to access memory not allocated to them, so, it must be increased. > > Admittedly, I'm not sure about all the details, but if you ran the submission > program without the softlimit setting it could possible eat up your system. > Setting the softlimit actually decreases the amount of memory the program > would otherwise use. Thanks Eric. I’m going to work on gradually incrementing it and see if it stops the errors. I just bumped it up to 200mb, and we’ll see how it goes… the box I’m running this on only has 2 gigs of RAM, and has to run nginx and a couple of other things too, so I’m going to be a little cautious. It’s a VM, so I can increase the RAM if I have to, but it’ll of course increase the cost each time I do, so I’m hoping I can squeak by with what I’ve got. I’ll report back if/when I hit a number that stops the segfaults. Cheers, Steve -- Steve Linberg, Chief Goblin Silicon Goblin Technologies http://silicongoblin.com Be kind. Remember, everyone you meet is fighting a hard battle.
Re: [qmailtoaster] vchkpw segfaults and spamdyke errors
> On Jun 6, 2016, at 11:58 PM, Eric <ebr...@whitehorsetc.com> wrote: > > http://thedjbway.b0llix.net/daemontools/softlimit.html > <http://thedjbway.b0llix.net/daemontools/softlimit.html> Well, I understand the basic concepts of softlimit. My question was really: is 1.28 gigabytes a reasonable setting for this value? That just seems awfully high to me, especially given that the default setting in the toaster is 64 megabytes (6400 vs your 128000) - that’s a factor of 20 increase. Again, unless I’ve got an off-by-a-digit issue somewhere, but I’m pretty sure I’m reading that correctly. 64 megabytes is the recommended setting from “Life with Qmail” and “The Qmail Handbook”, so I thought I was safe raising it to 128k, or even 160k, but over a gig seems like an awful lot. Does it really need to be that high? -- Steve Linberg, Chief Goblin Silicon Goblin Technologies http://silicongoblin.com Be kind. Remember, everyone you meet is fighting a hard battle.
Re: [qmailtoaster] vchkpw segfaults and spamdyke errors
> On Jun 6, 2016, at 1:21 PM, Eric <ebr...@whitehorsetc.com> wrote: > > Hi Steve, > > My /var/qmail/supervise/submission/run is as follows: > > > #!/bin/sh > QMAILDUID=`id -u vpopmail` > NOFILESGID=`id -g vpopmail` > MAXSMTPD=`cat /var/qmail/control/concurrencyincoming` > SMTPD="/var/qmail/bin/qmail-smtpd" > TCP_CDB="/etc/tcprules.d/tcp.smtp.cdb" > HOSTNAME=`hostname` > VCHKPW="/home/vpopmail/bin/vchkpw" > export REQUIRE_AUTH=1 > > exec /usr/bin/softlimit -m 128000 \ >/usr/bin/tcpserver -v -R -H -l $HOSTNAME -x $TCP_CDB -c "$MAXSMTPD" \ >-u "$QMAILDUID" -g "$NOFILESGID" 0 587 \ >$SMTPD $VCHKPW /bin/true 2>&1 > > > > > Note the difference in our softlimits: > 128000 > 16000 Hi Eric. Thanks for this - and maybe I misunderstand either the scope, or the intent, of this setting? I assumed the 16000 figure was meant to represent approximately 160 megabytes: 160,000,000, and that likewise, your 128000 is 1.28 gigs (1,280,000,000) - am I off by a factor of 10? Does this limit mean what I think it means? The original setting was 6400, or 64,000,000, unless I misplaced something? I thought going to 12800, and then 16000, was going roughly from 64megs to 128 megs to 160 megs, and that that should be enough. Sorry if I’m missing something stupidly obvious here… - Steve -- Steve Linberg, Chief Goblin Silicon Goblin Technologies http://silicongoblin.com Be kind. Remember, everyone you meet is fighting a hard battle.
Re: [qmailtoaster] vchkpw segfaults and spamdyke errors
> On Jun 6, 2016, at 11:33 AM, Jaime Lerner <jaimeler...@geekgoddess.com> wrote: > > What do you have in your /etc/resolv.conf file? If you have the ipv6 IPs for > Google in there rather than the ipv4 IPs, maybe that's your issue? Sheesh. That was it! It had the ipv6 addresses in it.Changed the addresses to ipv4, that should take care of that. Thanks (again!) Still stuck on the segfaults, though :( There’s a history of this kind of segfaults on RH systems that go back a few years; it’s strange because the symptoms are the same as when I couldn’t authenticate to the machine because the softlimit in /var/qmail/supervise/submission/run was too low. Since I raised it, I’m not having problems myself, but the segfaults that are occurring are identical. I might go through and do a more thorough shutdown of ipv6 on the box - there are a few clues here and there that it might be related in some distant way. Maybe it’s even related to the /etc/resolv.conf issue. I’ll see how it goes through the day. -- Steve Linberg, Chief Goblin Silicon Goblin Technologies http://silicongoblin.com Be kind. Remember, everyone you meet is fighting a hard battle.
[qmailtoaster] vchkpw segfaults and spamdyke errors
Greetings all. Overall, my new toaster build is working great; however, combing the logs, I still see a couple of issues I’d like to get to the bottom of. (CentOS 7.2, built the toaster a couple of weeks ago.) The first is that I’m still getting a ton of segfaults from vchkpw, even having raised the softlimit in /var/qmail/supervise/submission/run from the default of 6400 to 1, 12800 and even 16000. I sometimes have 20 or more in a row in my logs: Jun 6 08:43:18 xxx kernel: vchkpw[25196]: segfault at 0 ip 7fca89bdbad6 sp 7ffda62cef98 error 4 in libc-2.17.so[7fca89aa9000+1b7000] Jun 6 08:43:21 xxx kernel: vchkpw[25200]: segfault at 0 ip 7f2dd9f91ad6 sp 7ffc754d7b58 error 4 in libc-2.17.so[7f2dd9e5f000+1b7000] Jun 6 08:43:23 xxx kernel: vchkpw[25204]: segfault at 0 ip 7feb85bf8ad6 sp 7ffe1ad395c8 error 4 in libc-2.17.so[7feb85ac6000+1b7000] That said, I’m able to send / receive mail and log in to my imap system without any problems, so I suspect these are triggered by login attempts from someone else, but segfaults aren’t something I’m used to being comfortable with, and I’m not even sure where to begin troubleshooting this. Googling this hasn’t gotten me far. It may be a CentOS issue and not a toaster issue, but it’s still a bit unnerving. Is there anything else in the toaster config that I can look at or that might cause this? The second is hundreds of error messages from spamdyke in /var/log/maillog: Jun 6 10:56:32 xxx spamdyke[30667]: ERROR: invalid/unparsable nameserver found: 2001:4860:4860::8844 Jun 6 10:56:32 xxx spamdyke[30667]: ERROR: invalid/unparsable nameserver found: 2001:4860:4860:: These are constant, and always with those addresses, which I’m 99% sure are Google’s DNS servers in ipv6. I’m not actively using ipv6, and my first thought was to just turn it off (in /etc/sysctl.conf), but even after a reboot, I was still getting these messages, over and over in /var/log/maillog. Does anybody have any ideas on either of these issues? Thanks in advance. -- Steve Linberg, Chief Goblin Silicon Goblin Technologies http://silicongoblin.com Be kind. Remember, everyone you meet is fighting a hard battle.
[qmailtoaster] Using LetsEncrypt certs on Centos 7.2 / qmailtoaster
Just wanted to add this to the notes in case it helps anybody.
First, three rounds of thanks:
1. Of course, to DJB and everyone else who built this extraordinary software
2. to Eric B, for keeping the packages current on modern OSes… installation on
CentOS 7.2 was dizzyingly simple
3. To Jamie Lerner from the list here, who was extremely generous sharing the
details of her working setup with me… the process below is all hers, and all
credit to her. Thanks Jamie :)
This assumes you’ve got LetsEncrypt set up and installed and working already
for your web domain(s). I’m running nginx instead of apache for my web
services; I haven’t yet worked on making the qmailadmin functions accessible
over nginx because I’m comfortable with the command-line utilities. I’ll get to
it eventually.
I am migrating domains from an old qmailtoaster setup (based on Bill Shupp’s
version of the toaster from 10 or so years ago) to this new VM on DigitalOcean.
I did a couple of the smaller ones first just to test the process, and then
moved my main domain over the weekend. (I did accidentally destroy
deliverability to it by mistakenly including it in /var/qmail/control/locals…
don’t do that!)
You don’t need to get a cert for every vpopmail domain. Just do one for your
main domain, like “mail.domain.com”, and have your clients authenticate to it.
They can use their regular vpopmail credentials, but their inbound and outbound
domains will be “mail.domain.com” whatever their virtual domains are.
I needed to set up a website at mail.domain.com for the sole purpose of
obtaining the LetEncrypt certs; I’m not planning to use it for anything else.
There are other ways to get the certs, so you may not need to do this if you
choose another method.
BEFORE INSTALLING THE NEW CERTS, increase the softlimits in
/var/qmail/supervise/submission/run from 6400 to 12800. Although auth
initially worked with the default setup, vchkpw started segfaulting immediately
when I installed the letsencrypt certs, and authentication was impossible. That
was the fix.
Finally, to install the certs, first backup the self-signed cert that was
created as part of the qmailtoaster install in case you need it:
cd /var/qmail/control
cp -a servercert.pem servercert.pem-toaster-backup
Then concatenate the certs from your letsencrypt folder into a new cert:
cat
/etc/letsencrypt/archive/YOUR-MAIL-DOMAIN/{cert1,chain1,fullchain1,privkey1}.pem
> servercert.pem
Make sure the permissions and ownership are correct:
chmod 640 servercert.pem
chown vpopmail.vchkpw servercert.pem
And restart qmail and dovecot (not completely sure if this is necessary, just
playing it safe)
qmailctl restart
systemctl restart dovecot
And that’s it! Working for me.
Thanks again to all involved with this project, and again to Jamie Lerner to
taking the time to answer my questions, way above and beyond.
- Steve Linberg
--
Steve Linberg, Chief Goblin
Silicon Goblin Technologies
http://silicongoblin.com
Be kind. Remember, everyone you meet is fighting a hard battle.
