Re: Fwd: Re: [qmailtoaster] qq soft reject errors on high load

2017-09-04 Thread Rajesh M
eric i did not yet simulate it using your script these are errors in my live production server during peak hours today and there were mails with and without attachments. the max processes i got was 47 only. rajesh - Original Message - From: Eric Broch

Re: Fwd: Re: [qmailtoaster] qq soft reject errors on high load

2017-09-04 Thread Rajesh M
eric i got 3 types of errors like earlier. failure to fork, ripmime and /var/qmail/simscan/1simscan: check_spam had an error ret: -1 what exactly would be /var/qmail/simscan/1simscan ... there is no such file do want me to send you a few of my log files directly to your email id ? rajesh

RE: Fwd: Re: [qmailtoaster] qq soft reject errors on high load

2017-09-04 Thread Rajesh M
eric the max i saw was 47 connections using the shell script you gave me. there were a few errors related to qq soft reject but they got delivered. rajesh - Original Message - From: Eric Broch [mailto:ebr...@whitehorsetc.com] To: qmailtoaster-list@qmailtoaster.com Sent: Sat, 2 Sep

Fwd: Re: [qmailtoaster] qq soft reject errors on high load

2017-09-03 Thread Eric Broch
They're all temporary, for scanning purposes. Once the scanning is done, they all are deleted. On 9/3/2017 3:57 PM, Remo Mattei wrote: I do not see anything on my  :)  /var/qmail/simscan Remo On 9/3/17 12:13 PM, Eric Broch wrote: Rajesh, Those shouldn't be log files in that directory,

Re: [qmailtoaster] qq soft reject errors on high load

2017-09-03 Thread Rajesh M
eric the logs are not in that simscan directory.. they re in the /var/log which is also in the same partition. the files in the ls -l /var/qmail/simscan are temp ones which keep coming and going rajesh - Original Message - From: Eric Broch [mailto:ebr...@whitehorsetc.com] To:

Re: [qmailtoaster] qq soft reject errors on high load

2017-09-03 Thread Eric Broch
Are you sure they're log files in /var/qmail/simscan? They shouldn't be log files. On 9/3/2017 1:17 PM, Rajesh M wrote: i have over 50 gb of log files which can be deleted rajesh - Original Message - From: Remo Mattei [mailto:r...@mattei.org] To: qmailtoaster-list@qmailtoaster.com

Re: [qmailtoaster] qq soft reject errors on high load

2017-09-03 Thread Rajesh M
i have over 50 gb of log files which can be deleted rajesh - Original Message - From: Remo Mattei [mailto:r...@mattei.org] To: qmailtoaster-list@qmailtoaster.com Sent: Sun, 3 Sep 2017 12:13:28 -0700 Subject: 1gb is not very much! Il giorno 03 set 2017, alle ore 11:54, Rajesh M

Re: [qmailtoaster] qq soft reject errors on high load

2017-09-03 Thread Eric Broch
Rajesh, Those shouldn't be log files in that directory, but directories of this form... [root@mail ~]# ls -l /var/qmail/simscan total 7 drwxr-x--- 2 clamav vchkpw 4096 Apr 12  2016 1460525628.43125.19212 drwxr-x--- 2 clamav vchkpw 4096 Apr 12  2016 1460525917.767516.19248 drwxr-x--- 2 clamav

Re: [qmailtoaster] qq soft reject errors on high load

2017-09-03 Thread Remo Mattei
1gb is not very much! Il giorno 03 set 2017, alle ore 11:54, Rajesh M <24x7ser...@24x7server.net> ha scritto: eric around 1 gb spare. there are a lot of log files. rajesh - Original Message - From: Eric Broch [mailto:ebr...@whitehorsetc.com] To: qmailtoaster-list@qmailtoaster.com

Re: [qmailtoaster] qq soft reject errors on high load

2017-09-03 Thread Eric Broch
You got the script? On 9/3/2017 12:57 PM, Rajesh M wrote: eric yes. the idea is to use my backup server to run the script and send a large volume of email to the production server in the night to a single email account. rajesh - Original Message - From: Eric Broch

Re: [qmailtoaster] qq soft reject errors on high load

2017-09-03 Thread Rajesh M
eric around 1 gb spare. there are a lot of log files. rajesh - Original Message - From: Eric Broch [mailto:ebr...@whitehorsetc.com] To: qmailtoaster-list@qmailtoaster.com Sent: Sun, 3 Sep 2017 11:21:24 -0600 Subject: Rajesh, How much  available space is on the /var/qmail/simscan

Re: [qmailtoaster] qq soft reject errors on high load

2017-09-03 Thread Eric Broch
Rajesh, How much  available space is on the /var/qmail/simscan directory? Eric On 9/3/2017 12:19 AM, Rajesh M wrote: eric i would like to conduct more load intensive tests. is it possible to provide me the shell script that you use for this purpose. rajesh

Re: [qmailtoaster] qq soft reject errors on high load

2017-09-03 Thread Rajesh M
eric i would like to conduct more load intensive tests. is it possible to provide me the shell script that you use for this purpose. rajesh - Original Message - From: Eric Broch [mailto:ebr...@whitehorsetc.com] To: qmailtoaster-list@qmailtoaster.com Sent: Sat, 2 Sep 2017 14:04:36 -0600

RE: Fwd: Re: [qmailtoaster] qq soft reject errors on high load

2017-09-02 Thread Rajesh M
eric thanks for the info. the script you sent works the server processes roughly around 8 emails per day during week days and around 80 percent of it comes during around 12 hours. So during peak hours it is around 8000 emails per hour. i will test this out monday morning peak hours and

Fwd: Re: [qmailtoaster] qq soft reject errors on high load

2017-09-02 Thread Eric Broch
Rajesh, I'm still not sure the process limit is being reached. I'm testing a COS6/QMT server now and haven't been able to bring about 'qq soft reject' or failure of any sort. I'm hitting it with email using a delivery script from two servers with attachment size of 320KB. try the script

Re: [qmailtoaster] qq soft reject errors on high load

2017-09-02 Thread Rajesh M
eric i could not get the bash script working with either of the two options. still gives error line 23: `return-limits': not a valid identifier. can the limits issue in simscan be fixed ? thanks, rajesh - Original Message - From: Eric Broch [mailto:ebr...@whitehorsetc.com] To:

Re: [qmailtoaster] qq soft reject errors on high load

2017-09-02 Thread Eric Broch
Rajesh, If you've got the script working you can add a line above the call to 'return_limits':  echo "Count: $1 (`pgrep $1  | wc -l`)" This line will give you the number of the argument's ($1) processes as well as the rest of the output. #!/bin/bash if [ "$#" -ne "1" ]; then    echo ""   

Re: [qmailtoaster] qq soft reject errors on high load

2017-09-01 Thread Eric Broch
Rajesh, It has something to do with #!/bin/bash or #!/bin/sh. At top of script, when I use #!/bin/sh I get the same error you do, but when I use #!/bin/bash script works fine. Eric On 9/1/2017 11:07 PM, Rajesh M wrote: eric 1) is their any resolution for this issue ? 2) if i run the

Re: [qmailtoaster] qq soft reject errors on high load

2017-09-01 Thread Rajesh M
eric 1) is their any resolution for this issue ? 2) if i run the script it throws error as such . /limits.sh: line 23: `return-limits': not a valid identifier #!/bin/bash if [ "$#" -ne "1" ]; then echo "" echo -e "\033[01;32mLimit checker\033[00m" echo -e "\033[01;37mUsage:\033[01;33m

Re: [qmailtoaster] qq soft reject errors on high load

2017-09-01 Thread Jeff Koch
Hi Rajesh: Intel(R) Xeon(R) CPU E3-1230 V2 ASUS P8B-M, 16GB RAM On 9/1/2017 3:53 PM, Rajesh M wrote: jeff could you please let me know the cpu details of your machine is it a dell machine or some other with intel E5 processor ? rajesh - Original Message - From: Jeff Koch

Re: [qmailtoaster] qq soft reject errors on high load

2017-09-01 Thread Eric Broch
I sent myself a large file so that I could examine simscan with the below script (limits.sh). Here's the output: [root@pet105 simscan-1.4.0]# ./limits simscan [simscan #31535 -- limits] Limit Soft Limit   Hard Limit   Units Max cpu time  unlimited

Re: [qmailtoaster] qq soft reject errors on high load

2017-09-01 Thread Eric Broch
This looks like a decent script: #!/bin/bash if [ "$#" -ne "1" ]; then echo "" echo -e "\033[01;32mLimit checker\033[00m" echo -e "\033[01;37mUsage:\033[01;33m $0 process_name\033[00m" echo "" exit 0 fi return-limits(){ for process in $@; do       process_pids=`ps -C $process -o pid

Re: [qmailtoaster] qq soft reject errors on high load

2017-09-01 Thread Rajesh M
eric how do i check the number of user processes being consumed and by which user rajesh - Original Message - From: Eric Broch [mailto:ebr...@whitehorsetc.com] To: qmailtoaster-list@qmailtoaster.com Sent: Fri, 1 Sep 2017 14:25:36 -0600 Subject: Number of user processes. On 9/1/2017

Re: [qmailtoaster] qq soft reject errors on high load

2017-09-01 Thread Eric Broch
Number of user processes. On 9/1/2017 1:57 PM, Rajesh M wrote: eric what is RLIMIT ? is it related to memory ? rajesh - Original Message - From: Eric Broch [mailto:ebr...@whitehorsetc.com] To: qmailtoaster-list@qmailtoaster.com Sent: Fri, 1 Sep 2017 13:25:49 -0600 Subject: Here's a

Re: [qmailtoaster] qq soft reject errors on high load

2017-09-01 Thread Rajesh M
eric what is RLIMIT ? is it related to memory ? rajesh - Original Message - From: Eric Broch [mailto:ebr...@whitehorsetc.com] To: qmailtoaster-list@qmailtoaster.com Sent: Fri, 1 Sep 2017 13:25:49 -0600 Subject: Here's a section of code in simscan.c that controls the number of

Re: [qmailtoaster] qq soft reject errors on high load

2017-09-01 Thread Rajesh M
jeff could you please let me know the cpu details of your machine is it a dell machine or some other with intel E5 processor ? rajesh - Original Message - From: Jeff Koch [mailto:jeffk...@intersessions.com] To: qmailtoaster-list@qmailtoaster.com Sent: Fri, 1 Sep 2017 13:39:37 -0400

Re: [qmailtoaster] qq soft reject errors on high load

2017-09-01 Thread Eric Broch
Here's a section of code in simscan.c that controls the number of processes. #ifdef HAS_ULIMIT_NPROC   /* Set ulimits to prevent hangs if it forks too many processes */   getrlimit(RLIMIT_NPROC, );   limits.rlim_cur = 1024;   setrlimit(RLIMIT_NPROC, ); #endif The RLIMIT is set to 1024. I

Re: [qmailtoaster] qq soft reject errors on high load

2017-09-01 Thread Eric Broch
2) Problem forking (ripmime) vfork is also used to execute ripmime The error 'simscan: ripmime error' will happen for the same reasons as mentioned in the previous email (a and b below): a) [EAGAIN] The system-wide limit on the total number of processes under execution would be

Re: [qmailtoaster] qq soft reject errors on high load

2017-09-01 Thread Eric Broch
1)  Problem forking vfork() is used to execute qmail-queue The error 'simscan: error forking qmail-queue' will happen for either of the following reasons: a) [EAGAIN] The system-wide limit on the total number of processes under execution would be exceeded, or the system-imposed limit

Re: [qmailtoaster] qq soft reject errors on high load

2017-09-01 Thread Jeff Koch
Eric / Remo / Rajesh - we reported this same problem a number of months ago and thought we had fixed it. But alas, it's still with us. The good news is that we haven't had any customer complaints. I'll be very interested if it can be resolved. Jeff Koch On 9/1/2017 12:01 PM, Eric Broch

Re: [qmailtoaster] qq soft reject errors on high load

2017-09-01 Thread Rajesh M
eric the machine i have is a dell, hex core with hyperthreading, 16 gb ram -- hardly 4 gb being used, 600 gb 15krpm drive, 2 drives of 2000 gb each separately handling data. rajesh - Original Message - From: Eric Broch [mailto:ebr...@whitehorsetc.com] To:

Re: [qmailtoaster] qq soft reject errors on high load

2017-09-01 Thread Eric Broch
I'm not sure what's going on here. Is this a relatively new phenomenon? I wonder if it's a memory, or even a disk speed, issue since it only happens at peak hours? I think I'll appy Johannes Weberhofer's patch and put it out there for you...and cross or fingers. On 9/1/2017 6:28 AM,

Re: [qmailtoaster] qq soft reject errors on high load

2017-09-01 Thread Rajesh M
remo / eric i have still not being able to resolve the qq soft reject error. these are my findings 1) the errors i see are "error forking qmail-queue" and "ripmime error" which causes the qq soft reject. 2) the max concurrent connections in the logs is around 35. 3) These errors come up

Re: [qmailtoaster] qq soft reject errors on high load

2017-08-31 Thread Rajesh M
thank for sharing your experience remo, so would the following be correct ? chmod 2750 simscan chown clamav.clamav simscan do i need to stop qmail while applying this ? rajesh - Original Message - From: Remo Mattei [mailto:r...@mattei.org] To:

Re: [qmailtoaster] qq soft reject errors on high load

2017-08-31 Thread Remo Mattei
I had the same issue with qq. I changed the permissions and all good. I also raised the value of the softlimits exec /usr/bin/softlimit -m 104000 and my permissions are drwxr-s---   2 clamav clamav 4096 Aug 31 07:19 simscan So hopefully this helps. Remo On 8/31/17 6:51 AM, Rajesh M

Re: [qmailtoaster] qq soft reject errors on high load

2017-08-31 Thread Rajesh M
eric on my qmailtoaster server (centos 6 64 bit) there are several domains --- high traffic about 80 percent of them are routed via the sophos antispam filter and then passed on to the mailserver with spamassassin/clam disabled by adding the ip of the sophos server to the tcp.smtp file to

[qmailtoaster] qq soft reject errors on high load

2017-08-31 Thread Rajesh M
hi centos 6 64 bit SpamAssassin version 3.3.2 ClamAV 0.98.6/23745/Thu Aug 31 14:07:08 2017 we have a sophos antispam firewall which receives our emails and then routes them to our mail server (qmailtoaster, dovecot, spamassassin, clam) in the mail server we had stopped spamassassin however