Re: Huge active queue and system idle, not delivering

Patrick Chemla: Wietse: > > OK, so you can turn back on that connection caching. Note that > > qmail creates and destroys two processes per SMTP session, so > > reusing a session is also a win from a CPU resource point of view. Patrick: > If I do so, will postfix open more than one connexion to ea

Re: Huge active queue and system idle, not delivering

Le 11/01/2010 09:27, Stan Hoeppner a écrit : Patrick Chemla put forth on 1/11/2010 1:02 AM: Le 10/01/2010 23:58, Stan Hoeppner a écrit : On a technical level I'm happy you got it working. Just please tell us you're not sending mass spam with this setup. -- Stan I have to d

Re: Huge active queue and system idle, not delivering

Patrick Chemla put forth on 1/11/2010 1:02 AM: > Le 10/01/2010 23:58, Stan Hoeppner a écrit : >> On a technical level I'm happy you got it working. Just please tell >> us you're >> not sending mass spam with this setup. >> >> -- >> Stan >> > > I have to do it for a customer who send as he sa

Re: Huge active queue and system idle, not delivering

Le 11/01/2010 01:13, Wietse Venema a écrit : Patrick Chemla: Wietse, Please try the following, as asked half a week ago: postconf -e smtp_connection_cache_on_demand=no postfix reload and report if this makes a difference. Wietse I have tested this sin

Re: Huge active queue and system idle, not delivering

Le 10/01/2010 23:58, Stan Hoeppner a écrit : On a technical level I'm happy you got it working. Just please tell us you're not sending mass spam with this setup. -- Stan I have to do it for a customer who send as he said, only opt-in mass emails. He has a big blacklisted email database w

Re: Huge active queue and system idle, not delivering

Patrick Chemla: > Wietse, > >> Please try the following, as asked half a week ago: > >> > >> postconf -e smtp_connection_cache_on_demand=no > >> postfix reload > >> > >> and report if this makes a difference. > >>Wietse > >> > I have tested this since yesterday night. > > I got

Re: Huge active queue and system idle, not delivering

Patrick Chemla put forth on 1/10/2010 3:00 PM: > Wietse, >>> Please try the following, as asked half a week ago: >>> >>> postconf -e smtp_connection_cache_on_demand=no >>> postfix reload >>> >>> and report if this makes a difference. >>> Wietse >>> > I have tested this since yes

Re: Huge active queue and system idle, not delivering

Wietse, Please try the following, as asked half a week ago: postconf -e smtp_connection_cache_on_demand=no postfix reload and report if this makes a difference. Wietse I have tested this since yesterday night. I got some problems with Linux per user number of processes

Re: Huge active queue and system idle, not delivering

Wietse Venema: > Patrick Chemla: > > Hi all, > > > > I got these statistics: > > > > Jan 9 19:15:21 postfix postfix/scache[18038]: statistics: start > > interval Jan 9 19:09:03 > > Jan 9 19:15:21 postfix postfix/scache[18038]: statistics: domain lookup > > hits=110 miss=89 success=55% > > Ja

Re: Huge active queue and system idle, not delivering

Patrick Chemla: > Hi all, > > I got these statistics: > > Jan 9 19:15:21 postfix postfix/scache[18038]: statistics: start > interval Jan 9 19:09:03 > Jan 9 19:15:21 postfix postfix/scache[18038]: statistics: domain lookup > hits=110 miss=89 success=55% > Jan 9 19:15:21 postfix postfix/scach

Re: Huge active queue and system idle, not delivering

Patrick Chemla put forth on 1/9/2010 1:08 PM: > You mean 100% success? Yes. > Before I set up the postfix relay to load balance between 30 qmail > servers, each of them was able to accept in his own queue hundreds > thousands email. Email were sent by campaigns of thousands balanced on 3 > qmail

Re: Huge active queue and system idle, not delivering

Le 09/01/2010 20:54, Stan Hoeppner a écrit : Patrick Chemla put forth on 1/9/2010 12:37 PM: I wen t there but did not find explanations about miss address lookup or miss domain lookup. While I have 122,000 messages in active queue I still don't understand why statistics show max simultaneou

Re: Huge active queue and system idle, not delivering

Patrick Chemla put forth on 1/9/2010 12:37 PM: > I wen t there but did not find explanations about miss address lookup or > miss domain lookup. > While I have 122,000 messages in active queue I still don't understand > why statistics show max simultaneous domains=1. It should be dozens , or > hund

Re: Huge active queue and system idle, not delivering

Patrick Chemla put forth on 1/9/2010 11:07 AM: > Hi, > > I will try all your advises, but something still very strange for me: > > We see that postfix logs show that ehlo process is very slow through > postfix but very fast by hand. Even I have recorded through > tcpdump/WireShark and I can see t

Re: Huge active queue and system idle, not delivering

Hi Stan, Thanks for your interest. Le 09/01/2010 20:21, Stan Hoeppner a écrit : Patrick Chemla put forth on 1/9/2010 11:17 AM: Hi all, I got these statistics: Jan 9 19:15:21 postfix postfix/scache[18038]: statistics: start interval Jan 9 19:09:03 Jan 9 19:15:21 postfix postfix/scache[

Re: Huge active queue and system idle, not delivering

Patrick Chemla put forth on 1/9/2010 11:17 AM: > Hi all, > > I got these statistics: > > Jan 9 19:15:21 postfix postfix/scache[18038]: statistics: start > interval Jan 9 19:09:03 > Jan 9 19:15:21 postfix postfix/scache[18038]: statistics: domain lookup > hits=110 miss=89 success=55% > Jan 9 1

Re: Huge active queue and system idle, not delivering

Hi all, I got these statistics: Jan 9 19:15:21 postfix postfix/scache[18038]: statistics: start interval Jan 9 19:09:03 Jan 9 19:15:21 postfix postfix/scache[18038]: statistics: domain lookup hits=110 miss=89 success=55% Jan 9 19:15:21 postfix postfix/scache[18038]: statistics: address lo

Re: Huge active queue and system idle, not delivering

Hi, I will try all your advises, but something still very strange for me: We see that postfix logs show that ehlo process is very slow through postfix but very fast by hand. Even I have recorded through tcpdump/WireShark and I can see that messages are sent very very very quickly in about 1 s

Re: Huge active queue and system idle, not delivering

On Fri, Jan 08, 2010 at 03:24:25PM +0200, Patrick Chemla wrote: > When I do telnet a139.localpc2105.com 25, I get immediate response. What does "response" mean? Immediate connection completion means nothing. Do you get a 220 banner right away? Do you get all of it or just the first line in a mul

Re: Huge active queue and system idle, not delivering

Wietse Venema: > Patrick Chemla: > > Le 08/01/2010 00:43, Victor Duchovni a ?crit : > > > On Fri, Jan 08, 2010 at 12:30:34AM +0200, Patrick Chemla wrote: > > > > > > > > >> Jan 7 22:02:57 postfix postfix/qmgr[26441]: 5B91F873F6: removed > > >> Jan 7 22:02:57 postfix postfix/smtp[27180]: 375DD

Re: Huge active queue and system idle, not delivering

On Fri, 08 Jan 2010 15:24:25 +0200, Patrick Chemla > When I telnet a70.localpc2105.com 25 I get an immediate response. I assume you are telnet'ing from the Postfix server with the queue delay problem. At this point, after you receive the 220, type: ehlo your.postfix-server.tld and time the d

Re: Huge active queue and system idle, not delivering

Patrick Chemla: [ Charset ISO-8859-1 unsupported, converting... ] > Le 08/01/2010 00:43, Victor Duchovni a ?crit : > > On Fri, Jan 08, 2010 at 12:30:34AM +0200, Patrick Chemla wrote: > > > > > >> Jan 7 22:02:57 postfix postfix/qmgr[26441]: 5B91F873F6: removed > >> Jan 7 22:02:57 postfix postf

Re: Huge active queue and system idle, not delivering

Le 08/01/2010 00:43, Victor Duchovni a écrit : On Fri, Jan 08, 2010 at 12:30:34AM +0200, Patrick Chemla wrote: Jan 7 22:02:57 postfix postfix/qmgr[26441]: 5B91F873F6: removed Jan 7 22:02:57 postfix postfix/smtp[27180]: 375DDD5923: to=, relay=a139.localpc2105.com[10.0.0.139]:25, conn_use=5

Re: Huge active queue and system idle, not delivering

Le 08/01/2010 03:03, Wietse Venema a écrit : Patrick Chemla: But the CPU of the box is idle more than 80%. It is clear that it is not a matter of CPU, nor memory, nor disk. Something in the number of processes/users/simultaneous tasks is blocking. Indeed, the symptom of blocking i

Re: Huge active queue and system idle, not delivering

Patrick Chemla: > >> But the CPU of the box is idle more than 80%. It is clear that it is not a > >> matter of CPU, nor memory, nor disk. Something in the number of > >> processes/users/simultaneous tasks is blocking. Indeed, the symptom of blocking is in the third field of the Postfix "delays" lo

Re: Huge active queue and system idle, not delivering

On Fri, Jan 08, 2010 at 12:30:34AM +0200, Patrick Chemla wrote: > Jan 7 22:02:57 postfix postfix/qmgr[26441]: 5B91F873F6: removed > Jan 7 22:02:57 postfix postfix/smtp[27180]: 375DDD5923: > to=, relay=a139.localpc2105.com[10.0.0.139]:25, > conn_use=59, delay=61550, delays=17019/44435/96/0.17,

Re: Huge active queue and system idle, not delivering

Le 07/01/2010 20:37, Victor Duchovni a écrit : On Thu, Jan 07, 2010 at 08:29:44PM +0200, Patrick Chemla wrote: Here the logs: This is just the qmgr(8) warnings about a clogged queue. Other than telling us that all the mail is going to "localpc2105.com", this is not very useful. Where

Re: Huge active queue and system idle, not delivering

Le 07/01/2010 23:47, Stefan Caunter a écrit : On Thu, Jan 7, 2010 at 1:25 PM, Patrick Chemla wrote: said "I just found that Postfix could send 1 million emails per hour when I send less than a half million in 24 hours", but I can't make sense of that, sorry. I have to inject 2 t

Re: Huge active queue and system idle, not delivering

On Thu, Jan 07, 2010 at 10:54:15PM +0100, Ralf Hildebrandt wrote: > > It could be viewed that half a million delivered in 24 hours is fine. > > Are you signing the mail? This can help with delivery rates to the > > large webmailer mx destinations. > > There are many things to consider: > > * DKI

Re: Huge active queue and system idle, not delivering

* Stefan Caunter : > It could be viewed that half a million delivered in 24 hours is fine. > Are you signing the mail? This can help with delivery rates to the > large webmailer mx destinations. There are many things to consider: * DKIM signing - which is the prerequisite for getting into feedba

Re: Huge active queue and system idle, not delivering

On Thu, Jan 07, 2010 at 04:47:14PM -0500, Stefan Caunter wrote: > > > > I have to inject 2 to 4 millions emails to the postfix box in 24 hours, and > > I expect to deliver within the same delay. > > Actually, I can't deliver more than 500,000 per 24h hours. > > It could be viewed that half a mill

Re: Huge active queue and system idle, not delivering

On Thu, Jan 7, 2010 at 1:25 PM, Patrick Chemla wrote: >> said "I just found that Postfix could send 1 million emails per hour >> when I send less than a half million in 24 hours", but I can't make >> sense of that, sorry. >> > > I have to inject 2 to 4 millions emails to the postfix box in 24 hou

Re: Huge active queue and system idle, not delivering

On Thu, Jan 07, 2010 at 08:29:44PM +0200, Patrick Chemla wrote: > Here the logs: This is just the qmgr(8) warnings about a clogged queue. Other than telling us that all the mail is going to "localpc2105.com", this is not very useful. Where are the logs from smtp(8)? What transport is "localpc210

Re: Huge active queue and system idle, not delivering

On Thu, Jan 07, 2010 at 07:43:55PM +0200, Patrick Chemla wrote: > CPU is more than 85% idle on my postfix I5/750 box, but the outbound queue > is very very slow. Throughput == Concurrency / Latency What destination are most of the messages in the queue going to? What is the associated tran

Re: Huge active queue and system idle, not delivering

Le 07/01/2010 20:00, Wietse Venema a écrit : Patrick Chemla: Hi, I am running Postfix 2.5.6 on a Fedora 11 Linux system on a hardware based Intel I5/750 Quad Core, 8 Gb memory, 160Gb SSD hard disk. Incoming messages are entering very fast (500 smtp processes declared) and the active queue

Re: Huge active queue and system idle, not delivering

Le 07/01/2010 20:03, Barney Desmond a écrit : 2010/1/8 Patrick Chemla Incoming messages are entering very fast (500 smtp processes declared) and the active queue is actually of 2 millions messages waiting for delivery. here is my main.cf file: That's some very thorough information, y

Re: Huge active queue and system idle, not delivering

2010/1/8 Patrick Chemla > Incoming messages are entering very fast (500 smtp processes declared) and > the active queue is actually of 2 millions messages waiting for delivery. > > here is my main.cf file: That's some very thorough information, you've provided plenty of context and clear descrip

Re: Huge active queue and system idle, not delivering

Patrick Chemla: > Hi, > > I am running Postfix 2.5.6 on a Fedora 11 Linux system on a hardware > based Intel I5/750 Quad Core, 8 Gb memory, 160Gb SSD hard disk. > > Incoming messages are entering very fast (500 smtp processes declared) > and the active queue is actually of 2 millions messages

Huge active queue and system idle, not delivering

Hi, I am running Postfix 2.5.6 on a Fedora 11 Linux system on a hardware based Intel I5/750 Quad Core, 8 Gb memory, 160Gb SSD hard disk. Incoming messages are entering very fast (500 smtp processes declared) and the active queue is actually of 2 millions messages waiting for delivery. The