I just don't see the point of this. You may have stopped a large list inject being so aggressive on CPU usage, but you have increased the lifetime of processes by a factor of 30 or so (on your figures), and so increased the memory pressure and likelihood of swapping etc due to processes being just as fat but lasting longer.
The process we're talking about is OutgoingRunner, which is always in memory. I don't mind keeping the whole list in RAM -- I have 3 Gb, and the list (and chunks) probably eat up something like twice the size of the config.pck file, so about 44Mb. The only problem I see is if the list were locked during all that time, but I don't think (from what I understand from the code) that it is.
Nigel isn't talking about the Mailman processes. He's talking about the MTA processes. 44MB of memory is nothing on a large list, and since you have 3GB, that should not even be noticeable to you.
But the MTA processes sticking around much longer and not being able to work as efficiently, that is something that will hurt you far, far more.
Well that's already the case as postfix (my MTA) has 150 K messages to deliver, isn't it? But right now Mailman injects everything into postfix w/o considering that it might explode the load and system hit. It's brutal, with the load going to 25, and CPU to 70% (and I have a very nice machine, a brand new quadri-processor thing :-] ).
If it's 150,000 recipients, with SMTP_MAX_RCPTS = 900, then that's really just 167 messages, each addressed to multiple recipients (up to 900). Postfix is really good at handling things like this in parallel, so that quick sites get more mail dumped on them faster, and slower sites get mail delivered to them at a speed they are more comfortable with.
You really need to properly understand just what it is that you're mucking about with, before you start making random changes.
You may find you have made things less efficient by having 2 deliveries to a single list slowed down so that different messages to the same recipient can no longer be put in the same SMTP session (if your MTA does that).
I'm not sure postfix does that, no. And it's not really relevant in this case, as I really only have ONE big list with ONE message to send (and alot of small lists and normal emails to try to sneak through if possible).
Yes, postfix will coalesce messages together, if it finds multiple messages in the queue that are addressed to the same recipients. This is an old time/performance trick that even ancient versions of sendmail employed, and all modern MTAs likewise make use of.
And no, you really *DON'T* have just ONE message going out to ONE big list. When a message comes in for that list, it gets broken up into chunks (controlled by SMTP_MAX_RCPTS), and each chunk is then handed off to the MTA as a separate instance. This happens in pretty much the same way for all lists, regardless of how many recipients may be on a particular list, or how many messages may be coming in or going out for any given list.
Why not just run the cron jobs under nice instead?
What cron jobs? Sure I can renice the OutgoingRunner, but it won't change much (except that it will get swapped first if something needs to get swapped).
I think Nigel was misunderstanding your description of the problem.
-- Brad Knowles, <[EMAIL PROTECTED]>
"They that can give up essential liberty to obtain a little temporary
safety deserve neither liberty nor safety."
-Benjamin Franklin, Historical Review of Pennsylvania.GCS/IT d+(-) s:+(++)>: a C++(+++)$ UMBSHI++++$ P+>++ L+ !E-(---) W+++(--) N+ !w--- O- M++ V PS++(+++) PE- Y+(++) PGP>+++ t+(+++) 5++(+++) X++(+++) R+(+++) tv+(+++) b+(++++) DI+(++++) D+(++) G+(++++) e++>++++ h--- r---(+++)* z(+++)
_______________________________________________ Mailman-Developers mailing list [EMAIL PROTECTED] http://mail.python.org/mailman/listinfo/mailman-developers Unsubscribe: http://mail.python.org/mailman/options/mailman-developers/archive%40jab.org
