Khalil Abbas wrote:
>lol I'm not shouting, just too lazy to hit caps :)
Regardless of why it happened, it comes across to others as shouting.
>well I bought this server:
>
>RAM 1x 4 GB DDR2-RAM ECC
>HDD 2x 500 GB SATA II-HDD 7.200 rpm
>Barebone 1x Fujitsu PRIMERGY RX100 S5
>CPU 1x Intel Xeon X3320 Quadcore
>
>I did set the SMTP_MAX_RCPT to 5 .. then I created a list of 10,000 members,
>n it delivered them in like 5-6 minutes ..
>
>there was about 500 yahoo addresses and yahoo deferred a few messages..
>luckily Hotmail accepted all 5500 addresses..
>
>is that good?
It's not terrific. At that rate, assuming everything had completely
left your MTA in that time, it would take between 17 and 20 hours to
deliver to 2 million recipients. Is that acceptable?
Also, it is not clear that just because Hotmail accepted all 5500
addresses out of 10,000, that they will accept the next 55,000 out of
the next 100,000 (if there are that many). Also, once Yahoo starts
deferring, their deferral rate will increase if you keep sending more,
and this will add to the load on your MTA.
>do you think getting 2 or 3 or even 4 servers like this one and splitting
>the 2 million users over them is better?
I can't say. Perhaps the bottleneck is the bandwidth of your internet
connection. If so, increasinfg the ability to push more data faster
won't help at all.
>about the throttle patch, ok I confess, I AM IGNORANT! I have no idea how to
>install the patch :)
>
>I have mailman 2.1.9 by the way ..
The throttle.patch.txt file attached to this message contains a 2.1.9
version of the patch. The only difference is the line number of the
last portion.
To install it, save the file and cd to your mailman installation and
give the command
patch Mailman/Handlers/SMTPDirect.py < /path/to/throttle.patch.txt
and then restart Mailman.
If anything goes wrong, you can remove the patch with the command
patch -R Mailman/Handlers/SMTPDirect.py < /path/to/throttle.patch.txt
followed by another restart of Mailman.
--
Mark Sapiro <m...@msapiro.net> The highway is for gamblers,
San Francisco Bay Area, California better use your sense - B. Dylan
--- SMTPDirect.py.orig 2011-06-25 17:39:07.859375000 -0700
+++ SMTPDirect.py 2011-06-25 17:43:04.078125000 -0700
@@ -52,6 +52,37 @@
True = 1
False = 0
+# Throttling settings. Do not send to more than THROTTLE_LIMIT recipients
+# within THROTTLE_TIME seconds. Set THROTTLE_LIMIT to 0 to not throttle.
+THROTTLE_TIME = 60
+THROTTLE_LIMIT = 2000
+THROTTLE_DATA = []
+
+# check throttling. Doesn't return until total recipients in THROTTLE_TIME
+# <= THROTTLE_LIMIT.
+#
+def check_throttle(count):
+ global THROTTLE_DATA
+ if THROTTLE_LIMIT <= 0:
+ return
+ if count <= 0 or count > THROTTLE_LIMIT:
+ syslog('error', 'check_throttle: count=%d; THROTTLE_LIMIT=%d',
+ count, THROTTLE_LIMIT)
+ return
+ now = time.time()
+ total = 0
+ THROTTLE_DATA = [(num, tim) for (num, tim) in THROTTLE_DATA
+ if tim > now - THROTTLE_TIME]
+ for num, tim in THROTTLE_DATA:
+ total += num
+ total += count
+ if total > THROTTLE_LIMIT:
+ time.sleep(THROTTLE_TIME - now + THROTTLE_DATA[0][1])
+ check_throttle(count)
+ else:
+ THROTTLE_DATA.append((count, now))
+ return
+
# Manage a connection to the SMTP server
@@ -118,10 +149,14 @@
chunks = [[recip] for recip in recips]
msgdata['personalize'] = 1
deliveryfunc = verpdeliver
- elif mm_cfg.SMTP_MAX_RCPTS <= 0:
+ elif mm_cfg.SMTP_MAX_RCPTS <= 0 and (THROTTLE_LIMIT <=0 or
+ THROTTLE_LIMIT >= len(recips)):
chunks = [recips]
else:
- chunks = chunkify(recips, mm_cfg.SMTP_MAX_RCPTS)
+ if THROTTLE_LIMIT <= 0 or THROTTLE_LIMIT > mm_cfg.SMTP_MAX_RCPTS:
+ chunks = chunkify(recips, mm_cfg.SMTP_MAX_RCPTS)
+ else:
+ chunks = chunkify(recips, THROTTLE_LIMIT)
# See if this is an unshunted message for which some were undelivered
if msgdata.has_key('undelivered'):
chunks = msgdata['undelivered']
@@ -366,6 +401,8 @@
msgid = msg['message-id']
try:
# Send the message
+ # but wait if necessary for throttling
+ check_throttle(len(recips))
refused = conn.sendmail(envsender, recips, msgtext)
except smtplib.SMTPRecipientsRefused, e:
syslog('smtp-failure', 'All recipients refused: %s, msgid: %s',
------------------------------------------------------
Mailman-Users mailing list Mailman-Users@python.org
http://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe:
http://mail.python.org/mailman/options/mailman-users/archive%40jab.org
