------------------------------------------------------------
revno: 1032
committer: Mark Sapiro <[EMAIL PROTECTED]>
branch nick: 2.1
timestamp: Sun 2007-12-16 17:00:19 -0800
message:
Switchboard.py - Added a count to the queue entry metadata to count the number
of times the entry has been dequeued in a recover backup
scenario. When the count reaches three, the entry is
preserved to the shunt queue and the fact logged. This
prevents looping on a bad entry that causes a runner crash.
modified:
Mailman/Queue/Switchboard.py
NEWS
=== modified file 'Mailman/Queue/Switchboard.py'
--- a/Mailman/Queue/Switchboard.py 2007-05-08 03:16:04 +0000
+++ b/Mailman/Queue/Switchboard.py 2007-12-17 01:00:19 +0000
@@ -63,6 +63,11 @@
# Small increment to add to time in case two entries have the same time. This
# prevents skipping one of two entries with the same time until the next pass.
DELTA = .0001
+# We count the number of times a file has been dequeued. This can be more
+# than one if the file has been moved to .bak and recovered. In order to
+# prevent loops and a message flood, when the count reaches this value, we
+# move the file to the shunt queue as a .psv.
+MAX_BAK_COUNT = 3
@@ -147,19 +152,34 @@
filename = os.path.join(self.__whichq, filebase + '.pck')
backfile = os.path.join(self.__whichq, filebase + '.bak')
# Read the message object and metadata.
- fp = open(filename)
+ fp = open(filename, 'rb+')
# Move the file to the backup file name for processing. If this
# process crashes uncleanly the .bak file will be used to re-instate
- # the .pck file in order to try again. XXX what if something caused
- # Python to constantly crash? Is it possible that we'd end up mail
- # bombing recipients or crushing the archiver? How would we defend
- # against that?
+ # the .pck file in order to try again. We keep count in _bak_count
+ # in the metadata which we rewrite of the number of times we recover
+ # and dequeue this file. When the count reaches MAX_BAK_COUNT, we
+ # move the .bak file to a .psv file in the shunt queue.
os.rename(filename, backfile)
try:
msg = cPickle.load(fp)
+ data_pos = fp.tell()
data = cPickle.load(fp)
+ data['_bak_count'] = data.setdefault('_bak_count', 0) + 1
+ fp.seek(data_pos)
+ if data.get('_parsemsg'):
+ protocol = 0
+ else:
+ protocol = 1
+ cPickle.dump(data, fp, protocol)
+ fp.truncate()
+ fp.flush()
+ os.fsync(fp.fileno())
finally:
fp.close()
+ if data['_bak_count'] >= MAX_BAK_COUNT:
+ syslog('error', '.bak file max count, preserving file: %s',
+ filebase)
+ self.finish(filebase, preserve=True)
if data.get('_parsemsg'):
msg = email.message_from_string(msg, Message.Message)
return msg, data
=== modified file 'NEWS'
--- a/NEWS 2007-12-04 19:52:18 +0000
+++ b/NEWS 2007-12-17 01:00:19 +0000
@@ -4,7 +4,7 @@
Here is a history of user visible changes to Mailman.
-2.1.10b1 (04-Dec-2007)
+2.1.10b2 (xx-xxx-xxxx)
Security
@@ -141,6 +141,18 @@
- Coerced list name arguments to lower case in the change_pw, inject,
list_admins and list_owners command line tools (patch 1842412).
+ - Fixed cron/disabled to test if bounce info is stale before disabling
+ a member when the threshold has been reduced.
+
+ - It wasn't noted here, but in 2.1.9, queue runner processing was made
+ more robust by making backups of queue entries when they were dequeued
+ so they could be recovered in the event of a system failure. This
+ opened the possibility that if a message itself caused a runner to
+ crash, a loop could result that would endlessly reprocess the message.
+ This has now been fixed by adding a dequeue count to the entry and
+ moving the entry aside and logging the fact after the third dequeue of
+ the same entry.
+
2.1.9 (12-Sep-2006)
Security
--
https://code.launchpad.net/~mailman-coders/mailman/2.1
You are receiving this branch notification because you are subscribed to it.
To unsubscribe from this branch go to
https://code.launchpad.net/~mailman-coders/mailman/2.1/+subscription/mailman-checkins.
_______________________________________________
Mailman-checkins mailing list
[email protected]
Unsubscribe:
http://mail.python.org/mailman/options/mailman-checkins/archive%40jab.org
