------------------------------------------------------------
revno: 6636
committer: Barry Warsaw <[EMAIL PROTECTED]>
branch nick: 3.0
timestamp: Thu 2008-07-10 22:23:52 -0400
message:
Merging the Mail-Archive.com branch.
added:
mailman/archiving/
mailman/archiving/__init__.py
mailman/archiving/mailarchive.py
mailman/archiving/mhonarc.py
mailman/archiving/prototype.py
renamed:
mailman/app/archiving.py => mailman/archiving/pipermail.py
modified:
mailman/Defaults.py
mailman/bin/testall.py
mailman/configuration.py
mailman/docs/archivers.txt
mailman/docs/pipelines.txt
mailman/initialize.py
mailman/interfaces/mailinglist.py
mailman/loginit.py
mailman/pipeline/docs/cook-headers.txt
mailman/queue/docs/outgoing.txt
mailman/testing/helpers.py
mailman/testing/smtplistener.py
mailman/testing/testing.cfg.in
mailman/tests/test_documentation.py
setup.py
mailman/archiving/pipermail.py
------------------------------------------------------------
revno: 6635.1.4
committer: Barry Warsaw <[EMAIL PROTECTED]>
branch nick: madotcom
timestamp: Wed 2008-07-09 22:35:44 -0400
message:
Add rough support for a local MHonArc archiver, using the permalink
proposal
from the prototype archiver.
Add a mailman.archiver log.
added:
mailman/archiving/mhonarc.py
modified:
mailman/Defaults.py
mailman/archiving/mailarchive.py
mailman/archiving/prototype.py
mailman/bin/testall.py
mailman/docs/archivers.txt
mailman/loginit.py
setup.py
------------------------------------------------------------
revno: 6635.1.3
committer: Barry Warsaw <[EMAIL PROTECTED]>
branch nick: madotcom
timestamp: Sun 2008-07-06 10:27:05 -0400
message:
Use IMailingList.posting_address (a.k.a. List-Post header) in the
Mail-Archive.com hash calculation, not the post-id sequence number.
test_documentation now starts and stops the smtp listener, simplifing many
tests.
modified:
mailman/archiving/mailarchive.py
mailman/docs/archivers.txt
mailman/queue/docs/outgoing.txt
mailman/tests/test_documentation.py
------------------------------------------------------------
revno: 6635.1.2
committer: Barry Warsaw <[EMAIL PROTECTED]>
branch nick: madotcom
timestamp: Sat 2008-07-05 11:06:37 -0400
message:
Refactor the archivers so that they live in a separate sub-package.
Split out
the Pipermail, Prototype, and MailArchiver plugins into separate modules.
Put the archives registry on the config object and initialize it at the
right
time.
Update plugin entry points.
added:
mailman/archiving/
mailman/archiving/__init__.py
mailman/archiving/mailarchive.py
mailman/archiving/prototype.py
renamed:
mailman/app/archiving.py => mailman/archiving/pipermail.py
modified:
mailman/configuration.py
mailman/docs/archivers.txt
mailman/docs/pipelines.txt
mailman/initialize.py
mailman/pipeline/docs/cook-headers.txt
mailman/tests/test_documentation.py
setup.py
mailman/archiving/pipermail.py
------------------------------------------------------------
revno: 6635.1.1
committer: Barry Warsaw <[EMAIL PROTECTED]>
branch nick: madotcom
timestamp: Sat 2008-07-05 10:41:12 -0400
message:
Implement the basic Mail-Archive.com archiving algorithm.
Fix the start up of the SMTPServer to listen on a consistent port number.
post_number -> post_id
modified:
mailman/Defaults.py
mailman/app/archiving.py
mailman/bin/testall.py
mailman/docs/archivers.txt
mailman/interfaces/mailinglist.py
mailman/testing/helpers.py
mailman/testing/smtplistener.py
mailman/testing/testing.cfg.in
setup.py
=== modified file 'mailman/Defaults.py'
--- a/mailman/Defaults.py 2008-03-31 02:55:22 +0000
+++ b/mailman/Defaults.py 2008-07-10 02:35:44 +0000
@@ -220,6 +220,22 @@
# - $fqdn_listname -- the long name of the list being accessed
PUBLIC_ARCHIVE_URL = 'http://$hostname/pipermail/$fqdn_listname'
+# The public Mail-Archive.com service's base url.
+MAIL_ARCHIVE_BASEURL = 'http://go.mail-archive.com/'
+# The posting address for the Mail-Archive.com service
+MAIL_ARCHIVE_RECIPIENT = '[EMAIL PROTECTED]'
+
+# The command for archiving to a local MHonArc instance.
+MHONARC_COMMAND = """\
+/usr/bin/mhonarc \
+-add \
+-dbfile $PRIVATE_ARCHIVE_FILE_DIR/${listname}.mbox/mhonarc.db \
+-outdir $VAR_DIR/mhonarc/${listname} \
+-stderr $LOG_DIR/mhonarc \
+-stdout $LOG_DIR/mhonarc \
+-spammode \
+-umask 022"""
+
# Are archives on or off by default?
DEFAULT_ARCHIVE = On
=== added directory 'mailman/archiving'
=== added file 'mailman/archiving/__init__.py'
--- a/mailman/archiving/__init__.py 1970-01-01 00:00:00 +0000
+++ b/mailman/archiving/__init__.py 2008-07-05 15:06:37 +0000
@@ -0,0 +1,31 @@
+# Copyright (C) 2008 by the Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License
+# as published by the Free Software Foundation; either version 2
+# of the License, or (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write to the Free Software
+# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301,
+# USA.
+
+__metaclass__ = type
+__all__ = [
+ 'initialize',
+ ]
+
+
+from mailman.app.plugins import get_plugins
+from mailman.configuration import config
+
+
+def initialize():
+ """Initialize archivers."""
+ for archiver in get_plugins('mailman.archiver'):
+ config.archivers[archiver.name] = archiver
=== added file 'mailman/archiving/mailarchive.py'
--- a/mailman/archiving/mailarchive.py 1970-01-01 00:00:00 +0000
+++ b/mailman/archiving/mailarchive.py 2008-07-10 02:35:44 +0000
@@ -0,0 +1,87 @@
+# Copyright (C) 2008 by the Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License
+# as published by the Free Software Foundation; either version 2
+# of the License, or (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write to the Free Software
+# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301,
+# USA.
+
+"""The Mail-Archive.com archiver."""
+
+__metaclass__ = type
+__all__ = [
+ 'MailArchive',
+ ]
+
+
+import hashlib
+
+from base64 import urlsafe_b64encode
+from urllib import quote
+from urlparse import urljoin
+from zope.interface import implements
+
+from mailman.configuration import config
+from mailman.interfaces.archiver import IArchiver
+from mailman.queue import Switchboard
+
+
+
+class MailArchive:
+ """Public archiver at the Mail-Archive.com.
+
+ Messages get archived at http://go.mail-archive.com.
+ """
+
+ implements(IArchiver)
+
+ name = 'mail-archive'
+ is_enabled = False
+
+ @staticmethod
+ def list_url(mlist):
+ """See `IArchiver`."""
+ if mlist.archive_private:
+ return None
+ return urljoin(config.MAIL_ARCHIVE_BASEURL,
+ quote(mlist.posting_address))
+
+ @staticmethod
+ def permalink(mlist, msg):
+ """See `IArchiver`."""
+ if mlist.archive_private:
+ return None
+ message_id = msg.get('message-id')
+ # It is not the archiver's job to ensure the message has a Message-ID.
+ assert message_id is not None, 'No Message-ID found'
+ # The angle brackets are not part of the Message-ID. See RFC 2822.
+ if message_id.startswith('<') and message_id.endswith('>'):
+ message_id = message_id[1:-1]
+ else:
+ message_id = message_id.strip()
+ sha = hashlib.sha1(message_id)
+ sha.update(str(mlist.posting_address))
+ message_id_hash = urlsafe_b64encode(sha.digest())
+ del msg['x-message-id-hash']
+ msg['X-Message-ID-Hash'] = message_id_hash
+ return urljoin(config.MAIL_ARCHIVE_BASEURL, message_id_hash)
+
+ @staticmethod
+ def archive_message(mlist, msg):
+ """See `IArchiver`."""
+ if mlist.archive_private:
+ return
+ outq = Switchboard(config.OUTQUEUE_DIR)
+ outq.enqueue(
+ msg,
+ listname=mlist.fqdn_listname,
+ recips=[config.MAIL_ARCHIVE_RECIPIENT])
=== added file 'mailman/archiving/mhonarc.py'
--- a/mailman/archiving/mhonarc.py 1970-01-01 00:00:00 +0000
+++ b/mailman/archiving/mhonarc.py 2008-07-10 02:35:44 +0000
@@ -0,0 +1,95 @@
+# Copyright (C) 2008 by the Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License
+# as published by the Free Software Foundation; either version 2
+# of the License, or (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write to the Free Software
+# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301,
+# USA.
+
+"""MHonArc archiver."""
+
+__metaclass__ = type
+__all__ = [
+ 'MHonArc',
+ ]
+
+
+import hashlib
+import logging
+import subprocess
+
+from base64 import b32encode
+from string import Template
+from urlparse import urljoin
+from zope.interface import implements
+
+from mailman.configuration import config
+from mailman.interfaces.archiver import IArchiver
+
+
+log = logging.getLogger('mailman.archiver')
+
+
+
+class MHonArc:
+ """Local MHonArc archiver."""
+
+ implements(IArchiver)
+
+ name = 'mhonarc'
+ is_enabled = False
+
+ @staticmethod
+ def list_url(mlist):
+ """See `IArchiver`."""
+ # XXX What about private MHonArc archives?
+ web_host = config.domains.get(mlist.host_name, mlist.host_name)
+ return Template(config.PUBLIC_ARCHIVE_URL).safe_substitute(
+ listname=mlist.fqdn_listname,
+ hostname=web_host,
+ fqdn_listname=mlist.fqdn_listname,
+ )
+
+ @staticmethod
+ def permalink(mlist, msg):
+ """See `IArchiver`."""
+ # XXX What about private MHonArc archives?
+ message_id = msg.get('message-id')
+ # It is not the archiver's job to ensure the message has a Message-ID.
+ assert message_id is not None, 'No Message-ID found'
+ # The angle brackets are not part of the Message-ID. See RFC 2822.
+ if message_id.startswith('<') and message_id.endswith('>'):
+ message_id = message_id[1:-1]
+ else:
+ message_id = message_id.strip()
+ sha = hashlib.sha1(message_id)
+ message_id_hash = b32encode(sha.digest())
+ del msg['x-message-id-hash']
+ msg['X-Message-ID-Hash'] = message_id_hash
+ return urljoin(MHonArc.list_url(mlist), message_id_hash)
+
+ @staticmethod
+ def archive_message(mlist, msg):
+ """See `IArchiver`."""
+ substitutions = config.__dict__.copy()
+ substitutions['listname'] = mlist.fqdn_listname
+ command = Template(config.MHONARC_COMMAND).safe_substitute(
+ substitutions)
+ proc = subprocess.Popen(
+ command, stdout=subprocess.PIPE, stderr=subprocess.PIPE,
+ shell=True)
+ stdout, stderr = proc.communicate(msg.as_string())
+ if proc.returncode <> 0:
+ log.error('%s: mhonarc subprocess had non-zero exit code: %s' %
+ (msg['message-id'], proc.returncode))
+ log.info(stdout)
+ log.error(stderr)
=== renamed file 'mailman/app/archiving.py' => 'mailman/archiving/pipermail.py'
--- a/mailman/app/archiving.py 2008-07-03 02:29:20 +0000
+++ b/mailman/archiving/pipermail.py 2008-07-05 15:06:37 +0000
@@ -15,23 +15,18 @@
# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301,
# USA.
-"""Application level archiving support."""
+"""Pipermail archiver."""
__metaclass__ = type
__all__ = [
'Pipermail',
- 'Prototype',
]
import os
-import hashlib
-from base64 import b32encode
from cStringIO import StringIO
-from email.utils import make_msgid
from string import Template
-from urlparse import urljoin
from zope.interface import implements
from zope.interface.interface import adapter_hooks
@@ -79,7 +74,7 @@
implements(IArchiver)
name = 'pipermail'
- is_enabled = True
+ is_enabled = False
@staticmethod
def list_url(mlist):
@@ -112,43 +107,3 @@
fileobj.close()
# There's no good way to know the url for the archived message.
return None
-
-
-
-class Prototype:
- """A prototype of a third party archiver.
-
- Mailman proposes a draft specification for interoperability between list
- servers and archivers: <http://wiki.list.org/display/DEV/Stable+URLs>.
- """
-
- implements(IArchiver)
-
- name = 'prototype'
- is_enabled = False
-
- @staticmethod
- def list_url(mlist):
- """See `IArchiver`."""
- web_host = config.domains.get(mlist.host_name, mlist.host_name)
- return 'http://' + web_host
-
- @staticmethod
- def permalink(mlist, msg):
- """See `IArchiver`."""
- message_id = msg.get('message-id')
- # It is not the archiver's job to ensure the message has a Message-ID.
- assert message_id is not None, 'No Message-ID found'
- # The angle brackets are not part of the Message-ID. See RFC 2822.
- if message_id.startswith('<') and message_id.endswith('>'):
- message_id = message_id[1:-1]
- digest = hashlib.sha1(message_id).digest()
- message_id_hash = b32encode(digest)
- del msg['x-message-id-hash']
- msg['X-Message-ID-Hash'] = message_id_hash
- return urljoin(Prototype.list_url(mlist), message_id_hash)
-
- @staticmethod
- def archive_message(mlist, message):
- """See `IArchiver`."""
- raise NotImplementedError
=== added file 'mailman/archiving/prototype.py'
--- a/mailman/archiving/prototype.py 1970-01-01 00:00:00 +0000
+++ b/mailman/archiving/prototype.py 2008-07-10 02:35:44 +0000
@@ -0,0 +1,75 @@
+# Copyright (C) 2008 by the Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License
+# as published by the Free Software Foundation; either version 2
+# of the License, or (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write to the Free Software
+# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301,
+# USA.
+
+"""Prototypical permalinking archiver."""
+
+__metaclass__ = type
+__all__ = [
+ 'Prototype',
+ ]
+
+
+import hashlib
+
+from base64 import b32encode
+from urlparse import urljoin
+from zope.interface import implements
+
+from mailman.configuration import config
+from mailman.interfaces.archiver import IArchiver
+
+
+
+class Prototype:
+ """A prototype of a third party archiver.
+
+ Mailman proposes a draft specification for interoperability between list
+ servers and archivers: <http://wiki.list.org/display/DEV/Stable+URLs>.
+ """
+
+ implements(IArchiver)
+
+ name = 'prototype'
+ is_enabled = False
+
+ @staticmethod
+ def list_url(mlist):
+ """See `IArchiver`."""
+ web_host = config.domains.get(mlist.host_name, mlist.host_name)
+ return 'http://' + web_host
+
+ @staticmethod
+ def permalink(mlist, msg):
+ """See `IArchiver`."""
+ message_id = msg.get('message-id')
+ # It is not the archiver's job to ensure the message has a Message-ID.
+ assert message_id is not None, 'No Message-ID found'
+ # The angle brackets are not part of the Message-ID. See RFC 2822.
+ if message_id.startswith('<') and message_id.endswith('>'):
+ message_id = message_id[1:-1]
+ else:
+ message_id = message_id.strip()
+ digest = hashlib.sha1(message_id).digest()
+ message_id_hash = b32encode(digest)
+ del msg['x-message-id-hash']
+ msg['X-Message-ID-Hash'] = message_id_hash
+ return urljoin(Prototype.list_url(mlist), message_id_hash)
+
+ @staticmethod
+ def archive_message(mlist, message):
+ """See `IArchiver`."""
+ raise NotImplementedError
=== modified file 'mailman/bin/testall.py'
--- a/mailman/bin/testall.py 2008-07-03 19:32:25 +0000
+++ b/mailman/bin/testall.py 2008-07-10 02:35:44 +0000
@@ -31,9 +31,11 @@
import unittest
import pkg_resources
+from mailman import Defaults
from mailman.configuration import config
from mailman.i18n import _
from mailman.initialize import initialize_1, initialize_2
+from mailman.testing.helpers import SMTPServer
from mailman.version import MAILMAN_VERSION
@@ -226,10 +228,14 @@
with open(cfg_out, 'a') as fp:
print >> fp, 'VAR_DIR = "%s"' % var_dir
print >> fp, 'MAILMAN_USER = "%s"' % user_name
- print >> fp, 'MAILMAN_UID = %d' % user_id
+ print >> fp, 'MAILMAN_UID =', user_id
print >> fp, 'MAILMAN_GROUP = "%s"' % group_name
- print >> fp, 'MAILMAN_GID = %d' % group_id
+ print >> fp, 'MAILMAN_GID =', group_id
print >> fp, "LANGUAGES = 'en'"
+ print >> fp, 'SMTPPORT =', SMTPServer.port
+ # A fake MHonArc command, for testing.
+ print >> fp, 'MHONARC_COMMAND = """/bin/echo', \
+ Defaults.MHONARC_COMMAND, '"""'
initialize_1(cfg_out, propagate_logs=parser.options.stderr)
mailman_uid = pwd.getpwnam(config.MAILMAN_USER).pw_uid
=== modified file 'mailman/configuration.py'
--- a/mailman/configuration.py 2008-04-26 05:39:14 +0000
+++ b/mailman/configuration.py 2008-07-05 15:06:37 +0000
@@ -174,6 +174,7 @@
code = self.DEFAULT_SERVER_LANGUAGE
self.languages.enable_language(code)
# Create various registries.
+ self.archivers = {}
self.chains = {}
self.rules = {}
self.handlers = {}
=== modified file 'mailman/docs/archivers.txt'
--- a/mailman/docs/archivers.txt 2008-07-03 02:29:20 +0000
+++ b/mailman/docs/archivers.txt 2008-07-10 02:35:44 +0000
@@ -1,4 +1,5 @@
-= Archivers =
+Archivers
+=========
Mailman supports pluggable archivers, and it comes with several default
archivers.
@@ -23,15 +24,17 @@
Mailman defines a draft spec for how list servers and archivers can
interoperate.
- >>> from operator import attrgetter
- >>> name = attrgetter('name')
- >>> from mailman.app.plugins import get_plugins
- >>> archivers = {}
- >>> for archiver in sorted(get_plugins('mailman.archiver'), key=name):
+ >>> from mailman.configuration import config
+ >>> for archiver_name, archiver in sorted(config.archivers.items()):
... print archiver.name
... print ' ', archiver.list_url(mlist)
... print ' ', archiver.permalink(mlist, msg)
- ... archivers[archiver.name] = archiver
+ mail-archive
+ http://go.mail-archive.dev/test%40example.com
+ http://go.mail-archive.dev/ZaXPPxRMM9_hFZL4vTRlQlBx8pc=
+ mhonarc
+ http://www.example.com/.../[EMAIL PROTECTED]
+ http://www.example.com/.../RSZCG7IGPHFIRW3EMTVMMDNJMNCVCOLE
pipermail
http://www.example.com/pipermail/[EMAIL PROTECTED]
None
@@ -40,12 +43,13 @@
http://www.example.com/RSZCG7IGPHFIRW3EMTVMMDNJMNCVCOLE
-== Sending the message to the archiver ==
+Sending the message to the archiver
+-----------------------------------
The archiver is also able to archive the message.
>>> mlist.web_page_url = u'http://lists.example.com/'
- >>> archivers['pipermail'].archive_message(mlist, msg)
+ >>> config.archivers['pipermail'].archive_message(mlist, msg)
>>> import os
>>> from mailman.interfaces.archiver import IPipermailMailingList
@@ -57,7 +61,123 @@
Note however that the prototype archiver can't archive messages.
- >>> archivers['prototype'].archive_message(mlist, msg)
+ >>> config.archivers['prototype'].archive_message(mlist, msg)
Traceback (most recent call last):
...
NotImplementedError
+
+
+The Mail-Archive.com
+--------------------
+
+The Mail-Archive <http://www.mail-archive.com> is a public archiver that can
+be used to archive message for free. Mailman comes with a plugin for this
+archiver; by enabling it messages to public lists will get sent there
+automatically.
+
+ >>> archiver = config.archivers['mail-archive']
+ >>> archiver.list_url(mlist)
+ 'http://go.mail-archive.dev/test%40example.com'
+ >>> archiver.permalink(mlist, msg)
+ 'http://go.mail-archive.dev/ZaXPPxRMM9_hFZL4vTRlQlBx8pc='
+
+To archive the message, the archiver actually mails the message to a special
+address at the Mail-Archive.
+
+ >>> archiver.archive_message(mlist, msg)
+
+ >>> from mailman.queue.outgoing import OutgoingRunner
+ >>> from mailman.testing.helpers import make_testable_runner
+ >>> outgoing = make_testable_runner(OutgoingRunner)
+ >>> outgoing.run()
+
+ >>> from operator import itemgetter
+ >>> messages = list(smtpd.messages)
+ >>> len(messages)
+ 1
+
+ >>> print messages[0].as_string()
+ From: [EMAIL PROTECTED]
+ To: [EMAIL PROTECTED]
+ Subject: An archived message
+ Message-ID: <12345>
+ X-Message-ID-Hash: ZaXPPxRMM9_hFZL4vTRlQlBx8pc=
+ MIME-Version: 1.0
+ Content-Type: text/plain; charset="us-ascii"
+ Content-Transfer-Encoding: 7bit
+ Sender: [EMAIL PROTECTED]
+ Errors-To: [EMAIL PROTECTED]
+ X-Peer: 127.0.0.1:...
+ X-MailFrom: [EMAIL PROTECTED]
+ X-RcptTo: [EMAIL PROTECTED]
+ <BLANKLINE>
+ Here is an archived message.
+ _______________________________________________
+ Test mailing list
+ [EMAIL PROTECTED]
+ http://lists.example.com/listinfo/[EMAIL PROTECTED]
+
+ >>> smtpd.clear()
+
+However, if the mailing list is not public, the message will never be archived
+at this service.
+
+ >>> mlist.archive_private = True
+ >>> print archiver.list_url(mlist)
+ None
+ >>> print archiver.permalink(mlist, msg)
+ None
+ >>> archiver.archive_message(mlist, msg)
+ >>> list(smtpd.messages)
+ []
+
+Additionally, this archiver can handle malformed Message-IDs.
+
+ >>> mlist.archive_private = False
+ >>> del msg['message-id']
+ >>> msg['Message-ID'] = '12345>'
+ >>> archiver.permalink(mlist, msg)
+ 'http://go.mail-archive.dev/bXvG32YzcDEIVDaDLaUSVQekfo8='
+
+ >>> del msg['message-id']
+ >>> msg['Message-ID'] = '<12345'
+ >>> archiver.permalink(mlist, msg)
+ 'http://go.mail-archive.dev/9rockPrT1Mm-jOsLWS6_hseR_OY='
+
+ >>> del msg['message-id']
+ >>> msg['Message-ID'] = '12345'
+ >>> archiver.permalink(mlist, msg)
+ 'http://go.mail-archive.dev/ZaXPPxRMM9_hFZL4vTRlQlBx8pc='
+
+ >>> del msg['message-id']
+ >>> msg['Message-ID'] = ' 12345 '
+ >>> archiver.permalink(mlist, msg)
+ 'http://go.mail-archive.dev/ZaXPPxRMM9_hFZL4vTRlQlBx8pc='
+
+
+MHonArc
+-------
+
+The MHonArc archiver <http://www.mhonarc.org> is also available.
+
+ >>> archiver = config.archivers['mhonarc']
+ >>> archiver.name
+ 'mhonarc'
+
+Messages sent to a local MHonArc instance are added to its archive via a
+subprocess call.
+
+ >>> archiver.archive_message(mlist, msg)
+ >>> archive_log = open(os.path.join(config.LOG_DIR, 'archiver'))
+ >>> try:
+ ... contents = archive_log.read()
+ ... finally:
+ ... archive_log.close()
+ >>> print 'LOG:', contents
+ LOG: ... /usr/bin/mhonarc -add
+ -dbfile /.../private/[EMAIL PROTECTED]/mhonarc.db
+ -outdir /.../mhonarc/[EMAIL PROTECTED]
+ -stderr /.../logs/mhonarc
+ -stdout /.../logs/mhonarc
+ -spammode -umask 022
+ ...
=== modified file 'mailman/docs/pipelines.txt'
--- a/mailman/docs/pipelines.txt 2008-07-03 19:32:25 +0000
+++ b/mailman/docs/pipelines.txt 2008-07-05 15:06:37 +0000
@@ -21,6 +21,8 @@
Messages hit the pipeline after they've been accepted for posting.
+ >>> from mailman.configuration import config
+ >>> config.archivers['pipermail'].is_enabled = True
>>> msg = message_from_string("""\
... From: [EMAIL PROTECTED]
... To: [EMAIL PROTECTED]
@@ -69,7 +71,6 @@
And the message is now sitting in various other processing queues.
>>> from mailman.testing.helpers import get_queue_messages
- >>> from mailman.configuration import config
>>> messages = get_queue_messages(config.ARCHQUEUE_DIR)
>>> len(messages)
1
=== modified file 'mailman/initialize.py'
--- a/mailman/initialize.py 2008-04-26 05:39:14 +0000
+++ b/mailman/initialize.py 2008-07-05 15:06:37 +0000
@@ -65,10 +65,12 @@
mailman.configuration.config.db = database
# Initialize the rules and chains. Do the imports here so as to avoid
# circular imports.
+ from mailman.archiving import initialize as initialize_archivers
from mailman.app.chains import initialize as initialize_chains
from mailman.app.rules import initialize as initialize_rules
from mailman.app.pipelines import initialize as initialize_pipelines
from mailman.app.commands import initialize as initialize_commands
+ initialize_archivers()
initialize_rules()
initialize_chains()
initialize_pipelines()
=== modified file 'mailman/interfaces/mailinglist.py'
--- a/mailman/interfaces/mailinglist.py 2008-03-27 09:14:14 +0000
+++ b/mailman/interfaces/mailinglist.py 2008-07-05 14:41:12 +0000
@@ -147,7 +147,7 @@
last_post_date = Attribute(
"""The date and time a message was last posted to the mailing list.""")
- post_number = Attribute(
+ post_id = Attribute(
"""A monotonically increasing integer sequentially assigned to each
list posting.""")
=== modified file 'mailman/loginit.py'
--- a/mailman/loginit.py 2008-03-12 22:35:16 +0000
+++ b/mailman/loginit.py 2008-07-10 02:35:44 +0000
@@ -35,6 +35,7 @@
DATEFMT = '%b %d %H:%M:%S %Y'
LOGGERS = (
+ 'archiver', # All archiver output
'bounce', # All bounce processing logs go here
'config', # Configuration issues
'debug', # Only used for development
=== modified file 'mailman/pipeline/docs/cook-headers.txt'
--- a/mailman/pipeline/docs/cook-headers.txt 2008-07-03 02:29:20 +0000
+++ b/mailman/pipeline/docs/cook-headers.txt 2008-07-05 15:06:37 +0000
@@ -184,6 +184,7 @@
>>> mlist.include_rfc2369_headers = True
>>> mlist.include_list_post_header = True
>>> mlist.preferred_language = u'en'
+ >>> config.archivers['pipermail'].is_enabled = True
>>> msg = message_from_string("""\
... From: [EMAIL PROTECTED]
... Message-ID: <12345>
=== modified file 'mailman/queue/docs/outgoing.txt'
--- a/mailman/queue/docs/outgoing.txt 2008-07-03 19:32:25 +0000
+++ b/mailman/queue/docs/outgoing.txt 2008-07-06 14:27:05 +0000
@@ -26,21 +26,12 @@
... u'password', DeliveryMode.regular, u'en',
... ack=False, admin_notif=False)
- >>> from mailman.testing.helpers import SMTPServer
- >>> smtpd = SMTPServer()
- >>> smtpd.start()
- >>> from mailman.configuration import config
- >>> old_host = config.SMTPHOST
- >>> old_port = config.SMTPPORT
- >>> config.SMTPHOST = smtpd.host
- >>> config.SMTPPORT = smtpd.port
-
By setting the mailing list to personalize messages, each recipient will get a
unique copy of the message, with certain headers tailored for that recipient.
>>> from mailman.interfaces import Personalization
>>> mlist.personalize = Personalization.individual
- >>> config.db.commit()
+ >>> commit()
>>> msg = message_from_string("""\
... From: [EMAIL PROTECTED]
@@ -55,6 +46,7 @@
been processed by the rule set and pipeline. But we can simulate that here by
injecting a message directly into the outgoing queue.
+ >>> from mailman.configuration import config
>>> msgdata = {}
>>> handler = config.handlers['calculate-recipients']
>>> handler.process(mlist, msg, msgdata)
@@ -86,11 +78,3 @@
[EMAIL PROTECTED]
[EMAIL PROTECTED]
[EMAIL PROTECTED]
-
-
-Clean up
---------
-
- >>> smtpd.stop()
- >>> config.SMTPHOST = old_host
- >>> config.SMTPPORT = old_port
=== modified file 'mailman/testing/helpers.py'
--- a/mailman/testing/helpers.py 2008-07-03 19:32:25 +0000
+++ b/mailman/testing/helpers.py 2008-07-05 14:41:12 +0000
@@ -162,7 +162,7 @@
"""An smtp server for testing."""
host = 'localhost'
- port = 9025
+ port = 10825
def __init__(self):
self._messages = []
@@ -174,6 +174,10 @@
"""Start the smtp server in a thread."""
log.info('test SMTP server starting')
self._thread.start()
+ smtpd = smtplib.SMTP()
+ smtpd.connect(self.host, self.port)
+ smtpd.helo('test.localhost')
+ smtpd.quit()
def stop(self):
"""Stop the smtp server."""
=== modified file 'mailman/testing/smtplistener.py'
--- a/mailman/testing/smtplistener.py 2008-07-03 19:32:25 +0000
+++ b/mailman/testing/smtplistener.py 2008-07-05 14:41:12 +0000
@@ -58,12 +58,13 @@
def __init__(self, localaddr, queue):
smtpd.SMTPServer.__init__(self, localaddr, None)
+ log.info('[SMTPServer] listening: %s', localaddr)
self._queue = queue
def handle_accept(self):
"""Handle connections by creating our own Channel object."""
conn, addr = self.accept()
- log.info('accepted: %s', addr)
+ log.info('[SMTPServer] accepted: %s', addr)
Channel(self, conn, addr)
def process_message(self, peer, mailfrom, rcpttos, data):
@@ -72,7 +73,8 @@
message['X-Peer'] = '%s:%s' % peer
message['X-MailFrom'] = mailfrom
message['X-RcptTo'] = COMMASPACE.join(rcpttos)
- log.info('processed message: %s', message.get('message-id', 'n/a'))
+ log.info('[SMTPServer] processed message: %s',
+ message.get('message-id', 'n/a'))
self._queue.put(message)
def start(self):
=== modified file 'mailman/testing/testing.cfg.in'
--- a/mailman/testing/testing.cfg.in 2008-07-03 19:32:25 +0000
+++ b/mailman/testing/testing.cfg.in 2008-07-05 14:41:12 +0000
@@ -4,11 +4,14 @@
# both the process running the tests and all sub-processes (e.g. qrunners)
# must share the same configuration file.
-SMTPPORT = 10825
MAX_RESTARTS = 1
MTA = None
USE_LMTP = Yes
+# Make sure these goes to fake domains.
+MAIL_ARCHIVE_BASEURL = 'http://go.mail-archive.dev/'
+MAIL_ARCHIVE_RECIPIENT = '[EMAIL PROTECTED]'
+
add_domain('example.com', 'www.example.com')
# bin/testall will add additional runtime configuration variables here.
=== modified file 'mailman/tests/test_documentation.py'
--- a/mailman/tests/test_documentation.py 2008-03-13 01:52:43 +0000
+++ b/mailman/tests/test_documentation.py 2008-07-06 14:27:05 +0000
@@ -29,6 +29,7 @@
from mailman.Message import Message
from mailman.app.styles import style_manager
from mailman.configuration import config
+from mailman.testing.helpers import SMTPServer
DOT = '.'
@@ -54,12 +55,15 @@
def setup(testobj):
"""Test setup."""
+ smtpd = SMTPServer()
+ smtpd.start()
# In general, I don't like adding convenience functions, since I think
# doctests should do the imports themselves. It makes for better
# documentation that way. However, a few are really useful, or help to
# hide some icky test implementation details.
testobj.globs['message_from_string'] = specialized_message_from_string
testobj.globs['commit'] = config.db.commit
+ testobj.globs['smtpd'] = smtpd
@@ -79,6 +83,13 @@
for message in config.db.message_store.messages:
config.db.message_store.delete_message(message['message-id'])
config.db.commit()
+ # Reset all archivers by disabling them.
+ for archiver in config.archivers.values():
+ archiver.is_enabled = False
+ # Shutdown the smtp server.
+ smtpd = testobj.globs['smtpd']
+ smtpd.clear()
+ smtpd.stop()
=== modified file 'setup.py'
--- a/setup.py 2008-07-03 02:29:20 +0000
+++ b/setup.py 2008-07-10 02:35:44 +0000
@@ -91,10 +91,12 @@
'console_scripts': list(scripts),
# Entry point for plugging in different database backends.
'mailman.archiver' : [
- 'pipermail = mailman.app.archiving:Pipermail',
- 'prototype = mailman.app.archiving:Prototype',
+ 'mail-archive = mailman.archiving.mailarchive:MailArchive',
+ 'mhonarc = mailman.archiving.mhonarc:MHonArc',
+ 'pipermail = mailman.archiving.pipermail:Pipermail',
+ 'prototype = mailman.archiving.prototype:Prototype',
],
- 'mailman.scrubber' : 'stock = mailman.app.archiving:Pipermail',
+ 'mailman.scrubber' : 'stock = mailman.archiving.pipermail:Pipermail',
'mailman.commands' : list(commands),
'mailman.database' : 'stock = mailman.database:StockDatabase',
'mailman.mta' : 'stock = mailman.MTA:Manual',
--
Primary development focus
https://code.launchpad.net/~mailman-coders/mailman/3.0
You are receiving this branch notification because you are subscribed to it.
_______________________________________________
Mailman-checkins mailing list
[email protected]
Unsubscribe:
http://mail.python.org/mailman/options/mailman-checkins/archive%40jab.org
