Revision: 8219
http://svn.sourceforge.net/mailman/?rev=8219&view=rev
Author: bwarsaw
Date: 2007-05-17 06:17:32 -0700 (Thu, 17 May 2007)
Log Message:
-----------
I finally cracked a perplexing Elixir mystery: the kind_of keyword in a
Relationship (e.g. has_and_belongs_to_many()) names the full module path to
the Entity class that the relation is with. That makes sense but it's not
documented (I found an article in the mailing list archives with the clue).
The other trick is that you have to defer Elixir setup until after all
Entities have been loaded, because otherwise the relationships will try to
find the kind_of classes immediately. This will fail because the modules
won't all be loaded at that time. So we set elixir.delay_setup = True and
then explicitly call elixir.setup_all() when we know all Entities have been
imported.
This is a big breakthrough because now Addresses and Rosters work, as
evidenced by the new doctest cases.
Specific changes in this revision:
- New exception, ExistingAddressError to conform to IRoster interface. While
this inherits from RosterError, I'm not sure that's what we want long-term.
- Update the IRoster interface. Rosters now manage addresses, not members.
We'll be able to trace back from addresses to members in a different way.
There's also no need for an add() and remove() method, since IAddresses can
be added to the 'addresses' attribute directly (via append() and remove()).
create() method is added to return an IAddress instance, essentially making
IRoster both the factory and the manager.
- Update the IUser interface to remove the user_manager attribute. I don't
think this will be necessary.
- Update the IAddress interface to remove the 'user' and 'profile' attributes.
Now IAddresses just represent addresses. This is because addresses and
rosters will be used to track information about addresses which are not tied
to users (think, non-member postings). An IAddress now has just the email
address, real_name attributes, as well as the dates the IAddress was
registered (i.e. it became known to us) and the date it was validated on.
This latter may be None, representing a non-validated email address.
- Fleshed out the Elixir entities implementing IRoster and IAddress.
- Rework database initialization. I can basically get rid of the DBContext
now I think since we'll just use Elixir constructs to initialize the
database. The reason I'm not removing this totally yet is because I haven't
tested whether concurrency still works yet.
- Remove Mailman/database/{rosters,profiles}.py. These live in the (hopefully
soon to be deleted) Mailman/database/tables directory.
Modified Paths:
--------------
branches/exp-elixir-branch/Mailman/Errors.py
branches/exp-elixir-branch/Mailman/database/__init__.py
branches/exp-elixir-branch/Mailman/database/dbcontext.py
branches/exp-elixir-branch/Mailman/database/model/__init__.py
branches/exp-elixir-branch/Mailman/database/model/address.py
branches/exp-elixir-branch/Mailman/database/model/roster.py
branches/exp-elixir-branch/Mailman/interfaces/address.py
branches/exp-elixir-branch/Mailman/interfaces/roster.py
branches/exp-elixir-branch/Mailman/interfaces/user.py
Added Paths:
-----------
branches/exp-elixir-branch/Mailman/docs/addresses.txt
Removed Paths:
-------------
branches/exp-elixir-branch/Mailman/database/profiles.py
branches/exp-elixir-branch/Mailman/database/rosters.py
Modified: branches/exp-elixir-branch/Mailman/Errors.py
===================================================================
--- branches/exp-elixir-branch/Mailman/Errors.py 2007-05-12 01:38:52 UTC
(rev 8218)
+++ branches/exp-elixir-branch/Mailman/Errors.py 2007-05-17 13:17:32 UTC
(rev 8219)
@@ -230,3 +230,7 @@
class NoSuchRosterError(RosterError):
"""The named roster does not exist."""
+
+class ExistingAddressError(RosterError):
+ """The given email address already exists."""
+
Modified: branches/exp-elixir-branch/Mailman/database/__init__.py
===================================================================
--- branches/exp-elixir-branch/Mailman/database/__init__.py 2007-05-12
01:38:52 UTC (rev 8218)
+++ branches/exp-elixir-branch/Mailman/database/__init__.py 2007-05-17
13:17:32 UTC (rev 8219)
@@ -27,12 +27,11 @@
def initialize():
from Mailman.LockFile import LockFile
from Mailman.configuration import config
- from Mailman.database.dbcontext import DBContext
+ from Mailman.database import model
# Serialize this so we don't get multiple processes trying to create the
# database at the same time.
lockfile = os.path.join(config.LOCK_DIR, '<dbcreatelock>')
- dbcontext = DBContext()
with LockFile(lockfile):
- dbcontext.connect()
+ model.initialize()
config.list_manager = ListManager()
config.user_manager = UserManager()
Modified: branches/exp-elixir-branch/Mailman/database/dbcontext.py
===================================================================
--- branches/exp-elixir-branch/Mailman/database/dbcontext.py 2007-05-12
01:38:52 UTC (rev 8218)
+++ branches/exp-elixir-branch/Mailman/database/dbcontext.py 2007-05-17
13:17:32 UTC (rev 8219)
@@ -21,6 +21,7 @@
import weakref
from elixir import create_all, metadata, objectstore
+from sqlalchemy import create_engine
from string import Template
from urlparse import urlparse
@@ -62,8 +63,9 @@
# engines, and yes, we could have chmod'd the file after the fact, but
# half dozen and all...
self._touch(url)
- metadata.connect(url)
- metadata.engine.echo = config.SQLALCHEMY_ECHO
+ engine = create_engine(url)
+ engine.echo = config.SQLALCHEMY_ECHO
+ metadata.connect(engine)
# Load and create the Elixir active records. This works by
# side-effect.
import Mailman.database.model
Modified: branches/exp-elixir-branch/Mailman/database/model/__init__.py
===================================================================
--- branches/exp-elixir-branch/Mailman/database/model/__init__.py
2007-05-12 01:38:52 UTC (rev 8218)
+++ branches/exp-elixir-branch/Mailman/database/model/__init__.py
2007-05-17 13:17:32 UTC (rev 8219)
@@ -15,9 +15,6 @@
# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301,
# USA.
-import os
-import sys
-
__all__ = [
'Address',
'Language',
@@ -26,9 +23,66 @@
'Version',
]
+import os
+import sys
+import elixir
+from sqlalchemy import create_engine
+from string import Template
+from urlparse import urlparse
+
+import Mailman.Version
+
+elixir.delay_setup = True
+
+from Mailman.Errors import SchemaVersionMismatchError
+from Mailman.configuration import config
from Mailman.database.model.address import Address
from Mailman.database.model.language import Language
from Mailman.database.model.mailinglist import MailingList
from Mailman.database.model.roster import Roster
from Mailman.database.model.version import Version
+
+
+def initialize():
+ # Calculate the engine url
+ url = Template(config.SQLALCHEMY_ENGINE_URL).safe_substitute(config.paths)
+ # XXX By design of SQLite, database file creation does not honor
+ # umask. See their ticket #1193:
+ # http://www.sqlite.org/cvstrac/tktview?tn=1193,31
+ #
+ # This sucks for us because the mailman.db file /must/ be group writable,
+ # however even though we guarantee our umask is 002 here, it still gets
+ # created without the necessary g+w permission, due to SQLite's policy.
+ # This should only affect SQLite engines because its the only one that
+ # creates a little file on the local file system. This kludges around
+ # their bug by "touch"ing the database file before SQLite has any chance
+ # to create it, thus honoring the umask and ensuring the right
+ # permissions. We only try to do this for SQLite engines, and yes, we
+ # could have chmod'd the file after the fact, but half dozen and all...
+ touch(url)
+ engine = create_engine(url)
+ engine.echo = config.SQLALCHEMY_ECHO
+ elixir.metadata.connect(engine)
+ elixir.setup_all()
+ # Validate schema version.
+ v = Version.get_by(component='schema')
+ if not v:
+ # Database has not yet been initialized
+ v = Version(component='schema',
+ version=Mailman.Version.DATABASE_SCHEMA_VERSION)
+ elixir.objectstore.flush()
+ elif v.version <> Mailman.Version.DATABASE_SCHEMA_VERSION:
+ # XXX Update schema
+ raise SchemaVersionMismatchError(v.version)
+
+
+def touch(url):
+ parts = urlparse(url)
+ if parts.scheme <> 'sqlite':
+ return
+ path = os.path.normpath(parts.path)
+ fd = os.open(path, os.O_WRONLY | os.O_NONBLOCK | os.O_CREAT, 0666)
+ # Ignore errors
+ if fd > 0:
+ os.close(fd)
Modified: branches/exp-elixir-branch/Mailman/database/model/address.py
===================================================================
--- branches/exp-elixir-branch/Mailman/database/model/address.py
2007-05-12 01:38:52 UTC (rev 8218)
+++ branches/exp-elixir-branch/Mailman/database/model/address.py
2007-05-17 13:17:32 UTC (rev 8219)
@@ -16,13 +16,27 @@
# USA.
from elixir import *
+from email.utils import formataddr
+from zope.interface import implements
+from Mailman.interfaces import IAddress
+
class Address(Entity):
- with_fields(
- address = Field(Unicode),
- verified = Field(Boolean),
- bounce_info = Field(PickleType),
- )
+ implements(IAddress)
+
+ has_field('address', Unicode)
+ has_field('real_name', Unicode)
+ has_field('verified', Boolean)
+ has_field('registered_on', DateTime)
+ has_field('validated_on', DateTime)
# Relationships
- has_and_belongs_to_many('rosters', of_kind='Roster')
+ has_and_belongs_to_many('rosters',
+ of_kind='Mailman.database.model.roster.Roster')
+
+ def __str__(self):
+ return formataddr((self.real_name, self.address))
+
+ def __repr__(self):
+ return '<Address: %s [%s]>' % (
+ str(self), ('verified' if self.verified else 'not verified'))
Modified: branches/exp-elixir-branch/Mailman/database/model/roster.py
===================================================================
--- branches/exp-elixir-branch/Mailman/database/model/roster.py 2007-05-12
01:38:52 UTC (rev 8218)
+++ branches/exp-elixir-branch/Mailman/database/model/roster.py 2007-05-17
13:17:32 UTC (rev 8219)
@@ -18,15 +18,30 @@
from elixir import *
from zope.interface import implements
+from Mailman.Errors import ExistingAddressError
from Mailman.interfaces import IRoster
class Roster(Entity):
implements(IRoster)
- with_fields(
- name = Field(Unicode),
- )
+ has_field('name', Unicode)
+ # Relationships
+ has_and_belongs_to_many('addresses',
+ of_kind='Mailman.database.model.address.Address')
- # Relationships
- has_and_belongs_to_many('addresses', of_kind='Address')
+ def create(self, email_address, real_name=None):
+ """See IRoster"""
+ from Mailman.database.model.address import Address
+ addr = Address.get_by(address=email_address)
+ if addr:
+ raise ExistingAddressError(email_address)
+ addr = Address(address=email_address, real_name=real_name)
+ # Make sure all the expected links are made, including to the null
+ # (i.e. everyone) roster.
+ self.addresses.append(addr)
+ addr.rosters.append(self)
+ null_roster = Roster.get_by(name='')
+ null_roster.addresses.append(addr)
+ addr.rosters.append(null_roster)
+ return addr
Deleted: branches/exp-elixir-branch/Mailman/database/profiles.py
===================================================================
--- branches/exp-elixir-branch/Mailman/database/profiles.py 2007-05-12
01:38:52 UTC (rev 8218)
+++ branches/exp-elixir-branch/Mailman/database/profiles.py 2007-05-17
13:17:32 UTC (rev 8219)
@@ -1,77 +0,0 @@
-# Copyright (C) 2007 by the Free Software Foundation, Inc.
-#
-# This program is free software; you can redistribute it and/or
-# modify it under the terms of the GNU General Public License
-# as published by the Free Software Foundation; either version 2
-# of the License, or (at your option) any later version.
-#
-# This program is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-# GNU General Public License for more details.
-#
-# You should have received a copy of the GNU General Public License
-# along with this program; if not, write to the Free Software
-# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301,
-# USA.
-
-"""Mailman user profile information."""
-
-from sqlalchemy import *
-
-from Mailman import Defaults
-
-
-
-class Profile(object):
- pass
-
-
-
-# Both of these Enum types are stored in the database as integers, and
-# converted back into their enums on retrieval.
-
-class DeliveryModeType(types.TypeDecorator):
- impl = types.Integer
-
- def convert_bind_param(self, value, engine):
- return int(value)
-
- def convert_result_value(self, value, engine):
- return Defaults.DeliveryMode(value)
-
-
-class DeliveryStatusType(types.TypeDecorator):
- impl = types.Integer
-
- def convert_bind_param(self, value, engine):
- return int(value)
-
- def convert_result_value(self, value, engine):
- return Defaults.DeliveryStatus(value)
-
-
-
-def make_table(metadata, tables):
- table = Table(
- 'Profiles', metadata,
- Column('profile_id', Integer, primary_key=True),
- # OldStyleMemberships attributes, temporarily stored as pickles.
- Column('ack', Boolean),
- Column('delivery_mode', DeliveryModeType),
- Column('delivery_status', DeliveryStatusType),
- Column('hide', Boolean),
- Column('language', Unicode),
- Column('nodupes', Boolean),
- Column('nomail', Boolean),
- Column('notmetoo', Boolean),
- Column('password', Unicode),
- Column('realname', Unicode),
- Column('topics', PickleType),
- )
- # Avoid circular references
- from Mailman.database.address import Address
- # profile -> address*
- props = dict(addresses=relation(Address, cascade='all, delete-orphan'))
- mapper(Profile, table, properties=props)
- tables.bind(table)
Deleted: branches/exp-elixir-branch/Mailman/database/rosters.py
===================================================================
--- branches/exp-elixir-branch/Mailman/database/rosters.py 2007-05-12
01:38:52 UTC (rev 8218)
+++ branches/exp-elixir-branch/Mailman/database/rosters.py 2007-05-17
13:17:32 UTC (rev 8219)
@@ -1,59 +0,0 @@
-# Copyright (C) 2007 by the Free Software Foundation, Inc.
-#
-# This program is free software; you can redistribute it and/or
-# modify it under the terms of the GNU General Public License
-# as published by the Free Software Foundation; either version 2
-# of the License, or (at your option) any later version.
-#
-# This program is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-# GNU General Public License for more details.
-#
-# You should have received a copy of the GNU General Public License
-# along with this program; if not, write to the Free Software
-# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301,
-# USA.
-
-"""Collections of email addresses.
-
-Rosters contain email addresses. RosterSets contain Rosters. Most attributes
-on the listdata table take RosterSets so that it's easy to compose just about
-any combination of addresses.
-"""
-
-from sqlalchemy import *
-
-from Mailman.database.address import Address
-
-
-
-class Roster(object):
- pass
-
-
-class RosterSet(object):
- pass
-
-
-
-def make_table(metadata, tables):
- table = Table(
- 'Rosters', metadata,
- Column('roster_id', Integer, primary_key=True),
- )
- # roster* <-> address*
- props = dict(addresses=
- relation(Address,
- secondary=tables.address_rosters,
- lazy=False))
- mapper(Roster, table, properties=props)
- tables.bind(table)
- table = Table(
- 'RosterSets', metadata,
- Column('rosterset_id', Integer, primary_key=True),
- )
- # rosterset -> roster*
- props = dict(rosters=relation(Roster, cascade='all, delete=orphan'))
- mapper(RosterSet, table, properties=props)
- tables.bind(table)
Added: branches/exp-elixir-branch/Mailman/docs/addresses.txt
===================================================================
--- branches/exp-elixir-branch/Mailman/docs/addresses.txt
(rev 0)
+++ branches/exp-elixir-branch/Mailman/docs/addresses.txt 2007-05-17
13:17:32 UTC (rev 8219)
@@ -0,0 +1,140 @@
+Email addresses and rosters
+===========================
+
+Addresses represent email address, and nothing more. Some addresses are tied
+to users that Mailman knows about. For example, a list member is a user that
+the system knows about, but a non-member posting from a brand new email
+address is a counter-example.
+
+
+Creating a roster
+-----------------
+
+Email address objects are tied to rosters, and rosters are tied to the user
+manager. To get things started, access the global user manager and create a
+new roster.
+
+ >>> from Mailman.configuration import config
+ >>> mgr = config.user_manager
+ >>> roster_1 = mgr.create_roster('roster-1')
+ >>> sorted(roster_1.addresses)
+ []
+
+
+Creating addresses
+------------------
+
+Creating a simple email address object is straight forward.
+
+ >>> addr_1 = roster_1.create('[EMAIL PROTECTED]')
+ >>> addr_1.address
+ '[EMAIL PROTECTED]'
+ >>> addr_1.real_name is None
+ True
+
+You can also create an email address object with a real name.
+
+ >>> addr_2 = roster_1.create('[EMAIL PROTECTED]', 'Barney Person')
+ >>> addr_2.address
+ '[EMAIL PROTECTED]'
+ >>> addr_2.real_name
+ 'Barney Person'
+
+You can also iterate through all the addresses on a roster.
+
+ >>> sorted(addr.address for addr in roster_1.addresses)
+ ['[EMAIL PROTECTED]', '[EMAIL PROTECTED]']
+
+You can create another roster and add a bunch of existing addresses to the
+second roster.
+
+ >>> roster_2 = mgr.create_roster('roster-2')
+ >>> sorted(roster_2.addresses)
+ []
+ >>> for address in roster_1.addresses:
+ ... roster_2.addresses.append(address)
+ >>> roster_2.create('[EMAIL PROTECTED]', 'Charlie Person')
+ <Address: Charlie Person <[EMAIL PROTECTED]> [not verified]>
+ >>> sorted(addr.address for addr in roster_2.addresses)
+ ['[EMAIL PROTECTED]', '[EMAIL PROTECTED]', '[EMAIL PROTECTED]']
+
+The first roster hasn't been affected.
+
+ >>> sorted(addr.address for addr in roster_1.addresses)
+ ['[EMAIL PROTECTED]', '[EMAIL PROTECTED]']
+
+
+Removing addresses
+------------------
+
+You can remove an address from a roster just by deleting it.
+
+ >>> for addr in roster_1.addresses:
+ ... if addr.address == '[EMAIL PROTECTED]':
+ ... break
+ >>> addr.address
+ '[EMAIL PROTECTED]'
+ >>> roster_1.addresses.remove(addr)
+ >>> sorted(addr.address for addr in roster_1.addresses)
+ ['[EMAIL PROTECTED]']
+
+Again, this doesn't affect the other rosters.
+
+ >>> sorted(addr.address for addr in roster_2.addresses)
+ ['[EMAIL PROTECTED]', '[EMAIL PROTECTED]', '[EMAIL PROTECTED]']
+
+
+Registration and validation
+---------------------------
+
+Addresses have two dates, the date the address was registered on and the date
+the address was validated on. Neither date isset by default.
+
+ >>> addr = roster_1.create('[EMAIL PROTECTED]', 'David Person')
+ >>> addr.registered_on is None
+ True
+ >>> addr.validated_on is None
+ True
+
+The registered date takes a Python datetime object.
+
+ >>> from datetime import datetime
+ >>> addr.registered_on = datetime(2007, 5, 8, 22, 54, 1)
+ >>> print addr.registered_on
+ 2007-05-08 22:54:01
+ >>> addr.validated_on is None
+ True
+
+And of course, you can also set the validation date.
+
+ >>> addr.validated_on = datetime(2007, 5, 13, 22, 54, 1)
+ >>> print addr.registered_on
+ 2007-05-08 22:54:01
+ >>> print addr.validated_on
+ 2007-05-13 22:54:01
+
+
+The null roster
+---------------
+
+All address objects that have been created are members of the null roster.
+
+ >>> all = mgr.get_roster('')
+ >>> sorted(addr.address for addr in all.addresses)
+ ['[EMAIL PROTECTED]', '[EMAIL PROTECTED]',
+ '[EMAIL PROTECTED]', '[EMAIL PROTECTED]']
+
+And conversely, all addresses should have the null roster on their list of
+rosters.
+
+ >>> for addr in all.addresses:
+ ... assert all in addr.rosters, 'Address is missing null roster'
+
+
+Clean up
+--------
+
+ >>> for roster in mgr.rosters:
+ ... mgr.delete_roster(roster)
+ >>> list(mgr.rosters)
+ []
Modified: branches/exp-elixir-branch/Mailman/interfaces/address.py
===================================================================
--- branches/exp-elixir-branch/Mailman/interfaces/address.py 2007-05-12
01:38:52 UTC (rev 8218)
+++ branches/exp-elixir-branch/Mailman/interfaces/address.py 2007-05-17
13:17:32 UTC (rev 8219)
@@ -24,16 +24,21 @@
class IAddress(Interface):
"""Email address related information."""
- user = Attribute(
- """Read-only IUser owning this email address.""")
-
address = Attribute(
"""Read-only text email address.""")
+ real_name = Attribute(
+ """Optional real name associated with the email address.""")
+
+ registered_on = Attribute(
+ """The date and time at which this email address was registered.
+
+ Registeration is really the date at which this address became known to
+ us. It may have been explicitly registered by a user, or it may have
+ been implicitly registered, e.g. by showing up in a non-member
+ posting.""")
+
validated_on = Attribute(
"""The date and time at which this email address was validated, or
None if the email address has not yet been validated. The specific
method of validation is not defined here.""")
-
- profile = Attribute(
- """The profile for this address.""")
Modified: branches/exp-elixir-branch/Mailman/interfaces/roster.py
===================================================================
--- branches/exp-elixir-branch/Mailman/interfaces/roster.py 2007-05-12
01:38:52 UTC (rev 8218)
+++ branches/exp-elixir-branch/Mailman/interfaces/roster.py 2007-05-17
13:17:32 UTC (rev 8219)
@@ -25,14 +25,18 @@
"""A roster is a collection of IMembers."""
name = Attribute(
- """Read-only, required name for this roster. Rosters are considered
- equal if they have the same roster name.""")
+ """The name for this roster.
- members = Attribute(
- """An iterator of the IMembers of this roster.""")
+ Rosters are considered equal if they have the same name.""")
- def add(member):
- """Add the IMember to the roster."""
+ addresses = Attribute(
+ """An iterator over all the addresses managed by this roster.""")
- def remove(member):
- """Remove the IMember from the roster."""
+ def create(email_address, real_name=None):
+ """Create an IAddress and return it.
+
+ email_address is textual email address to add. real_name is the
+ optional real name that gets associated with the email address.
+
+ Raises ExistingAddressError if address already exists.
+ """
Modified: branches/exp-elixir-branch/Mailman/interfaces/user.py
===================================================================
--- branches/exp-elixir-branch/Mailman/interfaces/user.py 2007-05-12
01:38:52 UTC (rev 8218)
+++ branches/exp-elixir-branch/Mailman/interfaces/user.py 2007-05-17
13:17:32 UTC (rev 8219)
@@ -57,6 +57,3 @@
"""Return True if this user controls the given email address,
otherwise False. address must be a text email address
"""
-
- user_manager = Attribute(
- """Read-only IUserManager this user is being managed by.""")
This was sent by the SourceForge.net collaborative development platform, the
world's largest Open Source development site.
_______________________________________________
Mailman-checkins mailing list
[email protected]
Unsubscribe:
http://mail.python.org/mailman/options/mailman-checkins/archive%40jab.org
