Re: [Mailman-Developers] GSoC 15 - Interested in contributing to Hyperkitty

2015-03-24 Thread Aurelien Bompard
 I'm interested in contributing to the Hyperkitty archiver. Specifically,
 it looks like some requested features for Hyperkitty include rss syndication
 for entire mailing lists/specific users/specific threads, and the ability
 to view entire threads as plaintext and download that plaintext.

We don't have either feature yet, but I think RSS syndication would be
much too short for a GSOC project.
There's always been demand for a way to download a list archive as an
mbox file, and it's reasonably complicated if you want to avoid
blocking the UI, so this may be a good project similar to the
plaintext threads feature you mention.

Aurélien
___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/archive%40jab.org

Security Policy: http://wiki.list.org/x/QIA9

Re: [Mailman-Developers] Common use case archiving via configuration

2015-03-24 Thread Aurelien Bompard
 Would you mind if I used the Hyperkitty POST code as the basis for an example 
 POST archiver for Mailman?

Sure, no problem.
Aurélien
___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/archive%40jab.org

Security Policy: http://wiki.list.org/x/QIA9

Re: [Mailman-Developers] Getting approval to go ahead

2015-03-24 Thread Barry Warsaw
On Mar 24, 2015, at 02:38 PM, Andrew Stuart wrote:

I haven’t really worked on an open source project before.

You're fitting right in though! :)

It wouldn’t make sense to come up with an idea, write some code, submit it
and have it rejected because it’s not OK with the project owners and doesn’t
fit with project goals.

I think Terri summarized things well, so I'll just add a few things as it
relates to core.

A Launchpad bug is always a good place to augment a mailing list discussion.
You can hang branches and merge proposals off of bugs, and we can have a
conversation about the details of either the idea (on the bug) or the
implementation (on the merge proposal).

Always tag the bug 'mailman3' for Mailman 3.  It helps me find them better.  I
can then triage the bug, assign it to a milestone, raise or lower the
priority, etc.

Let's say you have a crazy idea for a new feature.  Bring it up here and let's
start the discussion going.  If it seems like a good idea, and aligned with
the project goals, create a bug and start hacking.  Attach your branch to the
bug.  When you think it's ready for review, create a merge proposal against
trunk (i.e. lp:mailman).

Some things that I look for right away: conformance to PEP 8 and our own style
guide.  I don't expect branches to be 100% conformant to style, but I think
it's not hard to get close if you follow the guides and try to look like
existing code.

Doctests that explain the feature and unit tests that cover the new code.
Doctests are documentation first, so are expected to explain good-path
behavior and usage.  Unit tests cover corner cases, error conditions, etc.
Tests should be included at each appropriate level (e.g. model, REST).  The
tests should fail without your fix and pass with them applied.  There should
be no regressions in the rest of the test suite.

Keep feature and bug branches small if possible, and concise, such that they
only implement the feature your working on or fix the reported bug.  A little
bit of extraneous stuff might be okay if it improves readability, but don't go
overboard.

That's probably the basics.  I promise I will try to be kind, fair, and
thankful for your contributions, though I might be rather strict in my
opinions.  If you really disagree, let's discuss respectfully!

Cheers,
-Barry
___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/archive%40jab.org

Security Policy: http://wiki.list.org/x/QIA9

Re: [Mailman-Developers] Common use case archiving via configuration

2015-03-24 Thread Barry Warsaw
On Mar 24, 2015, at 05:01 PM, Andrew Stuart wrote:

@barry - would you mind confirming please you’re OK with this please?

These are all pretty interesting.  We'd have to think about whether we'd want
some, any, or all them in the main tree, which essentially means we're
committing to supporting them.

Alternatively, we can create a contrib directory or repo somewhere where
semi-official additions like this could be managed.  The idea being that folks
could grab these, add a little configuration magic or drop them in the right
file system location, and they'd get the new functionality.  Doing it this way
would also validate or flesh out our plugin story.

Cheers,
-Barry


pgpoKhgw5wWuJ.pgp
Description: OpenPGP digital signature
___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/archive%40jab.org

Security Policy: http://wiki.list.org/x/QIA9

Re: [Mailman-Developers] GSoC 15 - Interested in contributing to Hyperkitty

2015-03-24 Thread David Udelson
I have submitted a proposal on Google Melange. Any feedback I could get
about my project constraints would be great!

Thanks,
David

On Tue, Mar 24, 2015 at 8:23 AM, Aurelien Bompard aurel...@bompard.org
wrote:

  I'm interested in contributing to the Hyperkitty archiver. Specifically,
  it looks like some requested features for Hyperkitty include rss
 syndication
  for entire mailing lists/specific users/specific threads, and the
 ability
  to view entire threads as plaintext and download that plaintext.

 We don't have either feature yet, but I think RSS syndication would be
 much too short for a GSOC project.
 There's always been demand for a way to download a list archive as an
 mbox file, and it's reasonably complicated if you want to avoid
 blocking the UI, so this may be a good project similar to the
 plaintext threads feature you mention.

 Aurélien
 ___
 Mailman-Developers mailing list
 Mailman-Developers@python.org
 https://mail.python.org/mailman/listinfo/mailman-developers
 Mailman FAQ: http://wiki.list.org/x/AgA3
 Searchable Archives:
 http://www.mail-archive.com/mailman-developers%40python.org/
 Unsubscribe:
 https://mail.python.org/mailman/options/mailman-developers/dru5%40cornell.edu

 Security Policy: http://wiki.list.org/x/QIA9

___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/archive%40jab.org

Security Policy: http://wiki.list.org/x/QIA9

Re: [Mailman-Developers] Getting approval to go ahead

2015-03-24 Thread Stephen J. Turnbull
Barry Warsaw writes:

  Keep feature and bug branches small if possible, and concise, such
  that they only implement the feature your working on or fix the
  reported bug.  A little bit of extraneous stuff might be okay if it
  improves readability, but don't go overboard.

I'd like to gloss this a bit.

(1) Suppose you change a small function whose formatting is not PEP 8
conformant (or otherwise so ugly you can't help fixing it -- of
course, check blame first, if Barry committed those lines, have
your eyes checked instead :-).  I would (1) fix the problem in the
current style, commit, and then (2) reformat the function to PEP
8, and give that a separate commit.  If the number of lines that
would *not* be in the diff at all if you didn't mess with style is
greater than the number of lines in the diff from step (1), don't
do step (2) without checking with the project leads.

(2) Documentation and test changes implied by your code changes should
go in the same branch.  In other projects I usually commit docstring
changes with the code change, and use two more commits, one for
standalone docs and one for unit tests.

(3) Unrelated documentation changes (eg, a typo you notice in another
function's docstring, or you want to add a docstring to an
undocumented function that you had to study) should go in a
separate *branch*.  I usually have a branch just to collect these
small improvements, and push them (or submit a merge request,
depending on the project workflow) all at once.

These practices are noticably tedious for the committer, but I find
they greatly improve the usefulness of commit logs and the readability
of diffs.  It's not officially in the Zen of Python, but widely
acknowledged, that Python should be written with the assumption that
the code will be read many thousand times more often than it's written.


___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/archive%40jab.org

Security Policy: http://wiki.list.org/x/QIA9


Re: [Mailman-Developers] Small change to REST API for held subscription requests

2015-03-24 Thread Barry Warsaw
On Mar 23, 2015, at 08:39 PM, Andrew Stuart wrote:

Any thoughts on how we could integrate with LDAP?

The intent is that the user database could be backed by, or augmented by LDAP,
although that is currently a goal, not reality.

Maybe of Mailman had some sort of event notification hook system for when
users add/edit/delete.  Or if there was some sort of generalised message bus
within the system that funnels messages about changes to the user database.
Something sort of LDAP synch process could then use those to synchronise an
LDAP database. An event hook or message queue would allow changes to the
users to be immediately pushed into LDAP.

Internally, Mailman does use zope.events to signal things nonlocally to other
parts of the system.  For example, events get fired when the configuration
changes, or when a message gets accepted.  Some of those events are connected
to handlers, but others are just waiting to be used by some future plugin or
other.  It's cheap to add events and handlers.

For users though we don't have much atm.  Just a PasswordChangeEvent and an
unsubscribe event.  There's no reason why we couldn't add more to help with
external user database synchronization.

Maybe the other possibility is some sort of direct integration such that
Mailman, if given details of an LDAP server, can directly make changes to the
LDAP system each time a user is added/edited/deleted.

The other point is that the use of interfaces and the Zope Component
Architecture (ZCA) is supposed to provide a layer of abstraction that could be
used as plug points for integrating with other backends.

Think of the IUserManager.  The existing implementation of that interface is
mailman.model.usermanager.UserManager, and it stores user information in the
local SQL database.  This connection between interface and implementation is
through src/mailman/config/configure.zcml.  Since there's only one user
manager in the system, it's defined as a utility:

  utility
provides=mailman.interfaces.usermanager.IUserManager
factory=mailman.model.usermanager.UserManager
/

Thus when the code calls getUtility(IUserManager) and gets back the built-in
UserManager object, after that, everything works through the interface.  You
could conceivably create a plugin with your own LDAP backed UserManager,
change the utility connection, and then the rest of Mailman would just keep on
working.

In theory of course.  No one to my knowledge has actually tried to explore
these ideas.

Cheers,
-Barry


pgpgY9r7T7bYU.pgp
Description: OpenPGP digital signature
___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/archive%40jab.org

Security Policy: http://wiki.list.org/x/QIA9

Re: [Mailman-Developers] Common use case archiving via configuration

2015-03-24 Thread Stephen J. Turnbull
Barry Warsaw writes:

  These [example? archivers] are all pretty interesting.  We'd have
  to think about whether we'd want some, any, or all them in the main
  tree, which essentially means we're committing to supporting them.
  
  Alternatively, we can create a contrib directory or repo

Batteries included, please!

A nice option would be that in the admin interface, the admin could
tick a box saying show me contrib archivers and then tick and
configure the appropriate one.  The instructions for contrib
configurations would say that Mailman is not committed to supporting
these but we'll help as time permits (and we always have time to
review patches -- a lie, but with good intent wink /).

___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/archive%40jab.org

Security Policy: http://wiki.list.org/x/QIA9


[Mailman-Developers] Fwd: Re: GSoC 15 - Interested in contributing to Postorius​

2015-03-24 Thread y1dothie
(Sorry Stephen)

-- Forwarded message --
From: Akshay Shah y1dot...@gmail.com
Date: Wed, Mar 25, 2015 at 12:21 AM
Subject: Re: [Mailman-Developers] GSoC 15 - Interested in contributing to 
Postorius
To: Stephen J. Turnbull step...@xemacs.org

 I wanted to work on Feature request: Editable fields in User profile
 https://bugs.launchpad.net/postorius/+bug/1432383 in the postorius web
 ui. Would this be an easy enough issue to pursue? Where do I ask if I have
 any questions about it? I have already commented on the page but not reply
 yet.
 On Tue, Mar 24, 2015 at 8:41 AM, Stephen J. Turnbull step...@xemacs.org
 wrote:
 Akshay Shah writes:

   My name is Akshay Shah.


 Pleased to meet you, Akshay.  Welcome to Mailman!

1. Can the Dashboard and Subscriber profile page be done together
   within the GSoC timeline?

 I think this would not be useful as the users' tasks are quite
 different.  It's better to pick one and do a super job on it than to
 do merely satisfactory work on both.

   2. I have fairly good amount of experience with NodeJs, will my
  resume be a good enough factor for you to decide or would you
  rather have me do a pre-project (I would love to do a
  pre-project)?

 Rather than a pre-project we prefer you to do a bugfix.  Any bug, easy
 is OK.  Feature implementation is also OK.  The point is that you
 become familiar with our workflow as much as it is for us to see your
 work output.

 https://bugs.launchpad.net/mailman

 Use the advanced search link with tag = easy (the tag entry field is
 way down at the bottom).  Note that most of the easy issues seem to
 be taken in that somebody's already working on them, check the
 comments to find out who.  Most likely most of the issue have *not*
 been checked for easy so don't let lack of an easy tag stop you
 from working on an interesting issue.  If it isn't clear how to
 proceed after a few minutes, you might want to ask here if the issue
 is maybe too hard for a GSoC application before continuing (and
 maybe try a different issue in the meantime).

   3. [NEW IDEA] Introducing search feature in the Web UI. Search
  lists, members, settings, and/or domains. Would introduction of
  Elastic Search with NodeJs be a feasible option for the GSoC
  Timeline? How detailed should I get about introduction of this
  topic?

 I don't see why this feature needs to be so complex.  There are a few
 sites like universities and project forges that deal with thousands
 of lists and tens of thousands of users, but thousands is still easy
 and efficient enough to do with a simple database lookup (and regexp
 matching on the results if desired) through the Django ORM (HyperKitty
 and Postorius) or SQLAlchemy and the REST interface (Mailman core),
 and it will be consistent with the code in the rest of the
 applications.

 If you have use cases where a more complex search would be helpful,
 please explain it.

 Regards


___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/archive%40jab.org

Security Policy: http://wiki.list.org/x/QIA9


Re: [Mailman-Developers] Getting approval to go ahead

2015-03-24 Thread Barry Warsaw
On Mar 25, 2015, at 01:02 PM, Stephen J. Turnbull wrote:

(1) Suppose you change a small function whose formatting is not PEP 8
conformant (or otherwise so ugly you can't help fixing it -- of
course, check blame first, if Barry committed those lines, have
your eyes checked instead :-).  I would (1) fix the problem in the
current style, commit, and then (2) reformat the function to PEP
8, and give that a separate commit.  If the number of lines that
would *not* be in the diff at all if you didn't mess with style is
greater than the number of lines in the diff from step (1), don't
do step (2) without checking with the project leads.

+1; of course, I'm happy to look at purely style fixing branches, again if
they're small, concise, and manageable.  If I can't tell that only style was
changed (i.e. no features or bug fixes snuck in) then I'd probably reject it.

My general approach for style nits is:

- If they're pervasive (e.g. lots of too-long lines), I'll move the mp to
  Needs Fixing and ask you to repair them.

- If they're fairly minor or rare, I'll comment about them in the mp and fix
  them up when I merge.  I don't want to be so nitpicky as to reject a good
  branch just for some minor style issues, but I *do* want the submitter to
  get a sense of how the next branch can avoid those issues.

(2) Documentation and test changes implied by your code changes should
go in the same branch.  In other projects I usually commit docstring
changes with the code change, and use two more commits, one for
standalone docs and one for unit tests.

+1.  This is actually something I wish vcses would handle better.  Maybe
there's git magic that would make this workflow better, but here's what I do
when reviewing a merge proposal (most relevantly, bug fixes):

- Run the full test suite with the full branch merged, but not yet committed.
  If there are *any* failures, the mp gets Needs Fixing'd.

- Unapply the fix, and run the test suite.  I should now see the new tests,
  and only the new tests fail.

- Review the tests; do they actually test what they claim, and do they
  actually test the new code?

- Reapply the fix.  Everything should pass again.

I can do this with `bzr shelve` but it's a little clunky.  It would be kind of
cool if TDD could be evident somehow in the branch, maybe e.g. as separate
commits.  `git rebase --interactive` might be promising.

(3) Unrelated documentation changes (eg, a typo you notice in another
function's docstring, or you want to add a docstring to an
undocumented function that you had to study) should go in a
separate *branch*.  I usually have a branch just to collect these
small improvements, and push them (or submit a merge request,
depending on the project workflow) all at once.

+1

These practices are noticably tedious for the committer, but I find
they greatly improve the usefulness of commit logs and the readability
of diffs.  It's not officially in the Zen of Python, but widely
acknowledged, that Python should be written with the assumption that
the code will be read many thousand times more often than it's written.

Absolutely.  Really great advice, Steve.

Cheers,
-Barry


pgpurSTtYwHB4.pgp
Description: OpenPGP digital signature
___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/archive%40jab.org

Security Policy: http://wiki.list.org/x/QIA9

Re: [Mailman-Developers] Common use case archiving via configuration

2015-03-24 Thread Barry Warsaw
On Mar 25, 2015, at 01:07 PM, Stephen J. Turnbull wrote:

A nice option would be that in the admin interface, the admin could
tick a box saying show me contrib archivers and then tick and
configure the appropriate one.  The instructions for contrib
configurations would say that Mailman is not committed to supporting
these but we'll help as time permits (and we always have time to
review patches -- a lie, but with good intent wink /).

An easy way to do that would be to include something like [provisional]
(thinking PEP 411) or [unsupported] in the archiver name.  This should then
be visible to the list admin when they're selecting them.

Cheers,
-Barry


pgpCpnl99JXPH.pgp
Description: OpenPGP digital signature
___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/archive%40jab.org

Security Policy: http://wiki.list.org/x/QIA9

Re: [Mailman-Developers] Common use case archiving via configuration

2015-03-24 Thread Andrew Stuart
Agreed Stephen.

It would be good if only configuration was required to drive the archivers even 
though they are only examples/provisional/contrib whatever. 

I’m hoping that projects like Hyperkitty and my server won’t need installers to 
touch the Mailman source code at all to get data flowing from Mailman.

Once you understand archivers they’re pretty easy but to newbies I suspect they 
are pretty intimidating to grasp.

What does batteries included mean in this context?

as


On 25 Mar 2015, at 3:07 pm, Stephen J. Turnbull step...@xemacs.org wrote:

Barry Warsaw writes:

 These [example? archivers] are all pretty interesting.  We'd have
 to think about whether we'd want some, any, or all them in the main
 tree, which essentially means we're committing to supporting them.
 
 Alternatively, we can create a contrib directory or repo

Batteries included, please!

A nice option would be that in the admin interface, the admin could
tick a box saying show me contrib archivers and then tick and
configure the appropriate one.  The instructions for contrib
configurations would say that Mailman is not committed to supporting
these but we'll help as time permits (and we always have time to
review patches -- a lie, but with good intent wink /).


___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/archive%40jab.org

Security Policy: http://wiki.list.org/x/QIA9

Re: [Mailman-Developers] Common use case archiving via configuration

2015-03-24 Thread Stephen J. Turnbull
Andrew Stuart writes:

  What does batteries included mean in this context?

That you shouldn't need to fetch the archivers from a separate archive
or repo to configure them.  Sure, you *could* register them as plugins
and have a button to fetch them from PyPI (or just do it if the user
selects one, but I don't see a huge benefit to that practice in this
case.

Thus I prefer a contrib directory or subtree to the repo proposal.



___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/archive%40jab.org

Security Policy: http://wiki.list.org/x/QIA9


Re: [Mailman-Developers] GSoC 15 - Interested in contributing to Hyperkitty

2015-03-24 Thread David Udelson
Sent this email to Aurélien's personal email by mistake the first time
(sorry Aurélien!). I'll get used to the mailing list evenually

 There's always been demand for a way to download a list archive as an
mbox file

This is a feature I am interested in pursuing. From a very (very) high
level, the process of creating an mbox file out a list archive should
entail:

1) Determine which messages to include in the mbox.
An entire list archive is clearly one choice, but is there also
interest in generating mbox files for specific threads, list archieves
between specific dates, etc.?
2) For each message, append plaintext to mbox file.
Is this the part where we risk blocking the UI? Certainly for
hundreds of thousands of messages, this will be a computationally intensive
step, so will this have to be run in a separate thread?
3) Present mbox file to user for download.
I'm hoping this is a trivial step, but I'm not sure about some of the
specifics. For example, is Hyperkitty only able to run on apache, or is the
choice of web server entirely up to the web admin? How we ultimately serve
the file will depend on these details.

Aurélien, just because I'm new to the codebase, what are the challenges you
see in implementing this feature? Whatever details you can give me will
help me set my milestones.

Thanks,
David

On Tue, Mar 24, 2015 at 8:23 AM, Aurelien Bompard aurel...@bompard.org
wrote:

  I'm interested in contributing to the Hyperkitty archiver. Specifically,
  it looks like some requested features for Hyperkitty include rss
 syndication
  for entire mailing lists/specific users/specific threads, and the
 ability
  to view entire threads as plaintext and download that plaintext.

 We don't have either feature yet, but I think RSS syndication would be
 much too short for a GSOC project.
 There's always been demand for a way to download a list archive as an
 mbox file, and it's reasonably complicated if you want to avoid
 blocking the UI, so this may be a good project similar to the
 plaintext threads feature you mention.

 Aurélien
 ___
 Mailman-Developers mailing list
 Mailman-Developers@python.org
 https://mail.python.org/mailman/listinfo/mailman-developers
 Mailman FAQ: http://wiki.list.org/x/AgA3
 Searchable Archives:
 http://www.mail-archive.com/mailman-developers%40python.org/
 Unsubscribe:
 https://mail.python.org/mailman/options/mailman-developers/dru5%40cornell.edu

 Security Policy: http://wiki.list.org/x/QIA9

___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/archive%40jab.org

Security Policy: http://wiki.list.org/x/QIA9

Re: [Mailman-Developers] Common use case archiving via configuration

2015-03-24 Thread Andrew Stuart
@barry - would you mind confirming please you’re OK with this please?

I’m going to implement some very simplistic, working, example archivers. As 
discussed with Stephen, although functional, these will be “working examples”, 
much like prototype.py is currently, and won’t be presented as complete archive 
integration solutions, rather as starting points/examples that happen to work 
for some common use cases. This is to avoid a concern that they might create a 
maintenance burden if end users see them as an official solution to archive 
integration.

— archiver filename: archiving/to_maildir.py
- config options:
  config[‘archiver.name’].folderpath =
— description: this is close to a straight copy of prototype.py and archives 
messages to a Maildir

— archiver filename: archiving/to_filesystem.py
- config options:
config[‘archiver.name’].folderpath =
config[‘archiver.name’].zip = True
- description: this writes emails to the file system, optionally zipped

— archiver filename: archiving/to_httppost.py
- config options:
config[‘archiver.name’].target_url =
config[‘archiver.name’].headers =
config[‘archiver.name’].url_parameters =
config[‘archiver.name’].form_fields =
config[‘archiver.name’].X_Message_ID_Hash_only =
- description: this (with Aurelien’s permission) will be based on Hyperkitty’s 
HTTP POST archiver and will behave in a similar manner.  Alternatively It’ll be 
written from scratch.

— archiver filename: archiving/to_smtp.py
- config options:
config[‘archiver.name’].destination_address =
- description: this is very close to a straight copy of mail_archive.py but 
will remove a few lines of code to make it a more generalised smtp sender

— archiver filename: archiving/to_nntp.py
- config options:
config[‘archiver.name’].server_address =
config[‘archiver.name’].username =
config[‘archiver.name’].password =
- description: this will use the standard library to post the archive message 
to an NNTP server

— archiver filename: archiving/to_git.py
- config options:
   config[‘archiver.name’].foldername
- description: this will write the email to a git repository on the local file 
system

— archiver filename: archiving/to_AMQP.py
- config options:
config[‘archiver.name’].server_address
config[‘archiver.name’].username =
config[‘archiver.name’].password =
config[‘archiver.name’].X_Message_ID_Hash_only =
config[‘archiver.name’].max_message_size =
- description: this will write the message to an AMQP server and optionally 
writes only the message id hash






___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/archive%40jab.org

Security Policy: http://wiki.list.org/x/QIA9

Re: [Mailman-Developers] Fine grained subscription control, scaling, noise, relevancy, subscription rules, dynamic lists

2015-03-24 Thread Andrew Stuart
@barry, @stephen, @terri

I  know this is a wall of text (sorry) but if you have time I’d be interested 
to hear your thoughts as Dynamic Sublists are in the GSOC projects list for 
this year.

as

On 20 Mar 2015, at 9:53 am, Andrew Stuart andrew.stu...@supercoders.com.au 
wrote:

Refer to the bottom of this email for some previous quotes from @barry on this 
topic. I’ve also had off list discussions with Barry in which he has mentioned 
this same topic so it seems to have some previous thought gone into it. 

I’m wanting to raise the topic of “fine grained subscription control” (for want 
of a better term) for discussion. Please note these thoughts refer to 
discussion style lists rather than notification style lists. Please prefix 
every sentence here with IMO - I’m not saying I’m right about anything, just 
putting forward food for thought. Also, I’m no Mailman expert so if my 
assumptions are plain wrong please let me know.

One of the core problems of mailing lists (at least as implemented in Mailman) 
is that there’s not much middle ground between being very involved (i.e receive 
all posts to list), or being almost-not-involved (i.e. receiving daily digests 
or no digest).

List noise and relevancy is the main problem and it’s a big impediment to lists 
being more widely used.

Very engaged users might be ok with constantly deleting messages that they are 
not interested in (i.e. irrelevant noise). Less engaged users will be annoyed 
with the noise and unsubscribe, or will switch to digest notification which I 
suggest to you is close to unsubscription anyway because that user no longer 
sees emails that they might have been interested in had they known there was a 
message on that subject. Digest users I suggest are at best observers rather 
than participators.

The more active the list, the greater the noise problem, the more users will 
drop out of the list.  As a list gains more subscribers, eventually even the 
most engaged user will have had enough of the volume of messages and will drop 
back to digest or write some mailbox rule that moves the emails to a folder, 
effectively dis-engaging them from the list conversation flow. Thus Mailman 
discussion-style lists currently don’t scale.

Even for low volume lists, noise is a major inhibitor to usage in certain 
contexts.  Consider for example the CIO of a company or the manager of a 
division of 30 developers. There are various mailing lists being used by the 
various projects that they are responsible for. The leader is not participating 
however because the noise from those lists would be overwhelming. They would 
however like to be partipcating in list discussions either that they initiated, 
are explicitly copied into, or relate to topics (keywords) that they are 
interested in. In reality, I’m not convinced Mailman would often be used in 
contexts like this currently because the relevancy/noise/digest/unsubscribe 
problem is a showstopper.

The systers have recognised this problem and their solution is Dynamic sublists 
as described here:
http://wiki.list.org/DEV/Dynamic%20Sublists
List subscribers can decide whether to be a part of new conversations or not. 
If the users decide to subscribe to new conversations, then they will receive 
all the messages of a new conversation unless they explicitly unsubscribe from 
it. If they otherwise decide to unsubscribe from new conversations,then they 
will receive only the first message of every new conversation unless they 
explicitly subscribe to it.”

I don’t think dynamic sublists are an effective solution (IMO, remember?).  
Getting the root message in all conversations is still too noisy and its a 
clumsy mechanic to have to opt in to further discussion and my thinking is that 
opt-in-to-discuss will impose a barrier to engagement that will reduce user 
interaction - put another way - “opt in to every discussion?  too hard”. 

Possibly a solution worth considering is fine-grained subscription control, in 
which there is a set of SEND and DONTSEND rules at a List default level and at 
a user level.

SEND:
Discussions where list member email address is in to/from/cc:
Discussions where to/from/cc contains one or more of: sa...@example.org, 
dav...@example.org, *.example.com
Discussions where subject contains keyword: hyperkitty
Discussions where body contains keyword: hyperkitty

DONTSEND:
Discussions where list member email address is in to/from/cc:
Discussions where to/from/cc contains one or more of: sa...@example.org, 
dav...@example.org
Discussions where subject contains keyword: Java
Discussions where body contains keyword: ruby

The technical hurdle to making this work is that Mailman needs access to 
historical messages to make it work (i.e. integrating some level of aqrchive 
like functionality into the Mailman database). I suggest this this may not be 
as hard as it sounds and hey, we’ve got a database there anyway so why not use 
it? 

One possibility is that this could be pushed into