Re: [Mailman-Developers] Authorization System in Core

[Andrew Stuart]

You are most welcome to do as you wish with Mailmania, rename, fork, rebuild, 
whatever - I’m not precious about it - it is owned by the FSF and GPL licensed. 
 It might be just a good “first try” and point the way to a better solution.  

It certainly would benefit from a cleanup from a more experienced Python 
developer than me - whilst I did everything I could to make it consistent I 
have no doubt much could be done to improve it.

As Barry says, Mailmania puts a server layer in front of the Mailman REST API 
that allows authenticated public access to the Mailman REST server.  It uses 
the Mailman conceptual authorisation model and implements that as a concrete 
set of authorisation rules.

It also includes an unfinished solution for archiving inbound mail to sqlite 
for full text indexing and integration with Apache Tika for extraction of text 
from documents attached to emails.

I apologise for effectively having abandoned it but I would say it is (or was 
last I looked) fully working with over 700 tests.  I just don’t have time to 
fit it in to everything in life right now..

If anyone does want to do any work on it I’ll do my level best to help them get 
it running, understand it and problem solve any issues.

I’m working on other stuff but still watching this list so I’ll try to respond 
ASAP to any questions….

thanks

as

___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/archive%40jab.org

Security Policy: http://wiki.list.org/x/QIA9

Re: [Mailman-Developers] User-centric authentication and access control

[Andrew Stuart]

Postorius is a Django app that talks directly to the Mailman 3 REST API and 
integrates everything needed to get the job done including user interface, 
authentication and authorization.

Mailmania is an authenticating reverse proxy server.  Effectively it would 
allow someone to write a purely JavaScript browser based interface to Mailman 
3, and it would provide the back end authorization and authentication.  No such 
JavaScript interface exists though. Mailmania provides a way for the public to 
talk to the Mailman 3 core REST API. 

In very loose terms you can imagine if you pulled Postorius apart into a 
backend and a JavaScript userinterface/front end, then Mailmania would be the 
equivalent of the back end. Sort of.

If you wanted to write a user interface for Mailman using ReactJS or AngularJS 
then you’d probably want it to talk to Mailmania.

If you want to actually have a working interface to Mailman 3 then you need 
Postorius as no-one has written any user interface code for Mailmania.

as

___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/archive%40jab.org

Security Policy: http://wiki.list.org/x/QIA9

Re: [Mailman-Developers] User-centric authentication and access control

[Andrew Stuart]

As far as I know that’s a pretty good summary.  I should say however I really 
don’t know much except about Mailmania.  You’d get better information on 
Postorius from Aurelien and Barry is the authority on the Mailman core.

It’s important to understand that Mailman core defines the resources and allows 
assignment of membership and ownership of those resources.  The business rules 
for how that is interpreted is left up to the application server, so Mailmania 
and Postorius almost certainly do it differently.  Anyway you probably don’t 
need to think about that.

The most battle tested parts of this are Mailman Core and Postorius. Mailmania 
isn’t used as far as I know - someone may do so one day but for the most part 
it’s leading a quiet life.

>>I'm assuming Mailmania relies on presenting a username and password directly 
>>to the Mailman REST API to authenticate?
That is correct.

>>Or would it also have some mechanism to let you customize the authentication 
>>mechanism?
All Mailmania’s login function does is check username and password with the 
core. If all good then it hands back a token.  There’s no reason you couldn’t 
modify that very simple logic to do some other check like looking up in LDAP 
and then handing back the token.   Probably you’d create an additional login 
function that does whatever auth lo9gic you want and then hands backs a JWT.

I don’t know anything about Django authentication.  Others pmay commment on 
that.

as





On 2 Sep 2015, at 12:50 pm, Waldbieser, Carl <waldb...@lafayette.edu> wrote:


I believe I have a better understanding of the puzzle pieces, now.  In a 
nutshell:

* Mailman 3 allows you to assign administrative privileges to individual users 
who may log in with their own credentials.
* Mailman 3 is composed of several separate projects that work together to form 
a coherent system.
* The Postorius component is the Django-based web UI component for Mailman 3.
* Because it is Django based, the authentication scheme for Postorious can be 
customized by configuring the appropriate Django authentication middleware 
(e.g. REMOTE_USER [1]).
* The Mailman REST API provides you with superuser-like access to Mailman, and 
should only be used to integrate trusted apps.
* Mailmania can be used to provide external access to the RESTful API.  
Requests are are authenticated, and the typical access controls are enforced.

It seems to me that if I have an authentication mechanism that supports 
exporting the authenticated user via REMOTE_USER into the web server 
environment, then I could use that mechanism with Django + Postorius.

I'm assuming Mailmania relies on presenting a username and password directly to 
the Mailman REST API to authenticate?  Or would it also have some mechanism to 
let you customize the authentication mechanism?

Thanks,
Carl Waldbieser
ITS Systems Programmer
Lafayette College

[1] https://docs.djangoproject.com/en/1.8/howto/auth-remote-user/

- Original Message -
From: "Andrew Stuart" <andrew.stu...@supercoders.com.au>
To: "waldbiec" <waldb...@lafayette.edu>
Cc: "mailman-developers" <mailman-developers@python.org>
Sent: Tuesday, September 1, 2015 9:58:32 PM
Subject: Re: [Mailman-Developers] User-centric authentication and access control

Postorius is a Django app that talks directly to the Mailman 3 REST API and 
integrates everything needed to get the job done including user interface, 
authentication and authorization.

Mailmania is an authenticating reverse proxy server.  Effectively it would 
allow someone to write a purely JavaScript browser based interface to Mailman 
3, and it would provide the back end authorization and authentication.  No such 
JavaScript interface exists though. Mailmania provides a way for the public to 
talk to the Mailman 3 core REST API. 

In very loose terms you can imagine if you pulled Postorius apart into a 
backend and a JavaScript userinterface/front end, then Mailmania would be the 
equivalent of the back end. Sort of.

If you wanted to write a user interface for Mailman using ReactJS or AngularJS 
then you’d probably want it to talk to Mailmania.

If you want to actually have a working interface to Mailman 3 then you need 
Postorius as no-one has written any user interface code for Mailmania.

as

___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/archive%40jab.org

Security Policy: http://wiki.list.org/x/QIA9

Re: [Mailman-Developers] User-centric authentication and access control

[Andrew Stuart]

It might be helpful if I explain what Mailmania does.

login - an anonymous HTTP request can be sent to the server asking to log in 
and POSTing Mailman username and password. Success results in a response with a 
JWT in the HTTP headers.
authentication - when an HTTP request arrives, it is examined to see if it 
contains a valid Jason web token (JWT) in its headers
authorization - when an HTTP request comes in, the request is examined to 
determine if the requesting user (as defined by the JWT attached to the 
request) has permission to access the resource they have requested. It has a 
set of business rules that make decisions about what resources a user is able 
to access. If authorization is approved, the request is proxied to the Mailman 
core REST API.

The Mailman permissions model is based on the idea that Mailman has the 
following resources:

a server
domains
lists
users

Users can have auth roles: (not the same as Mailman 2 roles I think)
userowner (this is implied, a user is its own owner)
listowner
listmember
listmoderator
listnonmember
domainowner
serverowner

Any HTTP request to Mailmania must be from a specific user, attempting to 
access a specific resource. Mailmania basically just figures out if that user 
is indeed allowed access to the resource that have asked for, and sends it 
through or denies it.

___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/archive%40jab.org

Security Policy: http://wiki.list.org/x/QIA9

Re: [Mailman-Developers] User-centric authentication and access control

[Andrew Stuart]

Sorry I should say - I don’t think roles exist in Mailman 3? Someone else will 
know but I don’t recall offhand.
___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/archive%40jab.org

Security Policy: http://wiki.list.org/x/QIA9

Re: [Mailman-Developers] User-centric authentication and access control

[Andrew Stuart]

If users still sign in directly, what do you see the functionality of roles 
being from a Mailman perspective? I don’t see how they relate Mailman resources 
specifically.

The authenticating proxy server (which is still awaiting a better name - Barry? 
:-)) currently gives individuals control over Mailing lists, assuming that 
individual has moderator status.

Currently it decides to grant an authentication token based on reciept of 
username and password, which it validates by asking Mailman core if the 
username and password match. If you know Python then that is the staring point 
for hooking in an additional level of authorisation.  If you can write some 
pseudocode for how your authentication process works I could comment further.

as
 


On 2 Sep 2015, at 1:15 am, Waldbieser, Carl  wrote:

I know that currently, mailman roles are set up such that the roles themselves 
have a shared password per role.  I want to be able to move away from that 
model and have roles assigned to individual user accounts that would allow 
access to the admin interfaces for individual lists.

For example, say we have mail lists "Campus" and "Board of Trustees".  I might 
have roles "campus_moderators", "campus_admins", "boardoftrustees_moderators", 
and "boardoftrustees_admins".
If I assign the role campus_admins to user "johnsmith", I would like this user 
to be able to access the mailman admin interface for the "Campus" list using 
his own credentials.  Ideally, "johnsmith" would not have to present his 
primary credentials to the mailman interface because our institution has a web 
single sign-on infrastructure (Web SSO).

I would like to actually move the authentication and role management *outside* 
of mailman and have the administrative interface consume the role based 
information from external sources (e.g. LDAP, CAS or SAML2 attribute release), 
so I am looking for a more "pluggable" authentication and access management 
architecture.

Does anything like this exist for Mailman, or is it on the roadmap?  Are there 
technical guidelines for how one might contribute toward something like this?

Thanks,
Carl Waldbieser
ITS Systems Programmer
Lafayette College
___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/andrew.stuart%40supercoders.com.au

Security Policy: http://wiki.list.org/x/QIA9

___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/archive%40jab.org

Security Policy: http://wiki.list.org/x/QIA9

Re: [Mailman-Developers] [Mailman 3] Granting Postorius web access to Mailman users?

[Andrew Stuart]

I think I would second the idea of having a single component/database that is 
responsible for the management of user information.

The problem is that the larger Mailman ecosystem is designed to be a loose 
collection of components so there is no way to define what information each 
component needs stored at the Mailman Core.  It wouldn’t make sense for the 
data store requirements of Postorius to be baked in to the Mailman Core data 
structures because Postorius is an optional component of the Mailman suite.  

In the case of your original question:
However, I am having some trouble understanding the procedure of granting 
users access to the Postorius webinterface. 

Whether or not a user has access via the web interface is application specific 
to Postorius so it should be stored in a local datastore managed by Postorius.

It’s both a strength and a weakness that Mailman Core is quite rigid about 
“taking care only of its own business”.  The big benefit of this strategy is 
that in future it will continue to be practical to upgrade Mailman Core exactly 
because it has tightly defined data structures and no knowledge of how other 
applications in the ecosystem behave.  Customised data structures in the 
Mailman Core would lead to major issues in forward compatibility - imagine the 
nightmare of trying to upgrade Mailman Core if it included application-specific 
customisations that had been wound in over time - it would be like chewing gum 
in your hair - you’d never get it out.

The important thing is that Mailman Core provides a resource permissions model 
that can be leveraged by other servers such as Postorius so that the central 
store of Mailman user authorisation information can be applied to application 
specific data stores.  Which is to say, Postorius (or any other application) 
can store data using the Mailman userid, and can check with the Mailman Core to 
see if it is a valid user.

Mailman Core’s commitment to minding its own business is a good compromise - 
its not ideal but it’s not that bad either - the key thing for third party 
application developers to understand is that Mailman provides a unique ID for 
every resource (users, domains, lists etc), and provides a permissions model 
that defines which of these resources users have access to.  Third party 
applications can connect logically to this conceptual framework in a fairly 
loosely coupled manner.

as
___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/archive%40jab.org

Security Policy: http://wiki.list.org/x/QIA9

Re: [Mailman-Developers] [Mailman 3] Granting Postorius web access to Mailman users?

[Andrew Stuart]

Having said all that, one thing that might work is for the Mailman Core to 
provide the ability to retrieve key/value pairs that can be attached to any 
given resource.  This is sufficiently simple to mean it would present no 
problems for forward compatibility.

key/value pairs would suffice for some applications but as soon as you tried to 
build any serious third party Mailman application you’d fairly rapidly move 
beyond the capabilities of a key/value store.

So there is something to be said for a very simple Mailman Core key/value store 
mechanism that would meet some simple needs. The argument against this is “well 
you’re going to outgrow it pretty quick anyway so why not immediately start 
with a more full featured data store”.

as
___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/archive%40jab.org

Security Policy: http://wiki.list.org/x/QIA9

[Mailman-Developers] Anyone with JavaScript skills up for a challenge....

[Andrew Stuart]

It would be good also if any daring developers with JavaScript skills were 
willing to have a go at getting something from the authenticating proxy server 
to display in a browser.

If you decide have a go at it, let me know and we can start by doing a short 
Google Hangouts session in which I can talk you through the process and 
concepts.

Step one would be to get the authenticating proxy server running on your 
Mailman server

There’s almost no documentation I am sorry to say, but there are installation 
instructions in the readme, and all of the Mailman REST API documentation 
remains accurate.

https://gitlab.com/astuart/mailmania

There is a handy list of API operations available here:
http://supercoders.com.au/swagger-ui/dist/index.html
Note that the routes are entirely accurate but none of the parameters are.

At its simplest, your JavaScript code would need to:

1: send a request to the login route (you will need to have a user configured 
in Mailman) 
POST /users/{id}/login

2: the login route will send back a JSON web token.

3: you can then send any API request you like to the server but you must 
provide the JSON web token in the X-Auth-Token: header

It will look something like this:
X-Auth-Token: 
eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJ1c2VyX2lkZW50aWZpZXIiOiJ0ZXN0c2VydmVyb3duZXJAbWFpbC5leGFtcGxlLm9yZyIsImV4cCI6MTQzNzE2MTIzOH0.UNo72oWoU4B35Bmmco7PeK7K8eyZcu3OHHBsyhwjhrQ.

4: thats it!



as

___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/archive%40jab.org

Security Policy: http://wiki.list.org/x/QIA9

[Mailman-Developers] A rambling tour of the authenticating proxy server

[Andrew Stuart]

If you have an hour or so to kill you can follow the fairly rambling tour that 
Barry and I went on of the authenticating proxy server here.  Sorry it’s not 
practised or focused but you should be able to glean some info out of it.

http://youtu.be/DWDjtWQNx7c

If you have questions - fire away.


___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/archive%40jab.org

Security Policy: http://wiki.list.org/x/QIA9

[Mailman-Developers] New name needed for authenticating proxy server

[Andrew Stuart]

Authenticating Proxy Server - such a catchy and inspiring name.

Barry has suggested and I heartily agree that we should solicit ideas from the 
Mailman community for a better name for it.

Your prize will be the great honour of a heartfelt “thanks!”

Anyone at all welcome to submit ideas.

Stephen Turnbull has suggested:
For the authentication service, does the cuckoo have any natural enemies?  Or 
we could call it some variation on Cliff Stoll. :-) Or maybe Barbarian Gate 
(from the cover of Cheswick  Bellovin).

as

___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/archive%40jab.org

Security Policy: http://wiki.list.org/x/QIA9

[Mailman-Developers] Guided tour of the authenticating proxy server source code Friday

[Andrew Stuart]

Hello all

Barry asked if I would mind donating the authentication proxy server to the FSF 
under a GPL license.  I thought this is a good idea and it is now at 
https://gitlab.com/astuart/mailmania and we are now finalising the copyright 
transfer.

Please note that right now I am cleaning up some things so not much point in 
trying it out as the tests are broken and some bits and pieces are missing but 
I am working now to get it finalised and hopefully all will be working within a 
day or two.

I am giving Barry a guided tour of the source code of the authenticating proxy 
server on Friday 17th (Australian Friday 17th) via Google Hangouts.

We’ll try to record it but its a good time to get questions answered so you 
might get more out of joining in.

Anyone else interested in joining us? If so, please say what city/timezone you 
are in so we can find a common time that suits.

thanks

as

___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/archive%40jab.org

Security Policy: http://wiki.list.org/x/QIA9

[Mailman-Developers] is_server_owner set to true regardless of value posted

[Andrew Stuart]

As far as I can tell, when creating users, if the is_server_owner field is 
provided in the POST data, then it will always be set to true. Is this the 
intended behaviour? I was trying to create users and explcitly setting it to 
False and wondering why they were being created as server_owners anyway.

I suppose this is a security issue although I imagine its unlikely any 
applications are using that field yet. The potential problem I suppose is that 
in the field there may be installations in which users created via POST have 
is_server_owner set to true.

See the commands below to see the problem in action:

# DELETE user to make sure its not present
(venv3.4)ubuntu@mail:~/mailmania$ curl -X DELETE --header authorization: Basic 
cmVzdGFkbWluOnJlc3RwYXNz 
http://localhost:8001/3.0/users/mailmanad...@example.org

# create user via POST with POSTed field value is_server_owner=anyoldvaluewilldo
(venv3.4)ubuntu@mail:~/mailmania$ curl -X POST --data 
email=mailmanad...@example.org --data display_name=displayname --data 
is_server_owner=anyoldvaluewilldo --header authorization: Basic 
cmVzdGFkbWluOnJlc3RwYXNz http://localhost:8001/3.0/users


# GET user - you can now see that is_server_owner is true
(venv3.4)ubuntu@mail:~/mailmania$ curl --header authorization: Basic 
cmVzdGFkbWluOnJlc3RwYXNz 
http://localhost:8001/3.0/users/mailmanad...@example.org
{display_name: displayname, password: 
$6$rounds=105489$ToJ.XV3yw0Mvee8r$pbMEE/6e1Xw8PPOunQGX1IL21NmNLrSwl3VFwZNtsoxwCjZ7iWZ.SjDmX7rs9nlM7pglz54GFjB8hmn.rOI.d/,
 http_etag: \7891a3ea8a2c71a67e738aede9b9d2cfbf438073\, user_id: 
264084494277271879132250546838180918030, self_link: 
http://localhost:8001/3.0/users/264084494277271879132250546838180918030;, 
created_on: 2015-07-12T05:08:07.320945, is_server_owner: true}

# DELETE user to make sure its not present
(venv3.4)ubuntu@mail:~/mailmania$ curl -X DELETE --header authorization: Basic 
cmVzdGFkbWluOnJlc3RwYXNz 
http://localhost:8001/3.0/users/mailmanad...@example.org

# create user via POST without any POSTed field value for is_server_owner
(venv3.4)ubuntu@mail:~/mailmania$ curl -X POST --data 
email=mailmanad...@example.org --data display_name=displayname --header 
authorization: Basic cmVzdGFkbWluOnJlc3RwYXNz http://localhost:8001/3.0/users

# GET user - you can now see that is_server_owner is false
(venv3.4)ubuntu@mail:~/mailmania$ curl --header authorization: Basic 
cmVzdGFkbWluOnJlc3RwYXNz http://localhost:8001/3.0/users
{total_size: 2, start: 0, entries: [{display_name: Insecure Testuser, 
password: 
$6$rounds=107421$p9wIIl6zAEHEe5ZC$JH0MhieMJr8URXGsQ5I6zLa3l4whbRsMhjKOntfDlxOrfstxaRvlMgPvuEYU05ptNnSJdSnB43Elw0zAshFux1,
 http_etag: \6394e37e623cbfa20445201f091c4aabaaa01340\, user_id: 
309469782534072420914095048754305489000, self_link: 
http://localhost:8001/3.0/users/309469782534072420914095048754305489000;, 
created_on: 2015-07-12T04:49:17.499529, is_server_owner: true}, 
{display_name: displayname, password: 
$6$rounds=98938$pOH6NcMh5RN1mNs0$8wUWsaEXVaqiA6z/B2qTXIYVDMVMunzaMmWCL3JmxL/EnknA4A0PFisJZA0VcDOzY0KqiU2sljlBPXuNrEypl.,
 http_etag: \3c81af299b5b36b104d9c7066fed63273a72bf48\, user_id: 
176959263898569148772584820722098620353, self_link: 
http://localhost:8001/3.0/users/176959263898569148772584820722098620353;, 
created_on: 2015-07-12T05:09:12.043209, is_server_owner: false}], 
http_etag: \943acca65a65afd2c1c40b6fc1e010b337e1bad6\”}






___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/archive%40jab.org

Security Policy: http://wiki.list.org/x/QIA9

[Mailman-Developers] Is there a way to get list of server_owners from REST API?

[Andrew Stuart]

I can see it is possible to make a user a “server_owner”.

Thus I can check a user to see if they are a server_owner.

Is there a way to get a list of all server_owners?

thanks
___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/archive%40jab.org

Security Policy: http://wiki.list.org/x/QIA9

Re: [Mailman-Developers] Is there a way to get list of server_owners from REST API?

[Andrew Stuart]

I don’t think I need it right now - I’m doing authorisation stuff and I think I 
always know the user in question.

But I can imagine that people writing applications might like to be able to get 
a complete list of people with server_owner rights, so it would be woth adding 
at some point.

Yes either at /users/server_owners or some endpoint relating to 
system/configuration/server_owners perhaps. 

as


On 11 Jul 2015, at 2:24 pm, Barry Warsaw ba...@list.org wrote:

On Jul 11, 2015, at 02:18 PM, Andrew Stuart wrote:

 Is there a way to get a list of all server_owners?

Not currently without iterating over all users.

Maybe there should be a resource like /users/server_owners?

That should be pretty easy to implement.

Cheers,
-Barry
___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/andrew.stuart%40supercoders.com.au

Security Policy: http://wiki.list.org/x/QIA9

___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/archive%40jab.org

Security Policy: http://wiki.list.org/x/QIA9

Re: [Mailman-Developers] REST API speed

[Andrew Stuart]

I’ve fumbled around here to see if I can make it work with gunicorn.

Correct me if I’m wrong but effectively gunicorn would need to run the rest 
server within itself, so it would be necessary to run the rest server outside 
the mailman runners? I’m not sure how to do so.

rest/wsgiapp.py says:

# Both the REST server and the signal handlers must run in the main
# thread; the former because of SQLite requirements (objects created
# in one thread cannot be shared with the other threads), and the
# latter because of Python's signal handling semantics.



rest/wsgiapp.py also has the following function which seems to be the right 
wsgi app function to pass into gunicorn. I couldn’t find the right incantation 
to get it to run standlone.


def make_application():
Create the WSGI application.

Use this if you want to integrate Mailman's REST server with your own WSGI
server.

return RootedAPI(Root())
___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/archive%40jab.org

Security Policy: http://wiki.list.org/x/QIA9

[Mailman-Developers] Currently on HN front page

[Andrew Stuart]

https://news.ycombinator.com/item?id=9336291

___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/archive%40jab.org

Security Policy: http://wiki.list.org/x/QIA9

[Mailman-Developers] How to validate mailing list-id?

[Andrew Stuart]

I’ve read RFC2919 https://www.ietf.org/rfc/rfc2919.txt which says:
The list header fields are subject to the encoding and character restrictions 
for mail headers as described in [RFC822].

RFC822 http://www.w3.org/Protocols/rfc822/ says:
The field-body may be composed of any ASCII characters, except CR or LF. 
(While CR and/or LF may be present in the actual text, they are removed by the 
action of unfolding the field.)

Am I correct in understanding then that a mailing list-id may be composed of 
any ASCII characters?

Thus this is a valid mailing list-id: ///*/etc/pw.txt

Is this function an accurate validator for a mailing list-id?

def is_valid_list_id(list_id):
try:
list_id.decode('ASCII')
except UnicodeDecodeError:
return False
else:
return True





___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/archive%40jab.org

Security Policy: http://wiki.list.org/x/QIA9

Re: [Mailman-Developers] Fine grained subscription control, scaling, noise, relevancy, subscription rules, dynamic lists

[Andrew Stuart]

I wasn’t aware of MM2 topics.  MM2 topics looks interesting but I agree its 
implementation is cumbersome. I think somewhere in the intersection of nosy 
lists, dlists, topics and fine-grained-subscription controls is an elegant way 
to reduce noise and improve relevance and user experience. I’m hoping to circle 
back to all this some months down the track, no time to do anything on it now.

as

___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/archive%40jab.org

Security Policy: http://wiki.list.org/x/QIA9

[Mailman-Developers] Accessibility of interfaces

[Andrew Stuart]

Dave

You seem to have an interest in Mailman being accessible.

Care to share some requirements/requests/tips/pointers for the various people 
who are writing user interface code that talks to Mailman.

I’d like to take this sort of thing into account early in the process if 
possible so now is a good time to talk about what you want/need.

as

___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/archive%40jab.org

Security Policy: http://wiki.list.org/x/QIA9

Re: [Mailman-Developers] Common use case archiving via configuration

[Andrew Stuart]

Agreed Stephen.

It would be good if only configuration was required to drive the archivers even 
though they are only examples/provisional/contrib whatever. 

I’m hoping that projects like Hyperkitty and my server won’t need installers to 
touch the Mailman source code at all to get data flowing from Mailman.

Once you understand archivers they’re pretty easy but to newbies I suspect they 
are pretty intimidating to grasp.

What does batteries included mean in this context?

as


On 25 Mar 2015, at 3:07 pm, Stephen J. Turnbull step...@xemacs.org wrote:

Barry Warsaw writes:

 These [example? archivers] are all pretty interesting.  We'd have
 to think about whether we'd want some, any, or all them in the main
 tree, which essentially means we're committing to supporting them.
 
 Alternatively, we can create a contrib directory or repo

Batteries included, please!

A nice option would be that in the admin interface, the admin could
tick a box saying show me contrib archivers and then tick and
configure the appropriate one.  The instructions for contrib
configurations would say that Mailman is not committed to supporting
these but we'll help as time permits (and we always have time to
review patches -- a lie, but with good intent wink /).


___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/archive%40jab.org

Security Policy: http://wiki.list.org/x/QIA9

Re: [Mailman-Developers] Common use case archiving via configuration

[Andrew Stuart]

@barry - would you mind confirming please you’re OK with this please?

I’m going to implement some very simplistic, working, example archivers. As 
discussed with Stephen, although functional, these will be “working examples”, 
much like prototype.py is currently, and won’t be presented as complete archive 
integration solutions, rather as starting points/examples that happen to work 
for some common use cases. This is to avoid a concern that they might create a 
maintenance burden if end users see them as an official solution to archive 
integration.

— archiver filename: archiving/to_maildir.py
- config options:
  config[‘archiver.name’].folderpath =
— description: this is close to a straight copy of prototype.py and archives 
messages to a Maildir

— archiver filename: archiving/to_filesystem.py
- config options:
config[‘archiver.name’].folderpath =
config[‘archiver.name’].zip = True
- description: this writes emails to the file system, optionally zipped

— archiver filename: archiving/to_httppost.py
- config options:
config[‘archiver.name’].target_url =
config[‘archiver.name’].headers =
config[‘archiver.name’].url_parameters =
config[‘archiver.name’].form_fields =
config[‘archiver.name’].X_Message_ID_Hash_only =
- description: this (with Aurelien’s permission) will be based on Hyperkitty’s 
HTTP POST archiver and will behave in a similar manner.  Alternatively It’ll be 
written from scratch.

— archiver filename: archiving/to_smtp.py
- config options:
config[‘archiver.name’].destination_address =
- description: this is very close to a straight copy of mail_archive.py but 
will remove a few lines of code to make it a more generalised smtp sender

— archiver filename: archiving/to_nntp.py
- config options:
config[‘archiver.name’].server_address =
config[‘archiver.name’].username =
config[‘archiver.name’].password =
- description: this will use the standard library to post the archive message 
to an NNTP server

— archiver filename: archiving/to_git.py
- config options:
   config[‘archiver.name’].foldername
- description: this will write the email to a git repository on the local file 
system

— archiver filename: archiving/to_AMQP.py
- config options:
config[‘archiver.name’].server_address
config[‘archiver.name’].username =
config[‘archiver.name’].password =
config[‘archiver.name’].X_Message_ID_Hash_only =
config[‘archiver.name’].max_message_size =
- description: this will write the message to an AMQP server and optionally 
writes only the message id hash






___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/archive%40jab.org

Security Policy: http://wiki.list.org/x/QIA9

Re: [Mailman-Developers] Fine grained subscription control, scaling, noise, relevancy, subscription rules, dynamic lists

[Andrew Stuart]

@barry, @stephen, @terri

I  know this is a wall of text (sorry) but if you have time I’d be interested 
to hear your thoughts as Dynamic Sublists are in the GSOC projects list for 
this year.

as

On 20 Mar 2015, at 9:53 am, Andrew Stuart andrew.stu...@supercoders.com.au 
wrote:

Refer to the bottom of this email for some previous quotes from @barry on this 
topic. I’ve also had off list discussions with Barry in which he has mentioned 
this same topic so it seems to have some previous thought gone into it. 

I’m wanting to raise the topic of “fine grained subscription control” (for want 
of a better term) for discussion. Please note these thoughts refer to 
discussion style lists rather than notification style lists. Please prefix 
every sentence here with IMO - I’m not saying I’m right about anything, just 
putting forward food for thought. Also, I’m no Mailman expert so if my 
assumptions are plain wrong please let me know.

One of the core problems of mailing lists (at least as implemented in Mailman) 
is that there’s not much middle ground between being very involved (i.e receive 
all posts to list), or being almost-not-involved (i.e. receiving daily digests 
or no digest).

List noise and relevancy is the main problem and it’s a big impediment to lists 
being more widely used.

Very engaged users might be ok with constantly deleting messages that they are 
not interested in (i.e. irrelevant noise). Less engaged users will be annoyed 
with the noise and unsubscribe, or will switch to digest notification which I 
suggest to you is close to unsubscription anyway because that user no longer 
sees emails that they might have been interested in had they known there was a 
message on that subject. Digest users I suggest are at best observers rather 
than participators.

The more active the list, the greater the noise problem, the more users will 
drop out of the list.  As a list gains more subscribers, eventually even the 
most engaged user will have had enough of the volume of messages and will drop 
back to digest or write some mailbox rule that moves the emails to a folder, 
effectively dis-engaging them from the list conversation flow. Thus Mailman 
discussion-style lists currently don’t scale.

Even for low volume lists, noise is a major inhibitor to usage in certain 
contexts.  Consider for example the CIO of a company or the manager of a 
division of 30 developers. There are various mailing lists being used by the 
various projects that they are responsible for. The leader is not participating 
however because the noise from those lists would be overwhelming. They would 
however like to be partipcating in list discussions either that they initiated, 
are explicitly copied into, or relate to topics (keywords) that they are 
interested in. In reality, I’m not convinced Mailman would often be used in 
contexts like this currently because the relevancy/noise/digest/unsubscribe 
problem is a showstopper.

The systers have recognised this problem and their solution is Dynamic sublists 
as described here:
http://wiki.list.org/DEV/Dynamic%20Sublists
List subscribers can decide whether to be a part of new conversations or not. 
If the users decide to subscribe to new conversations, then they will receive 
all the messages of a new conversation unless they explicitly unsubscribe from 
it. If they otherwise decide to unsubscribe from new conversations,then they 
will receive only the first message of every new conversation unless they 
explicitly subscribe to it.”

I don’t think dynamic sublists are an effective solution (IMO, remember?).  
Getting the root message in all conversations is still too noisy and its a 
clumsy mechanic to have to opt in to further discussion and my thinking is that 
opt-in-to-discuss will impose a barrier to engagement that will reduce user 
interaction - put another way - “opt in to every discussion?  too hard”. 

Possibly a solution worth considering is fine-grained subscription control, in 
which there is a set of SEND and DONTSEND rules at a List default level and at 
a user level.

SEND:
Discussions where list member email address is in to/from/cc:
Discussions where to/from/cc contains one or more of: sa...@example.org, 
dav...@example.org, *.example.com
Discussions where subject contains keyword: hyperkitty
Discussions where body contains keyword: hyperkitty

DONTSEND:
Discussions where list member email address is in to/from/cc:
Discussions where to/from/cc contains one or more of: sa...@example.org, 
dav...@example.org
Discussions where subject contains keyword: Java
Discussions where body contains keyword: ruby

The technical hurdle to making this work is that Mailman needs access to 
historical messages to make it work (i.e. integrating some level of aqrchive 
like functionality into the Mailman database). I suggest this this may not be 
as hard as it sounds and hey, we’ve got a database there anyway so why not use 
it? 

One possibility is that this could be pushed

Re: [Mailman-Developers] Getting approval to go ahead

[Andrew Stuart]

Thanks Terri,

Unfortunately, it is true that the lack of process has led people to put a 
lot of work into stuff that never got merged.
I’m mainly just trying to avoid this - writing stuff that doesn’t get merged.  

I figure step one is to get a quick nod from one of the architects/project 
leads, so if my code is rejected its not because the idea is not compatible 
with the project goals. 

I’m less after a formal process. Looking for an informal “sounds good, go 
ahead”, or “that’s not going to fit with the project goals, maybe change it 
this way or that or ditch the idea altogether”.

The question is, who do I ask for confirmation that the idea is solid and will 
be merged assuming all requirements are met for code 
quality/testing/documentation?

thanks

as


On 24 Mar 2015, at 4:27 pm, Terri Oda te...@toybox.ca wrote:

On 2015-03-23 8:38 PM, Andrew Stuart wrote:
 I haven’t really worked on an open source project before.
 
 It wouldn’t make sense to come up with an idea, write some code, submit it 
 and have it rejected because it’s not OK with the project owners and doesn’t 
 fit with project goals.
 
 So at what point do I know that if I write some code that the concept is 
 solid and will be integrated into the project assuming the quality is there? 
 Do I need explicit approval from someone like Barry or Stephen?
 
 Also, is there a development workflow checklist or something I can use to 
 ensure that I am meeting all requirements for submitted code?

Sadly, we aren't quite formal enough to have a procedure beyond post your plan 
to the mailing list and make sure no one (particularly not Barry!) tells you 
'no'.

Unfortunately, it is true that the lack of process has led people to put a lot 
of work into stuff that never got merged.

If you'd feel more comfy with a bit more of a process, you might consider doing 
up something like a pep https://www.python.org/dev/peps/pep-0001/

For core stuff you particularly want Barry's blessing, for Hyperkitty it's 
Aurelian, for Postorius it's Florian or me.


Terri



___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/archive%40jab.org

Security Policy: http://wiki.list.org/x/QIA9

Re: [Mailman-Developers] Common use case archiving via configuration

[Andrew Stuart]

Aurelien I’d be interested to hear your thoughts on this topic.

as


On 22 Mar 2015, at 10:13 am, Andrew Stuart andrew.stu...@supercoders.com.au 
wrote:

As I understand it, currently there are a few default archivers including 
prototype, mhonarc and mailarchive. New archivers can be plugged in by 
implementing an archiver which complies to the archiver interface.  Not hard 
but still requires coding skills.

Specifically if would be good if a Mailman admin could define in the Mailman 
config file a set of destination email addresses and POST urls that archive 
messages are sent to. This would meet many common archiving needs via 
configuration instead of code. 

It’s a little odd that Mailman basically does this at the moment but only for 
mail-archive.com I’m proposing this be a bit more generalised and also that a 
generalised POST archiver be included.

I’d also include a filesystem/zip archiver which writes each message to a 
zipped file on the disk.

I’d be happy to implement assuming anyone else sees value in it, ideally with 
some architectural input from core developers.

Thoughts anyone? 

as


___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/archive%40jab.org

Security Policy: http://wiki.list.org/x/QIA9

Re: [Mailman-Developers] Small change to REST API for held subscription requests

[Andrew Stuart]

Any thoughts on how we could integrate with LDAP?

I’d be happy to authenticate against LDAP but that leaves the open question of 
how to synchronize the Mailman user database with the LDAP server.

Maybe of Mailman had some sort of event notification hook system for when users 
add/edit/delete.  Or if there was some sort of generalised message bus within 
the system that funnels messages about changes to the user database.  Something 
sort of LDAP synch process could then use those to synchronise an LDAP 
database. An event hook or message queue would allow changes to the users to be 
immediately pushed into LDAP.

Maybe the other possibility is some sort of direct integration such that 
Mailman, if given details of an LDAP server, can directly make changes to the 
LDAP system each time a user is added/edited/deleted.

There was some discussion about LDAP integration here:
https://mail.python.org/pipermail/mailman-developers/2015-February/024412.html

LDAP seems like a pretty configuration heavy thing to put in to a standlone 
Mailman system.  I’m currently researching to find some sort of 
minimla/lightweight LDAP integration approach.

as



On 23 Mar 2015, at 9:46 am, Barry Warsaw ba...@list.org wrote:

On Mar 23, 2015, at 09:34 AM, Andrew Stuart wrote:

 How would user passwords work if not stored in the core?
 
 When someone logs in currently I just ask the core if the password is valid
 and off we go.

WFM.  As I said, the core doesn't (currently) use it, but certainly it's
available through the REST API, so if REST clients need it, we'll keep it.

-Barry

___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/archive%40jab.org

Security Policy: http://wiki.list.org/x/QIA9

[Mailman-Developers] Getting approval to go ahead

[Andrew Stuart]

I haven’t really worked on an open source project before.

It wouldn’t make sense to come up with an idea, write some code, submit it and 
have it rejected because it’s not OK with the project owners and doesn’t fit 
with project goals.

So at what point do I know that if I write some code that the concept is solid 
and will be integrated into the project assuming the quality is there? Do I 
need explicit approval from someone like Barry or Stephen?

Also, is there a development workflow checklist or something I can use to 
ensure that I am meeting all requirements for submitted code?

thanks

as
___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/archive%40jab.org

Security Policy: http://wiki.list.org/x/QIA9

Re: [Mailman-Developers] Common use case archiving via configuration

[Andrew Stuart]

Thanks Aurélien

Would you mind if I used the Hyperkitty POST code as the basis for an example 
POST archiver for Mailman?

i.e. this:
https://github.com/hyperkitty/mailman-hyperkitty/blob/master/mailman_hyperkitty/__init__.py

I’m happy to rewrite from scratch but it seems to make sense to use what you 
have.

as

On 24 Mar 2015, at 12:45 pm, Aurelien Bompard aurel...@bompard.org wrote:

 Aurelien I’d be interested to hear your thoughts on this topic.

Well, the implementation of the archiver for HyperKitty does POST to
archive emails :
https://github.com/hyperkitty/mailman-hyperkitty/blob/master/mailman_hyperkitty/__init__.py
However, there are a few things specific to HyperKitty:
- authentication is done via a shared API key, it was easier than HTTP
basic auth because those tokens are not passed to the WSGI application
by default (at least not in Apache's mod_wsgi):
https://code.google.com/p/modwsgi/wiki/ConfigurationDirectives#WSGIPassAuthorization
- when requesting a message URL, the server only replies with the
local part, the server name is taken from the configuration. The
reason is that the client usually talks over localhost, and we want
the front-facing server name.

Doing something more generic is possible but would be a bit harder,
and the current archiving plugin is still rather short.
I'm not sure we can design an HTTP API for every need at the moment.

Aurélien

___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/archive%40jab.org

Security Policy: http://wiki.list.org/x/QIA9

Re: [Mailman-Developers] Small change to REST API for held subscription requests

[Andrew Stuart]

How would user passwords work if not stored in the core?

When someone logs in currently I just ask the core if the password is valid and 
off we go.

as


On 23 Mar 2015, at 8:19 am, Barry Warsaw ba...@list.org wrote:

On Mar 22, 2015, at 09:55 PM, Florian Fuchs wrote:

 As for the `password` key: It's recognized by mailman.client and
 exposed within the mlist instance's `requests` property. But it's used
 nowhere in postorius and I would be surprised if it's used anywhere in
 hyperkitty (Aurélien...?). I guess it can go away.

TBH, I desperately want to purge user passwords from core too!  I think
they're almost completely unused; the only reason to keep them would be to
help Postorious or other REST clients.

 As for the email/address keys: We can also just keep the `address` key
 in mailman.client and let it use the same value as `email`. That way
 we would still have a fallback but the REST API would stay clean and
 wouldn't send any redundant information over the wire.

Cool.

 Can you notify me when your API changes are pushed to LP, so I can
 update mailman.client accordingly? Just to keep the window as small as
 possible where we have non-compatible states in the
 mailman/mailman.client trunks. Especially during GSoC application
 week...

Will do.  I'm working on the subscription policy branch again, and this is a
useful refactoring I'm committing before other work.  I'm trying to decide
whether to commit this to trunk now or when subpolicy lands.  Stay tuned!

Cheers,
-Barry
___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/andrew.stuart%40supercoders.com.au

Security Policy: http://wiki.list.org/x/QIA9

___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/archive%40jab.org

Security Policy: http://wiki.list.org/x/QIA9

Re: [Mailman-Developers] Small change to REST API for held subscription requests

[Andrew Stuart]

Fine with me.

as


On 22 Mar 2015, at 12:32 pm, Barry Warsaw ba...@list.org wrote:

I want to make some minor backward incompatible changes to the JSON
representation for pending mailing lists requests, e.g. subscription holds.
You'll see these for example in urls like:

 dump_json('http://localhost:9001/3.0/lists/a...@example.com/requests')
   entry 0:
   delivery_mode: regular
   display_name: Anne Person
   email: a...@example.com
   http_etag: ...
   language: en
   request_id: ...
   type: subscription
   when: 2005-08-01T07:49:23

The first change is that `type: subscription` requests will change their
`address` key to `email` for consistency with other parts of the API.  Second,
there will be no `password` key.

Let me know if this is a problem, and I can copy `email` to `address` so that
both would appear, and add a dummy `password` key.

Cheers,
-Barry
___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/andrew.stuart%40supercoders.com.au

Security Policy: http://wiki.list.org/x/QIA9

___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/archive%40jab.org

Security Policy: http://wiki.list.org/x/QIA9

[Mailman-Developers] Common use case archiving via configuration

[Andrew Stuart]

As I understand it, currently there are a few default archivers including 
prototype, mhonarc and mailarchive. New archivers can be plugged in by 
implementing an archiver which complies to the archiver interface.  Not hard 
but still requires coding skills.

Specifically if would be good if a Mailman admin could define in the Mailman 
config file a set of destination email addresses and POST urls that archive 
messages are sent to. This would meet many common archiving needs via 
configuration instead of code. 

It’s a little odd that Mailman basically does this at the moment but only for 
mail-archive.com I’m proposing this be a bit more generalised and also that a 
generalised POST archiver be included.

I’d also include a filesystem/zip archiver which writes each message to a 
zipped file on the disk.

I’d be happy to implement assuming anyone else sees value in it, ideally with 
some architectural input from core developers.

Thoughts anyone? 

as

___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/archive%40jab.org

Security Policy: http://wiki.list.org/x/QIA9

[Mailman-Developers] Fine grained subscription control, scaling, noise, relevancy, subscription rules, dynamic lists

[Andrew Stuart]

Refer to the bottom of this email for some previous quotes from @barry on this 
topic. I’ve also had off list discussions with Barry in which he has mentioned 
this same topic so it seems to have some previous thought gone into it. 

I’m wanting to raise the topic of “fine grained subscription control” (for want 
of a better term) for discussion. Please note these thoughts refer to 
discussion style lists rather than notification style lists. Please prefix 
every sentence here with IMO - I’m not saying I’m right about anything, just 
putting forward food for thought. Also, I’m no Mailman expert so if my 
assumptions are plain wrong please let me know.

One of the core problems of mailing lists (at least as implemented in Mailman) 
is that there’s not much middle ground between being very involved (i.e receive 
all posts to list), or being almost-not-involved (i.e. receiving daily digests 
or no digest).

List noise and relevancy is the main problem and it’s a big impediment to lists 
being more widely used.

Very engaged users might be ok with constantly deleting messages that they are 
not interested in (i.e. irrelevant noise). Less engaged users will be annoyed 
with the noise and unsubscribe, or will switch to digest notification which I 
suggest to you is close to unsubscription anyway because that user no longer 
sees emails that they might have been interested in had they known there was a 
message on that subject. Digest users I suggest are at best observers rather 
than participators.

The more active the list, the greater the noise problem, the more users will 
drop out of the list.  As a list gains more subscribers, eventually even the 
most engaged user will have had enough of the volume of messages and will drop 
back to digest or write some mailbox rule that moves the emails to a folder, 
effectively dis-engaging them from the list conversation flow. Thus Mailman 
discussion-style lists currently don’t scale.

Even for low volume lists, noise is a major inhibitor to usage in certain 
contexts.  Consider for example the CIO of a company or the manager of a 
division of 30 developers. There are various mailing lists being used by the 
various projects that they are responsible for. The leader is not participating 
however because the noise from those lists would be overwhelming. They would 
however like to be partipcating in list discussions either that they initiated, 
are explicitly copied into, or relate to topics (keywords) that they are 
interested in. In reality, I’m not convinced Mailman would often be used in 
contexts like this currently because the relevancy/noise/digest/unsubscribe 
problem is a showstopper.

The systers have recognised this problem and their solution is Dynamic sublists 
as described here:
http://wiki.list.org/DEV/Dynamic%20Sublists
List subscribers can decide whether to be a part of new conversations or not. 
If the users decide to subscribe to new conversations, then they will receive 
all the messages of a new conversation unless they explicitly unsubscribe from 
it. If they otherwise decide to unsubscribe from new conversations,then they 
will receive only the first message of every new conversation unless they 
explicitly subscribe to it.”

I don’t think dynamic sublists are an effective solution (IMO, remember?).  
Getting the root message in all conversations is still too noisy and its a 
clumsy mechanic to have to opt in to further discussion and my thinking is that 
opt-in-to-discuss will impose a barrier to engagement that will reduce user 
interaction - put another way - “opt in to every discussion?  too hard”. 

Possibly a solution worth considering is fine-grained subscription control, in 
which there is a set of SEND and DONTSEND rules at a List default level and at 
a user level.

SEND:
Discussions where list member email address is in to/from/cc:
Discussions where to/from/cc contains one or more of: sa...@example.org, 
dav...@example.org, *.example.com
Discussions where subject contains keyword: hyperkitty
Discussions where body contains keyword: hyperkitty

DONTSEND:
Discussions where list member email address is in to/from/cc:
Discussions where to/from/cc contains one or more of: sa...@example.org, 
dav...@example.org
Discussions where subject contains keyword: Java
Discussions where body contains keyword: ruby

The technical hurdle to making this work is that Mailman needs access to 
historical messages to make it work (i.e. integrating some level of aqrchive 
like functionality into the Mailman database). I suggest this this may not be 
as hard as it sounds and hey, we’ve got a database there anyway so why not use 
it? 

One possibility is that this could be pushed into archiving but I don’t think 
that actually is practical - such concepts really need to be built deep into 
the core. I’m not advocating archiving-in-core here because I think archiving 
should like outside core.  I do think however that there is value in the core 
having access to 

[Mailman-Developers] Message threading - requesting some wisdom

[Andrew Stuart]

Hi folks

I assume a great deal of thought has gone into message threading and how it 
works. Is there anyone willing to share the basics with me - specifically as it 
relates to Mailman and related projects?

There is of course plenty of information on google which I am researching 
including the seminal http://www.jwz.org/doc/threading.html but if there is 
anyone on the Mailman team willing to give me a Mailman oriented understanding 
that would be great. It appears to me there are a number of ways to implement 
threading, and how it is done is somewhat arbitrary.  For example, this 
description of Gmail threading 
http://xkahn.zoned.net/software/evolution/threads/ seems to suggest that Gmail 
includes arbitrary business rules for threading based on message content (i.e. 
subject prefix).

In particular I want to know the basics of message threading - i.e. 

** What data defines how emails are connected to one another?
** How is that connecting data organised (i.e. in a tree, a flat list, by date, 
whatever)?
** What happens when thread data is missing (i.e. a message in a thread is 
deleted)?
** Are there any algorithms/mechanisms/patterns that are effective for 
implementing threading?
** Any code in Mailman/Hyperkitty/elsewhere that is particularly good to study?
** Anything else I should know?

I’m wanting to find the most simple and effective way to deal with threading 
which of course has the potential to be complex if I get it wrong.

thanks!

as

___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/archive%40jab.org

Security Policy: http://wiki.list.org/x/QIA9

Re: [Mailman-Developers] GSOC 2015 :Mailman Client written in Javascript

[Andrew Stuart]

Hi Abhishek,

If you are going to do this sort of development you will need to become 
familiar with tools for observing the traffic being sent from the web browser 
to the web server.

What sort of operating system are you using?

Right clicking in a browser window that you run your JavaScript code in will 
generally allow you to bring up Firebug or some similar set of developer tools. 
 There will be a “network” option there somewhere that allows you to see what 
is being sent between browser and server.  You will be able to observe the 
request headers at that point.

You should also install curl and become familiar with using it to send requests 
to the Mailman REST API.  Make sure you have a good look through the curl 
options because there are lots of helpful options there for looking at headers 
etc.

If you are using Chrome there is a plugin called postman that allows you to 
send request to REST API’s.  This is also helpful.

Generally when you are trying to diagnose problems talking to the Mailman REST 
API you should first use the tools mentioned above to make sure your request is 
structured properly. Once you are certain you can execute your request properly 
using one of the tools above, then it is time to try to make your JavaScript 
code send the same request.  If your code is not working, use Firebug to 
observe the difference between what you are sending and what you made work 
using the command line tools.

If you are doing node.js or some other sort of server executed JavaScript you 
will need another tool for watching the network traffic.  I presume you are 
running Mailman on a LInux machine. When diagnosing communications between 
client and server it is helopful on the Linux side to be able to see the 
network traffic passing to and from the Mailman REST API.  To do so, I use the 
following Linux command:

sudo ngrep -W byline -d lo port 7001

This assumes your Mailman REST API is running on port 7001 which it probably is 
not - I don’t know which port you are running it on - you will have to do a 
mailman info” command to find out.  

It will take you a while to become familiar with the output of ngrep and see 
what is being sent, but it is worth the effort as this is the most direct route 
to solving many problem.

Finally, to solve the current issue that you have with basic authentication 
against the Mailman REST API you should just search google for +javascript 
+basic +auth +stackoverflow - surely there must be some good instructions out 
there somewhere for how to make basic auth work - it should be pretty 
straightforward.

thanks

as






On 6 Mar 2015, at 7:20 pm, Abhishek Kumar abhi170...@gmail.com wrote:

Hi,

Sorry for the same last 2 mails. Last one was sent accidentally.
Are there any ideas regarding the issues i mentioned in them..? Abhilash..?

On Fri, Mar 6, 2015 at 2:38 AM, Abhishek Kumar abhi170...@gmail.com wrote:

 Hi,
 
 Thanks Aanand and Abhilash for the help and suggestions. I need further
 assistance:
 
 I tried to  write the connection class's rough javascript code. It works
 if i bypass the authentication. I have encoded/decoded the authentication
 data correctly ( as seemed from printing them and what python client
 generates through b64encode(auth.encode('utf-8')).decode('utf-8') ). If
 authentication is allowed, the javascript client shows 401,
 Unauthorized. The REST API requires authentication.
 This happens because the request.auth on line 68 in
 src/mailman/rest/root.py  is None (see in this
 http://bazaar.launchpad.net/~mailman-coders/mailman/3.0/view/head:/src/mailman/rest/root.py
  file)
 even when i am passing the authentication data in the http header . However
 there are no issues when i use the python client through shell  and
 everyhing works fine. I was unable to figure out the issue. Any ideas
 regarding what should i do here..? The  js code(uses some nodejs) can be
 seen here http://pastebin.com/isxPbsNk.
 
 Also i need some guidance on what should i do now towards applying for
 this project. Should i go on writing the javascript version by discussing
 it with the community..? And regarding what would go into my proposal, my
 current understanding stands at  mainly writing javascript equivalent of
 everything present in _client.py(mailman client) alongwith documentation
 and tetsting. What more is required to be done in this project..?
 
 
 
 
 On Thu, Mar 5, 2015 at 12:33 PM, Abhilash Raj raj.abhila...@gmail.com
 wrote:
 
 Hi Abhishek,
 
 On Wednesday 04 March 2015 07:43 PM, Abhishek Kumar wrote:
 Hi,
 
 I was able to setup the mailman (Core, postorius, mailman.client,
 postorius_standalone)  by downloading them individually. Also with
 superuser, i can create domains. I have some questions:
 
 1) When i try to create list i get : HTTP Error 500: A server error
 occurred. Please contact the administrator
 
 Try to find out what error is raised, check the logs in
 var/logs/mailman.log . This error is very common due to the absence to
 

Re: [Mailman-Developers] How do I turn on/off moderation of subscription requests?

[Andrew Stuart]

@barry - after digging around further I found Aurelien had the same question a 
while back 

http://comments.gmane.org/gmane.mail.mailman.devel/19301

I also found a message from you in Dec 2014 (can’t find it now after a few 
minutes digging….ugh) saying that currently it’s not possible to create lists 
via the REST API that require moderator approval to subscribe.  I think it 
refered to the subpolicy branch 
https://code.launchpad.net/~barry/mailman/subpolicy

Currently if I subscribe a new user to a list via the REST API I don’t see 
their subscription come up as a held request in:

GET 3.0/lists/{VAR}/requests
POST 3.0/lists/{VAR}/requests/{VAR}
PATCH 3.0/lists/{VAR}/requests/{VAR}

I think this makes sense assuming the subpolicy functionality is not yet 
implemented.

Am I on the right track in understanding that its not possible right now to 
create a list and hold and moderate subscription requests (via REST)?

thanks

as




On 6 Mar 2015, at 8:39 am, Andrew Stuart andrew.stu...@supercoders.com.au 
wrote:

I want to be able to turn off and on the need for moderator to approve 
subscription requests.

Would someone mind please pointing out where this is configured?

thanks

as
___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/andrew.stuart%40supercoders.com.au

Security Policy: http://wiki.list.org/x/QIA9

___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/archive%40jab.org

Security Policy: http://wiki.list.org/x/QIA9

Re: [Mailman-Developers] Group Photo from PyCon Mailman sprint last year

[Andrew Stuart]

Who is who is this pic?

as


On 4 Mar 2015, at 7:09 pm, Terri Oda te...@toybox.ca wrote:

Thought you all might want this for your wall. ;)

https://www.flickr.com/photos/kushaldas/13953404515/in/set-72157644185026745

And since we're about a month out and it's probably not too late to make travel 
arrangements, I'd like to remind you all that we're planning to sprint again 
this year at PyCon!

PyCon is an excellent conference and well worth attending, but even if you 
aren't attending the main conference, the sprints are free.  Come help us 
release Mailman 3, finally! ;)

Terri
___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/andrew.stuart%40supercoders.com.au

Security Policy: http://wiki.list.org/x/QIA9

___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/archive%40jab.org

Security Policy: http://wiki.list.org/x/QIA9

Re: [Mailman-Developers] GSOC 2015 :Mailman Client written in Javascript

[Andrew Stuart]

Hi Abhishek

I suggest you find who the mentors are for your project and ask them, mentors 
are listed on this page:
http://wiki.list.org/DEV/Google_Summer_of_Code_2015#Mailman_Client_written_in_Javascript


Regarding how to build a JavaScript SDK.  For inspiration perhaps you might 
start by looking at another JavaScript SDK such as and see how they have 
implemented it. Careful not to copy code from them in case there are license 
issues.

http://aws.amazon.com/sdk-for-node-js/

as


On 6 Mar 2015, at 9:05 pm, Abhishek Kumar abhi170...@gmail.com wrote:

Hi,

Thanks for the suggestions Andrew. Using ngrep i found that i was using 
authentication key in header instead of authorization. After correcting it, the 
js client runs successfully..! :)  I have tried curl in past and will see how 
it could be helpful to me while working on this task. 

Now that the authentication is resolved, i want to work forward. Some questions:

1) When we talk about the javascript version, is usage of frameworks like 
jquery suggested..? Till now i have experience of using javascript only in 
webpages. But here i am writing a client which is not supposed to run in 
browser. Please correct me if i am wrong. 
Currently postorius uses the mailman.client to talk to the rest api. How would 
it be able to use the client in js..? Or the development js client is for 
another purpose..? If yes , what's that and how the js client would integrate 
with the mailman which is python based.

2)I asked this in the 2nd last mail but it got hidden in the duplicate last 
mail. I need some guidance on what should i do now towards applying for this 
project. Should i go on writing the javascript version by discussing it with 
the community..? And regarding what would go into my proposal, my current 
understanding stands at  mainly writing javascript equivalent of everything 
present in _client.py(mailman client) alongwith documentation and tetsting. 
What more is required to be done in this project..? 



On Fri, Mar 6, 2015 at 2:38 PM, Andrew Stuart 
andrew.stu...@supercoders.com.au wrote:
Hi Abhishek,

If you are going to do this sort of development you will need to become 
familiar with tools for observing the traffic being sent from the web browser 
to the web server.

What sort of operating system are you using?

Right clicking in a browser window that you run your JavaScript code in will 
generally allow you to bring up Firebug or some similar set of developer tools. 
 There will be a “network” option there somewhere that allows you to see what 
is being sent between browser and server.  You will be able to observe the 
request headers at that point.

You should also install curl and become familiar with using it to send requests 
to the Mailman REST API.  Make sure you have a good look through the curl 
options because there are lots of helpful options there for looking at headers 
etc.

If you are using Chrome there is a plugin called postman that allows you to 
send request to REST API’s.  This is also helpful.

Generally when you are trying to diagnose problems talking to the Mailman REST 
API you should first use the tools mentioned above to make sure your request is 
structured properly. Once you are certain you can execute your request properly 
using one of the tools above, then it is time to try to make your JavaScript 
code send the same request.  If your code is not working, use Firebug to 
observe the difference between what you are sending and what you made work 
using the command line tools.

If you are doing node.js or some other sort of server executed JavaScript you 
will need another tool for watching the network traffic.  I presume you are 
running Mailman on a LInux machine. When diagnosing communications between 
client and server it is helopful on the Linux side to be able to see the 
network traffic passing to and from the Mailman REST API.  To do so, I use the 
following Linux command:

sudo ngrep -W byline -d lo port 7001

This assumes your Mailman REST API is running on port 7001 which it probably is 
not - I don’t know which port you are running it on - you will have to do a 
mailman info” command to find out.

It will take you a while to become familiar with the output of ngrep and see 
what is being sent, but it is worth the effort as this is the most direct route 
to solving many problem.

Finally, to solve the current issue that you have with basic authentication 
against the Mailman REST API you should just search google for +javascript 
+basic +auth +stackoverflow - surely there must be some good instructions out 
there somewhere for how to make basic auth work - it should be pretty 
straightforward.

thanks

as






On 6 Mar 2015, at 7:20 pm, Abhishek Kumar abhi170...@gmail.com wrote:

Hi,

Sorry for the same last 2 mails. Last one was sent accidentally.
Are there any ideas regarding the issues i mentioned in them..? Abhilash..?

On Fri, Mar 6, 2015 at 2:38 AM, Abhishek Kumar abhi170...@gmail.com wrote

[Mailman-Developers] How do I turn on/off moderation of subscription requests?

[Andrew Stuart]

I want to be able to turn off and on the need for moderator to approve 
subscription requests.

Would someone mind please pointing out where this is configured?

thanks

as
___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/archive%40jab.org

Security Policy: http://wiki.list.org/x/QIA9

Re: [Mailman-Developers] attempting to delete mailing list or domain gives sqlalchemy error

[Andrew Stuart]

Hi Abhilash,

Sorry I should add that I am wondering if the presence of “acceptable aliases” 
in the list configuration is causing a problem deleting the list that has those 
aliases.  Is that a possibility? The sqlalchemy error mentions acceptable 
aliases and also my database state contains acceptable aliases records for the 
list in question.

thanks

Andrew

On 5 Mar 2015, at 11:16 am, Andrew Stuart andrew.stu...@supercoders.com.au 
wrote:

Hi Abhilash

Requesting your help - do you have some time to help me with this one please? 
You seem to know a fair bit about the database and sqlalchemy.

Do you have any guesses as to what is going on here?  It’s becoming quite a 
problem for me because the database in its stuck state won’t complete my test 
suite.

thanks

Andrew



On 2 Mar 2015, at 7:41 am, Andrew Stuart andrew.stu...@supercoders.com.au 
wrote:

After I run my tests, the Mailman database seems to get into a state in which 
it is not possible to delete my test mailing list and test domain.

I figure it is something to do with the database state but I can’t see what at 
this stage.


#
###Attempting to delete a domain gives 
sqlalchemy error 
#

curl   -X DELETE --verbose --header authorization: Basic 
cmVzdGFkbWluOnJlc3RwYXNz http://localhost:8001/3.0/domains/mail.example.org

Traceback (most recent call last):
File 
/home/ubuntu/venv3.4/lib/python3.4/site-packages/SQLAlchemy-0.9.8-py3.4.egg/sqlalchemy/engine/base.py,
 line 951, in _execute_context
  context)
File 
/home/ubuntu/venv3.4/lib/python3.4/site-packages/SQLAlchemy-0.9.8-py3.4.egg/sqlalchemy/engine/default.py,
 line 436, in do_execute
  cursor.execute(statement, parameters)
sqlite3.IntegrityError: NOT NULL constraint failed: 
acceptablealias.mailing_list_id

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File /usr/lib/python3.4/wsgiref/handlers.py, line 137, in run
  self.result = application(self.environ, self.start_response)
File /home/ubuntu/mailman/src/mailman/database/transaction.py, line 57, in 
wrapper
  config.db.commit()
File /home/ubuntu/mailman/src/mailman/database/base.py, line 56, in commit
  self.store.commit()
File 
/home/ubuntu/venv3.4/lib/python3.4/site-packages/SQLAlchemy-0.9.8-py3.4.egg/sqlalchemy/orm/session.py,
 line 776, in commit
  self.transaction.commit()
File 
/home/ubuntu/venv3.4/lib/python3.4/site-packages/SQLAlchemy-0.9.8-py3.4.egg/sqlalchemy/orm/session.py,
 line 377, in commit
  self._prepare_impl()
File 
/home/ubuntu/venv3.4/lib/python3.4/site-packages/SQLAlchemy-0.9.8-py3.4.egg/sqlalchemy/orm/session.py,
 line 357, in _prepare_impl
  self.session.flush()
File 
/home/ubuntu/venv3.4/lib/python3.4/site-packages/SQLAlchemy-0.9.8-py3.4.egg/sqlalchemy/orm/session.py,
 line 1919, in flush
  self._flush(objects)
File 
/home/ubuntu/venv3.4/lib/python3.4/site-packages/SQLAlchemy-0.9.8-py3.4.egg/sqlalchemy/orm/session.py,
 line 2037, in _flush
  transaction.rollback(_capture_exception=True)
File 
/home/ubuntu/venv3.4/lib/python3.4/site-packages/SQLAlchemy-0.9.8-py3.4.egg/sqlalchemy/util/langhelpers.py,
 line 60, in __exit__
  compat.reraise(exc_type, exc_value, exc_tb)
File 
/home/ubuntu/venv3.4/lib/python3.4/site-packages/SQLAlchemy-0.9.8-py3.4.egg/sqlalchemy/util/compat.py,
 line 182, in reraise
  raise value
File 
/home/ubuntu/venv3.4/lib/python3.4/site-packages/SQLAlchemy-0.9.8-py3.4.egg/sqlalchemy/orm/session.py,
 line 2001, in _flush
  flush_context.execute()
File 
/home/ubuntu/venv3.4/lib/python3.4/site-packages/SQLAlchemy-0.9.8-py3.4.egg/sqlalchemy/orm/unitofwork.py,
 line 372, in execute
  rec.execute(self)
File 
/home/ubuntu/venv3.4/lib/python3.4/site-packages/SQLAlchemy-0.9.8-py3.4.egg/sqlalchemy/orm/unitofwork.py,
 line 526, in execute
  uow
File 
/home/ubuntu/venv3.4/lib/python3.4/site-packages/SQLAlchemy-0.9.8-py3.4.egg/sqlalchemy/orm/persistence.py,
 line 60, in save_obj
  mapper, table, update)
File 
/home/ubuntu/venv3.4/lib/python3.4/site-packages/SQLAlchemy-0.9.8-py3.4.egg/sqlalchemy/orm/persistence.py,
 line 518, in _emit_update_statements
  execute(statement, params)
File 
/home/ubuntu/venv3.4/lib/python3.4/site-packages/SQLAlchemy-0.9.8-py3.4.egg/sqlalchemy/engine/base.py,
 line 729, in execute
  return meth(self, multiparams, params)
File 
/home/ubuntu/venv3.4/lib/python3.4/site-packages/SQLAlchemy-0.9.8-py3.4.egg/sqlalchemy/sql/elements.py,
 line 322, in _execute_on_connection
  return connection._execute_clauseelement(self, multiparams, params)
File 
/home/ubuntu/venv3.4/lib/python3.4/site-packages/SQLAlchemy-0.9.8-py3.4.egg/sqlalchemy/engine/base.py,
 line 826, in _execute_clauseelement
  compiled_sql, distilled_params
File 
/home/ubuntu/venv3.4/lib/python3.4/site-packages/SQLAlchemy-0.9.8-py3.4

Re: [Mailman-Developers] attempting to delete mailing list or domain gives sqlalchemy error

[Andrew Stuart]

Hi Abhilash

Requesting your help - do you have some time to help me with this one please? 
You seem to know a fair bit about the database and sqlalchemy.

Do you have any guesses as to what is going on here?  It’s becoming quite a 
problem for me because the database in its stuck state won’t complete my test 
suite.

thanks

Andrew



On 2 Mar 2015, at 7:41 am, Andrew Stuart andrew.stu...@supercoders.com.au 
wrote:

After I run my tests, the Mailman database seems to get into a state in which 
it is not possible to delete my test mailing list and test domain.

I figure it is something to do with the database state but I can’t see what at 
this stage.


#
###Attempting to delete a domain gives 
sqlalchemy error 
#

curl   -X DELETE --verbose --header authorization: Basic 
cmVzdGFkbWluOnJlc3RwYXNz http://localhost:8001/3.0/domains/mail.example.org

Traceback (most recent call last):
 File 
/home/ubuntu/venv3.4/lib/python3.4/site-packages/SQLAlchemy-0.9.8-py3.4.egg/sqlalchemy/engine/base.py,
 line 951, in _execute_context
   context)
 File 
/home/ubuntu/venv3.4/lib/python3.4/site-packages/SQLAlchemy-0.9.8-py3.4.egg/sqlalchemy/engine/default.py,
 line 436, in do_execute
   cursor.execute(statement, parameters)
sqlite3.IntegrityError: NOT NULL constraint failed: 
acceptablealias.mailing_list_id

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
 File /usr/lib/python3.4/wsgiref/handlers.py, line 137, in run
   self.result = application(self.environ, self.start_response)
 File /home/ubuntu/mailman/src/mailman/database/transaction.py, line 57, in 
wrapper
   config.db.commit()
 File /home/ubuntu/mailman/src/mailman/database/base.py, line 56, in commit
   self.store.commit()
 File 
/home/ubuntu/venv3.4/lib/python3.4/site-packages/SQLAlchemy-0.9.8-py3.4.egg/sqlalchemy/orm/session.py,
 line 776, in commit
   self.transaction.commit()
 File 
/home/ubuntu/venv3.4/lib/python3.4/site-packages/SQLAlchemy-0.9.8-py3.4.egg/sqlalchemy/orm/session.py,
 line 377, in commit
   self._prepare_impl()
 File 
/home/ubuntu/venv3.4/lib/python3.4/site-packages/SQLAlchemy-0.9.8-py3.4.egg/sqlalchemy/orm/session.py,
 line 357, in _prepare_impl
   self.session.flush()
 File 
/home/ubuntu/venv3.4/lib/python3.4/site-packages/SQLAlchemy-0.9.8-py3.4.egg/sqlalchemy/orm/session.py,
 line 1919, in flush
   self._flush(objects)
 File 
/home/ubuntu/venv3.4/lib/python3.4/site-packages/SQLAlchemy-0.9.8-py3.4.egg/sqlalchemy/orm/session.py,
 line 2037, in _flush
   transaction.rollback(_capture_exception=True)
 File 
/home/ubuntu/venv3.4/lib/python3.4/site-packages/SQLAlchemy-0.9.8-py3.4.egg/sqlalchemy/util/langhelpers.py,
 line 60, in __exit__
   compat.reraise(exc_type, exc_value, exc_tb)
 File 
/home/ubuntu/venv3.4/lib/python3.4/site-packages/SQLAlchemy-0.9.8-py3.4.egg/sqlalchemy/util/compat.py,
 line 182, in reraise
   raise value
 File 
/home/ubuntu/venv3.4/lib/python3.4/site-packages/SQLAlchemy-0.9.8-py3.4.egg/sqlalchemy/orm/session.py,
 line 2001, in _flush
   flush_context.execute()
 File 
/home/ubuntu/venv3.4/lib/python3.4/site-packages/SQLAlchemy-0.9.8-py3.4.egg/sqlalchemy/orm/unitofwork.py,
 line 372, in execute
   rec.execute(self)
 File 
/home/ubuntu/venv3.4/lib/python3.4/site-packages/SQLAlchemy-0.9.8-py3.4.egg/sqlalchemy/orm/unitofwork.py,
 line 526, in execute
   uow
 File 
/home/ubuntu/venv3.4/lib/python3.4/site-packages/SQLAlchemy-0.9.8-py3.4.egg/sqlalchemy/orm/persistence.py,
 line 60, in save_obj
   mapper, table, update)
 File 
/home/ubuntu/venv3.4/lib/python3.4/site-packages/SQLAlchemy-0.9.8-py3.4.egg/sqlalchemy/orm/persistence.py,
 line 518, in _emit_update_statements
   execute(statement, params)
 File 
/home/ubuntu/venv3.4/lib/python3.4/site-packages/SQLAlchemy-0.9.8-py3.4.egg/sqlalchemy/engine/base.py,
 line 729, in execute
   return meth(self, multiparams, params)
 File 
/home/ubuntu/venv3.4/lib/python3.4/site-packages/SQLAlchemy-0.9.8-py3.4.egg/sqlalchemy/sql/elements.py,
 line 322, in _execute_on_connection
   return connection._execute_clauseelement(self, multiparams, params)
 File 
/home/ubuntu/venv3.4/lib/python3.4/site-packages/SQLAlchemy-0.9.8-py3.4.egg/sqlalchemy/engine/base.py,
 line 826, in _execute_clauseelement
   compiled_sql, distilled_params
 File 
/home/ubuntu/venv3.4/lib/python3.4/site-packages/SQLAlchemy-0.9.8-py3.4.egg/sqlalchemy/engine/base.py,
 line 958, in _execute_context
   context)
 File 
/home/ubuntu/venv3.4/lib/python3.4/site-packages/SQLAlchemy-0.9.8-py3.4.egg/sqlalchemy/engine/base.py,
 line 1159, in _handle_dbapi_exception
   exc_info
 File 
/home/ubuntu/venv3.4/lib/python3.4/site-packages/SQLAlchemy-0.9.8-py3.4.egg/sqlalchemy/util/compat.py,
 line 188, in raise_from_cause
   reraise(type(exception

[Mailman-Developers] GSOC 2015: Message queue based email archiver

[Andrew Stuart]

The wiki says:

Hyperkitty is mailman's official archiver, however it would be a good idea 
to add an alternative to it based on message queues, either 
publish-subscribe or a fifo message queue.

Florian/Aurelien,

I’m curious to understand what would be the benefit of message queue based 
archiving and how do you see such a thing working? Is there some problem with 
the existing archiving that this would solve?

thanks

as

___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/archive%40jab.org

Security Policy: http://wiki.list.org/x/QIA9

[Mailman-Developers] attempting to delete mailing list or domain gives sqlalchemy error

[Andrew Stuart]

After I run my tests, the Mailman database seems to get into a state in which 
it is not possible to delete my test mailing list and test domain.

I figure it is something to do with the database state but I can’t see what at 
this stage.


#
###Attempting to delete a domain gives 
sqlalchemy error 
#

curl   -X DELETE --verbose --header authorization: Basic 
cmVzdGFkbWluOnJlc3RwYXNz http://localhost:8001/3.0/domains/mail.example.org

Traceback (most recent call last):
  File 
/home/ubuntu/venv3.4/lib/python3.4/site-packages/SQLAlchemy-0.9.8-py3.4.egg/sqlalchemy/engine/base.py,
 line 951, in _execute_context
context)
  File 
/home/ubuntu/venv3.4/lib/python3.4/site-packages/SQLAlchemy-0.9.8-py3.4.egg/sqlalchemy/engine/default.py,
 line 436, in do_execute
cursor.execute(statement, parameters)
sqlite3.IntegrityError: NOT NULL constraint failed: 
acceptablealias.mailing_list_id

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File /usr/lib/python3.4/wsgiref/handlers.py, line 137, in run
self.result = application(self.environ, self.start_response)
  File /home/ubuntu/mailman/src/mailman/database/transaction.py, line 57, in 
wrapper
config.db.commit()
  File /home/ubuntu/mailman/src/mailman/database/base.py, line 56, in commit
self.store.commit()
  File 
/home/ubuntu/venv3.4/lib/python3.4/site-packages/SQLAlchemy-0.9.8-py3.4.egg/sqlalchemy/orm/session.py,
 line 776, in commit
self.transaction.commit()
  File 
/home/ubuntu/venv3.4/lib/python3.4/site-packages/SQLAlchemy-0.9.8-py3.4.egg/sqlalchemy/orm/session.py,
 line 377, in commit
self._prepare_impl()
  File 
/home/ubuntu/venv3.4/lib/python3.4/site-packages/SQLAlchemy-0.9.8-py3.4.egg/sqlalchemy/orm/session.py,
 line 357, in _prepare_impl
self.session.flush()
  File 
/home/ubuntu/venv3.4/lib/python3.4/site-packages/SQLAlchemy-0.9.8-py3.4.egg/sqlalchemy/orm/session.py,
 line 1919, in flush
self._flush(objects)
  File 
/home/ubuntu/venv3.4/lib/python3.4/site-packages/SQLAlchemy-0.9.8-py3.4.egg/sqlalchemy/orm/session.py,
 line 2037, in _flush
transaction.rollback(_capture_exception=True)
  File 
/home/ubuntu/venv3.4/lib/python3.4/site-packages/SQLAlchemy-0.9.8-py3.4.egg/sqlalchemy/util/langhelpers.py,
 line 60, in __exit__
compat.reraise(exc_type, exc_value, exc_tb)
  File 
/home/ubuntu/venv3.4/lib/python3.4/site-packages/SQLAlchemy-0.9.8-py3.4.egg/sqlalchemy/util/compat.py,
 line 182, in reraise
raise value
  File 
/home/ubuntu/venv3.4/lib/python3.4/site-packages/SQLAlchemy-0.9.8-py3.4.egg/sqlalchemy/orm/session.py,
 line 2001, in _flush
flush_context.execute()
  File 
/home/ubuntu/venv3.4/lib/python3.4/site-packages/SQLAlchemy-0.9.8-py3.4.egg/sqlalchemy/orm/unitofwork.py,
 line 372, in execute
rec.execute(self)
  File 
/home/ubuntu/venv3.4/lib/python3.4/site-packages/SQLAlchemy-0.9.8-py3.4.egg/sqlalchemy/orm/unitofwork.py,
 line 526, in execute
uow
  File 
/home/ubuntu/venv3.4/lib/python3.4/site-packages/SQLAlchemy-0.9.8-py3.4.egg/sqlalchemy/orm/persistence.py,
 line 60, in save_obj
mapper, table, update)
  File 
/home/ubuntu/venv3.4/lib/python3.4/site-packages/SQLAlchemy-0.9.8-py3.4.egg/sqlalchemy/orm/persistence.py,
 line 518, in _emit_update_statements
execute(statement, params)
  File 
/home/ubuntu/venv3.4/lib/python3.4/site-packages/SQLAlchemy-0.9.8-py3.4.egg/sqlalchemy/engine/base.py,
 line 729, in execute
return meth(self, multiparams, params)
  File 
/home/ubuntu/venv3.4/lib/python3.4/site-packages/SQLAlchemy-0.9.8-py3.4.egg/sqlalchemy/sql/elements.py,
 line 322, in _execute_on_connection
return connection._execute_clauseelement(self, multiparams, params)
  File 
/home/ubuntu/venv3.4/lib/python3.4/site-packages/SQLAlchemy-0.9.8-py3.4.egg/sqlalchemy/engine/base.py,
 line 826, in _execute_clauseelement
compiled_sql, distilled_params
  File 
/home/ubuntu/venv3.4/lib/python3.4/site-packages/SQLAlchemy-0.9.8-py3.4.egg/sqlalchemy/engine/base.py,
 line 958, in _execute_context
context)
  File 
/home/ubuntu/venv3.4/lib/python3.4/site-packages/SQLAlchemy-0.9.8-py3.4.egg/sqlalchemy/engine/base.py,
 line 1159, in _handle_dbapi_exception
exc_info
  File 
/home/ubuntu/venv3.4/lib/python3.4/site-packages/SQLAlchemy-0.9.8-py3.4.egg/sqlalchemy/util/compat.py,
 line 188, in raise_from_cause
reraise(type(exception), exception, tb=exc_tb, cause=exc_value)
  File 
/home/ubuntu/venv3.4/lib/python3.4/site-packages/SQLAlchemy-0.9.8-py3.4.egg/sqlalchemy/util/compat.py,
 line 181, in reraise
raise value.with_traceback(tb)
  File 
/home/ubuntu/venv3.4/lib/python3.4/site-packages/SQLAlchemy-0.9.8-py3.4.egg/sqlalchemy/engine/base.py,
 line 951, in _execute_context
context)
  File 

Re: [Mailman-Developers] GSoC 2015: brainstorming ideas suitable for beginners?

[Andrew Stuart]

There’s a bug at:

https://bugs.launchpad.net/mailman/+bug/1423756

Requesting domainowner and serverowner

thanks

On 26 Feb 2015, at 10:10 am, Barry Warsaw ba...@list.org wrote:

On Feb 25, 2015, at 04:12 PM, Andrew Stuart wrote:

 I’m looking forward to being able to set and get domainowner and serverowner
 (or siteowner or whatever its called). It will allow me to delete lots of
 code and there’s no greater joy than deleting code.
 
 Are you anticipating this will be in V3.0?

Please file a bug and we'll see. ;)

Cheers,
-Barry

___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/andrew.stuart%40supercoders.com.au

Security Policy: http://wiki.list.org/x/QIA9

___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/archive%40jab.org

Security Policy: http://wiki.list.org/x/QIA9

Re: [Mailman-Developers] Ideas for injecting test messages

[Andrew Stuart]

Perhaps. I’m working close to full time on programming with Mailman and 
questions do come up.  I’ve had no previous experience of Mailman and for the 
most part I feel do all my own research and problem resolution.  I dig deep 
into the source and generally I think I ask questions when I’ve got to the 
bottom of the source code and still can’t understand why something is working 
in the way it does.

I know I’m more active on the list at the moment than anyone else.  However, 
this is the only case that I’m aware of that I’ve asked a question that some 
research on Google could have answered.

If you’d like less activity that’s OK I’ll take that on board.  This however is 
likely to be the outcome of Mailman 3 coming to completion with interfaces that 
others programmers can work with- active development along with lots of 
questions. The whole Mailman community is likely to become more noisy if you 
have succeeded in your goal of building something valuable, which I think is 
true.

I’ll take feedback from anyone on this - I don’t mind hearing what people 
think.  If you feel like you’d like the list to be less active from me that’s 
fine - if you think my questions lack research and effort that’s OK too.  Just 
let me know.

as



On 25 Feb 2015, at 10:18 pm, Abhilash Raj raj.abhila...@gmail.com wrote:

Hi Andrew,

I am not sure how do I put this, but I think you should spend little
more time investigating
about the questions you send in to this list. Questions and answers
seperated by a 3 hour
window makes me feel like its not even a question, but a personel log.
I agree mailman-devs
can answer your queries more quickly, but if you can find answers from
documentation then
there is no point in sending in these queries. It would be easier if
you collated your queries and
send-in one email in a day or two?

Also, I really appriciate the authentication proxy that you are trying
to build for mailman, please
don't take this as any form of discouragement. These are my personel
thoughts and in no way the
opinion of the whole mailman community.

If anyone feels this was totally unnecessary, please correct me.

On 25 February 2015 at 07:57, Andrew Stuart
andrew.stu...@supercoders.com.au wrote:
 Please disregard.  Apparently there’s websites that let you search the 
 Internet, through which I found:
 
 https://pythonhosted.org/mailman/src/mailman/commands/docs/inject.html
 
 
 On 25 Feb 2015, at 10:04 am, Andrew Stuart andrew.stu...@supercoders.com.au 
 wrote:
 
 I want to inject some test messages into Mailman to test moderation functions 
 via the REST API.
 
 Any ideas on a simple way to do that?
 
 Ideally I need to be able to clear them out somehow too.
 
 thanks
 
 as
 
 ___
 Mailman-Developers mailing list
 Mailman-Developers@python.org
 https://mail.python.org/mailman/listinfo/mailman-developers
 Mailman FAQ: http://wiki.list.org/x/AgA3
 Searchable Archives: 
 http://www.mail-archive.com/mailman-developers%40python.org/
 Unsubscribe: 
 https://mail.python.org/mailman/options/mailman-developers/andrew.stuart%40supercoders.com.au
 
 Security Policy: http://wiki.list.org/x/QIA9
 
 ___
 Mailman-Developers mailing list
 Mailman-Developers@python.org
 https://mail.python.org/mailman/listinfo/mailman-developers
 Mailman FAQ: http://wiki.list.org/x/AgA3
 Searchable Archives: 
 http://www.mail-archive.com/mailman-developers%40python.org/
 Unsubscribe: 
 https://mail.python.org/mailman/options/mailman-developers/raj.abhilash1%40gmail.com
 
 Security Policy: http://wiki.list.org/x/QIA9



-- 
thanks,
Abhilash Raj

___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/archive%40jab.org

Security Policy: http://wiki.list.org/x/QIA9

[Mailman-Developers] Ideas for injecting test messages

[Andrew Stuart]

I want to inject some test messages into Mailman to test moderation functions 
via the REST API.

Any ideas on a simple way to do that?

Ideally I need to be able to clear them out somehow too.

thanks

as

___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/archive%40jab.org

Security Policy: http://wiki.list.org/x/QIA9

Re: [Mailman-Developers] GSoC 2015: brainstorming ideas suitable for beginners?

[Andrew Stuart]

I’m looking forward to being able to set and get domainowner and serverowner 
(or siteowner or whatever its called). It will allow me to delete lots of code 
and there’s no greater joy than deleting code.

Are you anticipating this will be in V3.0?

thanks

as


On 24 Feb 2015, at 11:42 am, Barry Warsaw ba...@list.org wrote:

On Feb 22, 2015, at 11:20 AM, Andrew Stuart wrote:

 there was an “application data” table:
 
 
 resource_type  |  resource_id   |   private/public  |  key   |value 
 
 user
 list
 domain
 server
 
 and:
 the ability to set domainowner and serverowner permissions in 3.0,

No question, I think domain owner and server owner (or site owner) are
concepts that we should capture in the core.  I'd accept contributions for
this, so let me describe my thinking (open to suggestions!).

The first question is whether to associate addresses, users, or both with the
permissions.  If you look at the member table (Member model class) you'll see
we have a foreign key for both, but the logic ensure that we can't get
conflicting values.  This allows us to implement the address as a member or
the user as a member using their preferred address.  I'm not sure whether we
need this level of complexity for the site and domain owners.

The other observation is that we'd like to make it easy for both owners two
have multiple contact addresses, so that we have fallbacks in case one of them
starts to fail.

So I'm thinking we use a User for this association.  The user would have to
have at least one validated address linked to it.

For domains, we already have a domain table (Domain model class).  Since we'd
like to be able to associate multiple domain owners, it seems like we need a
many-to-many table associating domains to users.  (More than one user can own
a domain, and any user can own more than one domain.)  That table probably
doesn't need anything else.

Since we have no site table/model, I think it would be enough to add a flag to
the user table/model to indicate whether they are a site owner or not.

Now sprinkle with tests, database migrations, documentation, and REST, and I
think you have at least this part solved.

Cheers,
-Barry
___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/andrew.stuart%40supercoders.com.au

Security Policy: http://wiki.list.org/x/QIA9

___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/archive%40jab.org

Security Policy: http://wiki.list.org/x/QIA9

Re: [Mailman-Developers] Auth databases redux [was: GSoC 2015: brainstorming ... beginners?]

[Andrew Stuart]

Yes that’s a reasonable line of thought Stephen. You are correct I haven’t made 
any attempt to integrate with enterprise auth systems. Also, I’ll avoid 
describing my auth server as ‘the auth server’.

Is there anything that would define what sort of functionality is required for 
enterprise integration?

The two things that I’ve suggested might be worth including in the core are 
hopefully of broad value and not specific only to what I’m doing. (rounding out 
the permissions,  with serverowner and domainowner, including an application 
data table that stores keys and values).

thanks

as




On 22 Feb 2015, at 9:36 pm, Stephen J. Turnbull step...@xemacs.org wrote:

Andrew Stuart writes:

 Do you think this has to be integrated into the core for 3.0?
 I'd still prefer to keep it separate for better experimentation
 and testing.

 the alternative for me is to go ahead with deploying application
 data and domainowner and serverowner into the auth server,

I disagree with the implicit assumption right there.  From Mailman's
point of view, this is *Andrew's* auth server, not *the* auth server.
If we're serious about serving the enterprise, we need to figure these
things out、because Mailman is going to need to plug into an existing
auth and user database system.

I don't know if we care any more, but enterprise integration used to
be a goal.  It's still a requirement of some of our most loyal users
(universities, who FAQ us about using their databases to populate
lists for classes etc every year).


___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/archive%40jab.org

Security Policy: http://wiki.list.org/x/QIA9

Re: [Mailman-Developers] GSoC 2015: brainstorming ideas suitable for beginners?

[Andrew Stuart]

Do you think this has to be integrated into the core for 3.0?  I'd still 
prefer to keep it separate for better experimentation and testing. 

if:
 there was an “application data” table:


resource_type  |  resource_id   |   private/public  |  key   |value 

user
list
domain
server

and:
 the ability to set domainowner and serverowner permissions in 3.0,

then:
 there would be a huge payoff in terms of avoiding the downstream challenge of 
migration upgrade scripts, data migrations, testing, backups, the risk of 
something going wrong for the end-user sysadmin. 

else:
 the alternative for me is to go ahead with deploying application data and 
domainowner and serverowner into the auth server, and later needing to work out 
how to migrate these back into Mailman core in some later version - I’m 
thinking that if those tables were there in Mailman 3.0 rather than say version 
3.1 then a vast amount of downstream work, risk and thinking could be saved 
(for me). Not insurmountable but would really take alot of engineering to do 
safely. Getting random sysadmins to do stuff like data migrations that is a 
little anxiety provoking.

David Murray, who does the Python email package, implements APIs as 
“provisional” so they are not part of the official release but are present in 
the code, until some version further down the track where he formalises the 
provisional API following road testing and feedback.  Sort of a logical 
separation for experimentation and testing. Could that be a way to implement an 
application data table and serverowner and domainowner permissions in 3.0?  
That way, consumers of the REST API can use the provisional API functionality 
at our own risk that it might change before being formalised. I’d take that any 
day over the need to write stuff that a random sysadmin could run to migrate 
their data into a new table structure later.

Implementing an application data table and domainowner/serverowner as 
“provisional” would be a stitch in time for me anyway.

As I mentioned I’ve done it myself anyway in the auth server but theres a bunch 
of unaddressed challenges relating to replication and synchronisation - 
whenever the application data is accessed, there needs to be checks to verify 
that both the user and the resource actually exist in Mailman, and there’s no 
easy way to know when users and resources have been deleted from Mailman, 
potentially leaving stale application data.

As always though, I’m not super attached to the outcomes - you’re the boss 
Barry so I’m OK with whatever path.

as




On 22 Feb 2015, at 5:05 am, Barry Warsaw ba...@list.org wrote:

On Feb 20, 2015, at 07:25 PM, Andrew Stuart wrote:

 It’s workable as a part of the auth proxy but feels like it would fit better
 in the Mailman core database since the data is so tightly bound to Mailman
 resources.  It’ll need an effective replication mechanism to ensure
 consistency with Mailman which is a challenge I’m not relishing solving.

Do you think this has to be integrated into the core for 3.0?  I'd still
prefer to keep it separate for better experimentation and testing.  If the
design and API proves stable, then perhaps it'll make sense to integrate it
with the core for 3.1.

Cheers,
-Barry

___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/andrew.stuart%40supercoders.com.au

Security Policy: http://wiki.list.org/x/QIA9

___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/archive%40jab.org

Security Policy: http://wiki.list.org/x/QIA9

Re: [Mailman-Developers] GSoC 2015: brainstorming ideas suitable for beginners?

[Andrew Stuart]

 [1] The additional user info bit possibly re-raises the big question of a 
 central user data store for all Mailman components, which will clearly not 
 be solved as part of a GSoC project, especially a beginner-friendly one. 
 But we do already have a little bit of info, the user's email address and 
 the gravatar (if there is one). So all we'd have to store is an opt-in flag 
 somewhere in the Django-DB.

I have implemented a central data store into the REST proxy server.  It’s a 
simple table structure as follows:


resource_type  |  resource_id   |   private/public  |  key   |value 

user
list
domain
server

User access permissions for the data rows is the same as access to the 
resource_type  resource_id

It’s workable as a part of the auth proxy but feels like it would fit better in 
the Mailman core database since the data is so tightly bound to Mailman 
resources.  It’ll need an effective replication mechanism to ensure consistency 
with Mailman which is a challenge I’m not relishing solving.

as





On 18 Feb 2015, at 4:42 am, Florian Fuchs f...@florianfuchs.com wrote:

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1



Am 17.02.2015 um 10:03 schrieb Terri Oda:
 I'd love some help brainstorming some actual beginner-friendly
 ideas, though, because I'm currently all idea'd out.  Anyone want
 to suggest some simpler things they'd like to see in Mailman that
 we could maybe put on the list?

Just from the top of my head, either as a combination of the two or as
single projects (if fleshed out a bit further):

- - A Dashboard for Admins/Owners/Moderators: A single page that lists
all to dos in one place (subscription requests, held messages).
Probably useful for owners/moderators of multiple lists.

- - Subscriber pages: Let users opt-in to show their membership to other
list members. Possibly along with some additional user information[1].
This could be useful for more private lists if you want to get an idea
who your audience is.


Florian


[1] The additional user info bit possibly re-raises the big question
of a central user data store for all Mailman components, which will
clearly not be solved as part of a GSoC project, especially a
beginner-friendly one. But we do already have a little bit of info,
the user's email address and the gravatar (if there is one). So all
we'd have to store is an opt-in flag somewhere in the Django-DB.




 
 Suggesting and idea does not mean you have to volunteer to mentor
 them, although I'm always looking for new mentors too!
 
 Terri
 
 ___ Mailman-Developers
 mailing list Mailman-Developers@python.org 
 https://mail.python.org/mailman/listinfo/mailman-developers Mailman
 FAQ: http://wiki.list.org/x/AgA3 Searchable Archives: 
 http://www.mail-archive.com/mailman-developers%40python.org/ 
 Unsubscribe: 
 https://mail.python.org/mailman/options/mailman-developers/f%40florianfuchs.com
 
 
 
 Security Policy: http://wiki.list.org/x/QIA9
-BEGIN PGP SIGNATURE-
Version: GnuPG v1

iQEcBAEBAgAGBQJU432YAAoJEEceGbPdavl7CqEH/AvOLwc+qyWkmNw6Tahm6Jjb
IHVnz4yv4sTFmdB9iqYXkOQqt7XgpEoIYUZwV+j7+vltDSYSh2qecKyVyQjBkEUC
wc7hU8/czOw2E/I3MG1siRzCGeekGJSTFXf2NoKksUNca9DUQe0leNKs1VWXwaX5
uAsyitJxaiPJIsrKHeVhGO6g4rSmqBMi+Q+HTaBIZQqZZd1Qh7ZgUV4qXFuTjUXw
D4Ae20sdFAdLD412e3LrgI58JiXRNIHBfWzN1Zg0RoY95rw/AuDafrlPT37I2E7O
mahaaJAJrmPO3EimeDPmj1TTV7NWOpaU9Vlzd6H5BX4KweUxAaafYHFrkYXN/pU=
=L+aD
-END PGP SIGNATURE-
___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/andrew.stuart%40supercoders.com.au

Security Policy: http://wiki.list.org/x/QIA9

___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/archive%40jab.org

Security Policy: http://wiki.list.org/x/QIA9

[Mailman-Developers] Should REST API /addresses data be including the user_id field?

[Andrew Stuart]

This command:
curl   --header authorization: Basic cmVzdGFkbWluOnJlc3RwYXNz 
http://localhost:8001/3.0/addresses

Returns this data:
{entries: [{http_etag: \3045a243ea206ca6e112269f4c0ce8c4a66a92bf\, 
registered_on: 2015-02-19T03:52:55.460036, display_name: Server Owner, 
self_link: http://localhost:8001/3.0/addresses/theb...@example.org;, 
email: theb...@example.org, user: 
http://localhost:8001/3.0/users/49955403891644350774308934370095835248;, 
original_email: theb...@example.org}, {http_etag: 
\140a3ed1b92f9e14eab51b1183ba87c271191a0c\, registered_on: 
2015-02-19T23:39:43.340973, user: 
http://localhost:8001/3.0/users/316531173466190680782932225868344081837;, 
self_link: http://localhost:8001/3.0/addresses/testu...@example.org;, 
email: testu...@example.org, original_email: testu...@example.org}], 
start: 0, http_etag: \cb2df4aad285c85f0128b030ca4ce786308a0b4b\, 
total_size: 2}

Should the address entries include the user_id field?

thanks

as
___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/archive%40jab.org

Security Policy: http://wiki.list.org/x/QIA9

[Mailman-Developers] Is there a direct way to create an address via the REST API?

[Andrew Stuart]

I can create an address that is not linked to a user by 

1: creating a user with an address
2: unlinking the address from the user

Is there a more direct route to creating an address via REST?

I thought there was a POST /addresses method but I can’t seem to see it now - 
maybe it was never there.

thanks

as

___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/archive%40jab.org

Security Policy: http://wiki.list.org/x/QIA9

Re: [Mailman-Developers] /3.0/addresses/c...@example.com/user' - multi purpose REST API method

[Andrew Stuart]

Is this command meant to create the email address 1...@example.org , create a 
user and link them?

POST http://localhost:8001/3.0/addresses/1...@example.org/user

It returns 404

this bit of code in rest/addresses.py appears to be what is being executed and 
seems to be returning a 404 if the address does not already exist.

@child()
def user(self, request, segments):
/addresses/email/user
if self._address is None:
return NotFound(), []
# Avoid circular imports.
from mailman.rest.users import AddressUser
return AddressUser(self._address)

thanks

as


On 10 Feb 2015, at 1:53 pm, Barry Warsaw ba...@list.org wrote:

On Feb 08, 2015, at 12:28 AM, Andrew Stuart wrote:

 TLDR - This API methods does multiple things and depends on information apart
 from the URL to define its behaviour mode, making it hard to give a clear
 authentication scope to. I wonder if it should be broken out into individual
 REST API methods, each with a single purpose and a clear authentication
 scope.

Yes, that makes sense as a general principle.

 As background information, from the documentation:
 
 “”To link an address to a user, a POST request can be sent to the ``/user``
 sub-resource of the address.  If the user does not exist, it will be
 created.””
 
 dump_json('http://localhost:9001/3.0/addresses/c...@example.com/user',
   ...   {'display_name': 'Cris X. Person’})
 
 
 My question is - I am interested to understand the thinking behind this REST
 method.

It's not really that deep. ;)   As it turns out, the REST method is a thin
layer on top of IUserManager.create_user() which itself has been around for
quite a while.  In the model, the behavior is pretty convenient.

(I'll also note that create_user() is used when POSTing to the api/users
resource.)

Another guiding principle is that the edges should not have too much logic.
In MM2, there was a lot of logic ingrained in the cgi scripts, and that causes
a lot of problems if you want the same behavior in some other edge.  Therefore
I try to keep the REST API limited to parsing arguments, handling responses,
and reporting error conditions while the actual core logic is implemented in
the model or the app directory.

 Each of the above behaviours have a different authentication scope/role. For
 example:
 
 (permitted by superuser only) In one case it *creates an email address* and
 *creates a new user* and which are linked to each other.

The use case here is when someone walks up to Postorious and creates an
account for the first time.  They'd enter their email address and Postorious
would direct the core to create a user for them *and* an email address for
them.  It then makes sense to link the two together.  Of course, the new
address begins unverified.

 (permitted by superuser only, permitted by existing user) In one case it
 *creates an email address* and links that address to an existing user, when
 given a POSTed user_id.
 
 (permitted by superuser only) In one case it *creates an email address* and
 does not link it to any user, if the auto_create POST option is False.

The use case here is that someone already has a login to Postorious, and now
they want to associate a new email address with their existing one.  They log
in (establishing their ownership of an existing user), enter their email
address and if needed, that's created.

Now, it's questionable whether that address should be linked to the user until
they verify the address.  I think that happens now, but probably shouldn't,
otherwise it could be hijacked.

Another use case is where a list administrator wants to add a bunch of
addresses to their mailing list.  There may be reasons to opt-out (e.g. a
corporate list of employees, or a physical sign-up sheet that by its nature
implies opt-in).

Maybe one way to look at it is that any operation that links an address to a
user has to be a privileged url.  A user creating an account on Postorious, or
trying to add an address to an existing user are both operations where the
address being created will not be verified (until they do a mail-back or
confirmation click).  In those cases, maybe we need an entirely separate
resource (url) for the user operations, which would ensure the proper
confirmation dance were performed.

Thoughts?

Cheers,
-Barry
___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/andrew.stuart%40supercoders.com.au

Security Policy: http://wiki.list.org/x/QIA9

___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http

Re: [Mailman-Developers] REST API PUT versus PATCH - is there in any point in PUT and PATCH?

[Andrew Stuart]

No worries.  It’s a little repetitious but doesn’t make much difference either 
way.

On 19 Feb 2015, at 8:34 am, Barry Warsaw ba...@list.org wrote:

On Feb 19, 2015, at 08:10 AM, Andrew Stuart wrote:

 Is there any actual benefit in those PUT methods?  I can’t see what they
 offer over the PATCH methods.

PUT is defined in HTTP/1.1.  PATCH is (only) defined in RFC 5789, so it may
not be universally supported.  E.g. back in the restish days, I had to hack
WebOb to support it, and it was an ugly hack, though Falcon supports it out of
the box.  I haven't dug into HTTP/2 enough yet to know whether PATCH has
finally been adopted into the official standard.

Much of the philosophy of the REST API was influenced by RESTful Web Services
(Richardson  Ruby), which IIRC recommended implementing both methods[*].  I
recently received the follow up to that excellent book, RESTful Web APIs, but
haven't really had time to read it yet to see how the intervening 6 years have
changed their recommendations.  A *very* quick skim of the first index hit
provides some additional rationale as to why both are useful.

* PUT is idempotent.  If you send the same PUT request 10 times, the result is
 the same as if you'd only sent it once.

* PATCH is not idempotent, but it can save on bandwidth and complexity.  It's
 like a diff, in that it only contains the parts of the resource that have
 changed.

Cheers,
-Barry

[*] Leonard Richardson was a colleague of mine on the Launchpad project back
in the day.  We had many conversations about Launchpad's REST API and his
insights, expertise, and experience deeply shaped our current design.
___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/andrew.stuart%40supercoders.com.au

Security Policy: http://wiki.list.org/x/QIA9

___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/archive%40jab.org

Security Policy: http://wiki.list.org/x/QIA9

Re: [Mailman-Developers] Removal of REST API method /lists/{list_identifier}/config/{sub_resource}

[Andrew Stuart]

Alternatively I could just ignore these methods and not put them in the proxy 
although that grates against my sense of completion …   :-)

as

On 19 Feb 2015, at 7:51 am, Andrew Stuart andrew.stu...@supercoders.com.au 
wrote:


###
exhibit 1:
The REST API provides
GET /lists/{list_identifier}/config
PUT /lists/{list_identifier}/config
PATCH /lists/{list_identifier}/config

This allows reading of all list config variables (see bottom of this message 
for which ones). It also allows for setting of any individual config variable.
###
exhibit 2:
The REST API also provides
GET /lists/{list_identifier}/config/{config_variable}
PATCH /lists/{list_identifier}/config/{config_variable}

Which allows any individual config variable to be read and modified.
###


Effectively exhibit2 completely duplicates exhibit1 for no substantial gain.  

exhibit2 significantly increases the complexity of the overall system due to 
its fine grained duplication of everything in exhibit1. Right now I’m facing 
write a bunch of code to handle the fine grained config variable setting, all 
the while thinking “but this is all duplicate functionality”.

I’m wondering if anyone would miss exhibit 2 if it was deleted from the REST 
API, saving on testing and complexity but losing nothing.

Have I missed something? MAYBE its possible that different config items might 
have different user access requirements, justifying fine grained control, 
however I can’t see how right now.

Thoughts?

...   dict(
... acceptable_aliases=['o...@example.com', 't...@example.com'],
... admin_immed_notify=False,
... admin_notify_mchanges=True,
... administrivia=False,
... advertised=False,
... anonymous_list=True,
... archive_policy='never',
... autorespond_owner='respond_and_discard',
... autorespond_postings='respond_and_continue',
... autorespond_requests='respond_and_discard',
... autoresponse_grace_period='45d',
... autoresponse_owner_text='the owner',
... autoresponse_postings_text='the mailing list',
... autoresponse_request_text='the robot',
... display_name='Fnords',
... description='This is my mailing list',
... include_rfc2369_headers=False,
... allow_list_posts=False,
... digest_size_threshold=10.5,
... posting_pipeline='virgin',
... filter_content=True,
... first_strip_reply_to=True,
... convert_html_to_plaintext=True,
... collapse_alternatives=False,
... reply_goes_to_list='point_to_list',
... reply_to_address='b...@example.com',
... send_welcome_message=False,
... subject_prefix='[ant]',
... welcome_message_uri='mailman:///welcome.txt',
... default_member_action='hold',
... default_nonmember_action='discard',
... )

___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/andrew.stuart%40supercoders.com.au

Security Policy: http://wiki.list.org/x/QIA9

___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/archive%40jab.org

Security Policy: http://wiki.list.org/x/QIA9

[Mailman-Developers] REST API PUT versus PATCH - is there in any point in PUT and PATCH?

[Andrew Stuart]

There are several cases in the REST API where there is a PUT method that 
requires all fields to be present, and also a PATCH method which is the same 
except that not all fields are required.

Is there any actual benefit in those PUT methods?  I can’t see what they offer 
over the PATCH methods.

Maybe its worth thinking about removing all PUT methods that don’t offer any 
substantial gain over the PATCH methods, reducing need for testing, 
documentation and complexity.

Here are the PUT and PATCH methods in the REST API:

AddressesVARUserPUT
ListsVARArchiversPATCH
ListsVARArchiversPUT
ListsVARConfigPATCH
ListsVARConfigPUT
ListsVARConfigVARPATCH
MembersVARPATCH
MembersVARPreferencesPATCH
MembersVARPreferencesPUT
UsersVARPATCH
UsersVARPUT
UsersVARPreferencesPATCH
UsersVARPreferencesPUT



as
 

___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/archive%40jab.org

Security Policy: http://wiki.list.org/x/QIA9

Re: [Mailman-Developers] authentication, authorization and permissions FYI

[Andrew Stuart]

Sorry I meant SAML looks similar to JWT but XML based.

On 18 Feb 2015, at 10:45 am, Dick Visser vis...@terena.org wrote:

sorry meant to sent to list, just did that now



On Wed, Feb 18, 2015 at 12:42 AM, Andrew Stuart
andrew.stu...@supercoders.com.au wrote:
 Hi Dick,
 
 If you re-post as a reply to the mailing list I can send a reply there.
 
 thanks
 
 as
 
 
 On 18 Feb 2015, at 10:40 am, Dick Visser vis...@terena.org wrote:
 
 Hi Andrew
 
 Is this about the MM3 web interface?
 If so, I wonder if it could work with SAML authentication.
 This can be done with PySAML2 (https://github.com/rohe/pysaml2), or in
 an Apache module (mod_mellon).
 The latter takes care of the authentication, and any web app protected
 by it an rely on (for instance) REMOTE_USER always being available.
 
 Thanks
 
 Dick
 
 
 On Sun, Feb 15, 2015 at 9:05 AM, Andrew Stuart
 andrew.stu...@supercoders.com.au wrote:
 I’ve just finished (mostly) implementing most of the authentication, 
 authorization and permissions systems. Thought you might be interested in a 
 little on how it works.
 
 To discuss this further I need to explain a few things - apologies to those 
 who i am teaching to suck eggs.
 
 The auth server needs a few key subsystems to get the job done - 
 authorization, authentication and permissions.
 
 AUTHENTICATION - this is the process of the end user identifying themselves 
 in the first place (in this case, by logging in with a username and 
 password), then for every request beyond that, providing evidence that they 
 are who they say they are - in this case, that’s done by giving them a token 
 when they log in successfully, then requiring that the token be returned 
 along with each subsequent request.
 
 AUTHORIZATION - this is the process of determining if the authorized user is 
 allowed - according to the rules of our application - to access the resource 
 that they have requested. For example, user Jenny sends a request to access 
 the members of a list named supermaillingl...@marketingcompany.com. Before 
 we can execute that request we need to determine if she is *authorized* to 
 do so.
 
 PERMISSIONS - for the authorization process to do its job, it needs 
 information about WHO is allowed WHAT LEVEL OF ACCESS to WHICH RESOURCE. 
 This is the *permissions system*.  It’s VERY easy to understand. Nothing 
 more to it than a database table that looks like this
 user_id = Column(String, nullable=False)
 role = Column(String, nullable=False)
 resource_id = Column(String, nullable=False)
 resource_type = Column(Enum('list', 'domain', 'archive','server','domain', 
 name='resource_type'), nullable=False)
 
 The permissions system is a small extension of the existing Mailman 
 permissions system.
 user_id = the Mailman user id
 role = one of ['domainowner', 'serverowner', 'userowner', 'listowner', 
 'listmember', 'listmoderator']
 resource_type one of ['list', 'domain', 'archive','server',’domain']
 resource_id = Mailman list_id or Mailman domain or ‘server’ or ‘user'
 
 So with four pieces of information you can define access for any user to any 
 Mailman resource. The permissions system is nothing more than a plain SQL 
 table with four columns, that’s all. The important thing is that it unifies 
 all Mailman resources into those four pieces of permission information.
 
 Tying it all together - the auth proxy:
 
 1: receives an inbound HTTP API request
 2: determines (from the token that came with the request) the authenticated 
 identity of the requesting user
 3: determines (from the request URL/field params) which Mailman resource is 
 being requested (i.e. a list or a domain or something else)
 4: determines (from the request URL/field params) the type of activity that 
 the requesting user wishs to carry out against that resource (create new, 
 delete, rename, for example)
 5: asks the authorization process “is this user allowed to carry out this 
 type of activity against this resource?”
 6: the authorization process has its own set of business rules for working 
 out if a particular user is allowed to carry out the requested activity 
 against the requested resource.  It uses the permissions table to work out 
 if user is allowed to access resource in the requested manner.
 7: finally, if the authorisation process approves, then the request is 
 carried out.
 
 Phew!
 
 Any questions welcome. I’m hoping to have working code to show before Pycon. 
  I want to get it sufficiently complete such that it works when I first 
 release it.
 
 
 
 ___
 Mailman-Developers mailing list
 Mailman-Developers@python.org
 https://mail.python.org/mailman/listinfo/mailman-developers
 Mailman FAQ: http://wiki.list.org/x/AgA3
 Searchable Archives: 
 http://www.mail-archive.com/mailman-developers%40python.org/
 Unsubscribe: 
 https://mail.python.org/mailman/options/mailman-developers/visser%40terena.org
 
 Security Policy: http://wiki.list.org/x/QIA9
 
 
 
 --
 Dick Visser
 Sr

Re: [Mailman-Developers] Soliciting feedback on idea for rounding out the permissions model.

[Andrew Stuart]

Are there any other permissions you can think of?

I figured that an archive, which isn’t really a Mailman resource anyway(?), has 
the same permissions as the list that it gets its emails from.

Are there any other Mailman resources beyond user, list, domain, server?  There 
is member, but that is really more of a relationship between a user and a list 
- not a standalone resource that requires permissions - is that right?

as


On 17 Feb 2015, at 1:37 am, Barry Warsaw ba...@list.org wrote:

This looks like a reasonable analysis.

On Feb 16, 2015, at 11:39 AM, Andrew Stuart wrote:

 To implement, it would need to be possible to define as user as being a
 ‘serverowner’, and also to be able to define a user as being a ‘domainowner’
 for any given domain.

As you've noticed, we have IMember objects which encapsulate the list-centric
roles for users.  It's important to note though that this isn't quite complete
because it's possible for validated, non-user linked addresses to also be
subscribed to mailing lists, and then we have no trail back to a UUID.  I
think in normal operations though, this should be rare to nonexistent.

We have a natural place to hang user information on for domains, since we have
an IDomain interface (and a domain model/table).  In this case, I would
stipulate that only users can be associated with domains.  We'd probably need
to add a table to represent this relationship.

I don't know where to hang the siteowner information.  Maybe adding a column
to the user table?  It probably just needs to be a flag.

The alternative is to use some special list_id values in the member table to
denote a domain and the site, and then restrict the role to owner in both
cases.  I'm not sure I like this though.  It seems implicit rather than
explicit.

Are there any other permissions you can think of?

Cheers,
-Barry
___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/andrew.stuart%40supercoders.com.au

Security Policy: http://wiki.list.org/x/QIA9

___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/archive%40jab.org

Security Policy: http://wiki.list.org/x/QIA9

[Mailman-Developers] fqdn_listname - is it strict in what type of input it accepts?

[Andrew Stuart]

As I understand it, fqdn_listname is effectively the email address of a mailing 
list. i.e. testl...@example.org

The RFC 2369 list-id is the same thing but with a dot instead of an @ symbol. 
i.e. testlist.example.org

My question: is Mailman strict about accepting only email addresses in 
fqdn_listname fields? Or does it also accept RFC 2369 list-id in fqdn_listname 
fields?

I’m trying to wrap my head around some inconsistencies where some methods that 
use fqdn_listname seem to accept list-id, bt I can’t see how it possible to 
determine domain from a list-id, so how is it possible to accept either in a 
fqdn_listname field?

thanks

as

___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/archive%40jab.org

Security Policy: http://wiki.list.org/x/QIA9

Re: [Mailman-Developers] Soliciting feedback on idea for rounding out the permissions model.

[Andrew Stuart]

I’d be interested to hear what the Postorius/Hyperkitty folks think on the 
concept of more tangibly defining Mailman resource/user permissions.

Presumably Postorius and Hyperkitty somehow grant access to resources and store 
the permissions information externally?

Anyone got any thoughts?

as



On 17 Feb 2015, at 1:37 am, Barry Warsaw ba...@list.org wrote:

This looks like a reasonable analysis.

On Feb 16, 2015, at 11:39 AM, Andrew Stuart wrote:

 To implement, it would need to be possible to define as user as being a
 ‘serverowner’, and also to be able to define a user as being a ‘domainowner’
 for any given domain.

As you've noticed, we have IMember objects which encapsulate the list-centric
roles for users.  It's important to note though that this isn't quite complete
because it's possible for validated, non-user linked addresses to also be
subscribed to mailing lists, and then we have no trail back to a UUID.  I
think in normal operations though, this should be rare to nonexistent.

We have a natural place to hang user information on for domains, since we have
an IDomain interface (and a domain model/table).  In this case, I would
stipulate that only users can be associated with domains.  We'd probably need
to add a table to represent this relationship.

I don't know where to hang the siteowner information.  Maybe adding a column
to the user table?  It probably just needs to be a flag.

The alternative is to use some special list_id values in the member table to
denote a domain and the site, and then restrict the role to owner in both
cases.  I'm not sure I like this though.  It seems implicit rather than
explicit.

Are there any other permissions you can think of?

Cheers,
-Barry
___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/andrew.stuart%40supercoders.com.au

Security Policy: http://wiki.list.org/x/QIA9

___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/archive%40jab.org

Security Policy: http://wiki.list.org/x/QIA9

Re: [Mailman-Developers] A couple of archiving questions

[Andrew Stuart]

OK looks pretty easy. Seems like I’ll just write an archiver if I want access 
to archived messages.

Can there only be one active archiver or multiple?  Say I wanted all messages 
to go to a zip archiver as well as going to monharc or pipermail as well as my 
custom archive script.

as




On 16 Feb 2015, at 8:57 pm, Stephen J. Turnbull step...@xemacs.org wrote:

Andrew Stuart writes:

 Why would you want to access messages scheduled for archiving”
 via REST?  I have trouble imagining a use case.
 
 It would make it pretty easy to write an archiver if all I had to
 do is poll via the REST API for new messages waiting to be archived
 whenever I feel like it and put them somewhere.

It's no harder than that to write a Handler in Mailman 2, and IIRC the
additional burden (adding the message-id-hash) in Mailman 3 is
elsewhere in the pipeline.  Here's the whole thing for MM 2:

import time
from cStringIO import StringIO

from Mailman import mm_cfg
from Mailman.Queue.sbcache import get_switchboard



def process(mlist, msg, msgdata):
   # short circuits
   if msgdata.get('isdigest') or not mlist.archive:
   return
   # Common practice seems to favor X-No-Archive: yes.  No other value for
   # this header seems to make sense, so we'll just test for it's presence.
   # I'm keeping X-Archive: no for backwards compatibility.
   if msg.has_key('x-no-archive') or msg.get('x-archive', '').lower() == 'no':
   return
   # Send the message to the archiver queue
   archq = get_switchboard(mm_cfg.ARCHQUEUE_DIR)
   # Send the message to the queue
   archq.enqueue(msg, msgdata)

*One* function of *six* lines after deleting comments (and ignoring
imports), all of which you need to do on the archiver side of the REST
interface anyway.  Except the part about get the archiver queue
object, but you'll need some equivalent *on the Mailman core side* to
ensure that messages hang around until archived.  So this is more
complex than the current push design.  And it also leaves you
vulnerable to DoS'ing yourself if the polling process goes down and
the queue fills your disk -- probably not a *big* issue, but one that
needs a little thought at least to be sure it isn't.  (The self-DoS
problem is a non-problem for the current design, because if you are
expecting to archive locally you probably do have the storage for it.)

Note that the archq.enqueue() in the above is semantically just put
them somewhere (I'm quoting you).  That's where the devilish details
are in both Pipermail and in your abstract REST-pull-based archiver,
not in interfacing with the core pipeline.

I suppose you could push the MM 2 archive queue (which I'm pretty sure
currently is in Pipermail) into core in MM 3, and then you could use
REST to pull the messages out, but really, I don't see a big gain
except that you use REST for everything.  But writing an archiver is
not as simple as you seem to think (unless it just piles up the
messages somewhere, and we already have that as example code in MM 3:
mailman3/src/mailman/archiving/prototype.py -- hardly more complex
than the MM 2 code above, and it actually does the work of storing the
messages!)

Regards,
Steve

___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/archive%40jab.org

Security Policy: http://wiki.list.org/x/QIA9

[Mailman-Developers] authentication, authorization and permissions FYI

[Andrew Stuart]

I’ve just finished (mostly) implementing most of the authentication, 
authorization and permissions systems. Thought you might be interested in a 
little on how it works.

To discuss this further I need to explain a few things - apologies to those who 
i am teaching to suck eggs.

The auth server needs a few key subsystems to get the job done - authorization, 
authentication and permissions. 

AUTHENTICATION - this is the process of the end user identifying themselves in 
the first place (in this case, by logging in with a username and password), 
then for every request beyond that, providing evidence that they are who they 
say they are - in this case, that’s done by giving them a token when they log 
in successfully, then requiring that the token be returned along with each 
subsequent request.

AUTHORIZATION - this is the process of determining if the authorized user is 
allowed - according to the rules of our application - to access the resource 
that they have requested. For example, user Jenny sends a request to access the 
members of a list named supermaillingl...@marketingcompany.com. Before we can 
execute that request we need to determine if she is *authorized* to do so.

PERMISSIONS - for the authorization process to do its job, it needs information 
about WHO is allowed WHAT LEVEL OF ACCESS to WHICH RESOURCE. This is the 
*permissions system*.  It’s VERY easy to understand. Nothing more to it than a 
database table that looks like this
user_id = Column(String, nullable=False)
role = Column(String, nullable=False)
resource_id = Column(String, nullable=False)
resource_type = Column(Enum('list', 'domain', 'archive','server','domain', 
name='resource_type'), nullable=False)

The permissions system is a small extension of the existing Mailman permissions 
system.
user_id = the Mailman user id
role = one of ['domainowner', 'serverowner', 'userowner', 'listowner', 
'listmember', 'listmoderator']
resource_type one of ['list', 'domain', 'archive','server',’domain']
resource_id = Mailman list_id or Mailman domain or ‘server’ or ‘user'

So with four pieces of information you can define access for any user to any 
Mailman resource. The permissions system is nothing more than a plain SQL table 
with four columns, that’s all. The important thing is that it unifies all 
Mailman resources into those four pieces of permission information.

Tying it all together - the auth proxy:

1: receives an inbound HTTP API request
2: determines (from the token that came with the request) the authenticated 
identity of the requesting user
3: determines (from the request URL/field params) which Mailman resource is 
being requested (i.e. a list or a domain or something else)
4: determines (from the request URL/field params) the type of activity that the 
requesting user wishs to carry out against that resource (create new, delete, 
rename, for example)
5: asks the authorization process “is this user allowed to carry out this type 
of activity against this resource?”
6: the authorization process has its own set of business rules for working out 
if a particular user is allowed to carry out the requested activity against the 
requested resource.  It uses the permissions table to work out if user is 
allowed to access resource in the requested manner.
7: finally, if the authorisation process approves, then the request is carried 
out.

Phew!

Any questions welcome. I’m hoping to have working code to show before Pycon.  I 
want to get it sufficiently complete such that it works when I first release it.



___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/archive%40jab.org

Security Policy: http://wiki.list.org/x/QIA9

[Mailman-Developers] Soliciting feedback on idea for rounding out the permissions model.

[Andrew Stuart]

I had an idea about rounding out the Mailman permissions model, interested in 
hearing thoughts on it. Obviously there has been considerable discussion on 
this topic before.

Mailman already carries much of the information needed for determining user 
permissions to Mailman resources. Only two things are missing: 1: the ability 
to define a user as being a “serverowner”
2: the ability to define a user as being a “domainowner”

(You’ll need to look at this email in plain text to see the table properly).

The Mailman permissions model currently looks like this:

resource_type roles resource_iduser_identifier   where 
to find permission
-
user  userowner n/aUUID  
(defined in user record)
list  listowner list_idsubscriber
(defined in list member record)
list  listmemberlist_idsubscriber
(defined in list member record)
list  listmoderator list_idsubscriber
(defined in list member record)
list  listnonmember list_idsubscriber
(defined in list member record)

I am suggesting adding two further permissions to the existing permissions 
model, which would look like this: 

resource_type roles resource_iduser_id   where 
to find permission
-
serverserverowner   n/aUUID  (not 
currently defined in Mailman)
domaindomainowner   mail_host  UUID  (not 
currently defined in Mailman)


To implement, it would need to be possible to define as user as being a 
‘serverowner’, and also to be able to define a user as being a ‘domainowner’ 
for any given domain. 

If it were possible to do so within the Mailman core then there would be a 
completely usable permissions model entirely within Mailman, and no need to 
store any additional permissions data outside Mailman. The permissions model 
would allow definition of user access to any Mailman resource including domains 
and servers.

The interpretation of the permissions would still be up to the application that 
consumes the REST API, as is currently the case.

Thoughts?

as

___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/archive%40jab.org

Security Policy: http://wiki.list.org/x/QIA9

Re: [Mailman-Developers] A couple of archiving questions

[Andrew Stuart]

Why would you want to access messages scheduled for archiving” via REST?  I 
have trouble imagining a use case.

It would make it pretty easy to write an archiver if all I had to do is poll 
via the REST API for new messages waiting to be archived whenever I feel like 
it and put them somewhere. Clearly that’s not going to work if the emails are 
being shunted out the door to some other archiver via the existing archiving 
interface. But if that could be addressed somehow, REST API access would make 
life pretty easy for dealing with archive messages - mainly I’d like to avoid 
touching the Mailman core or write anything that needs installation into 
Mailman.

I’m not sure exactly what I have to do to write something that interfaces to 
the core archiver so I’m not sure how easy or hard that would be. 

as

On 16 Feb 2015, at 12:57 pm, Stephen J. Turnbull step...@xemacs.org wrote:

Andrew Stuart writes:

 Is there any reason why it should not be possible to be able to
 pull messages scheduled for archiving out through the REST API as
 file attachments?

Oops, my previous message missed the subtlety that you do have the
message at the point in time you're considering.  You still can't do
archiving with REST: REST is a pull interface, archiving is a push
operation.

Why would you want to access messages scheduled for archiving via
REST?  I have trouble imagining a use case.




___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/archive%40jab.org

Security Policy: http://wiki.list.org/x/QIA9

Re: [Mailman-Developers] Is there a way to get member id via REST API?

[Andrew Stuart]

In a similar vein - I seem to recall from when I first started looking at the 
REST API, noticing that some of the methods that return user information also 
don’t include the user_id as a standalone field, I had to parse it out of the 
self_link.  I can’t remember offhand which methods.  I’ll keep an eye out and 
if I notice will report.

as


On 14 Feb 2015, at 11:59 am, Barry Warsaw ba...@list.org wrote:

On Feb 14, 2015, at 09:43 AM, Andrew Stuart wrote:

 GET /addresses/{email}/memberships
 GET /members
 
 Both seem to return member id in self_link but I can’t see it elsewhere.
 
 Anywhere I can get member id without parsing it out of the self_link? Maybe
 I’m not looking in the right place.

Nope, I think that's missing.  Member ids are UUIDs so it makes sense to
include the member id as a separate key in the member resource JSON.   Just
like users it'll be an integer.

I'll add that now.

Cheers,
-Barry
___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/andrew.stuart%40supercoders.com.au

Security Policy: http://wiki.list.org/x/QIA9

___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/archive%40jab.org

Security Policy: http://wiki.list.org/x/QIA9

[Mailman-Developers] Question about member role and PATCH /members/{id}

[Andrew Stuart]

When a user is subscribed to a list, they can be assigned a role, one of 
member, owner, moderator nonmember.

The PATCH /members/{id} method does not allow changes to the role and 
display_name.

I’m just wondering what the thinking is behind this - is there a reason that 
role cannot be changed?

thanks

___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/archive%40jab.org

Security Policy: http://wiki.list.org/x/QIA9

[Mailman-Developers] Date normalization - opinions sought

[Andrew Stuart]

Does Mailman normalize dates on emails to UTC?

It seems like a good idea to me to normalise dates on emails to UTC but you 
guys would know best.

What would be the pros and cons of email date normalisation to UTC? Are there 
good reasons not to?

Which date field on an email is even the right one to use?  Sent date or 
received date? And WHICH received date , because there can be various dates 
can’t there?

thanks

as

___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/archive%40jab.org

Security Policy: http://wiki.list.org/x/QIA9

Re: [Mailman-Developers] When a user is deleted, is the uid record intentionally not deleted?

[Andrew Stuart]

Sounds good to me.

On 10 Feb 2015, at 1:59 pm, Barry Warsaw ba...@list.org wrote:

On Feb 06, 2015, at 04:17 PM, Andrew Stuart wrote:

 Yes it’s a tough question. I agree with leaving the uid records in the
 database to prevent reuse. You are correct my test code is exercising a full
 working system.  It would be good if my tests could leave the database in the
 same state after completion of each test.
 
 The ideal would be a method that is exactly the same as the existing user
 delete code but also deletes the users uid record.
 
 maybe as you suggest, url route names like:
 DELETE `api/testing/users/{user_identifier}'
 or
 DELETE `api/reserved/users/{user_identifier}’

Thinking about this some more, and remembering that this is an admin interface
(i.e. the authenticating proxy would *not* expose this), what about having
just api/uids which could be used to access and delete uids.  You couldn't
post/patch to either the collection url (e.g. api/uids) or to individual
uids (e.g. api/uids/7), but you could delete them *only if there is no user
object associated with the uid*, otherwise you'd get an error.

This means you'd still have to delete the user and its uid in two operations,
but as this is primarily for testing purposes, the extra overhead should be
acceptable.  We could even allow you to `DELETE api/uids` to clear out the
entire table (but again, only if there are no users...?).  I put that
restriction in because I wouldn't want a bug in the proxy (or some other REST
API client) to break the whole system, which would definitely be possible if
you accidentally cleared out the uid table.

If it makes it easier to restrict proxy access to some url space, then we
could put this under api/system or a new api/reserved resource.

 Personally I’d not bother with exposing or hiding reserved methods based on
 environment variables or anything else - as you say, it’s an internal API not
 mean to used by the public so developers should know not to be using reserved
 or testing methods in their production code. The extra functionality might
 introduce bugs and needs writing and testing and documenting - not worth it.

Agreed.

Thoughts?

Cheers,
-Barry
___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/andrew.stuart%40supercoders.com.au

Security Policy: http://wiki.list.org/x/QIA9

___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/archive%40jab.org

Security Policy: http://wiki.list.org/x/QIA9

Re: [Mailman-Developers] REST API delete user function does not delete every 2nd linked address

[Andrew Stuart]

I might need some help to track this down……..

When a user is deleted via the REST API, there is a function that appears to be 
iterating over the users addresses and unlinking them.

Is it possible that the unlinking process is somehow mutating the list of 
addresses that calling function is using? So as each address is unlinked 
somehow the underlying list of addresses is no longer able to be used as an 
effective address iterator?

Here are the relevant functions:

in rest/users.py (this is call from the REST API)

def on_delete(self, request, response):
Delete the named user, all her memberships, and addresses.
if self._user is None:
not_found(response)
return
for member in self._user.memberships.members:
member.unsubscribe()
user_manager = getUtility(IUserManager)
for address in self._user.addresses:
debug_log.info('address scheduled for deletion: {}'.format(address))
for address in self._user.addresses:
user_manager.delete_address(address)
user_manager.delete_user(self._user)
no_content(response)

* the debug.log line above returns a full list of all the addresses 
scheduled for deletion, proving that all the expected addresses are present in 
self._user.addresses
* THE NEXT LINE for address in self._user.addresses:” IS DOING THE 
SKIPPING.
* SO - COULD IT BE that self._user.addresses is being mutated by 
user_manager.delete_address(address), stuffing up the for loop that is meant to 
be iterating over all addresses? Which is to say, as each address is unlinked, 
the length of and values in self._user.addresses is changing?

in usermanager.py

@dbconnection
def delete_address(self, store, address):
See `IUserManager`.
# If there's a user controlling this address, it has to first be
# unlinked before the address can be deleted.
if address.user:
address.user.unlink(address)
store.delete(address)



___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/archive%40jab.org

Security Policy: http://wiki.list.org/x/QIA9

[Mailman-Developers] REST API delete user function does not delete every 2nd linked address

[Andrew Stuart]

(you’ll need to widen your message window to see this clearly)


The following series of curl commands creates a user with an email address of 
‘z...@example.org’, and then links 10 additional addresses to z...@example.org

curl -X POST --data email=z...@example.orgdisplay_name=z --header 
authorization: Basic cmVzdGFkbWluOnJlc3RwYXNz http://localhost:8001/3.0/users
curl -X POST --data email=z...@example.org --header authorization: Basic 
cmVzdGFkbWluOnJlc3RwYXNz 
http://localhost:8001/3.0/users/z...@example.org/addresses
curl -X POST --data email=z...@example.org --header authorization: Basic 
cmVzdGFkbWluOnJlc3RwYXNz 
http://localhost:8001/3.0/users/z...@example.org/addresses
curl -X POST --data email=z...@example.org --header authorization: Basic 
cmVzdGFkbWluOnJlc3RwYXNz 
http://localhost:8001/3.0/users/z...@example.org/addresses
curl -X POST --data email=z...@example.org --header authorization: Basic 
cmVzdGFkbWluOnJlc3RwYXNz 
http://localhost:8001/3.0/users/z...@example.org/addresses
curl -X POST --data email=z...@example.org --header authorization: Basic 
cmVzdGFkbWluOnJlc3RwYXNz 
http://localhost:8001/3.0/users/z...@example.org/addresses
curl -X POST --data email=z...@example.org --header authorization: Basic 
cmVzdGFkbWluOnJlc3RwYXNz 
http://localhost:8001/3.0/users/z...@example.org/addresses
curl -X POST --data email=z...@example.org --header authorization: Basic 
cmVzdGFkbWluOnJlc3RwYXNz 
http://localhost:8001/3.0/users/z...@example.org/addresses
curl -X POST --data email=z...@example.org --header authorization: Basic 
cmVzdGFkbWluOnJlc3RwYXNz 
http://localhost:8001/3.0/users/z...@example.org/addresses
curl -X POST --data email=z...@example.org --header authorization: Basic 
cmVzdGFkbWluOnJlc3RwYXNz 
http://localhost:8001/3.0/users/z...@example.org/addresses
curl -X POST --data email=z...@example.org --header authorization: Basic 
cmVzdGFkbWluOnJlc3RwYXNz 
http://localhost:8001/3.0/users/z...@example.org/addresses

A sqlite3 dump of the database shows the following in the addresses table:

INSERT INTO address VALUES(11,'z...@example.org',NULL,'z',NULL,'2015-02-08 
10:50:37.241724',5,17);
INSERT INTO address VALUES(12,'z...@example.org',NULL,'',NULL,'2015-02-08 
10:52:08.928901',5,19);
INSERT INTO address VALUES(13,'z...@example.org',NULL,'',NULL,'2015-02-08 
10:52:36.308978',5,20);
INSERT INTO address VALUES(14,'z...@example.org',NULL,'',NULL,'2015-02-08 
10:52:43.947229',5,21);
INSERT INTO address VALUES(15,'z...@example.org',NULL,'',NULL,'2015-02-08 
10:52:49.598934',5,22);
INSERT INTO address VALUES(16,'z...@example.org',NULL,'',NULL,'2015-02-08 
10:52:54.406450',5,23);
INSERT INTO address VALUES(17,'z...@example.org',NULL,'',NULL,'2015-02-08 
10:52:59.295078',5,24);
INSERT INTO address VALUES(18,'z...@example.org',NULL,'',NULL,'2015-02-08 
10:53:04.892381',5,25);
INSERT INTO address VALUES(19,'z...@example.org',NULL,'',NULL,'2015-02-08 
10:53:10.497119',5,26);
INSERT INTO address VALUES(20,'z...@example.org',NULL,'',NULL,'2015-02-08 
10:53:16.450507',5,27);
INSERT INTO address VALUES(21,'z...@example.org',NULL,'',NULL,'2015-02-08 
10:53:24.021368',5,28);

All good at this stage. No problems.

The following curl command deletes user ‘z...@example.org’

curl --verbose -X DELETE --header authorization: Basic 
cmVzdGFkbWluOnJlc3RwYXNz http://localhost:8001/3.0/users/z...@example.org

Strangely, a sqlite3 dump of the database now shows the following in the 
addresses table:

INSERT INTO address VALUES(12,'z...@example.org',NULL,'',NULL,'2015-02-08 
10:52:08.928901',NULL,19);
INSERT INTO address VALUES(14,'z...@example.org',NULL,'',NULL,'2015-02-08 
10:52:43.947229',NULL,21);
INSERT INTO address VALUES(16,'z...@example.org',NULL,'',NULL,'2015-02-08 
10:52:54.406450',NULL,23);
INSERT INTO address VALUES(18,'z...@example.org',NULL,'',NULL,'2015-02-08 
10:53:04.892381',NULL,25);
INSERT INTO address VALUES(20,'z...@example.org',NULL,'',NULL,'2015-02-08 
10:53:16.450507',NULL,27);

So 6 of the addresses that should have been deleted were deleted, but 5 remain. 
The linked address deletion seems to be skipping addresses for some reason.








___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/archive%40jab.org

Security Policy: http://wiki.list.org/x/QIA9

Re: [Mailman-Developers] REST API delete user function does not delete every 2nd linked address

[Andrew Stuart]

This appears to fix the problme in rest/users.py

def on_delete(self, request, response):
Delete the named user, all her memberships, and addresses.
if self._user is None:
not_found(response)
return
for member in self._user.memberships.members:
member.unsubscribe()
user_manager = getUtility(IUserManager)
addresses_for_deletion = []
for address in self._user.addresses:
# to avoid mutating the self._user.addresses iterator, create a 
separate list of addresses
addresses_for_deletion.append(address)
for address in addresses_for_deletion:
user_manager.delete_address(address)
user_manager.delete_user(self._user)
no_content(response)


On 8 Feb 2015, at 11:07 pm, Andrew Stuart andrew.stu...@supercoders.com.au 
wrote:

I might need some help to track this down……..

When a user is deleted via the REST API, there is a function that appears to be 
iterating over the users addresses and unlinking them.

Is it possible that the unlinking process is somehow mutating the list of 
addresses that calling function is using? So as each address is unlinked 
somehow the underlying list of addresses is no longer able to be used as an 
effective address iterator?

Here are the relevant functions:

in rest/users.py (this is call from the REST API)

   def on_delete(self, request, response):
   Delete the named user, all her memberships, and addresses.
   if self._user is None:
   not_found(response)
   return
   for member in self._user.memberships.members:
   member.unsubscribe()
   user_manager = getUtility(IUserManager)
   for address in self._user.addresses:
   debug_log.info('address scheduled for deletion: {}'.format(address))
   for address in self._user.addresses:
   user_manager.delete_address(address)
   user_manager.delete_user(self._user)
   no_content(response)

* the debug.log line above returns a full list of all the addresses 
scheduled for deletion, proving that all the expected addresses are present in 
self._user.addresses
* THE NEXT LINE for address in self._user.addresses:” IS DOING THE 
SKIPPING.
* SO - COULD IT BE that self._user.addresses is being mutated by 
user_manager.delete_address(address), stuffing up the for loop that is meant to 
be iterating over all addresses? Which is to say, as each address is unlinked, 
the length of and values in self._user.addresses is changing?

in usermanager.py

   @dbconnection
   def delete_address(self, store, address):
   See `IUserManager`.
   # If there's a user controlling this address, it has to first be
   # unlinked before the address can be deleted.
   if address.user:
   address.user.unlink(address)
   store.delete(address)



___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/andrew.stuart%40supercoders.com.au

Security Policy: http://wiki.list.org/x/QIA9

___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/archive%40jab.org

Security Policy: http://wiki.list.org/x/QIA9

Re: [Mailman-Developers] /3.0/addresses/c...@example.com/user' - multi purpose REST API method

[Andrew Stuart]

Actually I meant authorization not authentication.

On 8 Feb 2015, at 12:28 am, Andrew Stuart andrew.stu...@supercoders.com.au 
wrote:

TLDR - This API methods does multiple things and depends on information apart 
from the URL to define its behaviour mode, making it hard to give a clear 
authentication scope to. I wonder if it should be broken out into individual 
REST API methods, each with a single purpose and a clear authentication scope.

As background information, from the documentation:

“”To link an address to a user, a POST request can be sent to the ``/user`` 
sub-resource of the address.  If the user does not exist, it will be created.””

 dump_json('http://localhost:9001/3.0/addresses/c...@example.com/user',
   ...   {'display_name': 'Cris X. Person’})


My question is - I am interested to understand the thinking behind this REST 
method.

It is effectively a multi purpose method.  

In one case it *creates an email address* and *creates a new user* and which 
are linked to each other.  
In one case it *creates an email address* and links that address to an existing 
user, when given a POSTed user_id.
In one case it *creates an email address* and does not link it to any user, if 
the auto_create POST option is False.

In terms of building an authentication system this REST method is problematic. 
All (I think at this stage anyway) other methods in the REST API can be given a 
clear authentication scope/role based only on the URL. This REST API method, 
dual to its multi purpose, mode options-in-the-POST-data behaviour, cannot be 
given an authentication scope based only on the URL.

Each of the above behaviours have a different authentication scope/role. For 
example:

(permitted by superuser only) In one case it *creates an email address* and 
*creates a new user* and which are linked to each other.  

(permitted by superuser only, permitted by existing user) In one case it 
*creates an email address* and links that address to an existing user, when 
given a POSTed user_id.

(permitted by superuser only) In one case it *creates an email address* and 
does not link it to any user, if the auto_create POST option is False.

I’m interested to hear thoughts on this. It would be good if the authentication 
scope of every REST API URL could be defined based only on the URL, without the 
need to know either options provided in the form data, or whether or not other 
records such as user records exist.

In summary - This API methods does too many things and depends on information 
apart from the URL to define its behaviour, making it hard to give a clear 
authentication scope to. I wonder if it should be broken out into individual 
REST API methods, each with a single purpose and a clear authentication scope.








___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/andrew.stuart%40supercoders.com.au

Security Policy: http://wiki.list.org/x/QIA9

___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/archive%40jab.org

Security Policy: http://wiki.list.org/x/QIA9

[Mailman-Developers] /3.0/addresses/c...@example.com/user' - multi purpose REST API method

[Andrew Stuart]

TLDR - This API methods does multiple things and depends on information apart 
from the URL to define its behaviour mode, making it hard to give a clear 
authentication scope to. I wonder if it should be broken out into individual 
REST API methods, each with a single purpose and a clear authentication scope.

As background information, from the documentation:

“”To link an address to a user, a POST request can be sent to the ``/user`` 
sub-resource of the address.  If the user does not exist, it will be created.””

 dump_json('http://localhost:9001/3.0/addresses/c...@example.com/user',
...   {'display_name': 'Cris X. Person’})


My question is - I am interested to understand the thinking behind this REST 
method.

It is effectively a multi purpose method.  

In one case it *creates an email address* and *creates a new user* and which 
are linked to each other.  
In one case it *creates an email address* and links that address to an existing 
user, when given a POSTed user_id.
In one case it *creates an email address* and does not link it to any user, if 
the auto_create POST option is False.

In terms of building an authentication system this REST method is problematic. 
All (I think at this stage anyway) other methods in the REST API can be given a 
clear authentication scope/role based only on the URL. This REST API method, 
dual to its multi purpose, mode options-in-the-POST-data behaviour, cannot be 
given an authentication scope based only on the URL.

Each of the above behaviours have a different authentication scope/role. For 
example:

(permitted by superuser only) In one case it *creates an email address* and 
*creates a new user* and which are linked to each other.  

(permitted by superuser only, permitted by existing user) In one case it 
*creates an email address* and links that address to an existing user, when 
given a POSTed user_id.

(permitted by superuser only) In one case it *creates an email address* and 
does not link it to any user, if the auto_create POST option is False.

I’m interested to hear thoughts on this. It would be good if the authentication 
scope of every REST API URL could be defined based only on the URL, without the 
need to know either options provided in the form data, or whether or not other 
records such as user records exist.

In summary - This API methods does too many things and depends on information 
apart from the URL to define its behaviour, making it hard to give a clear 
authentication scope to. I wonder if it should be broken out into individual 
REST API methods, each with a single purpose and a clear authentication scope.








___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/archive%40jab.org

Security Policy: http://wiki.list.org/x/QIA9

Re: [Mailman-Developers] When a user is deleted, is the uid record intentionally not deleted?

[Andrew Stuart]

Yes it’s a tough question. I agree with leaving the uid records in the database 
to prevent reuse. You are correct my test code is exercising a full working 
system.  It would be good if my tests could leave the database in the same 
state after completion of each test.

The ideal would be a method that is exactly the same as the existing user 
delete code but also deletes the users uid record.

maybe as you suggest, url route names like:
DELETE `api/testing/users/{user_identifier}'
or
DELETE `api/reserved/users/{user_identifier}’

Personally I’d not bother with exposing or hiding reserved methods based on 
environment variables or anything else - as you say, it’s an internal API not 
mean to used by the public so developers should know not to be using reserved 
or testing methods in their production code. The extra functionality might 
introduce bugs and needs writing and testing and documenting - not worth it. 

The fastest and simplest way ahead might be to duplicate the existing delete 
method under a new url route name (as above) and add “delete uid” functionality 
on top of the existing “delete user” code.

All of this is up to you though - I don’t want to be making work for others.

Just a heads up - there may be further similar questions coming up about the 
database because I’m only half way through writing my tests.


as


On 6 Feb 2015, at 3:04 pm, Barry Warsaw ba...@list.org wrote:

On Feb 05, 2015, at 10:53 AM, Andrew Stuart wrote:

 I notice that when a user is deleted, the uid record remains.
 
 Is this because there is a bug in which the uid should be deleted but is not?
 
 or is this intentional behaviour to prevent the uid from being recreated as a
 duplicate?
 
 My testing code does large numbers of creations and deletions of users so the
 deleted uid records are building up.

I think it's mostly that this is an unexpected user case.

You've probably seen this, but the UniqueIDFactory behaves differently when
the real system is running vs. when the test suite is running.  In the latter
case, we don't actually create new UID records in the database.  Instead we
use a file (protected by a lock) to monotonically increment a user-friendly
UID.  The testing machinery is hooked up to this so that when we reset the
database between tests, the uid factory gets reset too.

Under normal execution, we create a uuid4() instance, which generates a random
UUID, which is stored in the database, exactly so we can guarantee no
duplicates (they are unlikely to happen, but we have to make sure).

What I didn't expect was for there to be a testing scenario which would hit
the core while it was under normal operation mode, rather than testing mode.
I think that's what you're doing, right?

If so, then indeed, the UID table never gets cleared, and the UID for deleted
users never gets deleted.

I'm wary of deleting the UID when the user is deleted.  It seems like this
could leave us open to vulnerabilities where a UID gets reassigned to a new
user.  Your tests can't directly manipulate the database since you have to go
through the REST API.

If this is really a problem for you, then we'll have to expose something in
the REST API for you to clear out the UID table, or at least specific UIDs.
What I could imagine is that, under some situations (e.g. the presence of an
environment variable perhaps?) a new top-level resource would get exposed
called `api/testing` and under that, would be resources you could touch
(DELETE?) that would reset bits of the database for you.

For example:

   DELETE api/testing/uid_table

would drop all UID rows.  It would be up to you to call that at the
appropriate time.

Thoughts?

Cheers,
-Barry
___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/andrew.stuart%40supercoders.com.au

Security Policy: http://wiki.list.org/x/QIA9

___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/archive%40jab.org

Security Policy: http://wiki.list.org/x/QIA9

[Mailman-Developers] Security - XSS JavaScript and SQL injection in Mailman, Postorius and Hyperkitty

[Andrew Stuart]

Does the code of Mailman 3, Hyperkitty and Postorius do anything to address 
concerns around SQL and JavaScript injections, either from inbound emails or 
via the fields coming in via web interface or REST API?

thanks
___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/archive%40jab.org

Security Policy: http://wiki.list.org/x/QIA9

[Mailman-Developers] When a user is deleted, is the uid record intentionally not deleted?

[Andrew Stuart]

I notice that when a user is deleted, the uid record remains.

Is this because there is a bug in which the uid should be deleted but is not?

or is this intentional behaviour to prevent the uid from being recreated as a 
duplicate?

My testing code does large numbers of creations and deletions of users so the 
deleted uid records are building up.


___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/archive%40jab.org

Security Policy: http://wiki.list.org/x/QIA9

[Mailman-Developers] Is there a way to delete an unlinked/orphaned address via the REST API?

[Andrew Stuart]

After unlinking an address from a user, there is an address in the addresses 
table that is orphaned - i.e. it does not have a linked user. This is the 
expected behaviour (I think) - that’s fine, no problems at this point.

However, I can’t see a way to delete these addresses via the REST API.  Does 
one exist? Should it exist? Perhaps so - without it there’s no way to delete 
orphaned email addresses.

thanks

as

___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/archive%40jab.org

Security Policy: http://wiki.list.org/x/QIA9

Re: [Mailman-Developers] Address already exists when creating users, but it creates a new user anyway

[Andrew Stuart]

After executing the curl command:

curl -X POST --data email=mailmanad...@example.org --header authorization: 
Basic cmVzdGFkbWluOnJlc3RwYXNz http://localhost:8001/3.0/users

The log file adds the following:

Feb 04 11:30:05 2015 (17776) BEGIN (implicit)
Feb 04 11:30:05 2015 (17776) SELECT count(*) AS count_1
FROM (SELECT uid.id AS uid_id, uid.uid AS uid_uid
FROM uid
WHERE uid.uid = ?) AS anon_1
Feb 04 11:30:05 2015 (17776) ('57fdf5a214334b388d6efc3c375dc392',)
Feb 04 11:30:05 2015 (17776) INSERT INTO uid (uid) VALUES (?)
Feb 04 11:30:05 2015 (17776) ('57fdf5a214334b388d6efc3c375dc392',)
Feb 04 11:30:05 2015 (17776) SELECT count(*) AS count_1
FROM (SELECT user.password AS user_password, user.id AS user_id, 
user.display_name AS user_display_name, user._user_id AS user__user_id, 
user._created_on AS user__created_on, user._preferred_address_id AS 
user__preferred_address_id, user.preferences_id AS user_preferences_id
FROM user
WHERE user._user_id = ?) AS anon_1
Feb 04 11:30:05 2015 (17776) ('57fdf5a214334b388d6efc3c375dc392',)
Feb 04 11:30:05 2015 (17776) INSERT INTO preferences (acknowledge_posts, 
hide_address, preferred_language, receive_list_copy, receive_own_postings, 
delivery_mode, delivery_status) VALUES (?, ?, ?, ?, ?, ?, ?)
Feb 04 11:30:05 2015 (17776) (None, None, None, None, None, None, None)
Feb 04 11:30:05 2015 (17776) INSERT INTO user (display_name, password, 
_user_id, _created_on, _preferred_address_id, preferences_id) VALUES (?, ?, ?, 
?, ?, ?)
Feb 04 11:30:05 2015 (17776) ('', None, '57fdf5a214334b388d6efc3c375dc392', 
'2015-02-04 11:30:05.112836', None, 7)
Feb 04 11:30:05 2015 (17776) SELECT count(*) AS count_1
FROM (SELECT address.verified_on AS address_verified_on, address.id AS 
address_id, address.email AS address_email, address._original AS 
address__original, address.display_name AS address_display_name, 
address.registered_on AS address_registered_on, address.user_id AS 
address_user_id, address.preferences_id AS address_preferences_id
FROM address
WHERE address.email = ?) AS anon_1
Feb 04 11:30:05 2015 (17776) ('mailmanad...@example.org',)
Feb 04 11:30:05 2015 (17776) SELECT address.verified_on AS address_verified_on, 
address.id AS address_id, address.email AS address_email, address._original AS 
address__original, address.display_name AS address_display_name, 
address.registered_on AS address_registered_on, address.user_id AS 
address_user_id, address.preferences_id AS address_preferences_id
FROM address
WHERE address.email = ?
 LIMIT ? OFFSET ?
Feb 04 11:30:05 2015 (17776) ('mailmanad...@example.org', 1, 0)
Feb 04 11:30:05 2015 (17776) COMMIT



___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/archive%40jab.org

Security Policy: http://wiki.list.org/x/QIA9

Re: [Mailman-Developers] Address already exists when creating users, but it creates a new user anyway

[Andrew Stuart]

 (
id INTEGER NOT NULL,
key VARCHAR,
request_type INTEGER,
data_hash VARCHAR,
mailing_list_id INTEGER,
PRIMARY KEY (id),
FOREIGN KEY(mailing_list_id) REFERENCES mailinglist (id)
);
CREATE TABLE address (
id INTEGER NOT NULL,
email VARCHAR,
_original VARCHAR,
display_name VARCHAR,
verified_on DATETIME,
registered_on DATETIME,
user_id INTEGER,
preferences_id INTEGER,
PRIMARY KEY (id),
FOREIGN KEY(user_id) REFERENCES user (id),
FOREIGN KEY(preferences_id) REFERENCES preferences (id)
);
INSERT INTO address 
VALUES(1,'mailmanad...@example.org',NULL,'',NULL,'2015-02-04 
10:11:03.108943',1,2);
CREATE TABLE member (
id INTEGER NOT NULL,
_member_id CHAR(32),
role INTEGER,
list_id VARCHAR,
moderation_action INTEGER,
address_id INTEGER,
preferences_id INTEGER,
user_id INTEGER,
PRIMARY KEY (id),
FOREIGN KEY(address_id) REFERENCES address (id),
FOREIGN KEY(preferences_id) REFERENCES preferences (id),
FOREIGN KEY(user_id) REFERENCES user (id)
);
CREATE TABLE autoresponserecord (
id INTEGER NOT NULL,
address_id INTEGER,
mailing_list_id INTEGER,
response_type INTEGER,
date_sent DATE,
PRIMARY KEY (id),
FOREIGN KEY(address_id) REFERENCES address (id),
FOREIGN KEY(mailing_list_id) REFERENCES mailinglist (id)
);
CREATE TABLE onelastdigest (
id INTEGER NOT NULL,
mailing_list_id INTEGER,
address_id INTEGER,
delivery_mode INTEGER,
PRIMARY KEY (id),
FOREIGN KEY(mailing_list_id) REFERENCES mailinglist (id),
FOREIGN KEY(address_id) REFERENCES address (id)
);
CREATE TABLE alembic_version (
version_num VARCHAR(32) NOT NULL
);
INSERT INTO alembic_version VALUES('33e1f5f6fa8');
CREATE INDEX ix_uid_uid ON uid (uid);
CREATE INDEX ix_user__user_id ON user (_user_id);
CREATE INDEX ix_user_preferences_id ON user (preferences_id);
CREATE INDEX ix_pendedkeyvalue_pended_id ON pendedkeyvalue (pended_id);
CREATE INDEX ix_listarchiver_mailing_list_id ON listarchiver (mailing_list_id);
CREATE INDEX ix_acceptablealias_alias ON acceptablealias (alias);
CREATE INDEX ix_acceptablealias_mailing_list_id ON acceptablealias 
(mailing_list_id);
CREATE INDEX ix_contentfilter_mailing_list_id ON contentfilter 
(mailing_list_id);
CREATE INDEX ix__request_mailing_list_id ON _request (mailing_list_id);
CREATE INDEX ix_address_user_id ON address (user_id);
CREATE INDEX ix_address_preferences_id ON address (preferences_id);
CREATE INDEX ix_autoresponserecord_address_id ON autoresponserecord 
(address_id);
CREATE INDEX ix_autoresponserecord_mailing_list_id ON autoresponserecord 
(mailing_list_id);
COMMIT;
(venv3.4)ubuntu@server01:~/mailman/var/data$


On 2 Feb 2015, at 9:28 am, Barry Warsaw ba...@list.org wrote:

On Feb 01, 2015, at 07:23 PM, Andrew Stuart wrote:

 In the commands below you can see that I start by getting a list of users, it
 returns total_size: 1

How was this user created and with what email address?

 I then execute a POST to the REST API multiple times.  Each time it says
 “Address already exists”, and then getting a list of users says total_size:
 2
 
 Is this the way its intended to work or a bug?

Nope, and I can't reproduce it.  Are you sure there are no other processes
running, and nothing else hitting the REST API that could be racing with that
POST?

Try tailing the mailman.log and watching for http activity.

Cheers,
-Barry
___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/andrew.stuart%40supercoders.com.au

Security Policy: http://wiki.list.org/x/QIA9

___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/archive%40jab.org

Security Policy: http://wiki.list.org/x/QIA9

Re: [Mailman-Developers] Address already exists when creating users, but it creates a new user anyway

[Andrew Stuart]

After a bit of deep cave diving into the code I suspect it’s this in 
model/usermanager.py

This function appears to create a user before it checks to see if the address 
exists, with the result that multiple users are created when the address 
already exists.

@implementer(IUserManager)
class UserManager:
See `IUserManager`.

def create_user(self, email=None, display_name=None):
See `IUserManager`.
user = User(display_name, Preferences())
if email:
address = self.create_address(email, display_name)
user.link(address)
return user


this seems to fix the problem - but please don’t trust my solution - needs 
verification.

@implementer(IUserManager)
class UserManager:
See `IUserManager`.

def create_user(self, email=None, display_name=None):
See `IUserManager`.
if email:
address = self.create_address(email, display_name)
user = User(display_name, Preferences())
if email:
user.link(address)
return user


On 5 Feb 2015, at 7:59 am, Barry Warsaw ba...@list.org wrote:

On Feb 05, 2015, at 07:23 AM, Andrew Stuart wrote:

 To get the source I do:
 
 bzr branch lp:mailman
 
 Is this the correct way?
 
 I’m using Python 3.4

Yep, exactly so.

Thanks.  I'll give it a go tonight.
-B

___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/archive%40jab.org

Security Policy: http://wiki.list.org/x/QIA9

[Mailman-Developers] How to drop and recreate database using Alembic?

[Andrew Stuart]

Anyone got quick tips for dropping the entiure database and recreating it with 
alembic?

I can’t find any instructions - if there are some would you mind please 
pointing me to them?

thanks

as

___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/archive%40jab.org

Security Policy: http://wiki.list.org/x/QIA9

Re: [Mailman-Developers] User id - integer or UUID?

[Andrew Stuart]

That’s fine I’m good with that.

The other possibility, distasteful as it may be, is to make the changes to 
publish the UUID to the REST API and deprecate the use of the user_id as 
integer. I suggest this because others who come after me will have the same 
question and may run into the same issues as I have in trying to work out a 
“broadly compatible” way of handling/storing integers. They’ll probably come up 
with their own solution - perhaps to store user ID’s as a variety of integer 
types/lengths - perhaps to convert to UUID - perhaps to store 
integer-as-string, and the question might be raised again.

Short term pain now to move to UUID and deprecate user_id as integer, possibly 
longer term gain with clarity that a user_id is a UUID.

Anyway I don’t mind much anyway I’ll just do as Barry says…..

as



On 2 Feb 2015, at 9:33 am, Barry Warsaw ba...@list.org wrote:

On Feb 01, 2015, at 11:54 AM, Andrew Stuart wrote:

 I’m going to store as string representation of the integer as I don’t want to
 be dealing with databases or anything else that makes a distinction between
 different types and lengths of integer.

That makes sense to me.  Since Aurelien was okay with the status quo, I never
did make the change described in that archive message.

 Ideally the identifier would match the core internal identifier which is UUID
 but since userid-as-UUID is not seen from the REST API side of things then
 I’ll act like that concept does not exist and hope I’m not making things more
 confused around user id.

Yep, for now I think we'll keep the int-as-user-id concept, and it will be
converted internally to a UUID.  If/when we change that I'll have to keep
backward compatibility with that API anyway.

Cheers,
-Barry
___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/andrew.stuart%40supercoders.com.au

Security Policy: http://wiki.list.org/x/QIA9

___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/archive%40jab.org

Security Policy: http://wiki.list.org/x/QIA9

Re: [Mailman-Developers] User id - integer or UUID?

[Andrew Stuart]

I’m going to store as string representation of the integer as I don’t want to 
be dealing with databases or anything else that makes a distinction between 
different types and lengths of integer. Ideally the identifier would match the 
core internal identifier which is UUID but since userid-as-UUID is not seen 
from the REST API side of things then I’ll act like that concept does not exist 
and hope I’m not making things more confused around user id.

On 1 Feb 2015, at 11:31 am, Andrew Stuart andrew.stu...@supercoders.com.au 
wrote:

I have read previous discussion here about format of user_id

https://mail.python.org/pipermail/mailman-developers/2014-October/024024.html

That’s fine, I understand the issues.

The question is, I am storing user ids and it’s not clear to me if I should be 
storing them as INTs as returned by the REST API or converting back to UUID?

It’s important to get right now because Im going to use the same approach 
everywhere.

thanks
___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/andrew.stuart%40supercoders.com.au

Security Policy: http://wiki.list.org/x/QIA9

___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/archive%40jab.org

Security Policy: http://wiki.list.org/x/QIA9

[Mailman-Developers] User id - integer or UUID?

[Andrew Stuart]

I have read previous discussion here about format of user_id

https://mail.python.org/pipermail/mailman-developers/2014-October/024024.html

That’s fine, I understand the issues.

The question is, I am storing user ids and it’s not clear to me if I should be 
storing them as INTs as returned by the REST API or converting back to UUID?

It’s important to get right now because Im going to use the same approach 
everywhere.

thanks
___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/archive%40jab.org

Security Policy: http://wiki.list.org/x/QIA9

[Mailman-Developers] Launch band song

[Andrew Stuart]

Hey Barry

If you are thinking of launching at Pycon, what do you think of the idea of 
playing some music - you’re a guitarist right? Maybe there could be “The 
Mailmen” - a Mailman band - if you can rope in some other musicians.  Perhaps 
the song could be a play on Elton John’s “Rocket Man”.

A bit of marketing theatre for the bloggers to post a Youtube video of.  Would 
help to get the word out there about the launch.

as

___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/archive%40jab.org

Security Policy: http://wiki.list.org/x/QIA9

Re: [Mailman-Developers] Who is the site administrator?

[Andrew Stuart]

OK.

How about I make a standalone User Authorisation based server that has a user 
data store with additional arbitrary user keys in it? It would also allow role 
information to be assigned to those users via it’s own REST API (which I would 
have to think about and make up).

Thus my API proxy (I’m calling it CrowdWave) will be a public front end that 
looks like this:

public HTTP request == CrowdWave API proxy == Mailman REST API
public HTTP request == CrowdWave API proxy == CrowdWave User Authorisation 
server

The CrowdWave User Authorisation server will use Falcon and Sqlalchemy with a 
default sqlite database.

I’ll just do it to meet my need for resolution of question of how to give 
public access to higher than list level functions.  I’m not going to overthink 
it because I just want a way to give public access to all Mailman REST API 
functions. Consistent with earlier discussion, the CrowdWave User Authorisation 
server would not in the short term be part of the Mailman project and would be 
a third party project.

After this is built it might be an effective thinking point for working out the 
official mechanism for authorisation. 

Barry - if you want me to implement a specific approach now would be a good 
time to say - I’m happy to do anything as long as I feel up to it from a coding 
perspective.  Right now I’m aiming for super simple.

What do you think?

as



On 26 Jan 2015, at 4:44 am, Barry Warsaw ba...@list.org wrote:

On Jan 24, 2015, at 04:05 PM, Andrew Stuart wrote:

 The main thing I’m looking for is whether there is an authorisation concept
 that operates at a higher level than the list.

No, there isn't[*].

 I wonder is there the concept of some sort of “special” mailing list that is
 different or hidden or privileged in some way? There is a reference in the
 documentation to a “site list” but I coudn’t find much more about it. If
 there is a “special” list then maybe site wide user priveleges can be stored
 against it.

There is no site list in Mailman 3.

 I can see there is a site_owner in the Mailman config file although this just
 seems to be an email address to get bounces in certain circumstances, is it
 used for anything?

Just for bounces, as you've discovered.

 Well I read the discussion thread.  I had no idea so much previous thinking
 had gone into this.  Funny that there was so much discussion but no-one has
 really chimed in with an opinion about how I’ve built the authentication
 proxy - maybe everyone’s gun-shy. :-)

I haven't yet had time to dig into your work - apologies for that!

 The only unresolved question for me coming out of all that is “where is it
 specified which users have access rights to carry out functions at a higher
 level than the list?”
 
 It appears that such information is not part of the Mailman core user data
 model and needs to be handled outside Mailman core i.e. at the application
 level.

This is correct.  It's the big reason why currently the API is an all-access
administrative API.  If the core was explicit in the roles an access right,
then an authenticating proxy wouldn't be necessary.

Sumana's summary is essentially correct.  There seems to be 6 (-ish) natural
roles that actors can play.  We can disregard the site administrator as that's
the person with shell access and the necessary permissions to pop the hood
and do anything.

Given that we have mailing lists organized into domains, it makes sense that
each mailing list have its owner, each domain have its manager, and the system
as a whole has a manager as well.  The system manager can do things like
create new domains, install global bans, and can delegate responsibility for
domains and mailing lists to others.

For a site with multiple domains, a domain manager can create lists within
that domain and delegate management of mailing lists to the owners of those
lists.  List owners can change any setting, and can delegate moderation
privileges to list moderators.

Users can manage their own subscriptions, and the various settings associated
with their memberships.

In Mailman 3, this breakdown is purely conceptual, and as you've seen there
has been heated discussion in the past.  I'm open to rebooting that.  I have
strong opinions but am willing to be persuaded, especially if there is code to
back it up. :)

 Personally I’d store such user information inside the Mailman core data
 model. Things start to look weird when application X has a different
 mechanism for deciding who can do admin tasks to application Y.  Maintaining
 such info outside the core also calls for user database synchronisation which
 would be good to avoid. But I don’t really mind I’m just trying to get a
 secure way of deciding which users to allow to do higher-than-list-level
 functions.

Alternatively, once the core bakes in these assumptions, then it becomes very
difficult for other tools to experiment with other ways of doing it, or
integrating the decision making with other external systems.

I've

Re: [Mailman-Developers] Who is the site administrator?

[Andrew Stuart]

Well I read the discussion thread.  I had no idea so much previous thinking had 
gone into this.  Funny that there was so much discussion but no-one has really 
chimed in with an opinion about how I’ve built the authentication proxy - maybe 
everyone’s gun-shy. :-)

The only unresolved question for me coming out of all that is “where is it 
specified which users have access rights to carry out functions at a higher 
level than the list?”

It appears that such information is not part of the Mailman core user data 
model and needs to be handled outside Mailman core i.e. at the application 
level.

Personally I’d store such user information inside the Mailman core data model. 
Things start to look weird when application X has a different mechanism for 
deciding who can do admin tasks to application Y.  Maintaining such info 
outside the core also calls for user database synchronisation which would be 
good to avoid. But I don’t really mind I’m just trying to get a secure way of 
deciding which users to allow to do higher-than-list-level functions.

My options are:

1: eliminate public access to all functions that operate higher than list level
2: bake my own additional level of user authentication that defines which users 
have higher level access rights.
3: find a cheat/workaround to store arbitrary keys against users (this my 
preferred solution)
4: something else

Option 1 drastically reduces the usefulness of the interface if only list level 
functions can be executed. Not ideal.
Option 2 I would prefer to avoid - my authentication proxy doesn’t actually 
know much about Mailman at an application level - all it does is authorise and 
authenticate and pass through requests to Mailman if the user is allowed to do 
it, and reject them if not.
Option 3: find some sort of cheat/workaround - preferablyt a way to store 
additional user information in a user profile field which allows storage of 
arbitrary keys.  Then I could stash some access control information in there 
that tells me that user X or user Y has some higher level access control. This 
would have to be do-able via the Mailman REST interface cause it’s the only way 
that I talk to the system. The ability to store arbitrary key/value information 
against a user would be the ideal outcome at this stage.
Option 4: - other ideas?

Does anyone know if there is a way to do Option 3 currently?

as


On 25 Jan 2015, at 7:37 am, Terri Oda te...@toybox.ca wrote:


On 2015-01-24, 5:08 AM, Andrew Stuart wrote:
 you can look back in the archives to the huge discussion we had about 
 unifying the user data store..
 
 Is there search to find that discussion or do I have to manually review all 
 the threads? Any idea what year it was in?

The data store discussion was I think two years ago around the time of pycon.  
I started it with a thread entitled Architecture for extra profile info

https://mail.python.org/pipermail/mailman-developers/2013-April/022852.html

(As Sumana says, mail-archive is usually the best bet for searching the mailman 
archives.  I searched locally, though.)

But I'm serious that it's only worth reading if you're incredibly bored.  It 
became a huge architecture discussion that only ended on the lists with an 
agree to disagree. (and I got bombarded with further comments about it on IRC 
for months after the list discussion ended, so I'm actually kind of still 
cranky about it.)

I think it's still a thing we need to look in to more seriously, but be warned 
that it's a topic folk have been passionate about in the past, to the point 
where progress was not made.  On the other hand, one of the people involved in 
that discussion has moved on to other things and the rest of us do 
collaborative work together more frequently, so it might be easier to reach a 
solution now.

Terri

___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/andrew.stuart%40supercoders.com.au

Security Policy: http://wiki.list.org/x/QIA9

___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/archive%40jab.org

Security Policy: http://wiki.list.org/x/QIA9

Re: [Mailman-Developers] Who is the site administrator?

[Andrew Stuart]

you can look back in the archives to the huge discussion we had about 
unifying the user data store..

Is there search to find that discussion or do I have to manually review all the 
threads? Any idea what year it was in?

thanks



On 24 Jan 2015, at 4:16 pm, Terri Oda te...@toybox.ca wrote:

On 2015-01-23, 9:05 PM, Andrew Stuart wrote:
 you can have users with site privileges
 
 I had a look at the data model but couldn’t see this. Would you mind please 
 pointing me in the direction of where user site privilege information is 
 stored?

I'm not sure the concept is visible in Mailman Core, but I'm pretty sure it's 
in Postorius.

(And if you're bored, you can look back in the archives to the huge discussion 
we had about unifying the user data store... ;)  )

___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/archive%40jab.org

Security Policy: http://wiki.list.org/x/QIA9

Re: [Mailman-Developers] How can I run the REST tests against live servers?

[Andrew Stuart]

Sure thing that’s fine.

FYI the Mailman REST authentication proxy works in this way:

** Python 3 only
** built using Falcon
** the proxy reads the structure of the Mailman 3 REST API by parsing the 
Mailman 3 Swagger API specification
** each Mailman 3 REST API URL/HTTP method call combination is defined in the 
Swagger spec and has a unique OperationID
** the proxy intercepts all inbound requests to the Mailman 3 REST API, this is 
done through Falcon’s “sink” capability
** the proxy does a regex match on the inbound URL and looks up the operationID 
to work out which Mailman 3 REST API method is being called
** in the proxy a set of request handlers is defined for each Mailman 3 API 
operation 
** the inbound request is pushed through the request handlers, typically the 
final request handler proxies the authenticated request to the Mailman REST 
API, unless one of the request handlers aborts the processing due to some sort 
of auth issue

A typical example of one API operation mapping to its set of handlers looks 
something like this:

{‘UsersGet’: [validateJSONAuthToken, refreshJSONAuthToken, userMustbeListAdmin, 
proxyRequestToMailman]}

In the above example the inbound request goes through a handler that validates 
its JWT authentication token, a handler that refreshes its JWT token, a handler 
that checks to ensure that the requesting user (as identified from the JWT 
token) has the appropriate rights to carry out the requested Mailman 3 
operation, then the request is proxied to the Mailman REST API and the result 
is send back to the client/requestor.

JSON web token authentication is automatically layered on top of the API.

Typically you would want the authenticating proxy to sit behind a reverse proxy 
such as HAproxy or Nginx which would do gzip compression and SSL.

Most of this is working.

as


On 22 Jan 2015, at 2:57 am, Barry Warsaw ba...@list.org wrote:

On Jan 22, 2015, at 12:02 AM, Andrew Stuart wrote:

 Barry - would you like the authenticating proxy to be part of Mailman core or
 a third party project?

For now, I think it should not be distributed as part of core.  Maybe after
the 3.0 release.  The code could still be donated to the FSF and managed under
the GNU Mailman project in a separate source repository.  That's kind of your
call; let me know if you'd like to do that.

 It’s close to being functionally operational (which is to say working but in
 a raw state).

Very cool.  Thanks for working on this important piece.

Cheers,
-Barry
___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/andrew.stuart%40supercoders.com.au

Security Policy: http://wiki.list.org/x/QIA9

___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/archive%40jab.org

Security Policy: http://wiki.list.org/x/QIA9

[Mailman-Developers] Who is the site administrator?

[Andrew Stuart]

Is this defined in any way?

Is there any concept that a given Mailman user has “site admin rights”?

Or is the concept of the “site administrator” not realised in the application 
and just considered to be someone with operating system level access to the 
system that Mailman is running on?

thanks

___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/archive%40jab.org

Security Policy: http://wiki.list.org/x/QIA9

Re: [Mailman-Developers] Who is the site administrator?

[Andrew Stuart]

you can have users with site privileges

I had a look at the data model but couldn’t see this. Would you mind please 
pointing me in the direction of where user site privilege information is stored?

I can see that there is the concept of “Rosters” but rosters seem to apply to 
lists and not at a higher level i.e. as far as I can see there’s no way to 
apply a roster to the system rather than to a list.

The main thing I’m looking for is whether there is an authorisation concept 
that operates at a higher level than the list.

I wonder is there the concept of some sort of “special” mailing list that is 
different or hidden or privileged in some way? There is a reference in the 
documentation to a “site list” but I coudn’t find much more about it. If there 
is a “special” list then maybe site wide user priveleges can be stored against 
it.

I can see there is a site_owner in the Mailman config file although this just 
seems to be an email address to get bounces in certain circumstances, is it 
used for anything?



as



On 24 Jan 2015, at 3:13 pm, Terri Oda te...@toybox.ca wrote:

On 2015-01-23, 7:12 PM, Andrew Stuart wrote:
 Is this defined in any way?
 
 Is there any concept that a given Mailman user has “site admin rights”?
 
 Or is the concept of the “site administrator” not realised in the application 
 and just considered to be someone with operating system level access to the 
 system that Mailman is running on?

In Mailman 2, there's no admin users in a typical sense but the site 
administrator is someone who knows the global password for a site and/or has 
shell access. The global password lets you log in as an admin to any list, and 
depending on your settings may let you do things like create and delete lists.  
And yes, it's really just one shared password

In Mailman 3 there's a much more nuanced user system and you can have users 
with site privileges (and there's no longer a need to share passwords).

Terri


___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/andrew.stuart%40supercoders.com.au

Security Policy: http://wiki.list.org/x/QIA9

___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/archive%40jab.org

Security Policy: http://wiki.list.org/x/QIA9

Re: [Mailman-Developers] How can I run the REST tests against live servers?

[Andrew Stuart]

Barry - would you like the authenticating proxy to be part of Mailman core or a 
third party project?

It’s close to being functionally operational (which is to say working but in a 
raw state).

as



On 12 Dec 2014, at 11:34 am, Barry Warsaw ba...@list.org wrote:

On Dec 12, 2014, at 08:35 AM, Andrew Stuart wrote:

 I’m writing an authenticating proxy for the Mailman REST API and want to make
 sure everything works as expected against real infrastructure
 (Postgres/Postfix/Sqlalchemy/Falcon).

We definitely want such a proxy.  It's always been our intent that the core's
REST API is an adminipstrative API with full access.  It should only be
exposed on localhost or other highly secure interface.

A proxy server such as you describe would be available on a public interface
and would allow people to script the management of their list memberships and
list administrations.

 I was hoping that just pointing the REST tests request port to my proxy would
 work but no dice - lots of failures - trying to diagnose the failures has led
 me back to trying to get the test suite to work against the live servers.
 
 I’m wondering if maybe the shortest route here might not be for me to write a
 new set of tests which purely talk REST and don’t touch the back end directly
 at all.  Except that’s not a short route. :-)
 
 Trying to bend the existing tests to meet my needs is looking harder than I
 have time to deal with.

Even in this case, I wouldn't expect the standard unittests to work.  For
example, if a normal user were to try to access the list admin API, you'd
expect that to fail.

I think one approach would be to continue to use the test servers, but have a
backdoor connection which would let you set up the state before each test and
reset the state after each test.  Then the tests themselves would use the
authenticated API to flex the things it can and shouldn't be able to do.
It would be possible for example to expose some helper resources in the admin
API which would not normally be exposed in production.  Something under a top
level /tests resource perhaps.

Cheers,
-Barry
___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/andrew.stuart%40supercoders.com.au

Security Policy: http://wiki.list.org/x/QIA9

___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/archive%40jab.org

Security Policy: http://wiki.list.org/x/QIA9

Re: [Mailman-Developers] Your confirmation is required to leave the Mailman-Developers mailing list

[Andrew Stuart]

Should Mailman be stripping these out as part of its processing?

On 12 Jan 2015, at 2:26 pm, Mark Sapiro m...@msapiro.net wrote:

On 01/11/2015 06:27 PM, Andrew Stuart wrote:
 errr…. is this someone saying they would prefer I wasn’t on this list? I’d 
 rather hear directly from someone if you feel that way.


Unlikely it's that. More likely, its because you've replied to posts
without removing the msg_footer from the quoted material in your reply.
See, e.g.,
https://mail.python.org/pipermail/mailman-developers/2015-January/024152.html.

Someone sees your post and thinks that's their unsubscribe link.

Be glad you didn't do what I once unthinkingly did at
https://mail.python.org/pipermail/mailman-users/2010-December/070723.html.
Every time Google, Bing, etc crawl that page, I get an unsub confirmation.

-- 
Mark Sapiro m...@msapiro.netThe highway is for gamblers,
San Francisco Bay Area, Californiabetter use your sense - B. Dylan
___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/andrew.stuart%40supercoders.com.au

Security Policy: http://wiki.list.org/x/QIA9

___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/archive%40jab.org

Security Policy: http://wiki.list.org/x/QIA9

Re: [Mailman-Developers] REST API returns are plain text for errors, JSON or nothing for successful

[Andrew Stuart]

Yes I’d favor leave as is, and at some stage in future convert the error 
messages to JSON.



On 12 Jan 2015, at 12:22 pm, Barry Warsaw ba...@list.org wrote:

On Jan 11, 2015, at 06:23 AM, Andrew Stuart wrote:

 Happy to do so if you like.  Would you mind pointing me in the direction of
 some instructions for filing bugs please?
 
 FWIW I’d probably favor a REST API that is behaving improperly by returning
 plain text error responses (i.e. the current situation) rather than a REST
 API that returns application/json content type for valid responses and
 text/plain for errors.  Once that gets baked into release 1 it’ll be
 challenging to correct back to application/json later because people will
 write clients to expect text/plain for errors.
 
 It’s a whole lot easier conceptually to deal with a REST API that always
 returns the same content-type regardless of the outcome of the API call.
 
 Some REST tools don’t have any mechanism for defining mixed response type
 APIs which leads to some weird outcomes when querying successfully versus
 getting errors.  In short, I’d leave as-is and eventually turn the error text
 into json, but obviously Barry you should say what you want.

I think at least the Content-Type should be accurate.

Your response sounds a little contradictory, but let me see if I can summarize
it.  From a practicality view point, you prefer the current implementation,
even if it isn't conceptually pure, at least for now.

That works for me because then I don't have to worry about changing
anything. :)  I'm open to re-evaluating that in some future Mailman release.

Cheers,
-Barry
___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/andrew.stuart%40supercoders.com.au

Security Policy: http://wiki.list.org/x/QIA9

___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/archive%40jab.org

Security Policy: http://wiki.list.org/x/QIA9

Re: [Mailman-Developers] REST API first pass at Swagger spec

[Andrew Stuart]

With Swagger you can either:
1: integrate the spec into the API source code
 or 
2: have one single standalone JSON file that contains the whole specification 
(api-docs.json in this case)

I’m not a fan of integrating the Swagger specification into the code - seems to 
make for a VERY large amount of work, a huge amount of complexity and massive 
potential for introduction of bugs.

I’ve taken the approach of a single static JSON file - MUCH more simple.  All 
Mailman would need is to include this file (api-docs.json)  as a static file 
that can be read or if there is no static file server in Mailman (is there? I 
haven’t looked) maybe create an api-docs.json endpoint in the API that returns 
the static file. Any tools that want to use the api-docs.json spec file don’t 
need to be included in Mailman core.

The api-docs.json spec file would need to be kept up to date when the Mailman 
API changes but it’s really straightforward - building it in the first place is 
just tedious research on the methods and properties but maintaining it should 
be a matter of a few minutes work to update it when the API changes.

I’ll need to finish it up.  I’m not sure what to do about getting a pair of 
eyes to QA verify it, but maybe we can cross that bridge after I’ve finished it.


On 12 Jan 2015, at 12:19 pm, Barry Warsaw ba...@list.org wrote:

On Jan 11, 2015, at 08:51 AM, Andrew Stuart wrote:

 Attached is a first pass at a Swagger spec for the REST API.
 
 You can find out about Swagger at http://swagger.io
 
 The Swagger spec that I am working on is attached to this message or can be
 found here: http://www.mailripper.com/api-docs.json
 
 If you want to see the attached spec in action against the Mailman REST API,
 go to http://www.mailripper.com/swagger-ui/index.html
 
 Give it a try at http://www.mailripper.com/swagger-ui/index.html - you are
 welcome to change anything, it’s a test Mailman server and you can add or
 delete anything - nothing you do will be a problem.
 
 There’s still a fair bit to do to finalise this but most of the API queries
 are working - enough for you to see how it works.

It looks interesting.  I have a little experience with WADL with the Launchpad
API.  There was a lot of infrastructure built into lazr.restful to
automatically generate the WADL, but it was so cumbersome that I was ecstatic
to remove it all for restish (at the time).

My big question is how to make sure that the swagger spec doesn't get out of
date to the implementation.  I definitely don't want to add a lot of cruft to
the implementation - the light weight of the current falcon-based
implementation is a huge benefit.  It would be okay (maybe ideal) if some how
the swagger spec was integrated with the core's test suite, so that at least
we'd see failures if the API were modified (e.g. new resources and methods
added) without the spec being similarly updated.

Ultimately I think if it isn't much effort to keep up to date, and is useful
to consumers of the API, it could be a good thing, but I don't really have any
opinions about swagger as compared to alternatives.

Cheers,
-Barry
___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/andrew.stuart%40supercoders.com.au

Security Policy: http://wiki.list.org/x/QIA9

___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/archive%40jab.org

Security Policy: http://wiki.list.org/x/QIA9

[Mailman-Developers] Fwd: Your confirmation is required to leave the Mailman-Developers mailing list

[Andrew Stuart]

errr…. is this someone saying they would prefer I wasn’t on this list? I’d 
rather hear directly from someone if you feel that way.




Begin forwarded message:

From: 
mailman-developers-confirm+e1a7bed1fbf00db07dc7590dcbdba1e364ecc...@python.org
To: andrew.stu...@supercoders.com.au
Subject: Your confirmation is required to leave the Mailman-Developers mailing 
list
Reply-To: 
mailman-developers-confirm+e1a7bed1fbf00db07dc7590dcbdba1e364ecc...@python.org
Date: 12 January 2015 1:23:31 pm AEDT

Mailing list removal confirmation notice for mailing list
Mailman-Developers

We have received a request from 68.196.224.128 for the removal of your
email address, andrew.stu...@supercoders.com.au from the
mailman-developers@python.org mailing list.  To confirm that you want
to be removed from this mailing list, simply reply to this message,
keeping the Subject: header intact.  Or visit this web page:

   
https://mail.python.org/mailman/confirm/mailman-developers/e1a7bed1fbf00db07dc7590dcbdba1e364ecc240


Or include the following line -- and only the following line -- in a
message to mailman-developers-requ...@python.org:

   confirm e1a7bed1fbf00db07dc7590dcbdba1e364ecc240

Note that simply sending a `reply' to this message should work from
most mail readers, since that usually leaves the Subject: line in the
right form (additional Re: text in the Subject: is okay).

If you do not wish to be removed from this list, please simply
disregard this message.  If you think you are being maliciously
removed from the list, or have any other questions, send them to
mailman-developers-ow...@python.org.

___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/archive%40jab.org

Security Policy: http://wiki.list.org/x/QIA9

Re: [Mailman-Developers] REST API first pass at Swagger spec

[Andrew Stuart]

 as soon as Andrew loses interest in keeping it up to date.”

;-) I was never interested in keeping it up to date in the first place but I 
can’t stand doing development without API documentation, a necessary first step 
to coding to the API.

I do like your idea about changes to the API not being complete until they have 
been implemented into the JSON spec.

On 12 Jan 2015, at 12:46 pm, Stephen J. Turnbull step...@xemacs.org wrote:

Barry Warsaw writes:

 My big question is how to make sure that the swagger spec doesn't
 get out of date to the implementation.

Doesn't Andrew's use case answer that?  I mean, if the Swagger spec is
out of date, the tests based on it should fail.  If the tests don't
fail, they're incomplete anyway, and that's a different problem that
no spec language can solve for you.

As a matter of process, if Swagger is readable enough, we make it
*the* spec, and you don't accept pull requests for the REST
implementation without a patch to the spec.  If Swagger isn't usable
in that fashion, I suppose it will go away again like WADL as soon as
Andrew loses interest in keeping it up to date. ;-)
___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/andrew.stuart%40supercoders.com.au

Security Policy: http://wiki.list.org/x/QIA9

___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/archive%40jab.org

Security Policy: http://wiki.list.org/x/QIA9

Re: [Mailman-Developers] REST API returns are plain text for errors, JSON or nothing for successful

[Andrew Stuart]

Hmmm.

Happy to do so if you like.  Would you mind pointing me in the direction of 
some instructions for filing bugs please?

FWIW I’d probably favor a REST API that is behaving improperly by returning 
plain text error responses (i.e. the current situation) rather than a REST API 
that returns application/json content type for valid responses and text/plain 
for errors.  Once that gets baked into release 1 it’ll be challenging to 
correct back to application/json later because people will write clients to 
expect text/plain for errors. 

It’s a whole lot easier conceptually to deal with a REST API that always 
returns the same content-type regardless of the outcome of the API call. 

Some REST tools don’t have any mechanism for defining mixed response type APIs 
which leads to some weird outcomes when querying successfully versus getting 
errors.  In short, I’d leave as-is and eventually turn the error text into 
json, but obviously Barry you should say what you want.

as



On 11 Jan 2015, at 6:10 am, Barry Warsaw ba...@list.org wrote:

On Jan 11, 2015, at 06:06 AM, Andrew Stuart wrote:

 I don’t think it’s a big deal in the short term because the plain text error
 message can easily be read from the response.
 
 The only thing that’s somewhat odd is that the content-type is
 application/json even though the content isn’t (for error messages).

That, at least, is a bug.  Care to file one?

Cheers,
-Barry
___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/andrew.stuart%40supercoders.com.au

Security Policy: http://wiki.list.org/x/QIA9

___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/archive%40jab.org

Security Policy: http://wiki.list.org/x/QIA9

Re: [Mailman-Developers] REST API returns are plain text for errors, JSON or nothing for successful

[Andrew Stuart]

I don’t think it’s a big deal in the short term because the plain text error 
message can easily be read from the response.

The only thing that’s somewhat odd is that the content-type is application/json 
even though the content isn’t (for error messages).


On 11 Jan 2015, at 5:39 am, Barry Warsaw ba...@list.org wrote:

On Jan 10, 2015, at 03:42 PM, Andrew Stuart wrote:

 Successful requests to the REST API return a 2xx HTTP code to indicate
 success either with or without JSON data depending on the context of the
 request.
 
 At least one (I’ve not checked the othes yet) 4xx error returned from the
 REST API returns a plain text error message rather than a JSON error message.
 
 For example attempting to create a domain that exists returns a 400 error
 with the response text “Domain exists”.
 
 Do you think this matters? Probably not much but does mean handling valid
 responses goes down a JSON path and handling error message goes down a plain
 text path. JSON error messages might arguable have been somewhat more
 consistent and programmable.

Some of this has to do with the historical legacy of the tools we've used to
implement the REST API (first, lazr.restful, then restish, and now hopefully
finally, falcon).  IIRC it wasn't possible to set a JSON body response on
error codes in lazr.restful and that just got inherited throughout the other
migrations.

I don't know if it matters.  I suppose we'll have to see what client authors
think.  Changing it would be an API change though, so perhaps we'd need to
wait until we rev the REST API.

Cheers,
-Barry
___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/andrew.stuart%40supercoders.com.au

Security Policy: http://wiki.list.org/x/QIA9

___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/archive%40jab.org

Security Policy: http://wiki.list.org/x/QIA9

[Mailman-Developers] REST API first pass at Swagger spec

[Andrew Stuart]

Attached is a first pass at a Swagger spec for the REST API.

You can find out about Swagger at http://swagger.io

The Swagger spec that I am working on is attached to this message or can be 
found here: http://www.mailripper.com/api-docs.json

If you want to see the attached spec in action against the Mailman REST API, go 
to http://www.mailripper.com/swagger-ui/index.html

Give it a try at http://www.mailripper.com/swagger-ui/index.html - you are 
welcome to change anything, it’s a test Mailman server and you can add or 
delete anything - nothing you do will be a problem.

There’s still a fair bit to do to finalise this but most of the API queries are 
working - enough for you to see how it works.

Any comments or feedback welcome.

as




api-docs.json
Description: application/json
___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/archive%40jab.org

Security Policy: http://wiki.list.org/x/QIA9

[Mailman-Developers] REST API returns are plain text for errors, JSON or nothing for successful

[Andrew Stuart]

Successful requests to the REST API return a 2xx HTTP code to indicate success 
either with or without JSON data depending on the context of the request.

At least one (I’ve not checked the othes yet) 4xx error returned from the REST 
API returns a plain text error message rather than a JSON error message. 

For example attempting to create a domain that exists returns a 400 error with 
the response text “Domain exists”.

Do you think this matters? Probably not much but does mean handling valid 
responses goes down a JSON path and handling error message goes down a plain 
text path. JSON error messages might arguable have been somewhat more 
consistent and programmable. 

thanks
___
Mailman-Developers mailing list
Mailman-Developers@python.org
https://mail.python.org/mailman/listinfo/mailman-developers
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: 
http://www.mail-archive.com/mailman-developers%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-developers/archive%40jab.org

Security Policy: http://wiki.list.org/x/QIA9