---------- Forwarded message ---------- From: Ankush Sharma <ankush.sharma.ec...@iitbhu.ac.in> Date: Sun, Mar 8, 2015 at 3:49 PM Subject: Re: [Mailman-Developers] Anatomy of list ids does not keep with that of urls causes some REST end points to return 404 always To: Richard Damon <rich...@damon-family.org>
Well, thanks Richard for replying. On Sat, Mar 7, 2015 at 8:04 PM, Richard Damon <rich...@damon-family.org> wrote: > On 3/7/15 5:27 AM, Ankush Sharma wrote: > >> Hello everyone, >> >> I want to discuss a serious issue regarding mailman architecture in >> context >> with bug #1429366 <https://bugs.launchpad.net/mailman/+bug/1429366>. >> The *hash(#) >> *is a valid character as far as the local part of the email addresses is >> concerned as per the email RFCs. So, as the mailing list addresses are >> email addresses too, we can use # in the list names too. And, in context >> with mailman it works well. We can create a list with list_id >> *sam#hashed.host.org >> <http://hashed.host.org>* for the address *sam#has...@host.org >> <has...@host.org> *. This works fine. But it makes the *list_id *to >> contain >> the hash character and therefore the REST endpoint for retrieving list >> wise >> info becomes invalid, i.e : >> >> *<api-root>/lists/sam#hashed.host.org <http://hashed.host.org>* >> >> Because in an URL the stuff after # is treated as *document starting >> point* >> i.e an id identifier or something of a dom element. This is not a valid >> PATH for the server. Therefore the *falcon wsgi request object* does not >> contain information of that and the *req.path* simply returns *sam* as the >> *list_id* ( http://bazaar.launchpad.net/~mailman-coders/mailman/3.0/ >> view/head:/src/mailman/rest/wsgiapp.py#L65 >> ) giving a 404 because there is no any list with list id *sam*. >> The mailman client works fine, it sends a GET to >> *<api-root>lists/sam#hashed.host.org >> <http://hashed.host.org>. * >> >> This causes the REST end points which needs *list_id* to return 404 or in >> worse we can have a list_id clash between ids* sam#XXXXX* and *sam*. >> Further more if the list_id starts with a # character then the server >> finds >> list_id to be *empty string* and therefore we get a KEY ERROR because >> *fqdn_listname >> *is not set too. The bug highly effects postorius too. The lists index >> template at */postorius/lists/ *cannot be rendered as it uses the former >> REST endpoint and again a 404 is given. And, until we delete this list >> from >> the database, we can't do anything except of getting a 404 and KEY ERROR >> each time. >> As far as security is concerned, if an another user created a public list >> using a hash character, then public list indexing would also fail. >> >> *Possible fixes:* >> 1. Put a constraint over the use of this kind of characters, but it's like >> falling back. >> 2. Do not derive list ids directly from the list addresses, use some kind >> of hashing or encoding technique to generate list_ids. Need to change a >> lot >> in architecture; :( . >> 3. Use obscure techniques to retain the hash part as discussed here >> http://stackoverflow.com/questions/317760/how-to-get- >> url-hash-from-server-side >> . >> >> I think this is quite a serious issue in mailman architecture; i.e >> obtaining list_ids directly from list addresses. I have been trying to fix >> this bug using some obscure techniques. But as it is related to mailman >> architecture, I thought it should be worthy to discuss on the mailing list >> first. >> >> Thanks, >> Ankush Sharma >> github.com/black-perl >> >> The simple, and standard complient, solution would seem to be to define > that special characters get URL encoded as needed (and decoded on the > receiving end. I had a feeling of using URL encoding universally for all the URL building routines but the issue I faced is of identifying the set of URL building routines that actually require to encode URLs to prevent chaos. That's why I didn't listed in on the possible fixes list. Because in that case it would be difficult to identify these guys with full confidence. > It should be standard procedure that ANY routine building a URL would use > URL encoding on the pieces unless it is positive that it is not needed. But now are things more clear. I will try to keep a positive approach towards all these URL building routines keeping them in the suspected list till I am not sure about one. (URL encoding is replacing the problem chars with %nn where nn is the hex value of the character, this now makes % a problem char so it is sent as part of the URL as %25) > > -- > Richard Damon > > _______________________________________________ > Mailman-Developers mailing list > Mailman-Developers@python.org > https://mail.python.org/mailman/listinfo/mailman-developers > Mailman FAQ: http://wiki.list.org/x/AgA3 > Searchable Archives: http://www.mail-archive.com/ > mailman-developers%40python.org/ > Unsubscribe: https://mail.python.org/mailman/options/mailman- > developers/ankush.sharma.ece12%40itbhu.ac.in > > Security Policy: http://wiki.list.org/x/QIA9 > Can it be a security concern as if any other user created such a public list then public listing would simply fail ? I haven't tried reproducing this part on my machine yet. But I am trying now. Thanks, Ankush Sharma github.com/black-perl _______________________________________________ Mailman-Developers mailing list Mailman-Developers@python.org https://mail.python.org/mailman/listinfo/mailman-developers Mailman FAQ: http://wiki.list.org/x/AgA3 Searchable Archives: http://www.mail-archive.com/mailman-developers%40python.org/ Unsubscribe: https://mail.python.org/mailman/options/mailman-developers/archive%40jab.org Security Policy: http://wiki.list.org/x/QIA9