Re: [Catalog-sig] hash tags

2013-03-08 Thread M.-A. Lemburg
On 08.03.2013 13:15, Christian Heimes wrote: Am 08.03.2013 12:49, schrieb M.-A. Lemburg: Together with the added hash tag on the download file URLs (*), this would solve the availability and the security aspects. Instead of deprecating external links altogether, we could then deprecate

Re: [Catalog-sig] Deprecation of External Urls, Statistics

2013-03-08 Thread Jesse Noller
As long as external URLs eventually are completely removed I'm okay with caching things On Mar 8, 2013, at 6:49 AM, M.-A. Lemburg m...@egenix.com wrote: On 08.03.2013 02:40, Donald Stufft wrote: So I updated my script (had to remove eventlet) and I believe it's now accurate. The total time

Re: [Catalog-sig] hash tags

2013-03-08 Thread Donald Stufft
Accidentally sent this to only MAL so resending! On Mar 8, 2013, at 7:50 AM, M.-A. Lemburg m...@egenix.com wrote: On 08.03.2013 13:15, Christian Heimes wrote: Am 08.03.2013 12:49, schrieb M.-A. Lemburg: Together with the added hash tag on the download file URLs (*), this would solve the

Re: [Catalog-sig] Deprecation of External Urls, Statistics

2013-03-08 Thread Donald Stufft
On Mar 8, 2013, at 8:13 AM, Donald Stufft don...@stufft.io wrote: On Mar 8, 2013, at 8:07 AM, Jesse Noller jnol...@gmail.com wrote: As long as external URLs eventually are completely removed I'm okay with caching things So I have mixed feelings on caching the urls. I'm not completely

Re: [Catalog-sig] Deprecation of External Urls, Statistics

2013-03-08 Thread Jesse Noller
On Mar 8, 2013, at 8:13 AM, Donald Stufft don...@stufft.io wrote: On Mar 8, 2013, at 8:07 AM, Jesse Noller jnol...@gmail.com wrote: As long as external URLs eventually are completely removed I'm okay with caching things So I have mixed feelings on caching the urls. I'm not completely

Re: [Catalog-sig] hash tags

2013-03-08 Thread M.-A. Lemburg
On 08.03.2013 14:09, Donald Stufft wrote: Accidentally sent this to only MAL so resending! On Mar 8, 2013, at 7:50 AM, M.-A. Lemburg m...@egenix.com wrote: On 08.03.2013 13:15, Christian Heimes wrote: Am 08.03.2013 12:49, schrieb M.-A. Lemburg: Together with the added hash tag on the

Re: [Catalog-sig] hash tags

2013-03-08 Thread M.-A. Lemburg
On 08.03.2013 13:50, M.-A. Lemburg wrote: On 08.03.2013 13:15, Christian Heimes wrote: I like to propose query string-like key/value pairs. key/value pairs are more flexible and allow us to add/remove new information in the future. Good idea. I'll add that as extension mechanism. I also

[Catalog-sig] Search engine relevance

2013-03-08 Thread Antoine Pitrou
Hello, It seems the PyPI search engine is quite crude and doesn't try to make the results relevant at all. For example, if I'm trying to search agi in the hope of finding modules relevant to the Asterisk Gateway Interface (nicknamed AGI), I get the following results:

Re: [Catalog-sig] Search engine relevance

2013-03-08 Thread Jacob Kaplan-Moss
Hi Antoine - Yes, PyPI's search engine is rather simplistic, I think that's a pretty well-known problem. For the time being you might try Crate instead (crate.io); I've found its search engine to be much much better. Jacob On Fri, Mar 8, 2013 at 8:00 AM, Antoine Pitrou solip...@pitrou.net

Re: [Catalog-sig] Search engine relevance

2013-03-08 Thread Yuval Greenfield
https://crate.io/?has_releases=onq=agi No results found. On Fri, Mar 8, 2013 at 6:51 AM, Jacob Kaplan-Moss ja...@jacobian.orgwrote: Hi Antoine - Yes, PyPI's search engine is rather simplistic, I think that's a pretty well-known problem. For the time being you might try Crate instead

Re: [Catalog-sig] Search engine relevance

2013-03-08 Thread Donald Stufft
On Mar 8, 2013, at 9:51 AM, Jacob Kaplan-Moss ja...@jacobian.org wrote: Hi Antoine - Yes, PyPI's search engine is rather simplistic, I think that's a pretty well-known problem. For the time being you might try Crate instead (crate.io); I've found its search engine to be much much

Re: [Catalog-sig] Search engine relevance

2013-03-08 Thread Antoine Pitrou
Yuval Greenfield ubershmekel at gmail.com writes: https://crate.io/?has_releases=onq=agi No results found. Thanks for the answers. Yes, crate.io is at least missing pyst2 which does mention AGI in its description: https://crate.io/packages/pyst2/ (pyst2 is rather unmaintained, but that

Re: [Catalog-sig] Search engine relevance

2013-03-08 Thread Donald Stufft
On Mar 8, 2013, at 10:24 AM, Antoine Pitrou solip...@pitrou.net wrote: Yuval Greenfield ubershmekel at gmail.com writes: https://crate.io/?has_releases=onq=agi No results found. Thanks for the answers. Yes, crate.io is at least missing pyst2 which does mention AGI in its description:

Re: [Catalog-sig] Search engine relevance

2013-03-08 Thread Yuval Greenfield
On Fri, Mar 8, 2013 at 7:24 AM, Antoine Pitrou solip...@pitrou.net wrote: Yes, crate.io is at least missing pyst2 which does mention AGI in its description: https://crate.io/packages/pyst2/ I agree. There's only one effective search engine for pypi I know of, e.g.

Re: [Catalog-sig] hash tags

2013-03-08 Thread PJ Eby
On Fri, Mar 8, 2013 at 7:50 AM, M.-A. Lemburg m...@egenix.com wrote: After the feedback I got from Holger and Phillip, I'm currently writing a new version, which drops some of the unneeded requirements and spells out a few more things. Here's a very short version... Installers are modified:

Re: [Catalog-sig] hash tags

2013-03-08 Thread Noah Kantrowitz
On Mar 8, 2013, at 4:50 AM, M.-A. Lemburg wrote: On 08.03.2013 13:15, Christian Heimes wrote: Am 08.03.2013 12:49, schrieb M.-A. Lemburg: Together with the added hash tag on the download file URLs (*), this would solve the availability and the security aspects. Instead of deprecating

Re: [Catalog-sig] Deprecation of External Urls, Statistics

2013-03-08 Thread PJ Eby
On Fri, Mar 8, 2013 at 8:13 AM, Donald Stufft don...@stufft.io wrote: It does solve the backwards compatibility issue of killing external urls immediately so I'm not flat out against it, but there may be legal issues involved too? I've mentioned this in the other thread as well, but the best

Re: [Catalog-sig] Deprecation of External Urls, Statistics

2013-03-08 Thread Donald Stufft
On Mar 8, 2013, at 2:54 PM, PJ Eby p...@telecommunity.com wrote: On Fri, Mar 8, 2013 at 8:13 AM, Donald Stufft don...@stufft.io wrote: It does solve the backwards compatibility issue of killing external urls immediately so I'm not flat out against it, but there may be legal issues involved

Re: [Catalog-sig] hash tags

2013-03-08 Thread M.-A. Lemburg
On 08.03.2013 20:52, Noah Kantrowitz wrote: On Mar 8, 2013, at 4:50 AM, M.-A. Lemburg wrote: On 08.03.2013 13:15, Christian Heimes wrote: Am 08.03.2013 12:49, schrieb M.-A. Lemburg: Together with the added hash tag on the download file URLs (*), this would solve the availability and the

Re: [Catalog-sig] hash tags

2013-03-08 Thread PJ Eby
On Fri, Mar 8, 2013 at 2:52 PM, Noah Kantrowitz n...@coderanger.net wrote: MD5 is _not_ acceptable for anything security related and we shouldn't be adding anything that increases our dependence on it. MD5's only use in the packaging world is to make people who forget that TCP has its own

Re: [Catalog-sig] hash tags

2013-03-08 Thread M.-A. Lemburg
On 08.03.2013 20:16, PJ Eby wrote: On Fri, Mar 8, 2013 at 7:50 AM, M.-A. Lemburg m...@egenix.com wrote: After the feedback I got from Holger and Phillip, I'm currently writing a new version, which drops some of the unneeded requirements and spells out a few more things. Here's a very short

Re: [Catalog-sig] hash tags

2013-03-08 Thread Donald Stufft
On Mar 8, 2013, at 4:12 PM, PJ Eby p...@telecommunity.com wrote: On Fri, Mar 8, 2013 at 2:52 PM, Noah Kantrowitz n...@coderanger.net wrote: MD5 is _not_ acceptable for anything security related and we shouldn't be adding anything that increases our dependence on it. MD5's only use in the

Re: [Catalog-sig] Search engine relevance

2013-03-08 Thread Richard Jones
That *was* the original search engine :-) Then after user complaints we devised a better solution... Always happy to take criticism of it and improve it! :-) Sent from my portable device, please excuse the brevity. On Mar 9, 2013 2:29 AM, Yuval Greenfield ubershme...@gmail.com wrote: On Fri,

Re: [Catalog-sig] hash tags

2013-03-08 Thread M.-A. Lemburg
On 08.03.2013 20:16, PJ Eby wrote: On Fri, Mar 8, 2013 at 7:50 AM, M.-A. Lemburg m...@egenix.com wrote: So far the only practical problem I've found with the approach is that the download page may not contain dynamic data, e.g. a date or timestamp, since that causes the hash tag not to

Re: [Catalog-sig] hash tags

2013-03-08 Thread Donald Stufft
On Mar 8, 2013, at 4:12 PM, PJ Eby p...@telecommunity.com wrote: On Fri, Mar 8, 2013 at 2:52 PM, Noah Kantrowitz n...@coderanger.net wrote: MD5 is _not_ acceptable for anything security related and we shouldn't be adding anything that increases our dependence on it. MD5's only use in the

Re: [Catalog-sig] hash tags

2013-03-08 Thread Donald Stufft
On Mar 8, 2013, at 4:28 PM, M.-A. Lemburg m...@egenix.com wrote: BTW: If we go with the CDN caching model for external files, we'd pull the download page links directly on the /simple/ index page - as files, not external links. We cannot download and rehost (even if we call it a cache)

Re: [Catalog-sig] hash tags

2013-03-08 Thread Noah Kantrowitz
On Mar 8, 2013, at 1:33 PM, Donald Stufft wrote: On Mar 8, 2013, at 4:28 PM, M.-A. Lemburg m...@egenix.com wrote: BTW: If we go with the CDN caching model for external files, we'd pull the download page links directly on the /simple/ index page - as files, not external links. We cannot

Re: [Catalog-sig] hash tags

2013-03-08 Thread Christian Heimes
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Am 08.03.2013 22:33, schrieb Donald Stufft: On Mar 8, 2013, at 4:28 PM, M.-A. Lemburg m...@egenix.com wrote: BTW: If we go with the CDN caching model for external files, we'd pull the download page links directly on the /simple/ index page -

Re: [Catalog-sig] hash tags

2013-03-08 Thread Donald Stufft
On Mar 8, 2013, at 4:50 PM, Christian Heimes christ...@python.org wrote: Am 08.03.2013 22:33, schrieb Donald Stufft: On Mar 8, 2013, at 4:28 PM, M.-A. Lemburg m...@egenix.com wrote: BTW: If we go with the CDN caching model for external files, we'd pull the download page links directly

Re: [Catalog-sig] hash tags

2013-03-08 Thread Christian Heimes
Am 08.03.2013 22:43, schrieb Daniel Holth: Check out https://blake2.net/ ; it is both faster and more secure than md5. md5 does have to go, no matter how secure it is in this particular application. SHA2 is the only choice that doesn't require a long explanation. When this came up a little

Re: [Catalog-sig] hash tags

2013-03-08 Thread Donald Stufft
On Mar 8, 2013, at 5:02 PM, Christian Heimes christ...@python.org wrote: Am 08.03.2013 22:43, schrieb Daniel Holth: Check out https://blake2.net/ ; it is both faster and more secure than md5. md5 does have to go, no matter how secure it is in this particular application. SHA2 is the only

Re: [Catalog-sig] hash tags

2013-03-08 Thread M.-A. Lemburg
On 08.03.2013 22:47, Donald Stufft wrote: On Mar 8, 2013, at 4:45 PM, M.-A. Lemburg m...@egenix.com wrote: On 08.03.2013 22:33, Donald Stufft wrote: On Mar 8, 2013, at 4:28 PM, M.-A. Lemburg m...@egenix.com wrote: BTW: If we go with the CDN caching model for external files, we'd pull the

Re: [Catalog-sig] hash tags

2013-03-08 Thread PJ Eby
On Fri, Mar 8, 2013 at 4:17 PM, M.-A. Lemburg m...@egenix.com wrote: On 08.03.2013 20:16, PJ Eby wrote: There is, as I said before, a MUCH simpler way to do this, that works right now: put direct #md5 download links in your description, and phase out the rel= attributes altogether. No, that

Re: [Catalog-sig] hash tags

2013-03-08 Thread PJ Eby
On Fri, Mar 8, 2013 at 4:26 PM, Donald Stufft don...@stufft.io wrote: On Mar 8, 2013, at 4:12 PM, PJ Eby p...@telecommunity.com wrote: On Fri, Mar 8, 2013 at 2:52 PM, Noah Kantrowitz n...@coderanger.net wrote: MD5 is _not_ acceptable for anything security related and we shouldn't be adding

Re: [Catalog-sig] hash tags

2013-03-08 Thread Christian Heimes
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Am 08.03.2013 23:03, schrieb Donald Stufft: Sha-1 is broken. Sha-2 or better is the only real acceptable one in the stdlib. Well, then SHA-384 it is. -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with

Re: [Catalog-sig] hash tags

2013-03-08 Thread PJ Eby
On Fri, Mar 8, 2013 at 4:28 PM, M.-A. Lemburg m...@egenix.com wrote: On 08.03.2013 20:16, PJ Eby wrote: So, since the page only contains links, might as well put the links straight on PyPI, or at most have an option/tool to load the links from an external source. I don't follow you. We only

Re: [Catalog-sig] hash tags

2013-03-08 Thread Donald Stufft
On Mar 8, 2013, at 5:08 PM, PJ Eby p...@telecommunity.com wrote: On Fri, Mar 8, 2013 at 4:26 PM, Donald Stufft don...@stufft.io wrote: On Mar 8, 2013, at 4:12 PM, PJ Eby p...@telecommunity.com wrote: On Fri, Mar 8, 2013 at 2:52 PM, Noah Kantrowitz n...@coderanger.net wrote: MD5 is _not_

Re: [Catalog-sig] hash tags

2013-03-08 Thread PJ Eby
On Fri, Mar 8, 2013 at 4:32 PM, Donald Stufft don...@stufft.io wrote: Here's some more information pulled straight from Wikiepdia: Trust me, I've read a LOT of Wikipedia (and even more from other sites, including at least the conclusions of a number of cryptography papers) about hashing attacks

Re: [Catalog-sig] hash tags

2013-03-08 Thread Giovanni Bajo
Il giorno 09/mar/2013, alle ore 00:15, Donald Stufft don...@stufft.io ha scritto: On Mar 8, 2013, at 5:50 PM, PJ Eby p...@telecommunity.com wrote: On Fri, Mar 8, 2013 at 4:32 PM, Donald Stufft don...@stufft.io wrote: Here's some more information pulled straight from Wikiepdia: Trust

Re: [Catalog-sig] hash tags

2013-03-08 Thread holger krekel
Hi Philip, all, On Fri, Mar 08, 2013 at 14:16 -0500, PJ Eby wrote: The key to making this transition isn't creating elaborate new standards for the tools, it's *creating new tools for the standards*. If we can find a way to improve PyPI and not require the world to change first, that's a big