Re: [Catalog-sig] Recent PyPI changes

2010-07-27 Thread Paul Nasrat
On Tue, Jul 27, 2010 at 9:52 AM, Chris Withers ch...@simplistix.co.uk wrote: Martin v. Löwis wrote: We (Richard Jones and me) made a number of changes to PyPI: - there is now a way to request release information in JSON,  see http://tinyurl.com/38lefsp - it's possible to run the code base

Re: [Catalog-sig] Recent PyPI changes

2010-07-27 Thread Chris Withers
Paul Nasrat wrote: The new pypi mirror schem has a naming scheme X.pypi.python.org as documented in PEP 381 PEP 381 looks awesome, how much of it is left to implement? Chris ___ Catalog-SIG mailing list Catalog-SIG@python.org

Re: [Catalog-sig] Recent PyPI changes

2010-07-27 Thread Alexis Metaireau
On Tue, 2010-07-27 at 09:52 +0100, Chris Withers wrote: there is now a way to request release information in JSON, see http://tinyurl.com/38lefsp That's indeed cool, and useful, but we can't rely on this while crawling, too bad this JSON is not replicated on the mirrors. It could help a

Re: [Catalog-sig] Recent PyPI changes

2010-07-27 Thread M.-A. Lemburg
Alexis Metaireau wrote: On Tue, 2010-07-27 at 09:52 +0100, Chris Withers wrote: there is now a way to request release information in JSON, see http://tinyurl.com/38lefsp That's indeed cool, and useful, but we can't rely on this while crawling, too bad this JSON is not replicated on the

Re: [Catalog-sig] Mirror list detection/construction - PEP 381

2010-07-27 Thread M.-A. Lemburg
Martin v. Löwis wrote: Thoughts? I've been thinking that *.pypi.python.org should always yield A records, not CNAMEs. +1 It may be that this becomes difficult with Google appengine, though. There's no rule without exception :-) -- Marc-Andre Lemburg eGenix.com Professional Python

Re: [Catalog-sig] Recent PyPI changes

2010-07-27 Thread Éric Araujo
We (Richard Jones and me) made a number of changes to PyPI: Great news, kudos! Regards ___ Catalog-SIG mailing list Catalog-SIG@python.org http://mail.python.org/mailman/listinfo/catalog-sig

[Catalog-sig] [monit] pypi.python.org - Connection succeeded

2010-07-27 Thread monitor
Connection succeeded Service pypi.python.org Date:Tue, 27 Jul 2010 06:55:58 -0500 Action: alert Host:jacobian.org Description: connection succeeded to INET[pypi.python.org:80] via TCP Your faithful employee, monit

Re: [Catalog-sig] Recent PyPI changes

2010-07-27 Thread Sridhar Ratnakumar
On 2010-07-27, at 2:55 AM, M.-A. Lemburg wrote: Alexis Metaireau wrote: On Tue, 2010-07-27 at 09:52 +0100, Chris Withers wrote: there is now a way to request release information in JSON, see http://tinyurl.com/38lefsp That's indeed cool, and useful, but we can't rely on this while

Re: [Catalog-sig] Recent PyPI changes

2010-07-27 Thread Sridhar Ratnakumar
On 2010-07-27, at 9:31 AM, Sridhar Ratnakumar wrote: On 2010-07-27, at 2:55 AM, M.-A. Lemburg wrote: Alexis Metaireau wrote: On Tue, 2010-07-27 at 09:52 +0100, Chris Withers wrote: there is now a way to request release information in JSON, see http://tinyurl.com/38lefsp That's indeed

Re: [Catalog-sig] Proposal to Reverse Ordering of Scraped Links in PyPI

2010-07-27 Thread Alexis Metaireau
On Tue, 2010-07-27 at 12:12 -0400, P.J. Eby wrote: At 12:21 PM 7/27/2010 +0200, Konrad Delong wrote: This *already* causes problems for pypi maintenance. For some time now, easy_install uses the '/simple' index (specifically intended for automated tools' consumption), rather than the

Re: [Catalog-sig] Proposal to Reverse Ordering of Scraped Links in PyPI

2010-07-27 Thread P.J. Eby
At 06:49 PM 7/27/2010 +0200, Alexis Metaireau wrote: On Tue, 2010-07-27 at 12:12 -0400, P.J. Eby wrote: At 12:21 PM 7/27/2010 +0200, Konrad Delong wrote: This *already* causes problems for pypi maintenance. For some time now, easy_install uses the '/simple' index (specifically

Re: [Catalog-sig] Recent PyPI changes

2010-07-27 Thread Martin v. Löwis
Am 27.07.2010 10:52, schrieb Chris Withers: Martin v. Löwis wrote: We (Richard Jones and me) made a number of changes to PyPI: - there is now a way to request release information in JSON, see http://tinyurl.com/38lefsp - it's possible to run the code base locally using sqlite, see the

Re: [Catalog-sig] Recent PyPI changes

2010-07-27 Thread Martin v. Löwis
Am 27.07.2010 11:24, schrieb Chris Withers: Paul Nasrat wrote: The new pypi mirror schem has a naming scheme X.pypi.python.org as documented in PEP 381 PEP 381 looks awesome, how much of it is left to implement? Nothing that I know of. Of course, clients now need to learn to use it.

[Catalog-sig] PyPI reverse download

2010-07-27 Thread Martin v. Löwis
I'll be implementing a feature for PyPI where you can POST to a certain action (revdownload), and then PyPI will POST the file requested to an URL that was passed; this is need to make blobs work on AppEngine. Any objections? Regards, Martin ___

Re: [Catalog-sig] PyPI reverse download

2010-07-27 Thread Jacob Kaplan-Moss
On Tue, Jul 27, 2010 at 1:25 PM, Martin v. Löwis mar...@v.loewis.de wrote: I'll be implementing a feature for PyPI where you can POST to a certain action (revdownload), and then PyPI will POST the file requested to an URL that was passed; this is need to make blobs work on AppEngine. Any

Re: [Catalog-sig] PyPI reverse download

2010-07-27 Thread Martin v. Löwis
Any objections? Seems like this is rife for abuse -- it's essentially an open relay for POST requests, so I could use it to amplify a DDOS attack. So probably sounds like there needs to be some sort of security, or whitelist of allowed URL (or prefixes?), or somesuch. I guess I restrict it

Re: [Catalog-sig] PyPI reverse download

2010-07-27 Thread Noah Kantrowitz
There are other ways to do this on GAE (was actually looking into this a few days ago). I can send some reference links when I get home from work. --Noah -Original Message- From: catalog-sig-bounces+noah=coderanger@python.org

Re: [Catalog-sig] PyPI reverse download

2010-07-27 Thread Martin v. Löwis
Am 27.07.2010 22:46, schrieb M.-A. Lemburg: Martin v. Löwis wrote: I'll be implementing a feature for PyPI where you can POST to a certain action (revdownload), and then PyPI will POST the file requested to an URL that was passed; this is need to make blobs work on AppEngine. Any

Re: [Catalog-sig] PyPI reverse download

2010-07-27 Thread Noah Kantrowitz
-Original Message- From: catalog-sig-bounces+noah=coderanger@python.org [mailto:catalog-sig-bounces+noah=coderanger@python.org] On Behalf Of Martin v. Löwis Sent: Tuesday, July 27, 2010 2:07 PM To: M.-A. Lemburg Cc: catalog-sig Subject: Re: [Catalog-sig] PyPI reverse

Re: [Catalog-sig] PyPI reverse download

2010-07-27 Thread Martin v. Löwis
However, apps can neither read from nor write to the blobstore. The only way to read from it is to serve the file, and the only way to write to it is through a POST from the outside. I've been able to write to it using urlfetch internally. You just craft a request with the POST from the app

Re: [Catalog-sig] PyPI reverse download

2010-07-27 Thread M.-A. Lemburg
Martin v. Löwis wrote: Am 27.07.2010 22:46, schrieb M.-A. Lemburg: Martin v. Löwis wrote: I'll be implementing a feature for PyPI where you can POST to a certain action (revdownload), and then PyPI will POST the file requested to an URL that was passed; this is need to make blobs work on