Re: [Catalog-sig] remove historic download/homepage links for a project

2013-02-28 Thread Lennart Regebro
On Fri, Mar 1, 2013 at 8:09 AM, Ronald Oussoren wrote: > I luckily haven't run into this with software I release on PyPI yet, but > sometimes > pulling back an update while working on a fix is the responsible thing to do. The the bug leads to data loss or security holes I agree. //Lennart _

Re: [Catalog-sig] remove historic download/homepage links for a project

2013-02-28 Thread Ronald Oussoren
On 1 Mar, 2013, at 4:08, Tres Seaver wrote: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA1 > > On 02/28/2013 06:21 PM, Richard Jones wrote: >> On 1 March 2013 04:10, Tres Seaver wrote: >>> -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 >>> >>> On 02/28/2013 11:27 AM, Ronald Oussoren wrote:

Re: [Catalog-sig] Deprecate External Links

2013-02-28 Thread Lennart Regebro
On Thu, Feb 28, 2013 at 8:52 PM, holger krekel wrote: > There are also packages which have some (older) release files on pypi > and newer ones outside (e.g. "lockfile" with 78256 downloads from > code.google.com). You didn't include such in your 2651 emails, or did you? No, I didn't, I assumed t

Re: [Catalog-sig] remove historic download/homepage links for a project

2013-02-28 Thread Tres Seaver
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 02/28/2013 06:21 PM, Richard Jones wrote: > On 1 March 2013 04:10, Tres Seaver wrote: >> -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 >> >> On 02/28/2013 11:27 AM, Ronald Oussoren wrote: >> >>> But necessary to have. Or am the only one that acci

Re: [Catalog-sig] PyPI terms

2013-02-28 Thread Terry Reedy
On 2/28/2013 1:19 PM, Noah Kantrowitz wrote: Because I happen to have YouTube open anyway: """ For clarity, you retain all of your ownership rights in your Content. However, by submitting Content to YouTube, you hereby grant YouTube a worldwide, non-exclusive, royalty-free, sublicenseable and t

Re: [Catalog-sig] Pypi cdn for hosted packages

2013-02-28 Thread Donald Stufft
On Thursday, February 28, 2013 at 10:13 AM, Noah Kantrowitz wrote: > Reponding from my phone quickly before this gets any further, will write more > later. Plan is to have pypi move package download links to a new hostname > (probably pypi-download.python.org (http://pypi-download.python.org)) an

Re: [Catalog-sig] Deprecate External Links

2013-02-28 Thread PJ Eby
On Thu, Feb 28, 2013 at 5:00 PM, Donald Stufft wrote: > SSL checking on upload should be possible, do you want > a patch? If it uses the 'requests' library, yes, I'll accept one. But I don't want to do any direct implementation of SSL cert checking in setuptools, at least in the short run (next

Re: [Catalog-sig] Deprecate External Links

2013-02-28 Thread Donald Stufft
On Thursday, February 28, 2013 at 6:31 PM, PJ Eby wrote: > On Thu, Feb 28, 2013 at 5:00 PM, Donald Stufft (mailto:donald.stu...@gmail.com)> wrote: > > SSL checking on upload should be possible, do you want > > a patch? > > > > > If it uses the 'requests' library, yes, I'll accept one. But I don

Re: [Catalog-sig] remove historic download/homepage links for a project

2013-02-28 Thread Richard Jones
On 1 March 2013 04:10, Tres Seaver wrote: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA1 > > On 02/28/2013 11:27 AM, Ronald Oussoren wrote: > >> But necessary to have. Or am the only one that accidently released a >> version that had serious bugs? > > Nope. The way to address such a version is

Re: [Catalog-sig] Deprecate External Links

2013-02-28 Thread Donald Stufft
On Thursday, February 28, 2013 at 1:23 PM, PJ Eby wrote: > On Thu, Feb 28, 2013 at 4:08 AM, Nick Coghlan (mailto:ncogh...@gmail.com)> wrote: > > On Thu, Feb 28, 2013 at 7:00 PM, holger krekel > (mailto:hol...@merlinux.eu)> wrote: > > > To summarize, having pip/easy_install report red warnings and

Re: [Catalog-sig] Deprecate External Links

2013-02-28 Thread Lennart Regebro
On Thu, Feb 28, 2013 at 7:38 PM, PJ Eby wrote: > I can't speak to pip, but since the relevant bits of distribute are > 95% the same as setuptools, I think I can say that it will have the > same technical issues, and that warning based on lack of an > --allow-hosts will be both simpler to implement

Re: [Catalog-sig] Deprecate External Links

2013-02-28 Thread holger krekel
On Thu, Feb 28, 2013 at 13:56 +0100, Reinout van Rees wrote: > On 28-02-13 10:43, holger krekel wrote: > >On Thu, Feb 28, 2013 at 06:38 +0100, Andreas Jung wrote: > >> > >>I give a shit at the arguments pulled out every time by package > >>maintainers using PyPI only for listing their packages. I a

Re: [Catalog-sig] Deprecate External Links

2013-02-28 Thread holger krekel
On Thu, Feb 28, 2013 at 16:30 +0100, Lennart Regebro wrote: > On Thu, Feb 28, 2013 at 10:43 AM, Lennart Regebro wrote: > > On Thu, Feb 28, 2013 at 9:28 AM, Nick Coghlan wrote: > >> Pissing off the maintainers off packages that currently rely on > >> external hosting by telling them they have to c

Re: [Catalog-sig] Migrating away from scanning home pages (was: Deprecate External Links)

2013-02-28 Thread PJ Eby
On Thu, Feb 28, 2013 at 5:55 AM, M.-A. Lemburg wrote: > I think we all agree that scanning arbitrary HTML pages > for download links is not a good idea and we need to > transition away from this towards a more reliable system. > > Here's an approach that would work to start the transition > while

Re: [Catalog-sig] Next generation package infrastructure (was: Deprecate External Links)

2013-02-28 Thread PJ Eby
On Thu, Feb 28, 2013 at 4:31 AM, M.-A. Lemburg wrote: > In order for this to work out, you will need to get the > support of people hosting packages externally and address > their concerns. > > The current discussion has been too dogmatic for my taste. > A more pragmatic approach would likely be a

Re: [Catalog-sig] remove historic download/homepage links for a project (was: Re: Deprecate External Links)

2013-02-28 Thread PJ Eby
On Thu, Feb 28, 2013 at 4:28 AM, holger krekel wrote: > I wrote a little command line tool "cleanpypi.py" for the > purposes of removing _all_ download/homepage metadata from all releases > of a project. See it attached - WARNING: it's a hack but worked for me. > It uses the xmlrpc interface to g

Re: [Catalog-sig] Deprecate External Links

2013-02-28 Thread PJ Eby
On Thu, Feb 28, 2013 at 4:28 AM, Lennart Regebro wrote: > My suggestions to move forward on this issue is as follows: > > 1. New versions of pip and distribute are released that will start > warning if they download distributions that are not from PyPI, unless > explicitly given a URL to download.

Re: [Catalog-sig] Pypi cdn for hosted packages

2013-02-28 Thread M.-A. Lemburg
On 28.02.2013 19:23, Noah Kantrowitz wrote: > > On Feb 28, 2013, at 10:20 AM, M.-A. Lemburg wrote: > >> On 28.02.2013 18:25, Noah Kantrowitz wrote: >>> You can go ahead and shut this down please, as I said our CDN partner has >>> already been selected. >> >> I know. Again: this is for testing a

Re: [Catalog-sig] PyPI terms

2013-02-28 Thread M.-A. Lemburg
On 28.02.2013 19:19, Noah Kantrowitz wrote: > > On Feb 28, 2013, at 10:14 AM, M.-A. Lemburg wrote: > >> On 28.02.2013 18:44, Noah Kantrowitz wrote: >>> >>> On Feb 28, 2013, at 2:22 AM, M.-A. Lemburg wrote: BTW: I've never seen a hosting website require agreeing to giving users of the we

Re: [Catalog-sig] Deprecate External Links

2013-02-28 Thread PJ Eby
On Thu, Feb 28, 2013 at 4:08 AM, Nick Coghlan wrote: > On Thu, Feb 28, 2013 at 7:00 PM, holger krekel wrote: >> To summarize, having pip/easy_install report red warnings and requiring >> to pass a "--htmlscrape=PROJ1,PROJ2" option or so is a good way to >> communicate, removing the ability is not

Re: [Catalog-sig] Pypi cdn for hosted packages

2013-02-28 Thread Noah Kantrowitz
On Feb 28, 2013, at 10:20 AM, M.-A. Lemburg wrote: > On 28.02.2013 18:25, Noah Kantrowitz wrote: >> You can go ahead and shut this down please, as I said our CDN partner has >> already been selected. > > I know. Again: this is for testing a CDN setup with installers, > mirrors, etc. It is not m

Re: [Catalog-sig] Pypi cdn for hosted packages

2013-02-28 Thread M.-A. Lemburg
On 28.02.2013 18:25, Noah Kantrowitz wrote: > You can go ahead and shut this down please, as I said our CDN partner has > already been selected. I know. Again: this is for testing a CDN setup with installers, mirrors, etc. It is not meant as permanent solution and will get shut down again, after

Re: [Catalog-sig] PyPI terms

2013-02-28 Thread Noah Kantrowitz
On Feb 28, 2013, at 10:14 AM, M.-A. Lemburg wrote: > On 28.02.2013 18:44, Noah Kantrowitz wrote: >> >> On Feb 28, 2013, at 2:22 AM, M.-A. Lemburg wrote: >>> BTW: I've never seen a hosting website require agreeing to >>> giving users of the website the same distribution rights >>> as the owner of

Re: [Catalog-sig] PyPI terms

2013-02-28 Thread M.-A. Lemburg
On 28.02.2013 18:44, Noah Kantrowitz wrote: > > On Feb 28, 2013, at 2:22 AM, M.-A. Lemburg wrote: >> BTW: I've never seen a hosting website require agreeing to >> giving users of the website the same distribution rights >> as the owner of the website. >> > > You should read terms of service more

Re: [Catalog-sig] PyPI terms (was: Deprecate External Links)

2013-02-28 Thread Noah Kantrowitz
On Feb 28, 2013, at 2:22 AM, M.-A. Lemburg wrote: > On 27.02.2013 19:11, Noah Kantrowitz wrote: >> >> On Feb 27, 2013, at 9:28 AM, M.-A. Lemburg wrote: >>> [reasons for not hosting distribution files on PyPI] >>> * giving up control >> >> This is the point of running a package server, the autho

Re: [Catalog-sig] PyPI limitations (was: Deprecate External Links)

2013-02-28 Thread Noah Kantrowitz
On Feb 28, 2013, at 2:14 AM, M.-A. Lemburg wrote: > On 27.02.2013 19:11, Noah Kantrowitz wrote: >> >> On Feb 27, 2013, at 9:28 AM, M.-A. Lemburg wrote: >> >>> On 27.02.2013 18:05, Noah Kantrowitz wrote: "M.-A. Lemburg" wrote: >> I propose we deprecate the external links th

Re: [Catalog-sig] Pypi cdn for hosted packages

2013-02-28 Thread Noah Kantrowitz
You can go ahead and shut this down please, as I said our CDN partner has already been selected. --Noah On Feb 28, 2013, at 9:19 AM, M.-A. Lemburg wrote: > I've created a wiki page with the CloudFront setup description: > > http://wiki.python.org/moin/CloudPyPI/ExampleCDN > > -- > Marc-Andre

Re: [Catalog-sig] Pypi cdn for hosted packages

2013-02-28 Thread M.-A. Lemburg
I've created a wiki page with the CloudFront setup description: http://wiki.python.org/moin/CloudPyPI/ExampleCDN -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Feb 28 2013) >>> Python Projects, Consulting and Support ... http://www.egenix.com/ >>>

Re: [Catalog-sig] remove historic download/homepage links for a project

2013-02-28 Thread Tres Seaver
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 02/28/2013 11:27 AM, Ronald Oussoren wrote: > But necessary to have. Or am the only one that accidently released a > version that had serious bugs? Nope. The way to address such a version is to release a new, fixed version (preferably one with a

Re: [Catalog-sig] Migrating away from scanning home pages

2013-02-28 Thread M.-A. Lemburg
I've added the proposal to the wiki to keep collecting comments and updates: http://wiki.python.org/moin/PyPI/DownloadMetaDataProposal On 28.02.2013 12:55, M.-A. Lemburg wrote: > On 28.02.2013 12:45, Donald Stufft wrote: >> On Thursday, February 28, 2013 at 5:55 AM, M.-A. Lemburg wrote: >>> I thi

Re: [Catalog-sig] remove historic download/homepage links for a project

2013-02-28 Thread M.-A. Lemburg
On 28.02.2013 17:27, Ronald Oussoren wrote: > > On 28 Feb, 2013, at 14:41, holger krekel wrote: >> >>> That's the #2 thing I hate about some packages: removed releases >>> that I faithfully pinned in my buildout (or requirements.txt). >>> Removing releases is, imho, irresponsible. >> >> it's bad,

Re: [Catalog-sig] Deprecate External Links

2013-02-28 Thread Doug Hellmann
On Feb 28, 2013, at 3:43 AM, Nick Coghlan wrote: > On Thu, Feb 28, 2013 at 6:12 PM, M.-A. Lemburg wrote: >> On 28.02.2013 07:39, Nick Coghlan wrote: >>> 1. The next generation metadata infrastructure will NOT support >>> external hosting of files indexed on PyPI - if you don't upload the >>> arc

Re: [Catalog-sig] remove historic download/homepage links for a project

2013-02-28 Thread Ronald Oussoren
On 28 Feb, 2013, at 14:41, holger krekel wrote: > >> That's the #2 thing I hate about some packages: removed releases >> that I faithfully pinned in my buildout (or requirements.txt). >> Removing releases is, imho, irresponsible. > > it's bad, yes. But necessary to have. Or am the only one tha

Re: [Catalog-sig] Deprecate External Links

2013-02-28 Thread Ian Cordasco
On Thu, Feb 28, 2013 at 10:30 AM, Lennart Regebro wrote: > On Thu, Feb 28, 2013 at 10:43 AM, Lennart Regebro wrote: >> On Thu, Feb 28, 2013 at 9:28 AM, Nick Coghlan wrote: >>> Pissing off the maintainers off packages that currently rely on >>> external hosting by telling them they have to change

Re: [Catalog-sig] Pypi cdn for hosted packages

2013-02-28 Thread Jesse Noller
You me talk - free things are afootsie On Thursday, February 28, 2013 at 10:13 AM, Noah Kantrowitz wrote: > Reponding from my phone quickly before this gets any further, will write more > later. Plan is to have pypi move package download links to a new hostname > (probably pypi-download.pytho

Re: [Catalog-sig] Deprecate External Links

2013-02-28 Thread Lennart Regebro
On Thu, Feb 28, 2013 at 10:43 AM, Lennart Regebro wrote: > On Thu, Feb 28, 2013 at 9:28 AM, Nick Coghlan wrote: >> Pissing off the maintainers off packages that currently rely on >> external hosting by telling them they have to change their release >> processes if they want to keep releasing soft

Re: [Catalog-sig] Deprecate External Links

2013-02-28 Thread Andreas Jung
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Reinout van Rees wrote: > On 28-02-13 10:43, holger krekel wrote: >> On Thu, Feb 28, 2013 at 06:38 +0100, Andreas Jung wrote: >>> >>> I give a shit at the arguments pulled out every time by package >>> maintainers using PyPI only for listing their

Re: [Catalog-sig] Pypi cdn for hosted packages

2013-02-28 Thread Noah Kantrowitz
Reponding from my phone quickly before this gets any further, will write more later. Plan is to have pypi move package download links to a new hostname (probably pypi-download.python.org) and then throw that behind fastly. This sidesteps 100% of issues with dynamic pages, etc. Simple index with

Re: [Catalog-sig] Deprecate External Links

2013-02-28 Thread Daniel Holth
On Thu, Feb 28, 2013 at 7:43 AM, Reinout van Rees wrote: > On 27-02-13 16:26, Donald Stufft wrote: >> >>2. External links decrease the expected uptime for a particular set >>of requirements. PyPI itself has become very stable, however >>the same cannot be said for all of the ho

Re: [Catalog-sig] Pypi cdn for hosted packages

2013-02-28 Thread Jim Fulton
On Thu, Feb 28, 2013 at 5:39 AM, Jesse Noller wrote: > Thread fork. > > Anyway. I know we have at least 1 major rep of a cloud provider on the list, > and I have at least one off in my pocket. > > I'd like to start discussing (completely ignoring past efforts and discussion > which got bogged do

Re: [Catalog-sig] Pypi cdn for hosted packages

2013-02-28 Thread M.-A. Lemburg
On 28.02.2013 15:02, M.-A. Lemburg wrote: > On 28.02.2013 14:37, Giovanni Bajo wrote: >> Il giorno 28/feb/2013, alle ore 13:53, Donald Stufft >> ha scritto: >> >>> On Thursday, February 28, 2013 at 7:49 AM, M.-A. Lemburg wrote: There you go: https://d1t66zoqn9vlte.cloudfront.net/si

Re: [Catalog-sig] Pypi cdn for hosted packages

2013-02-28 Thread M.-A. Lemburg
On 28.02.2013 14:37, Giovanni Bajo wrote: > Il giorno 28/feb/2013, alle ore 13:53, Donald Stufft > ha scritto: > >> On Thursday, February 28, 2013 at 7:49 AM, M.-A. Lemburg wrote: >>> There you go: >>> >>> https://d1t66zoqn9vlte.cloudfront.net/simple/ >> Same thing on Fastly http://pypi.python.o

Re: [Catalog-sig] Pypi cdn for hosted packages

2013-02-28 Thread M.-A. Lemburg
On 28.02.2013 13:56, Donald Stufft wrote: > The non /simple/ pages for either of this won't work since PyPI will > redirect to https://pypi.python.org/ FWIW. I've fixed this for CloudFront: https://d1t66zoqn9vlte.cloudfront.net/ https://d1t66zoqn9vlte.cloudfront.net/pypi both let you see PyPI f

Re: [Catalog-sig] remove historic download/homepage links for a project

2013-02-28 Thread holger krekel
On Thu, Feb 28, 2013 at 13:47 +0100, Reinout van Rees wrote: > On 28-02-13 10:28, holger krekel wrote: > >I wrote a little command line tool "cleanpypi.py" for the > >purposes of removing_all_ download/homepage metadata from all releases > >of a project. > > This sounds like you're removing older

Re: [Catalog-sig] Pypi cdn for hosted packages

2013-02-28 Thread Giovanni Bajo
Il giorno 28/feb/2013, alle ore 13:53, Donald Stufft ha scritto: > On Thursday, February 28, 2013 at 7:49 AM, M.-A. Lemburg wrote: >> There you go: >> >> https://d1t66zoqn9vlte.cloudfront.net/simple/ > Same thing on Fastly http://pypi.python.org.a.prod.fastly.net/simple/ > > Easy :) Given tha

Re: [Catalog-sig] Deprecate External Links

2013-02-28 Thread Donald Stufft
On Thursday, February 28, 2013 at 7:56 AM, Reinout van Rees wrote: > On 28-02-13 10:43, holger krekel wrote: > > On Thu, Feb 28, 2013 at 06:38 +0100, Andreas Jung wrote: > > > > > > I give a shit at the arguments pulled out every time by package > > > maintainers using PyPI only for listing their

Re: [Catalog-sig] Pypi cdn for hosted packages

2013-02-28 Thread Donald Stufft
On Thursday, February 28, 2013 at 7:49 AM, M.-A. Lemburg wrote: > There you go: > > https://d1t66zoqn9vlte.cloudfront.net/simple/ Same thing on Fastly http://pypi.python.org.a.prod.fastly.net/simple/ Easy :)___ Catalog-SIG mailing list Catalog-SIG@pyth

Re: [Catalog-sig] Pypi cdn for hosted packages

2013-02-28 Thread Donald Stufft
The non /simple/ pages for either of this won't work since PyPI will redirect to https://pypi.python.org/ FWIW. On Thursday, February 28, 2013 at 7:53 AM, Donald Stufft wrote: > On Thursday, February 28, 2013 at 7:49 AM, M.-A. Lemburg wrote: > > There you go: > > > > https://d1t66zoqn9vlte.clo

Re: [Catalog-sig] Deprecate External Links

2013-02-28 Thread Reinout van Rees
On 28-02-13 10:43, holger krekel wrote: On Thu, Feb 28, 2013 at 06:38 +0100, Andreas Jung wrote: I give a shit at the arguments pulled out every time by package maintainers using PyPI only for listing their packages. I am both annoyed and bothered by these people. I didn't see such positions

Re: [Catalog-sig] Pypi cdn for hosted packages

2013-02-28 Thread Jesse Noller
Can we please actually look at the free offers we are being given versus paying for something for once On Feb 28, 2013, at 7:40 AM, "M.-A. Lemburg" wrote: > On 28.02.2013 13:11, Donald Stufft wrote: >> On Thursday, February 28, 2013 at 6:48 AM, Giovanni Bajo wrote: >>> Il giorno 28/feb/2013, al

Re: [Catalog-sig] Pypi cdn for hosted packages

2013-02-28 Thread Jesse Noller
Good phew! On Feb 28, 2013, at 7:50 AM, "M.-A. Lemburg" wrote: > On 28.02.2013 13:43, Jesse Noller wrote: >> Can we please actually look at the free offers we are being given versus >> paying for something for once > > Sure. This is just for testing. > > -- > Marc-Andre Lemburg > eGenix.com

Re: [Catalog-sig] Pypi cdn for hosted packages

2013-02-28 Thread M.-A. Lemburg
On 28.02.2013 13:43, Jesse Noller wrote: > Can we please actually look at the free offers we are being given versus > paying for something for once Sure. This is just for testing. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Feb 28 2013) >>> Pyth

Re: [Catalog-sig] remove historic download/homepage links for a project

2013-02-28 Thread Reinout van Rees
On 28-02-13 10:28, holger krekel wrote: I wrote a little command line tool "cleanpypi.py" for the purposes of removing_all_ download/homepage metadata from all releases of a project. This sounds like you're removing older releases from pypi, effectively? That's the #2 thing I hate about some

Re: [Catalog-sig] Pypi cdn for hosted packages

2013-02-28 Thread M.-A. Lemburg
On 28.02.2013 13:40, M.-A. Lemburg wrote: > On 28.02.2013 13:11, Donald Stufft wrote: >> On Thursday, February 28, 2013 at 6:48 AM, Giovanni Bajo wrote: >>> Il giorno 28/feb/2013, alle ore 12:18, Donald Stufft >>> mailto:donald.stu...@gmail.com)> ha scritto: >>> On Thursday, February 28, 2013

Re: [Catalog-sig] Deprecate External Links

2013-02-28 Thread Reinout van Rees
On 27-02-13 16:26, Donald Stufft wrote: 2. External links decrease the expected uptime for a particular set of requirements. PyPI itself has become very stable, however the same cannot be said for all of the hosts linked that the toolchain processes. Each new host is an ad

Re: [Catalog-sig] Pypi cdn for hosted packages

2013-02-28 Thread M.-A. Lemburg
On 28.02.2013 13:11, Donald Stufft wrote: > On Thursday, February 28, 2013 at 6:48 AM, Giovanni Bajo wrote: >> Il giorno 28/feb/2013, alle ore 12:18, Donald Stufft >> mailto:donald.stu...@gmail.com)> ha scritto: >> >>> On Thursday, February 28, 2013 at 6:13 AM, Jesse Noller wrote: O

Re: [Catalog-sig] Pypi cdn for hosted packages

2013-02-28 Thread Donald Stufft
On Thursday, February 28, 2013 at 6:48 AM, Giovanni Bajo wrote: > Il giorno 28/feb/2013, alle ore 12:18, Donald Stufft (mailto:donald.stu...@gmail.com)> ha scritto: > > > On Thursday, February 28, 2013 at 6:13 AM, Jesse Noller wrote: > > > > > > > > > On Feb 28, 2013, at 5:41 AM, Donald Stufft

Re: [Catalog-sig] Migrating away from scanning home pages

2013-02-28 Thread M.-A. Lemburg
On 28.02.2013 12:45, Donald Stufft wrote: > On Thursday, February 28, 2013 at 5:55 AM, M.-A. Lemburg wrote: >> I think we all agree that scanning arbitrary HTML pages >> for download links is not a good idea and we need to >> transition away from this towards a more reliable system. >> >> Here's an

Re: [Catalog-sig] Pypi cdn for hosted packages

2013-02-28 Thread Jesse Noller
On Feb 28, 2013, at 6:48 AM, Giovanni Bajo wrote: > Il giorno 28/feb/2013, alle ore 12:18, Donald Stufft > ha scritto: > >> On Thursday, February 28, 2013 at 6:13 AM, Jesse Noller wrote: >>> >>> >>> On Feb 28, 2013, at 5:41 AM, Donald Stufft wrote: >>> On Thursday, February 28, 2013

Re: [Catalog-sig] Pypi cdn for hosted packages

2013-02-28 Thread Giovanni Bajo
Il giorno 28/feb/2013, alle ore 12:18, Donald Stufft ha scritto: > On Thursday, February 28, 2013 at 6:13 AM, Jesse Noller wrote: >> >> >> On Feb 28, 2013, at 5:41 AM, Donald Stufft wrote: >> >>> On Thursday, February 28, 2013 at 5:39 AM, Jesse Noller wrote: Thread fork. Anyw

Re: [Catalog-sig] Migrating away from scanning home pages (was: Deprecate External Links)

2013-02-28 Thread Donald Stufft
On Thursday, February 28, 2013 at 5:55 AM, M.-A. Lemburg wrote: > I think we all agree that scanning arbitrary HTML pages > for download links is not a good idea and we need to > transition away from this towards a more reliable system. > > Here's an approach that would work to start the transitio

Re: [Catalog-sig] Pypi cdn for hosted packages

2013-02-28 Thread Donald Stufft
On Thursday, February 28, 2013 at 6:13 AM, Jesse Noller wrote: > > > On Feb 28, 2013, at 5:41 AM, Donald Stufft (mailto:donald.stu...@gmail.com)> wrote: > > > On Thursday, February 28, 2013 at 5:39 AM, Jesse Noller wrote: > > > Thread fork. > > > > > > Anyway. I know we have at least 1 major

Re: [Catalog-sig] Pypi cdn for hosted packages

2013-02-28 Thread Jesse Noller
On Feb 28, 2013, at 5:41 AM, Donald Stufft wrote: > On Thursday, February 28, 2013 at 5:39 AM, Jesse Noller wrote: >> Thread fork. >> >> Anyway. I know we have at least 1 major rep of a cloud provider on the list, >> and I have at least one off in my pocket. >> >> I'd like to start discussin

Re: [Catalog-sig] Migrating away from scanning home pages (was: Deprecate External Links)

2013-02-28 Thread M.-A. Lemburg
I think we all agree that scanning arbitrary HTML pages for download links is not a good idea and we need to transition away from this towards a more reliable system. Here's an approach that would work to start the transition while not breaking old tools (sketching here to describe the basic idea)

Re: [Catalog-sig] Deprecate External Links

2013-02-28 Thread Donald Stufft
On Thursday, February 28, 2013 at 5:29 AM, M.-A. Lemburg wrote: > On 27.02.2013 19:21, Donald Stufft wrote: > > On Wednesday, February 27, 2013 at 1:11 PM, M.-A. Lemburg wrote: > > > On 27.02.2013 18:37, Donald Stufft wrote: > > > > On Wednesday, February 27, 2013 at 12:10 PM, M.-A. Lemburg wrote:

Re: [Catalog-sig] Pypi cdn for hosted packages

2013-02-28 Thread Donald Stufft
On Thursday, February 28, 2013 at 5:39 AM, Jesse Noller wrote: > Thread fork. > > Anyway. I know we have at least 1 major rep of a cloud provider on the list, > and I have at least one off in my pocket. > > I'd like to start discussing (completely ignoring past efforts and discussion > which g

[Catalog-sig] Pypi cdn for hosted packages

2013-02-28 Thread Jesse Noller
Thread fork. Anyway. I know we have at least 1 major rep of a cloud provider on the list, and I have at least one off in my pocket. I'd like to start discussing (completely ignoring past efforts and discussion which got bogged down) how we can start distributing the package data we host via C

Re: [Catalog-sig] Deprecate External Links

2013-02-28 Thread M.-A. Lemburg
On 27.02.2013 19:21, Donald Stufft wrote: > On Wednesday, February 27, 2013 at 1:11 PM, M.-A. Lemburg wrote: >> On 27.02.2013 18:37, Donald Stufft wrote: >>> On Wednesday, February 27, 2013 at 12:10 PM, M.-A. Lemburg wrote: Package installers only need access to the static files in the /s

Re: [Catalog-sig] Deprecate External Links

2013-02-28 Thread Martin v. Löwis
no support for UCS2/UCS4 binary distributions, unsupported distribution file formats (e.g. our prebuilt format), Not sure why PyPI would even care what charset the package files use, but if true thats certainly a bug and we can get that fixed. What file formats do pip/buildout support that PyPI

Re: [Catalog-sig] PyPI terms (was: Deprecate External Links)

2013-02-28 Thread M.-A. Lemburg
On 27.02.2013 19:11, Noah Kantrowitz wrote: > > On Feb 27, 2013, at 9:28 AM, M.-A. Lemburg wrote: >> [reasons for not hosting distribution files on PyPI] >> * giving up control > > This is the point of running a package server, the author gives up control > over distribution in order to reap the

Re: [Catalog-sig] PyPI limitations (was: Deprecate External Links)

2013-02-28 Thread M.-A. Lemburg
On 27.02.2013 19:11, Noah Kantrowitz wrote: > > On Feb 27, 2013, at 9:28 AM, M.-A. Lemburg wrote: > >> On 27.02.2013 18:05, Noah Kantrowitz wrote: >>> >>> >>> "M.-A. Lemburg" wrote: > I propose we deprecate the external links that PyPI has published > on the /simple/ indexes which exist

Re: [Catalog-sig] Deprecate External Links

2013-02-28 Thread Richard Jones
On 28 February 2013 20:09, holger krekel wrote: > On Thu, Feb 28, 2013 at 09:48 +1100, Richard Jones wrote: >> On 28 February 2013 08:31, PJ Eby wrote: >> > OTOH, I currently make development snapshots of setuptools and other >> > projects available by dumping them in a directory that's used as a

Re: [Catalog-sig] Deprecate External Links

2013-02-28 Thread Lennart Regebro
On Thu, Feb 28, 2013 at 9:28 AM, Nick Coghlan wrote: > Pissing off the maintainers off packages that currently rely on > external hosting by telling them they have to change their release > processes if they want to keep releasing software on PyPI and have > their users actually be able to downloa

Re: [Catalog-sig] Deprecate External Links

2013-02-28 Thread holger krekel
On Thu, Feb 28, 2013 at 06:38 +0100, Andreas Jung wrote: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA1 > > +1 for the proposal > > The complete discussion on this topic is once again absurd and bizarre. > We are discussing the issue with externally hosted packages every year > and the situati

Re: [Catalog-sig] Next generation package infrastructure (was: Deprecate External Links)

2013-02-28 Thread M.-A. Lemburg
On 28.02.2013 09:43, Nick Coghlan wrote: > On Thu, Feb 28, 2013 at 6:12 PM, M.-A. Lemburg wrote: >> On 28.02.2013 07:39, Nick Coghlan wrote: >>> 1. The next generation metadata infrastructure will NOT support >>> external hosting of files indexed on PyPI - if you don't upload the >>> archive files

Re: [Catalog-sig] Deprecate External Links

2013-02-28 Thread Lennart Regebro
On Thu, Feb 28, 2013 at 12:16 AM, Aaron Meurer wrote: > And by the way, this hasn't been mentioned, but I really mean *all* > mentions of Google Code on PyPI. pip crawls Google Code not just > because Google Code listed as an official site for my package or > because the latest release is there,

[Catalog-sig] remove historic download/homepage links for a project (was: Re: Deprecate External Links)

2013-02-28 Thread holger krekel
On Wed, Feb 27, 2013 at 16:16 -0700, Aaron Meurer wrote: > And by the way, this hasn't been mentioned, but I really mean *all* > mentions of Google Code on PyPI. pip crawls Google Code not just > because Google Code listed as an official site for my package or > because the latest release is there

Re: [Catalog-sig] Deprecate External Links

2013-02-28 Thread holger krekel
On Thu, Feb 28, 2013 at 09:48 +1100, Richard Jones wrote: > On 28 February 2013 08:31, PJ Eby wrote: > > OTOH, I currently make development snapshots of setuptools and other > > projects available by dumping them in a directory that's used as an > > external download URL. Replacing that would be

Re: [Catalog-sig] Deprecate External Links

2013-02-28 Thread Nick Coghlan
On Thu, Feb 28, 2013 at 7:00 PM, holger krekel wrote: > To summarize, having pip/easy_install report red warnings and requiring > to pass a "--htmlscrape=PROJ1,PROJ2" option or so is a good way to > communicate, removing the ability is not, at this point. +1 I'm a fan of updating the client side

Re: [Catalog-sig] Deprecate External Links

2013-02-28 Thread holger krekel
On Wed, Feb 27, 2013 at 22:04 +0100, Lennart Regebro wrote: > On Wed, Feb 27, 2013 at 8:49 PM, Monty Taylor wrote: > >> But wouldn't this only be a change in pip/easy_install, not PyPI > >> itself? I suppose you could explicitly break the external links by > >> having them point to nothing if you

Re: [Catalog-sig] Deprecate External Links

2013-02-28 Thread Nick Coghlan
On Thu, Feb 28, 2013 at 6:12 PM, M.-A. Lemburg wrote: > On 28.02.2013 07:39, Nick Coghlan wrote: >> 1. The next generation metadata infrastructure will NOT support >> external hosting of files indexed on PyPI - if you don't upload the >> archive files to PyPI, they won't be included in the next ge

Re: [Catalog-sig] Deprecate External Links

2013-02-28 Thread Nick Coghlan
On Thu, Feb 28, 2013 at 5:01 PM, Donald Stufft wrote: > I'm glad the next set of Metadata won't have external links, however > even if it showed up tomorrow it's going to be a long time until > people are completely migrated to it. Furthermore you estimate > months but the first phase will have po

Re: [Catalog-sig] Deprecate External Links

2013-02-28 Thread M.-A. Lemburg
On 28.02.2013 07:39, Nick Coghlan wrote: > On Thu, Feb 28, 2013 at 6:27 AM, Donald Stufft > wrote: >> Sometimes you need to break things. The goal is to do it with ample >> warning and migration time so that people have a chance to move >> to the new way of doing things. >> >> Again, I am not sug