Re: [Catalog-sig] Deprecate External Links

2013-02-28 Thread Nick Coghlan
On Thu, Feb 28, 2013 at 5:01 PM, Donald Stufft donald.stu...@gmail.com wrote: I'm glad the next set of Metadata won't have external links, however even if it showed up tomorrow it's going to be a long time until people are completely migrated to it. Furthermore you estimate months but the

Re: [Catalog-sig] Deprecate External Links

2013-02-28 Thread Nick Coghlan
On Thu, Feb 28, 2013 at 6:12 PM, M.-A. Lemburg m...@egenix.com wrote: On 28.02.2013 07:39, Nick Coghlan wrote: 1. The next generation metadata infrastructure will NOT support external hosting of files indexed on PyPI - if you don't upload the archive files to PyPI, they won't be included in

Re: [Catalog-sig] Deprecate External Links

2013-02-28 Thread holger krekel
On Wed, Feb 27, 2013 at 22:04 +0100, Lennart Regebro wrote: On Wed, Feb 27, 2013 at 8:49 PM, Monty Taylor mord...@inaugust.com wrote: But wouldn't this only be a change in pip/easy_install, not PyPI itself? I suppose you could explicitly break the external links by having them point to

Re: [Catalog-sig] Deprecate External Links

2013-02-28 Thread holger krekel
On Thu, Feb 28, 2013 at 09:48 +1100, Richard Jones wrote: On 28 February 2013 08:31, PJ Eby p...@telecommunity.com wrote: OTOH, I currently make development snapshots of setuptools and other projects available by dumping them in a directory that's used as an external download URL.

Re: [Catalog-sig] Next generation package infrastructure (was: Deprecate External Links)

2013-02-28 Thread M.-A. Lemburg
On 28.02.2013 09:43, Nick Coghlan wrote: On Thu, Feb 28, 2013 at 6:12 PM, M.-A. Lemburg m...@egenix.com wrote: On 28.02.2013 07:39, Nick Coghlan wrote: 1. The next generation metadata infrastructure will NOT support external hosting of files indexed on PyPI - if you don't upload the archive

Re: [Catalog-sig] Deprecate External Links

2013-02-28 Thread holger krekel
On Thu, Feb 28, 2013 at 06:38 +0100, Andreas Jung wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 +1 for the proposal The complete discussion on this topic is once again absurd and bizarre. We are discussing the issue with externally hosted packages every year and the situation has

Re: [Catalog-sig] Deprecate External Links

2013-02-28 Thread Richard Jones
On 28 February 2013 20:09, holger krekel hol...@merlinux.eu wrote: On Thu, Feb 28, 2013 at 09:48 +1100, Richard Jones wrote: On 28 February 2013 08:31, PJ Eby p...@telecommunity.com wrote: OTOH, I currently make development snapshots of setuptools and other projects available by dumping them

Re: [Catalog-sig] PyPI limitations (was: Deprecate External Links)

2013-02-28 Thread M.-A. Lemburg
On 27.02.2013 19:11, Noah Kantrowitz wrote: On Feb 27, 2013, at 9:28 AM, M.-A. Lemburg wrote: On 27.02.2013 18:05, Noah Kantrowitz wrote: M.-A. Lemburg m...@egenix.com wrote: I propose we deprecate the external links that PyPI has published on the /simple/ indexes which exist because of

Re: [Catalog-sig] PyPI terms (was: Deprecate External Links)

2013-02-28 Thread M.-A. Lemburg
On 27.02.2013 19:11, Noah Kantrowitz wrote: On Feb 27, 2013, at 9:28 AM, M.-A. Lemburg wrote: [reasons for not hosting distribution files on PyPI] * giving up control This is the point of running a package server, the author gives up control over distribution in order to reap the

Re: [Catalog-sig] Deprecate External Links

2013-02-28 Thread Martin v. Löwis
no support for UCS2/UCS4 binary distributions, unsupported distribution file formats (e.g. our prebuilt format), Not sure why PyPI would even care what charset the package files use, but if true thats certainly a bug and we can get that fixed. What file formats do pip/buildout support that PyPI

Re: [Catalog-sig] Deprecate External Links

2013-02-28 Thread Donald Stufft
On Thursday, February 28, 2013 at 5:29 AM, M.-A. Lemburg wrote: On 27.02.2013 19:21, Donald Stufft wrote: On Wednesday, February 27, 2013 at 1:11 PM, M.-A. Lemburg wrote: On 27.02.2013 18:37, Donald Stufft wrote: On Wednesday, February 27, 2013 at 12:10 PM, M.-A. Lemburg wrote:

Re: [Catalog-sig] Pypi cdn for hosted packages

2013-02-28 Thread Jesse Noller
On Feb 28, 2013, at 5:41 AM, Donald Stufft donald.stu...@gmail.com wrote: On Thursday, February 28, 2013 at 5:39 AM, Jesse Noller wrote: Thread fork. Anyway. I know we have at least 1 major rep of a cloud provider on the list, and I have at least one off in my pocket. I'd like to

Re: [Catalog-sig] Pypi cdn for hosted packages

2013-02-28 Thread Donald Stufft
On Thursday, February 28, 2013 at 6:13 AM, Jesse Noller wrote: On Feb 28, 2013, at 5:41 AM, Donald Stufft donald.stu...@gmail.com (mailto:donald.stu...@gmail.com) wrote: On Thursday, February 28, 2013 at 5:39 AM, Jesse Noller wrote: Thread fork. Anyway. I know we have at

Re: [Catalog-sig] Migrating away from scanning home pages (was: Deprecate External Links)

2013-02-28 Thread Donald Stufft
On Thursday, February 28, 2013 at 5:55 AM, M.-A. Lemburg wrote: I think we all agree that scanning arbitrary HTML pages for download links is not a good idea and we need to transition away from this towards a more reliable system. Here's an approach that would work to start the transition

Re: [Catalog-sig] Pypi cdn for hosted packages

2013-02-28 Thread Giovanni Bajo
Il giorno 28/feb/2013, alle ore 12:18, Donald Stufft donald.stu...@gmail.com ha scritto: On Thursday, February 28, 2013 at 6:13 AM, Jesse Noller wrote: On Feb 28, 2013, at 5:41 AM, Donald Stufft donald.stu...@gmail.com wrote: On Thursday, February 28, 2013 at 5:39 AM, Jesse Noller

Re: [Catalog-sig] remove historic download/homepage links for a project

2013-02-28 Thread Reinout van Rees
On 28-02-13 10:28, holger krekel wrote: I wrote a little command line tool cleanpypi.py for the purposes of removing_all_ download/homepage metadata from all releases of a project. This sounds like you're removing older releases from pypi, effectively? That's the #2 thing I hate about some

Re: [Catalog-sig] Pypi cdn for hosted packages

2013-02-28 Thread M.-A. Lemburg
On 28.02.2013 13:43, Jesse Noller wrote: Can we please actually look at the free offers we are being given versus paying for something for once Sure. This is just for testing. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Feb 28 2013) Python

Re: [Catalog-sig] Pypi cdn for hosted packages

2013-02-28 Thread Jesse Noller
Good phew! On Feb 28, 2013, at 7:50 AM, M.-A. Lemburg m...@egenix.com wrote: On 28.02.2013 13:43, Jesse Noller wrote: Can we please actually look at the free offers we are being given versus paying for something for once Sure. This is just for testing. -- Marc-Andre Lemburg

Re: [Catalog-sig] Pypi cdn for hosted packages

2013-02-28 Thread Jesse Noller
Can we please actually look at the free offers we are being given versus paying for something for once On Feb 28, 2013, at 7:40 AM, M.-A. Lemburg m...@egenix.com wrote: On 28.02.2013 13:11, Donald Stufft wrote: On Thursday, February 28, 2013 at 6:48 AM, Giovanni Bajo wrote: Il giorno

Re: [Catalog-sig] Deprecate External Links

2013-02-28 Thread Donald Stufft
On Thursday, February 28, 2013 at 7:56 AM, Reinout van Rees wrote: On 28-02-13 10:43, holger krekel wrote: On Thu, Feb 28, 2013 at 06:38 +0100, Andreas Jung wrote: I give a shit at the arguments pulled out every time by package maintainers using PyPI only for listing their packages. I

Re: [Catalog-sig] Pypi cdn for hosted packages

2013-02-28 Thread Giovanni Bajo
Il giorno 28/feb/2013, alle ore 13:53, Donald Stufft donald.stu...@gmail.com ha scritto: On Thursday, February 28, 2013 at 7:49 AM, M.-A. Lemburg wrote: There you go: https://d1t66zoqn9vlte.cloudfront.net/simple/ Same thing on Fastly http://pypi.python.org.a.prod.fastly.net/simple/ Easy

Re: [Catalog-sig] Pypi cdn for hosted packages

2013-02-28 Thread M.-A. Lemburg
On 28.02.2013 13:56, Donald Stufft wrote: The non /simple/ pages for either of this won't work since PyPI will redirect to https://pypi.python.org/ FWIW. I've fixed this for CloudFront: https://d1t66zoqn9vlte.cloudfront.net/ https://d1t66zoqn9vlte.cloudfront.net/pypi both let you see PyPI

Re: [Catalog-sig] Pypi cdn for hosted packages

2013-02-28 Thread M.-A. Lemburg
On 28.02.2013 15:02, M.-A. Lemburg wrote: On 28.02.2013 14:37, Giovanni Bajo wrote: Il giorno 28/feb/2013, alle ore 13:53, Donald Stufft donald.stu...@gmail.com ha scritto: On Thursday, February 28, 2013 at 7:49 AM, M.-A. Lemburg wrote: There you go:

Re: [Catalog-sig] Pypi cdn for hosted packages

2013-02-28 Thread Jim Fulton
On Thu, Feb 28, 2013 at 5:39 AM, Jesse Noller jnol...@gmail.com wrote: Thread fork. Anyway. I know we have at least 1 major rep of a cloud provider on the list, and I have at least one off in my pocket. I'd like to start discussing (completely ignoring past efforts and discussion which

Re: [Catalog-sig] Deprecate External Links

2013-02-28 Thread Daniel Holth
On Thu, Feb 28, 2013 at 7:43 AM, Reinout van Rees rein...@vanrees.org wrote: On 27-02-13 16:26, Donald Stufft wrote: 2. External links decrease the expected uptime for a particular set of requirements. PyPI itself has become very stable, however the same cannot be said for

Re: [Catalog-sig] Deprecate External Links

2013-02-28 Thread Ian Cordasco
On Thu, Feb 28, 2013 at 10:30 AM, Lennart Regebro rege...@gmail.com wrote: On Thu, Feb 28, 2013 at 10:43 AM, Lennart Regebro rege...@gmail.com wrote: On Thu, Feb 28, 2013 at 9:28 AM, Nick Coghlan ncogh...@gmail.com wrote: Pissing off the maintainers off packages that currently rely on external

Re: [Catalog-sig] remove historic download/homepage links for a project

2013-02-28 Thread Ronald Oussoren
On 28 Feb, 2013, at 14:41, holger krekel hol...@merlinux.eu wrote: That's the #2 thing I hate about some packages: removed releases that I faithfully pinned in my buildout (or requirements.txt). Removing releases is, imho, irresponsible. it's bad, yes. But necessary to have. Or am the

Re: [Catalog-sig] Deprecate External Links

2013-02-28 Thread Doug Hellmann
On Feb 28, 2013, at 3:43 AM, Nick Coghlan wrote: On Thu, Feb 28, 2013 at 6:12 PM, M.-A. Lemburg m...@egenix.com wrote: On 28.02.2013 07:39, Nick Coghlan wrote: 1. The next generation metadata infrastructure will NOT support external hosting of files indexed on PyPI - if you don't upload the

Re: [Catalog-sig] remove historic download/homepage links for a project

2013-02-28 Thread M.-A. Lemburg
On 28.02.2013 17:27, Ronald Oussoren wrote: On 28 Feb, 2013, at 14:41, holger krekel hol...@merlinux.eu wrote: That's the #2 thing I hate about some packages: removed releases that I faithfully pinned in my buildout (or requirements.txt). Removing releases is, imho, irresponsible. it's

Re: [Catalog-sig] Migrating away from scanning home pages

2013-02-28 Thread M.-A. Lemburg
I've added the proposal to the wiki to keep collecting comments and updates: http://wiki.python.org/moin/PyPI/DownloadMetaDataProposal On 28.02.2013 12:55, M.-A. Lemburg wrote: On 28.02.2013 12:45, Donald Stufft wrote: On Thursday, February 28, 2013 at 5:55 AM, M.-A. Lemburg wrote: I think we

Re: [Catalog-sig] Pypi cdn for hosted packages

2013-02-28 Thread M.-A. Lemburg
I've created a wiki page with the CloudFront setup description: http://wiki.python.org/moin/CloudPyPI/ExampleCDN -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Feb 28 2013) Python Projects, Consulting and Support ... http://www.egenix.com/

Re: [Catalog-sig] Pypi cdn for hosted packages

2013-02-28 Thread Noah Kantrowitz
You can go ahead and shut this down please, as I said our CDN partner has already been selected. --Noah On Feb 28, 2013, at 9:19 AM, M.-A. Lemburg wrote: I've created a wiki page with the CloudFront setup description: http://wiki.python.org/moin/CloudPyPI/ExampleCDN -- Marc-Andre

Re: [Catalog-sig] PyPI limitations (was: Deprecate External Links)

2013-02-28 Thread Noah Kantrowitz
On Feb 28, 2013, at 2:14 AM, M.-A. Lemburg wrote: On 27.02.2013 19:11, Noah Kantrowitz wrote: On Feb 27, 2013, at 9:28 AM, M.-A. Lemburg wrote: On 27.02.2013 18:05, Noah Kantrowitz wrote: M.-A. Lemburg m...@egenix.com wrote: I propose we deprecate the external links that PyPI has

Re: [Catalog-sig] PyPI terms (was: Deprecate External Links)

2013-02-28 Thread Noah Kantrowitz
On Feb 28, 2013, at 2:22 AM, M.-A. Lemburg wrote: On 27.02.2013 19:11, Noah Kantrowitz wrote: On Feb 27, 2013, at 9:28 AM, M.-A. Lemburg wrote: [reasons for not hosting distribution files on PyPI] * giving up control This is the point of running a package server, the author gives up

Re: [Catalog-sig] PyPI terms

2013-02-28 Thread M.-A. Lemburg
On 28.02.2013 18:44, Noah Kantrowitz wrote: On Feb 28, 2013, at 2:22 AM, M.-A. Lemburg wrote: BTW: I've never seen a hosting website require agreeing to giving users of the website the same distribution rights as the owner of the website. You should read terms of service more closely

Re: [Catalog-sig] PyPI terms

2013-02-28 Thread Noah Kantrowitz
On Feb 28, 2013, at 10:14 AM, M.-A. Lemburg wrote: On 28.02.2013 18:44, Noah Kantrowitz wrote: On Feb 28, 2013, at 2:22 AM, M.-A. Lemburg wrote: BTW: I've never seen a hosting website require agreeing to giving users of the website the same distribution rights as the owner of the website.

Re: [Catalog-sig] Pypi cdn for hosted packages

2013-02-28 Thread M.-A. Lemburg
On 28.02.2013 18:25, Noah Kantrowitz wrote: You can go ahead and shut this down please, as I said our CDN partner has already been selected. I know. Again: this is for testing a CDN setup with installers, mirrors, etc. It is not meant as permanent solution and will get shut down again, after

Re: [Catalog-sig] Next generation package infrastructure (was: Deprecate External Links)

2013-02-28 Thread PJ Eby
On Thu, Feb 28, 2013 at 4:31 AM, M.-A. Lemburg m...@egenix.com wrote: In order for this to work out, you will need to get the support of people hosting packages externally and address their concerns. The current discussion has been too dogmatic for my taste. A more pragmatic approach would

Re: [Catalog-sig] Migrating away from scanning home pages (was: Deprecate External Links)

2013-02-28 Thread PJ Eby
On Thu, Feb 28, 2013 at 5:55 AM, M.-A. Lemburg m...@egenix.com wrote: I think we all agree that scanning arbitrary HTML pages for download links is not a good idea and we need to transition away from this towards a more reliable system. Here's an approach that would work to start the

Re: [Catalog-sig] Deprecate External Links

2013-02-28 Thread holger krekel
On Thu, Feb 28, 2013 at 16:30 +0100, Lennart Regebro wrote: On Thu, Feb 28, 2013 at 10:43 AM, Lennart Regebro rege...@gmail.com wrote: On Thu, Feb 28, 2013 at 9:28 AM, Nick Coghlan ncogh...@gmail.com wrote: Pissing off the maintainers off packages that currently rely on external hosting by

Re: [Catalog-sig] Deprecate External Links

2013-02-28 Thread holger krekel
On Thu, Feb 28, 2013 at 13:56 +0100, Reinout van Rees wrote: On 28-02-13 10:43, holger krekel wrote: On Thu, Feb 28, 2013 at 06:38 +0100, Andreas Jung wrote: I give a shit at the arguments pulled out every time by package maintainers using PyPI only for listing their packages. I am both

Re: [Catalog-sig] Deprecate External Links

2013-02-28 Thread Donald Stufft
On Thursday, February 28, 2013 at 1:23 PM, PJ Eby wrote: On Thu, Feb 28, 2013 at 4:08 AM, Nick Coghlan ncogh...@gmail.com (mailto:ncogh...@gmail.com) wrote: On Thu, Feb 28, 2013 at 7:00 PM, holger krekel hol...@merlinux.eu (mailto:hol...@merlinux.eu) wrote: To summarize, having

Re: [Catalog-sig] remove historic download/homepage links for a project

2013-02-28 Thread Richard Jones
On 1 March 2013 04:10, Tres Seaver tsea...@palladion.com wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 02/28/2013 11:27 AM, Ronald Oussoren wrote: But necessary to have. Or am the only one that accidently released a version that had serious bugs? Nope. The way to address such a

Re: [Catalog-sig] Deprecate External Links

2013-02-28 Thread Donald Stufft
On Thursday, February 28, 2013 at 6:31 PM, PJ Eby wrote: On Thu, Feb 28, 2013 at 5:00 PM, Donald Stufft donald.stu...@gmail.com (mailto:donald.stu...@gmail.com) wrote: SSL checking on upload should be possible, do you want a patch? If it uses the 'requests' library, yes, I'll accept

Re: [Catalog-sig] Deprecate External Links

2013-02-28 Thread PJ Eby
On Thu, Feb 28, 2013 at 5:00 PM, Donald Stufft donald.stu...@gmail.com wrote: SSL checking on upload should be possible, do you want a patch? If it uses the 'requests' library, yes, I'll accept one. But I don't want to do any direct implementation of SSL cert checking in setuptools, at least

Re: [Catalog-sig] Pypi cdn for hosted packages

2013-02-28 Thread Donald Stufft
On Thursday, February 28, 2013 at 10:13 AM, Noah Kantrowitz wrote: Reponding from my phone quickly before this gets any further, will write more later. Plan is to have pypi move package download links to a new hostname (probably pypi-download.python.org (http://pypi-download.python.org)) and

Re: [Catalog-sig] PyPI terms

2013-02-28 Thread Terry Reedy
On 2/28/2013 1:19 PM, Noah Kantrowitz wrote: Because I happen to have YouTube open anyway: For clarity, you retain all of your ownership rights in your Content. However, by submitting Content to YouTube, you hereby grant YouTube a worldwide, non-exclusive, royalty-free, sublicenseable and

Re: [Catalog-sig] remove historic download/homepage links for a project

2013-02-28 Thread Tres Seaver
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 02/28/2013 06:21 PM, Richard Jones wrote: On 1 March 2013 04:10, Tres Seaver tsea...@palladion.com wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 02/28/2013 11:27 AM, Ronald Oussoren wrote: But necessary to have. Or am the only one

Re: [Catalog-sig] Deprecate External Links

2013-02-28 Thread Lennart Regebro
On Thu, Feb 28, 2013 at 8:52 PM, holger krekel hol...@merlinux.eu wrote: There are also packages which have some (older) release files on pypi and newer ones outside (e.g. lockfile with 78256 downloads from code.google.com). You didn't include such in your 2651 emails, or did you? No, I

Re: [Catalog-sig] remove historic download/homepage links for a project

2013-02-28 Thread Ronald Oussoren
On 1 Mar, 2013, at 4:08, Tres Seaver tsea...@palladion.com wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 02/28/2013 06:21 PM, Richard Jones wrote: On 1 March 2013 04:10, Tres Seaver tsea...@palladion.com wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 02/28/2013 11:27