Re: [Catalog-sig] Perhaps PyPI will do

2005-04-07 Thread holger krekel
Hi David, On Thu, Apr 07, 2005 at 09:32 -0700, David Ascher wrote: I find the discussion depressing in many ways. Did i miss some of the discussion? At least on catalog-sig and in the blogs it was going quite ok in my opionion. But maybe we had different expectations :-) holger

[Catalog-sig] current repo of pypi

2012-11-30 Thread Holger Krekel
Hello, The http://wiki.python.org/moin/CheeseShopDev page mentioned that the repo is undergoing migration. Is there some (even intermediate) url which i could pull today? thanks, holger ___ Catalog-SIG mailing list Catalog-SIG@python.org

[Catalog-sig] disabling the serving of links from description_html?

2012-12-18 Thread Holger Krekel
Hi Richard, hi all, While reading the pypi main and other sources i wondered how we could switch off serving links from description_html, at least on a per-project basis. It's really annoying that when you start to add some links to a long_description that installation of your package will thus

Re: [Catalog-sig] disabling the serving of links from description_html?

2012-12-18 Thread Holger Krekel
On Tue, Dec 18, 2012 at 5:46 PM, M.-A. Lemburg m...@egenix.com wrote: On 18.12.2012 15:54, Holger Krekel wrote: Hi Richard, hi all, While reading the pypi main and other sources i wondered how we could switch off serving links from description_html, at least on a per-project basis

[Catalog-sig] fresh pep381run pypi-mirroring fails since 1 week

2013-01-07 Thread Holger Krekel
Hi all, During the last 7 days i tried running pep381run with a fresh directory on two different hosts. They both failed while trying to copy azb_nester-1.2.0.tar.gz, see here for the traceback: http://bpaste.net/show/SoMoyjdJEIGvm99dH6gG/ It seems that azb_nester does not have any files

Re: [Catalog-sig] test pypi server?

2013-01-26 Thread Holger Krekel
Hey Chris, according to http://pypi.python.org there should be a test pypi server at http://testpypi.python.org/pypi but at the moment it gives 502 Bad Gateway. cheers, holger On Sat, Jan 26, 2013 at 10:33 AM, Chris Withers ch...@simplistix.co.ukwrote: Hi All, I remember mention of a test

Re: [Catalog-sig] Use user-specific site-packages by default?

2013-02-05 Thread Holger Krekel
On Tue, Feb 5, 2013 at 1:51 PM, Donald Stufft donald.stu...@gmail.comwrote: On Tuesday, February 5, 2013 at 5:16 AM, Lennart Regebro wrote: 1. Packages should only be installed from the given package indexes. No scraping of websites as at least easy_install/buildout does, no downloading

Re: [Catalog-sig] Use user-specific site-packages by default?

2013-02-05 Thread Holger Krekel
On Tue, Feb 5, 2013 at 2:05 PM, Jesse Noller jnol...@gmail.com wrote: On Feb 5, 2013, at 8:02 AM, Holger Krekel holger.kre...@gmail.com wrote: On Tue, Feb 5, 2013 at 1:51 PM, Donald Stufft donald.stu...@gmail.comwrote: On Tuesday, February 5, 2013 at 5:16 AM, Lennart Regebro wrote: 1

Re: [Catalog-sig] Use user-specific site-packages by default?

2013-02-05 Thread Holger Krekel
On Tue, Feb 5, 2013 at 2:13 PM, Lennart Regebro rege...@gmail.com wrote: On Tue, Feb 5, 2013 at 2:02 PM, Holger Krekel holger.kre...@gmail.com wrote: Dropping the crawling over external pages needs _much_ more than just a few months deprecation warnings, rather years. There are many

Re: [Catalog-sig] Use user-specific site-packages by default?

2013-02-05 Thread holger krekel
On Tue, Feb 05, 2013 at 15:46 +0100, Giovanni Bajo wrote: Il giorno 05/feb/2013, alle ore 15:06, Holger Krekel holger.kre...@gmail.com ha scritto: In the end, however, none of this prevents MITM attacks between a downloader and pypi.python.org. Or between the uploader

Re: [Catalog-sig] Use user-specific site-packages by default?

2013-02-05 Thread holger krekel
On Tue, Feb 05, 2013 at 16:07 +0100, Lennart Regebro wrote: On Tue, Feb 5, 2013 at 3:06 PM, Holger Krekel holger.kre...@gmail.com wrote: I wouldn't assume that maintainers are easily reachable. I've contacted at least three people of different (1K downloads) packages which never responded

Re: [Catalog-sig] Use user-specific site-packages by default?

2013-02-05 Thread holger krekel
On Tue, Feb 05, 2013 at 10:18 -0500, Donald Stufft wrote: On Tuesday, February 5, 2013 at 10:14 AM, holger krekel wrote: Transporting almost all externally reachable packages to be locally pypi served is also kind of a low hanging fruit, although probably slightly higher hanging than SSL

Re: [Catalog-sig] Use user-specific site-packages by default?

2013-02-05 Thread holger krekel
On Tue, Feb 05, 2013 at 15:54 -0500, Terry Reedy wrote: On 2/5/2013 11:35 AM, Lennart Regebro wrote: On Tue, Feb 5, 2013 at 5:03 PM, Donald Stufft donald.stu...@gmail.com wrote: Besides the issues with validating that the package We are mirroring is the authentic one there's also a legal

Re: [Catalog-sig] RubyGems Threat Model and Requirements

2013-02-12 Thread holger krekel
On Tue, Feb 12, 2013 at 12:44 -0500, Daniel Holth wrote: On Tue, Feb 12, 2013 at 11:27 AM, Giovanni Bajo ra...@develer.com wrote: Your Task #6/#7 (related to PyPI generating the trust file, and pip verifying it) are the ones where I think the input of the TUF team will be most

Re: [Catalog-sig] HTTPS now promoted on PyPI

2013-02-19 Thread holger krekel
On Tue, Feb 19, 2013 at 14:23 +0100, Giovanni Bajo wrote: Il giorno 19/feb/2013, alle ore 06:13, Richard Jones r1chardj0...@gmail.com ha scritto: Hi all, I've just altered the nginx configuration to promote (ie. redirect to) HTTPS for all GET/HEAD requests. This includes HSTS, but

Re: [Catalog-sig] Deprecate External Links

2013-02-27 Thread holger krekel
On Wed, Feb 27, 2013 at 14:49 -0500, Monty Taylor wrote: On 02/27/2013 02:47 PM, Aaron Meurer wrote: On Wed, Feb 27, 2013 at 11:37 AM, holger krekel hol...@merlinux.eu wrote: On Wed, Feb 27, 2013 at 19:34 +0100, Lennart Regebro wrote: On Wed, Feb 27, 2013 at 5:34 PM, M.-A. Lemburg m

Re: [Catalog-sig] Deprecate External Links

2013-02-28 Thread holger krekel
On Wed, Feb 27, 2013 at 22:04 +0100, Lennart Regebro wrote: On Wed, Feb 27, 2013 at 8:49 PM, Monty Taylor mord...@inaugust.com wrote: But wouldn't this only be a change in pip/easy_install, not PyPI itself? I suppose you could explicitly break the external links by having them point to

Re: [Catalog-sig] Deprecate External Links

2013-02-28 Thread holger krekel
On Thu, Feb 28, 2013 at 09:48 +1100, Richard Jones wrote: On 28 February 2013 08:31, PJ Eby p...@telecommunity.com wrote: OTOH, I currently make development snapshots of setuptools and other projects available by dumping them in a directory that's used as an external download URL.

Re: [Catalog-sig] Deprecate External Links

2013-02-28 Thread holger krekel
On Thu, Feb 28, 2013 at 06:38 +0100, Andreas Jung wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 +1 for the proposal The complete discussion on this topic is once again absurd and bizarre. We are discussing the issue with externally hosted packages every year and the situation has

Re: [Catalog-sig] Deprecate External Links

2013-02-28 Thread holger krekel
On Thu, Feb 28, 2013 at 16:30 +0100, Lennart Regebro wrote: On Thu, Feb 28, 2013 at 10:43 AM, Lennart Regebro rege...@gmail.com wrote: On Thu, Feb 28, 2013 at 9:28 AM, Nick Coghlan ncogh...@gmail.com wrote: Pissing off the maintainers off packages that currently rely on external hosting by

Re: [Catalog-sig] Deprecate External Links

2013-02-28 Thread holger krekel
On Thu, Feb 28, 2013 at 13:56 +0100, Reinout van Rees wrote: On 28-02-13 10:43, holger krekel wrote: On Thu, Feb 28, 2013 at 06:38 +0100, Andreas Jung wrote: I give a shit at the arguments pulled out every time by package maintainers using PyPI only for listing their packages. I am both

Re: [Catalog-sig] Deprecate External Links

2013-03-01 Thread holger krekel
On Fri, Mar 01, 2013 at 10:02 +0100, Reinout van Rees wrote: On 28-02-13 21:08, holger krekel wrote: I have seen that position in this discussion (I have to upload 120 files per release, so I won't do that, for instance). haven't seen that. Marc-Andre Lemburg said this, which I took

Re: [Catalog-sig] Deprecate External Links

2013-03-01 Thread holger krekel
On Fri, Mar 01, 2013 at 10:24 +0100, M.-A. Lemburg wrote: On 01.03.2013 10:02, Reinout van Rees wrote: On 28-02-13 21:08, holger krekel wrote: I have seen that position in this discussion (I have to upload 120 files per release, so I won't do that, for instance). haven't seen

[Catalog-sig] homepage/download metadata cleaning

2013-03-01 Thread holger krekel
Hi Richard, all, somewhere deep in the threads i mentioned i wrote a little cleanpypi.py script which takes a project name as an argument and then goes to pypi.python.org and removes all homepage/download metadata entries for this project. This sanitizes/speeds up installation because

Re: [Catalog-sig] homepage/download metadata cleaning

2013-03-01 Thread holger krekel
On Fri, Mar 01, 2013 at 06:09 -0500, Donald Stufft wrote: On Friday, March 1, 2013 at 6:04 AM, M.-A. Lemburg wrote: On 01.03.2013 11:19, holger krekel wrote: Hi Richard, all, somewhere deep in the threads i mentioned i wrote a little cleanpypi.py script which takes a project name

Re: [Catalog-sig] PyPI terms

2013-03-01 Thread holger krekel
On Fri, Mar 01, 2013 at 15:11 +0100, M.-A. Lemburg wrote: On 01.03.2013 15:02, Jesse Noller wrote: Okie doke. So we can move on to putting up the CDN and deprecating external links for now? I don't think anyone is against putting up a CDN. It should meet the same security requirements we

Re: [Catalog-sig] homepage/download metadata cleaning

2013-03-01 Thread holger krekel
On Fri, Mar 01, 2013 at 23:50 +0100, Lennart Regebro wrote: On Fri, Mar 1, 2013 at 8:31 PM, M.-A. Lemburg m...@egenix.com wrote: Hmm, then why not remove links that don't match the above from the /simple/ index pages ? I think we can do that, but if we *start* with that, we will just

Re: [Catalog-sig] Fw: Deprecate External Links

2013-03-05 Thread holger krekel
On Tue, Mar 05, 2013 at 04:19 -0500, Donald Stufft wrote: Forwarding this since I assume it was accidently sent to only me, and it's important to note that there is some sort of miscounting bug going on. Forwarded message: From: Donald Stufft donald.stu...@gmail.com To: M.-A.

Re: [Catalog-sig] hash tags

2013-03-08 Thread holger krekel
Hi Philip, all, On Fri, Mar 08, 2013 at 14:16 -0500, PJ Eby wrote: The key to making this transition isn't creating elaborate new standards for the tools, it's *creating new tools for the standards*. If we can find a way to improve PyPI and not require the world to change first, that's a big

[Catalog-sig] pre-PEP: transition to release-file hosting at pypi site

2013-03-10 Thread holger krekel
Hi Donald, Richard, Nick, Philip, Marc-Andre, all, after some more thinking i wrote a simplified PEP draft for transitioning hosting of release files to pypi.python.org. A PEP is warranted IMO because the according changes will affect all python package maintainers and the Python packaging

Re: [Catalog-sig] pre-PEP: transition to release-file hosting at pypi site

2013-03-10 Thread holger krekel
On Sun, Mar 10, 2013 at 13:35 -0400, Donald Stufft wrote: On Mar 10, 2013, at 11:07 AM, holger krekel hol...@merlinux.eu wrote: [...] Transitioning to pypi-cache mode - When transitioning from the currently implicit pypi-ext mode to pypi-cache

Re: [Catalog-sig] pre-PEP: transition to release-file hosting at pypi site

2013-03-10 Thread holger krekel
On Sun, Mar 10, 2013 at 14:29 -0400, Donald Stufft wrote: On Mar 10, 2013, at 2:18 PM, holger krekel hol...@merlinux.eu wrote: On Sun, Mar 10, 2013 at 13:35 -0400, Donald Stufft wrote: On Mar 10, 2013, at 11:07 AM, holger krekel hol...@merlinux.eu wrote: [...] Transitioning to pypi

Re: [Catalog-sig] pre-PEP: transition to release-file hosting at pypi site

2013-03-11 Thread holger krekel
10, 2013 at 15:41 -0400, PJ Eby wrote: On Sun, Mar 10, 2013 at 11:07 AM, holger krekel hol...@merlinux.eu wrote: Philip, Marc-Andre, Richard (Jones), Nick and catalog-sig/distutils-sig: scrutiny and feedback welcome. Hi Holger. I'm having some difficulty interpreting your proposal because

Re: [Catalog-sig] pre-PEP: transition to release-file hosting at pypi site

2013-03-11 Thread holger krekel
Hi again, A correction on one point of my last mail to you, On Mon, Mar 11, 2013 at 10:02 +, holger krekel wrote: My suggestion would be to do two things: First, make the state a boolean: crawl external links, with the current state yes and the future state no, with no simply

Re: [Catalog-sig] A 90% Solution

2013-03-12 Thread holger krekel
On Mon, Mar 11, 2013 at 19:04 -0400, PJ Eby wrote: Just a thought, but... If 90% of PyPI projects do not have any external files to download, then, wouldn't it make sense to: sidenote: we need to verify and clarify the 90/10 ratio. It would be the basis for action/changing pypi-state so we

[Catalog-sig] V2 pre-PEP: transitioning to release file hosting on PYPI

2013-03-12 Thread holger krekel
.. [2] Marc-Andre Lemburg, reasons for external hosting, http://mail.python.org/pipermail/catalog-sig/2013-March/005626.html .. [3] Holger Krekel, Script to remove homepage/download metadata for all releases http://mail.python.org/pipermail/catalog-sig/2013-February/005423.html

Re: [Catalog-sig] V2 pre-PEP: transitioning to release file hosting on PYPI

2013-03-12 Thread holger krekel
On Wed, Mar 13, 2013 at 01:19 +1000, Nick Coghlan wrote: That looks pretty good to me. My only comment is that qualifiers like new don't age well in an API. The explicit nocrawlhomepage and nocrawldownload might be a better choice. Right, we might also consider dropping rel-attributing given

Re: [Catalog-sig] V2 pre-PEP: transitioning to release file hosting on PYPI

2013-03-12 Thread holger krekel
On Tue, Mar 12, 2013 at 11:53 -0400, PJ Eby wrote: On Tue, Mar 12, 2013 at 7:38 AM, holger krekel hol...@merlinux.eu wrote: In addition, maintainers of installation tools are asked to release two updates. The first one shall provide clear warnings if external crawling needs to happen

Re: [Catalog-sig] V2 pre-PEP: transitioning to release file hosting on PYPI

2013-03-12 Thread holger krekel
Hi Marc-Andre, all, On Tue, Mar 12, 2013 at 17:06 +0100, M.-A. Lemburg wrote: On 12.03.2013 12:38, holger krekel wrote: Hi all, below is the new PEP pre-submit version (V2) which incorporates the latest suggestions and aims at a rapidly deployable solution. Thanks in particular

Re: [Catalog-sig] V2 pre-PEP: transitioning to release file hosting on PYPI

2013-03-12 Thread holger krekel
Hi Carl, On Tue, Mar 12, 2013 at 10:48 -0600, Carl Meyer wrote: Hi Holger, I am confused about the discrepancy between the title of this pre-PEP (transition to release file hosting on PyPI) and the contents of the PEP, which describe a transition to not crawling _HTML pages_ on external

Re: [Catalog-sig] pre-PEP: transition to release-file hosting at pypi site

2013-03-12 Thread holger krekel
On Tue, Mar 12, 2013 at 13:18 -0400, PJ Eby wrote: On Tue, Mar 12, 2013 at 12:29 PM, Jacob Kaplan-Moss ja...@jacobian.org wrote: On Tue, Mar 12, 2013 at 11:19 AM, M.-A. Lemburg m...@egenix.com wrote: So let's do this carefully and find a good solution before jumping to conclusions.

Re: [Catalog-sig] pre-PEP: transition to release-file hosting at pypi site

2013-03-12 Thread holger krekel
On Tue, Mar 12, 2013 at 12:18 -0600, Carl Meyer wrote: It seems to me that there's a remarkable level of consensus developing here (though it may not look like it), and a small set of remaining open questions. The consensus (as I see it): - Migrate away from scraping external HTML pages,

Re: [Catalog-sig] V2 pre-PEP: transitioning to release file hosting on PYPI

2013-03-12 Thread holger krekel
On Tue, Mar 12, 2013 at 19:07 +0100, M.-A. Lemburg wrote: Just a quick note (more later, if time permits)... On 12.03.2013 18:05, holger krekel wrote: Hi Marc-Andre, all, - Prepare PYPI implementation to allow a per-project hosting mode, effectively enabling or disabling external

Re: [Catalog-sig] pre-PEP: transition to release-file hosting at pypi site

2013-03-12 Thread holger krekel
On Tue, Mar 12, 2013 at 14:36 -0500, Jacob Kaplan-Moss wrote: On Tue, Mar 12, 2013 at 2:21 PM, PJ Eby p...@telecommunity.com wrote: The *only* thing I object to is the part where some people want to ban external links from /simple, always and forever, regardless of the package authors'

Re: [Catalog-sig] pre-PEP: transition to release-file hosting at pypi site

2013-03-12 Thread holger krekel
On Tue, Mar 12, 2013 at 15:21 -0400, PJ Eby wrote: On Tue, Mar 12, 2013 at 2:18 PM, Carl Meyer c...@oddbird.net wrote: It seems to me that there's a remarkable level of consensus developing here (though it may not look like it), and a small set of remaining open questions. The consensus

[Catalog-sig] V3 PEP-draft for transitioning to pypi-hosting of release files

2013-03-13 Thread holger krekel
to be sorted and clarified, among them the versioning-API suggestion by Marc-Andre. Thanks for everybody's support and feedback so far, holger PEP: XXX Title: Transitioning to release-file hosting on PyPI Version: $Revision$ Last-Modified: $Date$ Author: Holger Krekel hol...@merlinux.eu, Carl Meyer c

Re: [Catalog-sig] V3 PEP-draft for transitioning to pypi-hosting of release files

2013-03-14 Thread holger krekel
On Wed, Mar 13, 2013 at 23:43 -0700, Nick Coghlan wrote: On Wed, Mar 13, 2013 at 5:16 PM, Carl Meyer c...@oddbird.net wrote: There is no instead of. There are parallel proposals (see the TUF thread) to improve the security of the ecosystem, and those proposals are not mutually exclusive

[Catalog-sig] V4 Pre-PEP: transition to release-file hosting on PYPI

2013-03-15 Thread holger krekel
Version: $Revision$ Last-Modified: $Date$ Author: Holger Krekel hol...@merlinux.eu, Carl Meyer c...@oddbird.net Discussions-To: catalog-sig@python.org Status: Draft (PRE-submit V4) Type: Process Content-Type: text/x-rst Created: 10-Mar-2013 Post-History: Abstract This PEP proposes a backward

Re: [Catalog-sig] V4 Pre-PEP: transition to release-file hosting on PYPI

2013-03-15 Thread holger krekel
On Fri, Mar 15, 2013 at 11:15 -0400, PJ Eby wrote: Do we even need the internal/external rel info? I was planning to just use the URL hostname. i.e., are there any use cases for designating an externally-hosted file internal, or an internally-hosted file external? If not, it seems the

Re: [Catalog-sig] V4 Pre-PEP: transition to release-file hosting on PYPI

2013-03-15 Thread holger krekel
On Fri, Mar 15, 2013 at 22:01 -0400, PJ Eby wrote: On Fri, Mar 15, 2013 at 7:16 PM, Carl Meyer c...@oddbird.net wrote: Ok, pending agreement from Holger I'll make a change in the PEP to explicitly allow clients to make decisions based on either the rel attributes or based on hostnames.

Re: [Catalog-sig] Updated PEP 438

2013-03-21 Thread holger krekel
: Holger Krekel hol...@merlinux.eu, Carl Meyer c...@oddbird.net BDFL-Delegate: Richard Jones rich...@python.org Discussions-To: catalog-sig@python.org Status: Draft Type: Process Content-Type: text/x-rst Created: 15-Mar-2013 Post-History: Abstract This PEP proposes a backward-compatible two

Re: [Catalog-sig] Replacement client for pep381client

2013-03-21 Thread holger krekel
On Wed, Mar 20, 2013 at 19:27 -0700, Christian Theune wrote: On 2013-03-20 23:59:21 +, Christian Theune said: I'm currently re-initializing my own mirror. This basically can be run in-place by just removing the existing state data and calling my sync script (bsn-mirror) instead of

Re: [Catalog-sig] Merge catalog-sig and distutils-sig

2013-03-28 Thread holger krekel
On Thu, Mar 28, 2013 at 14:22 -0400, Donald Stufft wrote: Is there much point in keeping catalog-sig and distutils-sig separate? It seems to me that most of the same people are on both lists, and the topics almost always have consequences to both sides of the coin. So much so that it's

Re: [Catalog-sig] Merge catalog-sig and distutils-sig

2013-03-28 Thread holger krekel
On Thu, Mar 28, 2013 at 15:42 -0400, Donald Stufft wrote: On Mar 28, 2013, at 3:39 PM, PJ Eby p...@telecommunity.com wrote: On Thu, Mar 28, 2013 at 3:14 PM, Fred Drake f...@fdrake.net wrote: On Thu, Mar 28, 2013 at 2:22 PM, Donald Stufft don...@stufft.io wrote: Is there much point in