Re: [Distutils] distlib updated - comments sought

2012-10-09 Thread PJ Eby
On Fri, Oct 5, 2012 at 2:38 PM, Paul Moore wrote: > That's a good point. Actually, writing a "local PyPI server" is much > easier if all you have to implement is the simple static page > interface. So I take back some of my objection - both XML-RPC and the > static page interface make sense to sup

Re: [Distutils] distlib updated - comments sought

2012-10-08 Thread Daniel Holth
On Sat, Oct 6, 2012 at 3:44 AM, Antonio Cavallo wrote: > Here few lesson learnt using rpm and python at fairly large scale > (http://download.opensuse.org/repositories/home:/cavallo71:/opt-python-interpreters) > project of mine I wish could be of some help in thinking of a "packaging > system"

Re: [Distutils] distlib updated - comments sought

2012-10-06 Thread Vinay Sajip
Matthias Klose ubuntu.com> writes: > > PS: are there really ~2 packages on pypi, or do you count old versions > too? > Not counting old versions, my current list of packages where an archive is hosted on PyPI numbers a little under 18,000. While not bang up to date, it's unlikely to be t

Re: [Distutils] distlib updated - comments sought

2012-10-06 Thread Matthias Klose
On 05.10.2012 19:24, Paul Moore wrote: > On 5 October 2012 17:04, Daniel Holth wrote: > >> ~1300 of the ~2 packages on pypi have trouble using setup.py as >> their build system / metadata source format. > > That's interesting information. Do you know in what way they have > trouble with setu

Re: [Distutils] distlib updated - comments sought

2012-10-06 Thread Antonio Cavallo
On 5 Oct 2012, at 20:28, Stanley A. Klein wrote: > I assume that by an "installer" you mean the combination of package > building (done by e.g., rpmbuild) and installation (done by e.g., rpm or > yum). Hi, I think you've hit the jackpot here… Here few lesson learnt using rpm and python at fairly

Re: [Distutils] distlib updated - comments sought

2012-10-05 Thread Daniel Holth
On Fri, Oct 5, 2012 at 3:28 PM, Stanley A. Klein wrote: > The issue I had raised was attention to the needs of Linux packaging and > the Filesystem Hierarchy Standard, and apparently that is under > consideration. > > The response I received (from "Daniel Holth" ) said in > part "The FHS issues ar

Re: [Distutils] distlib updated - comments sought

2012-10-05 Thread Paul Moore
On 5 October 2012 20:28, Stanley A. Klein wrote: > What I didn't notice in the distlib documentation were low level functions > that would facilitate the allocation of Pypi files/directories to target > files/directories to help in preparation of the rpm spec. While I'm not particularly familiar

Re: [Distutils] distlib updated - comments sought

2012-10-05 Thread Stanley A. Klein
5 Oct 2012 15:49:04 + (UTC) > From: Vinay Sajip > To: Distutils-Sig@Python.Org > Subject: Re: [Distutils] distlib updated - comments sought > > Stanley A. Klein cpcug.org> writes: > >> I looked at the documentation and can't completely follow what you are

Re: [Distutils] distlib updated - comments sought

2012-10-05 Thread Daniel Holth
On Fri, Oct 5, 2012 at 2:40 PM, Vinay Sajip wrote: > Daniel Holth gmail.com> writes: > >> I'm basing this only on Vinay's numbers of >> how-many-packages-can-generate-a-yaml. It's probably mostly packages >> that import something he didn't have installed inside setup.py, but I >> don't have a goo

Re: [Distutils] distlib updated - comments sought

2012-10-05 Thread Vinay Sajip
Daniel Holth gmail.com> writes: > I'm basing this only on Vinay's numbers of > how-many-packages-can-generate-a-yaml. It's probably mostly packages > that import something he didn't have installed inside setup.py, but I > don't have a good way to find out exactly what is wrong with each one. > An

Re: [Distutils] distlib updated - comments sought

2012-10-05 Thread Paul Moore
On 5 October 2012 16:40, Daniel Holth wrote: >> Well, XML-RPC is of course preferable; the current code in distlib is just >> whatever I copied across from packaging, but the next step will be to look >> at the releases which are available from the different sources (XML-RPC, >> PyPI metadata URLs

Re: [Distutils] distlib updated - comments sought

2012-10-05 Thread Vinay Sajip
Paul Moore gmail.com> writes: > - a reasonable proportion of distlib offers alternatives to the > pkg_resources code, and more could be added. The next thing for me to look at is "entry points", which should be relatively straightforward to implement. Regards, Vinay Sajip

Re: [Distutils] distlib updated - comments sought

2012-10-05 Thread Vinay Sajip
Daniel Holth gmail.com> writes: > Bootstrapping is kinda annoying because Python doesn't include an > installer for pip or buildout or ... and it can be hard to choose > between the many excellent installers that are available on and off of > pypi. That was the point of packaging - to have somet

Re: [Distutils] distlib updated - comments sought

2012-10-05 Thread Daniel Holth
On Fri, Oct 5, 2012 at 1:24 PM, Paul Moore wrote: > On 5 October 2012 17:04, Daniel Holth wrote: > >> ~1300 of the ~2 packages on pypi have trouble using setup.py as >> their build system / metadata source format. > > That's interesting information. Do you know in what way they have > trouble

Re: [Distutils] distlib updated - comments sought

2012-10-05 Thread Paul Moore
On 5 October 2012 17:04, Daniel Holth wrote: > ~1300 of the ~2 packages on pypi have trouble using setup.py as > their build system / metadata source format. That's interesting information. Do you know in what way they have trouble with setup.py? Do they not use it at all, do they need featu

Re: [Distutils] distlib updated - comments sought

2012-10-05 Thread Daniel Holth
On Fri, Oct 5, 2012 at 10:23 AM, Paul Moore wrote: > On 5 October 2012 14:47, Daniel Holth wrote: > PPS If my locator interface ever matures enough, I'm happy to release > it on PyPI. But I don't want to compete with Vinay or a stdlib > implementation, so I'd rather co-operate on a unified view

Re: [Distutils] distlib updated - comments sought

2012-10-05 Thread Vinay Sajip
Daniel Holth gmail.com> writes: > IIRC the most practical limitation is that the XML-RPC interface > doesn't exist on the mirrors. Of course. It's easier to replicate a static website than an XML-RPC server to Web Scale ;-), but I presume it's only a question of hosting resources and developer t

Re: [Distutils] distlib updated - comments sought

2012-10-05 Thread Vinay Sajip
Stanley A. Klein cpcug.org> writes: > I looked at the documentation and can't completely follow what you are > doing, although I was told in another email that the issue I raised was > being addressed in pkg_resources. The goal of distlib is to provide a library of low-level functions relating t

Re: [Distutils] distlib updated - comments sought

2012-10-05 Thread Daniel Holth
On Fri, Oct 5, 2012 at 11:39 AM, Vinay Sajip wrote: > Paul Moore gmail.com> writes: > >> The first ones are fine, as they point to files. The second is often a >> file, and seems to frequently duplicate the first. I'm not sure how >> useful it is. The final one often points to a further webpage -

Re: [Distutils] distlib updated - comments sought

2012-10-05 Thread Vinay Sajip
Paul Moore gmail.com> writes: > The first ones are fine, as they point to files. The second is often a > file, and seems to frequently duplicate the first. I'm not sure how > useful it is. The final one often points to a further webpage - I > presume that's what you plan to scrape. That's where t

Re: [Distutils] distlib updated - comments sought

2012-10-05 Thread Stanley A. Klein
Klein On Fri, October 5, 2012 6:00 am, Vinay Sajip wrote: > Date: Thu, 4 Oct 2012 22:28:01 + (UTC) > From: Vinay Sajip > To: Distutils-Sig@Python.Org > Subject: [Distutils] distlib updated - comments sought > > I've updated distlib as follows: > >

Re: [Distutils] distlib updated - comments sought

2012-10-05 Thread Paul Moore
On 5 October 2012 15:37, Vinay Sajip wrote: >> PS If you want to start over-engineering the flexibility, users should >> have a way of choosing whether to use the webscraper or XMLRPC >> interfaces to PyPI. The former finds more packages (as I understand >> it) whereas the latter is much faster. A

Re: [Distutils] distlib updated - comments sought

2012-10-05 Thread Vinay Sajip
Paul Moore gmail.com> writes: > No-one could try to claim that the sort of web-scraping that > easy_install/pip does is a "simple" reference implementation, either. > If you take that viewpoint, I'd say the stdlib implementation should > *only* use the XMLRPC interface to PyPI. Code to use the "s

Re: [Distutils] distlib updated - comments sought

2012-10-05 Thread Paul Moore
On 5 October 2012 14:47, Daniel Holth wrote: > Is this supposed to go into the stdlib? Simple reference > implementations (wsgiref) fare better there than frameworks > (distutils). The pluggable installer metaframework belongs on pypi. I disagree. Having an installer depend on external packages i

Re: [Distutils] distlib updated - comments sought

2012-10-05 Thread Vinay Sajip
Daniel Holth gmail.com> writes: > Is this supposed to go into the stdlib? Simple reference > implementations (wsgiref) fare better there than frameworks > (distutils). The pluggable installer metaframework belongs on pypi. I'm thinking of distlib as a library rather than a framework. However, it

Re: [Distutils] distlib updated - comments sought

2012-10-05 Thread Daniel Holth
On Fri, Oct 5, 2012 at 6:29 AM, Vinay Sajip wrote: > Paul Moore gmail.com> writes: > >> Right, that's not going to be enough for me. I have a web page where >> files are hosted at >> http://my.domain/datastore/VARYING_ENCODED_ID/filename.tar.gz. The >> problem is that VARYING_ENCODED_ID changes r

Re: [Distutils] distlib updated - comments sought

2012-10-05 Thread Daniel Holth
Check out the reasonably modular and cleanly written pyg installer. https://github.com/rubik/pyg/ It has its own independent package database / metadata parser. https://github.com/rubik/pkgtools/blob/master/pkgtools/pkg.py It has a pypi interface too. ___

Re: [Distutils] distlib updated - comments sought

2012-10-05 Thread Tarek Ziadé
On 10/5/12 12:19 PM, Vinay Sajip wrote: Tarek Ziadé ziade.org> writes: I like the idea of having a main section to play with the tools, people can invoke with -m I don't think its a bad idea to have it in each module, like what Python does for SimpleHTTPServer or json.tools I've no problem

Re: [Distutils] distlib updated - comments sought

2012-10-05 Thread Daniel Holth
I mean should there be a pkgutil.pkg_listdir, etc. to accompany pkgutil.get_data Daniel Holth On Oct 5, 2012, at 5:32 AM, Vinay Sajip wrote: > Daniel Holth gmail.com> writes: > >> I still like the "mimic os.listdir but with an additional package >> argument" API because it has fewer parts. Is

Re: [Distutils] distlib updated - comments sought

2012-10-05 Thread Vinay Sajip
Paul Moore gmail.com> writes: > Right, that's not going to be enough for me. I have a web page where > files are hosted at > http://my.domain/datastore/VARYING_ENCODED_ID/filename.tar.gz. The > problem is that VARYING_ENCODED_ID changes regularly. It's possible to > calculate it in code, but only

Re: [Distutils] distlib updated - comments sought

2012-10-05 Thread Vinay Sajip
Tarek Ziadé ziade.org> writes: > I like the idea of having a main section to play with the tools, people > can invoke with -m > > I don't think its a bad idea to have it in each module, like what Python > does > for SimpleHTTPServer or json.tools > I've no problem with having a "if __name ==

Re: [Distutils] distlib updated - comments sought

2012-10-05 Thread Paul Moore
On 5 October 2012 10:27, Vinay Sajip wrote: >> +1 on this. Can you share a little more on your hint idea? I have a > >> specific use case in mind, and would like to know if it's covered. >> It's an intranet webpage that hosts packages in a very odd format, >> unfortunately, so I need to write code

Re: [Distutils] distlib updated - comments sought

2012-10-05 Thread Vinay Sajip
Daniel Holth gmail.com> writes: > I still like the "mimic os.listdir but with an additional package > argument" API because it has fewer parts. Is there a reason why we > can't just fix pkgutil? It is probably much easier to do it in > distlib. Sorry, I didn't get what you meant in the above. P

Re: [Distutils] distlib updated - comments sought

2012-10-05 Thread Tarek Ziadé
On 10/5/12 12:28 AM, Vinay Sajip wrote: I've updated distlib as follows: 1. I've added support for the equivalent of "pkg_resources.resource_filename" for returning a true filesystem path for a resource. I've added info about this in the "Distlib's design" document, though the reference

Re: [Distutils] distlib updated - comments sought

2012-10-04 Thread Daniel Holth
On Thu, Oct 4, 2012 at 6:28 PM, Vinay Sajip wrote: > I've updated distlib as follows: > > 1. I've added support for the equivalent of "pkg_resources.resource_filename" >for returning a true filesystem path for a resource. I've added info about >this in the "Distlib's design" document, thou

[Distutils] distlib updated - comments sought

2012-10-04 Thread Vinay Sajip
I've updated distlib as follows: 1. I've added support for the equivalent of "pkg_resources.resource_filename" for returning a true filesystem path for a resource. I've added info about this in the "Distlib's design" document, though the reference API has not been fully updated - I though