Re: [Python-Dev] PEP 376 and PEP 302 - allowing import hooks to provide distribution metadata
2009/7/6 P.J. Eby : > At 08:43 PM 7/5/2009 +0200, Tarek Ziadé wrote: >> >> But if it's based on PEP 302 protocols and if the pkgutil code works >> with the sys.meta_path hook, >> setuptools could then provide its loader, based on its EggFormats and >> act as a provider without being broken. > > You misunderstand me. The whole point of putting .egg-info in distutils in > the first place was to enable setuptools to detect the presence of > disutils-installed packages. That's what's broken by changing the name. This implies that there is no possibility that setuptools will be changed to support the new standard. That's patently false. Whether you *want* to change setuptools in such a way is up to you. And it's worth a note in the PEP if this change is made, that it will require a change to setuptools if that package is still to recognise distutils-installed packages. Paul. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 376 and PEP 302 - allowing import hooks to provide distribution metadata
I'm -1 on changing the name. For better or worse setuptools is the elephant in the room w.r.t. package management and it would IMHO be better to stay compatible (even if the stdlib only implements a subset of setuptools/pkg_resources) Ronald On 5 jul 2009, at 23:10, Nick Coghlan wrote: Tarek Ziadé wrote: 2009/7/5 P.J. Eby : If you want a clearer name, '.pydist' or some such would at least be reasonably specific. (It'd still have a backward compatibility problem, but at least then there'd be some benefit to the name change.) I do find "DIST_NAME.metadata" well-named and specific. But I guess that's just bikeshedding :) pydist has the advantage of both being more intuitive than 'egginfo' while still making it clear that this is Python related metadata rather than, say, something added by an OS packaging utility. So no, I don't think it's bikeshedding: 'metadata': accurate but generic. Not clear that it relates to Python specifically (except that it happens to stored in a Python related directory) 'egginfo': accurate, specific and serves as a good mnemonic. However, use of the 'egg' jargon means that someone is unlikely to guess what it means without being told, and it's less obvious that this is Python related 'pydist': accurate, specific and without the disadvantages of the 'egg' jargon PJE points out that existing tools (setuptools, pip, etc) won't be compatible with the new format at all if it uses a new name, but I am having trouble seeing that as a *bad* thing. By using a new name for the directory we *guarantee* that old packaging utilities won't get confused by the new format (they simply won't acknowledge its existence). So +1 for pydist as the directory extension in PEP 376 (for the above reasons). Cheers, Nick. -- Nick Coghlan | [email protected] | Brisbane, Australia --- ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/ronaldoussoren%40mac.com ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 376 and PEP 302 - allowing import hooks to provide distribution metadata
2009/7/6 Paul Moore : > 2009/7/6 P.J. Eby : >> At 08:43 PM 7/5/2009 +0200, Tarek Ziadé wrote: >>> >>> But if it's based on PEP 302 protocols and if the pkgutil code works >>> with the sys.meta_path hook, >>> setuptools could then provide its loader, based on its EggFormats and >>> act as a provider without being broken. >> >> You misunderstand me. The whole point of putting .egg-info in distutils in >> the first place was to enable setuptools to detect the presence of >> disutils-installed packages. That's what's broken by changing the name. > Forget about my previous mail, I didn't see that answer, > This implies that there is no possibility that setuptools will be > changed to support the new standard. That's patently false. Whether > you *want* to change setuptools in such a way is up to you. And it's > worth a note in the PEP if this change is made, that it will require a > change to setuptools if that package is still to recognise > distutils-installed packages. +1 setuptools is built on the top of distutils, not the contrary. And if setuptools wants to query installed distribution, it will have to be changed to use the query functions added in pkgutil. Tarek ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 376 and PEP 302 - allowing import hooks to provide distribution metadata
2009/7/6 Ronald Oussoren : > I'm -1 on changing the name. For better or worse setuptools is the elephant > in the room w.r.t. package management and it would IMHO be better to stay > compatible (even if the stdlib only implements a subset of > setuptools/pkg_resources) > I'd rather see the elephant evolves. I don't see why we should bend a standard we want to introduce in the stdlib, for a third-party package that is able to evolve to stick to a new standard without any problem. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 376 and PEP 302 - allowing import hooks to provide distribution metadata
2009/7/6 Tarek Ziadé : > 2009/7/6 Ronald Oussoren : >> I'm -1 on changing the name. For better or worse setuptools is the elephant >> in the room w.r.t. package management and it would IMHO be better to stay >> compatible (even if the stdlib only implements a subset of >> setuptools/pkg_resources) >> > > I'd rather see the elephant evolves. > > I don't see why we should bend a standard we want to introduce in the stdlib, > for a third-party package that is able to evolve to stick to a new standard > without any problem. I agree with this. The big problem is that setuptools development seems to have essentially stagnated, and while that shouldn't be something we care about, it does need to be considered. Although my instinct is to fight against "do it our way because we won't change to do it your way" arguments, I have to ask whether the (purely internal) name of the metadata directory is really worth fighting over. I'm +0 on changing the name, as long as it's the *only* "do it this way because setuptools isn't going to change" issue. If more such issues come up, I'll become +1 on a clean break, to avoid endless such arguments and restrictions. Paul. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 376 and PEP 302 - allowing import hooks to provide distribution metadata
P.J. Eby wrote: > At 08:43 PM 7/5/2009 +0200, Tarek Ziadé wrote: >> But if it's based on PEP 302 protocols and if the pkgutil code works >> with the sys.meta_path hook, >> setuptools could then provide its loader, based on its EggFormats and >> act as a provider without being broken. > > You misunderstand me. The whole point of putting .egg-info in distutils > in the first place was to enable setuptools to detect the presence of > disutils-installed packages. That's what's broken by changing the name. How much information does setuptools actually need in order to tell that a distribution is already present? Presumably the existing .egginfo files generated by distutils are sufficient for that task? If so, I'd still prefer to keep the new metadata safely out of reach of the legacy package management systems that don't understand it, while having distutils retain the ability to generate a legacy ".egginfo" file to warn off the existing package management tools from installing the same distribution again. So instead of ".egginfo becomes a directory" as in the current version of PEP 376, the existing distutils .egginfo file generation would instead remain unchanged with a ".pydist" directory being added alongside to hold all the new goodies. That way, the legacy formats (EGG-INFO, .egg-info file, .egg-info directory) continue to play as well together as they do now, while the new distutils system will understand .pydist directories natively and allow the injection of hook implementations by the third party package management tools in order to understand the old formats. That seems like it will give us the best of both worlds - a clean base to work from in the future, but without breaking the existing interoperability mechanisms (at least in the short term - maybe by the time 3.3 rolls around we will be able to start thinking about deprecating the legacy .egginfo formats, but even considering that question is a long way off). Cheers, Nick. -- Nick Coghlan | [email protected] | Brisbane, Australia --- ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Mercurial migration: progress report (PEP 385)
Dirkjan Ochtman wrote: > In response to some rumblings on python-committers and just to request > more feedback, a progress report. I know it's long, I've tried to put > to keep it concise and chunked, though. Two things: * We need some form of documentation of how committers are expected to work with the hg repo. (This is also missing for the Subversion repo, which due to the 3.x branch has gotten somewhat unclear - at least for me) It is currently not clear where to check in patches, whether and where to backport or forward-patch, which branches to consider closed, etc. E.g. if I check in something in trunk/ (Python 2.7), do I have to forward patch this change to the 3.0 branch (guess not), the py3k/ branch (Python 3.1), or will someone else take care of this, so that it's better not to interfere by doing it myself ? * Our tracker, checkin messages, online documentation and lots of Python resources on the web are full of references to the Subversion rXYZ notation of changesets. The PEP has to provide some way to gracefully provide a redirect from those URLs to the new ones (both for the source code browser and the bug search tool on python.org). The PEP will also have to address the same issue for checkin messages. Perhaps by converting the rXYZ notation to the hash based values during migration or by adding the has value after the rXYZ value in parens ?! -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Jul 06 2009) >>> Python/Zope Consulting and Support ...http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/ ::: Try our new mxODBC.Connect Python Database Interface for free ! eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 376 and PEP 302 - allowing import hooks to provide distribution metadata
2009/7/6 Nick Coghlan : > P.J. Eby wrote: >> At 08:43 PM 7/5/2009 +0200, Tarek Ziadé wrote: >>> But if it's based on PEP 302 protocols and if the pkgutil code works >>> with the sys.meta_path hook, >>> setuptools could then provide its loader, based on its EggFormats and >>> act as a provider without being broken. >> >> You misunderstand me. The whole point of putting .egg-info in distutils >> in the first place was to enable setuptools to detect the presence of >> disutils-installed packages. That's what's broken by changing the name. This is getting confusing. Is Phillip saying that setuptools will cope with the file changing to a directory without modification, but it can't handle a change in the name? My site-packages has a confusing mix of egginfo directories and files. Note that I NEVER use setuptools other than where an existing package's setup.py requires it. In that case, I still only do python setup.py bdist_wininst and install the generated installer. So is PEP 376 going to be able to cope with the stuff I have installed at the moment? If not, what's the point??? Example: I have the "coverage" package installed. It does not use setuptools at runtime - I could happily install it from a bdist_wininst on a machine without setuptools installed, and it would run fine. (In fact, that's what I did!) It has created a coverage-3.0-py2.6.egg-info *directory* in my site-packages, with no RECORD file - presumably, this is in a setuptools format. If setuptools is not going to change to conform to PEP 376, then any tools built using PEP 376 will fail to recognise my coverage install. Which, as far as I am concerned, is a completely standard bdist_wininst installer. In fact, the above strongly suggests to me that PEP 376 must NOT follow setuptools conventions - otherwise, it will end up clashing with the files setuptools is putting on my system. > How much information does setuptools actually need in order to tell that > a distribution is already present? Presumably the existing .egginfo > files generated by distutils are sufficient for that task? It appears so, but setuptools doesn't use (or create!!!) those files in its own installer formats. > If so, I'd still prefer to keep the new metadata safely out of reach of > the legacy package management systems that don't understand it, while > having distutils retain the ability to generate a legacy ".egginfo" file > to warn off the existing package management tools from installing the > same distribution again. So for a minimal case of a single .py file packaged up as a distribution, we now have the .py file, a legacy .egginfo file, a new .pydist directory containing RECORD and PKG-INFO files? That's getting silly. > > So instead of ".egginfo becomes a directory" as in the current version > of PEP 376, the existing distutils .egginfo file generation would > instead remain unchanged with a ".pydist" directory being added > alongside to hold all the new goodies. That way, the legacy formats > (EGG-INFO, .egg-info file, .egg-info directory) continue to play as well > together as they do now, while the new distutils system will understand > .pydist directories natively and allow the injection of hook > implementations by the third party package management tools in order to > understand the old formats. > > That seems like it will give us the best of both worlds - a clean base > to work from in the future, but without breaking the existing > interoperability mechanisms (at least in the short term - maybe by the > time 3.3 rolls around we will be able to start thinking about > deprecating the legacy .egginfo formats, but even considering that > question is a long way off). And yet, given that PEP 376 is explicitly being designed with a goal being to act as the common standard that *all* package management formats can use, is it not the whole point that once it's defined and we have achieved consensus, existing package managers will switch to using it rather than retaining a range of custom formats? I honestly fail to see why we are catering to this "setuptools cannot change" view. Usually, it's the standard library that cannot change, because of the longer release cycle of the core. If a standard is agreed, and setuptools won't conform to it, maybe it's time for someone to fork setuptools (unilaterally, if Phillip can't agree to an amicable transfer). Otherwise setuptools will remain a roadblock for any development of distutils. Frustrated-ly y'rs Paul. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 376 and PEP 302 - allowing import hooks to provide distribution metadata
On Mon, Jul 6, 2009 at 5:14 PM, Paul Moore wrote: > [...] > And yet, given that PEP 376 is explicitly being designed with a goal > being to act as the common standard that *all* package management > formats can use, is it not the whole point that once it's defined and > we have achieved consensus, existing package managers will switch to > using it rather than retaining a range of custom formats? That's the goal from the very beginning. > > I honestly fail to see why we are catering to this "setuptools cannot > change" view. Usually, it's the standard library that cannot change, > because of the longer release cycle of the core. If a standard is > agreed, and setuptools won't conform to it, maybe it's time for > someone to fork setuptools (unilaterally, if Phillip can't agree to an > amicable transfer). Otherwise setuptools will remain a roadblock for > any development of distutils. > > Frustrated-ly y'rs +1 why can't we just go ahead and continue the work as we started with PEP 376, introducing your work on PEP 302-like behavior. Then if we get a consensus on this PEP and introduce it in 2.7/3.2, setuptools will have to follow this consensus. FWIW I am also frustrated because the setuptools development has been stopped 9 months ago, and *doesn't work anymore with the current distutils trunk*. There are a lot of patches waiting in the setuptools tracker people wrote. I hate to say this but this project needs an active maintainer or it's going to die. So I've said in distutils-sig that I would start my own branch of setuptools before 2.7/3.2 is out for the people in the community that relies on this package. Not to maintain the project because I have a lot to do already on distutils side. But at least to make sure everyone can work with the upcoming releases of Python 2.x and 3.X without having their setup.py broken (and switch eventually to a plain distutils solution maybe) Tarek ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 376 and PEP 302 - allowing import hooks to provide distribution metadata
2009/7/6 Tarek Ziadé : > why can't we just go ahead and continue the work as we started with PEP 376, > introducing your work on PEP 302-like behavior. > > Then if we get a consensus on this PEP and introduce it in 2.7/3.2, > setuptools will have to follow this consensus. Essentially, because when I ask questions, responses along the lines of "you have to do it like X because setuptools does that" come back, and (not being a setuptools user) I can't tell whether there's a valid reason in there. I'm uncomfortable assuming that setuptools experience is irrelevant, but I can't distinguish between valid arguments and "setuptools can't change" arguments. I need to write another email with a list of outstanding open issues. If we can thrash them out *without* getting bogged down in setuptools compatibility questions, then I think we can move forward again. I'll do that this evening. Paul. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 376 and PEP 302 - allowing import hooks to provide distribution metadata
On Mon, Jul 6, 2009 at 5:53 PM, Paul Moore wrote: > 2009/7/6 Tarek Ziadé : >> why can't we just go ahead and continue the work as we started with PEP 376, >> introducing your work on PEP 302-like behavior. >> >> Then if we get a consensus on this PEP and introduce it in 2.7/3.2, >> setuptools will have to follow this consensus. > > Essentially, because when I ask questions, responses along the lines > of "you have to do it like X because setuptools does that" come back, > and (not being a setuptools user) I can't tell whether there's a valid > reason in there. Notice that we created PEP 376 in distutils-SIG with most of the valid reasons/use cases setuptools had, with the help of Phillip, before I brought it up again on python-dev. > > I'm uncomfortable assuming that setuptools experience is irrelevant, > but I can't distinguish between valid arguments and "setuptools can't > change" arguments. > > I need to write another email with a list of outstanding open issues. > If we can thrash them out *without* getting bogged down in setuptools > compatibility questions, then I think we can move forward again. I'll > do that this evening. Ok, I'll wait for your work to work on the PEP again then, and your push in the hg repo as well. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 376 and PEP 302 - allowing import hooks to provide distribution metadata
At 10:32 AM 7/6/2009 +0100, Paul Moore wrote: I'm +0 on changing the name, as long as it's the *only* "do it this way because setuptools isn't going to change" issue. Please note that I never said that. I was the one who suggested ".pydist", remember? I just don't want to have to complicate the pkg_resources directory scan to support a stupid name like "metadata", no offense to Tarek. ;-) I'd sooner not make the change at all, just because I doubt I'm the only person who has code that depends on the old name. (Consider the Linux OS packagers who treat .egg-info files specially at the moment.) ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 376 and PEP 302 - allowing import hooks to provide distribution metadata
At 04:14 PM 7/6/2009 +0100, Paul Moore wrote: 2009/7/6 Nick Coghlan : > P.J. Eby wrote: >> At 08:43 PM 7/5/2009 +0200, Tarek Ziadé wrote: >>> But if it's based on PEP 302 protocols and if the pkgutil code works >>> with the sys.meta_path hook, >>> setuptools could then provide its loader, based on its EggFormats and >>> act as a provider without being broken. >> >> You misunderstand me. The whole point of putting .egg-info in distutils >> in the first place was to enable setuptools to detect the presence of >> disutils-installed packages. That's what's broken by changing the name. This is getting confusing. Is Phillip saying that setuptools will cope with the file changing to a directory without modification, but it can't handle a change in the name? The existing versions of setuptools will read a file or a directory with no problem; it's the name change that will require a code change, and it's a rather more complex issue than just one name change, because it'll need to support both names. What's more, on the build/install side, it'll have to figure out whether to use the new name or the old name when creating a project's metadata for installation in single-version mode. In other words, this will likely affect pip as well, or at least the parts of setuptools that pip uses. My site-packages has a confusing mix of egginfo directories and files. Note that I NEVER use setuptools other than where an existing package's setup.py requires it. In that case, I still only do python setup.py bdist_wininst and install the generated installer. So is PEP 376 going to be able to cope with the stuff I have installed at the moment? If not, what's the point??? If I understand Tarek's proposal correctly, then no, it will not cope. If setuptools is not going to change to conform to PEP 376, then any tools built using PEP 376 will fail to recognise my coverage install. I'm all in favor of adding RECORD support to setuptools; it was in fact my idea to have the file there in the first place. Adding a RECORD file doesn't introduce any new and weird name migration requirements, which is why I'd rather not change the extension if we can avoid it. Reading both names is painful, writing both is more so, and I'm not sure how many tools/users *besides* setuptools will be affected by a name change. > How much information does setuptools actually need in order to tell that > a distribution is already present? Presumably the existing .egginfo > files generated by distutils are sufficient for that task? It appears so, but setuptools doesn't use (or create!!!) those files in its own installer formats. Setuptools treats an .egg-info file as if it were a PKG-INFO file contained in an .egg-info directory. This allows it to treat distutils-supplied .egg-info files as if they were setuptools-supplied .egg-info directories containing exactly one file. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Any updates on this subprocess/signal bug/issue (Re: subprocess and EINTR errnos)
On Wed, 17 Nov 2004, Peter Astrand wrote, > I've made a patch (attached) to subprocess.py (and test_subprocess.py) > that should guard against EINTR, but I haven't committed it yet. It's > quite large. > > Are Python modules supposed to handle EINTR? Why not let the C code handle > this? Or, perhaps the signal module should provide a sigaction function, > so that users can use SA_RESTART. This is a snippet from a email sent in 2004, I was wondering if there was any update on this issue. Are these issues supposed to be handled on a per application basis, or will a fix go into Python in the near future? -- Hatem Nassrat Refrences: http://mail.python.org/pipermail/python-dev/2004-November/049983.html ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 376 and PEP 302 - allowing import hooks to provide distribution metadata
On Mon, Jul 6, 2009 at 6:58 PM, P.J. Eby wrote: >> My site-packages has a confusing mix of egginfo directories and files. >> Note that I NEVER use setuptools other than where an existing >> package's setup.py requires it. In that case, I still only do python >> setup.py bdist_wininst and install the generated installer. If you end up with confusing mix of egginfo directories and files, that's because for the packages that uses setuptools in their setup.py, it patches distutils and to creates its own egg-info format. >> So is PEP 376 going to be able to cope with the stuff I have installed >> at the moment? If not, what's the point??? > > If I understand Tarek's proposal correctly, then no, it will not cope. Why that ? Can you detail ? On a system that uses only plain distutils distributions, it'll work. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 376 and PEP 302 - allowing import hooks to provide distribution metadata
At 07:18 PM 7/6/2009 +0200, Tarek Ziadé wrote: >> So is PEP 376 going to be able to cope with the stuff I have installed >> at the moment? If not, what's the point??? > > If I understand Tarek's proposal correctly, then no, it will not cope. Why that ? Can you detail ? On a system that uses only plain distutils distributions, it'll work. Paul's question, as I understood it, was whether pkgutil would handle the existing .egg-info's. Your proposal, as I understood it, was that pkgutil would not detect .egg-info's without a plugin of some sort... and you've not proposed a way of loading such a thing, which means that to use pkgutil with that plugin, an application would have to know that the plugin exists and import it. Unless you're planning to have a plugin registration facility (ala entry points) so that plugins can be used without each application needing to discover them on its own? ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 376 and PEP 302 - allowing import hooks to provide distribution metadata
2009/7/6 P.J. Eby : > At 04:14 PM 7/6/2009 +0100, Paul Moore wrote: >> This is getting confusing. Is Phillip saying that setuptools will cope >> with the file changing to a directory without modification, but it >> can't handle a change in the name? > > The existing versions of setuptools will read a file or a directory with no > problem; it's the name change that will require a code change, and it's a > rather more complex issue than just one name change, because it'll need to > support both names. If PEP 376 is the standard, setuptools can just use that. Whether you have to handle backward compatibility issues so that newer versions of setuptools work with older metadata layouts is largely an internal setuptools issue. While I don't want to gratuitously break setuptools, it already supports something like 3 formats, as I understand it, so adding a 4th shouldn't be the end of the world. > What's more, on the build/install side, it'll have to figure out whether to > use the new name or the old name when creating a project's metadata for > installation in single-version mode. In other words, this will likely > affect pip as well, or at least the parts of setuptools that pip uses. And yet, there have been no adverse comments from users of pip, or buildout, or any of the other tools that this PEP is targeted at. So either there's a huge chunk of the affected community that's unaware of this discussion (which I doubt), or they have no problem with this, or their interaction is via setuptools, and so as long as setuptools is changed, they are fine. (The other benefit of the PEP is that if setuptools *doesn't* change, they have alternative tools to build their own interface, although they may not want to be forced to do this...) >> If setuptools is not going to change to conform to PEP 376, then any >> tools built using PEP 376 will fail to recognise my coverage install. > > I'm all in favor of adding RECORD support to setuptools; it was in fact my > idea to have the file there in the first place. > > Adding a RECORD file doesn't introduce any new and weird name migration > requirements, which is why I'd rather not change the extension if we can > avoid it. You don't give any reasons why you pick and choose what changes you're happy with - "weird name migration requirements" isn't very specific. > Reading both names is painful, writing both is more so, and I'm not sure how > many tools/users *besides* setuptools will be affected by a name change. Don't forget - I made that point when I said I wasn't too bothered about what was an internal name change. >> > How much information does setuptools actually need in order to tell that >> > a distribution is already present? Presumably the existing .egginfo >> > files generated by distutils are sufficient for that task? >> >> It appears so, but setuptools doesn't use (or create!!!) those files >> in its own installer formats. > > Setuptools treats an .egg-info file as if it were a PKG-INFO file contained > in an .egg-info directory. This allows it to treat distutils-supplied > .egg-info files as if they were setuptools-supplied .egg-info directories > containing exactly one file. Hey, look - you get to *remove* some compatibility code! Paul. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Mercurial migration: progress report (PEP 385)
On Mon, Jul 6, 2009 at 07:51, M.-A. Lemburg wrote: > Dirkjan Ochtman wrote: > > In response to some rumblings on python-committers and just to request > > more feedback, a progress report. I know it's long, I've tried to put > > to keep it concise and chunked, though. > > Two things: > > * We need some form of documentation of how committers are expected > to work with the hg repo. (This is also missing for the Subversion > repo, which due to the 3.x branch has gotten somewhat unclear - at > least for me) > I am planning to get a version of the dev FAQ written up that covers most of what it already does now for svn. > > It is currently not clear where to check in patches, whether and > where to backport or forward-patch, which branches to consider > closed, etc. > > E.g. if I check in something in trunk/ (Python 2.7), do I have to > forward patch this change to the 3.0 branch (guess not), the py3k/ > branch (Python 3.1), or will someone else take care of this, so that > it's better not to interfere by doing it myself ? > This question is partially answered by http://www.python.org/dev/faq/#how-do-i-merge-between-branches, but I agree that we should have either this spelled out in the FAQ or a committer-specific doc at www.python.org/dev/ that makes this all very obvious. -Brett ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 376 and PEP 302 - allowing import hooks to provide distribution metadata
2009/7/6 P.J. Eby : > At 07:18 PM 7/6/2009 +0200, Tarek Ziadé wrote: >> >> >> So is PEP 376 going to be able to cope with the stuff I have installed >> >> at the moment? If not, what's the point??? >> > >> > If I understand Tarek's proposal correctly, then no, it will not cope. >> >> Why that ? Can you detail ? On a system that uses only plain distutils >> distributions, it'll work. > > Paul's question, as I understood it, was whether pkgutil would handle the > existing .egg-info's. Your proposal, as I understood it, was that pkgutil > would not detect .egg-info's without a plugin of some sort... and you've > not proposed a way of loading such a thing, which means that to use pkgutil > with that plugin, an application would have to know that the plugin exists > and import it. Close. My point was that distributions which use setuptools in their setup.py, *even if there's no runtime dependency on setuptools* end up with non-standard .egg-info's. There is no good reason for this, from my POV as a package user. So if setuptools is breaking otherwise entirely standard packages, so that they won't work with core distutils mechanisms, that's wrong. Setuptools is refusing to play nicely with other packaging systems. Now at the moment, this is a moot point, because the stdlib doesn't have any package discovery mechanisms. But when they *do* appear, I expect setuptools to work with them. So, when PEP 376 is accepted and implemented, I contend that, in order to be a well-behaved packaging tool, setuptools must: - Generate PEP 376 compatible metadata - Recognise PEP 376 compatible metadata - Unless installing a custom package format (such as eggs), which has its own PEP 302 installer, refrain from installing hooks that alter the way the standard distutils commands handle metadata. That's my view as an end user of Python packages. It means that if I choose not to use setuptools/easy_install to manage my packages, I shouldn't need them so that distutils or other core utilities (such as bdist_wininst) can do so. > Unless you're planning to have a plugin registration facility (ala entry > points) so that plugins can be used without each application needing to > discover them on its own? My understanding was that package management tools provide their own end-user level install and uninstall features (the PEP 376 uninstall is just for filesystems, to correspond to the distutils install command). PEP 376 is just standardising the format of metadata, and the APIs to allow read access. The PEP 302 hooks I'm writing provide the "plugin" ability for alternative storage formats. But there's no provision for alternative *metadata* formats. What's the point of a standard if people don't follow it? Paul. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] PEP 376 - Open questions
As promised, here are some open questions on PEP 376. - Will the public API names be changed from *egginfo* to *metadata*? - What precisely are the use cases for absolute path names? Concrete examples are needed. With the current spec, some things can go wrong (e.g., see below), so we need real use cases to know how to address this. - How will bdist_wininst/bdist_msi/bdist_rpm be updated? - How will the RECORD file be managed? (Particularly for the case of bdist_xxx) [1] - Can distutils be made to install files in places the current RECORD file spec can't handle? (I think the answer is "yes"). What happens then? - Should distribution names be case insensitive on case insensitive filesystems? For comparison, module/package names are always case sensitive even on case insensitive systems. - What will happen with the md5 hash? Are more types of hash going to be supported? What's the default? (Actually, the PEP doesn't need to care about the default, as the PEP says nothing about how RECORD files are written). [1] Note - the idea of using $EXEC_PREFIX / $PREFIX implies that the RECORD file is intended to be relocatable. Which is worrying, and also implies that an individual Distribution class must be able to handle filesystem files as well as whatever else it handles (consider mylib.zip on sys.path, containing a distribution which installed some files in $PREFIX). If this isn't possible, it should be clearly stated that it isn't possible. If it is, the ramifications are complex... I'm still unsure how the local vs relative, slash-separated filename formats should be handled. I don't actually think there's any real benefit in having 2 formats. I propose: - get_installed_files(), uses(), get_file_users() - always use local format absolute pathnames (for zipfiles and the like, these may not be "real" filenames, but they will be in "real filename" format, so other code will be able to manipulate them as filenames). - get_egginfo_files, get_egginfo_file - always use slash-separated forms, relative to the egginfo directory (so the name of the RECORD file is just 'RECORD') But there could be uses I haven't thought about, so this still counts as an open question at the moment (i.e., I'm reluctant to implement things this way until I've had some feedback). Paul. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Any updates on this subprocess/signal bug/issue (Re: subprocess and EINTR errnos)
On Wed, 17 Nov 2004, Peter Astrand wrote, > I've made a patch (attached) to subprocess.py (and test_subprocess.py) > that should guard against EINTR, but I haven't committed it yet. It's > quite large. > > Are Python modules supposed to handle EINTR? Why not let the C code handle > this? Or, perhaps the signal module should provide a sigaction function, > so that users can use SA_RESTART. This is a snippet from a email sent in 2004, I was wondering if there was any update on this issue. Are these issues supposed to be handled on a per application basis, or will a fix go into Python in the near future? -- Hatem Nassrat Refrences: http://mail.python.org/pipermail/python-dev/2004-November/049983.html ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] semi-regular check that all committers are subscribed to python-committers
Since no one objected, I will give Dirkjan commit privs so he can work on the hg migration. Now I just need you, Dirkjan, to get a contributor agreement sent in and then send python-dev your SSH2 key and we will get you set up. On Thu, Jul 2, 2009 at 13:08, Brett Cannon wrote: > > > On Thu, Jul 2, 2009 at 12:56, Dirkjan Ochtman wrote: > >> On Thu, Jul 2, 2009 at 21:04, Brett Cannon wrote: >> > If you have commit privileges on Python but are not subscribed to >> > python-committers, please let me know and I will subscribe you. >> Membership >> > on python-committers is mandatory as that's where we announce branch >> > freezes, etc. >> >> Would it be helpful if I was subscribed? I'm not strictly a committer, >> but what with the Mercurial migration I imagine it won't be long >> before the first cset bearing my name appears in the Python repository >> (plus I just saw there was some discussion about the hg migration >> there which I could weigh in on). > > > Maybe you should just get the commit privileges now. Any objections? > > -Brett > ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Any updates on this subprocess/signal bug/issue (Re: subprocess and EINTR errnos)
On Mon, Jul 06, 2009, Hatem Nassrat wrote: > On Wed, 17 Nov 2004, Peter Astrand wrote, >> >> I've made a patch (attached) to subprocess.py (and test_subprocess.py) >> that should guard against EINTR, but I haven't committed it yet. It's >> quite large. >> >> Are Python modules supposed to handle EINTR? Why not let the C code handle >> this? Or, perhaps the signal module should provide a sigaction function, >> so that users can use SA_RESTART. > > This is a snippet from a email sent in 2004, I was wondering if there > was any update on this issue. Are these issues supposed to be handled on > a per application basis, or will a fix go into Python in the near > future? For starters, if there is to be any progress, there needs to be an open issue on bugs.python.org -- have you searched to see if one already exists and created one if it doesn't? -- Aahz ([email protected]) <*> http://www.pythoncraft.com/ "as long as we like the same operating system, things are cool." --piranha ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 376 and PEP 302 - allowing import hooks to provide distribution metadata
At 07:15 PM 7/6/2009 +0100, Paul Moore wrote: My point was that distributions which use setuptools in their setup.py, *even if there's no runtime dependency on setuptools* end up with non-standard .egg-info's. There is no good reason for this, from my POV as a package user. So if setuptools is breaking otherwise entirely standard packages, so that they won't work with core distutils mechanisms, that's wrong. Setuptools is refusing to play nicely with other packaging systems. Now at the moment, this is a moot point, because the stdlib doesn't have any package discovery mechanisms. Not so; it has .egg-info. (See footnote below.) The PEP 302 hooks I'm writing provide the "plugin" ability for alternative storage formats. But there's no provision for alternative *metadata* formats. What's the point of a standard if people don't follow it? I'm confused. Are you saying pkgutil will recognize .egg-info, or not? The only practical difference between the existing formats and the proposed new one (AFAICR) is the extension of the directory name, and the presence or absence of the RECORD file. My POV is that it would be a shame to discard compatibility with existing installed software, just for a bikeshedding of the directory name. Especially if it means that any tools that currently generate .egg-info's (and I'm not 100% positive setuptools is the only case) will need to grow options for whether to generate the old name, possibly based on what Python version... except you can install new distutils into old Python... etc. If this were strictly a matter of, "from Python 3.x on, generate .pydist", it's a lot less of a problem for me and everybody else, because Python 3 is effectively going to force a fork (or at least a branch) of setuptools anyway. For Python 2.x, having *two* names for .egg-info (which is what I understand the proposal to effectively be) is just asking for trouble, IMO. And my concern is that currently these compatibility issues are being handwaved rather than considered up front. To reiterate, summarize, and attempt to make as concrete as I can: I'm fine with a rename, *if and only if* it can be done in a way that never requires a tool to generate BOTH files for compatibility reasons. AFAICT, this can only be done by making the extension dependent upon the Python version, and this version distinction must be made *even if* the distutils themselves are backported to an earlier Python version. (Alternatively, this could be resolved by *not* backporting the PEP 376-compatible distutils/pkgutil to 2.x.) Currently, there is no official Python 3 release of setuptools, so compatibility can (IMO) be considered moot there. What's more, it is easy to conditionally determine whether to use '.egg-info' or '.pydist' based on Python version. What is not easy and likely to be confusing for everyone, is having the possibility for *both* to exist in the same system, *without* also having bidirectional interoperability. So, I'm +1 for no change (obviously), and +0 for "always .pydist in 3.x and always .egg-info in 2.x", whether the latter part is achieved by making distutils/pkgutil use a version-dependent extension, or by refusing to backport distutils/pkgutil to 2.x. I'm -1 for having .pydist ever appear on a 2.x Python, either via stdlib or backporting. (These comments also apply for any name other than '.pydist'.) Footnote: there has been some discussion that implies that somehow .egg-info was hijacked into the stdlib, and that it should therefore not be supported. This is incorrect: at Pycon the year this happened (2005 IIRC), Guido *blessed setuptools for inclusion in the stdlib* in his keynote. Yes, *all* of setuptools, including easy_install, eggs, the whole nine yards. A few months later, when I actually checked in the code, there were protests from a few people on Python-Dev, some of which I agreed were valid. I reverted the checkin, floating instead a compromise proposal to add .egg-info support to distutils (i.e. the install_egg_info command) to allow setuptools and other 3rd party tools to detect the presence of distutils-installed packages. At the time, the assumption was that eggs and setuptools would still, eventually, make it into the stdlib -- at the time, Guido had not yet changed his mind about that. So the ".egg-info" name was not controversial, to the best of my recollection, and the inclusion did not violate any of the expressed concerns of the three persons who objected to including setuptools whole-hog. So, from my POV at least, dropping .egg-info is dropping backward compatibility on a feature that was *intentionally* added to the stdlib -- with approval from Guido and non-protest from Python-Dev -- specifically to provide that compatibility. That doesn't obligate it to stay... but I suggest that it *does* mean Python 3, not 2, is the appropriate place for dropping it (if it is indeed to be dropp
Re: [Python-Dev] PEP 376 - Open questions
At 07:38 PM 7/6/2009 +0100, Paul Moore wrote: As promised, here are some open questions on PEP 376. - Will the public API names be changed from *egginfo* to *metadata*? +1 (FWIW, 'metadata' is what pkg_resources API refers to this kind of stuff as.) - What precisely are the use cases for absolute path names? Concrete examples are needed. With the current spec, some things can go wrong (e.g., see below), so we need real use cases to know how to address this. The purpose is to be able to remove scripts, data files, documentation, etc. that's stored in some place that's not package-relative, and survive a move of the package directory itself. For example, suppose I install with --prefix=~, and have files in ~/bin and ~/lib. I then move ~/lib/python2.7 to ~/pylib. If the scripts were installed with a relative path, this would break the link to the script still living in ~/bin. Now that I'm thinking about this, it occurs to me that there is another way to address this besides mixing relative and absolute paths in RECORD, and that is to make *all* the paths relative, and include an original installation location in INSTALLER. Then, it would be possible to detect whether the installation location had been moved and handle it accordingly... for some definition of "accordingly". I agree that further discussion is probably needed to resolve this. - How will bdist_wininst/bdist_msi/bdist_rpm be updated? bdist_wininst, bdist_dumb, and various others use 'install --root' pattern to generate files for installation, which means that they would transparently end up writing a correct RECORD file, except for the inclusion of incorrect absolute paths for non-libdir-relative files. However, if we used the "all relative in RECORD, with a base in INSTALLER", these cases could transparently be treated as another instance of install directory relocation. I don't know if bdist_msi does a --root install before generating the .msi; if it does, then it should work the same way. - Can distutils be made to install files in places the current RECORD file spec can't handle? (I think the answer is "yes"). What happens then? Surely an absolute path is sufficient? - Should distribution names be case insensitive on case insensitive filesystems? For comparison, module/package names are always case sensitive even on case insensitive systems. IMO, they should be case-insensitive *regardless* of filesystem case, and I believe PyPI has already begun disallowing packages with only case-distinct names (e.g. 'Zope' vs. 'ZoPe') to be registered. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Mercurial migration: progress report (PEP 385)
M.-A. Lemburg wrote: > Dirkjan Ochtman wrote: > * Our tracker, checkin messages, online documentation and lots of >Python resources on the web are full of references to the >Subversion rXYZ notation of changesets. > >The PEP has to provide some way to gracefully provide a redirect >from those URLs to the new ones (both for the source code browser >and the bug search tool on python.org). > >The PEP will also have to address the same issue for checkin >messages. Perhaps by converting the rXYZ notation to the hash >based values during migration or by adding the has value after >the rXYZ value in parens ?! I've asked this question before: Dirkjan indicated that he will be writing a web redirection filter that translates from the SVN rXYZ revision numbers to the Hg hash value that corresponds to that revision in the history conversion. I suggest that we also run a version of that redirection filter to remap the old svn.python.org links in addition to making them accessible through hg.python.org as Dirkjan proposed. Cheers, Nick. [1] http://mail.python.org/pipermail/python-dev/2009-July/090397.html -- Nick Coghlan | [email protected] | Brisbane, Australia --- ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 376 and PEP 302 - allowing import hooks to provide distribution metadata
P.J. Eby wrote: > So, I'm +1 for no change (obviously), and +0 for "always .pydist in 3.x > and always .egg-info in 2.x", whether the latter part is achieved by > making distutils/pkgutil use a version-dependent extension, or by > refusing to backport distutils/pkgutil to 2.x. I'm -1 for having > .pydist ever appear on a 2.x Python, either via stdlib or backporting. > > (These comments also apply for any name other than '.pydist'.) Thanks for the explanation - Before this message I didn't fully grasp *how* installing a directory containing files that the other packages might not understand would help with the backwards compatibility problem. Now I understand that aspect better (i.e. it isn't a matter of "setuptools won't change" but "the user might already have an old version of setuptools/pip/etc on their system, or be installing a package that uses an old version of one of those libraries for its installer"), I no longer think it is worth the hassle of trying to use a "prettier" name for the metadata directory. So +1 on keeping the same extension name and -1 on the idea of changing it to anything other than the existing ".egg-info". I believe the idea of having different names in 2.x and 3.x would likely cause too many problems for simple bdist_* installers of modules that use only the common 2.x/3.x language subset, so I'm also -1 on that idea. Cheers, Nick. -- Nick Coghlan | [email protected] | Brisbane, Australia --- ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 376 - Open questions
(I cancelled sending this the first time, so apologies if a half-written version turns up) Paul Moore wrote: > As promised, here are some open questions on PEP 376. I'd add one more question to the list: is allowing backslash separated names in the RECORD file actually a good idea, or would it be better to always use forward slashes? My reason for this question is what happens if (for example) a bdist_* installer is generated on Linux and then used on Windows or vice-versa? If the expected RECORD format is different on the two platforms, won't it get things wrong? For the other questions, I don't have anything much to add to PJE's comments, except that the "all relative" paths idea won't work due to the Windows drive letter issue (i.e. if an installer puts files in C:\Program Files, there is no guarantee that a relative path between site-packages and Program Files even exists if Python is installed on a different drive). Cheers, Nick. -- Nick Coghlan | [email protected] | Brisbane, Australia --- ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Mercurial migration: progress report (PEP 385)
> I suggest that we also run a version of that redirection filter to remap > the old svn.python.org links in addition to making them accessible > through hg.python.org as Dirkjan proposed. Not sure what links you are talking about, but we also need to keep the current svn.python.org up as-is, for all the stuff that won't be migrated. Regards, Martin ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Mercurial migration: progress report (PEP 385)
Martin v. Löwis wrote: >> I suggest that we also run a version of that redirection filter to remap >> the old svn.python.org links in addition to making them accessible >> through hg.python.org as Dirkjan proposed. > > Not sure what links you are talking about, but we also need to keep the > current svn.python.org up as-is, for all the stuff that won't be migrated. In that case, no need to redirect anything (although it will still be possible to look up the old SVN revisions in the Hg web view). The previous discussion was based on the assumption that it was going to be possible to switch the SVN server off at some point rather than just disallowing commits to the projects that had migrated to Mercurial. Cheers, Nick. -- Nick Coghlan | [email protected] | Brisbane, Australia --- ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 376 and PEP 302 - allowing import hooks to provide distribution metadata
2009/7/6 Nick Coghlan : > P.J. Eby wrote: >> So, I'm +1 for no change (obviously), and +0 for "always .pydist in 3.x >> and always .egg-info in 2.x", whether the latter part is achieved by >> making distutils/pkgutil use a version-dependent extension, or by >> refusing to backport distutils/pkgutil to 2.x. I'm -1 for having >> .pydist ever appear on a 2.x Python, either via stdlib or backporting. >> >> (These comments also apply for any name other than '.pydist'.) > > Thanks for the explanation - Before this message I didn't fully grasp > *how* installing a directory containing files that the other packages > might not understand would help with the backwards compatibility problem. > > Now I understand that aspect better (i.e. it isn't a matter of > "setuptools won't change" but "the user might already have an old > version of setuptools/pip/etc on their system, or be installing a > package that uses an old version of one of those libraries for its > installer"), I no longer think it is worth the hassle of trying to use a > "prettier" name for the metadata directory. So +1 on keeping the same > extension name and -1 on the idea of changing it to anything other than > the existing ".egg-info". > > I believe the idea of having different names in 2.x and 3.x would likely > cause too many problems for simple bdist_* installers of modules that > use only the common 2.x/3.x language subset, so I'm also -1 on that idea. That suits me too. I'm happy for the name to stay as it is. Paul. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 376 - Open questions
2009/7/6 P.J. Eby : > At 07:38 PM 7/6/2009 +0100, Paul Moore wrote: >> >> As promised, here are some open questions on PEP 376. >> >> - Will the public API names be changed from *egginfo* to *metadata*? > > +1 (FWIW, 'metadata' is what pkg_resources API refers to this kind of stuff > as.) Thanks. I think this one is pretty non-controversial. >> - What precisely are the use cases for absolute path names? Concrete >> examples are needed. With the current spec, some things can go wrong >> (e.g., see below), so we need real use cases to know how to address >> this. > > The purpose is to be able to remove scripts, data files, documentation, etc. > that's stored in some place that's not package-relative, and survive a move > of the package directory itself. > > For example, suppose I install with --prefix=~, and have files in ~/bin and > ~/lib. I then move ~/lib/python2.7 to ~/pylib. If the scripts were > installed with a relative path, this would break the link to the script > still living in ~/bin. > > Now that I'm thinking about this, it occurs to me that there is another way > to address this besides mixing relative and absolute paths in RECORD, and > that is to make *all* the paths relative, and include an original > installation location in INSTALLER. Then, it would be possible to detect > whether the installation location had been moved and handle it > accordingly... for some definition of "accordingly". > > I agree that further discussion is probably needed to resolve this. I'm not 100% sure I follow your logic here. If you're renaming directories after install, without involving the package manager, I think you probably deserve whatever you get :-) Having said that, your proposal (all paths relative, plus an install location) is similar to something I mentioned earlier (but better explained :-)) The only case it doesn't cater for is when an installed file can't be expressed as relative to the "install location" (which would involve being on a different drive on Windows). That's likely to never work, so I doubt it counts in any case. I've got some concerns about the possibility of having locations "above" the install location - something like "../../scripts/driver.bat". I'm not sure if such things will work (consider if the install location is a zipfile or egg) but nor am I sure anything generates such names. As you say, more discussion (and use cases!) would help here. >> - How will bdist_wininst/bdist_msi/bdist_rpm be updated? > > bdist_wininst, bdist_dumb, and various others use 'install --root' pattern > to generate files for installation, which means that they would > transparently end up writing a correct RECORD file, except for the inclusion > of incorrect absolute paths for non-libdir-relative files. However, if we > used the "all relative in RECORD, with a base in INSTALLER", these cases > could transparently be treated as another instance of install directory > relocation. > > I don't know if bdist_msi does a --root install before generating the .msi; > if it does, then it should work the same way. Yes, that sounds like a plausible approach. >> - Can distutils be made to install files in places the current RECORD >> file spec can't handle? (I think the answer is "yes"). What happens >> then? > > Surely an absolute path is sufficient? Current spec says relative or $PREFIX/... or $EXEC_PREFIX/... I think that was based on earlier discussion, but it's hampered by the fact that nobody's pointing at a real case of a distribution that actually tries to *install* files in odd places. (Actually, something that installs a driver script, like twisted's twistd or nose's nosetests, would do - I'll see what I can find tomorrow). >> - Should distribution names be case insensitive on case insensitive >> filesystems? For comparison, module/package names are always case >> sensitive even on case insensitive systems. > > IMO, they should be case-insensitive *regardless* of filesystem case, and I > believe PyPI has already begun disallowing packages with only case-distinct > names (e.g. 'Zope' vs. 'ZoPe') to be registered. OK, that's another option :-) It's not hard to change the code, so as long as a final decision is reached at some point, we're OK here... Thanks for the comments, Paul. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 376 - Open questions
2009/7/6 Nick Coghlan :
> I'd add one more question to the list: is allowing backslash separated
> names in the RECORD file actually a good idea, or would it be better to
> always use forward slashes?
They do always use forward slashes.
> For the other questions, I don't have anything much to add to PJE's
> comments, except that the "all relative" paths idea won't work due to
> the Windows drive letter issue (i.e. if an installer puts files in
> C:\Program Files, there is no guarantee that a relative path between
> site-packages and Program Files even exists if Python is installed on a
> different drive).
The big question, though, is can an installer actually *do* that in
practical terms?
- There are *no* guaranteed absolute locations on Windows, so any such
oddly-located file would require user interaction to work. Certainly
bdist_wininst and bdist_msi don't do that.
- My experiments indicate that bdist_{wininst,msi} are broken with
respect to absolute paths anyway: they do a --root install to a
temporary directory (and the absolute paths don't end up in there) and
then package up that temporary directory.
I still want to see a real life example that demonstrates that there
is a genuine issue here. We're spending a lot of energy and complexity
trying to design a solution to a problem that actually doesn't appear
to exist in practice...
(To be honest, I'd be fairly confident in saying that absolute paths
can be ignored on Windows, subject to some corner cases that I haven't
thought through yet. My worry is that I don't know what Unix and Mac
users might do, so I can't just wish away the issue because it can't
arise on Windows. Can a Unix/Mac user offer a real-world example on
their own system?)
Paul.
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 376 and PEP 302 - allowing import hooks to provide distribution metadata
Paul Moore writes: > In fact, the above strongly suggests to me that PEP 376 must NOT > follow setuptools conventions - otherwise, it will end up clashing > with the files setuptools is putting on my system. I think the argument for a separate ‘.pydist’ metadata directory, normative in PEP 376, is a good one. > > If so, I'd still prefer to keep the new metadata safely out of reach > > of the legacy package management systems that don't understand it, > > while having distutils retain the ability to generate a legacy > > ".egginfo" file to warn off the existing package management tools > > from installing the same distribution again. > > So for a minimal case of a single .py file packaged up as a > distribution, we now have the .py file, a legacy .egginfo file, a new > .pydist directory containing RECORD and PKG-INFO files? > > That's getting silly. The burden of switching is that we need a compatibility bridge. If that means that a tool must support the old while also supporting the new, that's not silliness in the tool, but acknowledgement of the silliness we're migrating away from. That doesn't mean, though, that PEP 376 needs to mention the old format at all; just that distutils should support it, deprecated, until a time when it's deemed safe to remove. In fact, I think it's best that PEP 376 discuss *only* the format we're trying to migrate to, and support for existing formats is firmly outside its scope. > And yet, given that PEP 376 is explicitly being designed with a goal > being to act as the common standard that *all* package management > formats can use, is it not the whole point that once it's defined and > we have achieved consensus, existing package managers will switch to > using it rather than retaining a range of custom formats? Again, that needs to be a gradual process. Supporting a new format as defined in PEP 376 needs not (indeed, I would argue *should* not) mean the tool drop support for old formats immediately. There needs to be a compatibility chain to allow for fast migration *and* slow migration, since there will be a broad mix of both. Let PEP 376 discuss a format that is clean and new, but let discussions leading to that format acknowledge that it must allow (*not* require) parallel support of old formats for some time. -- \ “I love to go down to the schoolyard and watch all the little | `\ children jump up and down and run around yelling and screaming. | _o__) They don't know I'm only using blanks.” —Emo Philips | Ben Finney ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
