[issue8668] Packaging: add a 'develop' command
Éric Araujo added the comment: This is handled by setuptools/pip. It cannot be converted to a distutils feature request, as distutils does not handle dependencies. -- resolution: - out of date stage: needs patch - committed/rejected status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue8668 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8668] Packaging: add a 'develop' command
Alex Grönholm alex.gronholm+pyt...@nextday.fi added the comment: Python 3.3 is entering beta soon. The develop command is a must have, especially now that virtualenv is part of the official Python distribution. Can someone summarize what still needs to be done to get this feature merged? -- nosy: +agronholm ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue8668 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8668] Packaging: add a 'develop' command
Éric Araujo mer...@netwok.org added the comment: Merged yesterday and created this patch. Will refresh my memories by reading all messages again and review. -- dependencies: -Add **kwargs to reinitialize_command Added file: http://bugs.python.org/file24660/fe817128d2fc.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue8668 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8668] Packaging: add a 'develop' command
Changes by chris ch...@emerge-life.de: -- nosy: +chris ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue8668 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8668] Packaging: add a 'develop' command
Changes by David Barnett davidbarne...@gmail.com: -- nosy: +mu_mind ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue8668 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8668] Packaging: add a 'develop' command
higery shoulderhig...@gmail.com added the comment: What’s more practical for you, a review or a patch? I think a review is better(thanks for your time:) ), because some changes maybe needed after I have finished all the issues against the develop command. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue8668 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8668] Packaging: add a 'develop' command
Changes by higery shoulderhig...@gmail.com: Added file: http://bugs.python.org/file23387/adb2cb19ca9b.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue8668 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8668] Packaging: add a 'develop' command
Éric Araujo mer...@netwok.org added the comment: Thanks for the updated patch. There are things that should be removed (for example functions like get_develop_method, given that we only support packaging-based projects), and there are a few things to clean up in the tests. What’s more practical for you, a review or a patch? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue8668 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8668] Packaging: add a 'develop' command
higery shoulderhig...@gmail.com added the comment: higery, can you give us a status update? Do you have the time to ... Sorry to reply to you so late, I will read carefully these reviews asap and try to make an updated patch before 12th. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue8668 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8668] Packaging: add a 'develop' command
Éric Araujo mer...@netwok.org added the comment: higery, can you give us a status update? Do you have the time to address current reviews or would you like me to make an updated patch? I’d like to incorporate this command as soon as possible to let people play with it, and then we’ll see about integration with the install action. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue8668 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8668] Packaging: add a 'develop' command
Changes by Éric Araujo mer...@netwok.org: Added file: http://bugs.python.org/file23009/5359c895f841.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue8668 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8668] Packaging: add a 'develop' command
Changes by higery shoulderhig...@gmail.com: Added file: http://bugs.python.org/file22932/af7d14ff129b.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue8668 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8668] Packaging: add a 'develop' command
higery shoulderhig...@gmail.com added the comment: Through discussing inside or outside the mailing list on this bug tracker, current 'develop' has been made a kind of command other than an action. But there still isn't a consensus of the concrete implemention way, so I keep this command name as 'develop' and the simple usage of this command is : pysetup run develop BTW, most developers in this list have agreed that 'develop' is a kind of install command, so I also add an entry funtion 'install_editable' for it, like 'install_local_project' for the 'install_dist' command. In addition, current patch also supports several ways to run the 'develop' which is learned from the source code of install.py. When installing, there are three ways, '_run_packaging_install', '_run_setuptools_install', and '_run_distutils_install', so here for develop command, there are two ways, '_run_packaging_develop' and '_run_setuptools_develop'. You can know more through the source code and docs in this patch. BTW, should way convert the .egg-info directory to .dist-info directory? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue8668 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8668] Packaging: add a 'develop' command
Alexis Metaireau ale...@notmyidea.org added the comment: _run_setuptools_install is only intended to support setuptools setup.py, converting .egg-info to .dist-info, internally. IMO, you should not care about the differences between setuptools/distutils1/setuptools at this level, as it should be taken care at the install level. When installing old setuptools-based setup.py, the .egg-info should be gone, replaced by a shiny new .dist-info folder. I'm not sure why you're talking about it in the context of develop, can you clarify this? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue8668 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8668] Packaging: add a 'develop' command
Alexis Metaireau ale...@notmyidea.org added the comment: IOW, in my opinion, support for setuptools develop command is not needed in packaging core, and still be taken care directly be the users wanting to run python setup.py develop: I don't see any reason to make it avaible on the stdlib. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue8668 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8668] Packaging: add a 'develop' command
higery shoulderhig...@gmail.com added the comment: Alexis Metaireau ale...@notmyidea.org added the comment: I'm not sure why you're talking about it in the context of develop, can you clarify this? My consideration is : if in Packaging we always convert .egg-info directory to .dist-info directory, then my two different kinds of ways(setuptools' develop or packaging's develop) for the 'develop' command are unnecessary, we can just retain the new-style packaging's develop, which mean '_run_packaging_develop' function here. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue8668 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8668] Packaging: add a 'develop' command
Alexis Metaireau ale...@notmyidea.org added the comment: Yep, packaging is not keeping the .egginfo directories, or at least does not plan to keep them (It should be the case currently but I haven't checked recently) in the upcoming release, so I would go on removing support for setuptools' develop command here :) -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue8668 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8668] Packaging: add a 'develop' command
higery shoulderhig...@gmail.com added the comment: IOW, in my opinion, support for setuptools develop command is not needed in packaging core Then do you also mean support that for setuptools install is also not necessary in packaging core? and still be taken care directly be the users wanting to run python setup.py develop: I don't see any reason to make it avaible on the stdlib. I think the current implemention way of Packaging 'install' command just offers an executing router to run a proper 'install', which users doesn't need to know. It also the same to 'develop'. BTW, I think it's ok if setuptools is already installed when a user try to run 'python setup.py develop', if not, python will report error. In addition, Packaging use a different way to issue commands - cd to the source directory, and use 'pysetup run develop' to do the job. Old setup.py will be supported only by issuing commands under the source directory containing the setup.py, and if setuptools exists, Packaging will dispatch the job to it. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue8668 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8668] Packaging: add a 'develop' command
Alexis Metaireau ale...@notmyidea.org added the comment: On 08/18/2011 05:54 PM, higery wrote: Then do you also mean support that for setuptools install is also not necessary in packaging core? setuptools install is only supported in packaging because it's a widely used thing, and many python distributions are currently packaged using setuptools features in their setup.py I think the current implemention way of Packaging 'install' command just offers an executing router to run a proper 'install', which users doesn't need to know. What packaging.install does is, if the project has been packaged using setuptools, relying on it to install the projects, while not letting it take care of the dependencies. Then, the .egginfo is converted to a .distinfo, so the way setuptools distributions are installed is *not* compatible with setuptools installation procedure. It is important to note that setuptools will *not* be a dependency for packaging, but rather the end user will be prompted to install it if it's not present and he wants to install a setuptools based project (or if what he's trying to install relies on setuptools based projects). The problem with the develop command seems different to my eyes: develop is used by developers, not by end users. Thus, proposing them to keep using the old setuptools develop command in packaging may seem like encouraging them to continue using setuptools. Rather, I would prefer to say: well, if you want to use setuptools' develop command, you can continue to do so, but the way to go would probably to repackage your project using shiny new standards. That's one of the handles we have to help the transition to packaging, so taking this occasion seems important to me. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue8668 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8668] Packaging: add a 'develop' command
Éric Araujo mer...@netwok.org added the comment: [...] so I keep this command name as 'develop' and the simple usage of this command is : pysetup run develop So far, so good. BTW, most developers in this list have agreed that 'develop' is a kind of install command, so I also add an entry funtion 'install_editable' for it, That can be useful. On the other hand, pip uses subprocesses instead of calling Python functions, to make sure an error in the called code does not stop pip. I still think it’s a nice function to have. The name should be changed: “editable” is not used in our current terminology, so install_develop would be IMO better. I also think that “pysetup install --develop path/to/project” would be a nice high-level way to use develop, in the future. About supporting setup.py develop: I think too that it is not needed. Supporting setup.py install is a needed feature for *users*, as Alexis said; conversely, having a packaging-only develop feature can help seduce *developers* to use the new standards and tools. My vote is for removing setup.py develop support. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue8668 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8668] Packaging: add a 'develop' command
Éric Araujo mer...@netwok.org added the comment: Great. I’ve been re-reading some old threads in the spirit of “pth files are evil”, so I wondered whether we could/should avoid them, but I think the criticism was directed against pth files edited after the initial installation, and pth files abusing the format (import causes exec), IOW setuptools’ pth files; our current ideas avoid these two problems. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue8668 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8668] Packaging: add a 'develop' command
Carl Meyer c...@dirtcircle.com added the comment: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Éric Araujo mer...@netwok.org added the comment: [Carl] there's an implicit assumption that a .pth file is the most likely strategy. If you have other ideas, please share them. No, I think that's the most promising strategy. The implicit assumption comment was not criticism, just explanation for Michael. -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.10 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk4nF2wACgkQ1j/fhc23WEDvlwCeK3Y+MJGyb3uoEzYzJWaSCrTy WewAoI7UdW+nqP2SEtquvQXCndXX57VO =UFOY -END PGP SIGNATURE- -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue8668 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8668] Packaging: add a 'develop' command
Éric Araujo mer...@netwok.org added the comment: [Carl] there's an implicit assumption that a .pth file is the most likely strategy. If you have other ideas, please share them. [another message] I don't see why the installation-location-finding for develop should be any different than for a normal pysetup install. It’s only a technical limitation for now: the develop command is currently a standalone command, so it has to decide where to write stuff. If it were an option to install_dist instead of a standalone command, then it would have paths processing already written. Higery changed his code recently to get paths from the install_dist command instead of requiring site-packages. (You can read the reviews, if you don’t mind style comments mixed with more important issues.) Does pysetup install install to global site-packages by default, or try to find somewhere it can install without additional privileges? The install action can have a different behavior than the install_dist command. develop is only a command now, and I agree it should behave like install_dist (which it now does). (though I don't really see the value in arbitrary locations, since you then have to set up PYTHONPATH manually anyway). We don’t know what people do, what with /opt installs and plugins and whatever, so there’s just no value in not allowing any path for install. Certainly develop should support PEP 370, ideally with the same command-line flag as a regular install. Yes. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue8668 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8668] Packaging: add a 'develop' command
higery shoulderhig...@gmail.com added the comment: ** After the package has been installed in-place (using the develop command), how does one identify it as an in development project (or in development mode)? -- Case 3 and 6 touch on this topic (case 3 is a little vague at this time), but doesn't explain what type of action is intended. So if we install in-place (aka, develop), how does the python interpreter find the package? Are we using PYTHONPATH at this point (which would be contradict a requirement in case 6)? There is an .egg-link file that will be used by pkg_resources to find the develop-installed packages, so my current implementation of develop command in packaging module also adds a .distinfo-link file in the site-packages which will be used to identify a project is installed in development mode or not. -- Added file: http://bugs.python.org/file22628/unnamed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue8668 ___div class=gmail_quoteblockquote class=gmail_quote style=margin: 0pt 0pt 0pt 0.8ex; border-left: 1px solid rgb(204, 204, 204); padding-left: 1ex; ** After the package has been installed in-place (using the develop command), how does one identify it as an in development project (or in development mode)? -- Case 3 and 6 touch on this topic (case 3 is a little vague at this time), but doesn#39;t explain what type of action is intended. So if we install in-place (aka, develop), how does the python interpreter find the package? Are we using PYTHONPATH at this point (which would be contradict a requirement in  case 6)?br /blockquotedivbrThere is an .egg-link file that will be used by pkg_resources to find the develop-installed packages, so my current implementation of develop command in packaging module also adds a .distinfo-link file in the site-packages which will be used to identify a project is installed in development mode or not.br /div/divbr ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8668] Packaging: add a 'develop' command
Changes by Éric Araujo mer...@netwok.org: Removed file: http://bugs.python.org/file22628/unnamed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue8668 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8668] Packaging: add a 'develop' command
Éric Araujo mer...@netwok.org added the comment: For now, you should not worry about pkg_resources. Write a simple pure-packaging implementation compatible with packaging; the setuptools and distribute developers will see if they want to add forward compatibility with our system. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue8668 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8668] Packaging: add a 'develop' command
higery shoulderhig...@gmail.com added the comment: 2011/7/12 Michael Mulich rep...@bugs.python.org Michael Mulich michael.mul...@gmail.com added the comment: The wiki page has been edited to note what the develop command will write to the file system. I'll restate it here as well... The develop command writes three pieces of information to the filesystem: 1. It calls upon the build action(s) to build the package relative to the package's root directory. 2. It calls the [build|install]_distinfo action to write the .dist-info metadata inside the build directory. (see also Issue 12279) 3. It adds the build directory's path to a .pth file. You are right, what you listed above are also the things done by the 'develop' command of my current implementation. In addition, as I replied earlier, we can also add a .distinfo-link file more than the .pth file. -- Added file: http://bugs.python.org/file22630/unnamed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue8668 ___brdiv class=gmail_quote2011/7/12 Michael Mulich span dir=ltrlt;a href=mailto:rep...@bugs.python.org;rep...@bugs.python.org/agt;/spanbrblockquote class=gmail_quote style=margin: 0pt 0pt 0pt 0.8ex; border-left: 1px solid rgb(204, 204, 204); padding-left: 1ex; br Michael Mulich lt;a href=mailto:michael.mul...@gmail.com;michael.mul...@gmail.com/agt; added the comment:brbr The wiki page has been edited to note what the develop command willbr write to the file system. I#39;ll restate it here as well...br br The develop command writes three pieces of information to the filesystem:br  1. It calls upon the build action(s) to build the package relative tobr the package#39;s root directory.br  2. It calls the [build|install]_distinfo action to write thebr .dist-info metadata inside the build directory. (see also Issue 12279)br  3. It adds the build directory#39;s path to a .pth file.br/blockquotedivbrYou are right, what you listed above are also the things done by the #39;develop#39; command of my current implementation. In addition, as I replied earlier, we can also add a .distinfo-link file more than the .pth file.br /div/div ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8668] Packaging: add a 'develop' command
Changes by higery shoulderhig...@gmail.com: Removed file: http://bugs.python.org/file22630/unnamed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue8668 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8668] Packaging: add a 'develop' command
Changes by higery shoulderhig...@gmail.com: Added file: http://bugs.python.org/file22632/2750cd9e2111.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue8668 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8668] Packaging: add a 'develop' command
Changes by Éric Araujo mer...@netwok.org: Removed file: http://bugs.python.org/file22614/b1b9da3b3d20.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue8668 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8668] Packaging: add a 'develop' command
Michael Mulich michael.mul...@gmail.com added the comment: On Tue, Jul 12, 2011 at 9:39 AM, higery rep...@bugs.python.org wrote: The develop command writes three pieces of information to the filesystem: 1. It calls upon the build action(s) to build the package relative to the package's root directory. 2. It calls the [build|install]_distinfo action to write the .dist-info metadata inside the build directory. (see also Issue 12279) 3. It adds the build directory's path to a .pth file. You are right, what you listed above are also the things done by the 'develop' command of my current implementation. In addition, as I replied earlier, we can also add a .distinfo-link file more than the .pth file. I don't like the idea of a .distinfo-link file. Would it even be necessary if we already have a .pth entry? We should probably just use one of these files, either .distinfo-link or .pth. The .pth implementation has the least impact on code base and is already implemented. If we add support for a .distinfo-link, we would then need to modify database module to support that extension. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue8668 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8668] Packaging: add a 'develop' command
Éric Araujo mer...@netwok.org added the comment: Oh, I just realized that one thing I insisted on was wrong. I pushed for the modules to be built in the build dir, as well as the dist-info dir, so that the build dir can be added to sys.path to let both import and packaging.database find the files. But this breaks one important develop feature: editions to the code should be visible immediately, without having to re-run develop or build. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue8668 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8668] Packaging: add a 'develop' command
Éric Araujo mer...@netwok.org added the comment: Ah, higery’s code already has an answer for me: it writes *two* paths in the .pth file, one to the build dir (so that .dist-info is found) and one to the modules root (for modules, built in place). Anyone sees a problem with that? (For example huge sys.path.) In this scheme, when Python modules are edited, changes are visible instantly, when C modules are edited, a call to build_ext is required, and when the metadata is edited, build_distinfo is required. Does that sound good? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue8668 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8668] Packaging: add a 'develop' command
Éric Araujo mer...@netwok.org added the comment: I’ve reviewed the last patch. It looks like the code only installs to the global site-packages, and there is no support to install to the user site-packages or to another arbitrary location. On Windows, normal users seem to be able to write to the global site-packages (see #12260), but on other OSes with a proper rights model wink that won’t do. Luckily, PEP 370 brings us user site-packages (currently poorly documented, see #8617 and #10745), but only for 2.6, 2.7 and 3.x. It looks like Tarek is ready to drop 2.4 compatibility for distutils2, so the question is: what to do under 2.5? Generally, I don’t see why develop could not install to any directory. We want a default invocation without options to Just Work™, finding a writable directory already on sys.path and writing into it, but that doesn’t exclude letting the user do what they want. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue8668 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8668] Packaging: add a 'develop' command
Carl Meyer c...@dirtcircle.com added the comment: Ah, higery’s code already has an answer for me: it writes *two* paths in the .pth file, one to the build dir (so that .dist-info is found) and one to the modules root (for modules, built in place). Anyone sees a problem with that? (For example huge sys.path.) In this scheme, when Python modules are edited, changes are visible instantly, when C modules are edited, a call to build_ext is required, and when the metadata is edited, build_distinfo is required. Does that sound good? That sounds reasonable to me. I'm not worried about that making sys.path too long: whatever we do we aren't going to challenge buildout in that department ;-) -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue8668 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8668] Packaging: add a 'develop' command
Carl Meyer c...@dirtcircle.com added the comment: I’ve reviewed the last patch. It looks like the code only installs to the global site-packages, and there is no support to install to the user site-packages or to another arbitrary location. On Windows, normal users seem to be able to write to the global site-packages (see #12260), but on other OSes with a proper rights model wink that won’t do. Luckily, PEP 370 brings us user site-packages (currently poorly documented, see #8617 and #10745), but only for 2.6, 2.7 and 3.x. It looks like Tarek is ready to drop 2.4 compatibility for distutils2, so the question is: what to do under 2.5? Generally, I don’t see why develop could not install to any directory. We want a default invocation without options to Just Work™, finding a writable directory already on sys.path and writing into it, but that doesn’t exclude letting the user do what they want. I don't see why the installation-location-finding for develop should be any different than for a normal pysetup install. Does pysetup install install to global site-packages by default, or try to find somewhere it can install without additional privileges? Whatever it does by default, develop should do the same. If develop can install to arbitrary locations, then install should be able to as well (though I don't really see the value in arbitrary locations, since you then have to set up PYTHONPATH manually anyway). There is no reason for them to have different features in this area, it just adds confusion. Certainly develop should support PEP 370, ideally with the same command-line flag as a regular install. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue8668 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8668] Packaging: add a 'develop' command
Michael Mulich michael.mul...@gmail.com added the comment: After looking over the use cases, these are my findings and questions: * Cases 2, 3, 5 and 6 are strongly related. I'd suggest you condense them into a single use case. I agree with case 2 and 6 most, but have questions: ** Why wouldn't one simply use a virtualenv? -- Case 5 touches on this topic, but if we are installing in-place, who cares if can place a development package in the global site-packages directory? ** After the package has been installed in-place (using the develop command), how does one identify it as an in development project (or in development mode)? -- Case 3 and 6 touch on this topic (case 3 is a little vague at this time), but doesn't explain what type of action is intended. So if we install in-place (aka, develop), how does the python interpreter find the package? Are we using PYTHONPATH at this point (which would be contradict a requirement in case 6)? * Case 4 is a be unclear. Is Carl, the actor, pulling unreleased remote changes (hg pull --update) for these mercurial server plugins then running the develop command on them? * Case 1 is good and very clear, but I'd consider it a feature rather than required. Perhaps it should not be focused on first (priority). Thoughts? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue8668 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8668] Packaging: add a 'develop' command
Carl Meyer c...@dirtcircle.com added the comment: Can someone post a link here to the page of use cases that Michael just reviewed? I think the link came through on the Fellowship mailing list, but I'm not quickly finding it... -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue8668 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8668] Packaging: add a 'develop' command
Alexis Metaireau ale...@notmyidea.org added the comment: Carl, I believe that's this one: http://wiki.python.org/moin/UsecasesOfDevelop -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue8668 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8668] Packaging: add a 'develop' command
Carl Meyer c...@dirtcircle.com added the comment: On 07/11/2011 09:17 AM, Michael Mulich wrote: * Cases 2, 3, 5 and 6 are strongly related. I'd suggest you condense them into a single use case. I agree with case 2 and 6 most, but have questions: ** Why wouldn't one simply use a virtualenv? I don't know. I don't consider case 3 useful, because I don't consider I don't want to use a virtualenv (without some clearer technical justification) to be a prejudice the develop feature needs to support; especially if supporting it essentially means re-implementing a less-capable version of virtualenv within the develop command. -- Case 5 touches on this topic, but if we are installing in-place, who cares if can place a development package in the global site-packages directory? Several of these stories make the assumption that even the in-place installation will require placing a file in the installation location (a .pth file, if we follow the current setuptools implementation strategy). I think this is probably true, given the requirements in case 6 (which I agree with). So if you want an in-place install that's globally accessible, you'd need write access to global site-packages. ** After the package has been installed in-place (using the develop command), how does one identify it as an in development project (or in development mode)? -- Case 3 and 6 touch on this topic (case 3 is a little vague at this time), but doesn't explain what type of action is intended. So if we install in-place (aka, develop), how does the python interpreter find the package? Are we using PYTHONPATH at this point (which would be contradict a requirement in case 6)? These use cases (probably intentionally) don't touch on specific implementation strategies, but as I mentioned there's an implicit assumption that a .pth file is the most likely strategy. * Case 4 is a be unclear. Is Carl, the actor, pulling unreleased remote changes (hg pull --update) for these mercurial server plugins then running the develop command on them? Right, although the requirement for that story is that you don't have to re-run the develop command after every pull; if you develop-install it once, you can simply pull more code changes in and they'll immediately be available. I've added a line to that story to make it more clear. * Case 1 is good and very clear, but I'd consider it a feature rather than required. Perhaps it should not be focused on first (priority). Thoughts? I agree that's a second-level feature (or, perhaps more accurately, a bug in the existing setuptools feature that I was hoping could be addressed in the d2 version), not a primary requirement. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue8668 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8668] Packaging: add a 'develop' command
Michael Mulich michael.mul...@gmail.com added the comment: On Mon, Jul 11, 2011 at 1:14 PM, Carl Meyer rep...@bugs.python.org wrote: * Cases 2, 3, 5 and 6 are strongly related. I don't know. I don't consider case 3 useful, because I don't consider I don't want to use a virtualenv (without some clearer technical justification) to be a prejudice the develop feature needs to support; especially if supporting it essentially means re-implementing a less-capable version of virtualenv within the develop command. I think your later comments about the use of .pth files solves the issue for all four cases. We simply make reference in a .pth file in one of the approved site locations. For example, in case two we would write a .pth entry to site.USER_SITE. Sound about right? -- Case 5 Several of these stories make the assumption that even the in-place installation will require placing a file in the installation location (a .pth file, if we follow the current setuptools implementation strategy). I think this is probably true, given the requirements in case 6 (which I agree with). So if you want an in-place install that's globally accessible, you'd need write access to global site-packages. Basically write the .pth entry for the build to a site (the standard lib module) recognized location. * Case 4 Right, although the requirement for that story is that you don't have to re-run the develop command after every pull; if you develop-install it once, you can simply pull more code changes in and they'll immediately be available. I've added a line to that story to make it more clear. Ah, this case impacts the decision being made in issue 12279 (http://bugs.python.org/issue12279). The decision is to write a RECORD file or not. We wouldn't write a RECORD if you want to be able to update without rerunning the develop command. But this would be invalid based on PEP 376 guidelines. Please post your thoughts about this in that issue. The wiki page has been edited to note what the develop command will write to the file system. I'll restate it here as well... The develop command writes three pieces of information to the filesystem: 1. It calls upon the build action(s) to build the package relative to the package's root directory. 2. It calls the [build|install]_distinfo action to write the .dist-info metadata inside the build directory. (see also Issue 12279) 3. It adds the build directory's path to a .pth file. Thanks Carl -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue8668 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8668] Packaging: add a 'develop' command
Changes by Éric Araujo mer...@netwok.org: Added file: http://bugs.python.org/file22614/b1b9da3b3d20.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue8668 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8668] Packaging: add a 'develop' command
Éric Araujo mer...@netwok.org added the comment: Higery asked me why I recommended/asked that the .dist-info directory should be built in the build directory: “don't you feel it's a bit strange because we actually don't build the source except for some .c/.cpp files, but create a build directory?” There are a few converging reasons for that. Maybe a develop expert user will disagree with me, but I think that together they make a good case. The first reason is symmetry. The test command (http://docs.python.org/dev/packaging/commandref#test) needs to have the Python code in the build directory on sys.path, so that if you use build-time 2to3 conversion (currently undocumented :( apart from a line in http://docs.python.org/dev/packaging/setupcfg#command-sections), the tests need to find the 3.x code in the build dir, not the 2.x code in the project dir. I think that since we already put the build dir on sys.path to run tests, we might as well follow the same idead and put the dist-info dir in the build dir. The test command will then automatically have access to dist-info (see #12302), and other commands and tools can learn to look for dist-info in the build dir. The second reason is that I think we need to separate build_distinfo from install_distinfo (#12279). If we do this, then the new command has to write in the build dir, just like all other build_* commands (with possible exceptions). The third reason is that I don’t like that every project that uses setuptools needs to add ProjectName.egg-info in the VCS ignore file (like .hgignore); if we use the build dir, our generated files are already covered by the ignore of the build dir. (This makes me think that we may want to move MANIFEST to the build dir.) I hope my reasons are clearer now; let me know if I did not explain something clearly enough, or if you disagree :) -- nosy: +carljm ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue8668 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8668] Packaging: add a 'develop' command
Changes by Éric Araujo mer...@netwok.org: -- hgrepos: +29 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue8668 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8668] Packaging: add a 'develop' command
Changes by Éric Araujo mer...@netwok.org: -- keywords: +patch Added file: http://bugs.python.org/file22377/aa68d35988bb.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue8668 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8668] Packaging: add a 'develop' command
Peter Waller peter.wal...@gmail.com added the comment: Hi - Great to see this functionality coming. There is one feature of it that I would really like to see fixed, which is currently broken in setuptools/distribute - I'm sorry if this is the wrong forum for this note, but I wanted to add it to the discussion somewhere. That feature is the package_dir argument to setup(). If the sources aren't in the root directory, the package doesn't function correctly with `python setup.py develop` Here is a reference to an issue filed against distribute: https://bitbucket.org/tarek/distribute/issue/177/setuppy-develop-doesnt-support-package_dir-arg-to Is there any possibility of seeing this work correctly? A lot of packages use it, and for them, `develop` is currently broken. Apologies if this feature is implemented and I missed it, but I see no reference to package_dir in the patch, so I would be (pleasantly) surprised if it was implemented. I would be happy to provide a testcase on request. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue8668 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8668] Packaging: add a 'develop' command
Éric Araujo mer...@netwok.org added the comment: Yes, this should of course be supported. Note that in packaging, packages_dir has been simplified to packages_root, which means that all modules and packages must be in the same directory. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue8668 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8668] Packaging: add a 'develop' command
Changes by Éric Araujo mer...@netwok.org: -- dependencies: +Add **kwargs to get_reinitialized_command ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue8668 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8668] Packaging: add a 'develop' command
Changes by Peter Waller peter.wal...@gmail.com: -- nosy: +Peter.Waller ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue8668 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8668] Packaging: add a 'develop' command
Changes by Michael Mulich michael.mul...@gmail.com: -- nosy: +michael.mulich ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue8668 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8668] Packaging: add a 'develop' command
Éric Araujo mer...@netwok.org added the comment: FTR, Xu Dehai (higery) is working on this for GSoC. We discussed the requirements on the mailing list: http://groups.google.com/group/the-fellowship-of-the-packaging/browse_thread/thread/ae196efc4956b9e2 This message in particular defines clearly the requirements: http://groups.google.com/group/the-fellowship-of-the-packaging/msg/9ef937cf2a90a882 -- dependencies: +Add build_distinfo command to packaging nosy: +higery stage: - needs patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue8668 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8668] Packaging: add a 'develop' command
Changes by Éric Araujo mer...@netwok.org: -- assignee: tarek - eric.araujo keywords: +gsoc title: add a 'develop' command - Packaging: add a 'develop' command versions: +Python 3.3 -3rd party ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue8668 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
