Re: [Distutils] Distutils changes - end user requirements (Was: Deprecate MANIFEST.in)
Hi, I've been working on something similar with various approaches since a long while. I've finally got someting running on http://download.opensuse.org/repositories/home:/cavallo71:/python-opt It compiles on across different distribution but at the moment the suse rpmlint checks are too stringent and the compile fails (actually it works fine but it doesn't release the packages). Basically what I needed was (still is) a rpm integrated package to install on any linux distribution with a precise set of requirements: 1. The administrator should see the distro installed python (transparency) 2. The new interpreter should set on request the environment (explicit) 3. The installation should leverage the rpm infrastructure (consistency in a running system) These requirements are mandatory in order to garantee a smooth operation on across systems. Moreover a user shoudl use the distuils tools to install additional packages without any magic (so python setup.py bdist_rpm should generate a working installable package). Basically to activate the new python interpreter all a user have to do is typing something like: $ . /opt/cavallo-python-2.6.1/cavallo-python-env.sh (yes the name has to be changed at some point;)) Let me know if this sounds a good idea, Regards, Antonio Cavallo I'm packaging python for linux in a separate deirectory and I needed to be completely system trasnparent, On Thu, Apr 9, 2009 at 8:54 PM, Trent Mick tre...@activestate.com wrote: Paul Moore wrote: On that note, didn't ActiveState distribute their version of Python with a package manager (PPM)? As far as I know, that was only ever supported by ActiveState themselves, no-one else ever built PPM packages for their extensions, and it's been quietly dropped in recent versions (Google tells me that it vanished around Python 2.3). Yes, it was there a long time ago. It was dropped around ActivePython 2.3, as you say, because it wasn't well architected and, at the time, we didn't have the resources to maintain it. However, we've just recently hired a full time employee to (in part) start looking at a PyPM package manager again -- this time with an appropriate architecture. The hope is to have something to show in the fall (perhaps sooner). Trent -- Trent Mick trentm at activestate.com ___ Distutils-SIG maillist - distutils-...@python.org http://mail.python.org/mailman/listinfo/distutils-sig ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Distutils changes - end user requirements (Was: Deprecate MANIFEST.in)
Can any packager step up and explain why they execute python setup.py bdist_egg rather than python setup.py bdist_wininst when creating distributions for their Windows users? For starters, the former works on all my development platforms. I can script buildbot to do it the same on all platforms, for example. The latter I haven't tried. Just a minute... Zooko tries python setup.py bdist_wininst. (See appendix A.) Secondly, the package can be automatically installed when the user easy_installs a *different* package which depends on my package. The Tahoe project currently has fourteen such dependencies. The Tahoe install process works on all platforms including Windows: http://allmydata.org/source/tahoe/trunk/docs/install.html It isn't perfect, but I think it is pretty cool that the same install document works for both Windows users and everyone else. Thirdly, the entry_points mechanism for making executable scripts means I get executable scripts on all platforms with no extra effort on my part. I'm not saying that this is perfect, or that users don't want other things, or that developers shouldn't want other things. For Tahoe in specific, I really would like a normal installable package thing like Windows users expect for their apps. But until someone offers to spend their time crafting such a thing for Tahoe (ideally by using the bbfreeze tool [1] to automatically generate one from the Tahoe source distribution), I'm happier with what I've got now than with the alternative, which is a long list of around sixteen -- sixteen! -- different packages that the user would have to manually install before they can install Tahoe, where each of them might have different special install instructions on any of a dozen different supported platforms. Again, the setuptools-based system that Tahoe has now is not *alternative* to Windows-specific installers or Debian-specific apt- get'able packages. In fact, hopefully they will become increasingly *complementary* as tools like stdeb and bbfreeze mature. Regards, Zooko [1] http://allmydata.org/trac/tahoe/ticket/585 # make it work with bbfreeze Appendix A: Let's see, python setup.py bdist_wininst produced a file which I then uploaded to my web server: http://zooko.com/allmydata- tahoe-1.3.0-r3833.win32.exe. When I run that file, it first pops up a security warning that the producer of this file (me, in fact) is untrusted. Once I hit the button labelled Leave me alone about your stupid security warnings then it enters the normal installation wizardy thing. It has all my package metadata nicely displayed on the first page. Cool! Then it asks me which python to use -- it found 2.5 and 2.6 on my system. I choose, then click install, then finish and it goes away. Now what? Let's see, it didn't add anything to my Start Menu. I see that it installed it into c:/Python26/Lib/site-packages/allmydata , (which is the name of the top-level module in the allmydata-tahoe distribution), as well as writing a Removeallmydat-tahoe.exe and a allmydata-tahoe-wininst.log file. Oh! I see that it made a c:/ Python26/Scripts/tahoe.exe file! Cool. Okay, but I can't actually use it unless I now go and install a dozen or more dependencies (as mentioned in the body of this letter). Okay, finally, uninstall. Hm... It isn't there in the Add/Remove Programs list. Oh -- it is named Python 2.6 allmydata-tahoe-1.3.0- r3833 instead of allmydata-tahoe. And it worked to remove all the files as far as I can tell. It works! Cool! My verdict: as a consumer of other people's software, I wouldn't want to use a GUI tool like this if I could help it (it takes about 10 times as long to install a package that way as on the command-line, you can't script it, and you can't easily capture an encoding of all the options which were chosen for later reproduction). I'm sure that many other users would want to install applications this way, i.e. things which they actually care whether it is installed or not, but I suspect most users do not want to install dependencies this way, i.e. things that are there only because an application requires it. As a producer of packages I would be happy to use bdist_wininst since it is fully scriptable and I can just program my buildbot to do it for me, however I won't do so until the resulting installer can either automatically satisfy its dependencies or come with its dependencies bundled inside it. ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Distutils changes - end user requirements (Was: Deprecate MANIFEST.in)
2009/4/9 Lennart Regebro rege...@gmail.com: On Wed, Apr 8, 2009 at 12:27, Paul Moore p.f.mo...@gmail.com wrote: 1. (Meta-requirement) I want to be able to download a Windows installer[1] for *every* package I need. I would like to know why you have this requirement. Some sort of Windows binary, absolutely. But an installer? Would a package management GUI be enough? A Windows software that can install (and remove) eggs? Because I like to see my installed Python packages in Windows' Add/Remove Programs list. Think of it as an application of There should be one-- and preferably only one --obvious way to do it (in this case, install and remove software on my PC). Although maybe because I'm not Dutch, that doesn't apply :-) 1a. This means that the barrier for packagers building Windows installers should be as low as possible. Agreed. 1b. It also means that other formats (e.g. eggs) should offer no benefit over Windows installers Well... they don't, do they? Except that they can be easy_installed, right? Don't they? I have to admit that I'm baffled by how the features in setuptools/eggs/easy_install all hang together. What about the magic that creates executables from scripts? Entry points? Stuff like that. Don't you need to use eggs to make them work? If not, why do people distribute eggs rather than bdist_wininst installers? From what I understand, easy_install can convert and install a bdist_wininst if an egg doesn't exist, but there's no path the other way. So by what you're saying, eggs are a strict subset of bdist_wininst, and so people should be distributing bdist_wininst installers. But they aren't, so what gives? Can any packager step up and explain why they execute python setup.py bdist_egg rather than python setup.py bdist_wininst when creating distributions for their Windows users? Paul. ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Distutils changes - end user requirements (Was: Deprecate MANIFEST.in)
2009/4/9 zooko zo...@zooko.com: On Apr 8, 2009, at 4:27 AM, Paul Moore wrote: 1. (Meta-requirement) I want to be able to download a Windows installer[1] for *every* package I need. 1a. This means that the barrier for packagers building Windows installers should be as low as possible. 1b. It also means that other formats (e.g. eggs) should offer no benefit over Windows installers I personally, as a consumer of other people's software, prefer to acquire their packages as sdist .tar.gz's or as .eggs on all platforms, including Windows, which I use regularly. So your and my preferences as a consumer of packages differ on this. Are you able to explain *why* you prefer this? (I'm not trying to change your mind at all, just trying to get a wider understanding of how people's experiences differ). For me (using Windows only) the benefits of bdist_wininst/bdist_msi are: - Uses the system package manager (limited as it may be) - Means I don't need to have a compiler installed (even though I do on many of my machines) If I have to guess at your reasons, my assumption would be: - Consistent means of installing on all platforms - ??? How far off the mark am I? Written like that, my reasons (obviously, as I'm biased!) look stronger, so I am pretty sure I'm not understanding your situation properly. Fortunately, as far as the distutils is concerned Windows installers vs. sdists vs. eggs are not mutually exclusive. The distutils will provide a standard for metadata so that all of those distribution formats can have the same metadata, and it will allow automation such as bdist_msi and sdist_dsc (from stdeb) so that it can become easier for developers to produce these things. By the way, the exact same sorts of preference-among-consumers issues arises on the Linux side (which I also use regularly), where some people mistakenly think that they want distutils to support eggs worse, when what they really want is for it to support debs better (which I hope stdeb will ultimately do). The key thing for me would be if it was easy to convert any given binary format for a platform into another. This stems from the fact that compiling extensions is hard on Windows. So being able to convert eggs to and from bdist_wininst or bdist_msi installers would be sufficient. But standard metadata isn't quite enough by itself, as it still requires someone with a compiler and the relevant libraries configured to create the DLLs, etc. Given that all the *files* contained in any given binary distribution are identical (if they aren't that's a different problem!) then creating converters should be possible - although I'm not aware of anyone looking at this. Maybe it's a useful area to spend some effort on? Paul. ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Distutils changes - end user requirements (Was: Deprecate MANIFEST.in)
2009/4/9 Lennart Regebro rege...@gmail.com: 2009/4/9 Paul Moore p.f.mo...@gmail.com: Don't they? I have to admit that I'm baffled by how the features in setuptools/eggs/easy_install all hang together. What about the magic that creates executables from scripts? Entry points? Stuff like that. Don't you need to use eggs to make them work? No? Entry points work even if you have the source code in a tgz format and run setup.py install. The distribution format is not magical for that afaik. So by what you're saying, eggs are a strict subset of bdist_wininst, and so people should be distributing bdist_wininst installers. But they aren't, so what gives? Nobody knows about it? Possibly :-( But in any case, even if it would be a good idea to have every single Python package on the system listed in the Add/Remove programs list (Which I don't think it is, but that's a matter of taste, no logical arguments behind that), that would in practice mean that each and every package on PyPI must have a wininstaller, even if it is a pure-python package. That doesn't seem realistic to me. Personally, I'd be happy if every package that currently distributes any form of Windows binaries, distributed a Windows installer. That's about the same level of coverage as existed before setuptools appeared, so I don't think that's impossible to achieve. I agree that expecting *everything* to have a Windows installer is unreasonable. As regards your other points regarding Windows installers, I don't disagree entirely. But my personal preference is to work with the system packager, even if it's less functional than I'd like. Paul. ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Distutils changes - end user requirements (Was: Deprecate MANIFEST.in)
Paul Moore wrote: Can any packager step up and explain why they execute python setup.py bdist_egg rather than python setup.py bdist_wininst when creating distributions for their Windows users? I use both. If I'm installing a single package into a site-wide python, I use bdist_wininst. If I'm installing a package where I want to have multiple versions (in different directories, often with different python.exe's), I run bdist_egg. The latter case is usually installed with buildout. ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Distutils changes - end user requirements (Was: Deprecate MANIFEST.in)
2009/4/9 Paul Moore p.f.mo...@gmail.com: Don't they? I have to admit that I'm baffled by how the features in setuptools/eggs/easy_install all hang together. What about the magic that creates executables from scripts? Entry points? Stuff like that. Don't you need to use eggs to make them work? No? Entry points work even if you have the source code in a tgz format and run setup.py install. The distribution format is not magical for that afaik. So by what you're saying, eggs are a strict subset of bdist_wininst, and so people should be distributing bdist_wininst installers. But they aren't, so what gives? Nobody knows about it? But in any case, even if it would be a good idea to have every single Python package on the system listed in the Add/Remove programs list (Which I don't think it is, but that's a matter of taste, no logical arguments behind that), that would in practice mean that each and every package on PyPI must have a wininstaller, even if it is a pure-python package. That doesn't seem realistic to me. When I used Windows I would have preferred to have installed packages listed as parts in the Python installer, so that you can go in an modify the setup and add and remove parts. As with WinG or MS Office or whatever. That would also remove the need for a separate installer exe for each file. This is then basically the same requirements as the uninstall requirements from Linux people, except that you would have a GUI. -- Lennart Regebro: Python, Zope, Plone, Grok http://regebro.wordpress.com/ +33 661 58 14 64 ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Distutils changes - end user requirements (Was: Deprecate MANIFEST.in)
2009/4/9 Paul Moore p.f.mo...@gmail.com: Personally, I'd be happy if every package that currently distributes any form of Windows binaries, distributed a Windows installer. That's about the same level of coverage as existed before setuptools appeared, so I don't think that's impossible to achieve. I agree that expecting *everything* to have a Windows installer is unreasonable. I don't see the benefits of a windows installer in that situation. You are going to have a large set of python libraries, with a small subset of them in the Installed Software list, and the rest not. What did that achieve? As regards your other points regarding Windows installers, I don't disagree entirely. But my personal preference is to work with the system packager, even if it's less functional than I'd like. But the suggestion of having packages managed from the python setup program is working with the system packager just as much as selecting what parts of Office you want is. That's what Windows users would expect. I don't think they, when installing Plone expect to get a hundred Python packages listed in the Installed Software list, as an extreme example. And how would it handle multiple installations into Python, etc.? I just don't see the benefit. While having a Python package manager, I *can* see the benefit. But as always, I don't use Windows much nowadays, so I don't really care. I just want understand the thinking. -- Lennart Regebro: Python, Zope, Plone, Grok http://regebro.wordpress.com/ +33 661 58 14 64 ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Distutils changes - end user requirements (Was: Deprecate MANIFEST.in)
Paul Moore wrote: If not, why do people distribute eggs rather than bdist_wininst installers? For python only code, I can see one obvious reason: you can build one egg on your platform, and it is supposed to work everywhere. I may be wrong, but I don't think packagers who care about windows a lot would distribute eggs only. For numpy/scipy, some windows users do request eggs for numpy/scipy (which we do not provide for numpy/scipy). cheers, David ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Distutils changes - end user requirements (Was: Deprecate MANIFEST.in)
2009/4/9 Lennart Regebro rege...@gmail.com: 2009/4/9 Paul Moore p.f.mo...@gmail.com: Personally, I'd be happy if every package that currently distributes any form of Windows binaries, distributed a Windows installer. That's about the same level of coverage as existed before setuptools appeared, so I don't think that's impossible to achieve. I agree that expecting *everything* to have a Windows installer is unreasonable. I don't see the benefits of a windows installer in that situation. You are going to have a large set of python libraries, with a small subset of them in the Installed Software list, and the rest not. What did that achieve? Sorry, I don't follow. Maybe I was unclear. One more try, and if I still manage to confuse you, don't worry about it. It's not a major point. As far as I am concerned, all I need is for all the packages *I* use to be available as Windows installers, which I'll download and install. All of the Python packages I use are in Windows' Add/Remove Programs. Given that the packages I need isn't a particularly helpful definition, and given that I'm a terrible dabbler and like to try out new packages at an alarming rate (:-)), I'd like it if any package that is distributed in a Windows binary form at all, to be in available in Windows installer form (ie, I don't want to be forced into a situation where, in order to use a package, I have to use an installation method other than my preferred method). That sounds horribly dogmatic - everyone should work to make my life easier :-( It wasn't meant like that at all. For context, I'm thinking of the days before setuptools, when everything either had a Windows installer (bdist_wininst) or no Windows binaries at all. What frustrates me about the current situation is seeing cases where the developer has clearly gone to the effort to build Windows binaries, but they aren't in a form I can (or rather, want to) use. An egg-bdist_wininst converter would fix this issue. As would everyone standardising on bdist_wininst (which, as per the previous message, appears to be prefectly feasible given that bdist_wininst seems to be a strict superset of egg...) As regards your other points regarding Windows installers, I don't disagree entirely. But my personal preference is to work with the system packager, even if it's less functional than I'd like. But the suggestion of having packages managed from the python setup program is working with the system packager just as much as selecting what parts of Office you want is. That's what Windows users would expect. I don't think they, when installing Plone expect to get a hundred Python packages listed in the Installed Software list, as an extreme example. And how would it handle multiple installations into Python, etc.? Ah! I see what you're getting at. If it is indeed possible to modify the Python installer (which I believe is a MSI installer) to manage extensions as subpackages, then that would be a fine option, indeed. I wasn't aware that was even possible - and I certainly am not aware of anyone with expertise in how the standard Pytjhon binaries are packaged having said they could take such a project on. But if it could be done, then I'm all in favour. I just don't see the benefit. While having a Python package manager, I *can* see the benefit. But as always, I don't use Windows much nowadays, so I don't really care. I just want understand the thinking. Comparing the present with package management integrated into the Python installer, it's mainly a presumption that the latter isn't going to happen. I was assuming the choice was between what we have now and a standalone Python package management system which doesn't integrate into Add/Remove programs at all. That's a much clearer follow the system standard vs roll your own type of choice (which is still a personal choice, of course...) On that note, didn't ActiveState distribute their version of Python with a package manager (PPM)? As far as I know, that was only ever supported by ActiveState themselves, no-one else ever built PPM packages for their extensions, and it's been quietly dropped in recent versions (Google tells me that it vanished around Python 2.3). So that gives some real-world evidence of how non-standard Python package managers fare (albeit not very representative, given the limited use of ActiveState's Python). Paul. ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Distutils changes - end user requirements (Was: Deprecate MANIFEST.in)
Paul Moore wrote: An egg-bdist_wininst converter would fix this issue. As would everyone standardising on bdist_wininst (which, as per the previous message, appears to be prefectly feasible given that bdist_wininst seems to be a strict superset of egg...) I don't think this is true. I don't think bdist_wininst supports the buildout use case where I want to install multiple versions of multiple packages in different (or even the same) directory. This is a critical use case for Zope/Plone apps. ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Distutils changes - end user requirements (Was: Deprecate MANIFEST.in)
At 03:03 PM 4/9/2009 +0100, Paul Moore wrote: 2009/4/9 Eric Smith e...@trueblade.com: Paul Moore wrote: An egg-bdist_wininst converter would fix this issue. As would everyone standardising on bdist_wininst (which, as per the previous message, appears to be prefectly feasible given that bdist_wininst seems to be a strict superset of egg...) I don't think this is true. I don't think bdist_wininst supports the buildout use case where I want to install multiple versions of multiple packages in different (or even the same) directory. This is a critical use case for Zope/Plone apps. OK, I'm making that statement based on some earlier comments. I've probably misunderstood them. But I thought easy_install could take a bdist_wininst installer and use it to make an egg? Yes, it can. You could also, in principle, reverse the process. However, the resulting bdist_wininst wouldn't support multiple versions, because the bdist_wininst infrastructure doesn't support that. ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Distutils changes - end user requirements (Was: Deprecate MANIFEST.in)
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Paul Moore wrote: 2009/4/9 Lennart Regebro rege...@gmail.com: 2009/4/9 Paul Moore p.f.mo...@gmail.com: Don't they? I have to admit that I'm baffled by how the features in setuptools/eggs/easy_install all hang together. What about the magic that creates executables from scripts? Entry points? Stuff like that. Don't you need to use eggs to make them work? No? Entry points work even if you have the source code in a tgz format and run setup.py install. The distribution format is not magical for that afaik. So by what you're saying, eggs are a strict subset of bdist_wininst, and so people should be distributing bdist_wininst installers. But they aren't, so what gives? Nobody knows about it? Possibly :-( But in any case, even if it would be a good idea to have every single Python package on the system listed in the Add/Remove programs list (Which I don't think it is, but that's a matter of taste, no logical arguments behind that), that would in practice mean that each and every package on PyPI must have a wininstaller, even if it is a pure-python package. That doesn't seem realistic to me. Personally, I'd be happy if every package that currently distributes any form of Windows binaries, distributed a Windows installer. That's about the same level of coverage as existed before setuptools appeared, so I don't think that's impossible to achieve. I agree that expecting *everything* to have a Windows installer is unreasonable. Is there a technical reason why Windows users cannot build the installers themselves from pure Python sdists? I would rather distribute *no* binaries at all, myself, especially if self-help works. Stuff which requires a compiler is obviously a barrier for many Windows users: such packages normally need a Windows-savvy contributor to do the installer build, which often lags the 'sdist' release by a noticeable period. Tres. - -- === Tres Seaver +1 540-429-0999 tsea...@palladion.com Palladion Software Excellence by Designhttp://palladion.com -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFJ3i41+gerLs4ltQ4RAkhhAJ9zG5t8gSxVZcMzTCLe3ecNeVzRrwCffAGP f7TDGM1CMxXJtgPIihaZWyw= =zxkr -END PGP SIGNATURE- ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Distutils changes - end user requirements (Was: Deprecate MANIFEST.in)
Tres Seaver schrieb: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Paul Moore wrote: 2009/4/9 Lennart Regebro rege...@gmail.com: 2009/4/9 Paul Moore p.f.mo...@gmail.com: Don't they? I have to admit that I'm baffled by how the features in setuptools/eggs/easy_install all hang together. What about the magic that creates executables from scripts? Entry points? Stuff like that. Don't you need to use eggs to make them work? No? Entry points work even if you have the source code in a tgz format and run setup.py install. The distribution format is not magical for that afaik. So by what you're saying, eggs are a strict subset of bdist_wininst, and so people should be distributing bdist_wininst installers. But they aren't, so what gives? Nobody knows about it? Possibly :-( But in any case, even if it would be a good idea to have every single Python package on the system listed in the Add/Remove programs list (Which I don't think it is, but that's a matter of taste, no logical arguments behind that), that would in practice mean that each and every package on PyPI must have a wininstaller, even if it is a pure-python package. That doesn't seem realistic to me. Personally, I'd be happy if every package that currently distributes any form of Windows binaries, distributed a Windows installer. That's about the same level of coverage as existed before setuptools appeared, so I don't think that's impossible to achieve. I agree that expecting *everything* to have a Windows installer is unreasonable. Is there a technical reason why Windows users cannot build the installers themselves from pure Python sdists? No. There's even a script that automates the process completely. It allows to build bdist_wininst installers by drag and drop. http://code.activestate.com/recipes/117248/ Recipe 117248: installing source distributions on windows Distutil's bdist_wininst installers offer uninstallation support for Python extensions, many developers however only distribute sources in zip or tar.gz format. The typical steps to install such a distribution are: - download the file - unpack with winzip into a temporary directory - open a command prompt and type 'python setup.py install' - remove the temporary directory This script unpacks a source distribution into a temporary directory, builds a windows installer on the fly, executes it, and cleans everything up afterward. Not that it has gained much interest, afaik. -- Thanks, Thomas ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Distutils changes - end user requirements (Was: Deprecate MANIFEST.in)
2009/4/9 Thomas Heller thel...@ctypes.org: Is there a technical reason why Windows users cannot build the installers themselves from pure Python sdists? No. There's even a script that automates the process completely. It allows to build bdist_wininst installers by drag and drop. http://code.activestate.com/recipes/117248/ [...] Not that it has gained much interest, afaik. I, for one, never even knew it existed. That's pretty neat. Paul. ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Distutils changes - end user requirements (Was: Deprecate MANIFEST.in)
Paul Moore wrote: On that note, didn't ActiveState distribute their version of Python with a package manager (PPM)? As far as I know, that was only ever supported by ActiveState themselves, no-one else ever built PPM packages for their extensions, and it's been quietly dropped in recent versions (Google tells me that it vanished around Python 2.3). Yes, it was there a long time ago. It was dropped around ActivePython 2.3, as you say, because it wasn't well architected and, at the time, we didn't have the resources to maintain it. However, we've just recently hired a full time employee to (in part) start looking at a PyPM package manager again -- this time with an appropriate architecture. The hope is to have something to show in the fall (perhaps sooner). Trent -- Trent Mick trentm at activestate.com ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Distutils changes - end user requirements (Was: Deprecate MANIFEST.in)
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Paul Moore wrote: 2009/4/9 Tres Seaver tsea...@palladion.com: Is there a technical reason why Windows users cannot build the installers themselves from pure Python sdists? I would rather distribute *no* binaries at all, myself, especially if self-help works. Stuff which requires a compiler is obviously a barrier for many Windows users: such packages normally need a Windows-savvy contributor to do the installer build, which often lags the 'sdist' release by a noticeable period. No technical reason, no. It's as simple as python setup.py bdist_wininst or python setup.py mdist_msi. Personally, I'm happy doing that for any pure python package that doesn't provide an installer. The only downside is that not all packages document whether they are pure Python. It can be frustrating to download a package, unpack it, and try to build it only to find out that it has C code that won't build. Or even more subtle, it builds fine, but ignores important speedup code written in C... But the main reason is social - Windows users expect to download installers, and have a low tolerance for projects that don't provide such. And a low tolerance for anything involving a command line, in many cases. Call us bone idle if you must, but it's a fact you need to deal with in considering a Windows audience. *Shrug*. I mostly don't care about donating extra effort to help users who refuse to help themselves, or to contribute to the product. *That* isn't going to change, either. I've been willing to make exceptoins in the past for crucial packages with C extensions, but anything that requires booting to Windows is mostly not going to happen. However, it's equally true (I believe) that python setup.py bdist_wininst works fine on a Linux box. So it's not as if building Windows installers is a huge chore for developers, either. (I accept that there are other tasks, like distribution). It's a trade-off of developer time vs user time (and I fully accept that this trade-off comes out differently in an open source/volunteer environment). I don't think that works: $ ../../bin/python setup.py bdist_wininst running bdist_wininst running build installing to build/bdist.linux-i686/wininst running install_egg_info running egg_info writing pkginfo.egg-info/PKG-INFO writing top-level names to pkginfo.egg-info/top_level.txt writing dependency_links to pkginfo.egg-info/dependency_links.txt reading manifest file 'pkginfo.egg-info/SOURCES.txt' writing manifest file 'pkginfo.egg-info/SOURCES.txt' Copying pkginfo.egg-info to build/bdist.linux-i686/wininst/PURELIB/pkginfo-0.1-py2.6.egg-info running install_scripts creating '/tmp/tmpJ5kgq8.zip' and adding '.' to it adding 'PURELIB/pkginfo-0.1-py2.6.egg-info/dependency_links.txt' adding 'PURELIB/pkginfo-0.1-py2.6.egg-info/SOURCES.txt' adding 'PURELIB/pkginfo-0.1-py2.6.egg-info/top_level.txt' adding 'PURELIB/pkginfo-0.1-py2.6.egg-info/PKG-INFO' creating dist Warning: Can't read registry to find the necessary compiler setting Make sure that Python modules _winreg, win32api or win32con are installed. error: /home/tseaver/projects/Zope-CVS/lib/python2.6/distutils/command/wininst-6.0ux-i686.exe: No such file or directory Note that this package is pure python. Tres. - -- === Tres Seaver +1 540-429-0999 tsea...@palladion.com Palladion Software Excellence by Designhttp://palladion.com -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFJ3lRf+gerLs4ltQ4RAjwkAJ4kSb++3dwISpZWoFTWy3ymPfHdMACeMZ42 BzuguG6IxSvFV4Q8Lvh8I98= =dbEz -END PGP SIGNATURE- ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Distutils changes - end user requirements (Was: Deprecate MANIFEST.in)
2009/4/9 Tres Seaver tsea...@palladion.com: However, it's equally true (I believe) that python setup.py bdist_wininst works fine on a Linux box. So it's not as if building Windows installers is a huge chore for developers, either. (I accept that there are other tasks, like distribution). It's a trade-off of developer time vs user time (and I fully accept that this trade-off comes out differently in an open source/volunteer environment). I don't think that works: $ ../../bin/python setup.py bdist_wininst [...] Warning: Can't read registry to find the necessary compiler setting Make sure that Python modules _winreg, win32api or win32con are installed. error: /home/tseaver/projects/Zope-CVS/lib/python2.6/distutils/command/wininst-6.0ux-i686.exe: No such file or directory Note that this package is pure python. Hmm, I thought that worked. I'll see if I can build a suitable Linux VM to give it a try. Just in case it matters, what OS are you running? Paul. ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Distutils changes - end user requirements (Was: Deprecate MANIFEST.in)
2009/4/9 Tres Seaver tsea...@palladion.com: creating dist Warning: Can't read registry to find the necessary compiler setting Make sure that Python modules _winreg, win32api or win32con are installed. This is a warning, so can be ignored. error: /home/tseaver/projects/Zope-CVS/lib/python2.6/distutils/command/wininst-6.0ux-i686.exe: No such file or directory This is a bug. That ux-i686 part of the filename looks wrong. Yes, it's a bug in bdist_wininst, introduced in revision 62197, Mon Apr 7 01:53:39 2008 UTC. If I file a bug and it gets fixed, will you build installers for me? :-) (Seriously, thanks for trying this and locating the bug. I've raised it as http://bugs.python.org/issue5731). Paul. ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Distutils changes - end user requirements (Was: Deprecate MANIFEST.in)
On Thu, Apr 9, 2009 at 10:52 PM, Paul Moore p.f.mo...@gmail.com wrote: 2009/4/9 Tres Seaver tsea...@palladion.com: creating dist Warning: Can't read registry to find the necessary compiler setting Make sure that Python modules _winreg, win32api or win32con are installed. This is a warning, so can be ignored. error: /home/tseaver/projects/Zope-CVS/lib/python2.6/distutils/command/wininst-6.0ux-i686.exe: No such file or directory This is a bug. That ux-i686 part of the filename looks wrong. Yes, it's a bug in bdist_wininst, introduced in revision 62197, Mon Apr 7 01:53:39 2008 UTC. If I file a bug and it gets fixed, will you build installers for me? :-) (Seriously, thanks for trying this and locating the bug. I've raised it as http://bugs.python.org/issue5731). Great thanks, I'll add a test and fix it right now, so this discussion can continue, and distutils trunk will get some beta testers ;) Paul. ___ Distutils-SIG maillist - distutils-...@python.org http://mail.python.org/mailman/listinfo/distutils-sig -- Tarek Ziadé | Association AfPy | www.afpy.org Blog FR | http://programmation-python.org Blog EN | http://tarekziade.wordpress.com/ ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Distutils changes - end user requirements (Was: Deprecate MANIFEST.in)
On Apr 9, 2009, at 3:37 AM, Paul Moore wrote: I personally, as a consumer of other people's software, prefer to acquire their packages as sdist .tar.gz's or as .eggs on all platforms, including Windows, which I use regularly. So your and my preferences as a consumer of packages differ on this. ... If I have to guess at your reasons, my assumption would be: - Consistent means of installing on all platforms This is a big one. Another is that try to use command-line interfaces instead of GUIs whenever possible. I can, for example, script installation and run the same script on all my platforms, or schedule the script on all my platforms using buildbot. Another is that I can uninstall with rm, which I greatly prefer over the Windows Add/Remove programs interface. Another is that if I easy_install a *different* package which requires that package, then the .egg can be automatically installed with no manual effort on my part. Regards, Zooko ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Distutils changes - end user requirements (Was: Deprecate MANIFEST.in)
If I file a bug and it gets fixed, will you build installers for me? :-) (Seriously, thanks for trying this and locating the bug. I've raised it as http://bugs.python.org/issue5731). fixed in the trunk, r71413, I am now backporting it in the 2.6 maintenance branch, but I'd appreciate any feeback on Distutils current trunk Cheers Tarek ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Distutils changes - end user requirements (Was: Deprecate MANIFEST.in)
On Thu, Apr 09, 2009 at 10:24:18AM +0100, Paul Moore wrote: 2009/4/9 Lennart Regebro rege...@gmail.com: On Wed, Apr 8, 2009 at 12:27, Paul Moore p.f.mo...@gmail.com wrote: 1. (Meta-requirement) I want to be able to download a Windows installer[1] for *every* package I need. I would like to know why you have this requirement. Some sort of Windows binary, absolutely. But an installer? Would a package management GUI be enough? A Windows software that can install (and remove) eggs? Because I like to see my installed Python packages in Windows' Add/Remove Programs list. But that list is for *programs*, not *libraries*. I'd understand a wish that a Python app (that also depends on twenty other Python libraries) be bundled as a single Windows installer (that contains all those twenty libraries inside) for an easy next-next-next-finish installation. I don't understand a desire of downloading twenty .exe files and clicking on them all to get an app running. Did I misunderstand something? Marius Gedminas -- Added mysterious, undocumented --scanflags and --fuzzy options. -- nmap 3.0 announcement signature.asc Description: Digital signature ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Distutils changes - end user requirements (Was: Deprecate MANIFEST.in)
Paul Moore wrote: If I have to guess at your reasons, my assumption would be: - Consistent means of installing on all platforms - ??? I can imagine that people for whom Windows is not their primary platform might like to be able to install Python packages the same way they do on other platforms. There's less mental gear switching involved that way when moving from one platform to another. -- Greg ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Distutils changes - end user requirements (Was: Deprecate MANIFEST.in)
Paul Moore wrote: Personally, I'd be happy if every package that currently distributes any form of Windows binaries, distributed a Windows installer. ... I agree that expecting *everything* to have a Windows installer is unreasonable. Would it be feasible to have a generic installer for .egg files that does whatever a real Windows installer would have done? Distribute it with Python and make it launch when you double-click a .egg file. -- Greg ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Distutils changes - end user requirements (Was: Deprecate MANIFEST.in)
Marius Gedminas wrote: But that list is for *programs*, not *libraries*. The difference starts to be blurry for python (is numpy an application or a library ?), and it does not make much sense for windows users, I think. I don't understand a desire of downloading twenty .exe files and clicking on them all to get an app running. Did I misunderstand something? I think it is related to several issues: - no uninstall with eggs - .exe is a known thing for any windows user, .egg isn't - you can't easily get the dependencies, download them separately, and install this on another computer One thing which would be nice too is something to bundle several python packages together, a bit like pip freeze does, but to build one single installer. For all those reasons, having a commonly accepted, low-level set of data that can be fed/obtained from any tool is crucial. I don't see any other way to allow different tools to be built reliably, depending on the platform and/or goal. cheers, David ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Distutils changes - end user requirements (Was: Deprecate MANIFEST.in)
On 4/8/09 6:27 AM, Paul Moore wrote: 2009/4/8 Tarek Ziadé ziade.ta...@gmail.com: Nobody seems to be attempting to collect requirements here. I do, in the wiki. I am trying to synchronize the work done at Pycon, and in the future. I am trying to synthethize the needs there. http://wiki.python.org/moin/DistutilsVersionFight http://wiki.python.org/moin/Distutils/ManifestPluginSystem http://wiki.python.org/moin/Distutils/Metadata http://wiki.python.org/moin/Distutils/StaticMetadata http://wiki.python.org/moin/Distutils/StandardizeEggInfo Perhaps it would be a worthwhile effort to create a DistUtilsRequirements page on the Wiki? I think we all agree that the current situation needs help, but what is in scope for distutils and what needs to be supplied via plugins and third-party tools? I'm sure for many of you this seems obvious, but I think the flame wars are in part due to differing understandings of the goal. Off hand, here are some of the requirements I perceive for the packaging ecosystem on the whole. I do not think all of them should be met by distutils proper, but the overall ecosystem of python packaging tools needs to provide solutions for all of them. For users: 1. Simple package installation following the conventions of their platform. IE apt-get, aptitude, yum, synaptic, ports, msi. 2. Simple package installation following python conventions. IE easy_install, pip or whatever activestates thingy was called. 3. Simple installation including necessary dependencies. IE easy_install or apt-get. Raw rpm, deb, or msi installs don't follow this. 4. Simple application installation or deployment following both the system managed and application managed models Tarek alluded to. For distribution packagers: 1. Dependency meta-data. I can't make packages for appA that work correctly without knowing that it depends on libB version 1.0 or greater. 2. Declarative meta-data in the sdist. I need to be able to unpack an sdist file and pull out all the information i need to build packages without connecting to your dvcs or installing a bunch of third party stuff. 3. Rational versioning. I need to know how to express python package versions in a way I can map it's version number to the versioning scheme I use. IE RPM, Deb, PyPi. 4. Build tools. I don't want to become an expert on cython, swig, setuptools, or anything else. I want a standard mechanism to build packages regardless of their complexity. Distutils bdist sorta fits the bill here. 5. Reproducible sdists. I want to be able to take your package, make a small change, and rebuild the package without having to connect up to your vcs to do it. For package authors: 1. Easy builds for my own use and distribution. IE create setup.py, ./setup.py bdist_rpm, use the package! I get the impression setuptools is the more modern way to do this, but I must claim ignorance, personally. 2. Easy integration with VCS so I don't forget to include newly added files etc. Nobody wants to manually update MANIFEST, or MANIFEST.in, if the information needed is already in VCS. 3. Support for building C extensions and the like. Distutils covers this pretty well now, I think. 4. Support for including run scripts, hacking documentation, user documentation, support data files, etc. These need to be flagged in the metadata so distribution packages know what's what. I'm sure there's a lot more. If there's interest, I'll be happy to try to consolidate others contributions and clarifications and put it on the Wiki. Thanks --erik ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Distutils changes - end user requirements (Was: Deprecate MANIFEST.in)
On Apr 8, 2009, at 4:27 AM, Paul Moore wrote: 1. (Meta-requirement) I want to be able to download a Windows installer[1] for *every* package I need. 1a. This means that the barrier for packagers building Windows installers should be as low as possible. 1b. It also means that other formats (e.g. eggs) should offer no benefit over Windows installers I personally, as a consumer of other people's software, prefer to acquire their packages as sdist .tar.gz's or as .eggs on all platforms, including Windows, which I use regularly. So your and my preferences as a consumer of packages differ on this. Fortunately, as far as the distutils is concerned Windows installers vs. sdists vs. eggs are not mutually exclusive. The distutils will provide a standard for metadata so that all of those distribution formats can have the same metadata, and it will allow automation such as bdist_msi and sdist_dsc (from stdeb) so that it can become easier for developers to produce these things. By the way, the exact same sorts of preference-among-consumers issues arises on the Linux side (which I also use regularly), where some people mistakenly think that they want distutils to support eggs worse, when what they really want is for it to support debs better (which I hope stdeb will ultimately do). Regards, Zooko ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Distutils changes - end user requirements (Was: Deprecate MANIFEST.in)
On Wed, Apr 8, 2009 at 12:27, Paul Moore p.f.mo...@gmail.com wrote: 1. (Meta-requirement) I want to be able to download a Windows installer[1] for *every* package I need. I would like to know why you have this requirement. Some sort of Windows binary, absolutely. But an installer? Would a package management GUI be enough? A Windows software that can install (and remove) eggs? 1a. This means that the barrier for packagers building Windows installers should be as low as possible. Agreed. 1b. It also means that other formats (e.g. eggs) should offer no benefit over Windows installers Well... they don't, do they? Except that they can be easy_installed, right? -- Lennart Regebro: Pythonista, Barista, Notsotrista. http://regebro.wordpress.com/ +33 661 58 14 64 ___ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig
