[Distutils] Isolation, virtualenvs and buildout
Hi, I'm used to virtualenvs and I think I understood the general idea of build out. I need to build a pice of software out of several independent Python packages and I think buildout is the tool to use. But I don't get how buildout and virtualenv are related to each other. Asking Google, I found some hints that buildout is supposed to provide the same kind of isolation as virtualenvs - but only in the old 1.x version. The new(?) 2.0 version will change that. So I'm totally confused and ask for help. My use case is simple: I have some python modules on our private pypi server and would like to deploy a pice of software composed out of them (including all dependencies). Any starting guidance would be very appreciated! cheers, Achim ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Isolation, virtualenvs and buildout
On Mon, Nov 11, 2013 at 10:26 AM, Achim Domma do...@procoders.net wrote: Hi, I'm used to virtualenvs and I think I understood the general idea of build out. I need to build a pice of software out of several independent Python packages and I think buildout is the tool to use. But I don't get how buildout and virtualenv are related to each other. Asking Google, I found some hints that buildout is supposed to provide the same kind of isolation as virtualenvs - but only in the old 1.x version. The new(?) 2.0 version will change that. So I'm totally confused and ask for help. Buildout1 tried to provide isolation, but it was incomplete and too hard to maintain. virtualenv works crazy hard to provide isolation. (/me applauds virtualenv.) Many people run buildout inside virtualenvs. Personally, I always keep a unadulterated Python around that I run buildout with. Buildouts main contribution is that it doesn't install things into site packages and won't mess up your clean Python or your virtualenv. You could use virtualenv to make yourself a clean Python and run any number of buildouts with that virtualenv. My use case is simple: I have some python modules on our private pypi server and would like to deploy a pice of software composed out of them (including all dependencies). That's similar to our use cases. Any starting guidance would be very appreciated! FWIW, here's a script I use to make RPMs out of buildouts: https://gist.github.com/jimfulton/6629791 It builds on zc.sourcerelease, which builds self-contained source tar balls. We (ZC) deploy *software* using RPM and use other buildout based tools to deploy application configuration. I suspect in the future we'll to docker or maybe something simpler than RPM, which doesn't really match our use cases very well and tends to cause us no end of pain. Jim -- Jim Fulton http://www.linkedin.com/in/jimfulton ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
[Distutils] setuptools and use_2to3
I'm trying to convert a setup.py to use setuptools (specifically the docutils build, FWIW). The existing setup.py has some custom command stuff to run 2to3 on build. As I understand it, setuptools should do this automatically by using use_2to3=True. So, I've changed the code to just do: setup( ... packages=find_packages(), use_2to3=True, ... ) However, when I do python setup.py bdist_wheel, I find that docutils/__init__.py within the wheel still contains (for example) class ApplicationError(StandardError): ... So the StandardError - Exception fixer isn't being run. Am I doing something wrong here? The setuptools documentation isn't very clear on what I should do, or what to expect. Thanks, Paul ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] setuptools and use_2to3
On Mon, Nov 11, 2013 at 04:29:32PM +, Paul Moore wrote: I'm trying to convert a setup.py to use setuptools (specifically the docutils build, FWIW). The existing setup.py has some custom command stuff to run 2to3 on build. Aside: I believe there's consensus now that 2to3 is not the best way to support Python 2.x and 3.x. It's not that hard to have a single source tree in a portable subset of Python 2.x and 3.x. See python3porting.com and https://pypi.python.org/pypi/six. As I understand it, setuptools should do this automatically by using use_2to3=True. So, I've changed the code to just do: setup( ... packages=find_packages(), use_2to3=True, ... ) However, when I do python setup.py bdist_wheel, I find that docutils/__init__.py within the wheel still contains (for example) class ApplicationError(StandardError): ... So the StandardError - Exception fixer isn't being run. Am I doing something wrong here? The setuptools documentation isn't very clear on what I should do, or what to expect. I wouldn't be surprised to learn that bdist_wheel doesn't support use_2to3, but I don't actually *know* that for a fact, since I never used 2to3. Marius Gedminas -- What goes up, must come down. Ask any system administrator. signature.asc Description: Digital signature ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] setuptools and use_2to3
On 11 November 2013 17:26, Marius Gedminas mar...@pov.lt wrote: On Mon, Nov 11, 2013 at 04:29:32PM +, Paul Moore wrote: I'm trying to convert a setup.py to use setuptools (specifically the docutils build, FWIW). The existing setup.py has some custom command stuff to run 2to3 on build. Aside: I believe there's consensus now that 2to3 is not the best way to support Python 2.x and 3.x. It's not that hard to have a single source tree in a portable subset of Python 2.x and 3.x. See python3porting.com and https://pypi.python.org/pypi/six. Yes, I realise that. I'm not a fan of 2to3 at all myself, but it's what docutils uses and I'm not trying to do a full docutils port to shared source, just a quick local fork of setup.py to allow me to make some simple changes (using entry points rather than custom scripts, in particular). I only started looking at the 2to3 stuff because docutils' setup.py does some complex distutils customisation that I thought might give setuptools some problems. And as all it was doing was implementing things that are built into setuptools, it seemed worth looking at as a simplification. I wouldn't be surprised to learn that bdist_wheel doesn't support use_2to3, but I don't actually *know* that for a fact, since I never used 2to3. It's possible I guess. But I thought bdist_wheel used the normal build command, so stuff like 2to3 support should be unchanged. Thanks for the comments anyway, Paul ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] Isolation, virtualenvs and buildout
Achim Domma domma at procoders.net writes: Any starting guidance would be very appreciated! As of 2.0, Buildout no longer provides isolation so you should do something like: $ bin/virtualenv . $ bin/pip install zc.buildout $ bin/buildout init Then start writing your buildout.cfg. ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] setuptools and use_2to3
On Mon, Nov 11, 2013 at 6:26 PM, Marius Gedminas mar...@pov.lt wrote: Aside: I believe there's consensus now that 2to3 is not the best way to support Python 2.x and 3.x. It's not that hard to have a single source tree in a portable subset of Python 2.x and 3.x. See python3porting.com and https://pypi.python.org/pypi/six. Well, that depends. Mainly on what Python versions you need to support. ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
Re: [Distutils] setuptools and use_2to3
On 11 November 2013 20:45, Lennart Regebro rege...@gmail.com wrote: If 2to3 is being run you'll get quite a lot of fairly obvious output to that effect. So it sounds like it's not. A quick look at bdist_egg seems to indicate that it never runs the build_py step, which is what runs 2to3. So this may be a bug. Good thought. python setup.py build does run 2to3, and as you say the output is not something you can miss. So setup.py bdist_wheel isn't running build, which is likely a bug (but also not hard to work around). That's probably good enough for now - this whole exercise is a local hack, so just getting it to work is likely good enough. I'm not enough of a fan of the whole 2to3 process to be bothered spending time producing a reproducible test case, etc, for a good-quality bug report. I might just report it as it stands and leave it at that, though, so at least it doesn't get forgotten. Thanks. Paul ___ Distutils-SIG maillist - Distutils-SIG@python.org https://mail.python.org/mailman/listinfo/distutils-sig
