Re: Question about private modules in /usr/share
Hi Tianon. Thanks for the reply. On 21 Apr 2015, at 2:41 pm, Tianon Gravi admwig...@gmail.com wrote: On 20 April 2015 at 20:21, Potter, Tim (Cloud Services) timothy.pot...@hp.com wrote: It looks pretty good, but I think I have messed up something as my binary gives an import error since /usr/share/dwarf isn’t in the PYTHONPATH: # dwarf Traceback (most recent call last): File /usr/bin/dwarf, line 32, in module from dwarf import log # pylint: disable=W0611 ImportError: No module named dwarf I'm not an expert by any means, but it's been my experience that the more you trust Piotr's pybuild code to DTRT, the more it does. :) I cloned your package locally and removed both override_dh_* sections from debian/rules and deleted debian/links and it successfully created a package with just pybuild, including installing dwarf and dwarf-manage in /usr/bin. Is there a specific reason to override what pybuild is doing? Apparently that’s the difference between a Python app and a Python module. Also I did that because the wiki page said to. (-: Maybe someone else will jump in and explain the difference here. Here's what I get after removing those sections, building, installing, and testing the result: | # dwarf | libvirt: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock': No such file or directory This seems to be working properly (there's no ImportError), but I admit I'm not at all familiar with upstream here. :) Nice. That does look like it’s working much better. Tim. ♥, - Tianon 4096R / B42F 6819 007F 00F8 8E36 4FD4 036A 9C25 BF35 7DD4 smime.p7s Description: S/MIME cryptographic signature
Re: Question about private modules in /usr/share
[Potter, Tim (Cloud Services), 2015-04-21] binary gives an import error since /usr/share/dwarf isn’t in the PYTHONPATH: # dwarf Traceback (most recent call last): File /usr/bin/dwarf, line 32, in module from dwarf import log # pylint: disable=W0611 ImportError: No module named dwarf binary and module name are the same so distutils decided to copy script to module dir: | running install_scripts | copying build/scripts-2.7/dwarf - debian/dwarf/usr/share/dwarf/dwarf | copying build/scripts-2.7/dwarf-manage - debian/dwarf/usr/share/dwarf mv debian/dwarf/usr/share/dwarf/dwarf debian/dwarf/usr/share/dwarf/run in your install override, update symlink in debian/links and it should work -- Piotr Ożarowski Debian GNU/Linux Developer www.ozarowski.pl www.griffith.cc www.debian.org GPG Fingerprint: 1D2F A898 58DA AF62 1786 2DF7 AEF6 F1A2 A745 7645 -- To UNSUBSCRIBE, email to debian-python-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/20150421090124.gp2...@sar0.p1otr.com
Re: Question about private modules in /usr/share
On 21 Apr 2015, at 7:01 pm, Piotr Ożarowski pi...@debian.org wrote: [Potter, Tim (Cloud Services), 2015-04-21] binary gives an import error since /usr/share/dwarf isn’t in the PYTHONPATH: # dwarf Traceback (most recent call last): File /usr/bin/dwarf, line 32, in module from dwarf import log # pylint: disable=W0611 ImportError: No module named dwarf binary and module name are the same so distutils decided to copy script to module dir: | running install_scripts | copying build/scripts-2.7/dwarf - debian/dwarf/usr/share/dwarf/dwarf | copying build/scripts-2.7/dwarf-manage - debian/dwarf/usr/share/dwarf mv debian/dwarf/usr/share/dwarf/dwarf debian/dwarf/usr/share/dwarf/run in your install override, update symlink in debian/links and it should work Hi Piotr. Thanks a lot! I figured it out based on your comments. Tim. smime.p7s Description: S/MIME cryptographic signature
Question about private modules in /usr/share
Hi everyone. I recently filed in ITP (#782988) and had a stab at packaging the app based on the instructions at https://wiki.debian.org/Python/Packaging. It looks pretty good, but I think I have messed up something as my binary gives an import error since /usr/share/dwarf isn’t in the PYTHONPATH: # dwarf Traceback (most recent call last): File /usr/bin/dwarf, line 32, in module from dwarf import log # pylint: disable=W0611 ImportError: No module named dwarf I think I’ve followed the instructions correctly but I don’t quite understand the reasoning for step 5) renaming the script if the module has exactly the same name. My packaging so far is in a git-buildpackage compatible repo at https://anonscm.debian.org/cgit/users/tpot-guest/dwarf.git/ Any tips are much appreciated! Tim. smime.p7s Description: S/MIME cryptographic signature
Re: Question about private modules in /usr/share
On 20 April 2015 at 20:21, Potter, Tim (Cloud Services) timothy.pot...@hp.com wrote: It looks pretty good, but I think I have messed up something as my binary gives an import error since /usr/share/dwarf isn’t in the PYTHONPATH: # dwarf Traceback (most recent call last): File /usr/bin/dwarf, line 32, in module from dwarf import log # pylint: disable=W0611 ImportError: No module named dwarf I'm not an expert by any means, but it's been my experience that the more you trust Piotr's pybuild code to DTRT, the more it does. :) I cloned your package locally and removed both override_dh_* sections from debian/rules and deleted debian/links and it successfully created a package with just pybuild, including installing dwarf and dwarf-manage in /usr/bin. Is there a specific reason to override what pybuild is doing? Here's what I get after removing those sections, building, installing, and testing the result: | # dwarf | libvirt: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock': No such file or directory This seems to be working properly (there's no ImportError), but I admit I'm not at all familiar with upstream here. :) ♥, - Tianon 4096R / B42F 6819 007F 00F8 8E36 4FD4 036A 9C25 BF35 7DD4 -- To UNSUBSCRIBE, email to debian-python-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: https://lists.debian.org/cahnknk11ikxumq2psaoyyufmuh8dkf6musepwveocwe+06u...@mail.gmail.com
Re: Private modules
On Dec 24, 2012, at 12:43 AM, Dmitrijs Ledkovs wrote: The way I interpreted Paul's comment is that it implies don't use virtualenv inside the .deb package as to be distributed by Debian e.g. system-wide python packages should not be using virtualenv environment out of the box on Debian, as that opens a can of worms. I would definitely agree with that. -Barry signature.asc Description: PGP signature
Re: Private modules
On 22 December 2012 22:00, Bas Wijnen wij...@debian.org wrote: 6. Make /usr/bin/program a symlink to the actual file in the private directory. It will then search in its real place. (I've seen this used by angrydd.) This (symlinking /usr/bin/program) appears to be the recommended way to deal with it: http://wiki.debian.org/Python/Packaging#Example_2:_Python_application http://permalink.gmane.org/gmane.linux.debian.devel.python/6728 Thomas
Re: Private modules
On Sat, Dec 22, 2012 at 04:27:46PM -0500, Barry Warsaw wrote: On Dec 22, 2012, at 05:19 PM, Paul Tagliamonte wrote: Yeah, please don't use virtualenv, as much as I'd like to see a good way of using virtualenv in Debian. Can you expand on that? It should be usable to develop code, but do you mean something else? This really isn't the place for this, so if we could take this elsewhere, that'd be nice, but, given that we don't solve the problem virtualenv fixes, it'd be nice to support this in Debian. naturally, I don't care enough to fix it, but I'd imagine a system where we create a virtualenv by symlinking libraries into virtualenvs for each installed Python application, to allow us to co-install two versions of the same library for two different applications. Anyway, let's take this off list, it's a silly conversation we don't need to have :) Cheers, -Barry -- .''`. Paul Tagliamonte paul...@debian.org : :' : Proud Debian Developer `. `'` 4096R / 8F04 9AD8 2C92 066C 7352 D28A 7B58 5B30 807C 2A87 `- http://people.debian.org/~paultag signature.asc Description: Digital signature
Re: Private modules
On 22 December 2012 23:27, Barry Warsaw ba...@python.org wrote: On Dec 22, 2012, at 05:19 PM, Paul Tagliamonte wrote: Yeah, please don't use virtualenv, as much as I'd like to see a good way of using virtualenv in Debian. Can you expand on that? It should be usable to develop code, but do you mean something else? The way I interpreted Paul's comment is that it implies don't use virtualenv inside the .deb package as to be distributed by Debian e.g. system-wide python packages should not be using virtualenv environment out of the box on Debian, as that opens a can of worms. (virtualenv does nothing to help to what a coherent distribution tries to achieve). I do agree with you that one should not break vritualenv on purpose for those people who wish to use it. Regards, Dmitrijs. -- To UNSUBSCRIBE, email to debian-python-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/CANBHLUjiu=RfsBvaj-Q+11yQhFngZMhoPQh=6KFAwk=op4y...@mail.gmail.com
Re: Private modules
On Sat, Dec 22, 2012 at 11:00:35PM +0100, Bas Wijnen wrote: Hello, I have a Python program (of which I am upstream) and I want to package it. However, it has some private modules and I don't know how to treat them. Section 3.1.1 of the Python Packaging Policy says they should be in /usr/share/. This makes sense, being private data of the package.[1] However, this doesn't seem to be how Python programs normally work. I asked on IRC (freenode:#python), and the response was that Python programs don't have the concept of private modules. Any module, no matter how program-specific, is installed publicly. This feels wrong for me, and not the Debian Way. Remember, binaries can see local modules - if you drop the program into /usr/share as well, and link it into /usr/bin, it won't be system wide, and it will be importable -- so, there is this concept (sorta). This wouldn't be a problem in itself, but I need support from the Python system to make things work: my program needs to import my private module, which means its directory must be in sys.path. But how do I get it there? I see several options, none of which seems really good: 1. Hard-code the directory in the program. + It works. - It will then only work on Debian - It cannot run from a different path. 2. Let the upstream program assume that the module is in the search path (either installed publicly or in the same directory as the script). Use a debian patch to add Debian's private path to it. + It works. + The upstream script works for non-Debian people. - The patch may not apply cleanly in a next version, so it slightly increases maintainance burden. - The packaged program is not the upstream version (is that actually a problem?) 3. Create a module for handling this. I import the module and call a function on it providing my package name. It will add the required directories to sys.path. + This would be great, if it worked, but: - Othes operating systems don't have this module, so even though the approach is intended to make it more portable, it actually makes it Debian-specific. 4. Virtualenv was suggested on IRC. I'm not sure how I should use it; it seems like a tool for users with little permissions on the system, not for a package which can access anything. But I might be wrong, so I wanted to mention it. Yeah, please don't use virtualenv, as much as I'd like to see a good way of using virtualenv in Debian. + ? - ? 5. Create a wrapper executable script which just runs the program from /usr/share. Have the private module in the same directory as the program. (I've seen this used by freealchemist.) + It works. - It's a clumsy workaround. 6. Make /usr/bin/program a symlink to the actual file in the private directory. It will then search in its real place. (I've seen this used by angrydd.) + It works. + It's not as clumsy. - It still seems to be a workaround. - It is not obvious to a user who copies the file and changes it, why the copy doesn't find its modules. Any thoughts on this are welcome. Thanks, Bas Ps: please Cc me on replies, I'm not subscribed to this list. (And keep these lines, so I alse get replies to replies.) [1] Actually, it says it should go in /usr/share/modulename/, while I would expect it to go in /usr/share/packagename/.../modulename/. Am I misreading this? In particular, should package foo (with private module bar) really create /usr/share/bar? -- .''`. Paul Tagliamonte paul...@debian.org : :' : Proud Debian Developer `. `'` 4096R / 8F04 9AD8 2C92 066C 7352 D28A 7B58 5B30 807C 2A87 `- http://people.debian.org/~paultag signature.asc Description: Digital signature
Re: private modules and dh_python2
On Jun 14, 2011, at 12:02 AM, Piotr Ożarowski wrote: it is fine and it is useful... as a submodule, not as a top-level module Agreed! I think I get what you're driving at now. Some applications don't put their Python code or tests in a package. In those cases, yes by all means a private package makes complete sense. [...] you realize that setuptools/distribute hardcodes versions and forces you to depend on python-setuptools/python-pkg-resources, right? In the context of Debian, what are the practical problems of this? it's 217k of unneeded data that can be easily avoided and few CPU cycles that can be spared anyway, I always say to my sponsorees: if it's not useful outside this application, make it private and not pollute the global namespace. It always can be promoted to public one later Agreed. For applications that do it correctly, I think it's fine for their support package to be public. The principle is: Don't pollute the global namespace! -Barry signature.asc Description: PGP signature
Re: private modules and dh_python2
As an aside, I notice that Debian Python Policy begins by assuming people know
what private Python modules are, but this will not be the case for most
Python developers unfamiliar with Debian conventions. Section 2.1 briefly
mentions the distinction from a sys.path-visibility point of view, which
doesn't really explain much. I'd like to see a section early on which defines
private modules, where, when, why and how they should be used, etc.
On Jun 10, 2011, at 10:13 PM, Piotr Ożarowski wrote:
[Barry Warsaw, 2011-06-10]
Ah, yeah. Y'know, I am personally not a fan of private modules anyway :).
/me waits till Barry will try to package his 13th package with
Python application that uses lib or tests module names...
(or will he break after 4th? Bets anyone? ;)
I know this has been discussed before, and there's even debate about it
upstream, so apologies for rehashing it. My own personal opinion is that it's
fine to include things like a `test` (Python) subpackage in the (Debian)
package python-foo. It aligns with the Python consenting adults motto, and
I think such things *can* be useful. As long as the top-level package name is
unique, subpackage can't pollute the global namespace, so I don't care.
Note too in a setuptools(?)/distribute/packaging world, you might not
actually have a `foo` script in the source at all. For example, Mailman 3
has this in its setup.py:
template = Template('$script = mailman.bin.$script:main')
scripts = set(
template.substitute(script=script)
for script in ('mailman', 'runner', 'master', 'onebounce')
)
setup(
...
entry_points= {
'console_scripts' : list(scripts),
},
...)
so in fact /usr/bin/mailman doesn't exist until the system is built out.
/usr/bin/mailman gets generated and essentially imports the
`mailman.bin.mailman` module, then runs the main() function.
you realize that setuptools/distribute hardcodes versions and forces you
to depend on python-setuptools/python-pkg-resources, right?
In the context of Debian, what are the practical problems of this?
-Barry
signature.asc
Description: PGP signature
Re: private modules and dh_python2
[Barry Warsaw, 2011-06-13] it's fine to include things like a `test` (Python) subpackage in the (Debian) package python-foo. It aligns with the Python consenting adults motto, and I think such things *can* be useful. As long as the top-level package name is unique, subpackage can't pollute the global namespace, so I don't care. it is fine and it is useful... as a submodule, not as a top-level module [...] you realize that setuptools/distribute hardcodes versions and forces you to depend on python-setuptools/python-pkg-resources, right? In the context of Debian, what are the practical problems of this? it's 217k of unneeded data that can be easily avoided and few CPU cycles that can be spared anyway, I always say to my sponsorees: if it's not useful outside this application, make it private and not pollute the global namespace. It always can be promoted to public one later -- Piotr Ożarowski Debian GNU/Linux Developer www.ozarowski.pl www.griffith.cc www.debian.org GPG Fingerprint: 1D2F A898 58DA AF62 1786 2DF7 AEF6 F1A2 A745 7645 -- To UNSUBSCRIBE, email to debian-python-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20110613220226.ga4...@piotro.eu
private modules and dh_python2
Hi all, I just tried to package an application using a private module. In this case, the name of the script starting the application and the module have the same name. So if the module is in /usr/share/foo/foo, then the script can not be /usr/share/foo/foo as well and installing the script to /usr/share/foo/scripts/foo results in an import error. How should this be best handled? 1) Patch the upstream script to add /usr/share/foo to pythonpath if 'import foo' fails? 2) In this specific case (which might actually not be that uncommon), dh_python2 could rename the script foo to foo.py such that it can be installed to /usr/share/foo/foo.py (currently, the behavior of dh_python2 is to install the script to /usr/share/foo/foo/foo in this case of a naming clash). 3) Module and script having the same name is bad practice and the module should be renamed to foo-lib anyway. Thanks for any comments, Eike pgpSirwkYFb4o.pgp Description: PGP signature
Re: private modules and dh_python2
On Jun 10, 2011, at 09:01 PM, Eike Nicklas wrote: I just tried to package an application using a private module. In this case, the name of the script starting the application and the module have the same name. Is the script private too? Wouldn't that be better installed in /usr/bin/foo? -Barry signature.asc Description: PGP signature
Re: private modules and dh_python2
Hi Barry, thanks for the quick answer. On Fri, 10 Jun 2011 15:34:19 -0400 Barry Warsaw wrote: On Jun 10, 2011, at 09:01 PM, Eike Nicklas wrote: I just tried to package an application using a private module. In this case, the name of the script starting the application and the module have the same name. Is the script private too? Wouldn't that be better installed in /usr/bin/foo? Then 'import foo' fails if '/usr/share/foo/foo' is not explicitly added to pythonpath (that was the idea of having the module private in the first place ;-) ) As I understand http://wiki.debian.org/Python/Packaging#Example_2:_Python_application the idea is to install both script and module to /usr/share/foo where the script can 'locally' import the module and link /usr/bin/foo to the script. Eike pgpgJQzhrfGDt.pgp Description: PGP signature
Re: private modules and dh_python2
[Eike Nicklas, 2011-06-10] I just tried to package an application using a private module. In this case, the name of the script starting the application and the module have the same name. So if the module is in /usr/share/foo/foo, then the script can not be /usr/share/foo/foo as well and installing the script to /usr/share/foo/scripts/foo results in an import error. install foo to /usr/share/foo/ under a different name, see http://lists.debian.org/debian-python/2009/03/msg00091.html 1) Patch the upstream script to add /usr/share/foo to pythonpath if 'import foo' fails? try to avoid patching 2) In this specific case (which might actually not be that uncommon), dh_python2 could rename the script foo to foo.py such that it can be installed to /usr/share/foo/foo.py (currently, the behavior of dh_python2 is to install the script to /usr/share/foo/foo/foo in this case of a naming clash). renaming foo to foo.py is not a good idea, IMO (if there's a foo package (as in foo/__init__.py) it still will be imported, but it's unnecessary confusion. Just name it: run, start, init or something like this. The name doesn't really matter since you will link it as /usr/bin/foo later anyway 3) Module and script having the same name is bad practice and the module should be renamed to foo-lib anyway. that's ugly (even when you remove the unaccepted -) - I find all these foolib or pyfoo module names ugly (isn't it obvious that this module is a library or that it's for Python?) -- Piotr Ożarowski Debian GNU/Linux Developer www.ozarowski.pl www.griffith.cc www.debian.org GPG Fingerprint: 1D2F A898 58DA AF62 1786 2DF7 AEF6 F1A2 A745 7645 -- To UNSUBSCRIBE, email to debian-python-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20110610195211.gr24...@piotro.eu
Re: private modules and dh_python2
On Jun 10, 2011, at 09:48 PM, Eike Nicklas wrote:
Then 'import foo' fails if '/usr/share/foo/foo' is not explicitly added
to pythonpath (that was the idea of having the module private
in the first place ;-) )
Ah, yeah. Y'know, I am personally not a fan of private modules anyway :).
Note too in a setuptools(?)/distribute/packaging world, you might not actually
have a `foo` script in the source at all. For example, Mailman 3 has this in
its setup.py:
template = Template('$script = mailman.bin.$script:main')
scripts = set(
template.substitute(script=script)
for script in ('mailman', 'runner', 'master', 'onebounce')
)
setup(
...
entry_points= {
'console_scripts' : list(scripts),
},
...)
so in fact /usr/bin/mailman doesn't exist until the system is built out.
/usr/bin/mailman gets generated and essentially imports the
`mailman.bin.mailman` module, then runs the main() function.
-Barry
signature.asc
Description: PGP signature
Re: private modules and dh_python2
[Barry Warsaw, 2011-06-10]
Ah, yeah. Y'know, I am personally not a fan of private modules anyway :).
/me waits till Barry will try to package his 13th package with
Python application that uses lib or tests module names...
(or will he break after 4th? Bets anyone? ;)
Note too in a setuptools(?)/distribute/packaging world, you might not actually
have a `foo` script in the source at all. For example, Mailman 3 has this in
its setup.py:
template = Template('$script = mailman.bin.$script:main')
scripts = set(
template.substitute(script=script)
for script in ('mailman', 'runner', 'master', 'onebounce')
)
setup(
...
entry_points= {
'console_scripts' : list(scripts),
},
...)
so in fact /usr/bin/mailman doesn't exist until the system is built out.
/usr/bin/mailman gets generated and essentially imports the
`mailman.bin.mailman` module, then runs the main() function.
you realize that setuptools/distribute hardcodes versions and forces you
to depend on python-setuptools/python-pkg-resources, right?
--
Piotr Ożarowski Debian GNU/Linux Developer
www.ozarowski.pl www.griffith.cc www.debian.org
GPG Fingerprint: 1D2F A898 58DA AF62 1786 2DF7 AEF6 F1A2 A745 7645
--
To UNSUBSCRIBE, email to debian-python-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/20110610201342.gt24...@piotro.eu
Re: private modules and dh_python2
On Fri, 10 Jun 2011 21:52:11 +0200 Piotr Ożarowski wrote: install foo to /usr/share/foo/ under a different name, see http://lists.debian.org/debian-python/2009/03/msg00091.html Renaming is a great and simple idea, I'll do that. Thanks to all of you for the quick help, Eike pgpG23izpXYku.pgp Description: PGP signature
Packaging python programs (with private modules) using the current Python version and binNMU:able when that changes
(Hi, please keep me CC:d as I'm not subscribed (yet).) I'm packaging an application written in (pure) Python. It comes in the form of a module (or set of modules) and some minmal python scripts (with #!/usr/bin/env python or #!/usr/bin/python on top). dh_python2 will, unless told otherwise, and if modules are installed in the standard location, create symlinks in the module trees of all supported Python versions, which is a bit pointless for modules used only by the application in question. AFAIU the Python policy such private modules should not be installed in the standard location though, but in a private directory. All right, I just have to patch the scripts to add that directory to the search path and make sure that the modules are still bytecompiled in the postinst and recompiled when the default python version changes, right? In either case, it seems that I would like to fix the package to the current default Python version, but preferably a simple rebuild should be enough when the default version changes. If the package depends on a single Python version it can also depend on the corresponding pythonX.Y-foo virtual module packages, which _should_ be good because how can you otherwise be sure that python-foo actually provides modules for the current Python version? (For example, python-zodb retracted support for Python 2.5 in Lenny, which I didn't notice). So, my question basically is how to handle all this most easily and correctly. What have I missed? -- Magnus Holmgrenholmg...@debian.org Debian Developer signature.asc Description: This is a digitally signed message part.
Re: Tool support for private modules
Josselin Mouette [EMAIL PROTECTED] writes: Le jeudi 11 octobre 2007 à 10:50 +1000, Ben Finney a écrit : The main reason I use distutils is to assist those people using operating systems that *don't* have good package dependency management, which seems to be the primary target market for setuptools. This is a laudable goal, but not when done at the expense of proper support of operating systems which have one. Indeed. The rest of the message, which you chose not to address this time, asks for help avoiding exactly that trap. Care to answer some of the specific questions in that message and help Python packagers improve their practices? -- \ Remember men, we're fighting for this woman's honour; which is | `\probably more than she ever did. -- Groucho Marx | _o__) | Ben Finney -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Tool support for private modules
Bernd, please follow the Debian mailing list code of conduct URL:http://www.debian.org/MailingLists#codeofconduct, in particular by *not* sending personal copies of messages that are also sent to the list. Also, please preserve attribution lines so we can keep track of who wrote what quoted material. Bernd Zeimetz [EMAIL PROTECTED] writes: As an example, here's a Python package I'm trying to get packaged for Debian. [...] URL:http://cheeseshop.python.org/pypi/gracie/ The first thing I'd do here is to patch the ez_ stuff away, together with setuptools. I presume you mean the 'ez_setup.py' module (there's no other 'ez_*' files in that package). ez_... is known to break things badly (like trying to download things on buildds and other really broken things). Setuptools is broken by design imho (see a thread some time ago about this), but it may work. But as I run into trouble with it too often (like missing __init__.py files in random directories), I replace it by distutils. The main reason I use distutils is to assist those people using operating systems that *don't* have good package dependency management, which seems to be the primary target market for setuptools. I also want my package listed properly at the Python Cheeseshop; this is much easier using setuptools than distutils. Since we're not building eggs there's not need for setuptools at all (afaik distutils is able to build eggs now, too - at least the egg info files, which is enough for us). Better to patch those things before getting FTBFS reports form the buildds. Okay. So you're suggesting that I should continue to maintain the setuptools functionality upstream, but patch it out in the Debian package of the same software? I'm also unclear on what you mean by replace it by distutils. What does this mean for a package that already uses setuptools, and will continue to do so upstream? Although I didn't test it, there should be no special thign to do with your setup.py while building a package, python setup.py build/install --root= should do the job (with and without ez_... and setuptools). I'll address this in a separate thread, as it seems I'm not explaining the problem very well. -- \ I bought some powdered water, but I don't know what to add. | `\ -- Steven Wright | _o__) | Ben Finney -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Tool support for private modules
Ben Finney [EMAIL PROTECTED] writes: The main reason I use distutils is to assist those people using operating systems that *don't* have good package dependency management, which seems to be the primary target market for setuptools. This should, of course, read The main reason I use setuptools … -- \ Pinky, are you pondering what I'm pondering? I think so, | `\Brain, but this time *you* put the trousers on the chimp. -- | _o__)_Pinky and The Brain_ | Ben Finney -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Tool support for private modules
Le lundi 01 octobre 2007 à 23:56 +1000, Ben Finney a écrit : Hmm. I am hoping that modify the programs [to add an absolute path to the search path] is not a necessary part of this. If upstream hasn't thought of it, it is. You only need to add one line in the program, before the module is imported. How can upstream anticipate the arbitrary path I pass to './setup.py install --home=/foo/bar' if the path /foo/bar is up to me as a Debian packager? I don't see how any Python program can be written to allow for that without modification every time the /foo/bar changes. Distutils may be extended to do such a thing, but most of the software I've seen doing it is using automake or changing the files with custom hackery. -- .''`. Josselin Mouette/\./\ : :' : [EMAIL PROTECTED] `. `'[EMAIL PROTECTED] `- Debian GNU/Linux -- The power of freedom signature.asc Description: Ceci est une partie de message numériquement signée
Re: Tool support for private modules
(Please preserve attribution lines on quoted material. I don't know who wrote what in the following.) Josselin Mouette [EMAIL PROTECTED] writes: Le lundi 01 octobre 2007 à 23:56 +1000, Ben Finney a écrit : Hmm. I am hoping that modify the programs [to add an absolute path to the search path] is not a necessary part of this. If upstream hasn't thought of it, it is. You only need to add one line in the program, before the module is imported. [...] I don't see how any Python program can be written to allow for that without modification every time the /foo/bar changes. Distutils may be extended to do such a thing, but most of the software I've seen doing it is using automake or changing the files with custom hackery. I'm confused, then. How does this fit with the implication (in the above quoted teset) that upstream should have thought of [changing the location of the modules]? If downstream hackery is required, I don't see what upstream is expected to have done. -- \The most merciful thing in the world... is the inability of | `\ the human mind to correlate all its contents. -- Howard | _o__)Philips Lovecraft | Ben Finney -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Tool support for private modules
Le lundi 01 octobre 2007 à 14:42 +1000, Ben Finney a écrit : 3.1.1 Programs Shipping Private Modules A program using /usr/bin/python as interpreter can come up with private Python modules. These modules should be installed in /usr/share/module, or /usr/lib/module if the modules are architecture-dependent (e.g. extensions). /usr/lib/site-python is deprecated and should no longer be used for this purpose. I think the policy is a bit confusing here. The purpose of /usr/lib/site-python has never been to ship private modules, but rather to ship public modules supposed to work with any python version. This is now handled by python-support or python-central instead. But the Python distutils and setuptools will install the modules to /usr/lib/site-python/. Hrm, they shouldn't. With a default setup, public modules are shipped to /usr/lib/python2.X/site-packages. My 'debian/rules' already has a 'dh_pycentral' line. That would work if the files were shipped in /usr/lib/python2.X/site-packages. How can I best conform to the above policy? The dumb way, I imagine, would be to manually install the foles to those specific locations. I'm wondering what tools (e.g. debhelper) support something better, and how to use them to conform to the policy. If the modules are indeed private, it looks like you need to change the path by hand, and to add it with sys.path.append(/the/path) at the beginning of the binary. If, as the location suggests, they are public, you should look at the setup.py file to see if there's something unusual that would install the modules in this unusual location. -- .''`. Josselin Mouette/\./\ : :' : [EMAIL PROTECTED] `. `'[EMAIL PROTECTED] `- Debian GNU/Linux -- The power of freedom signature.asc Description: Ceci est une partie de message numériquement signée
Re: Tool support for private modules
Josselin Mouette [EMAIL PROTECTED] writes: Le lundi 01 octobre 2007 à 14:42 +1000, Ben Finney a écrit : But the Python distutils and setuptools will install the modules to /usr/lib/site-python/. Hrm, they shouldn't. With a default setup, public modules are shipped to /usr/lib/python2.X/site-packages. Yes, my apologies, that's what the tools do. I've no idea why I typed that unrelated path above. My 'debian/rules' already has a 'dh_pycentral' line. That would work if the files were shipped in /usr/lib/python2.X/site-packages. That's where the distutils and setuptools place them by default, yes. I don't know what magic is required to put them elsewhere; that may be part of the answer, if someone can instruct me on how to do it. The trouble is, these are modules that clearly fall under the private modules for the program description in the policy document. I fully agree with the policy that modules intended for internal use by a discrete set of programs should not be installed to the public site-packages directory. How can I use the tools available — distutils, setuptools, debhelper — to install these package-specific modules to a package-specific location, such that all the programs in the package will be able to find them? If, as the location suggests, they are public The location does indeed suggest that, but AFAICT that's only because both distutils and setuptools makes no distinction between modules intended for general use on the system and modules only intended for use by specific programs. If the modules are indeed private, it looks like you need to change the path by hand, and to add it with sys.path.append(/the/path) at the beginning of the binary. Hmm. I am hoping that modify the programs is not a necessary part of this. -- \ He who laughs last, thinks slowest. -- Anonymous | `\ | _o__) | Ben Finney -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Tool support for private modules
That's where the distutils and setuptools place them by default, yes. I don't know what magic is required to put them elsewhere; that may be part of the answer, if someone can instruct me on how to do it. You shoul dupload your work somewhere, teaching you is almost impossible if one can't see what's wrong. -- Bernd Zeimetz [EMAIL PROTECTED] http://bzed.de/ -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Tool support for private modules
Bernd Zeimetz [EMAIL PROTECTED] writes: You shoul dupload your work somewhere, teaching you is almost impossible if one can't see what's wrong. I'm not presenting something as wrong. I'm asking for guidance on how to do things right. If the policy recommends that packages be set up a particular way (put package-specific modules in '/usr/share/package/'), but the standard tools behave differently (put all modules by default in '/usr/lib/pythonX.Y/site-packages/'), then there's a step that needs to be taken to get from the default behaviour to the behaviour recommended by policy. I'm asking how to take that step, in a way that isn't brute-force manually hack each package to conform to policy. -- \ Dvorak users of the world flgkd! —Kirsten Chevalier, | `\rec.humor.oracle.d | _o__) | Ben Finney -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Tool support for private modules
If the policy recommends that packages be set up a particular way (put package-specific modules in '/usr/share/package/'), but the standard tools behave differently (put all modules by default in '/usr/lib/pythonX.Y/site-packages/'), then there's a step that needs to be taken to get from the default behaviour to the behaviour recommended by policy. The standard tools Do it right. So there must be a bug somewhere in your packaging or upstream's source. Unfortunately this bug is not shown in my fishbowl. You can read trough http://wiki.debian.org/DebianPython/NewPolicy and check if your packaging contains all the pieces mentioned there. If you think it does, show your code. -- Bernd Zeimetz [EMAIL PROTECTED] http://bzed.de/ -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Tool support for private modules
Le lundi 01 octobre 2007 à 18:37 +1000, Ben Finney a écrit : That would work if the files were shipped in /usr/lib/python2.X/site-packages. That's where the distutils and setuptools place them by default, yes. I don't know what magic is required to put them elsewhere; that may be part of the answer, if someone can instruct me on how to do it. The trouble is, these are modules that clearly fall under the private modules for the program description in the policy document. I fully agree with the policy that modules intended for internal use by a discrete set of programs should not be installed to the public site-packages directory. As long as they are using their own namespace without a too generic name, it's not *that* bad. It is not as if no other package was doing such things. How can I use the tools available — distutils, setuptools, debhelper — to install these package-specific modules to a package-specific location, such that all the programs in the package will be able to find them? The easiest way, if the modules are relocatable (99% of them are) is to simply move them after installation. Otherwise, you can pass specific arguments to setup.py. That would be, python setup.py install --home=/usr/share/$package --install-purelib=. More information: http://www.python.org/doc/2.4/inst/search-path.html If, as the location suggests, they are public The location does indeed suggest that, but AFAICT that's only because both distutils and setuptools makes no distinction between modules intended for general use on the system and modules only intended for use by specific programs. Well, in fact they do, but upstream developers generally simply ship public modules. If the modules are indeed private, it looks like you need to change the path by hand, and to add it with sys.path.append(/the/path) at the beginning of the binary. Hmm. I am hoping that modify the programs is not a necessary part of this. If upstream hasn't thought of it, it is. You only need to add one line in the program, before the module is imported. Cheers, -- .''`. Josselin Mouette/\./\ : :' : [EMAIL PROTECTED] `. `'[EMAIL PROTECTED] `- Debian GNU/Linux -- The power of freedom signature.asc Description: Ceci est une partie de message numériquement signée
Re: Tool support for private modules
Josselin Mouette [EMAIL PROTECTED] writes: Le lundi 01 octobre 2007 à 18:37 +1000, Ben Finney a écrit : How can I use the tools available — distutils, setuptools, debhelper — to install these package-specific modules to a package-specific location, such that all the programs in the package will be able to find them? The easiest way, if the modules are relocatable (99% of them are) is to simply move them after installation. Otherwise, you can pass specific arguments to setup.py. That would be, python setup.py install --home=/usr/share/$package --install-purelib=. More information: http://www.python.org/doc/2.4/inst/search-path.html Thanks, this is a useful pointer. I wasn't aware such fine-grained control was available over locations with distutils. (I've certainly never seen any Python author using that control.) Hmm. I am hoping that modify the programs [to add an absolute path to the search path] is not a necessary part of this. If upstream hasn't thought of it, it is. You only need to add one line in the program, before the module is imported. How can upstream anticipate the arbitrary path I pass to './setup.py install --home=/foo/bar' if the path /foo/bar is up to me as a Debian packager? I don't see how any Python program can be written to allow for that without modification every time the /foo/bar changes. -- \ Everything you read in newspapers is absolutely true, except | `\for that rare story of which you happen to have first-hand | _o__) knowledge. -- Erwin Knoll | Ben Finney -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Tool support for private modules
Ben Finney [EMAIL PROTECTED] writes: How can I best conform to the [Debian policy for Python modules specific to a single package]? As an example, here's a Python package I'm trying to get packaged for Debian. (I am the upstream author of this one, but I'm interested in a solution that *doesn't* involve significant changes to the upstream code.) URL:http://cheeseshop.python.org/pypi/gracie/ How should I modify 'setup.py' to allow binaries and modules for this package to be installed properly, and have the programs continue to find the modules? How should I construct the command line for 'setup.py' when I create the 'install-python%' target in 'debian/rules' for this package? -- \ Eccles: I just saw the Earth through the clouds! Lew: Did | `\ it look round? Eccles: Yes, but I don't think it saw me. | _o__) -- The Goon Show, _Wings Over Dagenham_ | Ben Finney -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Tool support for private modules
As an example, here's a Python package I'm trying to get packaged for
Debian. (I am the upstream author of this one, but I'm interested in a
solution that *doesn't* involve significant changes to the upstream
code.)
URL:http://cheeseshop.python.org/pypi/gracie/
The first thing I'd do here is to patch the ez_ stuff away, together
with setuptools. ez_... is known to break things badly (like trying to
download things on buildds and other really broken things). Setuptools
is broken by design imho (see a thread some time ago about this), but it
may work. But as I run into trouble with it too often (like missing
__init__.py files in random directories), I replace it by distutils.
Since we're not building eggs there's not need for setuptools at all
(afaik distutils is able to build eggs now, too - at least the egg info
files, which is enough for us).
Better to patch those things before getting FTBFS reports form the buildds.
Although I didn't test it, there should be no special thign to do with
your setup.py while building a package, python setup.py build/install
--root= should do the job (with and without ez_... and setuptools).
http://svn.debian.org/wsvn/python-modules/packages/python-contract/trunk/debian/rules?op=filerev=0sc=0
is a pretty simple debian/rules file, which also tries to build the
package for all python versions and runs the tests to ensure nothing is
buggy.
If you like it really simple, cdbs is something you can use:
http://svn.debian.org/wsvn/python-modules/packages/ll-core/trunk/debian/rules?op=filerev=0sc=0
Aa you can see on the cdbs example, sometimes upstream does not install
all files in the way one would like it to have in debian, often out of
personal taste - that's easy to fix though. Although at least I
appreciate if upstras isntalls everything at the proper place, at least
distutils supports installing completely non-py related files into the
right directories.
Depending on what you like you have to call dh_py{central,support} to
move your installed files in their directories, of course you need to
add the neccessary informations for them at the apropriate places (like
which python versions you want to support).
How should I construct the command line for 'setup.py' when I create
the 'install-python%' target in 'debian/rules' for this package?
Have a look at the first example.
Both will work for arch: all and arch: any packages. For arch: all
packages you don;t need to build/install with all python versions, just
using the default one is enough, everythign else will be handled by
python-support/central while installing the package. It makes sense to
build them all, though, especially when you want to run tests.
Hope that helps,
Bernd
--
Bernd Zeimetz
[EMAIL PROTECTED] http://bzed.de/
--
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Tool support for private modules
Howdy all, The Debian Python Policy, chapter 3 URL:http://www.debian.org/doc/packaging-manuals/python-policy/ch-programs.html says: 3.1.1 Programs Shipping Private Modules A program using /usr/bin/python as interpreter can come up with private Python modules. These modules should be installed in /usr/share/module, or /usr/lib/module if the modules are architecture-dependent (e.g. extensions). /usr/lib/site-python is deprecated and should no longer be used for this purpose. But the Python distutils and setuptools will install the modules to /usr/lib/site-python/. My 'debian/rules' already has a 'dh_pycentral' line. How can I best conform to the above policy? The dumb way, I imagine, would be to manually install the foles to those specific locations. I'm wondering what tools (e.g. debhelper) support something better, and how to use them to conform to the policy. -- \ God forbid that any book should be banned. The practice is as | `\ indefensible as infanticide. -- Dame Rebecca West | _o__) | Ben Finney -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
