Re: Private modules
On Dec 24, 2012, at 12:43 AM, Dmitrijs Ledkovs wrote: The way I interpreted Paul's comment is that it implies don't use virtualenv inside the .deb package as to be distributed by Debian e.g. system-wide python packages should not be using virtualenv environment out of the box on Debian, as that opens a can of worms. I would definitely agree with that. -Barry signature.asc Description: PGP signature
Re: Private modules
On 22 December 2012 22:00, Bas Wijnen wij...@debian.org wrote: 6. Make /usr/bin/program a symlink to the actual file in the private directory. It will then search in its real place. (I've seen this used by angrydd.) This (symlinking /usr/bin/program) appears to be the recommended way to deal with it: http://wiki.debian.org/Python/Packaging#Example_2:_Python_application http://permalink.gmane.org/gmane.linux.debian.devel.python/6728 Thomas
Re: Private modules
On Sat, Dec 22, 2012 at 04:27:46PM -0500, Barry Warsaw wrote: On Dec 22, 2012, at 05:19 PM, Paul Tagliamonte wrote: Yeah, please don't use virtualenv, as much as I'd like to see a good way of using virtualenv in Debian. Can you expand on that? It should be usable to develop code, but do you mean something else? This really isn't the place for this, so if we could take this elsewhere, that'd be nice, but, given that we don't solve the problem virtualenv fixes, it'd be nice to support this in Debian. naturally, I don't care enough to fix it, but I'd imagine a system where we create a virtualenv by symlinking libraries into virtualenvs for each installed Python application, to allow us to co-install two versions of the same library for two different applications. Anyway, let's take this off list, it's a silly conversation we don't need to have :) Cheers, -Barry -- .''`. Paul Tagliamonte paul...@debian.org : :' : Proud Debian Developer `. `'` 4096R / 8F04 9AD8 2C92 066C 7352 D28A 7B58 5B30 807C 2A87 `- http://people.debian.org/~paultag signature.asc Description: Digital signature
Re: Private modules
On 22 December 2012 23:27, Barry Warsaw ba...@python.org wrote: On Dec 22, 2012, at 05:19 PM, Paul Tagliamonte wrote: Yeah, please don't use virtualenv, as much as I'd like to see a good way of using virtualenv in Debian. Can you expand on that? It should be usable to develop code, but do you mean something else? The way I interpreted Paul's comment is that it implies don't use virtualenv inside the .deb package as to be distributed by Debian e.g. system-wide python packages should not be using virtualenv environment out of the box on Debian, as that opens a can of worms. (virtualenv does nothing to help to what a coherent distribution tries to achieve). I do agree with you that one should not break vritualenv on purpose for those people who wish to use it. Regards, Dmitrijs. -- To UNSUBSCRIBE, email to debian-python-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/CANBHLUjiu=RfsBvaj-Q+11yQhFngZMhoPQh=6KFAwk=op4y...@mail.gmail.com
Re: Private modules
On Sat, Dec 22, 2012 at 11:00:35PM +0100, Bas Wijnen wrote: Hello, I have a Python program (of which I am upstream) and I want to package it. However, it has some private modules and I don't know how to treat them. Section 3.1.1 of the Python Packaging Policy says they should be in /usr/share/. This makes sense, being private data of the package.[1] However, this doesn't seem to be how Python programs normally work. I asked on IRC (freenode:#python), and the response was that Python programs don't have the concept of private modules. Any module, no matter how program-specific, is installed publicly. This feels wrong for me, and not the Debian Way. Remember, binaries can see local modules - if you drop the program into /usr/share as well, and link it into /usr/bin, it won't be system wide, and it will be importable -- so, there is this concept (sorta). This wouldn't be a problem in itself, but I need support from the Python system to make things work: my program needs to import my private module, which means its directory must be in sys.path. But how do I get it there? I see several options, none of which seems really good: 1. Hard-code the directory in the program. + It works. - It will then only work on Debian - It cannot run from a different path. 2. Let the upstream program assume that the module is in the search path (either installed publicly or in the same directory as the script). Use a debian patch to add Debian's private path to it. + It works. + The upstream script works for non-Debian people. - The patch may not apply cleanly in a next version, so it slightly increases maintainance burden. - The packaged program is not the upstream version (is that actually a problem?) 3. Create a module for handling this. I import the module and call a function on it providing my package name. It will add the required directories to sys.path. + This would be great, if it worked, but: - Othes operating systems don't have this module, so even though the approach is intended to make it more portable, it actually makes it Debian-specific. 4. Virtualenv was suggested on IRC. I'm not sure how I should use it; it seems like a tool for users with little permissions on the system, not for a package which can access anything. But I might be wrong, so I wanted to mention it. Yeah, please don't use virtualenv, as much as I'd like to see a good way of using virtualenv in Debian. + ? - ? 5. Create a wrapper executable script which just runs the program from /usr/share. Have the private module in the same directory as the program. (I've seen this used by freealchemist.) + It works. - It's a clumsy workaround. 6. Make /usr/bin/program a symlink to the actual file in the private directory. It will then search in its real place. (I've seen this used by angrydd.) + It works. + It's not as clumsy. - It still seems to be a workaround. - It is not obvious to a user who copies the file and changes it, why the copy doesn't find its modules. Any thoughts on this are welcome. Thanks, Bas Ps: please Cc me on replies, I'm not subscribed to this list. (And keep these lines, so I alse get replies to replies.) [1] Actually, it says it should go in /usr/share/modulename/, while I would expect it to go in /usr/share/packagename/.../modulename/. Am I misreading this? In particular, should package foo (with private module bar) really create /usr/share/bar? -- .''`. Paul Tagliamonte paul...@debian.org : :' : Proud Debian Developer `. `'` 4096R / 8F04 9AD8 2C92 066C 7352 D28A 7B58 5B30 807C 2A87 `- http://people.debian.org/~paultag signature.asc Description: Digital signature
Re: private modules and dh_python2
On Jun 14, 2011, at 12:02 AM, Piotr Ożarowski wrote: it is fine and it is useful... as a submodule, not as a top-level module Agreed! I think I get what you're driving at now. Some applications don't put their Python code or tests in a package. In those cases, yes by all means a private package makes complete sense. [...] you realize that setuptools/distribute hardcodes versions and forces you to depend on python-setuptools/python-pkg-resources, right? In the context of Debian, what are the practical problems of this? it's 217k of unneeded data that can be easily avoided and few CPU cycles that can be spared anyway, I always say to my sponsorees: if it's not useful outside this application, make it private and not pollute the global namespace. It always can be promoted to public one later Agreed. For applications that do it correctly, I think it's fine for their support package to be public. The principle is: Don't pollute the global namespace! -Barry signature.asc Description: PGP signature
Re: private modules and dh_python2
As an aside, I notice that Debian Python Policy begins by assuming people know
what private Python modules are, but this will not be the case for most
Python developers unfamiliar with Debian conventions. Section 2.1 briefly
mentions the distinction from a sys.path-visibility point of view, which
doesn't really explain much. I'd like to see a section early on which defines
private modules, where, when, why and how they should be used, etc.
On Jun 10, 2011, at 10:13 PM, Piotr Ożarowski wrote:
[Barry Warsaw, 2011-06-10]
Ah, yeah. Y'know, I am personally not a fan of private modules anyway :).
/me waits till Barry will try to package his 13th package with
Python application that uses lib or tests module names...
(or will he break after 4th? Bets anyone? ;)
I know this has been discussed before, and there's even debate about it
upstream, so apologies for rehashing it. My own personal opinion is that it's
fine to include things like a `test` (Python) subpackage in the (Debian)
package python-foo. It aligns with the Python consenting adults motto, and
I think such things *can* be useful. As long as the top-level package name is
unique, subpackage can't pollute the global namespace, so I don't care.
Note too in a setuptools(?)/distribute/packaging world, you might not
actually have a `foo` script in the source at all. For example, Mailman 3
has this in its setup.py:
template = Template('$script = mailman.bin.$script:main')
scripts = set(
template.substitute(script=script)
for script in ('mailman', 'runner', 'master', 'onebounce')
)
setup(
...
entry_points= {
'console_scripts' : list(scripts),
},
...)
so in fact /usr/bin/mailman doesn't exist until the system is built out.
/usr/bin/mailman gets generated and essentially imports the
`mailman.bin.mailman` module, then runs the main() function.
you realize that setuptools/distribute hardcodes versions and forces you
to depend on python-setuptools/python-pkg-resources, right?
In the context of Debian, what are the practical problems of this?
-Barry
signature.asc
Description: PGP signature
Re: private modules and dh_python2
[Barry Warsaw, 2011-06-13] it's fine to include things like a `test` (Python) subpackage in the (Debian) package python-foo. It aligns with the Python consenting adults motto, and I think such things *can* be useful. As long as the top-level package name is unique, subpackage can't pollute the global namespace, so I don't care. it is fine and it is useful... as a submodule, not as a top-level module [...] you realize that setuptools/distribute hardcodes versions and forces you to depend on python-setuptools/python-pkg-resources, right? In the context of Debian, what are the practical problems of this? it's 217k of unneeded data that can be easily avoided and few CPU cycles that can be spared anyway, I always say to my sponsorees: if it's not useful outside this application, make it private and not pollute the global namespace. It always can be promoted to public one later -- Piotr Ożarowski Debian GNU/Linux Developer www.ozarowski.pl www.griffith.cc www.debian.org GPG Fingerprint: 1D2F A898 58DA AF62 1786 2DF7 AEF6 F1A2 A745 7645 -- To UNSUBSCRIBE, email to debian-python-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20110613220226.ga4...@piotro.eu
Re: private modules and dh_python2
On Jun 10, 2011, at 09:01 PM, Eike Nicklas wrote: I just tried to package an application using a private module. In this case, the name of the script starting the application and the module have the same name. Is the script private too? Wouldn't that be better installed in /usr/bin/foo? -Barry signature.asc Description: PGP signature
Re: private modules and dh_python2
Hi Barry, thanks for the quick answer. On Fri, 10 Jun 2011 15:34:19 -0400 Barry Warsaw wrote: On Jun 10, 2011, at 09:01 PM, Eike Nicklas wrote: I just tried to package an application using a private module. In this case, the name of the script starting the application and the module have the same name. Is the script private too? Wouldn't that be better installed in /usr/bin/foo? Then 'import foo' fails if '/usr/share/foo/foo' is not explicitly added to pythonpath (that was the idea of having the module private in the first place ;-) ) As I understand http://wiki.debian.org/Python/Packaging#Example_2:_Python_application the idea is to install both script and module to /usr/share/foo where the script can 'locally' import the module and link /usr/bin/foo to the script. Eike pgpgJQzhrfGDt.pgp Description: PGP signature
Re: private modules and dh_python2
[Eike Nicklas, 2011-06-10] I just tried to package an application using a private module. In this case, the name of the script starting the application and the module have the same name. So if the module is in /usr/share/foo/foo, then the script can not be /usr/share/foo/foo as well and installing the script to /usr/share/foo/scripts/foo results in an import error. install foo to /usr/share/foo/ under a different name, see http://lists.debian.org/debian-python/2009/03/msg00091.html 1) Patch the upstream script to add /usr/share/foo to pythonpath if 'import foo' fails? try to avoid patching 2) In this specific case (which might actually not be that uncommon), dh_python2 could rename the script foo to foo.py such that it can be installed to /usr/share/foo/foo.py (currently, the behavior of dh_python2 is to install the script to /usr/share/foo/foo/foo in this case of a naming clash). renaming foo to foo.py is not a good idea, IMO (if there's a foo package (as in foo/__init__.py) it still will be imported, but it's unnecessary confusion. Just name it: run, start, init or something like this. The name doesn't really matter since you will link it as /usr/bin/foo later anyway 3) Module and script having the same name is bad practice and the module should be renamed to foo-lib anyway. that's ugly (even when you remove the unaccepted -) - I find all these foolib or pyfoo module names ugly (isn't it obvious that this module is a library or that it's for Python?) -- Piotr Ożarowski Debian GNU/Linux Developer www.ozarowski.pl www.griffith.cc www.debian.org GPG Fingerprint: 1D2F A898 58DA AF62 1786 2DF7 AEF6 F1A2 A745 7645 -- To UNSUBSCRIBE, email to debian-python-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20110610195211.gr24...@piotro.eu
Re: private modules and dh_python2
On Jun 10, 2011, at 09:48 PM, Eike Nicklas wrote:
Then 'import foo' fails if '/usr/share/foo/foo' is not explicitly added
to pythonpath (that was the idea of having the module private
in the first place ;-) )
Ah, yeah. Y'know, I am personally not a fan of private modules anyway :).
Note too in a setuptools(?)/distribute/packaging world, you might not actually
have a `foo` script in the source at all. For example, Mailman 3 has this in
its setup.py:
template = Template('$script = mailman.bin.$script:main')
scripts = set(
template.substitute(script=script)
for script in ('mailman', 'runner', 'master', 'onebounce')
)
setup(
...
entry_points= {
'console_scripts' : list(scripts),
},
...)
so in fact /usr/bin/mailman doesn't exist until the system is built out.
/usr/bin/mailman gets generated and essentially imports the
`mailman.bin.mailman` module, then runs the main() function.
-Barry
signature.asc
Description: PGP signature
Re: private modules and dh_python2
[Barry Warsaw, 2011-06-10]
Ah, yeah. Y'know, I am personally not a fan of private modules anyway :).
/me waits till Barry will try to package his 13th package with
Python application that uses lib or tests module names...
(or will he break after 4th? Bets anyone? ;)
Note too in a setuptools(?)/distribute/packaging world, you might not actually
have a `foo` script in the source at all. For example, Mailman 3 has this in
its setup.py:
template = Template('$script = mailman.bin.$script:main')
scripts = set(
template.substitute(script=script)
for script in ('mailman', 'runner', 'master', 'onebounce')
)
setup(
...
entry_points= {
'console_scripts' : list(scripts),
},
...)
so in fact /usr/bin/mailman doesn't exist until the system is built out.
/usr/bin/mailman gets generated and essentially imports the
`mailman.bin.mailman` module, then runs the main() function.
you realize that setuptools/distribute hardcodes versions and forces you
to depend on python-setuptools/python-pkg-resources, right?
--
Piotr Ożarowski Debian GNU/Linux Developer
www.ozarowski.pl www.griffith.cc www.debian.org
GPG Fingerprint: 1D2F A898 58DA AF62 1786 2DF7 AEF6 F1A2 A745 7645
--
To UNSUBSCRIBE, email to debian-python-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/20110610201342.gt24...@piotro.eu
Re: private modules and dh_python2
On Fri, 10 Jun 2011 21:52:11 +0200 Piotr Ożarowski wrote: install foo to /usr/share/foo/ under a different name, see http://lists.debian.org/debian-python/2009/03/msg00091.html Renaming is a great and simple idea, I'll do that. Thanks to all of you for the quick help, Eike pgpG23izpXYku.pgp Description: PGP signature
