Re: [gentoo-dev] New eclass for Python
On Mon, Mar 26, 2012 at 6:23 PM, Krzysztof Pawlik nelch...@gentoo.org wrote:
On 26/03/12 18:11, Krzysztof Pawlik wrote:
On 26/03/12 09:20, justin wrote:
On 25/03/12 20:56, Krzysztof Pawlik wrote:
On 28/02/12 22:13, Krzysztof Pawlik wrote:
If there are no objections then during the weekend (March 3, 4) I will
add this
to portage (after finishing remaining TODO items, PyPy requires 4G of
RAM(!!)).
Hello,
Slightly late due to Real Life™ but finally it's in the main tree :)
(and yes - I've tested it with pypy - works as expected :)
Hi,
is there any documentation beside the man page somewhere?
No.
I tried to port some ebuilds but as soon I set
PYTHON_COMPAT=python2_7 python2_6 python2_5 pypy1_8
inherit python-distutils-ng
I get
REQUIRED_USE: USE flag 'python_targets_python3_1' is not in IUSE
Did I do something wrong, or is there something not straight in the eclass?
Can you send me the whole ebuild off-list?
There are two ebuilds using the eclass that I've used as tests:
http://git.overlays.gentoo.org/gitweb/?p=dev/nelchael.git;a=tree;f=dev-python;h=f1a8e00e3e6df33806d8972c8898f1187163bd3d;hb=HEAD
Ok, found a bug: REQUIRED_USE can't contain elements not in USE, so if you
excluded python3_1 from PYTHON_COMPAT it didn't appear in IUSE too -
REQUIRED_USE contained invalid value. Fixed by below patch:
nelchael@s-lappy ~/.../gentoo-x86/eclass$ cvs diff
Index: python-distutils-ng.eclass
===
RCS file: /var/cvsroot/gentoo-x86/eclass/python-distutils-ng.eclass,v
retrieving revision 1.2
diff -u -r1.2 python-distutils-ng.eclass
--- python-distutils-ng.eclass 26 Mar 2012 06:12:53 - 1.2
+++ python-distutils-ng.eclass 26 Mar 2012 16:20:52 -
@@ -105,11 +105,11 @@
esac
}
-required_use_str= || (
- python_targets_python2_5 python_targets_python2_6
python_targets_python2_7
- python_targets_python3_1 python_targets_python3_2
- python_targets_jython2_5
- python_targets_pypy1_7 python_targets_pypy1_8 )
+required_use_str=
+for impl in ${PYTHON_COMPAT}; do
+ required_use_str=${required_use_str} python_targets_${impl}
+done
+required_use_str= || ( ${required_use_str} )
if [[ ${PYTHON_OPTIONAL} = yes ]]; then
IUSE+=python
REQUIRED_USE+= python? ( ${required_use_str} )
--
Krzysztof Pawlik nelchael at gentoo.org key id: 0xF6A80E46
desktop-misc, java, vim, kernel, python, apache...
I have a feature request for distutil-ng (or maybe it's already
possible but I don't know how).
I have a package that depends on python-dateutil:python-2 for
python2_x and python-dateutil:python-3 for python3_x.
Would it be possible to have virtual targets like python, python2,
python3, pypi, jithon ?
Thanks,
--
Corentin Chary
http://xf.iksaif.net
Re: [gentoo-dev] New eclass for Python
On Wed, Apr 4, 2012 at 4:50 AM, Corentin Chary corentin.ch...@gmail.com wrote: I have a package that depends on python-dateutil:python-2 for python2_x and python-dateutil:python-3 for python3_x. Would it be possible to have virtual targets like python, python2, python3, pypi, jithon ? With regards to python-dateutil: As of python-dateutil-2.1, there are no longer separate slots for python-2 and python-3. As well, I masked the only version (2.0) with SLOT=python-3. For future compatibility, you should remove the slot from your dependencies and just depend on dev-python/python-dateutil.
Re: [gentoo-dev] New eclass for Python
On Wed, Apr 4, 2012 at 4:22 PM, Mike Gilbert flop...@gentoo.org wrote: On Wed, Apr 4, 2012 at 4:50 AM, Corentin Chary corentin.ch...@gmail.com wrote: I have a package that depends on python-dateutil:python-2 for python2_x and python-dateutil:python-3 for python3_x. Would it be possible to have virtual targets like python, python2, python3, pypi, jithon ? With regards to python-dateutil: As of python-dateutil-2.1, there are no longer separate slots for python-2 and python-3. As well, I masked the only version (2.0) with SLOT=python-3. For future compatibility, you should remove the slot from your dependencies and just depend on dev-python/python-dateutil. Yep, I just saw that. But well, celery is full of examples like thats since it have different dependencies for python2, python2.5, python2.5 + tests, jython, jython + tests, etc.. Having a way to group similar python would be great. -- Corentin Chary http://xf.iksaif.net
Re: [gentoo-dev] New eclass for Python
On Wed, Apr 4, 2012 at 4:50 AM, Corentin Chary corentin.ch...@gmail.com wrote: I have a feature request for distutil-ng (or maybe it's already possible but I don't know how). I have a package that depends on python-dateutil:python-2 for python2_x and python-dateutil:python-3 for python3_x. Would it be possible to have virtual targets like python, python2, python3, pypi, jithon ? One way to implement this would be to have a separate function that you would call to generate abi-specific dependencies.
Re: [gentoo-dev] New eclass for Python
On Wed, Apr 04, 2012 at 10:41:03AM -0400, Mike Gilbert wrote: On Wed, Apr 4, 2012 at 4:50 AM, Corentin Chary corentin.ch...@gmail.com wrote: I have a feature request for distutil-ng (or maybe it's already possible but I don't know how). I have a package that depends on python-dateutil:python-2 for python2_x and python-dateutil:python-3 for python3_x. Would it be possible to have virtual targets like python, python2, python3, pypi, jithon ? One way to implement this would be to have a separate function that you would call to generate abi-specific dependencies. Think through the performance implications of that, and the fact that moves control/decisions outside of the PM resolvers purview. If it's use controlled and is part of metadata, sure, but anything else? Not so much. :) ~harring
Re: [gentoo-dev] New eclass for Python
On Wed, Apr 4, 2012 at 8:07 PM, Brian Harring ferri...@gmail.com wrote: On Wed, Apr 04, 2012 at 10:41:03AM -0400, Mike Gilbert wrote: On Wed, Apr 4, 2012 at 4:50 AM, Corentin Chary corentin.ch...@gmail.com wrote: I have a feature request for distutil-ng (or maybe it's already possible but I don't know how). I have a package that depends on python-dateutil:python-2 for python2_x and python-dateutil:python-3 for python3_x. Would it be possible to have virtual targets like python, python2, python3, pypi, jithon ? One way to implement this would be to have a separate function that you would call to generate abi-specific dependencies. Think through the performance implications of that, and the fact that moves control/decisions outside of the PM resolvers purview. If it's use controlled and is part of metadata, sure, but anything else? Not so much. :) All I'm talking about is a function or function(s) to generate deps like these: python2_6? ( dev-python/python-dateutil[python2_6] ) python2_7? ( dev-python/python-dateutil[python2_7] ) I'm not sure how that would affect performance.
Re: [gentoo-dev] New eclass for Python
On Wed, Apr 04, 2012 at 08:36:37PM -0400, Mike Gilbert wrote: On Wed, Apr 4, 2012 at 8:07 PM, Brian Harring ferri...@gmail.com wrote: On Wed, Apr 04, 2012 at 10:41:03AM -0400, Mike Gilbert wrote: On Wed, Apr 4, 2012 at 4:50 AM, Corentin Chary corentin.ch...@gmail.com wrote: I have a feature request for distutil-ng (or maybe it's already possible but I don't know how). I have a package that depends on python-dateutil:python-2 for python2_x and python-dateutil:python-3 for python3_x. Would it be possible to have virtual targets like python, python2, python3, pypi, jithon ? One way to implement this would be to have a separate function that you would call to generate abi-specific dependencies. Think through the performance implications of that, and the fact that moves control/decisions outside of the PM resolvers purview. If it's use controlled and is part of metadata, sure, but anything else? ??Not so much. :) All I'm talking about is a function or function(s) to generate deps like these: python2_6? ( dev-python/python-dateutil[python2_6] ) python2_7? ( dev-python/python-dateutil[python2_7] ) I'm not sure how that would affect performance. Nah, that's fine- that's metadata level representation of it. Sorry, aparenytly I misread your original email- interpretted it as shelling out for that decision (something people propose occasionally, and usually needs to be stomped out w/ maximal force), rather than representing it properly in dependencies. What you're intending there is perfectly sane/acceptable; pardon the noise ;) ~brian
Re: [gentoo-dev] New eclass for Python
On 25/03/12 20:56, Krzysztof Pawlik wrote: On 28/02/12 22:13, Krzysztof Pawlik wrote: If there are no objections then during the weekend (March 3, 4) I will add this to portage (after finishing remaining TODO items, PyPy requires 4G of RAM(!!)). Hello, Slightly late due to Real Life™ but finally it's in the main tree :) (and yes - I've tested it with pypy - works as expected :) Hi, is there any documentation beside the man page somewhere? I tried to port some ebuilds but as soon I set PYTHON_COMPAT=python2_7 python2_6 python2_5 pypy1_8 inherit python-distutils-ng I get REQUIRED_USE: USE flag 'python_targets_python3_1' is not in IUSE Did I do something wrong, or is there something not straight in the eclass? Thanks justin signature.asc Description: OpenPGP digital signature
Re: [gentoo-dev] New eclass for Python
On 25/03/12 20:56, Krzysztof Pawlik wrote: On 28/02/12 22:13, Krzysztof Pawlik wrote: If there are no objections then during the weekend (March 3, 4) I will add this to portage (after finishing remaining TODO items, PyPy requires 4G of RAM(!!)). Hello, Slightly late due to Real Life™ but finally it's in the main tree :) (and yes - I've tested it with pypy - works as expected :) Hi, is it okay if I start fixing things for prefix? jusitn signature.asc Description: OpenPGP digital signature
Re: [gentoo-dev] New eclass for Python
On 26/03/12 09:21, justin wrote: On 25/03/12 20:56, Krzysztof Pawlik wrote: On 28/02/12 22:13, Krzysztof Pawlik wrote: If there are no objections then during the weekend (March 3, 4) I will add this to portage (after finishing remaining TODO items, PyPy requires 4G of RAM(!!)). Hello, Slightly late due to Real Life™ but finally it's in the main tree :) (and yes - I've tested it with pypy - works as expected :) Hi, is it okay if I start fixing things for prefix? Yes, by all means - please send the diff to this thread. jusitn ^^ your e-mail says justin -- typo? -- Krzysztof Pawlik nelchael at gentoo.org key id: 0xF6A80E46 desktop-misc, java, vim, kernel, python, apache... signature.asc Description: OpenPGP digital signature
Re: [gentoo-dev] New eclass for Python
On 26/03/12 09:20, justin wrote: On 25/03/12 20:56, Krzysztof Pawlik wrote: On 28/02/12 22:13, Krzysztof Pawlik wrote: If there are no objections then during the weekend (March 3, 4) I will add this to portage (after finishing remaining TODO items, PyPy requires 4G of RAM(!!)). Hello, Slightly late due to Real Life™ but finally it's in the main tree :) (and yes - I've tested it with pypy - works as expected :) Hi, is there any documentation beside the man page somewhere? No. I tried to port some ebuilds but as soon I set PYTHON_COMPAT=python2_7 python2_6 python2_5 pypy1_8 inherit python-distutils-ng I get REQUIRED_USE: USE flag 'python_targets_python3_1' is not in IUSE Did I do something wrong, or is there something not straight in the eclass? Can you send me the whole ebuild off-list? There are two ebuilds using the eclass that I've used as tests: http://git.overlays.gentoo.org/gitweb/?p=dev/nelchael.git;a=tree;f=dev-python;h=f1a8e00e3e6df33806d8972c8898f1187163bd3d;hb=HEAD -- Krzysztof Pawlik nelchael at gentoo.org key id: 0xF6A80E46 desktop-misc, java, vim, kernel, python, apache... signature.asc Description: OpenPGP digital signature
Re: [gentoo-dev] New eclass for Python
On 26/03/12 18:11, Krzysztof Pawlik wrote:
On 26/03/12 09:20, justin wrote:
On 25/03/12 20:56, Krzysztof Pawlik wrote:
On 28/02/12 22:13, Krzysztof Pawlik wrote:
If there are no objections then during the weekend (March 3, 4) I will add
this
to portage (after finishing remaining TODO items, PyPy requires 4G of
RAM(!!)).
Hello,
Slightly late due to Real Life™ but finally it's in the main tree :)
(and yes - I've tested it with pypy - works as expected :)
Hi,
is there any documentation beside the man page somewhere?
No.
I tried to port some ebuilds but as soon I set
PYTHON_COMPAT=python2_7 python2_6 python2_5 pypy1_8
inherit python-distutils-ng
I get
REQUIRED_USE: USE flag 'python_targets_python3_1' is not in IUSE
Did I do something wrong, or is there something not straight in the eclass?
Can you send me the whole ebuild off-list?
There are two ebuilds using the eclass that I've used as tests:
http://git.overlays.gentoo.org/gitweb/?p=dev/nelchael.git;a=tree;f=dev-python;h=f1a8e00e3e6df33806d8972c8898f1187163bd3d;hb=HEAD
Ok, found a bug: REQUIRED_USE can't contain elements not in USE, so if you
excluded python3_1 from PYTHON_COMPAT it didn't appear in IUSE too -
REQUIRED_USE contained invalid value. Fixed by below patch:
nelchael@s-lappy ~/.../gentoo-x86/eclass$ cvs diff
Index: python-distutils-ng.eclass
===
RCS file: /var/cvsroot/gentoo-x86/eclass/python-distutils-ng.eclass,v
retrieving revision 1.2
diff -u -r1.2 python-distutils-ng.eclass
--- python-distutils-ng.eclass 26 Mar 2012 06:12:53 - 1.2
+++ python-distutils-ng.eclass 26 Mar 2012 16:20:52 -
@@ -105,11 +105,11 @@
esac
}
-required_use_str= || (
- python_targets_python2_5 python_targets_python2_6
python_targets_python2_7
- python_targets_python3_1 python_targets_python3_2
- python_targets_jython2_5
- python_targets_pypy1_7 python_targets_pypy1_8 )
+required_use_str=
+for impl in ${PYTHON_COMPAT}; do
+ required_use_str=${required_use_str} python_targets_${impl}
+done
+required_use_str= || ( ${required_use_str} )
if [[ ${PYTHON_OPTIONAL} = yes ]]; then
IUSE+=python
REQUIRED_USE+= python? ( ${required_use_str} )
--
Krzysztof Pawlik nelchael at gentoo.org key id: 0xF6A80E46
desktop-misc, java, vim, kernel, python, apache...
signature.asc
Description: OpenPGP digital signature
Re: [gentoo-dev] New eclass for Python
On 28/02/12 22:13, Krzysztof Pawlik wrote: If there are no objections then during the weekend (March 3, 4) I will add this to portage (after finishing remaining TODO items, PyPy requires 4G of RAM(!!)). Hello, Slightly late due to Real Life™ but finally it's in the main tree :) (and yes - I've tested it with pypy - works as expected :) -- Krzysztof Pawlik nelchael at gentoo.org key id: 0xF6A80E46 desktop-misc, java, vim, kernel, python, apache... signature.asc Description: OpenPGP digital signature
Re: [gentoo-dev] New eclass for Python
On 3/25/12 11:56 AM, Krzysztof Pawlik wrote: On 28/02/12 22:13, Krzysztof Pawlik wrote: If there are no objections then during the weekend (March 3, 4) I will add this to portage (after finishing remaining TODO items, PyPy requires 4G of RAM(!!)). Hello, Slightly late due to Real Life™ but finally it's in the main tree :) (and yes - I've tested it with pypy - works as expected :) Wonderful =) I'd split away the python eclass to have a trimmed down eclass specific for the implementations btw. (the mess with the eselect-python shown we could fix that better) lu
Re: [gentoo-dev] New eclass for Python
On 29/02/12 22:57, Alexandre Rostovtsev wrote:
On Wed, 2012-02-29 at 21:24 +0100, Krzysztof Pawlik wrote:
On 29/02/12 20:51, Alexandre Rostovtsev wrote:
The proposed eclass omits three features from python.eclass which are
heavily used in the gnome stack.
Correct me if I'm wrong, but Gnome doesn't use standard distutils?
Gnome is mostly written in C and therefore uses standard autotools :)
Ok, thank you for this information.
Second, there doesn't seem to be any support for packages that do not
install in python's site-packages and do not allow multiple python ABIs.
If I have, for example, a package that installs python modules
in /usr/lib/appname or /usr/share/appname, how can I specify that
PYTHON_TARGETS=python2.6 or python2.7 or python3.2 is allowed, but
something like PYTHON_TARGETS=python2.7 python3.2 is not?
You're correct, note that I've stressed that this eclass is mainly for
distutils-based packages. I'm not using Gnome, so can you provide some
package
examples that I can look at?
personal opinion
If package decides to use given language then please, please play by the
rules
set by the rest of world (Ruby - gems, Python - distutils, Perl - CPAN,
PHP
- PEAR).
I don't like installing Python code outside of site-packages, the only
exception
to that rule is portage (at least for now).
/personal opinion
Some non-python packages allow python-based plugins. Obviously these
plugins live in the package's plugin directory (not in python's
site-packages) and use the package's main build system (not distutils),
and multiple python ABIs cannot be supported because that would result
in colliding plugins. Typical examples are app-editors/gedit,
media-gfx/gimp, media-sound/rhythmbox, or media-video/totem.
Some packages install a C library that links to a specific version of
libpython or that defines a particular python version string at compile
time, making it impossible to use the package with multiple python ABIs.
Examples I know are dev-libs/libpeas and dev-python/nautilus-python.
And then there are packages which could support e.g. multiple python2
ABIs in theory, but doing so in practice would require a fair bit of
patching, taking substantial effort with no real benefit for end users.
An example that springs to mind here is gnome-extra/zeitgeist.
I see - so it's the same case as with KDE (like Andreas wrote) - it's actually
not a python package but rather an embedded Python. That's very different case
than I'm targeting with this eclass (at least for now).
I'd be happy to hear how to solve this - what prefix or suffix to use? One
way
would be quite trivial: if only one implementation is enabled do not create
script-${impl}, go with single file, does that sound good?
That would be the ideal solution.
Ok, I've implemented this solution, now if one implementation is enabled there
will be only one script, no foo-IMPL mangling. I've added also ability to
install the script in other directory than /usr/bin. Hope this solves this case.
Thank you for your comments and suggestions :)
--
Krzysztof Pawlik nelchael at gentoo.org key id: 0xF6A80E46
desktop-misc, java, vim, kernel, python, apache...
signature.asc
Description: OpenPGP digital signature
Re: [gentoo-dev] New eclass for Python
On 28-02-2012 22:13:36 +0100, Krzysztof Pawlik wrote:
[good stuff]
Much appreciated!
From 2nd example ebuild:
python_install_all() {
rm -f ${D}/usr/bin/*.py || die
s/D/ED/ here for Prefix :)
I haven't checked the eclass in detail, but did you intend to make it
Prefix aware at all? I guess someone from us should test your work.
--
Fabian Groffen
Gentoo on a different level
signature.asc
Description: Digital signature
Re: [gentoo-dev] New eclass for Python
On 29/02/12 09:17, Fabian Groffen wrote:
On 28-02-2012 22:13:36 +0100, Krzysztof Pawlik wrote:
[good stuff]
Much appreciated!
From 2nd example ebuild:
python_install_all() {
rm -f ${D}/usr/bin/*.py || die
s/D/ED/ here for Prefix :)
I haven't checked the eclass in detail, but did you intend to make it
Prefix aware at all? I guess someone from us should test your work.
Yes, please - take a look at my overlay (it always has the latest version) and
check for prefix related issues. Take a look also at xlwt and xlrd in dev-python
in the same overlay.
--
Krzysztof Pawlik nelchael at gentoo.org key id: 0xF6A80E46
desktop-misc, java, vim, kernel, python, apache...
signature.asc
Description: OpenPGP digital signature
Re: [gentoo-dev] New eclass for Python
On 29/02/12 04:21, Sergei Trofimovich wrote:
On Tue, 28 Feb 2012 22:13:36 +0100
Krzysztof Pawlik nelch...@gentoo.org wrote:
I'm attaching the eclass itself and two ebuilds using it, code is also
available
in my overlay at
http://git.overlays.gentoo.org/gitweb/?p=dev/nelchael.git;a=summary
Nice!
eclass/python-distutils-ng.eclass
236 # TODO: Pick default implementation
237 for impl in python{2_7,2_6,2_5,3_2,2_1} pypy{1_8,1_7}
jython2_5; do
Looks like 2_1 should be 3_1
Yes! Thank you.
minor suggestion:
s/_python-distutils-ng_run_for_all_impls/_python-distutils-ng_run_for_each_impl/g
LGTM, changed.
--
Krzysztof Pawlik nelchael at gentoo.org key id: 0xF6A80E46
desktop-misc, java, vim, kernel, python, apache...
signature.asc
Description: OpenPGP digital signature
Re: [gentoo-dev] New eclass for Python
On 29/02/12 08:49, Paweł Hajdan, Jr. wrote: On 2/28/12 10:13 PM, Krzysztof Pawlik wrote: Highlights: - 400 lines of code including documentation - should work for 95% of packages (my educated guess) - did I mention it's *SIMPLE*? - easy to maintain read so it's also easy to use This is awesome! Compare that to over 3000 LOC of python.eclass. :) Count distutils.eclass too: $ wc -l python-distutils-ng.eclass python.eclass distutils.eclass 364 python-distutils-ng.eclass 3168 python.eclass 592 distutils.eclass Important thing: I'm not aiming at having 100% functionality of current python.eclass+distutils.eclass in the new one, I think that simplicity is more important that supporting every possible, obscure case that's out there. Exactly. If there are no objections then during the weekend (March 3, 4) I will add this to portage (after finishing remaining TODO items, PyPy requires 4G of RAM(!!)). I second this so much (didn't review the eclass in detail, but it's obviously better than python.eclass). Thank you :) What's the plan to retire python.eclass? I'm not entirely sure - there's no pressure to migrate off it, both can coexist happily. I don't think it can be retired until be have all functionality from python.eclass+distutils.eclass somewhere (especially tests!). If the package is distutils-based then migration should be pretty painless and new packages (or version/revision bumps) can start using it as soon as it hits portage. -- Krzysztof Pawlik nelchael at gentoo.org key id: 0xF6A80E46 desktop-misc, java, vim, kernel, python, apache... signature.asc Description: OpenPGP digital signature
Re: [gentoo-dev] New eclass for Python
On Wed, 29 Feb 2012 18:38:22 +0100 Krzysztof Pawlik nelch...@gentoo.org wrote: On 29/02/12 08:49, Paweł Hajdan, Jr. wrote: This is awesome! Compare that to over 3000 LOC of python.eclass. :) Count distutils.eclass too: $ wc -l python-distutils-ng.eclass python.eclass distutils.eclass 364 python-distutils-ng.eclass 3168 python.eclass 592 distutils.eclass cvs/gentoo-x86/eclass $ cloc --by-file \ --force-lang=Bourne Again Shell,eclass python.eclass \ distutils .eclass python-distutils-ng.eclass 3 text files. 3 unique files. 0 files ignored. http://cloc.sourceforge.net v 1.55 T=0.5 s (6.0 files/s, 8294.0 lines/s) --- File blank comment code --- python.eclass396 288 2494 distutils.eclass 111 86 395 python-distutils-ng.eclass51 110 216 --- SUM: 558 484 3105 --- jer
Re: [gentoo-dev] New eclass for Python
On Tue, 2012-02-28 at 22:13 +0100, Krzysztof Pawlik wrote: Hello, After some work during weekend on Python packages I've decided to start a rewrite of Python/distutils eclass for installing Python packages. My main goal was simplicity and functionality similar to ruby-ng.eclass (thanks Ruby team for your great work!). Python team members already contributed comments and suggestions and helped me to make the eclass better, thank you! Highlights: - *SIMPLE*next - uses PYTHON_TARGETS use-expand (no more python-updater, wh!) - EAPI4 required, uses REQUIRED_USE - 400 lines of code including documentation - should work for 95% of packages (my educated guess) - did I mention it's *SIMPLE*? - easy to maintain read so it's also easy to use Important thing: I'm not aiming at having 100% functionality of current python.eclass+distutils.eclass in the new one, I think that simplicity is more important that supporting every possible, obscure case that's out there. I'm attaching the eclass itself and two ebuilds using it, code is also available in my overlay at http://git.overlays.gentoo.org/gitweb/?p=dev/nelchael.git;a=summary If there are no objections then during the weekend (March 3, 4) I will add this to portage (after finishing remaining TODO items, PyPy requires 4G of RAM(!!)). The proposed eclass omits three features from python.eclass which are heavily used in the gnome stack. First, it does not set EPYTHON, which is a problem for the many packages whose build systems execute /usr/bin/python and assume that it's a generic python2 or the same version of python2.x for which the package is being built. Second, there doesn't seem to be any support for packages that do not install in python's site-packages and do not allow multiple python ABIs. If I have, for example, a package that installs python modules in /usr/lib/appname or /usr/share/appname, how can I specify that PYTHON_TARGETS=python2.6 or python2.7 or python3.2 is allowed, but something like PYTHON_TARGETS=python2.7 python3.2 is not? Third, python-distutils-ng_doscript() installs only one file at a time (no built-in support for multiple files at a time or for recursing through a directory tree), installs it only in /usr/bin (a package might want it in e.g. /usr/libexec or /usr/share/appname/scripts), and always creates scriptname-IMPLEMENTATION (polluting tab-completion in /usr/bin for the common case of programs that do not support multiple python ABIs). As a result, it is far less useful than python.eclass's python_convert_shebangs(). -Alexandre.
Re: [gentoo-dev] New eclass for Python
On 29/02/12 20:51, Alexandre Rostovtsev wrote:
On Tue, 2012-02-28 at 22:13 +0100, Krzysztof Pawlik wrote:
Hello,
After some work during weekend on Python packages I've decided to start a
rewrite of Python/distutils eclass for installing Python packages. My main
goal
was simplicity and functionality similar to ruby-ng.eclass (thanks Ruby team
for
your great work!). Python team members already contributed comments and
suggestions and helped me to make the eclass better, thank you!
Highlights:
- *SIMPLE*next
- uses PYTHON_TARGETS use-expand (no more python-updater, wh!)
- EAPI4 required, uses REQUIRED_USE
- 400 lines of code including documentation
- should work for 95% of packages (my educated guess)
- did I mention it's *SIMPLE*?
- easy to maintain read so it's also easy to use
Important thing: I'm not aiming at having 100% functionality of current
python.eclass+distutils.eclass in the new one, I think that simplicity is
more
important that supporting every possible, obscure case that's out there.
I'm attaching the eclass itself and two ebuilds using it, code is also
available
in my overlay at
http://git.overlays.gentoo.org/gitweb/?p=dev/nelchael.git;a=summary
If there are no objections then during the weekend (March 3, 4) I will add
this
to portage (after finishing remaining TODO items, PyPy requires 4G of
RAM(!!)).
The proposed eclass omits three features from python.eclass which are
heavily used in the gnome stack.
Correct me if I'm wrong, but Gnome doesn't use standard distutils?
Regardless of this eclass you can still use python.eclass, it not going anywhere
anytime soon (just like I wrote in one of previous e-mails).
First, it does not set EPYTHON, which is a problem for the many packages
whose build systems execute /usr/bin/python and assume that it's a
generic python2 or the same version of python2.x for which the package
is being built.
Setting EPYTHON is not a problem -- just another variable.
Second, there doesn't seem to be any support for packages that do not
install in python's site-packages and do not allow multiple python ABIs.
If I have, for example, a package that installs python modules
in /usr/lib/appname or /usr/share/appname, how can I specify that
PYTHON_TARGETS=python2.6 or python2.7 or python3.2 is allowed, but
something like PYTHON_TARGETS=python2.7 python3.2 is not?
You're correct, note that I've stressed that this eclass is mainly for
distutils-based packages. I'm not using Gnome, so can you provide some package
examples that I can look at?
personal opinion
If package decides to use given language then please, please play by the rules
set by the rest of world (Ruby - gems, Python - distutils, Perl - CPAN, PHP
- PEAR).
I don't like installing Python code outside of site-packages, the only exception
to that rule is portage (at least for now).
/personal opinion
Third, python-distutils-ng_doscript() installs only one file at a time
(no built-in support for multiple files at a time or for recursing
through a directory tree),
Yes, that why bash has `for' loop ;)
installs it only in /usr/bin (a package might
want it in e.g. /usr/libexec or /usr/share/appname/scripts)
Yes, I might add --destdir (or something like that) later on.
and always
creates scriptname-IMPLEMENTATION (polluting tab-completion in /usr/bin
for the common case of programs that do not support multiple python
ABIs). As a result, it is far less useful than python.eclass's
python_convert_shebangs().
Yes, that pollutes this name space, but I'm not buying this argument. Do you
know how CVS and .svn pollute my tab-completion list? I even set FIGNORE so I
can live with it.
I'd be happy to hear how to solve this - what prefix or suffix to use? One way
would be quite trivial: if only one implementation is enabled do not create
script-${impl}, go with single file, does that sound good?
--
Krzysztof Pawlik nelchael at gentoo.org key id: 0xF6A80E46
desktop-misc, java, vim, kernel, python, apache...
signature.asc
Description: OpenPGP digital signature
Re: [gentoo-dev] New eclass for Python
Am Mittwoch 29 Februar 2012, 21:24:49 schrieb Krzysztof Pawlik: Second, there doesn't seem to be any support for packages that do not install in python's site-packages and do not allow multiple python ABIs. If I have, for example, a package that installs python modules in /usr/lib/appname or /usr/share/appname, how can I specify that PYTHON_TARGETS=python2.6 or python2.7 or python3.2 is allowed, but something like PYTHON_TARGETS=python2.7 python3.2 is not? You're correct, note that I've stressed that this eclass is mainly for distutils-based packages. I'm not using Gnome, so can you provide some package examples that I can look at? personal opinion If package decides to use given language then please, please play by the rules set by the rest of world (Ruby - gems, Python - distutils, Perl - CPAN, PHP - PEAR). I don't like installing Python code outside of site-packages, the only exception to that rule is portage (at least for now). /personal opinion We will hit the same problem with KDE (actually we already hit it): it has various types of scripting support, and each installs a KDE library linked to whatever language interpreter. (Now, that library- is it a Python/Ruby library or a KDE library? Because it is at the proper place for KDE stuff :) It's not just about calling an external language but also about embedding the interpreter for in-app scripting... and KDE rather heavily relies on python. -- Andreas K. Huettel Gentoo Linux developer dilfri...@gentoo.org http://www.akhuettel.de/ signature.asc Description: This is a digitally signed message part.
Re: [gentoo-dev] New eclass for Python
On 29/02/12 22:08, Andreas K. Huettel wrote: Am Mittwoch 29 Februar 2012, 21:24:49 schrieb Krzysztof Pawlik: Second, there doesn't seem to be any support for packages that do not install in python's site-packages and do not allow multiple python ABIs. If I have, for example, a package that installs python modules in /usr/lib/appname or /usr/share/appname, how can I specify that PYTHON_TARGETS=python2.6 or python2.7 or python3.2 is allowed, but something like PYTHON_TARGETS=python2.7 python3.2 is not? You're correct, note that I've stressed that this eclass is mainly for distutils-based packages. I'm not using Gnome, so can you provide some package examples that I can look at? personal opinion If package decides to use given language then please, please play by the rules set by the rest of world (Ruby - gems, Python - distutils, Perl - CPAN, PHP - PEAR). I don't like installing Python code outside of site-packages, the only exception to that rule is portage (at least for now). /personal opinion We will hit the same problem with KDE (actually we already hit it): it has various types of scripting support, and each installs a KDE library linked to whatever language interpreter. (Now, that library- is it a Python/Ruby library or a KDE library? Because it is at the proper place for KDE stuff :) It's not just about calling an external language but also about embedding the interpreter for in-app scripting... and KDE rather heavily relies on python. I agree with last sentence - it's about embedding Python/Ruby/Foo, *this eclass is about installing packages for Python*. Compiling against an implementation is something completely different -- for example try building against Jython. -- Krzysztof Pawlik nelchael at gentoo.org key id: 0xF6A80E46 desktop-misc, java, vim, kernel, python, apache... signature.asc Description: OpenPGP digital signature
Re: [gentoo-dev] New eclass for Python
On Wed, 2012-02-29 at 21:24 +0100, Krzysztof Pawlik wrote:
On 29/02/12 20:51, Alexandre Rostovtsev wrote:
The proposed eclass omits three features from python.eclass which are
heavily used in the gnome stack.
Correct me if I'm wrong, but Gnome doesn't use standard distutils?
Gnome is mostly written in C and therefore uses standard autotools :)
Second, there doesn't seem to be any support for packages that do not
install in python's site-packages and do not allow multiple python ABIs.
If I have, for example, a package that installs python modules
in /usr/lib/appname or /usr/share/appname, how can I specify that
PYTHON_TARGETS=python2.6 or python2.7 or python3.2 is allowed, but
something like PYTHON_TARGETS=python2.7 python3.2 is not?
You're correct, note that I've stressed that this eclass is mainly for
distutils-based packages. I'm not using Gnome, so can you provide some package
examples that I can look at?
personal opinion
If package decides to use given language then please, please play by the rules
set by the rest of world (Ruby - gems, Python - distutils, Perl - CPAN, PHP
- PEAR).
I don't like installing Python code outside of site-packages, the only
exception
to that rule is portage (at least for now).
/personal opinion
Some non-python packages allow python-based plugins. Obviously these
plugins live in the package's plugin directory (not in python's
site-packages) and use the package's main build system (not distutils),
and multiple python ABIs cannot be supported because that would result
in colliding plugins. Typical examples are app-editors/gedit,
media-gfx/gimp, media-sound/rhythmbox, or media-video/totem.
Some packages install a C library that links to a specific version of
libpython or that defines a particular python version string at compile
time, making it impossible to use the package with multiple python ABIs.
Examples I know are dev-libs/libpeas and dev-python/nautilus-python.
And then there are packages which could support e.g. multiple python2
ABIs in theory, but doing so in practice would require a fair bit of
patching, taking substantial effort with no real benefit for end users.
An example that springs to mind here is gnome-extra/zeitgeist.
I'd be happy to hear how to solve this - what prefix or suffix to use? One way
would be quite trivial: if only one implementation is enabled do not create
script-${impl}, go with single file, does that sound good?
That would be the ideal solution.
-Alexandre.
[gentoo-dev] New eclass for Python
Hello,
After some work during weekend on Python packages I've decided to start a
rewrite of Python/distutils eclass for installing Python packages. My main goal
was simplicity and functionality similar to ruby-ng.eclass (thanks Ruby team for
your great work!). Python team members already contributed comments and
suggestions and helped me to make the eclass better, thank you!
Highlights:
- *SIMPLE*next
- uses PYTHON_TARGETS use-expand (no more python-updater, wh!)
- EAPI4 required, uses REQUIRED_USE
- 400 lines of code including documentation
- should work for 95% of packages (my educated guess)
- did I mention it's *SIMPLE*?
- easy to maintain read so it's also easy to use
Important thing: I'm not aiming at having 100% functionality of current
python.eclass+distutils.eclass in the new one, I think that simplicity is more
important that supporting every possible, obscure case that's out there.
I'm attaching the eclass itself and two ebuilds using it, code is also available
in my overlay at
http://git.overlays.gentoo.org/gitweb/?p=dev/nelchael.git;a=summary
If there are no objections then during the weekend (March 3, 4) I will add this
to portage (after finishing remaining TODO items, PyPy requires 4G of RAM(!!)).
--
Krzysztof Pawlik nelchael at gentoo.org key id: 0xF6A80E46
desktop-misc, java, vim, kernel, python, apache...
# Copyright 1999-2012 Gentoo Foundation
# Distributed under the terms of the GNU General Public License v2
# $Header$
EAPI=4
inherit python-distutils-ng
DESCRIPTION=Python library to create spreadsheet files compatible with Excel
HOMEPAGE=http://pypi.python.org/pypi/xlwt;
SRC_URI=mirror://pypi/${PN:0:1}/${PN}/${P}.tar.gz
LICENSE=BSD
SLOT=0
KEYWORDS=~amd64 ~ppc-aix ~hppa-hpux ~ia64-hpux ~x86-interix ~x86-linux
~sparc-solaris ~x86-solaris
IUSE=examples
DEPEND=
RDEPEND=
python_prepare_all() {
sed -i \
-e s,'doc,# 'doc,g \
-e s,'exa,# 'exa,g \
setup.py || die
}
python_install_all() {
dohtml xlwt/doc/*.html
if use examples; then
insinto /usr/share/doc/${PF}
doins -r xlwt/examples
fi
}
# Copyright 1999-2012 Gentoo Foundation
# Distributed under the terms of the GNU General Public License v2
# $Header$
EAPI=4
inherit python-distutils-ng
DESCRIPTION=Library for developers to extract data from Microsoft Excel (tm)
spreadsheet files
HOMEPAGE=http://pypi.python.org/pypi/xlrd;
SRC_URI=mirror://pypi/${PN:0:1}/${PN}/${P}.tar.gz
LICENSE=BSD
SLOT=0
KEYWORDS=~amd64 ~ppc-aix ~hppa-hpux ~ia64-hpux ~x86-interix ~x86-linux
~sparc-solaris ~x86-solaris
IUSE=examples
DEPEND=
RDEPEND=
python_prepare_all() {
sed -i \
-e s,'doc,# 'doc,g \
-e s,'exa,# 'exa,g \
setup.py || die
}
python_install_all() {
rm -f ${D}/usr/bin/*.py || die
python-distutils-ng_doscript scripts/runxlrd.py
dohtml xlrd/doc/*.html
if use examples; then
insinto /usr/share/doc/${PF}
doins -r xlrd/examples
fi
}
# Copyright 2009-2012 Gentoo Foundation.
# Distributed under the terms of the GNU General Public License v2
# $Header$
# This file contains descriptions of PYTHON_TARGETS USE_EXPAND flags.
python2_5 - Build with Python 2.5
python2_6 - Build with Python 2.6
python2_7 - Build with Python 2.7
python3_1 - Build with Python 3.1
python3_2 - Build with Python 3.2
jython2_5 - Build with Jython 2.5
pypy1_7 - Build with PyPy 1.7
pypy1_8 - Build with PyPy 1.8
# Copyright 1999-2012 Gentoo Foundation
# Distributed under the terms of the GNU General Public License v2
# $Header$
# @ECLASS: python-distutils-ng
# @MAINTAINER:
# Python herd pyt...@gentoo.org
# @AUTHOR:
# Author: Krzysztof Pawlik nelch...@gentoo.org
# @BLURB: An eclass for installing Python packages using distutils with proper
# support for multiple Python slots.
# @DESCRIPTION:
# The Python eclass is designed to allow an easier installation of Python
# packages and their incorporation into the Gentoo Linux system.
#
# TODO: Document implementations!
# @ECLASS-VARIABLE: PYTHON_COMPAT
# @DESCRIPTION:
# This variable contains a space separated list of implementations (see above) a
# package is compatible to. It must be set before the `inherit' call. The
# default is to enable all implementations.
if [[ -z ${PYTHON_COMPAT} ]]; then
# Default: pure python, support all implementations
PYTHON_COMPAT= python2_5 python2_6 python2_7
PYTHON_COMPAT+= python3_1 python3_2
PYTHON_COMPAT+= jython2_5
PYTHON_COMPAT+= pypy1_7 pypy1_8
fi
# @ECLASS-VARIABLE: PYTHON_OPTIONAL
# @DESCRIPTION:
# Set the value to yes to make the dependency on a Python interpreter
# optional.
# @ECLASS-VARIABLE: PYTHON_DISABLE_COMPILATION
# @DESCRIPTION:
# Set the value to yes to skip compilation and/or optimization of Python
# modules.
EXPORT_FUNCTIONS src_unpack src_prepare src_configure src_compile src_test
Re: [gentoo-dev] New eclass for Python
On Tue, 28 Feb 2012 22:13:36 +0100
Krzysztof Pawlik nelch...@gentoo.org wrote:
I'm attaching the eclass itself and two ebuilds using it, code is also
available
in my overlay at
http://git.overlays.gentoo.org/gitweb/?p=dev/nelchael.git;a=summary
Nice!
eclass/python-distutils-ng.eclass
236 # TODO: Pick default implementation
237 for impl in python{2_7,2_6,2_5,3_2,2_1} pypy{1_8,1_7}
jython2_5; do
Looks like 2_1 should be 3_1
minor suggestion:
s/_python-distutils-ng_run_for_all_impls/_python-distutils-ng_run_for_each_impl/g
--
Sergei
signature.asc
Description: PGP signature
Re: [gentoo-dev] New eclass for Python
On 2/28/12 10:13 PM, Krzysztof Pawlik wrote: Highlights: - 400 lines of code including documentation - should work for 95% of packages (my educated guess) - did I mention it's *SIMPLE*? - easy to maintain read so it's also easy to use This is awesome! Compare that to over 3000 LOC of python.eclass. :) Important thing: I'm not aiming at having 100% functionality of current python.eclass+distutils.eclass in the new one, I think that simplicity is more important that supporting every possible, obscure case that's out there. Exactly. If there are no objections then during the weekend (March 3, 4) I will add this to portage (after finishing remaining TODO items, PyPy requires 4G of RAM(!!)). I second this so much (didn't review the eclass in detail, but it's obviously better than python.eclass). What's the plan to retire python.eclass? signature.asc Description: OpenPGP digital signature
