Re: [PATCH 2/3] gnu: Add python-pyxb.
Hello, Marius Bakke skribis: > I'm not sure, we would have to do this with every package that uses > the pysam cython interface. This particular issue is described in the > manual: > > https://www.gnu.org/software/guix/manual/html_node/package-Reference.html > > The propagated-inputs section says "For example this is necessary when a > C/C++ library needs headers of another library to compile, or when a > pkg-config file refers to another one via its Requires field.". > > So I think it's a pretty common propagation scenario. See e.g. 618089f9 > for a less convoluted case (C++ library that propagates zlib headers). > > Perhaps a "propagated-build-inputs" functionality would be useful, so > that libraries are only propagated at build time and not to the profile. FWIW, Nixpkgs makes (or made) this distinction, but I could not think of a situation where it would make sense to propagate in one context and not the other. Ludo’.
Re: [PATCH 2/3] gnu: Add python-pyxb.
On 27/09/16 04:32, Marius Bakke wrote: Ben Woodcroft writes: On 24/09/16 13:15, Marius Bakke wrote: Ben Woodcroft writes: Htslib is propagated because it is included by the installed samtools and bcftools headers; at least one of pysams dependencies failed to build when it was a regular input. Ah, I see. Can this be avoided with patchelf? I don't think so. The htslib headers need to be propagated for anything that uses header files from pysam, since they contain contain code such as "include ". Perhaps you can update the comment to make it more clear? Patchelf is probably what made that work when htslib was bundled. Ah, ok I see, so the problem is that python-plastid requires the .h files during build. Given that this is the only package that has an issue, and perhaps pysam doesn't intend to provide these .h files as part of it's "interface", I think it might be best to simply put htslib as an input to python-plastid. I'm not sure, we would have to do this with every package that uses the pysam cython interface. Ah, you're right. I overlooked the cython interface. I pushed the series propagating htslib with pysam, ending in 99caa6f. Thanks for persevering. ben
Re: [PATCH 2/3] gnu: Add python-pyxb.
Ben Woodcroft writes: > On 24/09/16 13:15, Marius Bakke wrote: >> Ben Woodcroft writes: >> Htslib is propagated because it is included by the installed samtools and bcftools headers; at least one of pysams dependencies failed to build when it was a regular input. >>> Ah, I see. Can this be avoided with patchelf? >> I don't think so. The htslib headers need to be propagated for anything >> that uses header files from pysam, since they contain contain code such >> as "include ". Perhaps you can update the comment to make >> it more clear? >> >> Patchelf is probably what made that work when htslib was bundled. > Ah, ok I see, so the problem is that python-plastid requires the .h > files during build. Given that this is the only package that has an > issue, and perhaps pysam doesn't intend to provide these .h files as > part of it's "interface", I think it might be best to simply put htslib > as an input to python-plastid. I'm not sure, we would have to do this with every package that uses the pysam cython interface. This particular issue is described in the manual: https://www.gnu.org/software/guix/manual/html_node/package-Reference.html The propagated-inputs section says "For example this is necessary when a C/C++ library needs headers of another library to compile, or when a pkg-config file refers to another one via its Requires field.". So I think it's a pretty common propagation scenario. See e.g. 618089f9 for a less convoluted case (C++ library that propagates zlib headers). Perhaps a "propagated-build-inputs" functionality would be useful, so that libraries are only propagated at build time and not to the profile. If we don't do propagation, at least add a comment to python-plastid explaining that htslib is there to avoid propagating from pysam, since the plastid code contains no direct references to htslib. Your other changes LGTM. Thanks! Marius
Re: [PATCH 2/3] gnu: Add python-pyxb.
On 24/09/16 13:15, Marius Bakke wrote: Ben Woodcroft writes: Htslib is propagated because it is included by the installed samtools and bcftools headers; at least one of pysams dependencies failed to build when it was a regular input. Ah, I see. Can this be avoided with patchelf? I don't think so. The htslib headers need to be propagated for anything that uses header files from pysam, since they contain contain code such as "include ". Perhaps you can update the comment to make it more clear? Patchelf is probably what made that work when htslib was bundled. Ah, ok I see, so the problem is that python-plastid requires the .h files during build. Given that this is the only package that has an issue, and perhaps pysam doesn't intend to provide these .h files as part of it's "interface", I think it might be best to simply put htslib as an input to python-plastid. That all checks out, here's a patch series to do this plus update plastid. OK? ben >From b5ce635ef019f95ee2857060cbab1b2d4994f021 Mon Sep 17 00:00:00 2001 From: Marius Bakke Date: Fri, 16 Sep 2016 18:05:15 +0100 Subject: [PATCH 01/10] gnu: python-pysam: Update to 0.9.1.4. * gnu/packages/bioinformatics.scm (python-pysam, python2-pysam): Update to 0.9.1.4. --- gnu/packages/bioinformatics.scm | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/gnu/packages/bioinformatics.scm b/gnu/packages/bioinformatics.scm index 1bf91a9..6e83461 100644 --- a/gnu/packages/bioinformatics.scm +++ b/gnu/packages/bioinformatics.scm @@ -1290,13 +1290,13 @@ multiple sequence alignments.") (define-public python-pysam (package (name "python-pysam") -(version "0.8.4") +(version "0.9.1.4") (source (origin (method url-fetch) (uri (pypi-uri "pysam" version)) (sha256 (base32 -"1slx5mb94mzm5qzk52q270sab0sar95j67w1g1k452nz3s9j7krh" +"1i1djacqbr88y7w18b4aa78zxnsyr4sz7yqdq2spi7gs0y6pzvjn" (build-system python-build-system) (arguments `(#:tests? #f ; tests are excluded in the manifest -- 2.9.2 >From 2f815c0435eb514f850c6bfa8b0996fd6d3cb538 Mon Sep 17 00:00:00 2001 From: Marius Bakke Date: Sat, 17 Sep 2016 10:33:28 +0100 Subject: [PATCH 02/10] gnu: Add python-pyxb. * gnu/packages/xml.scm (python-pyxb, python2-pyxb): New variables. --- gnu/packages/xml.scm | 25 + 1 file changed, 25 insertions(+) diff --git a/gnu/packages/xml.scm b/gnu/packages/xml.scm index 7befad5..cd3426b 100644 --- a/gnu/packages/xml.scm +++ b/gnu/packages/xml.scm @@ -13,6 +13,7 @@ ;;; Copyright © 2016 Jan Nieuwenhuizen ;;; Copyright © 2016 ng0 ;;; Copyright © 2016 Tobias Geerinckx-Rice +;;; Copyright © 2016 Marius Bakke ;;; ;;; This file is part of GNU Guix. ;;; @@ -729,6 +730,30 @@ different Unicode encodings which happen automatically during parsing/saving.") (license license:expat))) +(define-public python-pyxb + (package +(name "python-pyxb") +(version "1.2.5") +(source (origin + (method url-fetch) + (uri (pypi-uri "PyXB" version)) + (sha256 + (base32 +"0rzzwibfqa28gxgcxx4cybx1qcg0g6fand06ykj3gz7z5kp653sf" +(build-system python-build-system) +(home-page "http://pyxb.sourceforge.net/";) +(synopsis "Python XML Schema Bindings") +(description + "PyXB (\"pixbee\") is a pure Python package that generates Python source +code for classes that correspond to data structures defined by XMLSchema.") +(license (list license:asl2.0; Most files. + license:expat ; pyxb/utils/six.py + license:gpl2 ; bundled jquery in doc is dual MIT/GPL2 + license:psfl ; pyxb/utils/activestate.py + +(define-public python2-pyxb + (package-with-python2 python-pyxb)) + (define-public xmlto (package (name "xmlto") -- 2.9.2 >From ee973d7f50f7a874a3917c3ea20bcc8e1c17cba3 Mon Sep 17 00:00:00 2001 From: Marius Bakke Date: Sat, 17 Sep 2016 10:42:56 +0100 Subject: [PATCH 03/10] gnu: python2-pbcore: Update to 1.2.10. * gnu/packages/bioinformatics.scm (python2-pbcore): Update to 1.2.10. [propagated-inputs]: New field. Add python2-pyxb. --- gnu/packages/bioinformatics.scm | 6 -- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/gnu/packages/bioinformatics.scm b/gnu/packages/bioinformatics.scm index 6e83461..a935d5a 100644 --- a/gnu/packages/bioinformatics.scm +++ b/gnu/packages/bioinformatics.scm @@ -3376,13 +3376,13 @@ interrupted by stop codons. OrfM finds and prints these ORFs.") (define-public python2-pbcore (package (name "python2-pbcore") -(version "1.2.8") +(version "1.2.10") (source (origin (method url-fetch) (uri (pypi-uri "pbcore" version)) (sha256 (base32 -"02pfn5raa3zf739672bg0dkx7z3j2c4nx7vmpfjqy5b12jrqpymk")))
Re: [PATCH 2/3] gnu: Add python-pyxb.
Ben Woodcroft writes: >> Htslib is propagated because it >> is included by the installed samtools and bcftools headers; at least one >> of pysams dependencies failed to build when it was a regular input. > Ah, I see. Can this be avoided with patchelf? I don't think so. The htslib headers need to be propagated for anything that uses header files from pysam, since they contain contain code such as "include ". Perhaps you can update the comment to make it more clear? Patchelf is probably what made that work when htslib was bundled. Cheers, Marius
Re: [PATCH 2/3] gnu: Add python-pyxb.
On 24/09/16 01:56, Marius Bakke wrote: Ben Woodcroft writes: Also, I notice that pysam bundles htslib, bcftools and samtools C code. Hopefully it should be straightforward enough to remove htslib as there are install instructions, I'm not sure about the other two. This shouldn't block the patch here, but would you mind taking a look? http://pysam.readthedocs.io/en/latest/installation.html#installation I had a go at this, and also enabled tests since I was reading the build system anyway. Samtools and bcftools does not seem possible to un-bundle at this time, but htslib was straightforward. OK. I don't think it needs to be propagated though, right? Also, would you mind separating the change to modify-phases syntax and unbundling of htslib into two patches please? Other than that this whole series LGTM. Modify-phases is now a separate patch. Thanks. Htslib is propagated because it is included by the installed samtools and bcftools headers; at least one of pysams dependencies failed to build when it was a regular input. Ah, I see. Can this be avoided with patchelf? ben
Re: [PATCH 2/3] gnu: Add python-pyxb.
Marius Bakke writes: > Ben Woodcroft writes: > >>> Also, I notice that pysam bundles htslib, bcftools and samtools C code. Hopefully it should be straightforward enough to remove htslib as there are install instructions, I'm not sure about the other two. This shouldn't block the patch here, but would you mind taking a look? http://pysam.readthedocs.io/en/latest/installation.html#installation >>> I had a go at this, and also enabled tests since I was reading the build >>> system anyway. Samtools and bcftools does not seem possible to un-bundle >>> at this time, but htslib was straightforward. >> OK. I don't think it needs to be propagated though, right? Also, would >> you mind separating the change to modify-phases syntax and unbundling of >> htslib into two patches please? Other than that this whole series LGTM. > > Modify-phases is now a separate patch. Htslib is propagated because it > is included by the installed samtools and bcftools headers; at least one > of pysams dependencies failed to build when it was a regular input. > > I added some comments and other cosmetic changes. Updated patch below. Oops, found a spelling error in the previous patch. Fixed. >From eaa38fb4174997f6282ff7d278a6aa4153b75bcf Mon Sep 17 00:00:00 2001 From: Marius Bakke Date: Fri, 16 Sep 2016 18:05:15 +0100 Subject: [PATCH 1/7] gnu: python-pysam: Update to 0.9.1.4. * gnu/packages/bioinformatics.scm (python-pysam, python2-pysam): Update to 0.9.1.4. --- gnu/packages/bioinformatics.scm | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/gnu/packages/bioinformatics.scm b/gnu/packages/bioinformatics.scm index 1bf91a9..6e83461 100644 --- a/gnu/packages/bioinformatics.scm +++ b/gnu/packages/bioinformatics.scm @@ -1290,13 +1290,13 @@ multiple sequence alignments.") (define-public python-pysam (package (name "python-pysam") -(version "0.8.4") +(version "0.9.1.4") (source (origin (method url-fetch) (uri (pypi-uri "pysam" version)) (sha256 (base32 -"1slx5mb94mzm5qzk52q270sab0sar95j67w1g1k452nz3s9j7krh" +"1i1djacqbr88y7w18b4aa78zxnsyr4sz7yqdq2spi7gs0y6pzvjn" (build-system python-build-system) (arguments `(#:tests? #f ; tests are excluded in the manifest -- 2.10.0 >From 5431b05fe66398f5c44a1f0a81e1e90663550ac1 Mon Sep 17 00:00:00 2001 From: Marius Bakke Date: Sat, 17 Sep 2016 10:33:28 +0100 Subject: [PATCH 2/7] gnu: Add python-pyxb. * gnu/packages/xml.scm (python-pyxb, python2-pyxb): New variables. --- gnu/packages/xml.scm | 25 + 1 file changed, 25 insertions(+) diff --git a/gnu/packages/xml.scm b/gnu/packages/xml.scm index 7befad5..cd3426b 100644 --- a/gnu/packages/xml.scm +++ b/gnu/packages/xml.scm @@ -13,6 +13,7 @@ ;;; Copyright © 2016 Jan Nieuwenhuizen ;;; Copyright © 2016 ng0 ;;; Copyright © 2016 Tobias Geerinckx-Rice +;;; Copyright © 2016 Marius Bakke ;;; ;;; This file is part of GNU Guix. ;;; @@ -729,6 +730,30 @@ different Unicode encodings which happen automatically during parsing/saving.") (license license:expat))) +(define-public python-pyxb + (package +(name "python-pyxb") +(version "1.2.5") +(source (origin + (method url-fetch) + (uri (pypi-uri "PyXB" version)) + (sha256 + (base32 +"0rzzwibfqa28gxgcxx4cybx1qcg0g6fand06ykj3gz7z5kp653sf" +(build-system python-build-system) +(home-page "http://pyxb.sourceforge.net/";) +(synopsis "Python XML Schema Bindings") +(description + "PyXB (\"pixbee\") is a pure Python package that generates Python source +code for classes that correspond to data structures defined by XMLSchema.") +(license (list license:asl2.0; Most files. + license:expat ; pyxb/utils/six.py + license:gpl2 ; bundled jquery in doc is dual MIT/GPL2 + license:psfl ; pyxb/utils/activestate.py + +(define-public python2-pyxb + (package-with-python2 python-pyxb)) + (define-public xmlto (package (name "xmlto") -- 2.10.0 >From 923fc43d8c665d1a2795ea2c221003e54544a06b Mon Sep 17 00:00:00 2001 From: Marius Bakke Date: Sat, 17 Sep 2016 10:42:56 +0100 Subject: [PATCH 3/7] gnu: python2-pbcore: Update to 1.2.10. * gnu/packages/bioinformatics.scm (python2-pbcore): Update to 1.2.10. [propagated-inputs]: New field. Add python2-pyxb. --- gnu/packages/bioinformatics.scm | 6 -- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/gnu/packages/bioinformatics.scm b/gnu/packages/bioinformatics.scm index 6e83461..a935d5a 100644 --- a/gnu/packages/bioinformatics.scm +++ b/gnu/packages/bioinformatics.scm @@ -3376,13 +3376,13 @@ interrupted by stop codons. OrfM finds and prints these ORFs.") (define-public python2-pbcore (package (name "python2-pbcore") -(version "1.2.8") +(version "1.2
Re: [PATCH 2/3] gnu: Add python-pyxb.
Ben Woodcroft writes: >> >>> Also, I notice that pysam bundles htslib, bcftools and samtools C code. >>> Hopefully it should be straightforward enough to remove htslib as there >>> are install instructions, I'm not sure about the other two. This >>> shouldn't block the patch here, but would you mind taking a look? >>> http://pysam.readthedocs.io/en/latest/installation.html#installation >> I had a go at this, and also enabled tests since I was reading the build >> system anyway. Samtools and bcftools does not seem possible to un-bundle >> at this time, but htslib was straightforward. > OK. I don't think it needs to be propagated though, right? Also, would > you mind separating the change to modify-phases syntax and unbundling of > htslib into two patches please? Other than that this whole series LGTM. Modify-phases is now a separate patch. Htslib is propagated because it is included by the installed samtools and bcftools headers; at least one of pysams dependencies failed to build when it was a regular input. I added some comments and other cosmetic changes. Updated patch below. Cheers! >From eaa38fb4174997f6282ff7d278a6aa4153b75bcf Mon Sep 17 00:00:00 2001 From: Marius Bakke Date: Fri, 16 Sep 2016 18:05:15 +0100 Subject: [PATCH 1/7] gnu: python-pysam: Update to 0.9.1.4. * gnu/packages/bioinformatics.scm (python-pysam, python2-pysam): Update to 0.9.1.4. --- gnu/packages/bioinformatics.scm | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/gnu/packages/bioinformatics.scm b/gnu/packages/bioinformatics.scm index 1bf91a9..6e83461 100644 --- a/gnu/packages/bioinformatics.scm +++ b/gnu/packages/bioinformatics.scm @@ -1290,13 +1290,13 @@ multiple sequence alignments.") (define-public python-pysam (package (name "python-pysam") -(version "0.8.4") +(version "0.9.1.4") (source (origin (method url-fetch) (uri (pypi-uri "pysam" version)) (sha256 (base32 -"1slx5mb94mzm5qzk52q270sab0sar95j67w1g1k452nz3s9j7krh" +"1i1djacqbr88y7w18b4aa78zxnsyr4sz7yqdq2spi7gs0y6pzvjn" (build-system python-build-system) (arguments `(#:tests? #f ; tests are excluded in the manifest -- 2.10.0 >From 5431b05fe66398f5c44a1f0a81e1e90663550ac1 Mon Sep 17 00:00:00 2001 From: Marius Bakke Date: Sat, 17 Sep 2016 10:33:28 +0100 Subject: [PATCH 2/7] gnu: Add python-pyxb. * gnu/packages/xml.scm (python-pyxb, python2-pyxb): New variables. --- gnu/packages/xml.scm | 25 + 1 file changed, 25 insertions(+) diff --git a/gnu/packages/xml.scm b/gnu/packages/xml.scm index 7befad5..cd3426b 100644 --- a/gnu/packages/xml.scm +++ b/gnu/packages/xml.scm @@ -13,6 +13,7 @@ ;;; Copyright © 2016 Jan Nieuwenhuizen ;;; Copyright © 2016 ng0 ;;; Copyright © 2016 Tobias Geerinckx-Rice +;;; Copyright © 2016 Marius Bakke ;;; ;;; This file is part of GNU Guix. ;;; @@ -729,6 +730,30 @@ different Unicode encodings which happen automatically during parsing/saving.") (license license:expat))) +(define-public python-pyxb + (package +(name "python-pyxb") +(version "1.2.5") +(source (origin + (method url-fetch) + (uri (pypi-uri "PyXB" version)) + (sha256 + (base32 +"0rzzwibfqa28gxgcxx4cybx1qcg0g6fand06ykj3gz7z5kp653sf" +(build-system python-build-system) +(home-page "http://pyxb.sourceforge.net/";) +(synopsis "Python XML Schema Bindings") +(description + "PyXB (\"pixbee\") is a pure Python package that generates Python source +code for classes that correspond to data structures defined by XMLSchema.") +(license (list license:asl2.0; Most files. + license:expat ; pyxb/utils/six.py + license:gpl2 ; bundled jquery in doc is dual MIT/GPL2 + license:psfl ; pyxb/utils/activestate.py + +(define-public python2-pyxb + (package-with-python2 python-pyxb)) + (define-public xmlto (package (name "xmlto") -- 2.10.0 >From 923fc43d8c665d1a2795ea2c221003e54544a06b Mon Sep 17 00:00:00 2001 From: Marius Bakke Date: Sat, 17 Sep 2016 10:42:56 +0100 Subject: [PATCH 3/7] gnu: python2-pbcore: Update to 1.2.10. * gnu/packages/bioinformatics.scm (python2-pbcore): Update to 1.2.10. [propagated-inputs]: New field. Add python2-pyxb. --- gnu/packages/bioinformatics.scm | 6 -- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/gnu/packages/bioinformatics.scm b/gnu/packages/bioinformatics.scm index 6e83461..a935d5a 100644 --- a/gnu/packages/bioinformatics.scm +++ b/gnu/packages/bioinformatics.scm @@ -3376,13 +3376,13 @@ interrupted by stop codons. OrfM finds and prints these ORFs.") (define-public python2-pbcore (package (name "python2-pbcore") -(version "1.2.8") +(version "1.2.10") (source (origin (method url-fetch) (uri (pypi-uri "pbcore"
Re: [PATCH 2/3] gnu: Add python-pyxb.
On 09/23/2016 01:15 AM, Marius Bakke wrote:
Ben Woodcroft writes:
Subject: [PATCH 1/3] gnu: python-pysam: Update to 0.9.1.4.
I'm not sure whether this is a product of the upgrade or not, but I
notice this in the build log. I think it is harmless though, WDYT?
starting phase `validate-runpath'
validating RUNPATH of 10 binaries in
"/gnu/store/bpiq3lm6b1kpf54i1vj2dl09ff293wic-python-pysam-0.9.1.4/lib"...
/gnu/store/bpiq3lm6b1kpf54i1vj2dl09ff293wic-python-pysam-0.9.1.4/lib/python3.4/site-packages/pysam-0.9.1.4-py3.4-linux-x86_64.egg/pysam/libchtslib.cpython-34m.so:
warning: RUNPATH contains bogus entries: ("pysam" "."
"build/lib.linux-x86_64-3.4/pysam")
I don't see this in the previous version, so it is a regression.
However, it should be mostly harmless. Readelf reports (when compiled
with external htslib, see below):
0x001d (RUNPATH)Library runpath:
[/gnu/store/m4gc2wx4q9if1vrhgclpspdil7rqsn21-python-3.4.3/lib:/gnu/store/ba22myqvxccwmmjwwq665rc43hanycxy-htslib-1.3.1/lib:build/lib.linux-x86_64-3.4/pysam:$ORIGIN:/gnu/store/m9vxvhdj691bq1f85lpflvnhcvrdilih-glibc-2.23/lib:/gnu/store/9nifwk709wajpyfwa0jzaa3p6mf10vxs-gcc-4.9.3-lib/lib:/gnu/store/xl19qrfzga52vrvp4ncccwjlnrjqwj95-ncurses-6.0/lib:/gnu/store/5992iq1v7arqa14ym3di58n4la0893nv-zlib-1.2.8/lib:/gnu/store/9nifwk709wajpyfwa0jzaa3p6mf10vxs-gcc-4.9.3-lib/lib/gcc/x86_64-unknown-linux-gnu/4.9.3/../../..]
Compared to the runpath of the same file currently in Guix:
0x001d (RUNPATH)Library runpath:
[/gnu/store/m4gc2wx4q9if1vrhgclpspdil7rqsn21-python-3.4.3/lib:/gnu/store/m9vxvhdj691bq1f85lpflvnhcvrdilih-glibc-2.23/lib:/gnu/store/9nifwk709wajpyfwa0jzaa3p6mf10vxs-gcc-4.9.3-lib/lib:/gnu/store/xl19qrfzga52vrvp4ncccwjlnrjqwj95-ncurses-6.0/lib:/gnu/store/5992iq1v7arqa14ym3di58n4la0893nv-zlib-1.2.8/lib:/gnu/store/9nifwk709wajpyfwa0jzaa3p6mf10vxs-gcc-4.9.3-lib/lib/gcc/x86_64-unknown-linux-gnu/4.9.3/../../..]
If a folder named "$CWD/build/lib.linux-x86_64-3.4/pysam exists, it
could potentially allow for code injection, which is troubling.
I opened an issue on their tracker, but don't think it's worth holding
the patch: https://github.com/pysam-developers/pysam/issues/347
Thanks, I agree.
Also, I notice that pysam bundles htslib, bcftools and samtools C code.
Hopefully it should be straightforward enough to remove htslib as there
are install instructions, I'm not sure about the other two. This
shouldn't block the patch here, but would you mind taking a look?
http://pysam.readthedocs.io/en/latest/installation.html#installation
I had a go at this, and also enabled tests since I was reading the build
system anyway. Samtools and bcftools does not seem possible to un-bundle
at this time, but htslib was straightforward.
OK. I don't think it needs to be propagated though, right? Also, would
you mind separating the change to modify-phases syntax and unbundling of
htslib into two patches please? Other than that this whole series LGTM.
Sorry, I keep asking one more thing..
ben
Re: [PATCH 2/3] gnu: Add python-pyxb.
Ben Woodcroft writes:
>> Subject: [PATCH 1/3] gnu: python-pysam: Update to 0.9.1.4.
> I'm not sure whether this is a product of the upgrade or not, but I
> notice this in the build log. I think it is harmless though, WDYT?
>
> starting phase `validate-runpath'
> validating RUNPATH of 10 binaries in
> "/gnu/store/bpiq3lm6b1kpf54i1vj2dl09ff293wic-python-pysam-0.9.1.4/lib"...
> /gnu/store/bpiq3lm6b1kpf54i1vj2dl09ff293wic-python-pysam-0.9.1.4/lib/python3.4/site-packages/pysam-0.9.1.4-py3.4-linux-x86_64.egg/pysam/libchtslib.cpython-34m.so:
>
> warning: RUNPATH contains bogus entries: ("pysam" "."
> "build/lib.linux-x86_64-3.4/pysam")
I don't see this in the previous version, so it is a regression.
However, it should be mostly harmless. Readelf reports (when compiled
with external htslib, see below):
0x001d (RUNPATH)Library runpath:
[/gnu/store/m4gc2wx4q9if1vrhgclpspdil7rqsn21-python-3.4.3/lib:/gnu/store/ba22myqvxccwmmjwwq665rc43hanycxy-htslib-1.3.1/lib:build/lib.linux-x86_64-3.4/pysam:$ORIGIN:/gnu/store/m9vxvhdj691bq1f85lpflvnhcvrdilih-glibc-2.23/lib:/gnu/store/9nifwk709wajpyfwa0jzaa3p6mf10vxs-gcc-4.9.3-lib/lib:/gnu/store/xl19qrfzga52vrvp4ncccwjlnrjqwj95-ncurses-6.0/lib:/gnu/store/5992iq1v7arqa14ym3di58n4la0893nv-zlib-1.2.8/lib:/gnu/store/9nifwk709wajpyfwa0jzaa3p6mf10vxs-gcc-4.9.3-lib/lib/gcc/x86_64-unknown-linux-gnu/4.9.3/../../..]
Compared to the runpath of the same file currently in Guix:
0x001d (RUNPATH)Library runpath:
[/gnu/store/m4gc2wx4q9if1vrhgclpspdil7rqsn21-python-3.4.3/lib:/gnu/store/m9vxvhdj691bq1f85lpflvnhcvrdilih-glibc-2.23/lib:/gnu/store/9nifwk709wajpyfwa0jzaa3p6mf10vxs-gcc-4.9.3-lib/lib:/gnu/store/xl19qrfzga52vrvp4ncccwjlnrjqwj95-ncurses-6.0/lib:/gnu/store/5992iq1v7arqa14ym3di58n4la0893nv-zlib-1.2.8/lib:/gnu/store/9nifwk709wajpyfwa0jzaa3p6mf10vxs-gcc-4.9.3-lib/lib/gcc/x86_64-unknown-linux-gnu/4.9.3/../../..]
If a folder named "$CWD/build/lib.linux-x86_64-3.4/pysam exists, it
could potentially allow for code injection, which is troubling.
I opened an issue on their tracker, but don't think it's worth holding
the patch: https://github.com/pysam-developers/pysam/issues/347
>
> Also, I notice that pysam bundles htslib, bcftools and samtools C code.
> Hopefully it should be straightforward enough to remove htslib as there
> are install instructions, I'm not sure about the other two. This
> shouldn't block the patch here, but would you mind taking a look?
> http://pysam.readthedocs.io/en/latest/installation.html#installation
I had a go at this, and also enabled tests since I was reading the build
system anyway. Samtools and bcftools does not seem possible to un-bundle
at this time, but htslib was straightforward.
Enabling tests required packaging bcftools as well. I've attached the
full patch series below. The first three commits are unchanged.
Thanks!
>From 0a6220a664ecf3aa967211db5bf9e9b9e539220d Mon Sep 17 00:00:00 2001
From: Marius Bakke
Date: Fri, 16 Sep 2016 18:05:15 +0100
Subject: [PATCH 1/6] gnu: python-pysam: Update to 0.9.1.4.
* gnu/packages/bioinformatics.scm (python-pysam, python2-pysam): Update
to 0.9.1.4.
---
gnu/packages/bioinformatics.scm | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/gnu/packages/bioinformatics.scm b/gnu/packages/bioinformatics.scm
index 7a27e0b..47b884e 100644
--- a/gnu/packages/bioinformatics.scm
+++ b/gnu/packages/bioinformatics.scm
@@ -1290,13 +1290,13 @@ multiple sequence alignments.")
(define-public python-pysam
(package
(name "python-pysam")
-(version "0.8.4")
+(version "0.9.1.4")
(source (origin
(method url-fetch)
(uri (pypi-uri "pysam" version))
(sha256
(base32
-"1slx5mb94mzm5qzk52q270sab0sar95j67w1g1k452nz3s9j7krh"
+"1i1djacqbr88y7w18b4aa78zxnsyr4sz7yqdq2spi7gs0y6pzvjn"
(build-system python-build-system)
(arguments
`(#:tests? #f ; tests are excluded in the manifest
--
2.10.0
>From 6a8ddd17d7f11e0687217c8649a452a1a56c596c Mon Sep 17 00:00:00 2001
From: Marius Bakke
Date: Sat, 17 Sep 2016 10:33:28 +0100
Subject: [PATCH 2/6] gnu: Add python-pyxb.
* gnu/packages/xml.scm (python-pyxb, python2-pyxb): New variables.
---
gnu/packages/xml.scm | 25 +
1 file changed, 25 insertions(+)
diff --git a/gnu/packages/xml.scm b/gnu/packages/xml.scm
index 7befad5..cd3426b 100644
--- a/gnu/packages/xml.scm
+++ b/gnu/packages/xml.scm
@@ -13,6 +13,7 @@
;;; Copyright © 2016 Jan Nieuwenhuizen
;;; Copyright © 2016 ng0
;;; Copyright © 2016 Tobias Geerinckx-Rice
+;;; Copyright © 2016 Marius Bakke
;;;
;;; This file is part of GNU Guix.
;;;
@@ -729,6 +730,30 @@ different Unicode encodings which happen automatically during
parsing/saving.")
(license license:expat)))
+(define-public python-pyxb
+ (package
+(name "python-pyxb")
+(version "1.2.5")
+(source (origin
+ (method url-fetc
Re: [PATCH 2/3] gnu: Add python-pyxb.
On 22/09/16 07:14, Marius Bakke wrote:
[..]
Hi Ben,
Thanks for the review. Updated patches attached.
Hi,
Subject: [PATCH 1/3] gnu: python-pysam: Update to 0.9.1.4.
I'm not sure whether this is a product of the upgrade or not, but I
notice this in the build log. I think it is harmless though, WDYT?
starting phase `validate-runpath'
validating RUNPATH of 10 binaries in
"/gnu/store/bpiq3lm6b1kpf54i1vj2dl09ff293wic-python-pysam-0.9.1.4/lib"...
/gnu/store/bpiq3lm6b1kpf54i1vj2dl09ff293wic-python-pysam-0.9.1.4/lib/python3.4/site-packages/pysam-0.9.1.4-py3.4-linux-x86_64.egg/pysam/libchtslib.cpython-34m.so:
warning: RUNPATH contains bogus entries: ("pysam" "."
"build/lib.linux-x86_64-3.4/pysam")
Also, I notice that pysam bundles htslib, bcftools and samtools C code.
Hopefully it should be straightforward enough to remove htslib as there
are install instructions, I'm not sure about the other two. This
shouldn't block the patch here, but would you mind taking a look?
http://pysam.readthedocs.io/en/latest/installation.html#installation
The other two patches LGTM.
ben
Re: [PATCH 2/3] gnu: Add python-pyxb.
Ben Woodcroft writes: > Hi again, > > > On 20/09/16 16:34, Ben Woodcroft wrote: >> Oops, forgot to cc the list. >> >> >> On 20/09/16 16:02, Ben Woodcroft wrote: >>> Hi Marius, >>> >>> Thanks for the patches. >>> >>> On 17/09/16 20:10, Marius Bakke wrote: [..] + (add-after 'unpack 'fix-tests + (lambda _ + ;; See https://github.com/pabigot/pyxb/issues/26 ... + (delete-file "tests/trac/test-trac-0091.py") >>> >>> The fix for that test failing seems straightforward enough, might be >>> better to include as a patch so those tests are run. >>> https://github.com/pabigot/pyxb/commit/d4bdd5a1c712cd70f96264ae13b55d015cbf3335 >>> >>> > Actually, 1.2.5 just got released which has both of these fixes. Can you > send an updated patch please? Hi Ben, Thanks for the review. Updated patches attached. >From 0a6220a664ecf3aa967211db5bf9e9b9e539220d Mon Sep 17 00:00:00 2001 From: Marius Bakke Date: Fri, 16 Sep 2016 18:05:15 +0100 Subject: [PATCH 1/3] gnu: python-pysam: Update to 0.9.1.4. * gnu/packages/bioinformatics.scm (python-pysam, python2-pysam): Update to 0.9.1.4. --- gnu/packages/bioinformatics.scm | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/gnu/packages/bioinformatics.scm b/gnu/packages/bioinformatics.scm index 7a27e0b..47b884e 100644 --- a/gnu/packages/bioinformatics.scm +++ b/gnu/packages/bioinformatics.scm @@ -1290,13 +1290,13 @@ multiple sequence alignments.") (define-public python-pysam (package (name "python-pysam") -(version "0.8.4") +(version "0.9.1.4") (source (origin (method url-fetch) (uri (pypi-uri "pysam" version)) (sha256 (base32 -"1slx5mb94mzm5qzk52q270sab0sar95j67w1g1k452nz3s9j7krh" +"1i1djacqbr88y7w18b4aa78zxnsyr4sz7yqdq2spi7gs0y6pzvjn" (build-system python-build-system) (arguments `(#:tests? #f ; tests are excluded in the manifest -- 2.10.0 >From 6a8ddd17d7f11e0687217c8649a452a1a56c596c Mon Sep 17 00:00:00 2001 From: Marius Bakke Date: Sat, 17 Sep 2016 10:33:28 +0100 Subject: [PATCH 2/3] gnu: Add python-pyxb. * gnu/packages/xml.scm (python-pyxb, python2-pyxb): New variables. --- gnu/packages/xml.scm | 25 + 1 file changed, 25 insertions(+) diff --git a/gnu/packages/xml.scm b/gnu/packages/xml.scm index 7befad5..cd3426b 100644 --- a/gnu/packages/xml.scm +++ b/gnu/packages/xml.scm @@ -13,6 +13,7 @@ ;;; Copyright © 2016 Jan Nieuwenhuizen ;;; Copyright © 2016 ng0 ;;; Copyright © 2016 Tobias Geerinckx-Rice +;;; Copyright © 2016 Marius Bakke ;;; ;;; This file is part of GNU Guix. ;;; @@ -729,6 +730,30 @@ different Unicode encodings which happen automatically during parsing/saving.") (license license:expat))) +(define-public python-pyxb + (package +(name "python-pyxb") +(version "1.2.5") +(source (origin + (method url-fetch) + (uri (pypi-uri "PyXB" version)) + (sha256 + (base32 +"0rzzwibfqa28gxgcxx4cybx1qcg0g6fand06ykj3gz7z5kp653sf" +(build-system python-build-system) +(home-page "http://pyxb.sourceforge.net/";) +(synopsis "Python XML Schema Bindings") +(description + "PyXB (\"pixbee\") is a pure Python package that generates Python source +code for classes that correspond to data structures defined by XMLSchema.") +(license (list license:asl2.0; Most files. + license:expat ; pyxb/utils/six.py + license:gpl2 ; bundled jquery in doc is dual MIT/GPL2 + license:psfl ; pyxb/utils/activestate.py + +(define-public python2-pyxb + (package-with-python2 python-pyxb)) + (define-public xmlto (package (name "xmlto") -- 2.10.0 >From 396cbcbfda24ce5cd1caecedf01407bf6a1f70a1 Mon Sep 17 00:00:00 2001 From: Marius Bakke Date: Sat, 17 Sep 2016 10:42:56 +0100 Subject: [PATCH 3/3] gnu: python2-pbcore: Update to 1.2.10. * gnu/packages/bioinformatics.scm (python2-pbcore): Update to 1.2.10. [propagated-inputs]: New field. Add python2-pyxb. --- gnu/packages/bioinformatics.scm | 6 -- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/gnu/packages/bioinformatics.scm b/gnu/packages/bioinformatics.scm index 47b884e..a598351 100644 --- a/gnu/packages/bioinformatics.scm +++ b/gnu/packages/bioinformatics.scm @@ -3376,13 +3376,13 @@ interrupted by stop codons. OrfM finds and prints these ORFs.") (define-public python2-pbcore (package (name "python2-pbcore") -(version "1.2.8") +(version "1.2.10") (source (origin (method url-fetch) (uri (pypi-uri "pbcore" version)) (sha256 (base32 -"02pfn5raa3zf739672bg0dkx7z3j2c4nx7vmpfjqy5b12jrqpymk" +"1kjmv891d6qbpp4shhhvkl02ff4q5xlpnls2513sm2cjcrs52f1i" (build-system python-build
Re: [PATCH 2/3] gnu: Add python-pyxb.
Hi again, On 20/09/16 16:34, Ben Woodcroft wrote: Oops, forgot to cc the list. On 20/09/16 16:02, Ben Woodcroft wrote: Hi Marius, Thanks for the patches. On 17/09/16 20:10, Marius Bakke wrote: [..] + (add-after 'unpack 'fix-tests + (lambda _ + ;; See https://github.com/pabigot/pyxb/issues/26 ... + (delete-file "tests/trac/test-trac-0091.py") The fix for that test failing seems straightforward enough, might be better to include as a patch so those tests are run. https://github.com/pabigot/pyxb/commit/d4bdd5a1c712cd70f96264ae13b55d015cbf3335 Actually, 1.2.5 just got released which has both of these fixes. Can you send an updated patch please? Thanks.
Re: [PATCH 2/3] gnu: Add python-pyxb.
Oops, forgot to cc the list.
On 20/09/16 16:02, Ben Woodcroft wrote:
Hi Marius,
Thanks for the patches.
On 17/09/16 20:10, Marius Bakke wrote:
[..]
+ (add-after 'unpack 'fix-tests
+ (lambda _
+ ;; See https://github.com/pabigot/pyxb/issues/26 ...
+ (delete-file "tests/trac/test-trac-0091.py")
The fix for that test failing seems straightforward enough, might be
better to include as a patch so those tests are run.
https://github.com/pabigot/pyxb/commit/d4bdd5a1c712cd70f96264ae13b55d015cbf3335
+ ;; ...and https://github.com/pabigot/pyxb/pull/58 .
+ (substitute* "tests/utils/test-utility.py"
+ (("__NoExt_re = re.compile.*$")
+"__NoExt_re = re.compile('(^|\\%s)[^\\.]+$' %
os.sep)\n"))
Good catch there. It would be preferable to add this change as a patch
as well, because it is more likely to apply correctly (not that it
won't here), and also because when it is no longer needed after this
package gets updated the patch will fail to apply alerting the updater
to the fact it should be removed. As a bonus, my experience is that
working with patches easier than using substitute* so require less
builds before getting to a happy package.
I have not actually built these packages, but they look fine
otherwise. Would you mind sending an updated patch for pyxb, and then
I'll probably apply all three (assuming I find nothing else, which I
don't anticipate).
Thanks,
ben
