[Urgent help needed ] python-pysam broken - may be Python3 3.6.3 to 3.6.4~rc1 transition issue (Was: Uploaded new version 1.6 of htslib, samtools & bcftools ...)

[Andreas Tille]

Hi,

as I reported yesterday, I've tested python-pysam 0.13.0+ds-1 as in Git
yesterday with the new samtools 1.6 package.  (For those who need a
prove here is the fill build[1].)  Unfortunately if I build today the
package does not build any more.  The python2 tests are running fine but
if it comes to python3 I get:


...
 ERRORS 
__ ERROR collecting .pybuild/pythonX.Y_3.6/build/tests/AlignedSegment_test.py __
ImportError while importing test module 
'/build/python-pysam-0.13.0+ds/.pybuild/pythonX.Y_3.6/build/tests/AlignedSegment_test.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
tests/AlignedSegment_test.py:2: in 
import pysam
pysam/__init__.py:28: in 
from pysam.samtools import *
pysam/samtools.py:1: in 
from utils import PysamDispatcher
E   ModuleNotFoundError: No module named 'utils'
__ ERROR collecting .pybuild/pythonX.Y_3.6/build/tests/AlignmentFile_test.py ___
tests/AlignmentFile_test.py:25: in 
import pysam
pysam/__init__.py:9: in 
import pysam.libcutils as libcutils
E   AttributeError: module 'pysam' has no attribute 'libcutils'
 ERROR collecting 
.pybuild/pythonX.Y_3.6/build/tests/StreamFiledescriptors_test.py.
tests/StreamFiledescriptors_test.py:8: in 
from pysam import AlignmentFile
pysam/__init__.py:9: in 
import pysam.libcutils as libcutils
E   AttributeError: module 'pysam' has no attribute 'libcutils'
___ ERROR collecting .pybuild/pythonX.Y_3.6/build/tests/VariantFile_test.py 
tests/VariantFile_test.py:4: in 
import pysam
pysam/__init__.py:9: in 
import pysam.libcutils as libcutils
E   AttributeError: module 'pysam' has no attribute 'libcutils'
_ ERROR collecting .pybuild/pythonX.Y_3.6/build/tests/compile_test.py __
ImportError while importing test module 
'/build/python-pysam-0.13.0+ds/.pybuild/pythonX.Y_3.6/build/tests/compile_test.py'.
Hint: make sure your test modules/packages have valid Python names.
...


When comparing the build log of today with the one from yesterday
I see differences in the Python versions:


-Preparing to unpack .../50-libpython-dev_2.7.14-3_amd64.deb ...
-Unpacking libpython-dev:amd64 (2.7.14-3) ...
+Preparing to unpack .../49-libpython-dev_2.7.14-4_amd64.deb ...
+Unpacking libpython-dev:amd64 (2.7.14-4) ...
...
-Setting up python3 (3.6.3-2) ...
-Setting up python3-dev (3.6.3-2) ...
-Setting up python3-pkg-resources (38.2.4-1) ...
-Setting up python3-all (3.6.3-2) ...
+Setting up python3 (3.6.4~rc1-1) ...
+Setting up python3-pkg-resources (38.2.4-2) ...
+Setting up python3-all (3.6.4~rc1-1) ...


So may be we are hit by a Python3 3.6.3 to 3.6.4~rc1 issue?

Does anybody have a clue?

Kind regards

Andreas.


[1] https://people.debian.org/~tille/packages/python-pysam/

-- 
http://fam-tille.de

Re: figtree autopkgtest

[Andreas Tille]

Dear Katerina,

[Somehow your text is really unreadable.  May be you try forcing
 GMail to use plain text.  You can possibly do so by marking your
 whole text using -A and than use the T_x icon on the bottom.]

On Thu, Dec 14, 2017 at 05:42:07PM +0200, Kalou.Katerina wrote:
> Dear Andreas,
> 
> I made the following changes in the figtree package : deleted (commented)
> the first suggested test which
> needs graphical output
> ​ and changed the last suggested test from 'gif' to 'png' output.  Finally
> , i manually exported
> the proper 'tree' files and added them to the 'experiments' folder, so that
> the 'autopkgtest -- null' is running succesfully
> on my machine.

Hmmm, despite the typo I fixed in cf1d7328b923a0485527a7b78a31cff51e8fe3e2 ?
;-)

I also changed the location of the examples.  For the packaging you
should *only* deal with the debian/ dir.  If you need to change / create
files in the upstream source tree you need to provide a quilt patch.
However, in this specific case I just moved the files to debian/examples
and added this to the debhelper install file.
 
> I tested around with the idea of starting
> xvfb-run
> ​and finding a way to export the proper '.tree' files
> without user interaction. I found the relative '
> ​​

I think that's really tricky if the application does not support this.
Just for the sake of interest I really managed this kind of test with
gdpc - but I think we are doing fine with the graphics export in the
figtree case.

> FigTreeNexusImporter​
> ​.java' function in the 'src' folder
> and tried to incorporate it in the test file ( as ' javac
> ​
> FigTreeNexusImporter​
> ​.java'
> but obviously i failed. I​
> ​f
> anyone who knows java wants to offer any help it would be a big help :)​

Hmmm, I can not help with Java and I think there is no need to.
 
> I think i messed up with the commit to the master branch of debian med in
> git.

What do you mean?  The commits seem to be OK.

> I should create
> a patch first and do 'git am' instead of that i did 'git push' , i was
> following a git tutorial and it was
> an accident. I am sorry about that. I will be more careful (and better
> prepared) next time.

Actually 'git push' is the way to go.
 
> I am looking for the next bug to close ( probably
> #879616
> ​ ?)​
> <879...@bugs.debian.org>

Please write a note to this bug report to inform others that you are
working on it.

> ​Thank you for all the help!​

You are welcome - thanks to you for the autopkgtest

   Andreas.

-- 
http://fam-tille.de

Re: figtree autopkgtest

[Kalou.Katerina]

Dear Andreas,

I made the following changes in the figtree package : deleted (commented)
the first suggested test which
needs graphical output
​ and changed the last suggested test from 'gif' to 'png' output.  Finally
, i manually exported
the proper 'tree' files and added them to the 'experiments' folder, so that
the 'autopkgtest -- null' is running succesfully
on my machine.

I tested around with the idea of starting
xvfb-run
​and finding a way to export the proper '.tree' files
without user interaction. I found the relative '
​​
FigTreeNexusImporter​
​.java' function in the 'src' folder
and tried to incorporate it in the test file ( as ' javac
​
FigTreeNexusImporter​
​.java'
but obviously i failed. I​
​f
anyone who knows java wants to offer any help it would be a big help :)​

I think i messed up with the commit to the master branch of debian med in
git. I should create
a patch first and do 'git am' instead of that i did 'git push' , i was
following a git tutorial and it was
an accident. I am sorry about that. I will be more careful (and better
prepared) next time.

I am looking for the next bug to close ( probably
#879616
​ ?)​
<879...@bugs.debian.org>

​Thank you for all the help!​


On Wed, Dec 13, 2017 at 5:30 PM, Andreas Tille  wrote:

> Hi Kate,
>
> cool, just take your time and let us know about problems.
>
> Thanks for your effort
>
>   Andreas.
>
> On Wed, Dec 13, 2017 at 09:14:06AM +0200, Kalou.Katerina wrote:
> > Hi Andreas,
> >
> > i'm really sorry i'm late-some real life problems kept me back.
> > I have already start working on it, i will have more news from me later
> > today!
> >
> > Hopefully this bug will be closed by tonight and i can start working on
> > another one :)
> >
> > kind regards,
> > kate
> >
> >
> > On Wed, Dec 13, 2017 at 9:02 AM, Andreas Tille  wrote:
> >
> > > Hi Kate,
> > >
> > > On Thu, Nov 30, 2017 at 03:23:18PM +0200, ka lou wrote:
> > > >
> > > > I am definitely going to keep on working on this, i 'll hit you up
> with
> > > > more comments and questions soon.
> > >
> > > Any news from the figtree autopkgtests?  I think the hints should have
> > > lead you to some progress and it would be great if we could close bug
> > > #879621 soon.
> > >
> > > Kind regards
> > >
> > >Andreas.
> > >
> > >
> > > --
> > > http://fam-tille.de
> > >
> > >
>
> --
> http://fam-tille.de
>

Re: Uploaded new version 1.6 of htslib, samtools & bcftools to experimental - any known issues with reverse dependencies

[Andreas Tille]

On Thu, Dec 14, 2017 at 02:30:53PM +0100, Mattia Rizzolo wrote:
> 
> ACK, should be all good to go now.

Thanks for the careful review.  That's really appreciated

   Andreas.

-- 
http://fam-tille.de

Re: Uploaded new version 1.6 of htslib, samtools & bcftools to experimental - any known issues with reverse dependencies

[Mattia Rizzolo]

On Thu, Dec 14, 2017 at 02:26:46PM +0100, Andreas Tille wrote:
> I've tested a local build and the means of
> 765419270800864fd29cea90db233a60cd46aea8 can all be droped.  I pushed
> two commits - hopefully better documented than before.

ACK, should be all good to go now.

-- 
regards,
Mattia Rizzolo

GPG Key: 66AE 2B4A FCCF 3F52 DA18  4D18 4B04 3FCD B944 4540  .''`.
more about me:  https://mapreri.org : :'  :
Launchpad user: https://launchpad.net/~mapreri  `. `'`
Debian QA page: https://qa.debian.org/developer.php?login=mattia  `-


signature.asc
Description: PGP signature

Re: Uploaded new version 1.6 of htslib, samtools & bcftools to experimental - any known issues with reverse dependencies

[Andreas Tille]

Hi Mattia,

On Thu, Dec 14, 2017 at 01:31:35PM +0100, Mattia Rizzolo wrote:
> On Wed, Dec 13, 2017 at 04:49:36PM +0100, Andreas Tille wrote:
> > Thanks for your careful checking.  Would you mind just commiting your
> > prefered solution to make sure I've correctly understood what you mean?
> > (There is no real policy in the med team here.)
> 
> I could do that, but I'm unsure as to why you did a thing.
> I'm looking at commit 765419270800864fd29cea90db233a60cd46aea8, where
> with a simple message "install htslib*.mk files that are used in
> bcftools" you did so many changes, including adding a dependency and
> installing a bunch of links.  The problem is the first link of that
> list.  The commit nor the changelog explains the reason of those
> changes, so I'd rather not remove something that is there for a
> reason...

Sorry for the sloppy commit messages.
 
> Then, now that the htslib*.mk are not installed anymore
> (commit 5fedaeab7241efc00f5d814ebb67f884d1c62e38) so *maybe* all the
> changes that were done in the same commit could be reverted as well.

I've tested a local build and the means of
765419270800864fd29cea90db233a60cd46aea8 can all be droped.  I pushed
two commits - hopefully better documented than before.

Sorry for the confusion

  Andreas.

-- 
http://fam-tille.de

Re: Uploaded new version 1.6 of htslib, samtools & bcftools to experimental - any known issues with reverse dependencies

[Mattia Rizzolo]

On Wed, Dec 13, 2017 at 04:49:36PM +0100, Andreas Tille wrote:
> Thanks for your careful checking.  Would you mind just commiting your
> prefered solution to make sure I've correctly understood what you mean?
> (There is no real policy in the med team here.)

I could do that, but I'm unsure as to why you did a thing.
I'm looking at commit 765419270800864fd29cea90db233a60cd46aea8, where
with a simple message "install htslib*.mk files that are used in
bcftools" you did so many changes, including adding a dependency and
installing a bunch of links.  The problem is the first link of that
list.  The commit nor the changelog explains the reason of those
changes, so I'd rather not remove something that is there for a
reason...

Then, now that the htslib*.mk are not installed anymore
(commit 5fedaeab7241efc00f5d814ebb67f884d1c62e38) so *maybe* all the
changes that were done in the same commit could be reverted as well.
But who knows…

-- 
regards,
Mattia Rizzolo

GPG Key: 66AE 2B4A FCCF 3F52 DA18  4D18 4B04 3FCD B944 4540  .''`.
more about me:  https://mapreri.org : :'  :
Launchpad user: https://launchpad.net/~mapreri  `. `'`
Debian QA page: https://qa.debian.org/developer.php?login=mattia  `-


signature.asc
Description: PGP signature

gatb-core: How to link binaries dynamically using CMake (Was: [MoM] Re: gatb-core packaging)

[Andreas Tille]

Hi,

I think all basic issues are now dealt with in gatb-core Git[1] (many
thanks specifically to Gert Wollny for sorting out some cmake issues I
had no idea how to deal with).  I've switched the packaging to d-shlibs
creating a the usual lib and lib-dev package with multiarch enabled.

Olivier, please point upstream to the set of patches - several of them
(for instance even spelling issues) could be applied upstream.

The last nuisance is that the binaries in /usr/bin and
/usr/lib/gatb-core are linked statically instead of using the dynamic
library.  Any hint how to fix this?

Kind regards

   Andreas.

[1] https://anonscm.debian.org/git/debian-med/gatb-core.git

-- 
http://fam-tille.de