Re: Bug#798066: Multiarch-renamed python extensions not found during autopkgtest testing

2015-12-12 Thread Afif Elghraoui

على الجمعـة 11 كانون الأول 2015 ‫03:09، كتب Antonio Terceiro:

> I also
> don't see other python packages that contain compiled extensions needing
> to do this sort of thing.
>

I have a few other packages that have this problem -- python-cobra is
one. For two other packages (python-pbh5tools and kineticstools), I
disabled tests that require the compiled extensions because they were
failing for this reason.


> I tried python-pysam here, and after some trial and error, I can also 
> reproduce
> the same issue outside of autopkgtest. The issue is that Python load path is
> being confused by the fact that you are on root of the source package:
> 
[...]

Many thanks for that information. I can reproduce that effect as well
and things are making much more sense now. Now, I just need to find a
proper solution to this python module priority problem.

> 
> So your problem has nothing to do autopkgtest, other than the fact that
> autopkgtest always starts the tests from the root of the source package.
> 

Thanks-- I didn't set out to bash autopkgtest with this. I just wanted
to know why the tests weren't running properly and how to resolve that
problem.


Many thanks and regards
Afif

-- 
Afif Elghraoui | عفيف الغراوي
http://afif.ghraoui.name



signature.asc
Description: OpenPGP digital signature


Re: Bug#798066: Multiarch-renamed python extensions not found during autopkgtest testing

2015-12-11 Thread Antonio Terceiro
On Thu, Dec 10, 2015 at 10:52:05PM -0800, Afif Elghraoui wrote:
> Hi,
> 
> على الأربعاء  9 كانون الأول 2015 ‫05:25، كتب Antonio Terceiro:
> > autopkgtest does not do anything special wrt dependencies, it will
> > install exactly what you told it to in debian/tests/control
> [...]
> > 
> > I am therefore closing this bug.
> 
> The problem is not that something is not installed. The problem is that
> the multiarch configuration for python is not right in autopkgtest (or
> schroot). During the package build, dh-python renames the compiled
> extension to contain the multiarch triplet. For some reason, only in
> autopkgtest, python is not properly configured to find the extension
> after it has been renamed, so autopkgtest runs requiring the compiled
> extensions fail.

ci.debian.net does not use schroot anymore, since a few weeks ago. and
autopkgtest does not do anything special to packages, it just installs
them on an otherwise regular testbed, be it a schroot chroot, an lxc
container (what ci.debian.net currently uses), and so on.

> So should we continue to have hacks like this to get autopkgtest to find
> multiarch-renamed extensions?
> 
> cd /usr/lib/python2.7/dist-packages/pysam
> gnutype=`dpkg-architecture -qDEB_TARGET_GNU_TYPE`
> for so in *.${gnutype}.so ; do sudo ln -sf $so `basename $so
> .${gnutype}.so`.so ; done
> 
> http://anonscm.debian.org/cgit/debian-med/python-pysam.git/tree/debian/tests/run-nose-tests

Again, there is nothing special in the autopkgtest test beds. I also
don't see other python packages that contain compiled extensions needing
to do this sort of thing.

I tried python-pysam here, and after some trial and error, I can also reproduce
the same issue outside of autopkgtest. The issue is that Python load path is
being confused by the fact that you are on root of the source package:

# pwd
/tmp/python-pysam-0.8.4+ds
# ls
AUTHORS  INSTALL MANIFEST.in  THANKS debian  install-CGAT-tools.sh  
pysam.py  samtools  setup.cfg  tests
COPYING  KNOWN_BUGS  README.rst   benchmark  doc pysam  
requirements.txt  save  setup.py   win32
# python
Python 2.7.11 (default, Dec  9 2015, 00:29:25) 
[GCC 5.3.1 20151205] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import pysam
Traceback (most recent call last):
  File "", line 1, in 
  File "pysam/__init__.py", line 1, in 
from pysam.libchtslib import *
ImportError: No module named libchtslib
>>> 
# cd /
# python
Python 2.7.11 (default, Dec  9 2015, 00:29:25) 
[GCC 5.3.1 20151205] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import pysam 
>>> 
# 

So your problem has nothing to do autopkgtest, other than the fact that
autopkgtest always starts the tests from the root of the source package.

-- 
Antonio Terceiro 


signature.asc
Description: PGP signature


Re: Bug#798066: Multiarch-renamed python extensions not found during autopkgtest testing

2015-12-10 Thread Afif Elghraoui
Hi,

على الأربعاء  9 كانون الأول 2015 ‫05:25، كتب Antonio Terceiro:
> autopkgtest does not do anything special wrt dependencies, it will
> install exactly what you told it to in debian/tests/control
[...]
> 
> I am therefore closing this bug.

The problem is not that something is not installed. The problem is that
the multiarch configuration for python is not right in autopkgtest (or
schroot). During the package build, dh-python renames the compiled
extension to contain the multiarch triplet. For some reason, only in
autopkgtest, python is not properly configured to find the extension
after it has been renamed, so autopkgtest runs requiring the compiled
extensions fail.

So should we continue to have hacks like this to get autopkgtest to find
multiarch-renamed extensions?

cd /usr/lib/python2.7/dist-packages/pysam
gnutype=`dpkg-architecture -qDEB_TARGET_GNU_TYPE`
for so in *.${gnutype}.so ; do sudo ln -sf $so `basename $so
.${gnutype}.so`.so ; done

http://anonscm.debian.org/cgit/debian-med/python-pysam.git/tree/debian/tests/run-nose-tests

regards
Afif

-- 
Afif Elghraoui | عفيف الغراوي
http://afif.ghraoui.name



Re: Bug#798066: Multiarch-renamed python extensions not found during autopkgtest testing

2015-12-09 Thread Antonio Terceiro
Control:

On Tue, Dec 08, 2015 at 11:25:47PM -0800, Afif Elghraoui wrote:
> Hi, debian-python,
> 
> For Debian Med's python packages that have compiled extensions, I
> noticed that test suites run via autopkgtest fail because the package
> cannot find those extensions, as they've been renamed with the multiarch
> triplet.
> 
> It seems to only be a problem with autopkgtest because I have in one
> case manually installed such a package and successfully ran its test
> suite. I therefore reported #798066 [1] against autopkgtest, but quite
> some time has elapsed since then and I'd like to get this resolved.
> 
> Does anyone know why this might be the case? Autopkgtest for these
> packages is currently not very helpful because there are many spurious
> failures due to this issue. There are more details in my original bug
> report [1].
> 
> Many thanks and regards
> Afif
> 
> 1. http://bugs.debian.org/798066

autopkgtest does not do anything special wrt dependencies, it will
install exactly what you told it to in debian/tests/control (see the
spec¹ for reference), so I doubt this would be a problem with
autopkgtest.

¹ 
http://anonscm.debian.org/cgit/autopkgtest/autopkgtest.git/tree/doc/README.package-tests.rst

I am therefore closing this bug.

-- 
Antonio Terceiro 


signature.asc
Description: PGP signature