Bug#806675: hmmer: nhmmer fails with fatal exception on armhf

[Edmund Grimley Evans]

> Presumably you can manually configure it to use the "dummy"
> implementation on an Intel processor. Does that work?

It appears not to. The 3 failures that occurred on armhf can be
obtained on amd64 thus:

./configure --enable-dummy
make
make check

Bug#806675: hmmer: nhmmer fails with fatal exception on armhf

[Edmund Grimley Evans]

ftp://selab.janelia.org/pub/software/hmmer3/3.1b2/Userguide.pdf says:

"""
Processor: HMMER depends on vector parallelization methods that are
supported on most modern processors. H3 requires either an
x86-compatible (IA32, IA64, or Intel64) processor that supports the
SSE2 vector instruction set, or a PowerPC processor that supports the
Altivec/VMX instruction set. SSE2 is supported on Intel processors
from Pentium 4 on, and AMD processors from K8 (Athlon 64) on; we
believe this includes almost all Intel processors since 2000 and AMD
processors since 2003. Altivec/VMX is supported on Motorola G4, IBM
G5, and IBM PowerPC processors starting with the Power6, which we
believe includes almost all PowerPC-based desktop systems since 1999
and servers since 2007.

If your platform does not support one of these vector instruction
sets, the configure script will revert to an unoptimized
implementation called the “dummy” implementation. The dummy
implementation is two orders of magnitude slower. It will enable you
to see H3’s scientific features on a much wider range of processors,
but is not suited for real production work.

We do aim to be portable to all modern processors. The acceleration
algorithms are designed to be portable despite their use of
specialized SIMD vector instructions. We hope to add support for the
Sun SPARC VIS instruction set, for example. We believe that the code
will be able to take advantage of GPGPUs and FPGAs in the future.
"""

So it seems that the failures on non-Intel architectures are not, in
fact, expected. The "dummy" implementation may be buggy.

Presumably you can manually configure it to use the "dummy"
implementation on an Intel processor. Does that work?

Bug#806675: hmmer: nhmmer fails with fatal exception on armhf

[Andreas Tille]

On Wed, Dec 02, 2015 at 06:17:52PM +, Edmund Grimley Evans wrote:
> > Presumably you can manually configure it to use the "dummy"
> > implementation on an Intel processor. Does that work?
> 
> It appears not to. The 3 failures that occurred on armhf can be
> obtained on amd64 thus:
> 
> ./configure --enable-dummy
> make
> make check

I also fail to see any sense in this approach:  Emulating some thing on
a even way less performant architecture to at least get a program that
is intended to do *performant* operations on *heavy* data is just hiding
your eyes from reality.  So I think restricting the package to
architectures it is intended to work on is the proper way to go.

Kind regards

   Andreas.

-- 
http://fam-tille.de

Bug#806675: hmmer: nhmmer fails with fatal exception on armhf

[Andreas Tille]

Hi Sean,

I'd like to forward this issue to you as the hmmer author.  Since in
Debian packages are usually build on several different architectures
sometimes hidden bugs are detected.  Since arm64 and ppc64el might
become relevant in bioinformatics in the not to distant future I'd
consider this issue important.

Any hint would be welcome.

Kind regards

   Andreas.

On Tue, Dec 01, 2015 at 09:10:30PM +, Edmund Grimley Evans wrote:
> A lot of the test failures seen in
> https://buildd.debian.org/status/fetch.php?pkg=hmmer=arm64=3.1b2-1=1436134634
> are not easily reproducible.
> 
> For example, when I tried
> 
> cd testsuite/
> touch tmp ; rm tmp* ; ./i9-optional-annotation.pl .. .. tmp
> 
> it gave "ok" most of the time, but every now and then it gave
> "FAIL: on line 0 target name, dtbl1".
> 
> One of the few easily reproducible failures I found on arm64 was this:
> 
> touch tmp1 ; rm tmp1* ; ./i8-nonresidues.pl .. .. tmp1
> 
> I didn't see that test ever produce a different output from
> "FAIL: expected one line in domtbl; saw 2". The "nonresidues"
> test also failed on ppc64el. Perhaps you should look at that
> one first. Or perhaps you're more worried about the general
> non-determinism that affects many of the tests. The obvious question
> to ask is: does this non-determinism only exist in the test suite,
> or does it also exist in the packaged programs?
> 
> ___
> Debian-med-packaging mailing list
> debian-med-packag...@lists.alioth.debian.org
> http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/debian-med-packaging
> 

-- 
http://fam-tille.de

Bug#806675: hmmer: nhmmer fails with fatal exception on armhf

[Edmund Grimley Evans]

A lot of the test failures seen in
https://buildd.debian.org/status/fetch.php?pkg=hmmer=arm64=3.1b2-1=1436134634
are not easily reproducible.

For example, when I tried

cd testsuite/
touch tmp ; rm tmp* ; ./i9-optional-annotation.pl .. .. tmp

it gave "ok" most of the time, but every now and then it gave
"FAIL: on line 0 target name, dtbl1".

One of the few easily reproducible failures I found on arm64 was this:

touch tmp1 ; rm tmp1* ; ./i8-nonresidues.pl .. .. tmp1

I didn't see that test ever produce a different output from
"FAIL: expected one line in domtbl; saw 2". The "nonresidues"
test also failed on ppc64el. Perhaps you should look at that
one first. Or perhaps you're more worried about the general
non-determinism that affects many of the tests. The obvious question
to ask is: does this non-determinism only exist in the test suite,
or does it also exist in the packaged programs?

Bug#806675: hmmer: nhmmer fails with fatal exception on armhf

[Andreas Tille]

Hi Sean,

thanks for the fast clarification.  It would be great if you could keep
me informed in case you would enhance the number of architectures to
make sure we will follow this move.

Kind regards

  Andreas.

On Wed, Dec 02, 2015 at 01:21:25AM +, Eddy, Sean wrote:
> HMMER3 is currently only supported on Intel or PPC compatible platforms 
> because it makes use of vector parallelization using SSE or Altivec/VMX. ARM 
> and PPC64le platforms are currently not supported.
> 
> Sean
> 
> 
> > On Dec 1, 2015, at 9:22 PM, Andreas Tille  wrote:
> > 
> > Hi Sean,
> > 
> > I'd like to forward this issue to you as the hmmer author.  Since in
> > Debian packages are usually build on several different architectures
> > sometimes hidden bugs are detected.  Since arm64 and ppc64el might
> > become relevant in bioinformatics in the not to distant future I'd
> > consider this issue important.
> > 
> > Any hint would be welcome.
> > 
> > Kind regards
> > 
> >   Andreas.
> > 
> > On Tue, Dec 01, 2015 at 09:10:30PM +, Edmund Grimley Evans wrote:
> >> A lot of the test failures seen in
> >> https://buildd.debian.org/status/fetch.php?pkg=hmmer=arm64=3.1b2-1=1436134634
> >> are not easily reproducible.
> >> 
> >> For example, when I tried
> >> 
> >>cd testsuite/
> >>touch tmp ; rm tmp* ; ./i9-optional-annotation.pl .. .. tmp
> >> 
> >> it gave "ok" most of the time, but every now and then it gave
> >> "FAIL: on line 0 target name, dtbl1".
> >> 
> >> One of the few easily reproducible failures I found on arm64 was this:
> >> 
> >>touch tmp1 ; rm tmp1* ; ./i8-nonresidues.pl .. .. tmp1
> >> 
> >> I didn't see that test ever produce a different output from
> >> "FAIL: expected one line in domtbl; saw 2". The "nonresidues"
> >> test also failed on ppc64el. Perhaps you should look at that
> >> one first. Or perhaps you're more worried about the general
> >> non-determinism that affects many of the tests. The obvious question
> >> to ask is: does this non-determinism only exist in the test suite,
> >> or does it also exist in the packaged programs?
> >> 
> >> ___
> >> Debian-med-packaging mailing list
> >> debian-med-packag...@lists.alioth.debian.org
> >> http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/debian-med-packaging
> >> 
> > 
> > -- 
> > http://fam-tille.de
> 
> 

-- 
http://fam-tille.de

Bug#806675: hmmer: nhmmer fails with fatal exception on armhf

[Sascha Steinbiss]

Hi Andreas,

> would you mind contacting upstream or arm porters about this since you
> seem to have done a quite detailed research on the issue ?

Sure, I will contact both and see if they know more.

Best,
Sascha

> 
> On Sun, Nov 29, 2015 at 10:20:54PM +, Sascha Steinbiss wrote:
>> Source: hmmer
>> Severity: normal
>> 
>> Dear Maintainer,
>> 
>> while reviewing the reproducible builds output for the barrnap package,
>> which depends on nhmmer, I noticed that a nhmmer problem seems to break
>> barrnap's post-build test run [1]. 
>> I'm suspecting this is platform related as it has never occured before
>> until the reproducibility builds started including ARM as a target platform 
>> [2].
>> 
>> I checked the unit test output for the hmmer armhf build [3] and some of the
>> tests at the end seem to crash or fail as well. This is also the case for the
>> other ARM platforms, powerpc and mips, but not for i386.
>> 
>> Cheers
>> Sascha
>> 
>> [1] https://reproducible.debian.net/rb-pkg/unstable/armhf/barrnap.html
>> [2] https://reproducible.debian.net/history/barrnap.html
>> [3] 
>> https://buildd.debian.org/status/fetch.php?pkg=hmmer=armhf=3.1b2-1=1436134926
>> 
> 
> -- 
> http://fam-tille.de

Bug#806675: hmmer: nhmmer fails with fatal exception on armhf

[Sascha Steinbiss]

Source: hmmer
Severity: normal

Dear Maintainer,

while reviewing the reproducible builds output for the barrnap package,
which depends on nhmmer, I noticed that a nhmmer problem seems to break
barrnap's post-build test run [1]. 
I'm suspecting this is platform related as it has never occured before
until the reproducibility builds started including ARM as a target platform [2].

I checked the unit test output for the hmmer armhf build [3] and some of the
tests at the end seem to crash or fail as well. This is also the case for the
other ARM platforms, powerpc and mips, but not for i386.

Cheers
Sascha

[1] https://reproducible.debian.net/rb-pkg/unstable/armhf/barrnap.html
[2] https://reproducible.debian.net/history/barrnap.html
[3] 
https://buildd.debian.org/status/fetch.php?pkg=hmmer=armhf=3.1b2-1=1436134926

Bug#806675: hmmer: nhmmer fails with fatal exception on armhf

[Andreas Tille]

Hi Sascha,

would you mind contacting upstream or arm porters about this since you
seem to have done a quite detailed research on the issue ?

Kind regards

   Andreas.

On Sun, Nov 29, 2015 at 10:20:54PM +, Sascha Steinbiss wrote:
> Source: hmmer
> Severity: normal
> 
> Dear Maintainer,
> 
> while reviewing the reproducible builds output for the barrnap package,
> which depends on nhmmer, I noticed that a nhmmer problem seems to break
> barrnap's post-build test run [1]. 
> I'm suspecting this is platform related as it has never occured before
> until the reproducibility builds started including ARM as a target platform 
> [2].
> 
> I checked the unit test output for the hmmer armhf build [3] and some of the
> tests at the end seem to crash or fail as well. This is also the case for the
> other ARM platforms, powerpc and mips, but not for i386.
> 
> Cheers
> Sascha
> 
> [1] https://reproducible.debian.net/rb-pkg/unstable/armhf/barrnap.html
> [2] https://reproducible.debian.net/history/barrnap.html
> [3] 
> https://buildd.debian.org/status/fetch.php?pkg=hmmer=armhf=3.1b2-1=1436134926
> 

-- 
http://fam-tille.de