Bug#916262: staden-io-lib: data/xx#unsorted.sam fails randomly

2018-12-12 Thread James Bonfield
On Wed, Dec 12, 2018 at 11:56:28AM +0100, Andreas Tille wrote:
> besides the remaining build issues on some architectures there
> is a new bug report about random failures.
> 
> I hope you will have a clue about this.

They all seem to be the same cause, which I haven't investigated yet.
One solution is just to give up with auto-generating data and bundle a
larger corpus of test data.

The problem comes from the auto-generation step producing files of
differing length.  I hard coded the random number generator and also
put the test system into serial mode so it cannot attempt to run
multiple data generation steps in parallel, but it hasn't cured it.

A simple method is probably the best way to go.

James
-- 
James Bonfield (j...@sanger.ac.uk)
The Sanger Institute, Hinxton, Cambs, CB10 1SA


-- 
 The Wellcome Sanger Institute is operated by Genome Research 
 Limited, a charity registered in England with number 1021457 and a 
 company registered in England with number 2742969, whose registered 
 office is 215 Euston Road, London, NW1 2BE. 



Bug#916262: staden-io-lib: data/xx#unsorted.sam fails randomly

2018-12-12 Thread Andreas Tille
On Wed, Dec 12, 2018 at 11:48:30AM +, James Bonfield wrote:
> A simple method is probably the best way to go.

+1

   Andreas.
 

-- 
http://fam-tille.de



Processed: Re: Bug#916262: staden-io-lib: data/xx#unsorted.sam fails randomly

2018-12-12 Thread Debian Bug Tracking System
Processing control commands:

> tags -1 upstream
Bug #916262 [src:staden-io-lib] staden-io-lib: data/xx#unsorted.sam fails 
randomly
Added tag(s) upstream.
> forwarded -1 James Bonfield 
Bug #916262 [src:staden-io-lib] staden-io-lib: data/xx#unsorted.sam fails 
randomly
Set Bug forwarded-to-address to 'James Bonfield '.

-- 
916262: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=916262
Debian Bug Tracking System
Contact ow...@bugs.debian.org with problems



Bug#916262: staden-io-lib: data/xx#unsorted.sam fails randomly

2018-12-12 Thread Andreas Tille
Control: tags -1 upstream
Control: forwarded -1 James Bonfield 

Hi James,

besides the remaining build issues on some architectures there
is a new bug report about random failures.

I hope you will have a clue about this.

Kind regards

   Andreas.

On Wed, Dec 12, 2018 at 12:36:43PM +0200, Adrian Bunk wrote:
> Source: staden-io-lib
> Version: 1.14.11-5
> Severity: serious
> Tags: ftbfs
> Control: block 912272 by -1
> 
> https://buildd.debian.org/status/package.php?p=staden-io-lib
> https://tests.reproducible-builds.org/debian/rb-pkg/unstable/amd64/staden-io-lib.html
> 
> ...
> === testing ./data/xx#unsorted.sam ===
>  ../progs/scramble  ./data/xx#unsorted.sam test.out/xx#unsorted.bam
>  ../progs/scramble  -r ./data/xx.fa test.out/xx#unsorted.bam 
> test.out/xx#unsorted.full.cram
>  ../progs/scramble  test.out/xx#unsorted.bam > test.out/tmp.sam
>  ../progs/scramble  test.out/xx#unsorted.full.cram > 
> test.out/xx#unsorted.full.sam
>  ../progs/scramble  -O bam test.out/xx#unsorted.full.cram > 
> test.out/xx#unsorted.full.bam
>  ../progs/scramble  test.out/xx#unsorted.full.bam test.out/tmp.sam
> ./compare_sam.pl --nopg test.out/xx#unsorted.full.sam 
> test.out/xx#unsorted.full.bam.sam
>  ../progs/scramble  -x -r ./data/xx.fa test.out/xx#unsorted.bam 
> test.out/xx#unsorted.full.cram
>  ../progs/scramble  test.out/xx#unsorted.full.cram > 
> test.out/xx#unsorted.full.sam
> ./compare_sam.pl --unknownrg ./data/xx#unsorted.sam 
> test.out/xx#unsorted.full.bam.sam
> 
> ../progs/scramble -H -r ./data/ce.fa test.out/ce#sorted.full.cram
> No region:   1024176
> FAIL: scram.test
> ...
> === testing ./data/xx#unsorted.sam ===
>  ../progs/scramble -t4 ./data/xx#unsorted.sam test.out/xx#unsorted.bam
>  ../progs/scramble -t4 -r ./data/xx.fa test.out/xx#unsorted.bam 
> test.out/xx#unsorted.full.cram
>  ../progs/scramble -t4 test.out/xx#unsorted.bam > test.out/tmp.sam
>  ../progs/scramble -t4 test.out/xx#unsorted.full.cram > 
> test.out/xx#unsorted.full.sam
>  ../progs/scramble -t4 -O bam test.out/xx#unsorted.full.cram > 
> test.out/xx#unsorted.full.bam
>  ../progs/scramble -t4 test.out/xx#unsorted.full.bam test.out/tmp.sam
> ./compare_sam.pl --nopg test.out/xx#unsorted.full.sam 
> test.out/xx#unsorted.full.bam.sam
>  ../progs/scramble -t4 -x -r ./data/xx.fa test.out/xx#unsorted.bam 
> test.out/xx#unsorted.full.cram
>  ../progs/scramble -t4 test.out/xx#unsorted.full.cram > 
> test.out/xx#unsorted.full.sam
> ./compare_sam.pl --unknownrg ./data/xx#unsorted.sam 
> test.out/xx#unsorted.full.bam.sam
> 
> ../progs/scramble -t4 -H -r ./data/ce.fa test.out/ce#sorted.full.cram
> No region:   1024176
> FAIL: scram_mt.test
> ...
> 
> 
> data/xx#unsorted.sam seems to fail randomly, with failures and passes
> seen on the same architecture (including amd64).
> 
> ___
> Debian-med-packaging mailing list
> debian-med-packag...@alioth-lists.debian.net
> https://alioth-lists.debian.net/cgi-bin/mailman/listinfo/debian-med-packaging

-- 
http://fam-tille.de



Bug#916262: staden-io-lib: data/xx#unsorted.sam fails randomly

2018-12-12 Thread Adrian Bunk
Source: staden-io-lib
Version: 1.14.11-5
Severity: serious
Tags: ftbfs
Control: block 912272 by -1

https://buildd.debian.org/status/package.php?p=staden-io-lib
https://tests.reproducible-builds.org/debian/rb-pkg/unstable/amd64/staden-io-lib.html

...
=== testing ./data/xx#unsorted.sam ===
 ../progs/scramble  ./data/xx#unsorted.sam test.out/xx#unsorted.bam
 ../progs/scramble  -r ./data/xx.fa test.out/xx#unsorted.bam 
test.out/xx#unsorted.full.cram
 ../progs/scramble  test.out/xx#unsorted.bam > test.out/tmp.sam
 ../progs/scramble  test.out/xx#unsorted.full.cram > 
test.out/xx#unsorted.full.sam
 ../progs/scramble  -O bam test.out/xx#unsorted.full.cram > 
test.out/xx#unsorted.full.bam
 ../progs/scramble  test.out/xx#unsorted.full.bam test.out/tmp.sam
./compare_sam.pl --nopg test.out/xx#unsorted.full.sam 
test.out/xx#unsorted.full.bam.sam
 ../progs/scramble  -x -r ./data/xx.fa test.out/xx#unsorted.bam 
test.out/xx#unsorted.full.cram
 ../progs/scramble  test.out/xx#unsorted.full.cram > 
test.out/xx#unsorted.full.sam
./compare_sam.pl --unknownrg ./data/xx#unsorted.sam 
test.out/xx#unsorted.full.bam.sam

../progs/scramble -H -r ./data/ce.fa test.out/ce#sorted.full.cram
No region:   1024176
FAIL: scram.test
...
=== testing ./data/xx#unsorted.sam ===
 ../progs/scramble -t4 ./data/xx#unsorted.sam test.out/xx#unsorted.bam
 ../progs/scramble -t4 -r ./data/xx.fa test.out/xx#unsorted.bam 
test.out/xx#unsorted.full.cram
 ../progs/scramble -t4 test.out/xx#unsorted.bam > test.out/tmp.sam
 ../progs/scramble -t4 test.out/xx#unsorted.full.cram > 
test.out/xx#unsorted.full.sam
 ../progs/scramble -t4 -O bam test.out/xx#unsorted.full.cram > 
test.out/xx#unsorted.full.bam
 ../progs/scramble -t4 test.out/xx#unsorted.full.bam test.out/tmp.sam
./compare_sam.pl --nopg test.out/xx#unsorted.full.sam 
test.out/xx#unsorted.full.bam.sam
 ../progs/scramble -t4 -x -r ./data/xx.fa test.out/xx#unsorted.bam 
test.out/xx#unsorted.full.cram
 ../progs/scramble -t4 test.out/xx#unsorted.full.cram > 
test.out/xx#unsorted.full.sam
./compare_sam.pl --unknownrg ./data/xx#unsorted.sam 
test.out/xx#unsorted.full.bam.sam

../progs/scramble -t4 -H -r ./data/ce.fa test.out/ce#sorted.full.cram
No region:   1024176
FAIL: scram_mt.test
...


data/xx#unsorted.sam seems to fail randomly, with failures and passes
seen on the same architecture (including amd64).