Bug#753485: Status of samtools
Hi, Andreas Only some parts of the patches were accepted. https://github.com/samtools/htslib/commit/e8806c71f3408d2cbff8caa8df17eef848d33a02 The pull request (https://github.com/samtools/htslib/pull/99/) contains two patches, one dealing with endianness related issues and the other one dealing with unaligned access issues. Most of the changes that fix endianness related issues were accepted, except the ones that fix detection of system endiannes on mips (mips big endian is not detected). This is still an issue in htslib and consequently in samtools. I will contact upstream to see if they would be willing to reconsider including these changes at least. On the other hand, most of the changes regarding unaligned access issues weren't accepted and upstream developers suggested this should be fixed in another way, that is by implementing a standard set of functions for accessing memory in both cases (where architecture supports unaligned access and for architectures which don't) and using these functions for accessing memory throughout the code. One of the samtools developers has done some work to implement this (https://github.com/jkbonfield/htslib/tree/SPARC), but changes from this experimental branch haven't been merged to the main branch yet. Best Regards Aleksandar Zlicic
Bug#753485: samtools: python-pysam FTBFS on mips/mipsel due to issues in samtools
Hi, Dominique System endianness issue refers to a bug in function swap_endian_data (bam.c:160). Function swap_endian_data() is used on big-endian architectures when data is read from input BAM files and/or written to output BAM files, for converting multibyte integer values from little endian to big endian byte order and vice versa (depending on the operation). Additional argument to the function is_host specifies whether the input data is in host byte order or not (reading or writing), since swapping of BAM auxiliary array length values has to be done at different times for reading and writing. -static void swap_endian_data(const bam1_core_t *c, int data_len, uint8_t *data) +static void swap_endian_data(const bam1_core_t *c, int data_len, uint8_t *data, int is_host) +if(!is_host) bam_swap_endian_4p(s+1); -bam_swap_endian_4p(s+1); +if(is_host) bam_swap_endian_4p(s+1); +s += n * Bsize + 4; The last line in this change updates pointer 's' so that it points to the next uxiliary data tag-value pair (there are 'n' elements, each element of the array is Bsize bytes long; additional 4 bytes are for the 4-byte integer where length of array is stored). The following fixes 'for' loop limits (there are 'n' elements and each element is Bsize bytes long): -for (i = 0; i n; i += 2) +for (i = 0; i n*Bsize; i += 2) -for (i = 0; i n; i += 4) +for (i = 0; i n*Bsize; i += 4) Changes in fix_alignment.patch are to avoid unaligned memory access for architectures that don't allow it. I am currently adapting changes to the latest version of samtools. When I'm done I will contact upstream. Best Regards Aleksandar Zlicic
Bug#753485: samtools: python-pysam FTBFS on mips/mipsel due to issues in samtools
Package: samtools
Version: 0.1.19-1
Tags: sid patch
Severity: important
User: debian-mips-dev-disc...@lists.alioth.debian.org
Usertags: mips-patch
'python-pysam' package FTBFS on mips/mipsel due to issues in 'samtools' package.
Build log for 'python-pysam' on mips:
https://buildd.debian.org/status/fetch.php?pkg=python-pysamarch=mipsver=0.7.7-1stamp=1397894446
Build log for 'python-pysam' on mipsel:
https://buildd.debian.org/status/fetch.php?pkg=python-pysamarch=mipselver=0.7.7-1stamp=1397887877
Output of samtools executable from samtools package is used as a reference
during tests in python-pysam (its output is compared with output from
pysam python module), but samtools has both unaligned memory access
and system endianness issues.
I have attached patches that fix this. With these patches applied to samtools,
python-pysam builds fine.
Best Regards
Aleksandar Zlicic
Index: samtools-0.1.19/bam_import.c
===
--- samtools-0.1.19.orig/bam_import.c
+++ samtools-0.1.19/bam_import.c
@@ -21,6 +21,10 @@ void bam_init_header_hash(bam_header_t *
void bam_destroy_header_hash(bam_header_t *header);
int32_t bam_get_tid(const bam_header_t *header, const char *seq_name);
+#if defined(__i386__) || defined(__i386) || defined(__amd64__) || defined(__amd64) || defined(__x86_64__) || defined(__x86_64) || defined(__i686__) || defined(__i686)
+# define ALLOW_UAC
+#endif
+
unsigned char bam_nt16_table[256] = {
15,15,15,15, 15,15,15,15, 15,15,15,15, 15,15,15,15,
15,15,15,15, 15,15,15,15, 15,15,15,15, 15,15,15,15,
@@ -365,6 +369,10 @@ int sam_read1(tamFile fp, bam_header_t *
if (dret != '\n' dret != '\r') { // aux
while (ks_getuntil(ks, KS_SEP_TAB, str, dret) = 0) {
uint8_t *s, type, key[2];
+#ifndef ALLOW_UAC
+ uint8_t tmpData[8];
+ int j;
+#endif
z += str-l + 1;
if (str-l 6 || str-s[2] != ':' || str-s[4] != ':')
parse_error(fp-n_lines, missing colon in auxiliary data);
@@ -410,14 +418,30 @@ int sam_read1(tamFile fp, bam_header_t *
}
}
} else if (type == 'f') {
+#ifndef ALLOW_UAC
+float *ptmpData = (float*)tmpData;
+#endif
s = alloc_data(b, doff + 5) + doff;
*s++ = 'f';
+#ifdef ALLOW_UAC
*(float*)s = (float)atof(str-s + 5);
+#else
+*ptmpData = (float)atof(str-s + 5);
+for(j=0;j4;j++) s[j] = tmpData[j];
+#endif
s += 4; doff += 5;
} else if (type == 'd') {
+#ifndef ALLOW_UAC
+float *ptmpData = (float*)tmpData;
+#endif
s = alloc_data(b, doff + 9) + doff;
*s++ = 'd';
+#ifdef ALLOW_UAC
*(float*)s = (float)atof(str-s + 9);
+#else
+*ptmpData = (float)atof(str-s + 9);
+for(j=0;j4;j++) s[j] = tmpData[j];
+#endif
s += 8; doff += 9;
} else if (type == 'Z' || type == 'H') {
int size = 1 + (str-l - 5) + 1;
Index: samtools-0.1.19/bam.c
===
--- samtools-0.1.19.orig/bam.c
+++ samtools-0.1.19/bam.c
@@ -157,7 +157,7 @@ int bam_header_write(bamFile fp, const b
return 0;
}
-static void swap_endian_data(const bam1_core_t *c, int data_len, uint8_t *data)
+static void swap_endian_data(const bam1_core_t *c, int data_len, uint8_t *data, int is_host)
{
uint8_t *s;
uint32_t i, *cigar = (uint32_t*)(data + c-l_qname);
@@ -174,16 +174,18 @@ static void swap_endian_data(const bam1_
else if (type == 'Z' || type == 'H') { while (*s) ++s; ++s; }
else if (type == 'B') {
int32_t n, Bsize = bam_aux_type2size(*s);
+ if(!is_host) bam_swap_endian_4p(s+1);
memcpy(n, s + 1, 4);
if (1 == Bsize) {
} else if (2 == Bsize) {
-for (i = 0; i n; i += 2)
+for (i = 0; i n*Bsize; i += 2)
bam_swap_endian_2p(s + 5 + i);
} else if (4 == Bsize) {
-for (i = 0; i n; i += 4)
+for (i = 0; i n*Bsize; i += 4)
bam_swap_endian_4p(s + 5 + i);
}
- bam_swap_endian_4p(s+1);
+ if(is_host) bam_swap_endian_4p(s+1);
+ s += n * Bsize + 4;
}
}
}
@@ -217,7 +219,7 @@ int bam_read1(bamFile fp, bam1_t *b)
}
if (bam_read(fp, b-data, b-data_len) != b-data_len) return -4;
b-l_aux = b-data_len - c-n_cigar * 4 - c-l_qname - c-l_qseq - (c-l_qseq+1)/2;
- if (bam_is_be) swap_endian_data(c, b-data_len, b-data);
+ if (bam_is_be) swap_endian_data(c, b-data_len, b-data, 0);
if (bam_no_B) bam_remove_B(b);
return 4 + block_len;
}
@@ -240,11 +242,11 @@ inline int bam_write1_core(bamFile fp, c
for (i = 0; i 8; ++i) bam_swap_endian_4p(x + i);
y = block_len;
bam_write(fp, bam_swap_endian_4p(y), 4);
- swap_endian_data(c, data_len, data);
+ swap_endian_data(c, data_len, data, 1);
} else bam_write(fp, block_len, 4);
bam_write(fp, x, BAM_CORE_SIZE);
bam_write(fp, data, data_len);
- if (bam_is_be) swap_endian_data(c, data_len, data);
+ if (bam_is_be) swap_endian_data(c, data_len, data, 0);
return 4 + block_len;
}
Bug#753490: python-pysam FTBFS on mips/mipsel
Package: python-pysam
Version: 0.7.7-1
Tags: sid patch
Severity: important
Justification: FTBFS
User: debian-mips-dev-disc...@lists.alioth.debian.org
Usertags: mips-patch
Control: block -1 by 753485
While trying to build python-pysam on mips/mipsel, build fails on testing.
Build log for 'python-pysam' on mips:
https://buildd.debian.org/status/fetch.php?pkg=python-pysamarch=mipsver=0.7.7-1stamp=1397894446
Build log for 'python-pysam' on mipsel:
https://buildd.debian.org/status/fetch.php?pkg=python-pysamarch=mipselver=0.7.7-1stamp=1397887877
The cause of test failures are issues in both 'samtools' and 'python-pysam'
packages.
For python-pysam to work samtools has to be fixed first since output of
samtools executable from samtools package is used as a reference
during tests in python-pysam (its output is compared with output from
pysam python module).
python-pysam has unaligned memory access and system endianness related issues
as well.
I have attached patches that fix this.
With samtools fixed (with solution proposed at
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=753485), and attached patches
are applied to python-pysam, I have successfully built python-pysam on
mips and mipsel.
Could you consider including these patches?
Best regards,
Aleksandar Zlicic
Index: python-pysam-0.7.7/samtools/bam_import.c.pysam.c
===
--- python-pysam-0.7.7.orig/samtools/bam_import.c.pysam.c
+++ python-pysam-0.7.7/samtools/bam_import.c.pysam.c
@@ -367,6 +367,9 @@ int sam_read1(tamFile fp, bam_header_t *
if (dret != '\n' dret != '\r') { // aux
while (ks_getuntil(ks, KS_SEP_TAB, str, dret) = 0) {
uint8_t *s, type, key[2];
+ uint8_t tmpData[8];
+ int j;
+
z += str-l + 1;
if (str-l 6 || str-s[2] != ':' || str-s[4] != ':')
parse_error(fp-n_lines, missing colon in auxiliary data);
@@ -412,15 +415,35 @@ int sam_read1(tamFile fp, bam_header_t *
}
}
} else if (type == 'f') {
+#ifdef ALLOW_UAC
s = alloc_data(b, doff + 5) + doff;
*s++ = 'f';
*(float*)s = (float)atof(str-s + 5);
s += 4; doff += 5;
+#else
+float *ptmpData = (float*)tmpData;
+s = alloc_data(b, doff + 5) + doff;
+*s++ = 'f';
+for(j=0;j4;j++) tmpData[j]=s[j];
+*ptmpData = (float)atof(str-s + 5);
+for(j=0;j4;j++) s[j] = tmpData[j];
+s += 4; doff += 5;
+#endif
} else if (type == 'd') {
+#ifdef ALLOW_UAC
s = alloc_data(b, doff + 9) + doff;
*s++ = 'd';
*(float*)s = (float)atof(str-s + 9);
s += 8; doff += 9;
+#else
+float *ptmpData = (float*)tmpData;
+s = alloc_data(b, doff + 9) + doff;
+*s++ = 'd';
+for(j=0;j8;j++) tmpData[j]=s[j];
+*ptmpData = (float)atof(str-s + 9);
+for(j=0;j8;j++) s[j] = tmpData[j];
+s += 8; doff += 9;
+#endif
} else if (type == 'Z' || type == 'H') {
int size = 1 + (str-l - 5) + 1;
if (type == 'H') { // check whether the hex string is valid
Index: python-pysam-0.7.7/samtools/bam.h
===
--- python-pysam-0.7.7.orig/samtools/bam.h
+++ python-pysam-0.7.7/samtools/bam.h
@@ -70,6 +70,12 @@ typedef gzFile bamFile;
/* no bam_write/bam_tell/bam_seek() here */
#endif
+#if defined(__i386__) || defined(__i386) || defined(__amd64__) || defined(__amd64) || defined(__x86_64__) || defined(__x86_64) || defined(__i686__) || defined(__i686)
+# define ALLOW_UAC
+#else
+# undef ALLOW_UAC
+#endif
+
/*! @typedef
@abstract Structure for the alignment header.
@field n_targets number of reference sequences
Index: python-pysam-0.7.7/samtools/bam_aux.c.pysam.c
===
--- python-pysam-0.7.7.orig/samtools/bam_aux.c.pysam.c
+++ python-pysam-0.7.7/samtools/bam_aux.c.pysam.c
@@ -180,7 +180,18 @@ float bam_aux2f(const uint8_t *s)
int type;
type = *s++;
if (s == 0) return 0.0;
+#ifdef ALLOW_UAC
if (type == 'f') return *(float*)s;
+#else
+ if (type == 'f')
+ {
+ uint8_t tmpData[4];
+ int j;
+ float *ptmpData = (float*)tmpData;
+ for(j=0;j4;j++) tmpData[j]=s[j];
+ return *ptmpData;
+ }
+#endif
else return 0.0;
}
@@ -189,7 +200,19 @@ double bam_aux2d(const uint8_t *s)
int type;
type = *s++;
if (s == 0) return 0.0;
+#ifdef ALLOW_UAC
if (type == 'd') return *(double*)s;
+#else
+ if (type == 'd')
+ {
+ uint8_t tmpData[8];
+ int j;
+ double *ptmpData = (double*)tmpData;
+ for(j=0;j8;j++) tmpData[j]=s[j];
+ return *ptmpData;
+
+ }
+#endif
else return 0.0;
}
Index: python-pysam-0.7.7/pysam/csamtools.pyx
===
--- python-pysam-0.7.7.orig/pysam/csamtools.pyx
+++ python-pysam-0.7.7/pysam/csamtools.pyx
@@ -2763,7 +2763,7 @@ cdef class AlignedRead:
src = self._delegate
-fmts, args = [], []
+fmts, args = [=], []
if tags != None:
Index: python-pysam-0.7.7
Bug#753485: [Debian-med-packaging] Bug#753485: samtools: python-pysam FTBFS on mips/mipsel due to issues in samtools
Ok, I will contact upstream. Best Regards Aleksandar Zlicic From: Charles Plessy [ple...@debian.org] Sent: Wednesday, July 02, 2014 3:16 PM To: Aleksandar Zlicic; 753...@bugs.debian.org Subject: Re: [Debian-med-packaging] Bug#753485: samtools: python-pysam FTBFS on mips/mipsel due to issues in samtools Le Wed, Jul 02, 2014 at 12:30:24PM +, Aleksandar Zlicic a écrit : Package: samtools Version: 0.1.19-1 Tags: sid patch Severity: important User: debian-mips-dev-disc...@lists.alioth.debian.org Usertags: mips-patch 'python-pysam' package FTBFS on mips/mipsel due to issues in 'samtools' package. Build log for 'python-pysam' on mips: https://buildd.debian.org/status/fetch.php?pkg=python-pysamarch=mipsver=0.7.7-1stamp=1397894446 Build log for 'python-pysam' on mipsel: https://buildd.debian.org/status/fetch.php?pkg=python-pysamarch=mipselver=0.7.7-1stamp=1397887877 Output of samtools executable from samtools package is used as a reference during tests in python-pysam (its output is compared with output from pysam python module), but samtools has both unaligned memory access and system endianness issues. I have attached patches that fix this. With these patches applied to samtools, python-pysam builds fine. Hi Aleksandar, I admit that I am a bit worried of applying a patch that I am not able to review. Could you ask upstream (https://github.com/samtools/samtools) to have a look ? Have a nice day, -- Charles Plessy Debian Med packaging team, http://www.debian.org/devel/debian-med Tsurumi, Kanagawa, Japan -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#749367: htslib FTBFS on testing for mips/mipsel
Sure, I will upload the patches.
I am currently working on adapting the changes to the latest upstream version.
Also I have added a minor change to patch '02-fix-alignment.patch',
which you will find attached.
Best Regards
Aleksandar Zlicic
Index: htslib-0.2.0~rc3/sam.c
===
--- htslib-0.2.0~rc3.orig/sam.c
+++ htslib-0.2.0~rc3/sam.c
@@ -7,6 +7,7 @@
#include htslib/sam.h
#include htslib/bgzf.h
#include cram/cram.h
+#include cram/os.h
#include hfile.h
#include htslib/khash.h
@@ -726,18 +727,69 @@ int sam_format1(const bam_hdr_t *h, cons
s = bam_get_aux(b); // aux
while (s b-data + b-l_data) {
uint8_t type, key[2];
+ uint8_t tmpData[8];
+ int j;
key[0] = s[0]; key[1] = s[1];
s += 2; type = *s++;
kputc('\t', str); kputsn((char*)key, 2, str); kputc(':', str);
if (type == 'A') { kputsn(A:, 2, str); kputc(*s, str); ++s; }
else if (type == 'C') { kputsn(i:, 2, str); kputw(*s, str); ++s; }
else if (type == 'c') { kputsn(i:, 2, str); kputw(*(int8_t*)s, str); ++s; }
+#ifdef ALLOW_UAC
else if (type == 'S') { kputsn(i:, 2, str); kputw(*(uint16_t*)s, str); s += 2; }
else if (type == 's') { kputsn(i:, 2, str); kputw(*(int16_t*)s, str); s += 2; }
else if (type == 'I') { kputsn(i:, 2, str); kputuw(*(uint32_t*)s, str); s += 4; }
else if (type == 'i') { kputsn(i:, 2, str); kputw(*(int32_t*)s, str); s += 4; }
else if (type == 'f') { ksprintf(str, f:%g, *(float*)s); s += 4; }
else if (type == 'd') { ksprintf(str, d:%g, *(double*)s); s += 8; }
+#else
+ else if (type == 'S')
+{
+uint16_t *ptmpData = (uint16_t*)tmpData;
+for(j=0;j2;j++) tmpData[j]=s[j];
+kputsn(i:, 2, str);
+kputw(*ptmpData, str);
+s += 2;
+}
+ else if (type == 's')
+{
+int16_t *ptmpData = (int16_t*)tmpData;
+for(j=0;j2;j++) tmpData[j]=s[j];
+kputsn(i:, 2, str);
+kputw(*ptmpData, str);
+s += 2;
+}
+ else if (type == 'I')
+{
+uint32_t *ptmpData = (uint32_t*)tmpData;
+for(j=0;j4;j++) tmpData[j]=s[j];
+kputsn(i:, 2, str);
+kputuw(*ptmpData, str);
+s += 4;
+}
+ else if (type == 'i')
+{
+int32_t *ptmpData = (int32_t*)tmpData;
+for(j=0;j4;j++) tmpData[j]=s[j];
+kputsn(i:, 2, str);
+kputw(*ptmpData, str);
+s += 4;
+}
+ else if (type == 'f')
+{
+float *ptmpData = (float*)tmpData;
+for(j=0;j4;j++) tmpData[j]=s[j];
+ksprintf(str, f:%g, *ptmpData);
+s += 4;
+}
+ else if (type == 'd')
+{
+float *ptmpData = (float*)tmpData;
+for(j=0;j8;j++) tmpData[j]=s[j];
+ksprintf(str, d:%g, *ptmpData);
+s += 8;
+}
+#endif
else if (type == 'Z' || type == 'H') { kputc(type, str); kputc(':', str); while (*s) kputc(*s++, str); ++s; }
else if (type == 'B') {
uint8_t sub_type = *(s++);
@@ -749,11 +801,49 @@ int sam_format1(const bam_hdr_t *h, cons
kputc(',', str);
if ('c' == sub_type) { kputw(*(int8_t*)s, str); ++s; }
else if ('C' == sub_type) { kputw(*(uint8_t*)s, str); ++s; }
+#ifdef ALLOW_UAC
else if ('s' == sub_type) { kputw(*(int16_t*)s, str); s += 2; }
else if ('S' == sub_type) { kputw(*(uint16_t*)s, str); s += 2; }
else if ('i' == sub_type) { kputw(*(int32_t*)s, str); s += 4; }
else if ('I' == sub_type) { kputuw(*(uint32_t*)s, str); s += 4; }
else if ('f' == sub_type) { ksprintf(str, %g, *(float*)s); s += 4; }
+#else
+else if ('s' == sub_type)
+{
+int16_t *ptmpData = (int16_t*)tmpData;
+for(j=0;j2;j++) tmpData[j]=s[j];
+kputw(*ptmpData, str);
+s += 2;
+}
+else if ('S' == sub_type)
+{
+uint16_t *ptmpData = (uint16_t*)tmpData;
+for(j=0;j2;j++) tmpData[j]=s[j];
+kputw(*ptmpData, str);
+s += 2;
+}
+else if ('i' == sub_type)
+{
+int32_t *ptmpData = (int32_t*)tmpData;
+for(j=0;j4;j++) tmpData[j]=s[j];
+kputw(*ptmpData
Bug#749367: htslib FTBFS on testing for mips/mipsel
Package: htslib
Version: 0.2.0~rc3-1
Tags: sid patch
Severity: important
Justification: FTBFS
User: debian-mips-dev-disc...@lists.alioth.debian.org
Usertags: mips-patch
While trying to build htslib on mips/mipsel, build fails on testing.
On mips, package fails with the following error:
./compare_sam.pl xx#unsorted.sam xx#unsorted.tmp.cram.sam_
./test_view -t xx.fa -C xx#unsorted.tmp.bam xx#unsorted.tmp.bam.cram
./test_view -b -D xx#unsorted.tmp.bam.cram xx#unsorted.tmp.bam.cram.bam
./test_view xx#unsorted.tmp.bam.cram.bam xx#unsorted.tmp.bam.cram.bam.sam_
./compare_sam.pl xx#unsorted.sam xx#unsorted.tmp.bam.cram.bam.sam_
Successes 235
Failures 5
make[2]: *** [test] Error 1
make[2]: Leaving directory `/«PKGBUILDDIR»'
dh_auto_test: make -j1 test returned exit code 2
make[1]: *** [override_dh_auto_test] Error 2
make[1]: Leaving directory `/«PKGBUILDDIR»'
make: *** [build-arch] Error 2
The reason for that are endianness related issues:
- system endianness is not detected properly
- function which converts multibyte data from big endian byte to little
endian byte order (and vice versa) has errors (this was partially fixed
in latest upstream release).
This is fixed in '01-fix-endianness.patch'.
After applying it, build fails with error:
./compare_sam.pl xx#unsorted.sam xx#unsorted.tmp.cram.sam_
./test_view -t xx.fa -C xx#unsorted.tmp.bam xx#unsorted.tmp.bam.cram
./test_view -b -D xx#unsorted.tmp.bam.cram xx#unsorted.tmp.bam.cram.bam
./test_view xx#unsorted.tmp.bam.cram.bam xx#unsorted.tmp.bam.cram.bam.sam_
./compare_sam.pl xx#unsorted.sam xx#unsorted.tmp.bam.cram.bam.sam_
make[2]: *** [test] Error 1
Successes 234
Failures 6
make[2]: Leaving directory `/«PKGBUILDDIR»'
dh_auto_test: make -j1 test returned exit code 2
make[1]: *** [override_dh_auto_test] Error 2
make[1]: Leaving directory `/«PKGBUILDDIR»'
make: *** [build-arch] Error 2
The same error appears on mipsel.
Reason for this is that unaligned memory access is not handled well
for achitectures that don't allow unaligned access, which causes bus errors
on both mips and mipsel.
This is fixed in '02-fix-alignment.patch'.
With these patches applied I have successfully built htslib on mips/mipsel.
Could you please consider including this patch?
Best regards,
Aleksandar Zlicic
--- a/cram/cram_encode.c
+++ b/cram/cram_encode.c
@@ -1858,6 +1858,12 @@
return rg;
}
+static inline int is_big_endian(){
+int x = 0x01;
+char *c = (char*)x;
+return (c[0] != 0x01);
+}
+
/*
* Encodes auxiliary data. Largely duplicated from above, but done so to
* keep it simple and avoid a myriad of version ifs.
@@ -1949,10 +1955,21 @@
case 'B': {
int type = aux[3], blen;
- uint32_t count = (uint32_t)unsigned char *)aux)[4] 0) +
+uint32_t count;
+if(is_big_endian())
+{
+ count = (uint32_t)unsigned char *)aux)[7] 0) +
+ (((unsigned char *)aux)[6] 8) +
+ (((unsigned char *)aux)[5]16) +
+ (((unsigned char *)aux)[4]24));
+}
+else
+{
+ count = (uint32_t)unsigned char *)aux)[4] 0) +
(((unsigned char *)aux)[5] 8) +
(((unsigned char *)aux)[6]16) +
(((unsigned char *)aux)[7]24));
+}
// skip TN field
aux+=3;
--- a/cram/os.h
+++ b/cram/os.h
@@ -88,8 +88,15 @@
* processor type too.
*/
-/* Set by autoconf */
-#define SP_LITTLE_ENDIAN
+#if !defined(SP_BIG_ENDIAN) !defined(SP_LITTLE_ENDIAN)
+
+# if __BYTE_ORDER == __BIG_ENDIAN
+# define SP_BIG_ENDIAN
+#elif __BYTE_ORDER == __LITTLE_ENDIAN
+# define SP_LITTLE_ENDIAN
+#endif
+
+#endif
/* Mac FAT binaries or unknown. Auto detect based on CPU type */
#if !defined(SP_BIG_ENDIAN) !defined(SP_LITTLE_ENDIAN)
--- a/sam.c
+++ b/sam.c
@@ -216,7 +216,12 @@
else return 0;
}
-static void swap_data(const bam1_core_t *c, int l_data, uint8_t *data)
+typedef enum swap_data_rw {
+SWAP_DATA_READ,
+SWAP_DATA_WRITE
+}swap_data_rw_t;
+
+static void swap_data(const bam1_core_t *c, int l_data, uint8_t *data, swap_data_rw_t rw_mode)
{
uint8_t *s;
uint32_t *cigar = (uint32_t*)(data + c-l_qname);
@@ -234,15 +239,23 @@
else if (type == 'Z' || type == 'H') { while (*s) ++s; ++s; }
else if (type == 'B') {
int32_t n, Bsize = aux_type2size(*s);
+ if(SWAP_DATA_READ == rw_mode)
+ {
+ed_swap_4p(s+1);
+ }
memcpy(n, s + 1, 4);
if (Bsize == 2) {
-for (i = 0; i n; i += 2)
+for (i = 0; i n*Bsize; i += 2)
ed_swap_2p(s + 5 + i);
} else if (Bsize == 4) {
-for (i = 0; i n; i += 4)
+for (i = 0; i n*Bsize; i += 4)
ed_swap_4p(s + 5 + i);
}
- ed_swap_4p(s+1);
+ if(SWAP_DATA_WRITE == rw_mode)
+ {
+ed_swap_4p(s+1);
+ }
+ s += (4 + Bsize * n + 1);
}
}
}
@@ -274,7 +287,7 @@
}
if (bgzf_read(fp, b-data, b-l_data) != b-l_data) return -4;
//b-l_aux = b-l_data - c-n_cigar * 4 - c-l_qname - c-l_qseq - (c-l_qseq+1)/2;
- if (fp-is_be) swap_data(c
Bug#742940: since: Does not work on mips (includes test suite failures and FTBFS)
The cause of problem is that st_dev member of structure stat,
defined in /usr/include/mipsel-linux-gnu/bits/stat.h,
is expected to be of type dev_t.
But for mips/mipsel ABIO32 it is defined as 'unsigned long int'.
dev_t type is defined as 'unsigned long long int'.
Attached patch fixes package on mips/mipsel.
Could you please consider including these changes
in order to build since for mips/mipsel architectures?
Thanks
Aleksandar Zlicic
Index: since-1.1/since.c
===
--- since-1.1.orig/since.c 2014-04-03 14:01:50.0 +
+++ since-1.1/since.c 2014-04-07 15:32:15.0 +
@@ -52,7 +52,11 @@
struct data_file{
int d_fd;
char *d_name;
+#if defined (__mips__ ) (_MIPS_SIM == _ABIO32)
+ unsigned long int d_dev;
+#else
dev_t d_dev;
+#endif
ino_t d_ino;
off_t d_had;
off_t d_now;
Bug#736565: FTBFS on non-PC architectures: FAIL: scram_mt.test
Hello,
The cause of this issue is an error in function bam_get_seq(),
or to be more precise, function's implementation for non-intel architectures.
Function bam_get_seq is used for reading data from alignment blocks
in BAM files, block by block.
In special cases, when the alignment block being read is the last block in BAM
file,
and the block has no cigar operations and segment sequence stored inside of it,
function incorrectly detects end-of-file.
In that case,
block has actually been successfully read,
but because of erroneous end-of-file detection,
it isn't written to output file in SAM format.
Attached patch fixes this issue.
It is tested on mips, mipsel and powerpc.
Best Regards
Aleksandar Zlicic
--- a/io_lib/bam.c
+++ b/io_lib/bam.c
@@ -1636,7 +1636,7 @@
/* The remainder, word aligned */
blk_size = blk_ret;
-if ((blk_ret = bam_read(b, (char *)bam_cigar(bs), blk_size+4)) == 0)
+if (((blk_ret = bam_read(b, (char *)bam_cigar(bs), blk_size+4)) == 0) blk_size!=0)
return 0;
if (blk_size+4 != blk_ret) {
