Bug#956324: Clustalo bus error on mipsel (Was: Bug#956324: python-biopython: FTBFS on mipsel)
Hi Adrian, On Mon, May 11, 2020 at 10:04:24AM +0300, Adrian Bunk wrote: > A bug buried somewhere in the Debian bts would not change anything. Probably that's correct. > What would have to happen would be a Debian MIPS porter debugging > this problem and then submitting a minimal testcase to gcc upstream. I'd be really happy about this. > Assuming this will be confirmed to be a mipsel-specific gcc bug, > this looks likely but it is still possible that there actually > is a bug in clustalo that works by chance on other architectures. > Unlikely, but not impossible. The debugging sessions on clustalo and suggested patches uncovered code parts that are not looking nice - so I have no idea about the likelihood of this. For me it seems a possibility that should be kept in mind and which is why I tried my best to support the debugging sessions. > In a more positive note, I assume people would have already noticed if > OpenMP on mipsel in stable and unstable would be completely broken. Probably. Kind regards Andreas. -- http://fam-tille.de
Bug#956324: Clustalo bus error on mipsel (Was: Bug#956324: python-biopython: FTBFS on mipsel)
On Sun, May 10, 2020 at 08:56:50PM +0200, Andreas Tille wrote: >... > On Sun, May 10, 2020 at 11:19:27AM +0300, Adrian Bunk wrote: > > What does fix the problem is disabling OpenMP. > > I suspect OpenMP is somehow broken in gcc >= 8 on mipsel. > > I wonder how we could keep this finding to other packages. I bet it > would not the only issue but it has shown up due to intense testing. >... > I've now uploaded now with this patch closing the bug - but as I tried to > express I'd sleep a bit better if this issue would be recorded somewhere > else than in a closed bug. A bug buried somewhere in the Debian bts would not change anything. What would have to happen would be a Debian MIPS porter debugging this problem and then submitting a minimal testcase to gcc upstream. Assuming this will be confirmed to be a mipsel-specific gcc bug, this looks likely but it is still possible that there actually is a bug in clustalo that works by chance on other architectures. Unlikely, but not impossible. In a more positive note, I assume people would have already noticed if OpenMP on mipsel in stable and unstable would be completely broken. > Thanks again for your research > > Andreas. cu Adrian
Bug#956324: Clustalo bus error on mipsel (Was: Bug#956324: python-biopython: FTBFS on mipsel)
On Sun, May 10, 2020 at 03:51:28PM -0400, Jeffrey Walton wrote: > > I've now uploaded now with this patch closing the bug - but as I tried to > > express I'd sleep a bit better if this issue would be recorded somewhere > > else than in a closed bug. > > Maybe GCC? I believe the component is libgomp. > > If you have a good idea of the problematic source file, then add the > preprocessoed source with the report. Also see > https://gcc.gnu.org/bugs/. > > If you don't have an idea, then maybe someone like Andrew or Ian can > provide some suggestions. I admit I don't have any idea, sorry. Kind regards Andreas. -- http://fam-tille.de
Bug#956324: Clustalo bus error on mipsel (Was: Bug#956324: python-biopython: FTBFS on mipsel)
On Sun, May 10, 2020 at 2:57 PM Andreas Tille wrote: > > ... > > --- clustalo-1.2.4/debian/rules 2020-04-14 12:19:44.0 +0300 > > +++ clustalo-1.2.4/debian/rules 2020-04-14 12:19:44.0 +0300 > > @@ -9,6 +9,11 @@ > > %: > > dh $@ > > > > +ifneq (,$(filter $(DEB_HOST_ARCH), mipsel)) > > +override_dh_auto_configure: > > + dh_auto_configure -- --without-openmp > > +endif > > + > > override_dh_auto_build-indep: > > # nothing to do here > > I've now uploaded now with this patch closing the bug - but as I tried to > express I'd sleep a bit better if this issue would be recorded somewhere > else than in a closed bug. Maybe GCC? I believe the component is libgomp. If you have a good idea of the problematic source file, then add the preprocessoed source with the report. Also see https://gcc.gnu.org/bugs/. If you don't have an idea, then maybe someone like Andrew or Ian can provide some suggestions. Jeff
Bug#956324: Clustalo bus error on mipsel (Was: Bug#956324: python-biopython: FTBFS on mipsel)
Hi Adrian, thanks a lot for your investigation. On Sun, May 10, 2020 at 11:19:27AM +0300, Adrian Bunk wrote: > What does fix the problem is disabling OpenMP. > I suspect OpenMP is somehow broken in gcc >= 8 on mipsel. I wonder how we could keep this finding to other packages. I bet it would not the only issue but it has shown up due to intense testing. > Below is a workaround patch (lower performance of clustalo on mipsel > shouldn't be a major problem). Its definitely no problem - I doubt that the package is used at all on mipsel. Its simply of theoretical interest and might uncover hidden issues (finally some suggestions to enhance the code were found not only for mipsel). > --- clustalo-1.2.4/debian/rules 2020-04-14 12:19:44.0 +0300 > +++ clustalo-1.2.4/debian/rules 2020-04-14 12:19:44.0 +0300 > @@ -9,6 +9,11 @@ > %: > dh $@ > > +ifneq (,$(filter $(DEB_HOST_ARCH), mipsel)) > +override_dh_auto_configure: > + dh_auto_configure -- --without-openmp > +endif > + > override_dh_auto_build-indep: > # nothing to do here I've now uploaded now with this patch closing the bug - but as I tried to express I'd sleep a bit better if this issue would be recorded somewhere else than in a closed bug. Thanks again for your research Andreas. -- http://fam-tille.de
Bug#956324: Clustalo bus error on mipsel (Was: Bug#956324: python-biopython: FTBFS on mipsel)
Hi, I tried to narrow down what broke clustalo on mipsel. The test from 1.2.4-5 passes with the binary in buster, but not when rebuilding it in buster. gcc versions: 7: works 8: broken 9: broken 10: broken clustalo in buster was built with gcc 7, a rebuild with the gcc 8 now in buster would result in a broken binary. Rebuilding with gcc 8 with -O0 does not work around the problem, this is unlikely to be the the common case of optimization changes breaking buggy code. What does fix the problem is disabling OpenMP. I suspect OpenMP is somehow broken in gcc >= 8 on mipsel. Below is a workaround patch (lower performance of clustalo on mipsel shouldn't be a major problem). cu Adrian --- clustalo-1.2.4/debian/rules 2020-04-14 12:19:44.0 +0300 +++ clustalo-1.2.4/debian/rules 2020-04-14 12:19:44.0 +0300 @@ -9,6 +9,11 @@ %: dh $@ +ifneq (,$(filter $(DEB_HOST_ARCH), mipsel)) +override_dh_auto_configure: + dh_auto_configure -- --without-openmp +endif + override_dh_auto_build-indep: # nothing to do here
Bug#956324: Clustalo bus error on mipsel (Was: Bug#956324: python-biopython: FTBFS on mipsel)
On Fri, May 1, 2020 at 3:05 AM Jeffrey Walton wrote:
>
> On Fri, May 1, 2020 at 2:14 AM Andreas Tille wrote:
> >
> > ...
> > ==13209== Process terminating with default action of signal 10 (SIGBUS)
> > ==13209==at 0x12D5CC: PairDistances (pair_dist.c:346)
> > ==13209==by 0x119410: AlignmentOrder (clustal-omega.c:835)
> > ==13209==by 0x11A6C4: Align (clustal-omega.c:1221)
> > ==13209==by 0x1171C8: MyMain (mymain.c:1192)
> > ==13209==by 0x113CCC: main (main.cpp:469)
>
> Here is line 346 in
> https://salsa.debian.org/med-team/clustalo/-/blob/master/src/clustal/pair_dist.c#L346:
>
> NewProgress(&prProgress, LogGetFP(&rLog, LOG_INFO),
> "Ktuple-distance calculation progress", bPrintCR);
>
> For testing, change LogGetFP(&rLog, LOG_INFO) for stdout for testing. I.e.,
>
> NewProgress(&prProgress, stdout,,
> "Ktuple-distance calculation progress", bPrintCR);
>
> It looks like LogGetFP retrieves an entry in an array of FILE*. From
> https://salsa.debian.org/med-team/clustalo/-/blob/master/src/clustal/log.h:
>
> typedef struct {
> /* the higher the level, the more priority it has. numbers must be
> * sequential
> */
>
> /* array of function pointers */
> void (*prFunc[LOG_NUM_LEVELS]) (FILE *prFP, char *pcFormat,
> va_list rVArgList);
> FILE *prFP[LOG_NUM_LEVELS];
> char *prPrefix[LOG_NUM_LEVELS];
>
> /* everything above this level will be printed */
> int iLogLevelEnabled;
> } log_t;
>
> And
> https://salsa.debian.org/med-team/clustalo/-/blob/master/src/clustal/log.c:
>
> FILE *
> LogGetFP(log_t *prLog, int iLevel)
> {
> assert(iLevel>=0 && iLevel<=LOG_NUM_LEVELS);
> return prLog->prFP[iLevel];
> }
>
> That should help determine if something is sideways in the log_t structure.
Also, I think this should be:
> FILE *
> LogGetFP(log_t *prLog, int iLevel)
> {
> assert(iLevel>=0 && iLevel return prLog->prFP[iLevel];
> }
That is, 'iLevel
Bug#956324: Clustalo bus error on mipsel (Was: Bug#956324: python-biopython: FTBFS on mipsel)
On Fri, May 1, 2020 at 2:14 AM Andreas Tille wrote:
>
> ...
> ==13209== Process terminating with default action of signal 10 (SIGBUS)
> ==13209==at 0x12D5CC: PairDistances (pair_dist.c:346)
> ==13209==by 0x119410: AlignmentOrder (clustal-omega.c:835)
> ==13209==by 0x11A6C4: Align (clustal-omega.c:1221)
> ==13209==by 0x1171C8: MyMain (mymain.c:1192)
> ==13209==by 0x113CCC: main (main.cpp:469)
Here is line 346 in
https://salsa.debian.org/med-team/clustalo/-/blob/master/src/clustal/pair_dist.c#L346:
NewProgress(&prProgress, LogGetFP(&rLog, LOG_INFO),
"Ktuple-distance calculation progress", bPrintCR);
For testing, change LogGetFP(&rLog, LOG_INFO) for stdout for testing. I.e.,
NewProgress(&prProgress, stdout,,
"Ktuple-distance calculation progress", bPrintCR);
It looks like LogGetFP retrieves an entry in an array of FILE*. From
https://salsa.debian.org/med-team/clustalo/-/blob/master/src/clustal/log.h:
typedef struct {
/* the higher the level, the more priority it has. numbers must be
* sequential
*/
/* array of function pointers */
void (*prFunc[LOG_NUM_LEVELS]) (FILE *prFP, char *pcFormat,
va_list rVArgList);
FILE *prFP[LOG_NUM_LEVELS];
char *prPrefix[LOG_NUM_LEVELS];
/* everything above this level will be printed */
int iLogLevelEnabled;
} log_t;
And https://salsa.debian.org/med-team/clustalo/-/blob/master/src/clustal/log.c:
FILE *
LogGetFP(log_t *prLog, int iLevel)
{
assert(iLevel>=0 && iLevel<=LOG_NUM_LEVELS);
return prLog->prFP[iLevel];
}
That should help determine if something is sideways in the log_t structure.
Jeff
Bug#956324: Clustalo bus error on mipsel (Was: Bug#956324: python-biopython: FTBFS on mipsel)
Hi Matthew, On Thu, Apr 30, 2020 at 05:53:29PM -0700, Matthew Fernandez wrote: > > Is the priority goal here to simply ship a non-crashing clustalo mipsel > binary that BioPython can depend on? If so, maybe we can just disable > compiler optimisation (-O0) and this may avoid provoking the bus error. Of > course this doesn’t fix the underlying problem(s), but it looks as if > debugging this to its root cause is going to result in the Debian package > carrying a lot of invasive dquilt patches on top of upstream. OTOH I don’t > know the performance requirements of BioPython and maybe an unoptimised > clustalo is unacceptable to it. I need to admit the priority goal is to be really sure that the clustalo code has no hidden issues which might crash on architectures that are used in practical applications. I would have no problems to simply exclude clustalo for mipsel architecture completely - if I could be sure that it works properly. So the major reason for this debugging session is to make sure that clustalo runs properly *on all other* architectures while loosing mipsel would be no real loss for practival usage of clustalo and its rdepends. To follow your hints I cared for -O0 optimisation flags (which is possible via configure options which uncovers some syntax errors in comments :-( which I fixed as well) and tried to rebuild on the mipsel host provided by Debian. Unfortunately the bus error remains. Due to the advise here that valgrind works properly only for -O0 I'm reposting the valgrind output here: (sid_mipsel-dchroot)tille@eller:~/clustalo$ valgrind -s src/clustalo -i debian/tests/biopython_testdata/f002 --guidetree-out temp_test.dnd -o temp_test.aln --outfmt clustal --force ==13209== Memcheck, a memory error detector ==13209== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al. ==13209== Using Valgrind-3.15.0 and LibVEX; rerun with -h for copyright info ==13209== Command: src/clustalo -i debian/tests/biopython_testdata/f002 --guidetree-out temp_test.dnd -o temp_test.aln --outfmt clustal --force ==13209== ==13209== Conditional jump or move depends on uninitialised value(s) ==13209==at 0x4007828: cached_fpabi_reject_phdr_p (dl-machine-reject-phdr.h:57) ==13209==by 0x4007828: elf_machine_reject_phdr_p (dl-machine-reject-phdr.h:217) ==13209==by 0x4008080: open_verify.constprop.0 (dl-load.c:1688) ==13209==by 0x4009D7C: _dl_map_object (dl-load.c:2181) ==13209==by 0x40011F8: map_doit (rtld.c:607) ==13209==by 0x401B2A8: _dl_catch_exception (dl-error-skeleton.c:196) ==13209==by 0x401B334: _dl_catch_error (dl-error-skeleton.c:215) ==13209==by 0x4001138: do_preload (rtld.c:778) ==13209==by 0x4002560: handle_preload_list (rtld.c:879) ==13209==by 0x4005B08: dl_main (rtld.c:1684) ==13209==by 0x401A094: _dl_sysdep_start (dl-sysdep.c:253) ==13209==by 0x400199C: _dl_start_final (rtld.c:447) ==13209==by 0x4001D44: _dl_start (rtld.c:539) ==13209== ==13209== Invalid read of size 1 ==13209==at 0x486F558: ??? (in /usr/lib/mipsel-linux-gnu/libargtable2.so.0.1.8) ==13209==by 0x486F5F0: ??? (in /usr/lib/mipsel-linux-gnu/libargtable2.so.0.1.8) ==13209== Address 0x4d655c5 is 0 bytes after a block of size 37 alloc'd ==13209==at 0x484B89C: malloc (in /usr/lib/mipsel-linux-gnu/valgrind/vgpreload_memcheck-mips32-linux.so) ==13209==by 0x134D1C: CkMalloc (util.c:83) ==13209== ==13209== Invalid read of size 4 ==13209==at 0x12D5CC: PairDistances (pair_dist.c:346) ==13209==by 0x119410: AlignmentOrder (clustal-omega.c:835) ==13209==by 0x11A6C4: Align (clustal-omega.c:1221) ==13209==by 0x1171C8: MyMain (mymain.c:1192) ==13209==by 0x113CCC: main (main.cpp:469) ==13209== Address 0x8060 is not stack'd, malloc'd or (recently) free'd ==13209== ==13209== ==13209== Process terminating with default action of signal 10 (SIGBUS) ==13209==at 0x12D5CC: PairDistances (pair_dist.c:346) ==13209==by 0x119410: AlignmentOrder (clustal-omega.c:835) ==13209==by 0x11A6C4: Align (clustal-omega.c:1221) ==13209==by 0x1171C8: MyMain (mymain.c:1192) ==13209==by 0x113CCC: main (main.cpp:469) ==13209== ==13209== HEAP SUMMARY: ==13209== in use at exit: 8,039 bytes in 34 blocks ==13209== total heap usage: 112 allocs, 78 frees, 77,811 bytes allocated ==13209== ==13209== LEAK SUMMARY: ==13209==definitely lost: 128 bytes in 2 blocks ==13209==indirectly lost: 0 bytes in 0 blocks ==13209== possibly lost: 0 bytes in 0 blocks ==13209==still reachable: 7,911 bytes in 32 blocks ==13209== suppressed: 0 bytes in 0 blocks ==13209== Rerun with --leak-check=full to see details of leaked memory ==13209== ==13209== Use --track-origins=yes to see where uninitialised values come from ==13209== ERROR SUMMARY: 3 errors from 3 contexts (suppressed: 0 from 0) ==13209== ==13209== 1 errors in context 1 of 3: ==13209== Invalid read of size 4 ==13209==at 0x12D5CC: PairDistances (pair_dist.c:346) ==13
Bug#956324: Clustalo bus error on mipsel (Was: Bug#956324: python-biopython: FTBFS on mipsel)
> One small issue... Valgrind recommends -O0 or -O1 TIL :) Thanks, Jeff! > You can sometimes locate a bus error at build time with -Wcast-align. > At runtime you can usually locate them with -fsanitize=undefined. I had previously tried UBSan and, while it turned up a number of shifting and strict aliasing violations, it did not find anything that I believe could be the cause of the mipsel bus error. > I can't track the trail any further. The source code is missing for > pvm_pkdouble and pvm_upkdouble. I would be very suspect of > pvm_pkdouble and pvm_upkdouble. Unfortunately I think the reason you can’t find their source is that this code is unused. It is inside a ifdef SRE_ENABLE_PVM block and this macro is not defined during the build. You can confirm this by deleting these lines and recompiling. AFAICT the code in src/squid is a library taken wholesale from somewhere else and then modified (see src/squid/clustalo.README). It contains a lot of code that goes unused in Clustal Omega. Is the priority goal here to simply ship a non-crashing clustalo mipsel binary that BioPython can depend on? If so, maybe we can just disable compiler optimisation (-O0) and this may avoid provoking the bus error. Of course this doesn’t fix the underlying problem(s), but it looks as if debugging this to its root cause is going to result in the Debian package carrying a lot of invasive dquilt patches on top of upstream. OTOH I don’t know the performance requirements of BioPython and maybe an unoptimised clustalo is unacceptable to it.
Bug#956324: Clustalo bus error on mipsel (Was: Bug#956324: python-biopython: FTBFS on mipsel)
Hi Jeffrey,
thanks a lot for this analysis. Any chance that somebody could
turn this into a patch I could try?
Kind regards
Andreas.
On Thu, Apr 30, 2020 at 03:40:12PM -0400, Jeffrey Walton wrote:
> On Fri, Apr 17, 2020 at 7:21 AM Andreas Tille wrote:
> > ...
> > So it seems the bus error occures somehow here:
> >
> >
> > https://salsa.debian.org/med-team/clustalo/-/blob/master/src/clustal/pair_dist.c#L346
>
> NewProgress is at
> https://salsa.debian.org/med-team/clustalo/-/blob/master/src/clustal/progress.c#L53.
> It uses a progress_t at
> https://salsa.debian.org/med-team/clustalo/-/blob/master/src/clustal/progress.h#L25.
>
> typedef struct {
> /* where to write to */
> FILE *prFile;
> /* prefix printed before each step */
> char *pcPrefix;
> bool bPrintCR;
> char pcLastLogMsg[1024];
> Stopwatch_t *prStopwatch;
> } progress_t;
>
> And stopwatch_t at
> https://salsa.debian.org/med-team/clustalo/-/blob/master/src/squid/stopwatch.h:
>
> struct stopwatch_s {
> time_t t0; /* Wall clock time, ANSI time() */
> #ifdef SRE_STRICT_ANSI
> clock_t cpu0; /* CPU time, ANSI clock()*/
> #else
> struct tms cpu0; /* CPU/system time, POSIX times()*/
> #endif
>
> double elapsed; /* elapsed time, seconds */
> double user; /* CPU time, seconds */
> double sys; /* system time, seconds */
> };
>
> There are your doubles that are likely the problem.
>
> And what do we have here at
> https://salsa.debian.org/med-team/clustalo/-/blob/master/src/squid/stopwatch.c#L276
> ...
>
> void
> StopwatchPVMPack(Stopwatch_t *w)
> {
> pvm_pkdouble(&(w->elapsed), 1, 1);
> pvm_pkdouble(&(w->user),1, 1);
> pvm_pkdouble(&(w->sys), 1, 1);
> }
> void
> StopwatchPVMUnpack(Stopwatch_t *w)
> {
> pvm_upkdouble(&(w->elapsed), 1, 1);
> pvm_upkdouble(&(w->user),1, 1);
> pvm_upkdouble(&(w->sys), 1, 1);
> }
>
> I can't track the trail any further. The source code is missing for
> pvm_pkdouble and pvm_upkdouble. I would be very suspect of
> pvm_pkdouble and pvm_upkdouble.
>
> Jeff
>
--
http://fam-tille.de
Bug#956324: Clustalo bus error on mipsel (Was: Bug#956324: python-biopython: FTBFS on mipsel)
On Fri, Apr 17, 2020 at 7:21 AM Andreas Tille wrote:
> ...
> So it seems the bus error occures somehow here:
>
>
> https://salsa.debian.org/med-team/clustalo/-/blob/master/src/clustal/pair_dist.c#L346
NewProgress is at
https://salsa.debian.org/med-team/clustalo/-/blob/master/src/clustal/progress.c#L53.
It uses a progress_t at
https://salsa.debian.org/med-team/clustalo/-/blob/master/src/clustal/progress.h#L25.
typedef struct {
/* where to write to */
FILE *prFile;
/* prefix printed before each step */
char *pcPrefix;
bool bPrintCR;
char pcLastLogMsg[1024];
Stopwatch_t *prStopwatch;
} progress_t;
And stopwatch_t at
https://salsa.debian.org/med-team/clustalo/-/blob/master/src/squid/stopwatch.h:
struct stopwatch_s {
time_t t0; /* Wall clock time, ANSI time() */
#ifdef SRE_STRICT_ANSI
clock_t cpu0; /* CPU time, ANSI clock()*/
#else
struct tms cpu0; /* CPU/system time, POSIX times()*/
#endif
double elapsed; /* elapsed time, seconds */
double user; /* CPU time, seconds */
double sys; /* system time, seconds */
};
There are your doubles that are likely the problem.
And what do we have here at
https://salsa.debian.org/med-team/clustalo/-/blob/master/src/squid/stopwatch.c#L276
...
void
StopwatchPVMPack(Stopwatch_t *w)
{
pvm_pkdouble(&(w->elapsed), 1, 1);
pvm_pkdouble(&(w->user),1, 1);
pvm_pkdouble(&(w->sys), 1, 1);
}
void
StopwatchPVMUnpack(Stopwatch_t *w)
{
pvm_upkdouble(&(w->elapsed), 1, 1);
pvm_upkdouble(&(w->user),1, 1);
pvm_upkdouble(&(w->sys), 1, 1);
}
I can't track the trail any further. The source code is missing for
pvm_pkdouble and pvm_upkdouble. I would be very suspect of
pvm_pkdouble and pvm_upkdouble.
Jeff
Bug#956324: Clustalo bus error on mipsel (Was: Bug#956324: python-biopython: FTBFS on mipsel)
On Fri, Apr 17, 2020 at 7:21 AM Andreas Tille wrote: > > Control: tags -1 help > > as it can be seen on the recent build log of clustalo on mips[1] the > build fails with > > # Run additional test from python-biopython package to verify that > # this will work as well > src/clustalo -i debian/tests/biopython_testdata/f002 --guidetree-out > temp_test.dnd -o temp_test.aln --outfmt clustal --force > make[1]: *** [debian/rules:25: override_dh_auto_test-arch] Bus error You can sometimes locate a bus error at build time with -Wcast-align. At runtime you can usually locate them with -fsanitize=undefined. On x86 -Wcast-align usually triggers when casting to/from floats and doubles. I don't know what triggers on MIPS, bit it may include integers. You should look for all calls that perform casts to/from void* and uint8_t* with a dereference. I.e., uint8_t y[8] = ...; double x = *(double*)y; If MIPS is sensitive to alignment (beyond x86 slop), then something like this will get you into trouble, too: uint8_t y[8] = ...; unsigned long x = *(unsigned long*)y; I don't know about MIPS, but... I've experienced the problem with long's on SPARCv8. SPARCv8 needs 8-byte alignments because they have a specialized 64-bit move instruction. Or SPARCv8 needs an extra compile/link switch that disables the 64-bit move. The option is -xmemalign, which tells the toolchain what alignments can be assumed. Jeff
Bug#956324: Clustalo bus error on mipsel (Was: Bug#956324: python-biopython: FTBFS on mipsel)
On Thu, Apr 30, 2020 at 10:33 AM Matthew Fernandez wrote: > > > On Apr 30, 2020, at 00:31, Andreas Tille wrote: > > > > On Wed, Apr 29, 2020 at 05:51:26PM -0700, Matthew Fernandez wrote: > > > >> The other option I suggested was Valgrind, but if you can’t run apt-file > >> you probably can’t install Valgrind either. > > > > Well, I guess apt-get is permitted for sudo but not apt-file. So I can > > probably install valgrind inside the chroot environment. I've never > > worked with valgrind. What am I supposed to do? > > Valgrind, in its default mode, checks for a variety of memory issues > (use-after-free, write out-of bounds, …). You don’t need any special > configure/build options, but you probably want to enable debug symbols > (`export CFLAGS=-g; export CXXFLAGS=-g`). Then you can prefix the test you’re > running with Valgrind: `valgrind ./src/clustalo -i > debian/tests/biopython_testdata/f002 …`. One small issue... Valgrind recommends -O0 or -O1: Compile your program with -g to include debugging information so that Memcheck's error messages include exact line numbers. Using -O0 is also a good idea, if you can tolerate the slowdown. With -O1 line numbers in error messages can be inaccurate, although generally speaking running Memcheck on code compiled at -O1 works fairly well, and the speed improvement compared to running -O0 is quite significant. Use of -O2 and above is not recommended as Memcheck occasionally reports uninitialised-value errors which don't really exist. Also see the Valgrind Quick Start Guide, Section 2. Preparing Your Programs, at http://valgrind.org/docs/manual/QuickStart.html. Jeff
Bug#956324: Clustalo bus error on mipsel (Was: Bug#956324: python-biopython: FTBFS on mipsel)
On Thu, Apr 30, 2020 at 07:17:50AM -0700, Matthew Fernandez wrote: > > Valgrind, in its default mode, checks for a variety of memory issues > (use-after-free, write out-of bounds, …). You don’t need any special > configure/build options, but you probably want to enable debug symbols > (`export CFLAGS=-g; export CXXFLAGS=-g`). Then you can prefix the test you’re > running with Valgrind: `valgrind ./src/clustalo -i > debian/tests/biopython_testdata/f002 …`. Running on mipsel: (sid_mipsel-dchroot)tille@eller:~/clustalo$ valgrind src/clustalo -i debian/tests/biopython_testdata/f002 --guidetree-out temp_test.dnd -o temp_test.aln --outfmt clustal --force ==15149== Memcheck, a memory error detector ==15149== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al. ==15149== Using Valgrind-3.15.0 and LibVEX; rerun with -h for copyright info ==15149== Command: src/clustalo -i debian/tests/biopython_testdata/f002 --guidetree-out temp_test.dnd -o temp_test.aln --outfmt clustal --force ==15149== ==15149== Conditional jump or move depends on uninitialised value(s) ==15149==at 0x4007828: cached_fpabi_reject_phdr_p (dl-machine-reject-phdr.h:57) ==15149==by 0x4007828: elf_machine_reject_phdr_p (dl-machine-reject-phdr.h:217) ==15149==by 0x4008080: open_verify.constprop.0 (dl-load.c:1688) ==15149==by 0x4009D7C: _dl_map_object (dl-load.c:2181) ==15149==by 0x40011F8: map_doit (rtld.c:607) ==15149==by 0x401B2A8: _dl_catch_exception (dl-error-skeleton.c:196) ==15149==by 0x401B334: _dl_catch_error (dl-error-skeleton.c:215) ==15149==by 0x4001138: do_preload (rtld.c:778) ==15149==by 0x4002560: handle_preload_list (rtld.c:879) ==15149==by 0x4005B08: dl_main (rtld.c:1684) ==15149==by 0x401A094: _dl_sysdep_start (dl-sysdep.c:253) ==15149==by 0x400199C: _dl_start_final (rtld.c:447) ==15149==by 0x4001D44: _dl_start (rtld.c:539) ==15149== ==15149== Invalid read of size 1 ==15149==at 0x486F558: ??? (in /usr/lib/mipsel-linux-gnu/libargtable2.so.0.1.8) ==15149==by 0x486F5F0: ??? (in /usr/lib/mipsel-linux-gnu/libargtable2.so.0.1.8) ==15149== Address 0x4d655c5 is 0 bytes after a block of size 37 alloc'd ==15149==at 0x484B89C: malloc (in /usr/lib/mipsel-linux-gnu/valgrind/vgpreload_memcheck-mips32-linux.so) ==15149==by 0x126674: CkMalloc (util.c:83) ==15149==by 0x126864: CkStrdup (util.c:213) ==15149==by 0x1108F4: ConvertOldCmdline(int*, char***, int, char**) (main.cpp:358) ==15149==by 0x10FCDC: main (main.cpp:467) ==15149== ==15149== Invalid read of size 4 ==15149==at 0x1221B8: PairDistances (pair_dist.c:346) ==15149==by 0x114774: AlignmentOrder (clustal-omega.c:835) ==15149==by 0x115024: Align (clustal-omega.c:1221) ==15149==by 0x112EB4: MyMain (mymain.c:1192) ==15149==by 0x10FCF0: main (main.cpp:469) ==15149== Address 0x83b0 is not stack'd, malloc'd or (recently) free'd ==15149== ==15149== ==15149== Process terminating with default action of signal 10 (SIGBUS) ==15149==at 0x1221B8: PairDistances (pair_dist.c:346) ==15149==by 0x114774: AlignmentOrder (clustal-omega.c:835) ==15149==by 0x115024: Align (clustal-omega.c:1221) ==15149==by 0x112EB4: MyMain (mymain.c:1192) ==15149==by 0x10FCF0: main (main.cpp:469) ==15149== ==15149== HEAP SUMMARY: ==15149== in use at exit: 8,039 bytes in 34 blocks ==15149== total heap usage: 112 allocs, 78 frees, 77,811 bytes allocated ==15149== ==15149== LEAK SUMMARY: ==15149==definitely lost: 128 bytes in 2 blocks ==15149==indirectly lost: 0 bytes in 0 blocks ==15149== possibly lost: 0 bytes in 0 blocks ==15149==still reachable: 7,911 bytes in 32 blocks ==15149== suppressed: 0 bytes in 0 blocks ==15149== Rerun with --leak-check=full to see details of leaked memory ==15149== ==15149== Use --track-origins=yes to see where uninitialised values come from ==15149== For lists of detected and suppressed errors, rerun with: -s ==15149== ERROR SUMMARY: 3 errors from 3 contexts (suppressed: 0 from 0) Bus error Is this different from your tests on amd64? > > On the other hand: Is valgrind possibly able to uncover issues also > > on any other architecture? > > You mean if we were to use Valgrind to debug this on e.g. x86? In my own > experiments on amd64, both ASan and Valgrind found multiple issues in both > Clustal Omega and its dependency, argtable2. However I don’t believe any of > the remaining ones I was seeing after the last patches I sent you could be > causing the mipsel bus error; they were all memory leaks. Thanks again for your great support and the time you've spent. Kind regards Andreas. -- http://fam-tille.de
Bug#956324: Clustalo bus error on mipsel (Was: Bug#956324: python-biopython: FTBFS on mipsel)
> On Apr 30, 2020, at 00:31, Andreas Tille wrote: > > On Wed, Apr 29, 2020 at 05:51:26PM -0700, Matthew Fernandez wrote: > >> The other option I suggested was Valgrind, but if you can’t run apt-file you >> probably can’t install Valgrind either. > > Well, I guess apt-get is permitted for sudo but not apt-file. So I can > probably install valgrind inside the chroot environment. I've never > worked with valgrind. What am I supposed to do? Valgrind, in its default mode, checks for a variety of memory issues (use-after-free, write out-of bounds, …). You don’t need any special configure/build options, but you probably want to enable debug symbols (`export CFLAGS=-g; export CXXFLAGS=-g`). Then you can prefix the test you’re running with Valgrind: `valgrind ./src/clustalo -i debian/tests/biopython_testdata/f002 …`. Valgrind and ASan serve roughly the same purpose in this scenario, but I usually tend to prefer ASan because it is more efficient and tends to give you more accurate debugging clues. > On the other hand: Is valgrind possibly able to uncover issues also > on any other architecture? You mean if we were to use Valgrind to debug this on e.g. x86? In my own experiments on amd64, both ASan and Valgrind found multiple issues in both Clustal Omega and its dependency, argtable2. However I don’t believe any of the remaining ones I was seeing after the last patches I sent you could be causing the mipsel bus error; they were all memory leaks.
Bug#956324: Clustalo bus error on mipsel (Was: Bug#956324: python-biopython: FTBFS on mipsel)
Hi Matthew, On Wed, Apr 29, 2020 at 05:51:26PM -0700, Matthew Fernandez wrote: > > Any more help from debian-mipsel is really appreciated. > > Hm yes, “--disable-libsanitizer” is rather ominous. I guess the mipsel GCC > package has been built without ASan support. Surprising that it fails so > messily (the front end seems to think -fsanitize=address is an accepted > command line option), but libasan does indeed seem not available on mipsel > [0]. OK, so it seems this method to track down the issue on mips does not work. > The other option I suggested was Valgrind, but if you can’t run apt-file you > probably can’t install Valgrind either. Well, I guess apt-get is permitted for sudo but not apt-file. So I can probably install valgrind inside the chroot environment. I've never worked with valgrind. What am I supposed to do? On the other hand: Is valgrind possibly able to uncover issues also on any other architecture? > If anyone spectating has ideas, please chime in. Definitely. We obviously could need some help. Kind regards Andreas. -- http://fam-tille.de
Bug#956324: Clustalo bus error on mipsel (Was: Bug#956324: python-biopython: FTBFS on mipsel)
> On Apr 29, 2020, at 09:04, Andreas Tille wrote: > > On Wed, Apr 29, 2020 at 07:14:30AM -0700, Matthew Fernandez wrote: > >> For those on this thread who have access to mipsel hardware or can shell in >> to one of the mipsel build machines, I would suggest running an >> ASan-instrumented test there (`export CFLAGS="-g -fsanitize=address"; export >> CXXFLAGS="-g -fsanitize=address"`) and see what we learn. > > I tried on real hardware. Unfortunately I'm running into > > > > configure:3720: $? = 0 > configure:3709: gcc -v >&5 > Using built-in specs. > COLLECT_GCC=gcc > COLLECT_LTO_WRAPPER=/usr/lib/gcc/mipsel-linux-gnu/9/lto-wrapper > Target: mipsel-linux-gnu > Configured with: ../src/configure -v --with-pkgversion='Debian 9.3.0-10' > --with-bugurl=file:///usr/share/doc/gcc-9/README.Bugs > --enable-languages=c,ada,c++,go,d,fortran,objc,obj-c++,gm2 --prefix=/usr > --with-gcc-major-version-only --program-suffix=-9 > --program-prefix=mipsel-linux-gnu- --enable-shared --enable-linker-build-id > --libexecdir=/usr/lib --without-included-gettext --enable-threads=posix > --libdir=/usr/lib --enable-nls --enable-clocale=gnu --enable-libstdcxx-debug > --enable-libstdcxx-time=yes --with-default-libstdcxx-abi=new > --enable-gnu-unique-object --disable-libitm --disable-libsanitizer > --disable-libquadmath --disable-libquadmath-support --enable-plugin > --enable-default-pie --with-system-zlib --with-target-system-zlib=auto > --enable-multiarch --disable-werror --enable-multilib --with-arch-32=mips32r2 > --with-fp-32=xx --with-madd4=no --with-lxc1-sxc1=no --enable-targets=all > --with-arch-64=mips64r2 --enable-checking=release --build=mipsel-linux-gnu > --host=mipsel-linux-gnu --target=mipsel-linux-gnu > Thread model: posix > gcc version 9.3.0 (Debian 9.3.0-10) > configure:3720: $? = 0 > configure:3709: gcc -V >&5 > gcc: error: unrecognized command line option '-V' > gcc: fatal error: no input files > compilation terminated. > configure:3720: $? = 1 > configure:3709: gcc -qversion >&5 > gcc: error: unrecognized command line option '-qversion'; did you mean > '--version'? > gcc: fatal error: no input files > compilation terminated. > configure:3720: $? = 1 > configure:3740: checking whether the C compiler works > configure:3762: gcc -g -O2 -fdebug-prefix-map=/home/tille/clustalo=. > -fstack-protector-strong -Wformat -Werror=format-security -fsanitize=address > -Wdate-time -D_FORTIFY_SOURCE=2 -Wl,-z,relro -Wl,-z,now conftest.c >&5 > /usr/bin/ld: cannot find libasan_preinit.o: No such file or directory > /usr/bin/ld: cannot find -lasan > collect2: error: ld returned 1 exit status > configure:3766: $? = 1 > configure:3804: result: no > configure: failed program was: > > > > I have no idea why libasan_preinit.o and libasan.a are not installed. > The package libgcc-9-dev is installed for sure and on amd64 both files > are included here. It seems the sudo command on eller does not permit > me executing `apt-file update` in a chroot so I have no idea where these > files are on mipsel (if they exist at all). > > Any more help from debian-mipsel is really appreciated. Hm yes, “--disable-libsanitizer” is rather ominous. I guess the mipsel GCC package has been built without ASan support. Surprising that it fails so messily (the front end seems to think -fsanitize=address is an accepted command line option), but libasan does indeed seem not available on mipsel [0]. The other option I suggested was Valgrind, but if you can’t run apt-file you probably can’t install Valgrind either. If anyone spectating has ideas, please chime in. [0]: https://packages.debian.org/search?searchon=contents&keywords=libasan.so&mode=exactfilename&suite=sid&arch=mipsel
Bug#956324: Clustalo bus error on mipsel (Was: Bug#956324: python-biopython: FTBFS on mipsel)
Hi Matthew, On Wed, Apr 29, 2020 at 07:14:30AM -0700, Matthew Fernandez wrote: > > To add another data point to this discussion, one other (fruitless) thing I > tried previously was cross-compiling Clustal Omega. From an amd64 host, it’s > possible to target mipsel using the GCC cross-compilers in the standard > Debian repositories. You can then run the resulting binary using Qemu’s user > mode. Using this technique, the f002 test runs to completion with no bus > error. This is not really surprising as AFAIK unaligned accesses that would > trigger a bus error on mipsel hardware would be silently allowed in this > configuration (Qemu doesn’t faithfully emulate this hardware behaviour and > amd64 allows unaligned access). > > Unfortunately the repositories’ cross-compilers have been built without ASan > enabled and you can’t attach to an emulated mipsel process with a native > Valgrind. So debugging memory safety issues is not straightforward. To go > further with this approach, you would have to build a mipsel-targeting > cross-compiler with ASan enabled or cross-compile Valgrind to mipsel. For a > true masochist, it may be possible to attach to the process with GDB or rr > and reverse-step from the location Andreas has quoted, but I wouldn’t trust > the debugger not to crash in this configuration. Even then the issue may not > be reproducible because it may be dependent on transformations/optimizations > only performed by the particular version of the native mipsel compiler called > during packaging. Thanks a lot for your effort. > For those on this thread who have access to mipsel hardware or can shell in > to one of the mipsel build machines, I would suggest running an > ASan-instrumented test there (`export CFLAGS="-g -fsanitize=address"; export > CXXFLAGS="-g -fsanitize=address"`) and see what we learn. I tried on real hardware. Unfortunately I'm running into configure:3720: $? = 0 configure:3709: gcc -v >&5 Using built-in specs. COLLECT_GCC=gcc COLLECT_LTO_WRAPPER=/usr/lib/gcc/mipsel-linux-gnu/9/lto-wrapper Target: mipsel-linux-gnu Configured with: ../src/configure -v --with-pkgversion='Debian 9.3.0-10' --with-bugurl=file:///usr/share/doc/gcc-9/README.Bugs --enable-languages=c,ada,c++,go,d,fortran,objc,obj-c++,gm2 --prefix=/usr --with-gcc-major-version-only --program-suffix=-9 --program-prefix=mipsel-linux-gnu- --enable-shared --enable-linker-build-id --libexecdir=/usr/lib --without-included-gettext --enable-threads=posix --libdir=/usr/lib --enable-nls --enable-clocale=gnu --enable-libstdcxx-debug --enable-libstdcxx-time=yes --with-default-libstdcxx-abi=new --enable-gnu-unique-object --disable-libitm --disable-libsanitizer --disable-libquadmath --disable-libquadmath-support --enable-plugin --enable-default-pie --with-system-zlib --with-target-system-zlib=auto --enable-multiarch --disable-werror --enable-multilib --with-arch-32=mips32r2 --with-fp-32=xx --with-madd4=no --with-lxc1-sxc1=no --enable-targets=all --with-arch-64=mips64r2 --enable-checking=release --build=mipsel-linux-gnu --host=mipsel-linux-gnu --target=mipsel-linux-gnu Thread model: posix gcc version 9.3.0 (Debian 9.3.0-10) configure:3720: $? = 0 configure:3709: gcc -V >&5 gcc: error: unrecognized command line option '-V' gcc: fatal error: no input files compilation terminated. configure:3720: $? = 1 configure:3709: gcc -qversion >&5 gcc: error: unrecognized command line option '-qversion'; did you mean '--version'? gcc: fatal error: no input files compilation terminated. configure:3720: $? = 1 configure:3740: checking whether the C compiler works configure:3762: gcc -g -O2 -fdebug-prefix-map=/home/tille/clustalo=. -fstack-protector-strong -Wformat -Werror=format-security -fsanitize=address -Wdate-time -D_FORTIFY_SOURCE=2 -Wl,-z,relro -Wl,-z,now conftest.c >&5 /usr/bin/ld: cannot find libasan_preinit.o: No such file or directory /usr/bin/ld: cannot find -lasan collect2: error: ld returned 1 exit status configure:3766: $? = 1 configure:3804: result: no configure: failed program was: I have no idea why libasan_preinit.o and libasan.a are not installed. The package libgcc-9-dev is installed for sure and on amd64 both files are included here. It seems the sudo command on eller does not permit me executing `apt-file update` in a chroot so I have no idea where these files are on mipsel (if they exist at all). Any more help from debian-mipsel is really appreciated. Kind regards Andreas. -- http://fam-tille.de
Bug#956324: Clustalo bus error on mipsel (Was: Bug#956324: python-biopython: FTBFS on mipsel)
> On Apr 29, 2020, at 02:12, Andreas Tille wrote: > > Hi, > > On Wed, Apr 29, 2020 at 10:30:35AM +0800, 黄佳文 wrote: >> I am a developer from Loongson company (R & D CPU/mip64el), I've been >> looking at this recently. > > Very nice to see mips developers to care for biological software. :-) > >> I did two experiments, and I found that when I used Python 3,7 to compile >> python-biopython, Build successfully. >> In the same environment, I just upgrade Python 3.7 to Python 3.8, and then >> compile python-biopytho, Build fails, but not bus error. >> I found through online query that some symbol tables were added and deleted >> in upgrading Python 3.7 to 3.8. The following are symbol tables: > > Sorry to insist here - I do not think that it is a Python version problem > at all. The issue can be reproduced in clustalo only which is pure C code. > May be you have a look at > > https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=956324#59 > > and the following discussion. Despite Matthew has found some issues inside > the C code it did not helped to prevent: > > > Starting program: /home/tille/clustalo/src/clustalo -i > debian/tests/biopython_testdata/f002 --guidetree-out temp_test.dnd -o > temp_test.aln --outfmt clustal --force > [Thread debugging using libthread_db enabled] > Using host libthread_db library "/lib/mipsel-linux-gnu/libthread_db.so.1". > > Program received signal SIGBUS, Bus error. > 0x5556a1b8 in PairDistances (distmat=0x7fff278c, mseq=0x55692a30, > pairdist_type=, bPercID=, istart=0, iend=3, > jstart=0, jend=3, fdist_in=0x0, >fdist_out=0x0) at pair_dist.c:346 > 346 NewProgress(&prProgress, LogGetFP(&rLog, LOG_INFO), > > > That's the issue we need to care about here. To add another data point to this discussion, one other (fruitless) thing I tried previously was cross-compiling Clustal Omega. From an amd64 host, it’s possible to target mipsel using the GCC cross-compilers in the standard Debian repositories. You can then run the resulting binary using Qemu’s user mode. Using this technique, the f002 test runs to completion with no bus error. This is not really surprising as AFAIK unaligned accesses that would trigger a bus error on mipsel hardware would be silently allowed in this configuration (Qemu doesn’t faithfully emulate this hardware behaviour and amd64 allows unaligned access). Unfortunately the repositories’ cross-compilers have been built without ASan enabled and you can’t attach to an emulated mipsel process with a native Valgrind. So debugging memory safety issues is not straightforward. To go further with this approach, you would have to build a mipsel-targeting cross-compiler with ASan enabled or cross-compile Valgrind to mipsel. For a true masochist, it may be possible to attach to the process with GDB or rr and reverse-step from the location Andreas has quoted, but I wouldn’t trust the debugger not to crash in this configuration. Even then the issue may not be reproducible because it may be dependent on transformations/optimizations only performed by the particular version of the native mipsel compiler called during packaging. For those on this thread who have access to mipsel hardware or can shell in to one of the mipsel build machines, I would suggest running an ASan-instrumented test there (`export CFLAGS="-g -fsanitize=address"; export CXXFLAGS="-g -fsanitize=address"`) and see what we learn.
Bug#956324: Clustalo bus error on mipsel (Was: Bug#956324: python-biopython: FTBFS on mipsel)
Hi, On Wed, Apr 29, 2020 at 10:30:35AM +0800, 黄佳文 wrote: > I am a developer from Loongson company (R & D CPU/mip64el), I've been > looking at this recently. Very nice to see mips developers to care for biological software. :-) > I did two experiments, and I found that when I used Python 3,7 to compile > python-biopython, Build successfully. > In the same environment, I just upgrade Python 3.7 to Python 3.8, and then > compile python-biopytho, Build fails, but not bus error. > I found through online query that some symbol tables were added and deleted > in upgrading Python 3.7 to 3.8. The following are symbol tables: Sorry to insist here - I do not think that it is a Python version problem at all. The issue can be reproduced in clustalo only which is pure C code. May be you have a look at https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=956324#59 and the following discussion. Despite Matthew has found some issues inside the C code it did not helped to prevent: Starting program: /home/tille/clustalo/src/clustalo -i debian/tests/biopython_testdata/f002 --guidetree-out temp_test.dnd -o temp_test.aln --outfmt clustal --force [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib/mipsel-linux-gnu/libthread_db.so.1". Program received signal SIGBUS, Bus error. 0x5556a1b8 in PairDistances (distmat=0x7fff278c, mseq=0x55692a30, pairdist_type=, bPercID=, istart=0, iend=3, jstart=0, jend=3, fdist_in=0x0, fdist_out=0x0) at pair_dist.c:346 346 NewProgress(&prProgress, LogGetFP(&rLog, LOG_INFO), That's the issue we need to care about here. Kind regards and thanks a lot for the attempt to help Andreas.
Bug#956324: [Debian-med-packaging] Bug#956324: Clustalo bus error on mipsel (Was: Bug#956324: python-biopython: FTBFS on mipsel)
Control: affects -1 src:python-biopython On Tue, 14 Apr 2020 at 17:39, Andreas Tille wrote: > Control: reassign -1 clustalo > Control: retitle -1 "clustalo: Bus error on mipsel" > Control: tags -1 upstream > Control: forwarded -1 clust...@ucd.ie Marking this bug as affecting src:python-biopython, so it will show up here again: https://buildd.debian.org/status/package.php?p=python-biopython&suite=unstable
Bug#956324: Clustalo bus error on mipsel (Was: Bug#956324: python-biopython: FTBFS on mipsel)
Hi Matthew, many thanks again for your investigation. On Sat, Apr 18, 2020 at 01:15:49PM -0700, Matthew Fernandez wrote: > > Upstream is in the row of this investigation. Its quite interesting > > that the issue could also observed on amd64. So probably this is a real > > issue which is just exposed on mipsel. I think just bumping the array > > boundaries is cheap. If there will be no response from upstream (or > > somebody else who is comfortable with the algorithm which I'm not) I'll > > check again on mipsel and in case it will work I'll upload. > > Fair enough. While we’re discussing this, here’s another patch for a memory > leak that fixes a typoed ifdef and a missing free(): > > diff --git a/src/squid/clustal.c b/src/squid/clustal.c > index 650004a..bb1fec8 100644 > --- a/src/squid/clustal.c > +++ b/src/squid/clustal.c > @@ -207,7 +207,7 @@ WriteClustal(FILE *fp, MSA *msa) >intnamelen; /* maximum name length used */ >intpos; /* position counter */ > #ifdef CLUSTALO > - char *buf; /* buffer for writing seq*/ > + char *buf = NULL; /* buffer for writing seq > */ >intcpl = msa->alenalen+10 : (iWrap > 0 ? iWrap : > 60); /* char per line (< 64) */ > #else >char buf[80]; /* buffer for writing seq*/ > @@ -410,8 +410,9 @@ WriteClustal(FILE *fp, MSA *msa) > #endif > } > > -#ifdef CLUSTAL > +#ifdef CLUSTALO >free(piResCnt); piResCnt = NULL; > + free(buf); buf = NULL; > #endif > >return; I tried the suggested patches again step by step on mipsel but the bus error issue remains. :-( Kind regards Andreas. -- http://fam-tille.de
Bug#956324: Clustalo bus error on mipsel (Was: Bug#956324: python-biopython: FTBFS on mipsel)
> On Apr 17, 2020, at 22:39, Andreas Tille wrote:
>
> Hi Matthew,
>
> thanks a lot for your detailed investigation.
>
> On Fri, Apr 17, 2020 at 04:28:23PM -0700, Matthew Fernandez wrote:
>>> Program received signal SIGBUS, Bus error.
>>> 0x5556a1b8 in PairDistances (distmat=0x7fff278c, mseq=0x55692a30,
>>> pairdist_type=, bPercID=, istart=0, iend=3,
>>> jstart=0, jend=3, fdist_in=0x0,
>>> fdist_out=0x0) at pair_dist.c:346
>>> 346 NewProgress(&prProgress, LogGetFP(&rLog, LOG_INFO),
>>
>> OK, let me try a little harder :)
>>
>>$ # enable debugging symbols and Address Sanitizer
>>$ CFLAGS="-g -fsanitize=address" CXXFLAGS="-g -fsanitize=address"
>> ./configure
>>…
>>$ make clean && make
>>…
>>$ ./src/clustalo -i debian/tests/biopython_testdata/f002 --guidetree-out
>> temp_test.dnd -o temp_test.aln --outfmt clustal --force
>>=
>>==30264==ERROR: AddressSanitizer: dynamic-stack-buffer-overflow on
>> address 0x7ffcfcbf5784 at pc 0x5620f0aa478c bp 0x7ffcfcbf56c0 sp
>> 0x7ffcfcbf56b8
>>WRITE of size 4 at 0x7ffcfcbf5784 thread T0
>>#0 0x5620f0aa478b in PairDistances
>> /home/matthew/clustal-omega-1.2.4/src/clustal/pair_dist.c:336
>>#1 0x5620f0a91d9f in AlignmentOrder
>> /home/matthew/clustal-omega-1.2.4/src/clustal-omega.c:835
>>#2 0x5620f0a95c04 in Align
>> /home/matthew/clustal-omega-1.2.4/src/clustal-omega.c:1221
>>#3 0x5620f0a90d76 in MyMain
>> /home/matthew/clustal-omega-1.2.4/src/mymain.c:1192
>>#4 0x5620f0a88ca2 in main
>> /home/matthew/clustal-omega-1.2.4/src/main.cpp:469
>>#5 0x7f3773d9009a in __libc_start_main ../csu/libc-start.c:308
>>#6 0x5620f0a89ad9 in _start
>> (/home/matthew/clustal-omega-1.2.4/src/clustalo+0x2dad9)
>>
>>Address 0x7ffcfcbf5784 is located in stack of thread T0
>>SUMMARY: AddressSanitizer: dynamic-stack-buffer-overflow
>> /home/matthew/clustal-omega-1.2.4/src/clustal/pair_dist.c:336 in
>> PairDistances
>>Shadow bytes around the buggy address:
>> 0x10001f976aa0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
>> 0x10001f976ab0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
>> 0x10001f976ac0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
>> 0x10001f976ad0: 00 00 00 00 00 00 00 00 00 00 00 00 ca ca ca ca
>> 0x10001f976ae0: 04 cb cb cb cb cb cb cb 00 00 00 00 ca ca ca ca
>>=>0x10001f976af0:[04]cb cb cb cb cb cb cb 00 00 00 00 00 00 00 00
>> 0x10001f976b00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
>> 0x10001f976b10: f1 f1 f1 f1 00 f2 f2 f2 f2 f2 f2 f2 00 f2 f2 f2
>> 0x10001f976b20: f2 f2 f2 f2 00 00 00 00 00 00 00 00 00 f2 f2 f2
>> 0x10001f976b30: f3 f3 f3 f3 00 00 00 00 00 00 00 00 00 00 00 00
>> 0x10001f976b40: 00 00 00 00 00 00 f1 f1 f1 f1 00 f2 f2 f2 f2 f2
>>Shadow byte legend (one shadow byte represents 8 application bytes):
>> Addressable: 00
>> Partially addressable: 01 02 03 04 05 06 07
>> Heap left redzone: fa
>> Freed heap region: fd
>> Stack left redzone: f1
>> Stack mid redzone: f2
>> Stack right redzone: f3
>> Stack after return: f5
>> Stack use after scope: f8
>> Global redzone: f9
>> Global init order: f6
>> Poisoned by user:f7
>> Container overflow: fc
>> Array cookie:ac
>> Intra object redzone:bb
>> ASan internal: fe
>> Left alloca redzone: ca
>> Right alloca redzone:cb
>>==30264==ABORTING
>>
>> Looking at line 336 of pair_dist.c, it looks like the bound on the
>> containing loop is wrong. So let’s try adjusting that:
>>
>>$ vim src/clustal/pair_dist.c
>>$ git diff src/clustal/pair_dist.c
>>diff --git a/src/clustal/pair_dist.c b/src/clustal/pair_dist.c
>>index e6dbdc3..bb79e61 100644
>>--- a/src/clustal/pair_dist.c
>>+++ b/src/clustal/pair_dist.c
>>@@ -321,7 +321,7 @@ PairDistances(symmatrix_t **distmat, mseq_t *mseq,
>> int pairdist_type, bool bPerc
>>
>> /* FIXME: can get rid of iChunkStart, iChunkEnd now that we're
>> using the arrays */
>> iChunkStart = iend;
>>-for(iChunk = 0; iChunk <= iNumberOfThreads; iChunk++)
>>+for(iChunk = 0; iChunk < iNumberOfThreads; iChunk++)
>> {
>> iChunkEnd = iChunkStart;
>> if (iChunk == iNumberOfThreads - 1){
>>$ make
>>…
>>$ ./src/clustalo -i debian/tests/biopython_testdata/f002 --guidetree-out
>> temp_test.dnd -o temp_test.aln --outfmt clustal --force
>>=
>>==30601==ERROR: AddressSanitizer: global-buffer-overflow on address
>> 0x561188847864 at pc 0x5611886da6e7 bp 0x7fffe6d77ef0 sp 0x7fffe6d77ee8
>>READ of size 4 at 0x561188847864 thread T0
>>#0 0
Bug#956324: Clustalo bus error on mipsel (Was: Bug#956324: python-biopython: FTBFS on mipsel)
Hi Matthew,
thanks a lot for your detailed investigation.
On Fri, Apr 17, 2020 at 04:28:23PM -0700, Matthew Fernandez wrote:
> > Program received signal SIGBUS, Bus error.
> > 0x5556a1b8 in PairDistances (distmat=0x7fff278c, mseq=0x55692a30,
> > pairdist_type=, bPercID=, istart=0, iend=3,
> > jstart=0, jend=3, fdist_in=0x0,
> >fdist_out=0x0) at pair_dist.c:346
> > 346 NewProgress(&prProgress, LogGetFP(&rLog, LOG_INFO),
>
> OK, let me try a little harder :)
>
> $ # enable debugging symbols and Address Sanitizer
> $ CFLAGS="-g -fsanitize=address" CXXFLAGS="-g -fsanitize=address"
> ./configure
> …
> $ make clean && make
> …
> $ ./src/clustalo -i debian/tests/biopython_testdata/f002 --guidetree-out
> temp_test.dnd -o temp_test.aln --outfmt clustal --force
> =
> ==30264==ERROR: AddressSanitizer: dynamic-stack-buffer-overflow on
> address 0x7ffcfcbf5784 at pc 0x5620f0aa478c bp 0x7ffcfcbf56c0 sp
> 0x7ffcfcbf56b8
> WRITE of size 4 at 0x7ffcfcbf5784 thread T0
> #0 0x5620f0aa478b in PairDistances
> /home/matthew/clustal-omega-1.2.4/src/clustal/pair_dist.c:336
> #1 0x5620f0a91d9f in AlignmentOrder
> /home/matthew/clustal-omega-1.2.4/src/clustal-omega.c:835
> #2 0x5620f0a95c04 in Align
> /home/matthew/clustal-omega-1.2.4/src/clustal-omega.c:1221
> #3 0x5620f0a90d76 in MyMain
> /home/matthew/clustal-omega-1.2.4/src/mymain.c:1192
> #4 0x5620f0a88ca2 in main
> /home/matthew/clustal-omega-1.2.4/src/main.cpp:469
> #5 0x7f3773d9009a in __libc_start_main ../csu/libc-start.c:308
> #6 0x5620f0a89ad9 in _start
> (/home/matthew/clustal-omega-1.2.4/src/clustalo+0x2dad9)
>
> Address 0x7ffcfcbf5784 is located in stack of thread T0
> SUMMARY: AddressSanitizer: dynamic-stack-buffer-overflow
> /home/matthew/clustal-omega-1.2.4/src/clustal/pair_dist.c:336 in PairDistances
> Shadow bytes around the buggy address:
> 0x10001f976aa0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> 0x10001f976ab0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> 0x10001f976ac0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> 0x10001f976ad0: 00 00 00 00 00 00 00 00 00 00 00 00 ca ca ca ca
> 0x10001f976ae0: 04 cb cb cb cb cb cb cb 00 00 00 00 ca ca ca ca
> =>0x10001f976af0:[04]cb cb cb cb cb cb cb 00 00 00 00 00 00 00 00
> 0x10001f976b00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> 0x10001f976b10: f1 f1 f1 f1 00 f2 f2 f2 f2 f2 f2 f2 00 f2 f2 f2
> 0x10001f976b20: f2 f2 f2 f2 00 00 00 00 00 00 00 00 00 f2 f2 f2
> 0x10001f976b30: f3 f3 f3 f3 00 00 00 00 00 00 00 00 00 00 00 00
> 0x10001f976b40: 00 00 00 00 00 00 f1 f1 f1 f1 00 f2 f2 f2 f2 f2
> Shadow byte legend (one shadow byte represents 8 application bytes):
> Addressable: 00
> Partially addressable: 01 02 03 04 05 06 07
> Heap left redzone: fa
> Freed heap region: fd
> Stack left redzone: f1
> Stack mid redzone: f2
> Stack right redzone: f3
> Stack after return: f5
> Stack use after scope: f8
> Global redzone: f9
> Global init order: f6
> Poisoned by user:f7
> Container overflow: fc
> Array cookie:ac
> Intra object redzone:bb
> ASan internal: fe
> Left alloca redzone: ca
> Right alloca redzone:cb
> ==30264==ABORTING
>
> Looking at line 336 of pair_dist.c, it looks like the bound on the containing
> loop is wrong. So let’s try adjusting that:
>
> $ vim src/clustal/pair_dist.c
> $ git diff src/clustal/pair_dist.c
> diff --git a/src/clustal/pair_dist.c b/src/clustal/pair_dist.c
> index e6dbdc3..bb79e61 100644
> --- a/src/clustal/pair_dist.c
> +++ b/src/clustal/pair_dist.c
> @@ -321,7 +321,7 @@ PairDistances(symmatrix_t **distmat, mseq_t *mseq,
> int pairdist_type, bool bPerc
>
> /* FIXME: can get rid of iChunkStart, iChunkEnd now that we're
> using the arrays */
> iChunkStart = iend;
> -for(iChunk = 0; iChunk <= iNumberOfThreads; iChunk++)
> +for(iChunk = 0; iChunk < iNumberOfThreads; iChunk++)
> {
> iChunkEnd = iChunkStart;
> if (iChunk == iNumberOfThreads - 1){
> $ make
> …
> $ ./src/clustalo -i debian/tests/biopython_testdata/f002 --guidetree-out
> temp_test.dnd -o temp_test.aln --outfmt clustal --force
> =
> ==30601==ERROR: AddressSanitizer: global-buffer-overflow on address
> 0x561188847864 at pc 0x5611886da6e7 bp 0x7fffe6d77ef0 sp 0x7fffe6d77ee8
> READ of size 4 at 0x561188847864 thread T0
> #0 0x5611886da6e6 in FullAlignment::Build(HMM&, Hit&, char*)
> /home/matthew/clustal-omega-1.2.
Bug#956324: Clustalo bus error on mipsel (Was: Bug#956324: python-biopython: FTBFS on mipsel)
> On Apr 17, 2020, at 13:18, Andreas Tille wrote: > > Hi Matthew, > > On Fri, Apr 17, 2020 at 08:18:29AM -0700, Matthew Fernandez wrote: >>> Thanks for the patch which I applied to packaging Git. I assume you >>> want to express that while these fixes are definitely good coding >>> practice the bus error problem is not fixed by it, right? >> >> Thanks, Andreas. It may fix the bus error, but I don’t have a MIPS machine >> to test on. Some of those logging calls had the potential to leave you with >> a misaligned stack pointer. IIUC unaligned loads on MIPS could cause such a >> bus error. > > I tried with hope ... but failed: > > (sid_mipsel-dchroot)tille@eller:~/clustalo$ gdb --args src/clustalo -i > debian/tests/biopython_testdata/f002 --guidetree-out temp_test.dnd -o > temp_test.aln --outfmt clustal --force > GNU gdb (Debian 9.1-3) 9.1 > ... > Reading symbols from src/clustalo... > (gdb) run > Starting program: /home/tille/clustalo/src/clustalo -i > debian/tests/biopython_testdata/f002 --guidetree-out temp_test.dnd -o > temp_test.aln --outfmt clustal --force > [Thread debugging using libthread_db enabled] > Using host libthread_db library "/lib/mipsel-linux-gnu/libthread_db.so.1". > > Program received signal SIGBUS, Bus error. > 0x5556a1b8 in PairDistances (distmat=0x7fff278c, mseq=0x55692a30, > pairdist_type=, bPercID=, istart=0, iend=3, > jstart=0, jend=3, fdist_in=0x0, >fdist_out=0x0) at pair_dist.c:346 > 346 NewProgress(&prProgress, LogGetFP(&rLog, LOG_INFO), OK, let me try a little harder :) $ # enable debugging symbols and Address Sanitizer $ CFLAGS="-g -fsanitize=address" CXXFLAGS="-g -fsanitize=address" ./configure … $ make clean && make … $ ./src/clustalo -i debian/tests/biopython_testdata/f002 --guidetree-out temp_test.dnd -o temp_test.aln --outfmt clustal --force = ==30264==ERROR: AddressSanitizer: dynamic-stack-buffer-overflow on address 0x7ffcfcbf5784 at pc 0x5620f0aa478c bp 0x7ffcfcbf56c0 sp 0x7ffcfcbf56b8 WRITE of size 4 at 0x7ffcfcbf5784 thread T0 #0 0x5620f0aa478b in PairDistances /home/matthew/clustal-omega-1.2.4/src/clustal/pair_dist.c:336 #1 0x5620f0a91d9f in AlignmentOrder /home/matthew/clustal-omega-1.2.4/src/clustal-omega.c:835 #2 0x5620f0a95c04 in Align /home/matthew/clustal-omega-1.2.4/src/clustal-omega.c:1221 #3 0x5620f0a90d76 in MyMain /home/matthew/clustal-omega-1.2.4/src/mymain.c:1192 #4 0x5620f0a88ca2 in main /home/matthew/clustal-omega-1.2.4/src/main.cpp:469 #5 0x7f3773d9009a in __libc_start_main ../csu/libc-start.c:308 #6 0x5620f0a89ad9 in _start (/home/matthew/clustal-omega-1.2.4/src/clustalo+0x2dad9) Address 0x7ffcfcbf5784 is located in stack of thread T0 SUMMARY: AddressSanitizer: dynamic-stack-buffer-overflow /home/matthew/clustal-omega-1.2.4/src/clustal/pair_dist.c:336 in PairDistances Shadow bytes around the buggy address: 0x10001f976aa0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0x10001f976ab0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0x10001f976ac0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0x10001f976ad0: 00 00 00 00 00 00 00 00 00 00 00 00 ca ca ca ca 0x10001f976ae0: 04 cb cb cb cb cb cb cb 00 00 00 00 ca ca ca ca =>0x10001f976af0:[04]cb cb cb cb cb cb cb 00 00 00 00 00 00 00 00 0x10001f976b00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0x10001f976b10: f1 f1 f1 f1 00 f2 f2 f2 f2 f2 f2 f2 00 f2 f2 f2 0x10001f976b20: f2 f2 f2 f2 00 00 00 00 00 00 00 00 00 f2 f2 f2 0x10001f976b30: f3 f3 f3 f3 00 00 00 00 00 00 00 00 00 00 00 00 0x10001f976b40: 00 00 00 00 00 00 f1 f1 f1 f1 00 f2 f2 f2 f2 f2 Shadow byte legend (one shadow byte represents 8 application bytes): Addressable: 00 Partially addressable: 01 02 03 04 05 06 07 Heap left redzone: fa Freed heap region: fd Stack left redzone: f1 Stack mid redzone: f2 Stack right redzone: f3 Stack after return: f5 Stack use after scope: f8 Global redzone: f9 Global init order: f6 Poisoned by user:f7 Container overflow: fc Array cookie:ac Intra object redzone:bb ASan internal: fe Left alloca redzone: ca Right alloca redzone:cb ==30264==ABORTING Looking at line 336 of pair_dist.c, it looks like the bound on the containing loop is wrong. So let’s try adjusting that: $ vim src/clustal/pair_dist.c $ git diff src/clustal/pair_dist.c diff --git a/src/clustal/pair_dist.c b/src/clustal/pair_dist.c index e6dbdc3..bb79e61 100644 --- a/src/clustal/pair_dist.c +++ b/src/clustal/pair_dist.c @@ -321,7 +321,7 @@ PairDistances(symmatrix_t **distmat, mseq_t *mseq, int pairdist_type, bool bPerc
Bug#956324: Clustalo bus error on mipsel (Was: Bug#956324: python-biopython: FTBFS on mipsel)
Hi Matthew, On Fri, Apr 17, 2020 at 08:18:29AM -0700, Matthew Fernandez wrote: > > Thanks for the patch which I applied to packaging Git. I assume you > > want to express that while these fixes are definitely good coding > > practice the bus error problem is not fixed by it, right? > > Thanks, Andreas. It may fix the bus error, but I don’t have a MIPS machine > to test on. Some of those logging calls had the potential to leave you with > a misaligned stack pointer. IIUC unaligned loads on MIPS could cause such a > bus error. I tried with hope ... but failed: (sid_mipsel-dchroot)tille@eller:~/clustalo$ gdb --args src/clustalo -i debian/tests/biopython_testdata/f002 --guidetree-out temp_test.dnd -o temp_test.aln --outfmt clustal --force GNU gdb (Debian 9.1-3) 9.1 ... Reading symbols from src/clustalo... (gdb) run Starting program: /home/tille/clustalo/src/clustalo -i debian/tests/biopython_testdata/f002 --guidetree-out temp_test.dnd -o temp_test.aln --outfmt clustal --force [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib/mipsel-linux-gnu/libthread_db.so.1". Program received signal SIGBUS, Bus error. 0x5556a1b8 in PairDistances (distmat=0x7fff278c, mseq=0x55692a30, pairdist_type=, bPercID=, istart=0, iend=3, jstart=0, jend=3, fdist_in=0x0, fdist_out=0x0) at pair_dist.c:346 346 NewProgress(&prProgress, LogGetFP(&rLog, LOG_INFO), Thank you for the fixes anyway Andreas. -- http://fam-tille.de
Bug#956324: Clustalo bus error on mipsel (Was: Bug#956324: python-biopython: FTBFS on mipsel)
On Fri, 17 Apr 2020 at 08:09, Andreas Tille wrote: > Hi Matthew, > > On Fri, Apr 17, 2020 at 07:40:54AM -0700, Matthew Fernandez wrote: > > > > As a jumping off point, the attached patch fixes some issues with > logging calls in the upstream 1.2.4 source release. > > Thanks for the patch which I applied to packaging Git. I assume you > want to express that while these fixes are definitely good coding > practice the bus error problem is not fixed by it, right? Thanks, Andreas. It may fix the bus error, but I don’t have a MIPS machine to test on. Some of those logging calls had the potential to leave you with a misaligned stack pointer. IIUC unaligned loads on MIPS could cause such a bus error.
Bug#956324: Clustalo bus error on mipsel (Was: Bug#956324: python-biopython: FTBFS on mipsel)
Hi Matthew, On Fri, Apr 17, 2020 at 07:40:54AM -0700, Matthew Fernandez wrote: > > As a jumping off point, the attached patch fixes some issues with logging > calls in the upstream 1.2.4 source release. Thanks for the patch which I applied to packaging Git. I assume you want to express that while these fixes are definitely good coding practice the bus error problem is not fixed by it, right? Kind regards and thank a lot in any case Andreas. -- http://fam-tille.de
Bug#956324: Clustalo bus error on mipsel (Was: Bug#956324: python-biopython: FTBFS on mipsel)
> On Apr 17, 2020, at 04:20, Andreas Tille wrote: > > Control: tags -1 help > > Hi, > > as it can be seen on the recent build log of clustalo on mips[1] the > build fails with > > > # Run additional test from python-biopython package to verify that > # this will work as well > src/clustalo -i debian/tests/biopython_testdata/f002 --guidetree-out > temp_test.dnd -o temp_test.aln --outfmt clustal --force > make[1]: *** [debian/rules:25: override_dh_auto_test-arch] Bus error > > > I injected that extra test since this very test failed inside the > python-biopython package. To track it down I logged in to > eller.debian.org tried to build the package and was able to reproduce > the error. Thus I fired up gdb in this session and got: > > > (sid_mipsel-dchroot)tille@eller:~/clustalo-1.2.4$ gdb --args src/clustalo -i > debian/tests/biopython_testdata/f002 --guidetree-out temp_test.dnd -o > temp_test.aln --outfmt clustal --force > GNU gdb (Debian 9.1-3) 9.1 > ... > Reading symbols from src/clustalo... > (gdb) run > Starting program: /home/tille/clustalo-1.2.4/src/clustalo -i > debian/tests/biopython_testdata/f002 --guidetree-out temp_test.dnd -o > temp_test.aln --outfmt clustal --force > [Thread debugging using libthread_db enabled] > Using host libthread_db library "/lib/mipsel-linux-gnu/libthread_db.so.1". > > Program received signal SIGBUS, Bus error. > 0x5556a188 in PairDistances (distmat=0x7fff278c, mseq=0x55692a38, > pairdist_type=, bPercID=, istart=0, iend=3, > jstart=0, jend=3, fdist_in=0x0, fdist_out=0x0) at pair_dist.c:346 > 346 NewProgress(&prProgress, LogGetFP(&rLog, LOG_INFO), > (gdb) > > > So it seems the bus error occures somehow here: > > > https://salsa.debian.org/med-team/clustalo/-/blob/master/src/clustal/pair_dist.c#L346 > > > I have no idea how to continue from here. I'm hoping that someone with > some more detailed knowledge might have a clue how to fix this issue on > mipsel. Otherwise I personally do not see any better solution than to > remove clustalo from mipsel architecture. As a jumping off point, the attached patch fixes some issues with logging calls in the upstream 1.2.4 source release. clustalo-log-format-calls.patch Description: Binary data
Bug#956324: Clustalo bus error on mipsel (Was: Bug#956324: python-biopython: FTBFS on mipsel)
Control: tags -1 help Hi, as it can be seen on the recent build log of clustalo on mips[1] the build fails with # Run additional test from python-biopython package to verify that # this will work as well src/clustalo -i debian/tests/biopython_testdata/f002 --guidetree-out temp_test.dnd -o temp_test.aln --outfmt clustal --force make[1]: *** [debian/rules:25: override_dh_auto_test-arch] Bus error I injected that extra test since this very test failed inside the python-biopython package. To track it down I logged in to eller.debian.org tried to build the package and was able to reproduce the error. Thus I fired up gdb in this session and got: (sid_mipsel-dchroot)tille@eller:~/clustalo-1.2.4$ gdb --args src/clustalo -i debian/tests/biopython_testdata/f002 --guidetree-out temp_test.dnd -o temp_test.aln --outfmt clustal --force GNU gdb (Debian 9.1-3) 9.1 ... Reading symbols from src/clustalo... (gdb) run Starting program: /home/tille/clustalo-1.2.4/src/clustalo -i debian/tests/biopython_testdata/f002 --guidetree-out temp_test.dnd -o temp_test.aln --outfmt clustal --force [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib/mipsel-linux-gnu/libthread_db.so.1". Program received signal SIGBUS, Bus error. 0x5556a188 in PairDistances (distmat=0x7fff278c, mseq=0x55692a38, pairdist_type=, bPercID=, istart=0, iend=3, jstart=0, jend=3, fdist_in=0x0, fdist_out=0x0) at pair_dist.c:346 346 NewProgress(&prProgress, LogGetFP(&rLog, LOG_INFO), (gdb) So it seems the bus error occures somehow here: https://salsa.debian.org/med-team/clustalo/-/blob/master/src/clustal/pair_dist.c#L346 I have no idea how to continue from here. I'm hoping that someone with some more detailed knowledge might have a clue how to fix this issue on mipsel. Otherwise I personally do not see any better solution than to remove clustalo from mipsel architecture. Kind regards Andreas. [1] https://buildd.debian.org/status/fetch.php?pkg=clustalo&arch=mipsel&ver=1.2.4-5&stamp=1586883766&raw=0 -- http://fam-tille.de
Bug#956324: Clustalo bus error on mipsel (Was: Bug#956324: python-biopython: FTBFS on mipsel)
Control: reassign -1 clustalo Control: retitle -1 "clustalo: Bus error on mipsel" Control: tags -1 upstream Control: forwarded -1 clust...@ucd.ie Hi, I took over the test done by biopython into the clustalo build time and autopkgtest. As Peter assumed this is an issue in clustalo as you can see in the build log on mipsel here: https://buildd.debian.org/status/fetch.php?pkg=clustalo&arch=mipsel&ver=1.2.4-5&stamp=1586866059&raw=0 ... # Run additional test from python-biopython package to verify that # this will work as well src/clustalo -i debian/tests/biopython_testdata/f002 --guidetree-out temp_test.dnd -o temp_test.aln --outfmt clustal --force make[1]: *** [debian/rules:25: override_dh_auto_test-arch] Bus error Any help from the Debian Mips team would be really welcome. The covid-19 effort is occupying all hands for lots of things so we probably have limited time to debug this kind of issues. Kind regards Andreas. -- http://fam-tille.de
Bug#956324: python-biopython: FTBFS on mipsel
I agree this is most likely a clustalo error. I think the first two examples use temporary input files created during the Biopython tests. However, the third failing example ought to be useful in isolation: clustalo -i Fasta/f002 --guidetree-out temp_test.dnd -o temp_test.aln --outfmt clustal --force The input file is here (plain text with no extension): https://github.com/biopython/biopython/blob/master/Tests/Fasta/f002 Peter On Fri, Apr 10, 2020 at 6:29 AM Andreas Tille wrote: > Control: forwarded -1 Peter Cock > > Hi Peter, > > the log that is linked to below says in the end: > > > == > ERROR: test_input_filename_with_space > (test_ClustalOmega_tool.ClustalOmegaTestNormalConditions) > Test an input filename containing a space. > -- > Traceback (most recent call last): > File > "/<>/.pybuild/cpython3_3.8/build/Tests/test_ClustalOmega_tool.py", > line 175, in test_input_filename_with_space > self.standard_test_procedure(cline) > File > "/<>/.pybuild/cpython3_3.8/build/Tests/test_ClustalOmega_tool.py", > line 63, in standard_test_procedure > output, error = cline() > File > "/<>/.pybuild/cpython3_3.8/build/Bio/Application/__init__.py", > line 530, in __call__ > raise ApplicationError(return_code, str(self), > Bio.Application.ApplicationError: Non-zero return code 138 from 'clustalo > -i "Clustalw/temp horses.fasta" -o temp_test.aln --outfmt clustal --force', > message 'Bus error' > > == > ERROR: test_large_fasta_file > (test_ClustalOmega_tool.ClustalOmegaTestNormalConditions) > Test a large fasta input file. > -- > Traceback (most recent call last): > File > "/<>/.pybuild/cpython3_3.8/build/Tests/test_ClustalOmega_tool.py", > line 210, in test_large_fasta_file > self.standard_test_procedure(cline) > File > "/<>/.pybuild/cpython3_3.8/build/Tests/test_ClustalOmega_tool.py", > line 63, in standard_test_procedure > output, error = cline() > File > "/<>/.pybuild/cpython3_3.8/build/Bio/Application/__init__.py", > line 530, in __call__ > raise ApplicationError(return_code, str(self), > Bio.Application.ApplicationError: Non-zero return code 138 from 'clustalo > -i temp_cw_prot.fasta -o temp_cw_prot.aln --outfmt clustal --force', > message 'Bus error' > > == > ERROR: test_newtree_files > (test_ClustalOmega_tool.ClustalOmegaTestNormalConditions) > Test requesting a guide tree. > -- > Traceback (most recent call last): > File > "/<>/.pybuild/cpython3_3.8/build/Tests/test_ClustalOmega_tool.py", > line 224, in test_newtree_files > self.standard_test_procedure(cline) > File > "/<>/.pybuild/cpython3_3.8/build/Tests/test_ClustalOmega_tool.py", > line 63, in standard_test_procedure > output, error = cline() > File > "/<>/.pybuild/cpython3_3.8/build/Bio/Application/__init__.py", > line 530, in __call__ > raise ApplicationError(return_code, str(self), > Bio.Application.ApplicationError: Non-zero return code 138 from 'clustalo > -i Fasta/f002 --guidetree-out temp_test.dnd -o temp_test.aln --outfmt > clustal --force', message 'Bus error' > > -- > > > I'm not sure whether you have any clue about this since it looks rather > like a clustalo issue - but I guess you could be interested. Please let > us know what you think about this. > > Kind regards > >Andreas. > > On Thu, Apr 09, 2020 at 10:38:04PM +0200, Sebastian Ramacher wrote: > > Source: python-biopython > > Version: 1.76+dfsg-1 > > Severity: serious > > Tags: ftbfs sid bullseye > > Justification: fails to build from source (but built successfully in the > past) > > > > python-biopython failed to build on mipsel: > > > https://buildd.debian.org/status/fetch.php?pkg=python-biopython&arch=mipsel&ver=1.76%2Bdfsg-1%2Bb1&stamp=1586419338&raw=0 > > > > Cheers > > -- > > Sebastian Ramacher > > > > > ___ > > Debian-med-packaging mailing list > > debian-med-packag...@alioth-lists.debian.net > > > https://alioth-lists.debian.net/cgi-bin/mailman/listinfo/debian-med-packaging > > > -- > http://fam-tille.de >
Bug#956324: python-biopython: FTBFS on mipsel
Control: forwarded -1 Peter Cock Hi Peter, the log that is linked to below says in the end: == ERROR: test_input_filename_with_space (test_ClustalOmega_tool.ClustalOmegaTestNormalConditions) Test an input filename containing a space. -- Traceback (most recent call last): File "/<>/.pybuild/cpython3_3.8/build/Tests/test_ClustalOmega_tool.py", line 175, in test_input_filename_with_space self.standard_test_procedure(cline) File "/<>/.pybuild/cpython3_3.8/build/Tests/test_ClustalOmega_tool.py", line 63, in standard_test_procedure output, error = cline() File "/<>/.pybuild/cpython3_3.8/build/Bio/Application/__init__.py", line 530, in __call__ raise ApplicationError(return_code, str(self), Bio.Application.ApplicationError: Non-zero return code 138 from 'clustalo -i "Clustalw/temp horses.fasta" -o temp_test.aln --outfmt clustal --force', message 'Bus error' == ERROR: test_large_fasta_file (test_ClustalOmega_tool.ClustalOmegaTestNormalConditions) Test a large fasta input file. -- Traceback (most recent call last): File "/<>/.pybuild/cpython3_3.8/build/Tests/test_ClustalOmega_tool.py", line 210, in test_large_fasta_file self.standard_test_procedure(cline) File "/<>/.pybuild/cpython3_3.8/build/Tests/test_ClustalOmega_tool.py", line 63, in standard_test_procedure output, error = cline() File "/<>/.pybuild/cpython3_3.8/build/Bio/Application/__init__.py", line 530, in __call__ raise ApplicationError(return_code, str(self), Bio.Application.ApplicationError: Non-zero return code 138 from 'clustalo -i temp_cw_prot.fasta -o temp_cw_prot.aln --outfmt clustal --force', message 'Bus error' == ERROR: test_newtree_files (test_ClustalOmega_tool.ClustalOmegaTestNormalConditions) Test requesting a guide tree. -- Traceback (most recent call last): File "/<>/.pybuild/cpython3_3.8/build/Tests/test_ClustalOmega_tool.py", line 224, in test_newtree_files self.standard_test_procedure(cline) File "/<>/.pybuild/cpython3_3.8/build/Tests/test_ClustalOmega_tool.py", line 63, in standard_test_procedure output, error = cline() File "/<>/.pybuild/cpython3_3.8/build/Bio/Application/__init__.py", line 530, in __call__ raise ApplicationError(return_code, str(self), Bio.Application.ApplicationError: Non-zero return code 138 from 'clustalo -i Fasta/f002 --guidetree-out temp_test.dnd -o temp_test.aln --outfmt clustal --force', message 'Bus error' -- I'm not sure whether you have any clue about this since it looks rather like a clustalo issue - but I guess you could be interested. Please let us know what you think about this. Kind regards Andreas. On Thu, Apr 09, 2020 at 10:38:04PM +0200, Sebastian Ramacher wrote: > Source: python-biopython > Version: 1.76+dfsg-1 > Severity: serious > Tags: ftbfs sid bullseye > Justification: fails to build from source (but built successfully in the past) > > python-biopython failed to build on mipsel: > https://buildd.debian.org/status/fetch.php?pkg=python-biopython&arch=mipsel&ver=1.76%2Bdfsg-1%2Bb1&stamp=1586419338&raw=0 > > Cheers > -- > Sebastian Ramacher > ___ > Debian-med-packaging mailing list > debian-med-packag...@alioth-lists.debian.net > https://alioth-lists.debian.net/cgi-bin/mailman/listinfo/debian-med-packaging -- http://fam-tille.de
Bug#956324: python-biopython: FTBFS on mipsel
Source: python-biopython Version: 1.76+dfsg-1 Severity: serious Tags: ftbfs sid bullseye Justification: fails to build from source (but built successfully in the past) python-biopython failed to build on mipsel: https://buildd.debian.org/status/fetch.php?pkg=python-biopython&arch=mipsel&ver=1.76%2Bdfsg-1%2Bb1&stamp=1586419338&raw=0 Cheers -- Sebastian Ramacher signature.asc Description: PGP signature
