Hallo Aaron,
... thanks for your help first. Here my gcc/glibc versions:
[EMAIL PROTECTED] ~]# gcc --version
gcc (GCC) 3.4.3 20041212 (Red Hat 3.4.3-9.EL4)
[EMAIL PROTECTED] rpm]# /lib/libc.so.6
GNU C Library stable release version 2.3.4, by Roland McGrath et al.
Copyright (C) 2004 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.
There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A
PARTICULAR PURPOSE.
Compiled by GNU CC version 3.4.3 20041212 (Red Hat 3.4.3-9.EL4).
Compiled on a Linux 2.4.20 system on 2004-12-20.
Available extensions:
GNU libio by Per Bothner
crypt add-on version 2.1 by Michael Glad and others
linuxthreads-0.10 by Xavier Leroy
The C stubs add-on version 2.1.2.
BIND-8.2.3-T5B
NIS(YP)/NIS+ NSS modules 0.19 by Thorsten Kukuk
Glibc-2.0 compatibility add-on by Cristian Gafton
GNU Libidn by Simon Josefsson
libthread_db work sponsored by Alpha Processor Inc
Thread-local storage support included.
I configured it as follows (using the 200506 version of the NCBI toolbox):
[EMAIL PROTECTED] mpiblast-1.4.0]$ ./configure --with-ncbi=/usr/local/ncbi
--with-mpi=/opt/lam-7.0.6 --enable-MPI_Alloc_mem
checking build system type... i686-pc-linux
checking host system type... i686-pc-linux
checking target system type... i686-pc-linux
checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... yes
checking for gawk... gawk
checking whether make sets $(MAKE)... yes
checking for MPI... /opt/lam-7.0.6/bin
checking for mpicc... /opt/lam-7.0.6/bin/mpicc
checking for mpicxx... no
checking for mpic++... /opt/lam-7.0.6/bin/mpic++
checking for C compiler default output file name... a.out
checking whether the C compiler works... yes
checking whether we are cross compiling... no
checking for suffix of executables...
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether /opt/lam-7.0.6/bin/mpicc accepts -g... yes
checking for /opt/lam-7.0.6/bin/mpicc option to accept ANSI C... none needed
checking for style of include used by make... GNU
checking dependency style of /opt/lam-7.0.6/bin/mpicc... gcc3
checking whether we are using the GNU C++ compiler... yes
checking whether /opt/lam-7.0.6/bin/mpic++ accepts -g... yes
checking dependency style of /opt/lam-7.0.6/bin/mpic++... gcc3
checking for ranlib... ranlib
checking for a BSD-compatible install... /usr/bin/install -c
checking whether ln -s works... yes
checking how to run the C preprocessor... /opt/lam-7.0.6/bin/mpicc -E
checking for egrep... grep -E
checking for ANSI C header files... yes
checking for special C compiler options needed for large files... no
checking for _FILE_OFFSET_BITS value needed for large files... 64
checking for _LARGE_FILES value needed for large files... no
checking for getopt_long... yes
checking for getopt... yes
checking for NCBI library... /usr/local/ncbi
checking for /usr/local/ncbi/demo/formatdb.c... yes
checking for /usr/local/ncbi/build/ncbithr.o... yes
checking for MPI_Alloc_mem... checking for MPE timing option... false
checking for an ANSI C-conforming const... yes
checking for inline... inline
checking whether time.h and sys/time.h may both be included... yes
checking whether /opt/lam-7.0.6/bin/mpicc needs -traditional... no
configure: creating ./config.status
config.status: creating Makefile
config.status: creating src/Makefile
config.status: creating src/config.h
config.status: executing depfiles commands
[EMAIL PROTECTED] mpiblast-1.4.0]$ make
Making all in src
make[1]: Entering directory `/home/mike/mpiblast-1.4.0/src'
make all-am
make[2]: Entering directory `/home/mike/mpiblast-1.4.0/src'
if /opt/lam-7.0.6/bin/mpic++ -DHAVE_CONFIG_H -I. -I. -I. -I../include/
-I/usr/local/ncbi/include/ -DUSING_MPI
-DINSTALL_PREFIX="\"/usr/local\"" -g -O2 -MT mpiblast-blastjob.o -MD
-MP -MF ".deps/mpiblast-blastjob.Tpo" -c -o mpiblast-blastjob.o `test -f
'blastjob.cpp' || echo './'`blastjob.cpp; \
then mv -f ".deps/mpiblast-blastjob.Tpo" ".deps/mpiblast-blastjob.Po";
else rm -f ".deps/mpiblast-blastjob.Tpo"; exit 1; fi
if /opt/lam-7.0.6/bin/mpic++ -DHAVE_CONFIG_H -I. -I. -I. -I../include/
-I/usr/local/ncbi/include/ -DUSING_MPI
-DINSTALL_PREFIX="\"/usr/local\"" -g -O2 -MT mpiblast-db_spec.o -MD -MP
-MF ".deps/mpiblast-db_spec.Tpo" -c -o mpiblast-db_spec.o `test -f
'db_spec.cpp' || echo './'`db_spec.cpp; \
then mv -f ".deps/mpiblast-db_spec.Tpo" ".deps/mpiblast-db_spec.Po";
else rm -f ".deps/mpiblast-db_spec.Tpo"; exit 1; fi
if /opt/lam-7.0.6/bin/mpicc -DHAVE_CONFIG_H -I. -I. -I. -I../include/
-I/usr/local/ncbi/include/ -DUSING_MPI
-DINSTALL_PREFIX="\"/usr/local\"" -g -O2 -MT mpiblast-blast_hooks.o -MD
-MP -MF ".deps/mpiblast-blast_hooks.Tpo" -c -o mpiblast-blast_hooks.o
`test -f 'blast_hooks.c' || echo './'`blast_hooks.c; \
then mv -f ".deps/mpiblast-blast_hooks.Tpo"
".deps/mpiblast-blast_hooks.Po"; else rm -f
".deps/mpiblast-blast_hooks.Tpo"; exit 1; fi
if /opt/lam-7.0.6/bin/mpic++ -DHAVE_CONFIG_H -I. -I. -I. -I../include/
-I/usr/local/ncbi/include/ -DUSING_MPI
-DINSTALL_PREFIX="\"/usr/local\"" -g -O2 -MT mpiblast-file_util.o -MD
-MP -MF ".deps/mpiblast-file_util.Tpo" -c -o mpiblast-file_util.o `test
-f 'file_util.cpp' || echo './'`file_util.cpp; \
then mv -f ".deps/mpiblast-file_util.Tpo" ".deps/mpiblast-file_util.Po";
else rm -f ".deps/mpiblast-file_util.Tpo"; exit 1; fi
if /opt/lam-7.0.6/bin/mpic++ -DHAVE_CONFIG_H -I. -I. -I. -I../include/
-I/usr/local/ncbi/include/ -DUSING_MPI
-DINSTALL_PREFIX="\"/usr/local\"" -g -O2 -MT mpiblast-fragment_list.o
-MD -MP -MF ".deps/mpiblast-fragment_list.Tpo" -c -o
mpiblast-fragment_list.o `test -f 'fragment_list.cpp' || echo
'./'`fragment_list.cpp; \
then mv -f ".deps/mpiblast-fragment_list.Tpo"
".deps/mpiblast-fragment_list.Po"; else rm -f
".deps/mpiblast-fragment_list.Tpo"; exit 1; fi
if /opt/lam-7.0.6/bin/mpic++ -DHAVE_CONFIG_H -I. -I. -I. -I../include/
-I/usr/local/ncbi/include/ -DUSING_MPI
-DINSTALL_PREFIX="\"/usr/local\"" -g -O2 -MT mpiblast-mpiblast_util.o
-MD -MP -MF ".deps/mpiblast-mpiblast_util.Tpo" -c -o
mpiblast-mpiblast_util.o `test -f 'mpiblast_util.cpp' || echo
'./'`mpiblast_util.cpp; \
then mv -f ".deps/mpiblast-mpiblast_util.Tpo"
".deps/mpiblast-mpiblast_util.Po"; else rm -f
".deps/mpiblast-mpiblast_util.Tpo"; exit 1; fi
if /opt/lam-7.0.6/bin/mpic++ -DHAVE_CONFIG_H -I. -I. -I. -I../include/
-I/usr/local/ncbi/include/ -DUSING_MPI
-DINSTALL_PREFIX="\"/usr/local\"" -g -O2 -MT mpiblast-mpiblast.o -MD
-MP -MF ".deps/mpiblast-mpiblast.Tpo" -c -o mpiblast-mpiblast.o `test -f
'mpiblast.cpp' || echo './'`mpiblast.cpp; \
then mv -f ".deps/mpiblast-mpiblast.Tpo" ".deps/mpiblast-mpiblast.Po";
else rm -f ".deps/mpiblast-mpiblast.Tpo"; exit 1; fi
if /opt/lam-7.0.6/bin/mpic++ -DHAVE_CONFIG_H -I. -I. -I. -I../include/
-I/usr/local/ncbi/include/ -DUSING_MPI
-DINSTALL_PREFIX="\"/usr/local\"" -g -O2 -MT mpiblast-mpiblast_config.o
-MD -MP -MF ".deps/mpiblast-mpiblast_config.Tpo" -c -o
mpiblast-mpiblast_config.o `test -f 'mpiblast_config.cpp' || echo
'./'`mpiblast_config.cpp; \
then mv -f ".deps/mpiblast-mpiblast_config.Tpo"
".deps/mpiblast-mpiblast_config.Po"; else rm -f
".deps/mpiblast-mpiblast_config.Tpo"; exit 1; fi
if /opt/lam-7.0.6/bin/mpicc -DHAVE_CONFIG_H -I. -I. -I. -I../include/
-I/usr/local/ncbi/include/ -DUSING_MPI
-DINSTALL_PREFIX="\"/usr/local\"" -g -O2 -MT
mpiblast-distributed_bioseq.o -MD -MP -MF
".deps/mpiblast-distributed_bioseq.Tpo" -c -o
mpiblast-distributed_bioseq.o `test -f 'distributed_bioseq.c' || echo
'./'`distributed_bioseq.c; \
then mv -f ".deps/mpiblast-distributed_bioseq.Tpo"
".deps/mpiblast-distributed_bioseq.Po"; else rm -f
".deps/mpiblast-distributed_bioseq.Tpo"; exit 1; fi
if /opt/lam-7.0.6/bin/mpicc -DHAVE_CONFIG_H -I. -I. -I. -I../include/
-I/usr/local/ncbi/include/ -DUSING_MPI
-DINSTALL_PREFIX="\"/usr/local\"" -g -O2 -MT mpiblast-mpi_util.o -MD
-MP -MF ".deps/mpiblast-mpi_util.Tpo" -c -o mpiblast-mpi_util.o `test -f
'mpi_util.c' || echo './'`mpi_util.c; \
then mv -f ".deps/mpiblast-mpi_util.Tpo" ".deps/mpiblast-mpi_util.Po";
else rm -f ".deps/mpiblast-mpi_util.Tpo"; exit 1; fi
/opt/lam-7.0.6/bin/mpic++ -DINSTALL_PREFIX="\"/usr/local\"" -g -O2 -o
mpiblast mpiblast-blastjob.o mpiblast-db_spec.o mpiblast-blast_hooks.o
mpiblast-file_util.o mpiblast-fragment_list.o mpiblast-mpiblast_util.o
mpiblast-mpiblast.o mpiblast-mpiblast_config.o
mpiblast-distributed_bioseq.o mpiblast-mpi_util.o
/usr/local/ncbi/build/ncbithr.o -lncbitool -lncbiobj -lncbi
-L/usr/local/ncbi/lib/ -lm -lpthread
/usr/local/ncbi/lib//libncbi.a(ncbifile.o)(.text+0x945): In function
`Nlm_TmpNam':
: warning: the use of `tempnam' is dangerous, better use `mkstemp'
if /opt/lam-7.0.6/bin/mpic++ -DHAVE_CONFIG_H -I. -I. -I. -I../include/
-I/usr/local/ncbi/include/ -DINSTALL_PREFIX="\"/usr/local\"" -g -O2
-MT mpiformatdb.o -MD -MP -MF ".deps/mpiformatdb.Tpo" -c -o
mpiformatdb.o mpiformatdb.cpp; \
then mv -f ".deps/mpiformatdb.Tpo" ".deps/mpiformatdb.Po"; else rm -f
".deps/mpiformatdb.Tpo"; exit 1; fi
if /opt/lam-7.0.6/bin/mpic++ -DHAVE_CONFIG_H -I. -I. -I. -I../include/
-I/usr/local/ncbi/include/ -DINSTALL_PREFIX="\"/usr/local\"" -g -O2
-MT mpiblast_config.o -MD -MP -MF ".deps/mpiblast_config.Tpo" -c -o
mpiblast_config.o mpiblast_config.cpp; \
then mv -f ".deps/mpiblast_config.Tpo" ".deps/mpiblast_config.Po"; else
rm -f ".deps/mpiblast_config.Tpo"; exit 1; fi
if /opt/lam-7.0.6/bin/mpic++ -DHAVE_CONFIG_H -I. -I. -I. -I../include/
-I/usr/local/ncbi/include/ -DINSTALL_PREFIX="\"/usr/local\"" -g -O2
-MT file_util.o -MD -MP -MF ".deps/file_util.Tpo" -c -o file_util.o
file_util.cpp; \
then mv -f ".deps/file_util.Tpo" ".deps/file_util.Po"; else rm -f
".deps/file_util.Tpo"; exit 1; fi
if /opt/lam-7.0.6/bin/mpic++ -DHAVE_CONFIG_H -I. -I. -I. -I../include/
-I/usr/local/ncbi/include/ -DINSTALL_PREFIX="\"/usr/local\"" -g -O2
-MT mpiblast_util.o -MD -MP -MF ".deps/mpiblast_util.Tpo" -c -o
mpiblast_util.o mpiblast_util.cpp; \
then mv -f ".deps/mpiblast_util.Tpo" ".deps/mpiblast_util.Po"; else rm
-f ".deps/mpiblast_util.Tpo"; exit 1; fi
if /opt/lam-7.0.6/bin/mpic++ -DHAVE_CONFIG_H -I. -I. -I. -I../include/
-I/usr/local/ncbi/include/ -DINSTALL_PREFIX="\"/usr/local\"" -g -O2
-MT fragment_list.o -MD -MP -MF ".deps/fragment_list.Tpo" -c -o
fragment_list.o fragment_list.cpp; \
then mv -f ".deps/fragment_list.Tpo" ".deps/fragment_list.Po"; else rm
-f ".deps/fragment_list.Tpo"; exit 1; fi
if /opt/lam-7.0.6/bin/mpic++ -DHAVE_CONFIG_H -I. -I. -I. -I../include/
-I/usr/local/ncbi/include/ -DINSTALL_PREFIX="\"/usr/local\"" -g -O2
-MT db_spec.o -MD -MP -MF ".deps/db_spec.Tpo" -c -o db_spec.o db_spec.cpp; \
then mv -f ".deps/db_spec.Tpo" ".deps/db_spec.Po"; else rm -f
".deps/db_spec.Tpo"; exit 1; fi
if /opt/lam-7.0.6/bin/mpicc -DHAVE_CONFIG_H -I. -I. -I. -I../include/
-I/usr/local/ncbi/include/ -DINSTALL_PREFIX="\"/usr/local\"" -g -O2
-MT mpiblast_formatdb.o -MD -MP -MF ".deps/mpiblast_formatdb.Tpo" -c -o
mpiblast_formatdb.o mpiblast_formatdb.c; \
then mv -f ".deps/mpiblast_formatdb.Tpo" ".deps/mpiblast_formatdb.Po";
else rm -f ".deps/mpiblast_formatdb.Tpo"; exit 1; fi
if /opt/lam-7.0.6/bin/mpicc -DHAVE_CONFIG_H -I. -I. -I. -I../include/
-I/usr/local/ncbi/include/ -DINSTALL_PREFIX="\"/usr/local\"" -g -O2
-MT mpi_util.o -MD -MP -MF ".deps/mpi_util.Tpo" -c -o mpi_util.o
mpi_util.c; \
then mv -f ".deps/mpi_util.Tpo" ".deps/mpi_util.Po"; else rm -f
".deps/mpi_util.Tpo"; exit 1; fi
/opt/lam-7.0.6/bin/mpic++ -DINSTALL_PREFIX="\"/usr/local\"" -g -O2 -o
mpiformatdb mpiformatdb.o mpiblast_config.o file_util.o mpiblast_util.o
fragment_list.o db_spec.o mpiblast_formatdb.o mpi_util.o
/usr/local/ncbi/build/ncbithr.o -lncbitool -lncbiobj -lncbi
-L/usr/local/ncbi/lib/ -lm -lpthread
/usr/local/ncbi/lib//libncbi.a(ncbifile.o)(.text+0x945): In function
`Nlm_TmpNam':
: warning: the use of `tempnam' is dangerous, better use `mkstemp'
if /opt/lam-7.0.6/bin/mpic++ -DHAVE_CONFIG_H -I. -I. -I. -I../include/
-I/usr/local/ncbi/include/ -DINSTALL_PREFIX="\"/usr/local\"" -g -O2
-MT mpiblast_cleanup.o -MD -MP -MF ".deps/mpiblast_cleanup.Tpo" -c -o
mpiblast_cleanup.o mpiblast_cleanup.cpp; \
then mv -f ".deps/mpiblast_cleanup.Tpo" ".deps/mpiblast_cleanup.Po";
else rm -f ".deps/mpiblast_cleanup.Tpo"; exit 1; fi
/opt/lam-7.0.6/bin/mpic++ -DINSTALL_PREFIX="\"/usr/local\"" -g -O2 -o
mpiblast_cleanup mpiblast_cleanup.o mpiblast_config.o file_util.o
mpiblast_util.o fragment_list.o mpi_util.o
/usr/local/ncbi/build/ncbithr.o -lncbitool -lncbiobj -lncbi
-L/usr/local/ncbi/lib/ -lm -lpthread
/usr/local/ncbi/lib//libncbi.a(ncbifile.o)(.text+0x945): In function
`Nlm_TmpNam':
: warning: the use of `tempnam' is dangerous, better use `mkstemp'
make[2]: Leaving directory `/home/mike/mpiblast-1.4.0/src'
make[1]: Leaving directory `/home/mike/mpiblast-1.4.0/src'
make[1]: Entering directory `/home/mike/mpiblast-1.4.0'
make[1]: Nothing to be done for `all-am'.
make[1]: Leaving directory `/home/mike/mpiblast-1.4.0'
[EMAIL PROTECTED] mpiblast-1.4.0]$
Aaron Darling wrote:
Hi Mike,
What compiler are you using? What ./configure command-line options
did you use? Did you run ./configure with --enable-MPI_Alloc_mem? I
personally develop and test on SuSE, rocks 3.3 (which is a RHEL
derivitive), windows, and occasionally OS X.
To further track this problem down it may be necessary to compile with
debug options and run the program with a debugger attached. If you
can send me the query data set (off the list) I can try to reproduce
the problem.
-Aaron
Mike Schilling wrote:
Unfortunately I send the mail partially already - but incomplete -
sorry for posting again ...
-------------
Hallo everybody,
... I try to run mpiblast on a 17 node (2proc each) oscar 4.2 cluster
based on RHEL 4. The master node is a dual Xeon2.4 while the 16
workers are dual Pentium III (933).
All cluster tests described in the oscar installation were successful
and also I was able to compile the NCBI toolbox (patch included)
without problems. Mpiblast was compiled using the following options:
--with-ncbi=/usr/local/ncbi and --with-mpi=/opt/lam-7.0.6
This was successful as well. Blasting very small contigs (8kb)
against the uniprot database works perfect and fast. When I try to go
with querys over 30kb in size the following error occur after roughly
5 min:
mpirun -np 34 /usr/local/bin/mpiblast
--debug=/database/tmpfiledir/debug -p blastx -d uniprot -i
/database/tmpfiledir/contig76_53738-84646.tmp -o
/database/results/contigs_masked.out
*** glibc detected *** free(): invalid pointer: 0x08a34870 ***
-----------------------------------------------------------------------------
One of the processes started by mpirun has exited with a nonzero exit
code. This typically indicates that the process finished in error.
If your process did not finish in error, be sure to include a "return
0" or "exit(0)" in your C code before exiting the application.
PID 20219 failed on node n7 (10.0.0.8) due to signal 6.
-----------------------------------------------------------------------------
2 134.052 Bailing out with signal -1
3 134.054 Bailing out with signal -1
4 134.055 Bailing out with signal -1
5 134.057 Bailing out with signal -1
67 134.059 Bailing out with signal -1
134.06 Bailing out with signal -1
89 134.063 Bailing out with signal -1
134.063 Bailing out with signal -1
1011 134.066 Bailing out with signal -1
134.066 Bailing out with signal -1
12 134.069 Bailing out with signal -1
15 134.071 Bailing out with signal -1
13 134.071 Bailing out with signal -1
17 134.076 Bailing out with signal -1
16 134.076 Bailing out with signal -1
18 134.077 Bailing out with signal -1
19 134.078 Bailing out with signal -1
20 134.08 Bailing out with signal -1
21 134.081 Bailing out with signal -1
22 23 134.084 Bailing out with signal -1
134.084 Bailing out with signal -124 134.086 Bailing out with
signal -1
25 134.088 Bailing out with signal
-1
27 134.09 Bailing out with signal -1
26 134.091 Bailing out with signal -1
28 134.09329 134.094 Bailing out with signal -1
Bailing out with signal -1
30 134.096 Bailing out with signal -1
32 134.099 Bailing out with signal -1
34 134.1 Bailing out with signal -1
33 134.101 Bailing out with signal -1
31 134.097 Bailing out with signal -1
35 134.1 Bailing out with signal -1
... there is no further bad message when I switch on debug logs - it
seems to break suddenly and the address of the "glibc free()" message
sometimes shows different numbers on different nodes ...
I tried the "-ssi rpi lamd" option of mpirun as well since there was
something in the manual in connection with lamd - same result ....
Next - I compiled both - the 2004 and the 2005 version of the ncbi
toolbox with a similar result. Also compiling against mpich or lam
has no influence. As well I tried to compile the 1.3.0 release of
mpiblast but without success.
Are there any things which I can do to get more debug output? Do you
have a recommendation about a kernel or a specific RHEL version (or
maybe a specific cluster software) where it runs since I do not
believe that it relies on hardware.
any help would be appreciated
best regards
Mike
-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep through log
files
for problems? Stop! Download the new AJAX search engine that makes
searching your log files as easy as surfing the web. DOWNLOAD SPLUNK!
http://ads.osdn.com/?ad_id=7637&alloc_id=16865&op=click
_______________________________________________
Mpiblast-users mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/mpiblast-users
--
+--------------------------------------------------------------------+
| Mike Schilling Head IT Operations world wide |
| MWG Biotech AG voice : int+49 8092 8289303 |
| Anzinger Strasse 7, fax : int+49 8092 8289561 |
| D-85560 Ebersberg, Germany email : [EMAIL PROTECTED] |
| web : http://www.mwgdna.com |
+--------------------------------------------------------------------+
| Contrary to popular belief, UNIX is user friendly. It just happens |
| to be very selective about who it decides to make friends with. |
+--------------------------------------------------------------------+
-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep through log files
for problems? Stop! Download the new AJAX search engine that makes
searching your log files as easy as surfing the web. DOWNLOAD SPLUNK!
http://ads.osdn.com/?ad_id=7637&alloc_id=16865&op=click
_______________________________________________
Mpiblast-users mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/mpiblast-users
