Hi Jeff, I have changed the position of malloc.h in header files list. I moved it up above mpi.h. Now I am getting a different error message see following, [asad@stat74 T]$ mpirun --np 4 nice -10 ./lisa09EMRIT-P +---[ lisa14.c ]--------------------------------------------------- | This is process #0 of 4 | initialising : 21.12.2008 21:59:28 Allocating 134217728 bytes for read buffer... Allocating 134217728 bytes for read buffer... Allocating 134217728 bytes for read buffer... Allocating 134217728 bytes for read buffer... Allocating 134217728 bytes for read buffer... Allocating 134217728 bytes for read buffer... Allocating 134217728 bytes for read buffer... -+--> INJECTING SIGNAL... Allocating 134217728 bytes for read buffer... -+--> INJECTING SIGNAL... -+--> INJECTING SIGNAL... -+--> INJECTING SIGNAL... 524765 524765 524765 524765 | initialising `TDI framework' (might take several minutes)... Signal:6 info.si_errno:0(Success) si_code:-6() [0] func:/usr/lib/openmpi/libopal.so.0 [0x46efd6] [1] func:[0x110440] [2] func:[0x110402] [3] func:/lib/libc.so.6(gsignal+0x50) [0x2c1fa0] [4] func:/lib/libc.so.6(abort+0x101) [0x2c38b1] [5] func:/usr/lib/openmpi/libopal.so.0 [0x473d34] [6] func:/usr/lib/openmpi/libopal.so.0(free+0x36) [0x473786] [7] func:./lisa09EMRIT-P(main+0x454) [0x80566f4] [8] func:/lib/libc.so.6(__libc_start_main+0xe0) [0x2aef70] [9] func:./lisa09EMRIT-P [0x804af21] *** End of error message ***
Its good that now no user defined functions in code are involved in error (it was giving error messages for some of my defined functions too). malloc.h has also been disappeared in this error message as was in previous message. I have used valgrind to see where the error is but I am unable to interpret the following valgrind output error message. --16515-- object doesn't have a symbol table --16515-- Reading syms from /usr/lib/openmpi/openmpi/mca_osc_pt2pt.so (0x4114000) --16515-- object doesn't have a symbol table ==16513== Warning: set address range perms: large range 536870912 (defined) ==16512== Warning: set address range perms: large range 536870912 (defined) ==16514== Warning: set address range perms: large range 536870912 (defined) ==16515== Warning: set address range perms: large range 536870912 (defined) +---[ lisa14.c ]--------------------------------------------------- | This is process #0 of 4 | initialising : 21.12.2008 21:15:37 Allocating 134217728 bytes for read buffer... ==16515== Warning: set address range perms: large range 134221824 (defined) Allocating 134217728 bytes for read buffer... ==16514== Warning: set address range perms: large range 134221824 (defined) Allocating 134217728 bytes for read buffer... ==16513== Warning: set address range perms: large range 134221824 (defined) Allocating 134217728 bytes for read buffer... ==16512== Warning: set address range perms: large range 134221824 (defined) ==16515== Warning: set address range perms: large range 134217728 (defined) ==16514== Warning: set address range perms: large range 134217728 (defined) ==16513== Warning: set address range perms: large range 134217728 (defined) ==16512== Warning: set address range perms: large range 134217728 (defined) ==16515== Warning: set address range perms: large range 134221824 (noaccess) ==16514== Warning: set address range perms: large range 134221824 (noaccess) ==16513== Warning: set address range perms: large range 134221824 (noaccess) ==16512== Warning: set address range perms: large range 134221824 (noaccess) --16515-- memcheck GC: 1024 nodes, 1024 survivors (100.0%) --16515-- memcheck GC: increase table size to 2048 --16513-- memcheck GC: 1024 nodes, 1024 survivors (100.0%) --16513-- memcheck GC: increase table size to 2048 --16514-- memcheck GC: 1024 nodes, 1024 survivors (100.0%) --16514-- memcheck GC: increase table size to 2048 --16512-- memcheck GC: 1024 nodes, 1024 survivors (100.0%) --16512-- memcheck GC: increase table size to 2048 --16513-- memcheck GC: 2048 nodes, 2048 survivors (100.0%) --16513-- memcheck GC: increase table size to 4096 --16514-- memcheck GC: 2048 nodes, 2048 survivors (100.0%) --16514-- memcheck GC: increase table size to 4096 --16515-- memcheck GC: 2048 nodes, 2048 survivors (100.0%) --16515-- memcheck GC: increase table size to 4096 --16512-- memcheck GC: 2048 nodes, 2048 survivors (100.0%) --16512-- memcheck GC: increase table size to 4096 Allocating 134217728 bytes for read buffer... ==16515== Warning: set address range perms: large range 134221824 (defined) Allocating 134217728 bytes for read buffer... ==16513== Warning: set address range perms: large range 134221824 (defined) Allocating 134217728 bytes for read buffer... ==16514== Warning: set address range perms: large range 134221824 (defined) ==16515== Warning: set address range perms: large range 134217728 (defined) ==16513== Warning: set address range perms: large range 134217728 (defined) ==16514== Warning: set address range perms: large range 134217728 (defined) ==16515== Warning: set address range perms: large range 134221824 (noaccess) Allocating 134217728 bytes for read buffer... ==16512== Warning: set address range perms: large range 134221824 (defined) ==16513== Warning: set address range perms: large range 134221824 (noaccess) ==16512== Warning: set address range perms: large range 134217728 (defined) ==16514== Warning: set address range perms: large range 134221824 (noaccess) ==16512== Warning: set address range perms: large range 134221824 (noaccess) -+--> INJECTING SIGNAL... -+--> INJECTING SIGNAL... -+--> INJECTING SIGNAL... -+--> INJECTING SIGNAL... 524765 524765 524765 524765 Signal:6 info.si_errno:0(Success) si_code:-6() --16515-- Reading syms from /lib/libgcc_s-4.1.2-20070925.so.1 (0x794000) --16515-- object doesn't have a symbol table [0] func:/usr/lib/openmpi/libopal.so.0 [0x46efd6] [1] func:/lib/libpthread.so.0 [0x443290] [2] func:/lib/ld-linux.so.2 [0x2767f2] [3] func:/lib/libc.so.6(gsignal+0x50) [0x2c1fa0] [4] func:/lib/libc.so.6(abort+0x101) [0x2c38b1] [5] func:/usr/lib/openmpi/libopal.so.0 [0x473d34] [6] func:/usr/lib/openmpi/libopal.so.0(free+0x36) [0x473786] [7] func:./lisa09EMRIT-P(main+0x44c) [0x80566ec] [8] func:/lib/libc.so.6(__libc_start_main+0xe0) [0x2aef70] [9] func:./lisa09EMRIT-P [0x804af21] *** End of error message *** ==16515== ==16515== Process terminating with default action of signal 6 (SIGABRT) ==16515== at 0x2C1FA0: raise (in /lib/libc-2.6.so) ==16515== by 0x2C39FD: abort (in /lib/libc-2.6.so) ==16515== by 0x473D33: (within /usr/lib/openmpi/libopal.so.0.0.0) ==16515== by 0x473785: free (in /usr/lib/openmpi/libopal.so.0.0.0) ==16515== by 0x80566EB: main (in /home/phd/asad/MLDCs/T/lisa09EMRIT-P) ==16515== ==16515== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 108 from 1) --16515-- --16515-- supp: 108 dl-hack3 ==16515== malloc/free: in use at exit: 0 bytes in 0 blocks. ==16515== malloc/free: 0 allocs, 0 frees, 0 bytes allocated. ==16515== ==16515== All heap blocks were freed -- no leaks are possible. --16515-- memcheck: sanity checks: 22725 cheap, 910 expensive --16515-- memcheck: auxmaps: 0 auxmap entries (0k, 0M) in use --16515-- memcheck: auxmaps: 0 searches, 0 comparisons --16515-- memcheck: SMs: n_issued = 6014 (96224k, 93M) --16515-- memcheck: SMs: n_deissued = 4726 (75616k, 73M) --16515-- memcheck: SMs: max_noaccess = 65535 (1048560k, 1023M) --16515-- memcheck: SMs: max_undefined = 38 (608k, 0M) --16515-- memcheck: SMs: max_defined = 10813 (173008k, 168M) --16515-- memcheck: SMs: max_non_DSM = 3109 (49744k, 48M) --16515-- memcheck: max sec V bit nodes: 2094 (106k, 0M) --16515-- memcheck: set_sec_vbits8 calls: 3371606 (new: 2094, updates: 3369512) --16515-- memcheck: max shadow mem size: 50154k, 48M --16515-- translate: fast SP updates identified: 23,603 ( 89.1%) --16515-- translate: generic_known SP updates identified: 2,000 ( 7.5%) --16515-- translate: generic_unknown SP updates identified: 877 ( 3.3%) --16515-- tt/tc: 625,692 tt lookups requiring 672,349 probes --16515-- tt/tc: 625,692 fast-cache updates, 9 flushes --16515-- transtab: new 20,764 (798,227 -> 14,916,839; ratio 186:10) [0 scs] --16515-- transtab: dumped 0 (0 -> ??) --16515-- transtab: discarded 193 (3,337 -> ??) --16515-- scheduler: 2,272,550,211 jumps (bb entries). --16515-- scheduler: 22,725/32,791,636 major/minor sched events. --16515-- sanity: 22726 cheap, 910 expensive checks. --16515-- exectx: 30,011 lists, 11 contexts (avg 0 per list) --16515-- exectx: 108 searches, 97 full compares (898 per 1000) --16515-- exectx: 0 cmp2, 232 cmp4, 0 cmpAll [8]+ Killed mpirun -np 4 valgrind -v ./lisa09EMRIT-P Could please tell me how ti interpret the results of valgrind? Thanking you in advance. Asad
