Gilles, At the moment ompi/mca/osc/sm/osc_sm_component.c is using the following:
#ifdef HAVE_GETPAGESIZE pagesize = getpagesize(); #else pagesize = 4096; #endif While other places in the code use sysconf(), but not always consistently. And on some systems _SC_PAGESIZE is spelled as _SC_PAGE_SIZE. Fortunately configure already checks these variations for you. So, I suggest #ifdef HAVE_GETPAGESIZE pagesize = getpagesize(); #elif defined(_SC_PAGESIZE ) pagesize = sysconf(_SC_PAGESIZE); #elif defined(_SC_PAGE_SIZE) pagesize = sysconf(_SC_PAGE_SIZE); #else pagesize = 65536; /* safer to overestimate than under */ #endif opal_pagesize() anyone? -Paul On Fri, Aug 1, 2014 at 12:50 AM, Gilles Gouaillardet < gilles.gouaillar...@iferc.org> wrote: > Paul, > > you are absolutly right ! > > in ompi/mca/coll/ml/coll_ml_lmngr.c at line 53, > cm->lmngr_alignment is hard coded to 4096 > > as a proof of concept, i hard coded it to 65536 and now coll/ml works just > fine > > i will now write a patch that uses sysconf(_SC_PAGESIZE) instead > > Cheers, > > Gilles > > On 2014/08/01 15:56, Paul Hargrove wrote: > > Hmm, maybe this has nothing to do with big-endian. > Below is a backtrace from ring_c on an IA64 platform (definitely > little-endian) that looks very similar to me. > > It happens that sysconf(_SC_PAGESIZE) returns 64K on both of these systems. > So, I wonder if that might be related. > > -Paul > > $ mpirun -mca btl sm,self -np 2 examples/ring_c' > [altix][[26769,1],0][/eng/home/PHHargrove/OMPI/openmpi-trunk-linux-ia64/openmpi-1.9a1r32386/ompi/mca/coll/ml/coll_ml_lmngr.c:231:mca_coll_ml_lmngr_init] > COLL-ML [altix:20418] *** Process received signal *** > [altix:20418] Signal: Segmentation fault (11) > [altix:20418] Signal code: Invalid permissions (2) > [altix:20418] Failing at address: 0x16 > [altix][[26769,1],1][/eng/home/PHHargrove/OMPI/openmpi-trunk-linux-ia64/openmpi-1.9a1r32386/ompi/mca/coll/ml/coll_ml_lmngr.c:231:mca_coll_ml_lmngr_init] > COLL-ML [altix:20419] *** Process received signal *** > [altix:20419] Signal: Segmentation fault (11) > [altix:20419] Signal code: Invalid permissions (2) > [altix:20419] Failing at address: 0x16 > [altix:20418] [ 0] [0xa000000000010800] > [altix:20418] [ 1] /lib/libc.so.6.1(strlen-0x92e930)[0x200000000051b2a0] > [altix:20418] [altix:20419] [ 0] [0xa000000000010800] > [altix:20419] [ 1] [ 2] > /lib/libc.so.6.1(strlen-0x92e930)[0x200000000051b2a0] > [altix:20419] [ 2] > /lib/libc.so.6.1(_IO_vfprintf-0x998610)[0x20000000004b15d0] > [altix:20419] [ 3] /lib/libc.so.6.1(+0x82860)[0x20000000004b2860] > [altix:20419] [ 4] > /lib/libc.so.6.1(_IO_vfprintf-0x99f140)[0x20000000004aaaa0] > [altix:20419] [ 5] > /eng/home/PHHargrove/OMPI/openmpi-trunk-linux-ia64/INST/lib/openmpi/mca_coll_ml.so(+0xc5a70)[0x2000000001e55a70] > [altix:20419] [ 6] > /eng/home/PHHargrove/OMPI/openmpi-trunk-linux-ia64/INST/lib/openmpi/mca_coll_ml.so(+0xc84a0)[0x2000000001e584a0] > [altix:20419] [ 7] > /eng/home/PHHargrove/OMPI/openmpi-trunk-linux-ia64/INST/lib/openmpi/mca_coll_ml.so(mca_coll_ml_lmngr_alloc+0x100f520)[0x2000000001e59110] > [altix:20419] [ 8] > /eng/home/PHHargrove/OMPI/openmpi-trunk-linux-ia64/INST/lib/openmpi/mca_coll_ml.so(mca_coll_ml_allocate_block+0xf6e940)[0x2000000001db8540] > [altix:20419] [ 9] > /eng/home/PHHargrove/OMPI/openmpi-trunk-linux-ia64/INST/lib/openmpi/mca_coll_ml.so(+0x10130)[0x2000000001da0130] > [altix:20419] [10] > /eng/home/PHHargrove/OMPI/openmpi-trunk-linux-ia64/INST/lib/openmpi/mca_coll_ml.so(+0x19970)[0x2000000001da9970] > [altix:20419] [11] > /eng/home/PHHargrove/OMPI/openmpi-trunk-linux-ia64/INST/lib/openmpi/mca_coll_ml.so(mca_coll_ml_comm_query+0xf6d6b0)[0x2000000001db5830] > [altix:20419] [12] > /eng/home/PHHargrove/OMPI/openmpi-trunk-linux-ia64/INST/lib/libmpi.so.0(+0x22fbd0)[0x200000000028fbd0] > [altix:20419] [13] > /eng/home/PHHargrove/OMPI/openmpi-trunk-linux-ia64/INST/lib/libmpi.so.0(+0x22fac0)[0x200000000028fac0] > [altix:20419] [14] > /eng/home/PHHargrove/OMPI/openmpi-trunk-linux-ia64/INST/lib/libmpi.so.0(+0x22f7e0)[0x200000000028f7e0] > [altix:20419] [15] > /eng/home/PHHargrove/OMPI/openmpi-trunk-linux-ia64/INST/lib/libmpi.so.0(+0x22eac0)[0x200000000028eac0] > [altix:20419] [16] > /eng/home/PHHargrove/OMPI/openmpi-trunk-linux-ia64/INST/lib/libmpi.so.0(mca_coll_base_comm_select-0xbcbb90)[0x200000000027e080] > [altix:20419] [17] > /eng/home/PHHargrove/OMPI/openmpi-trunk-linux-ia64/INST/lib/libmpi.so.0(ompi_mpi_init-0xd38e70)[0x2000000000110db0] > [altix:20419] [18] > /eng/home/PHHargrove/OMPI/openmpi-trunk-linux-ia64/INST/lib/libmpi.so.0(MPI_Init-0xc8bf40)[0x20000000001bdcf0] > [altix:20419] [19] examples/ring_c[0x4000000000000c00] > [altix:20419] [20] > /lib/libc.so.6.1(__libc_start_main-0x9f56b0)[0x2000000000454590] > [altix:20419] [21] examples/ring_c[0x4000000000000a20] > [altix:20419] *** End of error message *** > /lib/libc.so.6.1(_IO_vfprintf-0x998610)[0x20000000004b15d0] > [altix:20418] [ 3] /lib/libc.so.6.1(+0x82860)[0x20000000004b2860] > [altix:20418] [ 4] > /lib/libc.so.6.1(_IO_vfprintf-0x99f140)[0x20000000004aaaa0] > > > > > On Thu, Jul 31, 2014 at 11:47 PM, Paul Hargrove <phhargr...@lbl.gov> > <phhargr...@lbl.gov> wrote: > > > Gilles's findings are consistent with mine which showed the SEGVs to be in > the coll/ml code. > I've built with --enable-debug and so below is a backtrace (well, two > actually) that might be helpful. > Unfortunately the output of the two ranks did get slightly entangled. > > -Paul > > $ ../INST/bin/mpirun -mca btl sm,self -np 2 examples/ring_c' > [bd-login][[43502,1],0][/home/hargrov1/OMPI/openmpi-trunk-linux-ppc64-gcc/openmpi-1.9a1r32369/ompi/mca/coll/ml/coll_ml_lmngr.c:231:mca_coll_ml_lmngr_init] > COLL-ML [bd-login:09106] *** Process received signal *** > [bd-login][[43502,1],1][/home/hargrov1/OMPI/openmpi-trunk-linux-ppc64-gcc/openmpi-1.9a1r32369/ompi/mca/coll/ml/coll_ml_lmngr.c:231:mca_coll_ml_lmngr_init] > COLL-ML [bd-login:09107] *** Process received signal *** > [bd-login:09107] Signal: Segmentation fault (11) > [bd-login:09107] Signal code: Address not mapped (1) > [bd-login:09107] Failing at address: 0x10 > [bd-login:09107] [ 0] [bd-login:09106] Signal: Segmentation fault (11) > [bd-login:09106] Signal code: Address not mapped (1) > [bd-login:09106] Failing at address: 0x10 > [bd-login:09106] [ 0] [0xfffa96c0418] > [bd-login:09106] [ 1] [0xfff8f580418] > [bd-login:09107] [ 1] /lib64/libc.so.6(_IO_vfprintf-0x157168)[0x80c9b5b968] > [bd-login:09107] [ 2] /lib64/libc.so.6(_IO_vfprintf-0x157168)[0x80c9b5b968] > [bd-login:09106] [ 2] /lib64/libc.so.6[0x80c9b600b4] > [bd-login:09106] [ 3] /lib64/libc.so.6[0x80c9b600b4] > [bd-login:09107] [ 3] /lib64/libc.so.6(_IO_vfprintf-0x157010)[0x80c9b5bac0] > [bd-login:09107] [ 4] /lib64/libc.so.6(_IO_vfprintf-0x157010)[0x80c9b5bac0] > [bd-login:09106] [ 4] > /home/hargrov1/OMPI/openmpi-trunk-linux-ppc64-gcc/INST/lib/openmpi/mca_coll_ml.so(+0x66580)[0xfffa8296580] > [bd-login:09106] [ 5] > /home/hargrov1/OMPI/openmpi-trunk-linux-ppc64-gcc/INST/lib/openmpi/mca_coll_ml.so(+0x67604)[0xfffa8297604] > [bd-login:09106] [ 6] > /home/hargrov1/OMPI/openmpi-trunk-linux-ppc64-gcc/INST/lib/openmpi/mca_coll_ml.so(mca_coll_ml_lmngr_alloc-0x1af04)[0xfffa829784c] > [bd-login:09106] [ 7] > /home/hargrov1/OMPI/openmpi-trunk-linux-ppc64-gcc/INST/lib/openmpi/mca_coll_ml.so(mca_coll_ml_allocate_block-0x607b4)[0xfffa8250d4c] > [bd-login:09106] > /home/hargrov1/OMPI/openmpi-trunk-linux-ppc64-gcc/INST/lib/openmpi/mca_coll_ml.so(+0x66580)[0xfff8e156580] > [bd-login:09107] [ 5] > /home/hargrov1/OMPI/openmpi-trunk-linux-ppc64-gcc/INST/lib/openmpi/mca_coll_ml.so(+0x67604)[0xfff8e157604] > [bd-login:09107] [ 6] > /home/hargrov1/OMPI/openmpi-trunk-linux-ppc64-gcc/INST/lib/openmpi/mca_coll_ml.so(mca_coll_ml_lmngr_alloc-0x1af04)[0xfff8e15784c] > [bd-login:09107] [ 7] > /home/hargrov1/OMPI/openmpi-trunk-linux-ppc64-gcc/INST/lib/openmpi/mca_coll_ml.so(mca_coll_ml_allocate_block-0x607b4)[0xfff8e110d4c] > [bd-login:09107] [ 8] > /home/hargrov1/OMPI/openmpi-trunk-linux-ppc64-gcc/INST/lib/openmpi/mca_coll_ml.so(+0x165e4)[0xfff8e1065e4] > [bd-login:09107] [ 9] > /home/hargrov1/OMPI/openmpi-trunk-linux-ppc64-gcc/INST/lib/openmpi/mca_coll_ml.so(+0x1a7d8)[0xfff8e10a7d8] > [bd-login:09107] [10] > /home/hargrov1/OMPI/openmpi-trunk-linux-ppc64-gcc/INST/lib/openmpi/mca_coll_ml.so(mca_coll_ml_comm_query-0x61b50)[0xfff8e10f970] > [bd-login:09107] [11] [ 8] > /home/hargrov1/OMPI/openmpi-trunk-linux-ppc64-gcc/INST/lib/openmpi/mca_coll_ml.so(+0x165e4)[0xfffa82465e4] > [bd-login:09106] [ 9] > /home/hargrov1/OMPI/openmpi-trunk-linux-ppc64-gcc/INST/lib/openmpi/mca_coll_ml.so(+0x1a7d8)[0xfffa824a7d8] > [bd-login:09106] [10] > /home/hargrov1/OMPI/openmpi-trunk-linux-ppc64-gcc/INST/lib/openmpi/mca_coll_ml.so(mca_coll_ml_comm_query-0x61b50)[0xfffa824f970] > [bd-login:09106] [11] > /home/hargrov1/OMPI/openmpi-trunk-linux-ppc64-gcc/INST/lib/libmpi.so.0(+0x165ba0)[0xfff8f4b5ba0] > [bd-login:09107] [12] > /home/hargrov1/OMPI/openmpi-trunk-linux-ppc64-gcc/INST/lib/libmpi.so.0(+0x165b14)[0xfff8f4b5b14] > [bd-login:09107] [13] > /home/hargrov1/OMPI/openmpi-trunk-linux-ppc64-gcc/INST/lib/libmpi.so.0(+0x165ba0)[0xfffa95f5ba0] > [bd-login:09106] [12] > /home/hargrov1/OMPI/openmpi-trunk-linux-ppc64-gcc/INST/lib/libmpi.so.0(+0x165b14)[0xfffa95f5b14] > [bd-login:09106] [13] > /home/hargrov1/OMPI/openmpi-trunk-linux-ppc64-gcc/INST/lib/libmpi.so.0(+0x1659a8)[0xfffa95f59a8] > [bd-login:09106] [14] > /home/hargrov1/OMPI/openmpi-trunk-linux-ppc64-gcc/INST/lib/libmpi.so.0(+0x1659a8)[0xfff8f4b59a8] > [bd-login:09107] [14] > /home/hargrov1/OMPI/openmpi-trunk-linux-ppc64-gcc/INST/lib/libmpi.so.0(+0x1657ac)[0xfffa95f57ac] > [bd-login:09106] [15] > /home/hargrov1/OMPI/openmpi-trunk-linux-ppc64-gcc/INST/lib/libmpi.so.0(+0x1657ac)[0xfff8f4b57ac] > [bd-login:09107] [15] > /home/hargrov1/OMPI/openmpi-trunk-linux-ppc64-gcc/INST/lib/libmpi.so.0(mca_coll_base_comm_select-0x9b89c)[0xfff8f4ae3ec] > [bd-login:09107] > /home/hargrov1/OMPI/openmpi-trunk-linux-ppc64-gcc/INST/lib/libmpi.so.0(mca_coll_base_comm_select-0x9b89c)[0xfffa95ee3ec] > [bd-login:09106] [16] [16] > /home/hargrov1/OMPI/openmpi-trunk-linux-ppc64-gcc/INST/lib/libmpi.so.0(ompi_mpi_init-0x13f790)[0xfff8f401408] > [bd-login:09107] [17] > /home/hargrov1/OMPI/openmpi-trunk-linux-ppc64-gcc/INST/lib/libmpi.so.0(ompi_mpi_init-0x13f790)[0xfffa9541408] > [bd-login:09106] [17] > /home/hargrov1/OMPI/openmpi-trunk-linux-ppc64-gcc/INST/lib/libmpi.so.0(MPI_Init-0xf28d4)[0xfffa9591c74] > [bd-login:09106] [18] examples/ring_c[0x1000099c] > [bd-login:09106] [19] > /home/hargrov1/OMPI/openmpi-trunk-linux-ppc64-gcc/INST/lib/libmpi.so.0(MPI_Init-0xf28d4)[0xfff8f451c74] > [bd-login:09107] [18] examples/ring_c[0x1000099c] > [bd-login:09107] [19] /lib64/libc.so.6[0x80c9b2bcd8] > [bd-login:09107] [20] /lib64/libc.so.6[0x80c9b2bcd8] > [bd-login:09106] [20] > /lib64/libc.so.6(__libc_start_main-0x184e00)[0x80c9b2bed0] > [bd-login:09107] *** End of error message *** > /lib64/libc.so.6(__libc_start_main-0x184e00)[0x80c9b2bed0] > [bd-login:09106] *** End of error message *** > -------------------------------------------------------------------------- > mpirun noticed that process rank 0 with PID 0 on node bd-login exited on > signal 11 (Segmentation fault). > -------------------------------------------------------------------------- > > > > > > > > On Thu, Jul 31, 2014 at 11:39 PM, Gilles Gouaillardet > <gilles.gouaillar...@iferc.org> wrote: > > > Paul and Ralph, > > for what it's worth : > > a) i faced the very same issue on my (sloooow) qemu emulated ppc64 vm > b) i was able to run very basic programs when passing --mca coll ^ml to > mpirun > > Cheers, > > Gilles > > On 2014/08/01 12:30, Ralph Castain wrote: > > Yes, I fear this will require some effort to chase all the breakage down > given that (to my knowledge, at least) we lack PPC machines in the devel > group. > > > On Jul 31, 2014, at 5:46 PM, Paul Hargrove <phhargr...@lbl.gov> > <phhargr...@lbl.gov> <phhargr...@lbl.gov> <phhargr...@lbl.gov> wrote: > > > On the path to verifying George's atomics patch, I have started just by > verifying that I can still build the UNPATCHED trunk on each of the platforms > I listed. > > I have tried two PPC64/Linux systems so far and am seeing the same problem on > both. Though I can pass "make check" both platforms SEGV on > mpirun -mca btl sm,self -np 2 examples/ring_c > > Is this the expected state of the trunk on big-endian systems? > I am thinking in particular of > http://www.open-mpi.org/community/lists/devel/2014/07/15365.php in which > Ralph wrote: > > Yeah, my fix won't work for big endian machines - this is going to be an > issue across the > code base now, so we'll have to troll and fix it. I was doing the minimal > change required to > fix the trunk in the meantime. > > If this big-endian failure is not known/expected let me know and I'll > provide details. > Since testing George's patch only requires "make check" I can proceed with > that regardless. > > -Paul > > > On Thu, Jul 31, 2014 at 4:25 PM, George Bosilca <bosi...@icl.utk.edu> > <bosi...@icl.utk.edu> <bosi...@icl.utk.edu> <bosi...@icl.utk.edu> wrote: > Awesome, thanks Paul. When the results will be in we will fix whatever is > needed for these less common architectures. > > George. > > > > On Thu, Jul 31, 2014 at 7:24 PM, Paul Hargrove <phhargr...@lbl.gov> > <phhargr...@lbl.gov> <phhargr...@lbl.gov> <phhargr...@lbl.gov> wrote: > > > On Thu, Jul 31, 2014 at 4:22 PM, Paul Hargrove <phhargr...@lbl.gov> > <phhargr...@lbl.gov> <phhargr...@lbl.gov> <phhargr...@lbl.gov> wrote: > > On Thu, Jul 31, 2014 at 4:13 PM, George Bosilca <bosi...@icl.utk.edu> > <bosi...@icl.utk.edu> <bosi...@icl.utk.edu> <bosi...@icl.utk.edu> wrote: > Paul, I know you have a pretty diverse range computers. Can you try to > compile and run a "make check" with the following patch? > > I will see what I can do for ARMv7, MIPS, PPC and IA64 (or whatever subset of > those is still supported). > The ARM and MIPS system are emulators and take forever to build OMPI. > However, I am not even sure how soon I'll get to start this testing. > > > Add SPARC (v8plus and v9) to that list. > > > > -- > Paul H. Hargrove phhargr...@lbl.gov > Future Technologies Group > Computer and Data Sciences Department Tel: +1-510-495-2352 > Lawrence Berkeley National Laboratory Fax: +1-510-486-6900 > > _______________________________________________ > devel mailing listde...@open-mpi.org > Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/devel > Link to this post: > http://www.open-mpi.org/community/lists/devel/2014/07/15411.php > > > _______________________________________________ > devel mailing listde...@open-mpi.org > Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/devel > Link to this post: > http://www.open-mpi.org/community/lists/devel/2014/07/15412.php > > > > -- > Paul H. Hargrove phhargr...@lbl.gov > Future Technologies Group > Computer and Data Sciences Department Tel: +1-510-495-2352 > Lawrence Berkeley National Laboratory Fax: +1-510-486-6900 > _______________________________________________ > devel mailing listde...@open-mpi.org > Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/devel > Link to this post: > http://www.open-mpi.org/community/lists/devel/2014/07/15414.php > > > > _______________________________________________ > devel mailing listde...@open-mpi.org > Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/devel > Link to this post: > http://www.open-mpi.org/community/lists/devel/2014/07/15425.php > > > > _______________________________________________ > devel mailing listde...@open-mpi.org > Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/devel > Link to this > post:http://www.open-mpi.org/community/lists/devel/2014/08/15436.php > > -- > Paul H. Hargrove phhargr...@lbl.gov > Future Technologies Group > Computer and Data Sciences Department Tel: +1-510-495-2352 > Lawrence Berkeley National Laboratory Fax: +1-510-486-6900 > > > > _______________________________________________ > devel mailing listde...@open-mpi.org > Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/devel > Link to this post: > http://www.open-mpi.org/community/lists/devel/2014/08/15438.php > > > > _______________________________________________ > devel mailing list > de...@open-mpi.org > Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/devel > Link to this post: > http://www.open-mpi.org/community/lists/devel/2014/08/15439.php > -- Paul H. Hargrove phhargr...@lbl.gov Future Technologies Group Computer and Data Sciences Department Tel: +1-510-495-2352 Lawrence Berkeley National Laboratory Fax: +1-510-486-6900