[hwloc-devel] Create success (hwloc r1.1a1r2663)
Creating nightly hwloc snapshot SVN tarball was a success. Snapshot: hwloc 1.1a1r2663 Start time: Fri Oct 29 21:01:04 EDT 2010 End time: Fri Oct 29 21:03:21 EDT 2010 Your friendly daemon, Cyrador
Re: [hwloc-devel] Strange results on itanium 2
On Saturday, October 30, 2010 12:24:14 am Brice Goglin wrote: > Le 30/10/2010 00:01, Jirka Hladky a écrit : > > On Friday, October 29, 2010 10:59:25 pm Brice Goglin wrote: > >> Le 29/10/2010 21:57, Jirka Hladky a écrit : > >>> Hi Samuel, > >>> > >>> I have attached the output of > >>> tests/linux/gather-topology.sh `uname --kernel-release`_`uname -- > >>> nodename`_gather-topology > >>> > >>> I'm sorry for the long delay - systems has been used by somebody else, > >>> I had to wait for it to be free. > >>> > >>> System is running kernel 2.6.18-227.el5 (RHEL 5.6). ia64 is not > >>> supported on RHEL 6.0 so I cannot really test it on the new kernel. > >>> > >>> It would be really interesting if you can recognize if it's a kernel > >>> bug or hwloc problem. > >> > >> /sys/devices/system/cpu/cpu*/cache/index4/shared_cpu_map contains an > >> empty map. This index4 is a L3 cache. But this map means that this cache > >> is near none of the cores... The instruction L2 has the same problem > >> (index3 instead of 4). This is a kernel bug. > >> > >> But, we already have a dedicated work-around in hwloc > >> > >> (src/topology-linux.c): > >> if (hwloc_bitmap_weight(cacheset) < 1) > >> > >> /* mask is wrong (happens on ia64), assumes it's not shared > >> */ hwloc_bitmap_only(cacheset, i); > >> > >> This work-around worked fine on old itaniums since they had one L3, one > >> L2 and one L1 per core. Your machine has hyperthreading, so our > >> work-around creates one L3 per thread, while L1 and L2 (properly > >> reported by the kernel) are core-specific. Maybe hwloc should just > >> ignore caches with invalid shared_cpu_map. > >> > >> Brice > >> > >> ___ > >> hwloc-devel mailing list > >> hwloc-de...@open-mpi.org > >> http://www.open-mpi.org/mailman/listinfo.cgi/hwloc-devel > > > > Hi Brice, > > > > thanks for looking into it! I'm going to open a BZ for it and put you on > > the Cc. > > Thanks. Bug 647949 - Wrong /sys/devices/system/cpu/cpu*/cache/index4/shared_cpu_map for L3 and L2 cache on HP Integrity BL870c box with 2 Intel Itanium2 9140N processors https://bugzilla.redhat.com/show_bug.cgi?id=647949 I put you on Cc. > > > BTW, is there some documentation on /sys/devices/system/cpu/* tree? > > There's some doc in Documentation/ABI/*/sysfs-devices-* inside the > kernel source tree. Thanks! BTW, it's Intel Itanium2 9140N CPU with 18MB of L3 cache. Kernel reports only 9MB of L3 cache. It's another bug. Bonne nuit Jirka
Re: [hwloc-devel] Strange results on itanium 2
Le 30/10/2010 00:01, Jirka Hladky a écrit : > On Friday, October 29, 2010 10:59:25 pm Brice Goglin wrote: > >> Le 29/10/2010 21:57, Jirka Hladky a écrit : >> >>> Hi Samuel, >>> >>> I have attached the output of >>> tests/linux/gather-topology.sh `uname --kernel-release`_`uname -- >>> nodename`_gather-topology >>> >>> I'm sorry for the long delay - systems has been used by somebody else, I >>> had to wait for it to be free. >>> >>> System is running kernel 2.6.18-227.el5 (RHEL 5.6). ia64 is not supported >>> on RHEL 6.0 so I cannot really test it on the new kernel. >>> >>> It would be really interesting if you can recognize if it's a kernel bug >>> or hwloc problem. >>> >> /sys/devices/system/cpu/cpu*/cache/index4/shared_cpu_map contains an >> empty map. This index4 is a L3 cache. But this map means that this cache >> is near none of the cores... The instruction L2 has the same problem >> (index3 instead of 4). This is a kernel bug. >> >> But, we already have a dedicated work-around in hwloc >> (src/topology-linux.c): >> if (hwloc_bitmap_weight(cacheset) < 1) >> /* mask is wrong (happens on ia64), assumes it's not shared */ >> hwloc_bitmap_only(cacheset, i); >> >> This work-around worked fine on old itaniums since they had one L3, one >> L2 and one L1 per core. Your machine has hyperthreading, so our >> work-around creates one L3 per thread, while L1 and L2 (properly >> reported by the kernel) are core-specific. Maybe hwloc should just >> ignore caches with invalid shared_cpu_map. >> >> Brice >> >> ___ >> hwloc-devel mailing list >> hwloc-de...@open-mpi.org >> http://www.open-mpi.org/mailman/listinfo.cgi/hwloc-devel >> > > Hi Brice, > > thanks for looking into it! I'm going to open a BZ for it and put you on the > Cc. > Thanks. > BTW, is there some documentation on /sys/devices/system/cpu/* tree? > There's some doc in Documentation/ABI/*/sysfs-devices-* inside the kernel source tree. Brice
Re: [hwloc-devel] Strange results on itanium 2
On Friday, October 29, 2010 10:59:25 pm Brice Goglin wrote: > Le 29/10/2010 21:57, Jirka Hladky a écrit : > > Hi Samuel, > > > > I have attached the output of > > tests/linux/gather-topology.sh `uname --kernel-release`_`uname -- > > nodename`_gather-topology > > > > I'm sorry for the long delay - systems has been used by somebody else, I > > had to wait for it to be free. > > > > System is running kernel 2.6.18-227.el5 (RHEL 5.6). ia64 is not supported > > on RHEL 6.0 so I cannot really test it on the new kernel. > > > > It would be really interesting if you can recognize if it's a kernel bug > > or hwloc problem. > > /sys/devices/system/cpu/cpu*/cache/index4/shared_cpu_map contains an > empty map. This index4 is a L3 cache. But this map means that this cache > is near none of the cores... The instruction L2 has the same problem > (index3 instead of 4). This is a kernel bug. > > But, we already have a dedicated work-around in hwloc > (src/topology-linux.c): > if (hwloc_bitmap_weight(cacheset) < 1) > /* mask is wrong (happens on ia64), assumes it's not shared */ > hwloc_bitmap_only(cacheset, i); > > This work-around worked fine on old itaniums since they had one L3, one > L2 and one L1 per core. Your machine has hyperthreading, so our > work-around creates one L3 per thread, while L1 and L2 (properly > reported by the kernel) are core-specific. Maybe hwloc should just > ignore caches with invalid shared_cpu_map. > > Brice > > ___ > hwloc-devel mailing list > hwloc-de...@open-mpi.org > http://www.open-mpi.org/mailman/listinfo.cgi/hwloc-devel Hi Brice, thanks for looking into it! I'm going to open a BZ for it and put you on the Cc. BTW, is there some documentation on /sys/devices/system/cpu/* tree? Thanks! Jirka
Re: [OMPI devel] Cost for AllGatherV() Operation
Tim, The collective in Open MPI works differently than in MPICH. They are dynamically selected based on the number of processes involved and the amount of data to be exchanged. Therefore, it is difficult to answer your question without knowing this information. There are 4 algorithms for MPI_Allgather in Open MPI: - recursive doubling - Bruck - ring - neighbor exchange I think their complexity is described in "Performance analysis of MPI collective operations" (http://www.springerlink.com/content/542207241006p64h/). george. On Oct 29, 2010, at 15:42 , Tim Stitt wrote: > Dear OpenMPI Developers, > > I would be grateful if someone could briefly describe the cost (complexity) > for the allgatherv() collective operation in the current release of OpenMPI. > > For MPICH2 I believe the cost is ceiling(lg p). Can anyone comment on the > algorithms and cost used in the OpenMPI implementation? > > Thanks in advance, > > Tim. > ___ > devel mailing list > de...@open-mpi.org > http://www.open-mpi.org/mailman/listinfo.cgi/devel
Re: [hwloc-devel] Strange results on itanium 2
Hi Samuel, I have attached the output of tests/linux/gather-topology.sh `uname --kernel-release`_`uname -- nodename`_gather-topology I'm sorry for the long delay - systems has been used by somebody else, I had to wait for it to be free. System is running kernel 2.6.18-227.el5 (RHEL 5.6). ia64 is not supported on RHEL 6.0 so I cannot really test it on the new kernel. It would be really interesting if you can recognize if it's a kernel bug or hwloc problem. Thanks a lot! Jirka On Wednesday, October 27, 2010 02:59:30 pm Samuel Thibault wrote: > Hello, > > Jirka Hladky, le Wed 27 Oct 2010 14:53:01 +0200, a écrit : > > I have attached a tar file with full information. > > Could you also post the result of test/linux/gather-topology.sh ? > > Samuel > ___ > hwloc-devel mailing list > hwloc-de...@open-mpi.org > http://www.open-mpi.org/mailman/listinfo.cgi/hwloc-devel 2.6.18-227.el5_hp-bl870c-01.rhts.eng.bos.redhat.com_gather-topology.tar.gz Description: application/compressed-tar Machine (phys=0 local=16591312KB total=16591312KB) Socket #0 (phys=0) L2Cache #0 (256KB) L1Cache #0 (16KB) Core #0 (phys=0) L3Cache #0 (9216KB) PU #0 (phys=0) L3Cache #1 (9216KB) PU #1 (phys=1) L2Cache #1 (256KB) L1Cache #1 (16KB) Core #1 (phys=1) L3Cache #2 (9216KB) PU #2 (phys=2) L3Cache #3 (9216KB) PU #3 (phys=3) Socket #1 (phys=1) L2Cache #2 (256KB) L1Cache #2 (16KB) Core #2 (phys=0) L3Cache #4 (9216KB) PU #4 (phys=4) L3Cache #5 (9216KB) PU #5 (phys=5) L2Cache #3 (256KB) L1Cache #3 (16KB) Core #3 (phys=1) L3Cache #6 (9216KB) PU #6 (phys=6) L3Cache #7 (9216KB) PU #7 (phys=7) depth 0:1 Machine (type #1) depth 1: 2 Sockets (type #3) depth 2: 4 Caches (type #4) depth 3: 4 Caches (type #4) depth 4:4 Cores (type #5) depth 5: 8 Caches (type #4) depth 6: 8 PUs (type #6) Topology not from this system
[OMPI devel] Cost for AllGatherV() Operation
Dear OpenMPI Developers, I would be grateful if someone could briefly describe the cost (complexity) for the allgatherv() collective operation in the current release of OpenMPI. For MPICH2 I believe the cost is ceiling(lg p). Can anyone comment on the algorithms and cost used in the OpenMPI implementation? Thanks in advance, Tim.
Re: [OMPI devel] === CREATE FAILURE (trunk) ===
I have fixes for this, but they're .m4 changes (stupid VPATH stuff; sorry) -- so I'll commit them tonight after 6pm US Eastern. On Oct 28, 2010, at 9:16 PM, MPI Team wrote: > > ERROR: Command returned a non-zero exist status (trunk): > make distcheck > > Start time: Thu Oct 28 21:00:05 EDT 2010 > End time: Thu Oct 28 21:16:19 EDT 2010 > > === > [... previous lines snipped ...] > checking for OPAL CXXFLAGS... -pthread > checking for OPAL CXXFLAGS_PREFIX... > checking for OPAL LDFLAGS... > checking for OPAL LIBS... -ldl -Wl,--export-dynamic -lrt -lnsl -lutil -lm > -ldl > checking for OPAL extra include dirs... > checking for ORTE CPPFLAGS... > checking for ORTE CXXFLAGS... -pthread > checking for ORTE CXXFLAGS_PREFIX... > checking for ORTE CFLAGS... -pthread > checking for ORTE CFLAGS_PREFIX... > checking for ORTE LDFLAGS... > checking for ORTE LIBS... -ldl -Wl,--export-dynamic -lrt -lnsl -lutil -lm > -ldl > checking for ORTE extra include dirs... > checking for OMPI CPPFLAGS... > checking for OMPI CFLAGS... -pthread > checking for OMPI CFLAGS_PREFIX... > checking for OMPI CXXFLAGS... -pthread > checking for OMPI CXXFLAGS_PREFIX... > checking for OMPI FFLAGS... -pthread > checking for OMPI FFLAGS_PREFIX... > checking for OMPI FCFLAGS... -pthread > checking for OMPI FCFLAGS_PREFIX... > checking for OMPI LDFLAGS... > checking for OMPI LIBS... -ldl -Wl,--export-dynamic -lrt -lnsl -lutil -lm > -ldl > checking for OMPI extra include dirs... > > *** Final output > configure: creating ./config.status > config.status: creating ompi/include/ompi/version.h > config.status: creating orte/include/orte/version.h > config.status: creating opal/include/opal/version.h > config.status: creating opal/mca/backtrace/Makefile > config.status: creating opal/mca/backtrace/printstack/Makefile > config.status: creating opal/mca/backtrace/execinfo/Makefile > config.status: creating opal/mca/backtrace/darwin/Makefile > config.status: creating opal/mca/backtrace/none/Makefile > config.status: creating opal/mca/carto/Makefile > config.status: creating opal/mca/carto/auto_detect/Makefile > config.status: creating opal/mca/carto/file/Makefile > config.status: creating opal/mca/compress/Makefile > config.status: creating opal/mca/compress/gzip/Makefile > config.status: creating opal/mca/compress/bzip/Makefile > config.status: creating opal/mca/crs/Makefile > config.status: creating opal/mca/crs/none/Makefile > config.status: creating opal/mca/crs/self/Makefile > config.status: creating opal/mca/crs/blcr/Makefile > config.status: creating opal/mca/event/Makefile > config.status: creating opal/mca/event/libevent207/Makefile > config.status: error: cannot find input file: > `opal/mca/event/libevent207/libevent/include/event2/event-config.h.in' > make: *** [distcheck] Error 1 > === > > Your friendly daemon, > Cyrador > ___ > testing mailing list > test...@open-mpi.org > http://www.open-mpi.org/mailman/listinfo.cgi/testing -- Jeff Squyres jsquy...@cisco.com For corporate legal information go to: http://www.cisco.com/web/about/doing_business/legal/cri/
