[OMPI users] OMPI Coll Framework and RDMA

Hi, I have a quick question. Is there an openib (in btl framework) equivalent in coll framework? I have an MPI application with gatherv and scatterv. I am wondering if I can leverage RDMA capabilities of the underlying Infiniband fabric. Thanks, -- Joba

Re: [OMPI users] OMPI Coll Framework and RDMA

Division > Oak Ridge National Laboratory > > > > > > > On Jun 6, 2013, at 1:59 PM, Jingcha Joba <pukkimon...@gmail.com pukkimon...@gmail.com>> wrote: > > Hi, > > I have a quick question. > > Is there an openib (in btl framework) equivalent in coll f

Re: [OMPI users] OMPI Coll Framework and RDMA

(p2p layer) that is > implemented on top of BTL. > Consequently it laverages RDMA capabilities to some extend. > > Pavel (Pasha) Shamis > --- > Computer Science Research Group > Computer Science and Math Division > Oak Ridge National Laboratory > > > > >

Re: [OMPI users] Problem running an mpi applicatio​n on nodes with more than one interface

m. >>> Trying to run mpi between 2 separate machines, with each machine having >>> 2 ethernet ports, causes really weird behaviour on the most basic code. >>> I had to disable one of the ethernet ports on each of the machines >>> and it worked just fine after that. No

Re: [OMPI users] Could not execute the executable "/home/MET/hrm/bin/hostlist": Exec format error

Just to be sure, can u try echo "${MPIRUN} -np ${NPROC} -hostfile ${ABSDIR}/hostlist -mca btl sm,openib,self --mca btl_openib_use_srq 1 ./hrm >> ${OUTFILE}_hrm 2>&1" and check if you are indeed getting the correct argument. If that looks fine, can u add --mca btl_openib_verbose 1 to the mpirun

Re: [OMPI users] Could not execute the executable "/home/MET/hrm/bin/hostlist": Exec format error

iles]$ echo ${MPIRUN} -np ${NPROC} -hostfile > $i{ABSDIR}/hostlist -mca btl sm,openib,self --mca btl_openib_use_srq 1 > ./hrm >> ${OUTFILE}_hrm 2>&1 > [pmdtest@pmd02 d00_dayfiles]$ > > > On Wed, Feb 29, 2012 at 12:01 PM, Jingcha Joba <pukkimon...@gmail.com>wrote:

Re: [OMPI users] [EXTERNAL] Re: Question regarding osu-benchamarks 3.1.1

When I ran my osu tests , I was able to get the numbers out of all the tests except latency_mt (which was obvious, as I didnt compile open-mpi with multi threaded support). A good way to know if the problem is with openmpi or with your custom OFED stack would be to use some other device like tcp

Re: [OMPI users] Very slow MPI_GATHER

two things: 1. Too many mpi processes on one node leading to processes pre-empting each other 2. Contention in your network. On Wed, Feb 29, 2012 at 8:01 AM, Pinero, Pedro_jose < pedro_jose.pin...@atmel.com> wrote: > Hi, > > ** ** > > I am using OMPI v.1.5.5 to communicate 200 Processes in a

Re: [OMPI users] [EXTERNAL] Re: Question regarding osu-benchamarks 3.1.1

and generally made it much mo'betta. Brian > is re-implementing that stuff, and I believe it'll also be much mo'betta. > > My point: I wouldn't worry if not all one-sided benchmarks run with OMPI. > No one uses them (yet) anyway. > > > On Feb 29, 2012, at 1:42 PM, Jingcha Joba w

Re: [OMPI users] [EXTERNAL] Re: Question regarding osu-benchamarks 3.1.1

"hangs" when sending a message more than 512KB (or may be 1MB)? Also is there a way to know if for a particular MPI call, OF uses send/recv or RDMA exchange? On Wed, Feb 29, 2012 at 11:36 AM, Jeffrey Squyres <jsquy...@cisco.com>wrote: > On Feb 29, 2012, at 2:30 PM, Jingcha Joba w

Re: [OMPI users] [EXTERNAL] Re: Question regarding osu-benchamarks 3.1.1

Aah... So when openMPI is compile with OFED, and run on a Infiniband/RoCE devices, I would use the mpi would simply direct to ofed to do point to point calls in the ofed way? > > More specifically: all things being equal, you don't care which is used. > You just want your message to get to the

Re: [OMPI users] [EXTERNAL] Re: Question regarding osu-benchamarks 3.1.1

lt;jsquy...@cisco.com>wrote: > >> On Feb 29, 2012, at 2:57 PM, Jingcha Joba wrote: >> >> > So if I understand correctly, if a message size is smaller than it will >> use the MPI way (non-RDMA, 2 way communication), if its larger, then it >> would use the

Re: [OMPI users] Redefine proc in cartesian topologies

mpirun -np 4 --host node1,node2,node1,node2 ./app Is this what you want? On Thu, Mar 1, 2012 at 10:57 AM, Claudio Pastorino < claudio.pastor...@gmail.com> wrote: > Dear all, > I apologize in advance if this is not the right list to post this. I > am a newcomer and please let me know if I should

Re: [OMPI users] MPI doesn't recognize multiple cores available on multicore machines

Try using slots in hostfile ? -- Sent from my iPhone On Apr 24, 2012, at 2:52 PM, Kyle Boe wrote: > I'm having a problem trying to use OpenMPI on some multicore machines I have. > The code I am running was giving me errors which suggested that MPI was > assigning multiple

Re: [OMPI users] MPI doesn't recognize multiple cores available on multicore machines

My bad. I did not read the bottom part of the email. Not sure If this would help, but can u try, --mca btl sm,self ? -- Sent from my iPhone On Apr 24, 2012, at 3:46 PM, Kyle Boe wrote: > Right, I tried using a hostfile, and it made no difference. This is running > OpenMPI

Re: [OMPI users] OpenMPI and CUDA on cluster

Best way to find if the application binary can "see" the cuda library is to do ldd. On Tue, May 1, 2012 at 2:03 AM, Rohan Deshpande wrote: > Hi, > > I am trying to execute OpenMPI and CUDA code on a cluster. The code works > fine on single machine but when I try to

Re: [OMPI users] Regarding the execution time calculation

Not related to this question , but just curious, is Wtime context switch safe ? -- Sent from my iPhone On May 3, 2012, at 4:48 AM, seshendra seshu wrote: > Hi, > I have written an parallel program and when i run my program on 4,8,16 nodes > and calculated the execution

Re: [OMPI users] Regarding the execution time calculation

<jsquy...@cisco.com> wrote: > On May 3, 2012, at 2:02 PM, Jingcha Joba wrote: > > > Not related to this question , but just curious, is Wtime context switch > safe ? > > Not sure exactly what you're asking here...? > > -- > Jeff Squyres > jsquy...@cisco.c

Re: [OMPI users] Regarding the execution time calculation

rote: > ** > MPI_Wtime() returns the elapsed time since some arbitrary time in the > past. It is a measure of "wallclock" time, not of CPU time or anything. > > > On 5/4/2012 3:08 PM, Jingcha Joba wrote: > > Lets say I have a code like this > > start = MPI_Wti

Re: [OMPI users] Regarding the execution time calculation

Jeff, So in the above stated example, end-start will be: + 20ms ? (time slice of P2 + P3 = 20ms) On Mon, May 7, 2012 at 1:42 PM, Jeff Squyres (jsquyres) <jsquy...@cisco.com>wrote: > On May 7, 2012, at 2:39 PM, Jingcha Joba wrote: > > > OK.This explains that if a process ge

Re: [OMPI users] Regarding the execution time calculation

True. I was curious as to what happens when I am time sharing the CPU. -- Sent from my iPhone On May 8, 2012, at 3:11 AM, TERRY DONTJE <terry.don...@oracle.com> wrote: > On 5/7/2012 8:40 PM, Jeff Squyres (jsquyres) wrote: >> >> On May 7, 2012, at 8:31 PM, Jingcha J

Re: [OMPI users] Returned "Unreachable" (-12) instead of "Success" (0)

Not sure what the solution is, but ompi does not bind to privilege ports. In fact , ompi_info --param btl tcp | grep port_min should show that the minimum port where TCP BTL will try to bind (by default ) is 1024. -- Sent from my iPhone On May 18, 2012, at 1:02 AM, devendra rai

Re: [OMPI users] Intra-node communication

This should not happen. Typically, Intra node communication latency are way way cheaper than inter node. Can you please tell us how u ran your application ? Thanks -- Sent from my iPhone On Jun 1, 2012, at 7:34 AM, Mudassar Majeed wrote: > Dear MPI people, >

Re: [OMPI users] testing for openMPI

Hello Duke, Welcome to the forum. The way openmpi schedules by default is to fill all the slots in a host, before moving on to next host. Check this link for some info: http://www.open-mpi.org/faq/?category=running#mpirun-scheduling -- Jingcha On Thu, Jun 7, 2012 at 2:11 AM, Duke

Re: [OMPI users] Help with buidling MPI(Error: mpi.h not found)

mpi.h shoud be located under /include/ When you compile your hello.c include that path with -I. On Tue, Jun 12, 2012 at 10:25 AM, wrote: > Hi, > > ** ** > > I was directed to the OpenMPI website from the Boost Libraries page to > install an MPI Installation. >

Re: [OMPI users] Newbie question?

John, BTL refers to Byte Transfer Layer, a framework to send/receive point to point messages on different network. It has several components (implementations) like openib, tcp, mx, shared mem, etc. ^openib means "not" to use openib component for p2p messages. On a side note, do you have an

Re: [OMPI users] Newbie question?

> > On a side note, do you have an RDMA supporting device ( > > Infiniband/RoCE/iWarp) ? > > I'm just an engineer trying to get something to work on an AMD dual core > notebook for the powers-that-be at a small engineering concern (all MEs) in > Huntsville, AL - i.e., NASA work. > If on a

Re: [OMPI users] High cpu usage

Hi , First, I don't see any cpu utilization but %time (of a function wrt others in a process/application). Generally for high cpu utilization, there could be many reason. Two of them that comes to my mind is, 1. Depends on the network stack, eg. the "tcp" way will use more CPU than the