Re: [OMPI devel] Fwd: [OMPI users] OpenMPI with openib partitions

2008-10-07 Thread Jeff Squyres
"Soon." :-) On Oct 7, 2008, at 1:30 PM, Matt Burgess wrote: Pasha, That's great, thanks for the help. When exactly do you expect that 1.2.8 will be released? Thanks, Matt On Tue, Oct 7, 2008 at 1:29 PM, Pavel Shamis (Pasha) > wrote: Matt, For all 1.2.X versions you should use btl_openi

Re: [OMPI devel] Fwd: [OMPI users] OpenMPI with openib partitions

2008-10-07 Thread Matt Burgess
Pasha, That's great, thanks for the help. When exactly do you expect that 1.2.8 will be released? Thanks, Matt On Tue, Oct 7, 2008 at 1:29 PM, Pavel Shamis (Pasha) < pa...@dev.mellanox.co.il> wrote: > Matt, > For all 1.2.X versions you should use btl_openib_ib_pkey_val > In ongoing 1.3 version

Re: [OMPI devel] Fwd: [OMPI users] OpenMPI with openib partitions

2008-10-07 Thread Pavel Shamis (Pasha)
Matt, For all 1.2.X versions you should use btl_openib_ib_pkey_val In ongoing 1.3 version the parameter was renamed to btl_openib_of_pkey_val. BTW we plan to release 1.2.8 version very soon and it will include the partition bug fix. Regards, Pasha Matt Burgess wrote: Pasha, With your patch

Re: [OMPI devel] Fwd: [OMPI users] OpenMPI with openib partitions

2008-10-07 Thread Matt Burgess
Pasha, With your patch and parameter suggestion, it works! So to be clear btl_openib_ib_pkey_val is for 1.2.6 and btl_openib_of_pkey_val is for 1.2.7? Thanks again, Matt On Tue, Oct 7, 2008 at 12:24 PM, Pavel Shamis (Pasha) < pa...@dev.mellanox.co.il> wrote: > Matt, > Can you please run " cat /

Re: [OMPI devel] Fwd: [OMPI users] OpenMPI with openib partitions

2008-10-07 Thread Pavel Shamis (Pasha)
Matt, Can you please run " cat /sys/class/infiniband/mlx4_0/ports/1/pkeys/* " on your d2-ib,d3-ib. I would like to check the partition configuration. Ohh, BTW I see that the command line in previous email was wrong, Please use follow command line (the parameter name should be "btl_openib_ib_pk

Re: [OMPI devel] Fwd: [OMPI users] OpenMPI with openib partitions

2008-10-07 Thread Matt Burgess
Pasha, Thanks for the patch. Unfortunately, it doesn't seem like that fixed the problem. I realized earlier I didn't mention what version of OpenMPI I was trying - it's 1.2.6. Should I be trying 1.2.7 with this patch? Thanks, Matt 2008/10/7 Pavel Shamis (Pasha) > Matt, > Can you please try att

Re: [OMPI devel] Fwd: [OMPI users] OpenMPI with openib partitions

2008-10-07 Thread Pavel Shamis (Pasha)
Matt, Can you please try attached patch ? I guess it will resolve this issue. Thanks, Pasha Matt Burgess wrote: Lenny, Thanks for the info. It doesn't seem to be be working still. My command line is: /opt/openmpi-ib/1.2.6/bin/mpirun -np 2 -H d2-ib,d3-ib -mca btl openib,self -mca btl_openib

[OMPI devel] Fwd: [OMPI users] OpenMPI with openib partitions

2008-10-07 Thread Matt Burgess
Lenny, Thanks for the info. It doesn't seem to be be working still. My command line is: /opt/openmpi-ib/1.2.6/bin/mpirun -np 2 -H d2-ib,d3-ib -mca btl openib,self -mca btl_openib_of_pkey_val 33033 /cluster/pallas/x86_64-ib/IMB-MPI1 I don't have a "/sys/class/infiniband/mthca0/ports/1/pkeys/" but