Re: [OMPI devel] open ib dependency question

2008-07-11 Thread Bogdan Costescu

On Thu, 10 Jul 2008, Pavel Shamis (Pasha) wrote:


FYI the issue was resolved - https://svn.open-mpi.org/trac/ompi/ticket/1376


Indeed, no more IBCM error message displayed with r18878. Thank you !

--
Bogdan Costescu

IWR, University of Heidelberg, INF 368, D-69120 Heidelberg, Germany
Phone: +49 6221 54 8869/8240, Fax: +49 6221 54 8868/8850
E-mail: bogdan.coste...@iwr.uni-heidelberg.de


Re: [OMPI devel] open ib dependency question

2008-07-10 Thread Pavel Shamis (Pasha)

FYI the issue was resolved - https://svn.open-mpi.org/trac/ompi/ticket/1376
Regards,
Pasha

Bogdan Costescu wrote:

On Thu, 3 Jul 2008, Pavel Shamis (Pasha) wrote:

I had similar issue recently. It will be nice to have option to 
disable/enable *CM via config flags.


Not sure if this is related... I am looking at using a nightly 1.3 
snapshot and I get this type of error messages when running:


[n020205][[36506,1],0][connect/btl_openib_connect_ibcm.c:723:ibcm_component_query] 
failed to open IB CM device: /dev/infiniband/ucm0


which is actually right, as /dev/infiniband on the nodes doesn't 
contain ucm0. On the same cluster, OpenMPI 1.2.7rc2 works fine; the 
configure options for building them are the same.


The output of ldd shows for the binary linked with 1.3a:

libibcm.so.1 => /opt/ofed/1.2.5.4/lib64/libibcm.so.1

while this is missing from the binary linked with 1.2. Even after 
printing these messages, the binary linked with 1.3a works; it works 
even when I specify "--mca btl openib,self" so I think that the IB 
stack is still being used (there is also a TCP/GigE network which 
could be chosen otherwise).


I don't know whether this is caused by a somehow inconsistent setup of 
the system, but I would welcome an option to make 1.3a behave like 1.2.






Re: [OMPI devel] open ib dependency question

2008-07-08 Thread Pavel Shamis (Pasha)




Pasha -- do you want to open a ticket?


Done. https://svn.open-mpi.org/trac/ompi/ticket/1376
Pasha.


Re: [OMPI devel] open ib dependency question

2008-07-06 Thread Pavel Shamis (Pasha)

I added patch to the ticked, please review.

Pasha.
Jeff Squyres wrote:

Ok:

https://svn.open-mpi.org/trac/ompi/ticket/1375

I think any of us could do this -- it's pretty straightforward.  No 
guarantees on when I can get to it; my 1.3 list is already pretty long...



On Jul 3, 2008, at 6:20 AM, Pavel Shamis (Pasha) wrote:


Jeff Squyres wrote:

Do you need configury to disable building ibcm / rdmacm support?

The more I think about it, the more I think that these would be good 
features to have for v1.3...
I had similar issue recently. It will be nice to have option to 
disable/enable *CM via config flags.



On Jul 3, 2008, at 2:52 AM, Don Kerr wrote:

I did not think it was required but it hung me up when I built ompi 
on one system which had the ibcm libraries and then ran on a system 
without the ibcm libs. I had another issue on the system without 
ibcm libs which prevented my building there but I will go down that 
path again. Thanks.


Jeff Squyres wrote:
That is the IBCM library for the IBCM CPC -- IB connection manager 
stuff.


It's not *necessary*; you could use the OOB CPC if you want to.

That being said, OMPI currently builds support for it (and links 
it in) if it finds the right headers and library files. We don't 
currently have configury to disable this behavior (and *not* build 
RDMACM and/or IBCM support).


Do you have a problem / need to disable building support for IBCM?



On Jul 2, 2008, at 12:02 PM, Don Kerr wrote:



It appears that the mca_btl_openib.so has a dependency on 
libibcm.so. Is this necessary?



___
devel mailing list
de...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/devel




___
devel mailing list
de...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/devel





___
devel mailing list
de...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/devel







Re: [OMPI devel] open ib dependency question

2008-07-06 Thread Pavel Shamis (Pasha)
I see the same issue on my Mellanox OFED 1.3. IBCM module is loaded but 
is no such device in system.

Jeff, looks like some bug in IBCM stuff... (not ompi)
I think we should print the error only if  ibcm  was  explicitly 
selected by user. But from the cpc level it is no way to know

about explicit selectionMaybe just hide the print ?

Bogdan Costescu wrote:

On Thu, 3 Jul 2008, Pavel Shamis (Pasha) wrote:

I had similar issue recently. It will be nice to have option to 
disable/enable *CM via config flags.


Not sure if this is related... I am looking at using a nightly 1.3 
snapshot and I get this type of error messages when running:


[n020205][[36506,1],0][connect/btl_openib_connect_ibcm.c:723:ibcm_component_query] 
failed to open IB CM device: /dev/infiniband/ucm0


which is actually right, as /dev/infiniband on the nodes doesn't 
contain ucm0. On the same cluster, OpenMPI 1.2.7rc2 works fine; the 
configure options for building them are the same.


The output of ldd shows for the binary linked with 1.3a:

libibcm.so.1 => /opt/ofed/1.2.5.4/lib64/libibcm.so.1

while this is missing from the binary linked with 1.2. Even after 
printing these messages, the binary linked with 1.3a works; it works 
even when I specify "--mca btl openib,self" so I think that the IB 
stack is still being used (there is also a TCP/GigE network which 
could be chosen otherwise).


I don't know whether this is caused by a somehow inconsistent setup of 
the system, but I would welcome an option to make 1.3a behave like 1.2.






Re: [OMPI devel] open ib dependency question

2008-07-04 Thread Bogdan Costescu

On Thu, 3 Jul 2008, Pavel Shamis (Pasha) wrote:

I had similar issue recently. It will be nice to have option to 
disable/enable *CM via config flags.


Not sure if this is related... I am looking at using a nightly 1.3 
snapshot and I get this type of error messages when running:


[n020205][[36506,1],0][connect/btl_openib_connect_ibcm.c:723:ibcm_component_query]
 failed to open IB CM device: /dev/infiniband/ucm0

which is actually right, as /dev/infiniband on the nodes doesn't 
contain ucm0. On the same cluster, OpenMPI 1.2.7rc2 works fine; the 
configure options for building them are the same.


The output of ldd shows for the binary linked with 1.3a:

libibcm.so.1 => /opt/ofed/1.2.5.4/lib64/libibcm.so.1

while this is missing from the binary linked with 1.2. Even after 
printing these messages, the binary linked with 1.3a works; it works 
even when I specify "--mca btl openib,self" so I think that the IB 
stack is still being used (there is also a TCP/GigE network which 
could be chosen otherwise).


I don't know whether this is caused by a somehow inconsistent setup of 
the system, but I would welcome an option to make 1.3a behave like 
1.2.


--
Bogdan Costescu

IWR, University of Heidelberg, INF 368, D-69120 Heidelberg, Germany
Phone: +49 6221 54 8869/8240, Fax: +49 6221 54 8868/8850
E-mail: bogdan.coste...@iwr.uni-heidelberg.de


Re: [OMPI devel] open ib dependency question

2008-07-03 Thread Don Kerr

capturing in the bug is good enough for me at this point, thanks Jeff

Jeff Squyres wrote:

Ok:

https://svn.open-mpi.org/trac/ompi/ticket/1375

I think any of us could do this -- it's pretty straightforward. No 
guarantees on when I can get to it; my 1.3 list is already pretty long...



On Jul 3, 2008, at 6:20 AM, Pavel Shamis (Pasha) wrote:


Jeff Squyres wrote:

Do you need configury to disable building ibcm / rdmacm support?

The more I think about it, the more I think that these would be good 
features to have for v1.3...
I had similar issue recently. It will be nice to have option to 
disable/enable *CM via config flags.



On Jul 3, 2008, at 2:52 AM, Don Kerr wrote:

I did not think it was required but it hung me up when I built ompi 
on one system which had the ibcm libraries and then ran on a system 
without the ibcm libs. I had another issue on the system without 
ibcm libs which prevented my building there but I will go down that 
path again. Thanks.


Jeff Squyres wrote:
That is the IBCM library for the IBCM CPC -- IB connection manager 
stuff.


It's not *necessary*; you could use the OOB CPC if you want to.

That being said, OMPI currently builds support for it (and links 
it in) if it finds the right headers and library files. We don't 
currently have configury to disable this behavior (and *not* build 
RDMACM and/or IBCM support).


Do you have a problem / need to disable building support for IBCM?



On Jul 2, 2008, at 12:02 PM, Don Kerr wrote:



It appears that the mca_btl_openib.so has a dependency on 
libibcm.so. Is this necessary?



___
devel mailing list
de...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/devel




___
devel mailing list
de...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/devel





___
devel mailing list
de...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/devel





Re: [OMPI devel] open ib dependency question

2008-07-03 Thread Pavel Shamis (Pasha)



Ok:

https://svn.open-mpi.org/trac/ompi/ticket/1375

I think any of us could do this -- it's pretty straightforward.  No 
guarantees on when I can get to it; my 1.3 list is already pretty long...

No problem. I will take this one.
Pasha.



On Jul 3, 2008, at 6:20 AM, Pavel Shamis (Pasha) wrote:


Jeff Squyres wrote:

Do you need configury to disable building ibcm / rdmacm support?

The more I think about it, the more I think that these would be good 
features to have for v1.3...
I had similar issue recently. It will be nice to have option to 
disable/enable *CM via config flags.



On Jul 3, 2008, at 2:52 AM, Don Kerr wrote:

I did not think it was required but it hung me up when I built ompi 
on one system which had the ibcm libraries and then ran on a system 
without the ibcm libs. I had another issue on the system without 
ibcm libs which prevented my building there but I will go down that 
path again. Thanks.


Jeff Squyres wrote:
That is the IBCM library for the IBCM CPC -- IB connection manager 
stuff.


It's not *necessary*; you could use the OOB CPC if you want to.

That being said, OMPI currently builds support for it (and links 
it in) if it finds the right headers and library files. We don't 
currently have configury to disable this behavior (and *not* build 
RDMACM and/or IBCM support).


Do you have a problem / need to disable building support for IBCM?



On Jul 2, 2008, at 12:02 PM, Don Kerr wrote:



It appears that the mca_btl_openib.so has a dependency on 
libibcm.so. Is this necessary?



___
devel mailing list
de...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/devel




___
devel mailing list
de...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/devel





___
devel mailing list
de...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/devel







Re: [OMPI devel] open ib dependency question

2008-07-03 Thread Jeff Squyres

Ok:

https://svn.open-mpi.org/trac/ompi/ticket/1375

I think any of us could do this -- it's pretty straightforward.  No  
guarantees on when I can get to it; my 1.3 list is already pretty  
long...



On Jul 3, 2008, at 6:20 AM, Pavel Shamis (Pasha) wrote:


Jeff Squyres wrote:

Do you need configury to disable building ibcm / rdmacm support?

The more I think about it, the more I think that these would be  
good features to have for v1.3...
I had similar issue recently. It will be nice to have option to  
disable/enable *CM via config flags.



On Jul 3, 2008, at 2:52 AM, Don Kerr wrote:

I did not think it was required but it hung me up when I built  
ompi on one system which had the ibcm libraries and then ran on a  
system without the ibcm libs. I had another issue on the system  
without ibcm libs which prevented my building there but I will go  
down that path again. Thanks.


Jeff Squyres wrote:
That is the IBCM library for the IBCM CPC -- IB connection  
manager stuff.


It's not *necessary*; you could use the OOB CPC if you want to.

That being said, OMPI currently builds support for it (and links  
it in) if it finds the right headers and library files. We don't  
currently have configury to disable this behavior (and *not*  
build RDMACM and/or IBCM support).


Do you have a problem / need to disable building support for IBCM?



On Jul 2, 2008, at 12:02 PM, Don Kerr wrote:



It appears that the mca_btl_openib.so has a dependency on  
libibcm.so. Is this necessary?



___
devel mailing list
de...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/devel




___
devel mailing list
de...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/devel





___
devel mailing list
de...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/devel



--
Jeff Squyres
Cisco Systems



Re: [OMPI devel] open ib dependency question

2008-07-03 Thread Pavel Shamis (Pasha)

Jeff Squyres wrote:

Do you need configury to disable building ibcm / rdmacm support?

The more I think about it, the more I think that these would be good 
features to have for v1.3...
I had similar issue recently. It will be nice to have option to 
disable/enable *CM via config flags.



On Jul 3, 2008, at 2:52 AM, Don Kerr wrote:

I did not think it was required but it hung me up when I built ompi 
on one system which had the ibcm libraries and then ran on a system 
without the ibcm libs. I had another issue on the system without ibcm 
libs which prevented my building there but I will go down that path 
again. Thanks.


Jeff Squyres wrote:
That is the IBCM library for the IBCM CPC -- IB connection manager 
stuff.


It's not *necessary*; you could use the OOB CPC if you want to.

That being said, OMPI currently builds support for it (and links it 
in) if it finds the right headers and library files. We don't 
currently have configury to disable this behavior (and *not* build 
RDMACM and/or IBCM support).


Do you have a problem / need to disable building support for IBCM?



On Jul 2, 2008, at 12:02 PM, Don Kerr wrote:



It appears that the mca_btl_openib.so has a dependency on 
libibcm.so. Is this necessary?



___
devel mailing list
de...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/devel




___
devel mailing list
de...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/devel







Re: [OMPI devel] open ib dependency question

2008-07-03 Thread Jeff Squyres

Do you need configury to disable building ibcm / rdmacm support?

The more I think about it, the more I think that these would be good  
features to have for v1.3...



On Jul 3, 2008, at 2:52 AM, Don Kerr wrote:

I did not think it was required but it hung me up when I built ompi  
on one system which had the ibcm libraries and then ran on a system  
without the ibcm libs. I had another issue on the system without  
ibcm libs which prevented my building there but I will go down that  
path again. Thanks.


Jeff Squyres wrote:
That is the IBCM library for the IBCM CPC -- IB connection manager  
stuff.


It's not *necessary*; you could use the OOB CPC if you want to.

That being said, OMPI currently builds support for it (and links it  
in) if it finds the right headers and library files. We don't  
currently have configury to disable this behavior (and *not* build  
RDMACM and/or IBCM support).


Do you have a problem / need to disable building support for IBCM?



On Jul 2, 2008, at 12:02 PM, Don Kerr wrote:



It appears that the mca_btl_openib.so has a dependency on  
libibcm.so. Is this necessary?



___
devel mailing list
de...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/devel




___
devel mailing list
de...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/devel



--
Jeff Squyres
Cisco Systems



Re: [OMPI devel] open ib dependency question

2008-07-03 Thread Don Kerr
I did not think it was required but it hung me up when I built ompi on 
one system which had the ibcm libraries and then ran on a system without 
the ibcm libs. I had another issue on the system without ibcm libs which 
prevented my building there but I will go down that path again. Thanks.


Jeff Squyres wrote:

That is the IBCM library for the IBCM CPC -- IB connection manager stuff.

It's not *necessary*; you could use the OOB CPC if you want to.

That being said, OMPI currently builds support for it (and links it 
in) if it finds the right headers and library files. We don't 
currently have configury to disable this behavior (and *not* build 
RDMACM and/or IBCM support).


Do you have a problem / need to disable building support for IBCM?



On Jul 2, 2008, at 12:02 PM, Don Kerr wrote:



It appears that the mca_btl_openib.so has a dependency on libibcm.so. 
Is this necessary?



___
devel mailing list
de...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/devel





Re: [OMPI devel] open ib dependency question

2008-07-02 Thread Jeff Squyres
That is the IBCM library for the IBCM CPC -- IB connection manager  
stuff.


It's not *necessary*; you could use the OOB CPC if you want to.

That being said, OMPI currently builds support for it (and links it  
in) if it finds the right headers and library files.  We don't  
currently have configury to disable this behavior (and *not* build  
RDMACM and/or IBCM support).


Do you have a problem / need to disable building support for IBCM?



On Jul 2, 2008, at 12:02 PM, Don Kerr wrote:



It appears that the mca_btl_openib.so  has a dependency on  
libibcm.so.  Is this necessary?



___
devel mailing list
de...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/devel



--
Jeff Squyres
Cisco Systems



[OMPI devel] open ib dependency question

2008-07-02 Thread Don Kerr


It appears that the mca_btl_openib.so  has a dependency on libibcm.so.  
Is this necessary?