Re: [OMPI devel] open ib dependency question
On Thu, 10 Jul 2008, Pavel Shamis (Pasha) wrote: FYI the issue was resolved - https://svn.open-mpi.org/trac/ompi/ticket/1376 Indeed, no more IBCM error message displayed with r18878. Thank you ! -- Bogdan Costescu IWR, University of Heidelberg, INF 368, D-69120 Heidelberg, Germany Phone: +49 6221 54 8869/8240, Fax: +49 6221 54 8868/8850 E-mail: bogdan.coste...@iwr.uni-heidelberg.de
Re: [OMPI devel] open ib dependency question
FYI the issue was resolved - https://svn.open-mpi.org/trac/ompi/ticket/1376 Regards, Pasha Bogdan Costescu wrote: On Thu, 3 Jul 2008, Pavel Shamis (Pasha) wrote: I had similar issue recently. It will be nice to have option to disable/enable *CM via config flags. Not sure if this is related... I am looking at using a nightly 1.3 snapshot and I get this type of error messages when running: [n020205][[36506,1],0][connect/btl_openib_connect_ibcm.c:723:ibcm_component_query] failed to open IB CM device: /dev/infiniband/ucm0 which is actually right, as /dev/infiniband on the nodes doesn't contain ucm0. On the same cluster, OpenMPI 1.2.7rc2 works fine; the configure options for building them are the same. The output of ldd shows for the binary linked with 1.3a: libibcm.so.1 => /opt/ofed/1.2.5.4/lib64/libibcm.so.1 while this is missing from the binary linked with 1.2. Even after printing these messages, the binary linked with 1.3a works; it works even when I specify "--mca btl openib,self" so I think that the IB stack is still being used (there is also a TCP/GigE network which could be chosen otherwise). I don't know whether this is caused by a somehow inconsistent setup of the system, but I would welcome an option to make 1.3a behave like 1.2.
Re: [OMPI devel] open ib dependency question
Pasha -- do you want to open a ticket? Done. https://svn.open-mpi.org/trac/ompi/ticket/1376 Pasha.
Re: [OMPI devel] open ib dependency question
I added patch to the ticked, please review. Pasha. Jeff Squyres wrote: Ok: https://svn.open-mpi.org/trac/ompi/ticket/1375 I think any of us could do this -- it's pretty straightforward. No guarantees on when I can get to it; my 1.3 list is already pretty long... On Jul 3, 2008, at 6:20 AM, Pavel Shamis (Pasha) wrote: Jeff Squyres wrote: Do you need configury to disable building ibcm / rdmacm support? The more I think about it, the more I think that these would be good features to have for v1.3... I had similar issue recently. It will be nice to have option to disable/enable *CM via config flags. On Jul 3, 2008, at 2:52 AM, Don Kerr wrote: I did not think it was required but it hung me up when I built ompi on one system which had the ibcm libraries and then ran on a system without the ibcm libs. I had another issue on the system without ibcm libs which prevented my building there but I will go down that path again. Thanks. Jeff Squyres wrote: That is the IBCM library for the IBCM CPC -- IB connection manager stuff. It's not *necessary*; you could use the OOB CPC if you want to. That being said, OMPI currently builds support for it (and links it in) if it finds the right headers and library files. We don't currently have configury to disable this behavior (and *not* build RDMACM and/or IBCM support). Do you have a problem / need to disable building support for IBCM? On Jul 2, 2008, at 12:02 PM, Don Kerr wrote: It appears that the mca_btl_openib.so has a dependency on libibcm.so. Is this necessary? ___ devel mailing list de...@open-mpi.org http://www.open-mpi.org/mailman/listinfo.cgi/devel ___ devel mailing list de...@open-mpi.org http://www.open-mpi.org/mailman/listinfo.cgi/devel ___ devel mailing list de...@open-mpi.org http://www.open-mpi.org/mailman/listinfo.cgi/devel
Re: [OMPI devel] open ib dependency question
I see the same issue on my Mellanox OFED 1.3. IBCM module is loaded but is no such device in system. Jeff, looks like some bug in IBCM stuff... (not ompi) I think we should print the error only if ibcm was explicitly selected by user. But from the cpc level it is no way to know about explicit selectionMaybe just hide the print ? Bogdan Costescu wrote: On Thu, 3 Jul 2008, Pavel Shamis (Pasha) wrote: I had similar issue recently. It will be nice to have option to disable/enable *CM via config flags. Not sure if this is related... I am looking at using a nightly 1.3 snapshot and I get this type of error messages when running: [n020205][[36506,1],0][connect/btl_openib_connect_ibcm.c:723:ibcm_component_query] failed to open IB CM device: /dev/infiniband/ucm0 which is actually right, as /dev/infiniband on the nodes doesn't contain ucm0. On the same cluster, OpenMPI 1.2.7rc2 works fine; the configure options for building them are the same. The output of ldd shows for the binary linked with 1.3a: libibcm.so.1 => /opt/ofed/1.2.5.4/lib64/libibcm.so.1 while this is missing from the binary linked with 1.2. Even after printing these messages, the binary linked with 1.3a works; it works even when I specify "--mca btl openib,self" so I think that the IB stack is still being used (there is also a TCP/GigE network which could be chosen otherwise). I don't know whether this is caused by a somehow inconsistent setup of the system, but I would welcome an option to make 1.3a behave like 1.2.
Re: [OMPI devel] open ib dependency question
On Thu, 3 Jul 2008, Pavel Shamis (Pasha) wrote: I had similar issue recently. It will be nice to have option to disable/enable *CM via config flags. Not sure if this is related... I am looking at using a nightly 1.3 snapshot and I get this type of error messages when running: [n020205][[36506,1],0][connect/btl_openib_connect_ibcm.c:723:ibcm_component_query] failed to open IB CM device: /dev/infiniband/ucm0 which is actually right, as /dev/infiniband on the nodes doesn't contain ucm0. On the same cluster, OpenMPI 1.2.7rc2 works fine; the configure options for building them are the same. The output of ldd shows for the binary linked with 1.3a: libibcm.so.1 => /opt/ofed/1.2.5.4/lib64/libibcm.so.1 while this is missing from the binary linked with 1.2. Even after printing these messages, the binary linked with 1.3a works; it works even when I specify "--mca btl openib,self" so I think that the IB stack is still being used (there is also a TCP/GigE network which could be chosen otherwise). I don't know whether this is caused by a somehow inconsistent setup of the system, but I would welcome an option to make 1.3a behave like 1.2. -- Bogdan Costescu IWR, University of Heidelberg, INF 368, D-69120 Heidelberg, Germany Phone: +49 6221 54 8869/8240, Fax: +49 6221 54 8868/8850 E-mail: bogdan.coste...@iwr.uni-heidelberg.de
Re: [OMPI devel] open ib dependency question
capturing in the bug is good enough for me at this point, thanks Jeff Jeff Squyres wrote: Ok: https://svn.open-mpi.org/trac/ompi/ticket/1375 I think any of us could do this -- it's pretty straightforward. No guarantees on when I can get to it; my 1.3 list is already pretty long... On Jul 3, 2008, at 6:20 AM, Pavel Shamis (Pasha) wrote: Jeff Squyres wrote: Do you need configury to disable building ibcm / rdmacm support? The more I think about it, the more I think that these would be good features to have for v1.3... I had similar issue recently. It will be nice to have option to disable/enable *CM via config flags. On Jul 3, 2008, at 2:52 AM, Don Kerr wrote: I did not think it was required but it hung me up when I built ompi on one system which had the ibcm libraries and then ran on a system without the ibcm libs. I had another issue on the system without ibcm libs which prevented my building there but I will go down that path again. Thanks. Jeff Squyres wrote: That is the IBCM library for the IBCM CPC -- IB connection manager stuff. It's not *necessary*; you could use the OOB CPC if you want to. That being said, OMPI currently builds support for it (and links it in) if it finds the right headers and library files. We don't currently have configury to disable this behavior (and *not* build RDMACM and/or IBCM support). Do you have a problem / need to disable building support for IBCM? On Jul 2, 2008, at 12:02 PM, Don Kerr wrote: It appears that the mca_btl_openib.so has a dependency on libibcm.so. Is this necessary? ___ devel mailing list de...@open-mpi.org http://www.open-mpi.org/mailman/listinfo.cgi/devel ___ devel mailing list de...@open-mpi.org http://www.open-mpi.org/mailman/listinfo.cgi/devel ___ devel mailing list de...@open-mpi.org http://www.open-mpi.org/mailman/listinfo.cgi/devel
Re: [OMPI devel] open ib dependency question
Ok: https://svn.open-mpi.org/trac/ompi/ticket/1375 I think any of us could do this -- it's pretty straightforward. No guarantees on when I can get to it; my 1.3 list is already pretty long... No problem. I will take this one. Pasha. On Jul 3, 2008, at 6:20 AM, Pavel Shamis (Pasha) wrote: Jeff Squyres wrote: Do you need configury to disable building ibcm / rdmacm support? The more I think about it, the more I think that these would be good features to have for v1.3... I had similar issue recently. It will be nice to have option to disable/enable *CM via config flags. On Jul 3, 2008, at 2:52 AM, Don Kerr wrote: I did not think it was required but it hung me up when I built ompi on one system which had the ibcm libraries and then ran on a system without the ibcm libs. I had another issue on the system without ibcm libs which prevented my building there but I will go down that path again. Thanks. Jeff Squyres wrote: That is the IBCM library for the IBCM CPC -- IB connection manager stuff. It's not *necessary*; you could use the OOB CPC if you want to. That being said, OMPI currently builds support for it (and links it in) if it finds the right headers and library files. We don't currently have configury to disable this behavior (and *not* build RDMACM and/or IBCM support). Do you have a problem / need to disable building support for IBCM? On Jul 2, 2008, at 12:02 PM, Don Kerr wrote: It appears that the mca_btl_openib.so has a dependency on libibcm.so. Is this necessary? ___ devel mailing list de...@open-mpi.org http://www.open-mpi.org/mailman/listinfo.cgi/devel ___ devel mailing list de...@open-mpi.org http://www.open-mpi.org/mailman/listinfo.cgi/devel ___ devel mailing list de...@open-mpi.org http://www.open-mpi.org/mailman/listinfo.cgi/devel
Re: [OMPI devel] open ib dependency question
Ok: https://svn.open-mpi.org/trac/ompi/ticket/1375 I think any of us could do this -- it's pretty straightforward. No guarantees on when I can get to it; my 1.3 list is already pretty long... On Jul 3, 2008, at 6:20 AM, Pavel Shamis (Pasha) wrote: Jeff Squyres wrote: Do you need configury to disable building ibcm / rdmacm support? The more I think about it, the more I think that these would be good features to have for v1.3... I had similar issue recently. It will be nice to have option to disable/enable *CM via config flags. On Jul 3, 2008, at 2:52 AM, Don Kerr wrote: I did not think it was required but it hung me up when I built ompi on one system which had the ibcm libraries and then ran on a system without the ibcm libs. I had another issue on the system without ibcm libs which prevented my building there but I will go down that path again. Thanks. Jeff Squyres wrote: That is the IBCM library for the IBCM CPC -- IB connection manager stuff. It's not *necessary*; you could use the OOB CPC if you want to. That being said, OMPI currently builds support for it (and links it in) if it finds the right headers and library files. We don't currently have configury to disable this behavior (and *not* build RDMACM and/or IBCM support). Do you have a problem / need to disable building support for IBCM? On Jul 2, 2008, at 12:02 PM, Don Kerr wrote: It appears that the mca_btl_openib.so has a dependency on libibcm.so. Is this necessary? ___ devel mailing list de...@open-mpi.org http://www.open-mpi.org/mailman/listinfo.cgi/devel ___ devel mailing list de...@open-mpi.org http://www.open-mpi.org/mailman/listinfo.cgi/devel ___ devel mailing list de...@open-mpi.org http://www.open-mpi.org/mailman/listinfo.cgi/devel -- Jeff Squyres Cisco Systems
Re: [OMPI devel] open ib dependency question
Jeff Squyres wrote: Do you need configury to disable building ibcm / rdmacm support? The more I think about it, the more I think that these would be good features to have for v1.3... I had similar issue recently. It will be nice to have option to disable/enable *CM via config flags. On Jul 3, 2008, at 2:52 AM, Don Kerr wrote: I did not think it was required but it hung me up when I built ompi on one system which had the ibcm libraries and then ran on a system without the ibcm libs. I had another issue on the system without ibcm libs which prevented my building there but I will go down that path again. Thanks. Jeff Squyres wrote: That is the IBCM library for the IBCM CPC -- IB connection manager stuff. It's not *necessary*; you could use the OOB CPC if you want to. That being said, OMPI currently builds support for it (and links it in) if it finds the right headers and library files. We don't currently have configury to disable this behavior (and *not* build RDMACM and/or IBCM support). Do you have a problem / need to disable building support for IBCM? On Jul 2, 2008, at 12:02 PM, Don Kerr wrote: It appears that the mca_btl_openib.so has a dependency on libibcm.so. Is this necessary? ___ devel mailing list de...@open-mpi.org http://www.open-mpi.org/mailman/listinfo.cgi/devel ___ devel mailing list de...@open-mpi.org http://www.open-mpi.org/mailman/listinfo.cgi/devel
Re: [OMPI devel] open ib dependency question
Do you need configury to disable building ibcm / rdmacm support? The more I think about it, the more I think that these would be good features to have for v1.3... On Jul 3, 2008, at 2:52 AM, Don Kerr wrote: I did not think it was required but it hung me up when I built ompi on one system which had the ibcm libraries and then ran on a system without the ibcm libs. I had another issue on the system without ibcm libs which prevented my building there but I will go down that path again. Thanks. Jeff Squyres wrote: That is the IBCM library for the IBCM CPC -- IB connection manager stuff. It's not *necessary*; you could use the OOB CPC if you want to. That being said, OMPI currently builds support for it (and links it in) if it finds the right headers and library files. We don't currently have configury to disable this behavior (and *not* build RDMACM and/or IBCM support). Do you have a problem / need to disable building support for IBCM? On Jul 2, 2008, at 12:02 PM, Don Kerr wrote: It appears that the mca_btl_openib.so has a dependency on libibcm.so. Is this necessary? ___ devel mailing list de...@open-mpi.org http://www.open-mpi.org/mailman/listinfo.cgi/devel ___ devel mailing list de...@open-mpi.org http://www.open-mpi.org/mailman/listinfo.cgi/devel -- Jeff Squyres Cisco Systems
Re: [OMPI devel] open ib dependency question
I did not think it was required but it hung me up when I built ompi on one system which had the ibcm libraries and then ran on a system without the ibcm libs. I had another issue on the system without ibcm libs which prevented my building there but I will go down that path again. Thanks. Jeff Squyres wrote: That is the IBCM library for the IBCM CPC -- IB connection manager stuff. It's not *necessary*; you could use the OOB CPC if you want to. That being said, OMPI currently builds support for it (and links it in) if it finds the right headers and library files. We don't currently have configury to disable this behavior (and *not* build RDMACM and/or IBCM support). Do you have a problem / need to disable building support for IBCM? On Jul 2, 2008, at 12:02 PM, Don Kerr wrote: It appears that the mca_btl_openib.so has a dependency on libibcm.so. Is this necessary? ___ devel mailing list de...@open-mpi.org http://www.open-mpi.org/mailman/listinfo.cgi/devel
Re: [OMPI devel] open ib dependency question
That is the IBCM library for the IBCM CPC -- IB connection manager stuff. It's not *necessary*; you could use the OOB CPC if you want to. That being said, OMPI currently builds support for it (and links it in) if it finds the right headers and library files. We don't currently have configury to disable this behavior (and *not* build RDMACM and/or IBCM support). Do you have a problem / need to disable building support for IBCM? On Jul 2, 2008, at 12:02 PM, Don Kerr wrote: It appears that the mca_btl_openib.so has a dependency on libibcm.so. Is this necessary? ___ devel mailing list de...@open-mpi.org http://www.open-mpi.org/mailman/listinfo.cgi/devel -- Jeff Squyres Cisco Systems
[OMPI devel] open ib dependency question
It appears that the mca_btl_openib.so has a dependency on libibcm.so. Is this necessary?