Re: [ofa-general] Installing SDP on existing OFED 1.3.1 install - DRBD SDP/Infiniband Support
On Tuesday 01 September 2009 16:24, Robert Dunklewrote:
> Hi Jack,
>
> Thanks for the tip. (*Embarassed*)
>
> I think a Kernel upgrade since might have broken the source RPM. Is
> there any way for me to fix this? (I have an identical hardware server
> that was originally installed with SDP, details of that below, it seems
> the system without sdp that I'm trying to add it to got a minor kernel
> upgrade)
>
> Thanks again,
>
> Rob
You need to see where the kernel is taking ib_sdp from.
Do "modinfo ib_sdp" to see the ib_sdp.ko file which gets loaded.
Note that on the working system, you have:
/lib/modules/2.6.18-92.1.6.el5xen/updates/kernel/drivers/infiniband/ulp/sdp/ib_sdp.ko
Note **updates** in the above directory path -- indicating that this
sdp was installed as part of your OFED install. You have no such path for
ib_sdp.ko in
your broken system.
1. Was OFED installed on the broken system with your xen kernel?
(check this by seeing if directory /lib/modules/2.6.18-92.1.13.el5xen/updates
exists).
2. Where did you place the ib_sdp.ko module that you built?
3. The version disagreement below indicates that the sdp module
expects different versions of ib_core.ko and rdma_cm.ko than are
currently loaded.
-Jack
>
> Error on system I'm trying to add to ("Broken System"):
> ib_sdp: disagrees about version of symbol ib_unregister_client
> ib_sdp: Unknown symbol ib_unregister_client
> ib_sdp: disagrees about version of symbol ib_create_cq
> ib_sdp: Unknown symbol ib_create_cq
> ib_sdp: disagrees about version of symbol rdma_resolve_addr
> ib_sdp: Unknown symbol rdma_resolve_addr
> ib_sdp: disagrees about version of symbol ib_dereg_mr
> ib_sdp: Unknown symbol ib_dereg_mr
> ib_sdp: disagrees about version of symbol rdma_reject
> ib_sdp: Unknown symbol rdma_reject
> ib_sdp: disagrees about version of symbol rdma_disconnect
> ib_sdp: Unknown symbol rdma_disconnect
> ib_sdp: disagrees about version of symbol rdma_resolve_route
> ib_sdp: Unknown symbol rdma_resolve_route
> ib_sdp: disagrees about version of symbol rdma_bind_addr
> ib_sdp: Unknown symbol rdma_bind_addr
> ib_sdp: disagrees about version of symbol ib_register_client
> ib_sdp: Unknown symbol ib_register_client
> ib_sdp: disagrees about version of symbol rdma_create_qp
> ib_sdp: Unknown symbol rdma_create_qp
> ib_sdp: disagrees about version of symbol ib_destroy_cq
> ib_sdp: Unknown symbol ib_destroy_cq
> ib_sdp: disagrees about version of symbol rdma_create_id
> ib_sdp: Unknown symbol rdma_create_id
> ib_sdp: disagrees about version of symbol rdma_notify
> ib_sdp: Unknown symbol rdma_notify
> ib_sdp: disagrees about version of symbol rdma_listen
> ib_sdp: Unknown symbol rdma_listen
> ib_sdp: disagrees about version of symbol ib_get_dma_mr
> ib_sdp: Unknown symbol ib_get_dma_mr
> ib_sdp: disagrees about version of symbol ib_alloc_pd
> ib_sdp: Unknown symbol ib_alloc_pd
> ib_sdp: disagrees about version of symbol rdma_connect
> ib_sdp: Unknown symbol rdma_connect
> ib_sdp: disagrees about version of symbol rdma_destroy_id
> ib_sdp: Unknown symbol rdma_destroy_id
> ib_sdp: disagrees about version of symbol rdma_accept
> ib_sdp: Unknown symbol rdma_accept
> ib_sdp: disagrees about version of symbol ib_destroy_qp
> ib_sdp: Unknown symbol ib_destroy_qp
> ib_sdp: disagrees about version of symbol ib_dealloc_pd
> ib_sdp: Unknown symbol ib_dealloc_pd
>
> Working system files:
> /lib/modules/2.6.18-92.1.6.el5xen/kernel/drivers/infiniband/ulp/sdp
> /lib/modules/2.6.18-92.1.6.el5xen/kernel/drivers/infiniband/ulp/sdp/ib_s
> dp.ko
> /lib/modules/2.6.18-92.1.6.el5xen/updates/kernel/drivers/infiniband/ulp/
> sdp
> /lib/modules/2.6.18-92.1.6.el5xen/updates/kernel/drivers/infiniband/ulp/
> sdp/ib_s
> dp.ko
> /lib/modules/2.6.18-92.el5/kernel/drivers/infiniband/ulp/sdp
> /lib/modules/2.6.18-92.el5/kernel/drivers/infiniband/ulp/sdp/ib_sdp.ko
>
> Broken System Files:
> # locate sdp | more
> /etc/libsdp.conf
> /lib/modules/2.6.18-92.1.13.el5xen/kernel/drivers/infiniband/ulp/sdp
> /lib/modules/2.6.18-92.1.13.el5xen/kernel/drivers/infiniband/ulp/sdp/ib_
> sdp.ko
> /lib/modules/2.6.18-92.el5/kernel/drivers/infiniband/ulp/sdp
> /lib/modules/2.6.18-92.el5/kernel/drivers/infiniband/ulp/sdp/ib_sdp.ko
>
> Uname -r on Broken:
> 2.6.18-92.1.13.el5xen
>
> Uname -r on Works:
> 2.6.18-92.1.6.el5xen
>
>
> -Original Message-
> From: Jack Morgenstein [mailto:[email protected]]
> Sent: 01 September 2009 14:05
> To: Robert Dunkley
> Cc: [email protected]
> Subject: Re: [ofa-general] Installing SDP on existing OFED 1.3.1 install
> - DRBD SDP/Infiniband Support
>
> On Tuesday 01 September 2009 14:52, Robert Dunkley wrote:
&g
Re: [ofa-general] Installing SDP on existing OFED 1.3.1 install - DRBD SDP/Infiniband Support
On Tuesday 01 September 2009 14:52, Robert Dunkley wrote: > Hi Jack, > > Thanks for the reply, it now tries to load the ib_sdp module but fails: > > # /etc/rc.d/init.d/openibd restart > Unloading HCA driver: [ OK ] > Loading HCA driver and Access Layer: [ OK ] > Setting up InfiniBand network interfaces: > Bringing up interface ib0: [ OK ] > Setting up service network . . . [ done ] > Loading ib_sdp [FAILED] > > Where does the full log for this go? Am I missing some sort of > dependency? (Loaded modules shown below) > do "dmesg" from a console window to see what the failure is. - Jack > # /etc/rc.d/init.d/openibd status > > HCA driver loaded > > Configured devices: > ib0 > > Currently active devices: > ib0 > > The following OFED modules are loaded: > > rdma_ucm > rdma_cm > ib_addr > ib_ipoib > mlx4_core > mlx4_ib > ib_mthca > ib_uverbs > ib_umad > ib_sa > ib_cm > ib_mad > ib_core > iw_cxgb3 > > > Thanks, > > Rob > > > -Original Message- > From: Jack Morgenstein [mailto:[email protected]] > Sent: 01 September 2009 12:17 > To: [email protected] > Cc: Robert Dunkley > Subject: Re: [ofa-general] Installing SDP on existing OFED 1.3.1 install > - DRBD SDP/Infiniband Support > > On Tuesday 01 September 2009 13:44, Robert Dunkley wrote: > > Hi everyone, > > > > A DRBD release candidate with specific SDP/Infiniband support was > > released last week. > > > > I have an existing OFED 1.3.1 install without the SDP protocol loaded, > I > > need to add it. I still have the original source I installed with and > > found what looked like a suitable SRPM, I built the SRPM, installed > the > > resulting RPM and then restarted OpenSM and OpenIBD but OpenIBD does > not > > seem to have loaded "ib_sdp". I don't want to reboot this server. Does > > anyone know where I am going wrong? > > > Try adding the lines: > > # Load SDP module > SDP_LOAD=yes > > to file /etc/infiniband/openib.conf > > and then restart the driver > > (if SDP_LOAD is already in the file, and set to "no", just change it to > "yes"). > > -Jack > > > Thanks, > > > > Rob > > > > The SAQ Group > > > > Registered Office: 18 Chapel Street, Petersfield, Hampshire GU32 3DZ > > SAQ is the trading name of SEMTEC Limited. Registered in England & > Wales > > Company Number: 06481952 > > > > http://www.saqnet.co.uk AS29219 > > > > SAQ Group Delivers high quality, honestly priced communication and > I.T. services to UK Business. > > > > Broadband : Domains : Email : Hosting : CoLo : Servers : Racks : > Transit : Backups : Managed Networks : Remote Support. > > > > ISPA Member > > > > ___ > > general mailing list > > [email protected] > > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > > > To unsubscribe, please visit > http://openib.org/mailman/listinfo/openib-general > > > ___ general mailing list [email protected] http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
RE: [ofa-general] Installing SDP on existing OFED 1.3.1 install - DRBD SDP/Infiniband Support
Hi Jack, Thanks for the reply, it now tries to load the ib_sdp module but fails: # /etc/rc.d/init.d/openibd restart Unloading HCA driver: [ OK ] Loading HCA driver and Access Layer: [ OK ] Setting up InfiniBand network interfaces: Bringing up interface ib0: [ OK ] Setting up service network . . . [ done ] Loading ib_sdp [FAILED] Where does the full log for this go? Am I missing some sort of dependency? (Loaded modules shown below) # /etc/rc.d/init.d/openibd status HCA driver loaded Configured devices: ib0 Currently active devices: ib0 The following OFED modules are loaded: rdma_ucm rdma_cm ib_addr ib_ipoib mlx4_core mlx4_ib ib_mthca ib_uverbs ib_umad ib_sa ib_cm ib_mad ib_core iw_cxgb3 Thanks, Rob -Original Message- From: Jack Morgenstein [mailto:[email protected]] Sent: 01 September 2009 12:17 To: [email protected] Cc: Robert Dunkley Subject: Re: [ofa-general] Installing SDP on existing OFED 1.3.1 install - DRBD SDP/Infiniband Support On Tuesday 01 September 2009 13:44, Robert Dunkley wrote: > Hi everyone, > > A DRBD release candidate with specific SDP/Infiniband support was > released last week. > > I have an existing OFED 1.3.1 install without the SDP protocol loaded, I > need to add it. I still have the original source I installed with and > found what looked like a suitable SRPM, I built the SRPM, installed the > resulting RPM and then restarted OpenSM and OpenIBD but OpenIBD does not > seem to have loaded "ib_sdp". I don't want to reboot this server. Does > anyone know where I am going wrong? > Try adding the lines: # Load SDP module SDP_LOAD=yes to file /etc/infiniband/openib.conf and then restart the driver (if SDP_LOAD is already in the file, and set to "no", just change it to "yes"). -Jack > Thanks, > > Rob > > The SAQ Group > > Registered Office: 18 Chapel Street, Petersfield, Hampshire GU32 3DZ > SAQ is the trading name of SEMTEC Limited. Registered in England & Wales > Company Number: 06481952 > > http://www.saqnet.co.uk AS29219 > > SAQ Group Delivers high quality, honestly priced communication and I.T. services to UK Business. > > Broadband : Domains : Email : Hosting : CoLo : Servers : Racks : Transit : Backups : Managed Networks : Remote Support. > > ISPA Member > > ___ > general mailing list > [email protected] > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general > ___ general mailing list [email protected] http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [ofa-general] Installing SDP on existing OFED 1.3.1 install - DRBD SDP/Infiniband Support
On Tuesday 01 September 2009 13:44, Robert Dunkley wrote: > Hi everyone, > > A DRBD release candidate with specific SDP/Infiniband support was > released last week. > > I have an existing OFED 1.3.1 install without the SDP protocol loaded, I > need to add it. I still have the original source I installed with and > found what looked like a suitable SRPM, I built the SRPM, installed the > resulting RPM and then restarted OpenSM and OpenIBD but OpenIBD does not > seem to have loaded "ib_sdp". I don't want to reboot this server. Does > anyone know where I am going wrong? > Try adding the lines: # Load SDP module SDP_LOAD=yes to file /etc/infiniband/openib.conf and then restart the driver (if SDP_LOAD is already in the file, and set to "no", just change it to "yes"). -Jack > Thanks, > > Rob > > The SAQ Group > > Registered Office: 18 Chapel Street, Petersfield, Hampshire GU32 3DZ > SAQ is the trading name of SEMTEC Limited. Registered in England & Wales > Company Number: 06481952 > > http://www.saqnet.co.uk AS29219 > > SAQ Group Delivers high quality, honestly priced communication and I.T. > services to UK Business. > > Broadband : Domains : Email : Hosting : CoLo : Servers : Racks : Transit : > Backups : Managed Networks : Remote Support. > > ISPA Member > > ___ > general mailing list > [email protected] > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general > ___ general mailing list [email protected] http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
