Re: [ofa-general] IB function calls in kernel module fail
On Wed, 2009-02-18 at 16:40 +0200, Tziporet Koren wrote:
> Brian J. Murrell wrote:
> > Ahhh. But should he just include /src/openib/include/ or
> > also
> > /src/openib/kernel_addons/backport//include/
> > (as described in /src/openib/ofed_patch.mk as well?
> >
> > And in what order should these be specified in?
> >
> >
> You need both
> Order not important
Are you sure about this? I have been, in the past, unsure about this
ordering too, but have been seeing evidence that order is important.
Take for example in the current ~vlad/ofed_kernel-1.4 tree, there is an
exportfs.h in both /src/openib/include/linux and
/src/openib/kernel_addons/backport/2.6.16_sles10_sp2/include/linux.
Having discussed the presence of this (newish) header in the SLES10 SP2
backports tree with Jeff it's clear that it should be used in preference
to the one in the general include tree.
Therefore, if one is not careful about ordering (so that backport
headers take precedence) over the general ones, one would get the wrong
exportfs.h header for SLES10 SP2 builds.
But then there is the question of the kernel headers and ordering. Many
of backport headers use #include_next to get the next found instance of
a header. I have always assumed that was to get the kernel's version of
a header included in the backport header. But if the order is:
1. backports headers
2. ofa general headers
3. kernel headers
then an "#include_next " in
/src/openib/kernel_addons/backport//include/linux/foo.h
could potentially pick up /src/openib/include/linux/foo.h rather
than /include/linux/foo.h which I think is what is
intended/desired in most cases.
But if the ordering is changed to:
1. backports headers
2. kernel headers
3. ofa general headers
Then the desired preference of
/src/openib/include/{rdma,scsi}/* headers vs. the ones
included in the kernel will be lost.
How can we reconcile this?
b.
signature.asc
Description: This is a digitally signed message part
___
general mailing list
general@lists.openfabrics.org
http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general
To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [ofa-general] IB function calls in kernel module fail
Brian J. Murrell wrote: Ahhh. But should he just include /src/openib/include/ or also /src/openib/kernel_addons/backport//include/ (as described in /src/openib/ofed_patch.mk as well? And in what order should these be specified in? You need both Order not important Tziporet ___ general mailing list general@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [ofa-general] IB function calls in kernel module fail
On Tue, 2009-02-17 at 11:57 +0200, Tziporet Koren wrote: > neutron wrote: > > One remaining question. > > > > In my code of kernel module, do I need to #include the header files > > from /src/openib/include/ > > Or I just include the header files from /include/. > > > > > You should use the headers from ofed if you wish to use OFED kernel modules. Ahhh. But should he just include /src/openib/include/ or also /src/openib/kernel_addons/backport//include/ (as described in /src/openib/ofed_patch.mk as well? And in what order should these be specified in? b. signature.asc Description: This is a digitally signed message part ___ general mailing list general@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [ofa-general] IB function calls in kernel module fail
neutron wrote: One remaining question. In my code of kernel module, do I need to #include the header files from /src/openib/include/ Or I just include the header files from /include/. You should use the headers from ofed if you wish to use OFED kernel modules. Tziporet ___ general mailing list general@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [ofa-general] IB function calls in kernel module fail
One remaining question. In my code of kernel module, do I need to #include the header files from /src/openib/include/ Or I just include the header files from /include/. Thanks! On Mon, Feb 16, 2009 at 6:19 AM, Tziporet Koren wrote: > neutron wrote: >> >> Hi all, >> >> I'm writing a kernel module that make use of basic IB verbs to >> communicate, like: >> ib_register_client, ib_unregister_client, ib_alloc_pd, >> ib_create_qp, ib_reg_phys_mr, etc. >> >> I can compile the code into a kernel module: ib_rdma_lat.ko. This >> module is to test the RDMA write latency from kernel module. >> >> But when I "insmod", I got error reports at /var/log/messages: >> >> Feb 15 16:33:28 wci11 kernel: ib_rdma_lat: disagrees about version of >> symbol ib_unregister_client >> Feb 15 16:33:28 wci11 kernel: ib_rdma_lat: Unknown symbol >> ib_unregister_client >> Feb 15 16:33:28 wci11 kernel: ib_rdma_lat: disagrees about version of >> symbol ib_create_cq >> Feb 15 16:33:28 wci11 kernel: ib_rdma_lat: Unknown symbol ib_create_cq >> Feb 15 16:33:28 wci11 kernel: ib_rdma_lat: disagrees about version of >> symbol ib_reg_phys_mr >> Feb 15 16:33:28 wci11 kernel: ib_rdma_lat: Unknown symbol ib_reg_phys_mr >> Feb 15 16:33:28 wci11 kernel: ib_rdma_lat: disagrees about version of >> symbol ib_dereg_mr >> Feb 15 16:33:28 wci11 kernel: ib_rdma_lat: Unknown symbol ib_dereg_mr >> Feb 15 16:33:28 wci11 kernel: ib_rdma_lat: disagrees about version of >> symbol ib_register_client >> Feb 15 16:33:28 wci11 kernel: ib_rdma_lat: Unknown symbol >> ib_register_client >> Feb 15 16:33:28 wci11 kernel: ib_rdma_lat: disagrees about version of >> symbol ib_destroy_cq >> Feb 15 16:33:28 wci11 kernel: ib_rdma_lat: Unknown symbol ib_destroy_cq >> Feb 15 16:33:28 wci11 kernel: ib_rdma_lat: disagrees about version of >> symbol ib_query_port >> Feb 15 16:33:28 wci11 kernel: ib_rdma_lat: Unknown symbol ib_query_port >> Feb 15 16:33:28 wci11 kernel: ib_rdma_lat: disagrees about version of >> symbol ib_alloc_pd >> Feb 15 16:33:28 wci11 kernel: ib_rdma_lat: Unknown symbol ib_alloc_pd >> Feb 15 16:33:28 wci11 kernel: ib_rdma_lat: disagrees about version of >> symbol ib_create_qp >> Feb 15 16:33:28 wci11 kernel: ib_rdma_lat: Unknown symbol ib_create_qp >> Feb 15 16:33:28 wci11 kernel: ib_rdma_lat: disagrees about version of >> symbol ib_modify_qp >> Feb 15 16:33:28 wci11 kernel: ib_rdma_lat: Unknown symbol ib_modify_qp >> Feb 15 16:33:28 wci11 kernel: ib_rdma_lat: disagrees about version of >> symbol ib_destroy_qp >> Feb 15 16:33:28 wci11 kernel: ib_rdma_lat: Unknown symbol ib_destroy_qp >> Feb 15 16:33:28 wci11 kernel: ib_rdma_lat: disagrees about version of >> symbol ib_dealloc_pd >> Feb 15 16:33:28 wci11 kernel: ib_rdma_lat: Unknown symbol ib_dealloc_pd >> >> I'm running rhel5. I have rebooted the node many times but didn't >> help at all. >> >> > > From OFED_tips: > 4. External Module Compilation Over OFED-1.4 > === > > To build kernel modules depending on OFED's modules, take the > Modules.symvers > file from /src/openib/Module.symvers (part of the kernel-ib-devel > RPM) > and copy it to the modules subdir and then compile your module. > > If /src/openib/Module.symvers does not exist or it is empty, use the > create_Module.symvers.sh (a part of the ofed-docs RPM) script to create the > Module.symvers file. > > See "Module versioning & Module.symvers" in the modules.txt from kernel > documentation (e.g. linux-2.6.20/Documentation/kbuild/modules.txt). > > > Tziporet > > ___ general mailing list general@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [ofa-general] IB function calls in kernel module fail
The problem solved following your advice. Thanks a ton!! On Mon, Feb 16, 2009 at 6:19 AM, Tziporet Koren wrote: > neutron wrote: >> >> Hi all, >> >> I'm writing a kernel module that make use of basic IB verbs to >> communicate, like: >> ib_register_client, ib_unregister_client, ib_alloc_pd, >> ib_create_qp, ib_reg_phys_mr, etc. >> >> I can compile the code into a kernel module: ib_rdma_lat.ko. This >> module is to test the RDMA write latency from kernel module. >> >> But when I "insmod", I got error reports at /var/log/messages: >> >> Feb 15 16:33:28 wci11 kernel: ib_rdma_lat: disagrees about version of >> symbol ib_unregister_client >> Feb 15 16:33:28 wci11 kernel: ib_rdma_lat: Unknown symbol >> ib_unregister_client >> Feb 15 16:33:28 wci11 kernel: ib_rdma_lat: disagrees about version of >> symbol ib_create_cq >> Feb 15 16:33:28 wci11 kernel: ib_rdma_lat: Unknown symbol ib_create_cq >> Feb 15 16:33:28 wci11 kernel: ib_rdma_lat: disagrees about version of >> symbol ib_reg_phys_mr >> Feb 15 16:33:28 wci11 kernel: ib_rdma_lat: Unknown symbol ib_reg_phys_mr >> Feb 15 16:33:28 wci11 kernel: ib_rdma_lat: disagrees about version of >> symbol ib_dereg_mr >> Feb 15 16:33:28 wci11 kernel: ib_rdma_lat: Unknown symbol ib_dereg_mr >> Feb 15 16:33:28 wci11 kernel: ib_rdma_lat: disagrees about version of >> symbol ib_register_client >> Feb 15 16:33:28 wci11 kernel: ib_rdma_lat: Unknown symbol >> ib_register_client >> Feb 15 16:33:28 wci11 kernel: ib_rdma_lat: disagrees about version of >> symbol ib_destroy_cq >> Feb 15 16:33:28 wci11 kernel: ib_rdma_lat: Unknown symbol ib_destroy_cq >> Feb 15 16:33:28 wci11 kernel: ib_rdma_lat: disagrees about version of >> symbol ib_query_port >> Feb 15 16:33:28 wci11 kernel: ib_rdma_lat: Unknown symbol ib_query_port >> Feb 15 16:33:28 wci11 kernel: ib_rdma_lat: disagrees about version of >> symbol ib_alloc_pd >> Feb 15 16:33:28 wci11 kernel: ib_rdma_lat: Unknown symbol ib_alloc_pd >> Feb 15 16:33:28 wci11 kernel: ib_rdma_lat: disagrees about version of >> symbol ib_create_qp >> Feb 15 16:33:28 wci11 kernel: ib_rdma_lat: Unknown symbol ib_create_qp >> Feb 15 16:33:28 wci11 kernel: ib_rdma_lat: disagrees about version of >> symbol ib_modify_qp >> Feb 15 16:33:28 wci11 kernel: ib_rdma_lat: Unknown symbol ib_modify_qp >> Feb 15 16:33:28 wci11 kernel: ib_rdma_lat: disagrees about version of >> symbol ib_destroy_qp >> Feb 15 16:33:28 wci11 kernel: ib_rdma_lat: Unknown symbol ib_destroy_qp >> Feb 15 16:33:28 wci11 kernel: ib_rdma_lat: disagrees about version of >> symbol ib_dealloc_pd >> Feb 15 16:33:28 wci11 kernel: ib_rdma_lat: Unknown symbol ib_dealloc_pd >> >> I'm running rhel5. I have rebooted the node many times but didn't >> help at all. >> >> > > From OFED_tips: > 4. External Module Compilation Over OFED-1.4 > === > > To build kernel modules depending on OFED's modules, take the > Modules.symvers > file from /src/openib/Module.symvers (part of the kernel-ib-devel > RPM) > and copy it to the modules subdir and then compile your module. > > If /src/openib/Module.symvers does not exist or it is empty, use the > create_Module.symvers.sh (a part of the ofed-docs RPM) script to create the > Module.symvers file. > > See "Module versioning & Module.symvers" in the modules.txt from kernel > documentation (e.g. linux-2.6.20/Documentation/kbuild/modules.txt). > > > Tziporet > > ___ general mailing list general@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [ofa-general] IB function calls in kernel module fail
neutron wrote: Hi all, I'm writing a kernel module that make use of basic IB verbs to communicate, like: ib_register_client, ib_unregister_client, ib_alloc_pd, ib_create_qp, ib_reg_phys_mr, etc. I can compile the code into a kernel module: ib_rdma_lat.ko. This module is to test the RDMA write latency from kernel module. But when I "insmod", I got error reports at /var/log/messages: Feb 15 16:33:28 wci11 kernel: ib_rdma_lat: disagrees about version of symbol ib_unregister_client Feb 15 16:33:28 wci11 kernel: ib_rdma_lat: Unknown symbol ib_unregister_client Feb 15 16:33:28 wci11 kernel: ib_rdma_lat: disagrees about version of symbol ib_create_cq Feb 15 16:33:28 wci11 kernel: ib_rdma_lat: Unknown symbol ib_create_cq Feb 15 16:33:28 wci11 kernel: ib_rdma_lat: disagrees about version of symbol ib_reg_phys_mr Feb 15 16:33:28 wci11 kernel: ib_rdma_lat: Unknown symbol ib_reg_phys_mr Feb 15 16:33:28 wci11 kernel: ib_rdma_lat: disagrees about version of symbol ib_dereg_mr Feb 15 16:33:28 wci11 kernel: ib_rdma_lat: Unknown symbol ib_dereg_mr Feb 15 16:33:28 wci11 kernel: ib_rdma_lat: disagrees about version of symbol ib_register_client Feb 15 16:33:28 wci11 kernel: ib_rdma_lat: Unknown symbol ib_register_client Feb 15 16:33:28 wci11 kernel: ib_rdma_lat: disagrees about version of symbol ib_destroy_cq Feb 15 16:33:28 wci11 kernel: ib_rdma_lat: Unknown symbol ib_destroy_cq Feb 15 16:33:28 wci11 kernel: ib_rdma_lat: disagrees about version of symbol ib_query_port Feb 15 16:33:28 wci11 kernel: ib_rdma_lat: Unknown symbol ib_query_port Feb 15 16:33:28 wci11 kernel: ib_rdma_lat: disagrees about version of symbol ib_alloc_pd Feb 15 16:33:28 wci11 kernel: ib_rdma_lat: Unknown symbol ib_alloc_pd Feb 15 16:33:28 wci11 kernel: ib_rdma_lat: disagrees about version of symbol ib_create_qp Feb 15 16:33:28 wci11 kernel: ib_rdma_lat: Unknown symbol ib_create_qp Feb 15 16:33:28 wci11 kernel: ib_rdma_lat: disagrees about version of symbol ib_modify_qp Feb 15 16:33:28 wci11 kernel: ib_rdma_lat: Unknown symbol ib_modify_qp Feb 15 16:33:28 wci11 kernel: ib_rdma_lat: disagrees about version of symbol ib_destroy_qp Feb 15 16:33:28 wci11 kernel: ib_rdma_lat: Unknown symbol ib_destroy_qp Feb 15 16:33:28 wci11 kernel: ib_rdma_lat: disagrees about version of symbol ib_dealloc_pd Feb 15 16:33:28 wci11 kernel: ib_rdma_lat: Unknown symbol ib_dealloc_pd I'm running rhel5. I have rebooted the node many times but didn't help at all. From OFED_tips: 4. External Module Compilation Over OFED-1.4 === To build kernel modules depending on OFED's modules, take the Modules.symvers file from /src/openib/Module.symvers (part of the kernel-ib-devel RPM) and copy it to the modules subdir and then compile your module. If /src/openib/Module.symvers does not exist or it is empty, use the create_Module.symvers.sh (a part of the ofed-docs RPM) script to create the Module.symvers file. See "Module versioning & Module.symvers" in the modules.txt from kernel documentation (e.g. linux-2.6.20/Documentation/kbuild/modules.txt). Tziporet ___ general mailing list general@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
