It seems that lustre puts its modules in /lib/modules/2.6.16.21-0.8-default despite the fact that my kernel is 2.6.16.21-0.8-smp !
uname -a Linux n32 2.6.16.21-0.8-smp #4 SMP Sun Sep 24 08:47:30 BST 2006 i686 i686 i386 GNU/Linux make[3]: Nothing to be done for `install-exec-am'. /bin/sh ../../mkinstalldirs /lib/modules/2.6.16.21-0.8-default/kernel/fs/lustre /usr/bin/install -c -m 644 lquota.ko /lib/modules/2.6.16.21-0.8-default/kernel/fs/lustre/lquota I therefore ends up with a /lib/modules/2.6.16.21-0.8-smp and /lib/modules/2.6.16.21-0.8-default i'm now searching why lustre thinks my kernel is 2.6.16.21-0.8-default and not 2.6.16.21-0.8-smp Thierry. On Mon, 25 Sep 2006, Thierry Delaitre wrote: > > On Mon, 25 Sep 2006, Michael S. Tsirkin wrote: > > > Quoting r. Thierry Delaitre <[EMAIL PROTECTED]>: > > > > > > I've set the o2ib path to /usr/local/ofed/src/openib-1.1 as shown in the > > > lustre's configure line below. Lustre's configure script looks for a > > > driver/infiniband directory which only seems to exist under > > > /usr/local/ofed/src/openib-1.1 > > > > > > ./configure --with-o2ib=/usr/local/ofed/src/openib-1.1/ > > > > > > Thierry. > > > > > > > replace /usr/local/ofed with the prefix you specified. > > > > This looks wrong - openib-1.1 is the pristine sources. > > openib/include is the exported interface and is what you should use > > for dependent modules. > > No idea why would lustre need drivers/infiniband. > > Try creating a softlink: > > > > mkdir /usr/local/ofed/src/openib/drivers/infiniband > > ln -s /usr/local/ofed/src/openib/include > > /usr/local/ofed/src/openib/drivers/infiniband > > I untarred lustre 1.5.95, compiled it (./configure > --with-o2ib=/usr/local/ofed/src/openib) . did a make install, depmod -a > and still get the following: > > my modprobe.conf is the following > > options lnet ip2nets="o2ib0 161.74.83.[0-255]" > > lctl network up > LNET configure error 100: Network is down > > ko2iblnd: disagrees about version of symbol ib_create_cq > ko2iblnd: Unknown symbol ib_create_cq > ko2iblnd: disagrees about version of symbol ib_dereg_mr > ko2iblnd: Unknown symbol ib_dereg_mr > ko2iblnd: disagrees about version of symbol ib_destroy_cq > ko2iblnd: Unknown symbol ib_destroy_cq > ko2iblnd: disagrees about version of symbol ib_get_dma_mr > ko2iblnd: Unknown symbol ib_get_dma_mr > ko2iblnd: disagrees about version of symbol ib_alloc_pd > ko2iblnd: Unknown symbol ib_alloc_pd > ko2iblnd: disagrees about version of symbol ib_modify_qp > ko2iblnd: Unknown symbol ib_modify_qp > ko2iblnd: disagrees about version of symbol ib_dealloc_pd > ko2iblnd: Unknown symbol ib_dealloc_pd > LustreError: 4177:0:(api-ni.c:1002:lnet_startup_lndnis()) Can't load LND > o2ib, module ko2iblnd, rc=256 > > lsmod | grep ib > libcfs 103060 1 lnet > ib_ucm 19332 0 > ib_addr 10756 1 rdma_cm > ib_cm 31968 2 ib_ucm,rdma_cm > ib_ipoib 48400 0 > ib_sa 16652 3 rdma_cm,ib_cm,ib_ipoib > ib_uverbs 38312 2 rdma_ucm,ib_ucm > ib_umad 17968 0 > ib_mthca 116240 0 > ib_mad 36116 4 ib_cm,ib_sa,ib_umad,ib_mthca > ib_core 49024 9 > ib_ucm,rdma_cm,ib_cm,ib_ipoib,ib_sa,ib_uverbs,ib_umad,ib_mthca,ib_mad > > nm /lib/modules/2.6.16.21-0.8-smp/kernel/drivers/infiniband/core/ib_core.ko | > grep ib_alloc_pd > d5dcb698 A __crc_ib_alloc_pd > 0000001c r __kcrctab_ib_alloc_pd > 0000006a r __kstrtab_ib_alloc_pd > 00000038 r __ksymtab_ib_alloc_pd > 00000c65 T ib_alloc_pd > > from lustre's config.log: > > configure:6500: checking whether to enable OpenIB gen2 support > configure:6586: cp conftest.c build && make modules CC=gcc -f > /root/lustre-1.5.95/build/Makefile LUSTRE_LINUX > _CONFIG=/usr/src/linux/.config -o tmp_include_depends -o scripts -o > include/config/MARKER -C /usr/src/linux > EXTRA_CFLAGS=-Werror-implicit-function-declaration -g > -I/root/lustre-1.5.95/lnet/include -I/root/lustre-1.5.95/lustre/include > -I/usr/local/ofed/src/openib/include M=/root/lustre-1.5.95/build > /root/lustre-1.5.95/build/conftest.c:42: warning: function declaration > isn't a prototype > /root/lustre-1.5.95/build/conftest.c: In function 'main': > /root/lustre-1.5.95/build/conftest.c:49: warning: unused variable 'rej_reason' > /root/lustre-1.5.95/build/conftest.c:48: warning: unused variable 'pool_fmr' > /root/lustre-1.5.95/build/conftest.c:47: warning: unused variable 'qp_attr' > /root/lustre-1.5.95/build/conftest.c:46: warning: unused variable > 'device_attr' > /root/lustre-1.5.95/build/conftest.c:45: warning: unused variable 'conn_param' > WARNING: "rdma_create_id" [/root/lustre-1.5.95/build/conftest.ko] undefined! > configure:6589: $? = 0 > configure:6591: test -s build/conftest.o > configure:6594: $? = 0 > configure:6597: result: yes > > > Thierry. > > _______________________________________________ > openib-general mailing list > [email protected] > http://openib.org/mailman/listinfo/openib-general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general > > ---------------------------------------- Dr Thierry DELAITRE Systems and Services Manager, CSCS University of Westminster 115 New Cavendish Street, London W1W 6UW Tel: 020 7911 5000 ext: 3586 Fax: 020 7911 5089 Mobile short dial code 1788 http://www.cscs.wmin.ac.uk/~delaitt ---------------------------------------- This e-mail and its attachments are intended for the above named only and may be confidential. If they have come to you in error you must not copy or show them to anyone, nor should you take any action based on them, other than to notify the error by replying to the sender. _______________________________________________ openib-general mailing list [email protected] http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
