Well,
Finally I started from scratch with a colleague and following the old
method is working well
./configure --with-o2ib=no --with-kmp-moddir=/lib/modules/$(uname -r)/extra
make
make debs
The process ends with an error because there's a glitch somewhere on the
generated filename
mv: cannot stat ‘../linux-patch-lustre__all.deb’: No such file or directory
...
The file name expected lacks the version number
But my packages are here and I can mount my lustre share
(still have to fix debian/control* for headers package name, and fix
version number in the kernel-source makefile provided by debian and run
make modules_prepare make scripts)
Le 01/12/2015 11:11, Jérôme BECOT a écrit :
thanks
They load .. huh .. at least, the kernel tries to load them first
without this. Still the symbol problem remains.
Le 01/12/2015 11:02, Thomas Stibor a écrit :
Looks like that still staging Lustre modules are loaded first,
and then the remaining newer compiled modules. To make sure that ONLY
Lustre kernel modules from "extra" directory are loaded one can do
the following:
DEPMOD_DIR='/etc/depmod.d'
mkdir -p ${DEPMOD_DIR}
echo "search extra built-in" > ${DEPMOD_DIR}/lustre.conf
depmod -a
This sets the search order in directory "extra" first and thus your
compiled modules
in /lib/modules/3.16.0-4-amd64/extra are loaded (not the staging one).
Cheers
Thomas
On 11/30/2015 06:18 PM, Jérôme BECOT wrote:
So,
I could move on with building the modules. They are successfully
build and installed but I had to change the module destination
directory in config/lustre-build-linux.m4 to target
/lib/modules/`uname -r`/extra directory (else it goes in "kernel"
subdirectory and they never get loaded)
Then it won't load anyway. I'm getting [440444.832446] lnet: no
symbol version for module_layout in dmesg
If i copy the Module.symvers generated into the lustre-release
folder to /usr/src/linux, then i get
[438311.953707] lnet: disagrees about version of symbol
libcfs_deregister_ioctl
[438311.953710] lnet: Unknown symbol libcfs_deregister_ioctl (err -22)
[438311.953725] lnet: Unknown symbol cfs_str2num_check (err 0)
[438311.953760] lnet: Unknown symbol cfs_gettok (err 0)
[438311.953782] lnet: Unknown symbol lprocfs_call_handler (err 0)
(this is a sample of various symbol error)
Any clue ? I'm close to make it now (i guess)
Le 28/11/2015 18:07, Dilger, Andreas a écrit :
The 2.3.64 version means you are using the in-kernel Lustre client
(confirmed by the waning messages about "staging"), and not the
2.7.x version from the Lustre master branch.
It looks like Ubuntu is building the in-kernel client, and your
modules are not being loaded.
Cheers, Andreas
On Nov 28, 2015, at 03:26, Jérôme BECOT
<[email protected]<mailto:[email protected]>> wrote:
Hi there,
We run lustre 2.6/2.7 on our Centos 6.6 (servers) and 7 (clients)
cluster. We have a few webservers running Debian that need to
access the storage. I followed the procedure given by Thomas Stibor
about Ubuntu 14 last year.
I could successfully compile the binaries and modules after some
digging. He also left an already compiled lustre 2.7.63 and modules
for kernel 3.16.0-4 online.
If I install his binaries, it works well. If I install the one
generated by the procedure, the modules don't load and a weird
thing happen. Running dmesg warns me about one surprising thing :
> With his packages
[212417.535369] LNet: HW CPU cores: 1, npartitions: 1
[212417.538430] alg: No test for adler32 (adler32-zlib)
[212417.538456] alg: No test for crc32 (crc32-table)
[212425.548907] Lustre: Lustre: Build Version:
v2_7_60_0-ge686e57-CHANGED-3.16.0-4-amd64
[212425.565330] LNet: Added LNI 172.27.7.118@tcp1 [8/256/0/180]
[212425.565354] LNet: Accept secure, port 988
[212425.595531] Lustre: Mounted lustre-client
With mine
[209942.090874] LNet: HW CPU cores: 1, npartitions: 1
[209942.092902] alg: No test for adler32 (adler32-zlib)
[209950.092501] lnet: module is from the staging directory, the
quality is unknown, you have been warned.
[209950.093589] lvfs: module is from the staging directory, the
quality is unknown, you have been warned.
[209950.094634] obdclass: module is from the staging directory, the
quality is unknown, you have been warned.
[209950.098595] Lustre: Lustre: Build Version:
v2_3_64_0-g6e62c21-CHANGED-3.9.0
[209950.099999] ptlrpc: module is from the staging directory, the
quality is unknown, you have been warned.
[209950.104615] ksocklnd: module is from the staging directory, the
quality is unknown, you have been warned.
[209950.105237] LNetError:
845:0:(linux-tcpip.c:82:libcfs_ipif_query()) Can't get flags for
interface eth0
[209950.105862] LNetError:
845:0:(socklnd.c:2824:ksocknal_startup()) Can't get interface eth0
info: -515
[209951.104194] LNetError: 105-4: Error -100 starting up LNI tcp
[209951.104852] LustreError:
845:0:(events.c:566:ptlrpc_init_portals()) network initialisation
failed
[209990.787541] ptlrpc: module is from the staging directory, the
quality is unknown, you have been warned.
I pulled the master git branch, and coul obtain
linux-patch-lustre_2.7.63.0-16-g8524994_all.deb
lustre-client-modules-3.16.7-ckt11-lustre-my-build_2.7.63.0-16-g8524994_amd64.deb
lustre-tests_2.7.63.0-16-g8524994_amd64.deb
lustre_2.7.63.0-16-g8524994_amd64.changes
lustre-dev_2.7.63.0-16-g8524994_amd64.deb
lustre-utils_2.7.63.0-16-g8524994_amd64.deb
lustre_2.7.63.0-16-g8524994.dsc lustre-release
lustre_2.7.63.0-16-g8524994.tar.gz
lustre-source_2.7.63.0-16-g8524994_all.deb
I just don't get it. Why the shown version of the module is 2.3 ?
I tried to compile from the 2.7 branch but the 2.7.0 version
doesn't compile with kernel 3.16, as suggested in LU-7042
I probably miss something. Sorry I'm not familiar with compilers
(usually the ./configure && make && make install works or warns you
about missing dependencies and i don't go farther).
Thank you
Jerome
PS : Here is the procedure followed to compile
apt-get install linux-headers-3.16.0-4-amd64 libtool automake
linux-source-3.16
git clone git://git.whamcloud.com/fs/lustre-release.git
cd /usr/src/
tar xf linux-source-3.16.tar.xz
cd /home/build/lustre-release/
vi debian/rules (edit so it doesn't stop with autogen because it
doesn't find the kernel sources)
(modify --with-linux= option with current path
--with-linux=/usr/src/linux-3.16 in kdist_config section)
aptitude install module-assistant libreadline-dev debhelper dpatch
libsnmp-dev quilt devscripts
vi debian/control
(modify the dependency for unsatisfied linux-headers called
linux-headers-something in jessie)
vi compile.sh
#!/bin/bash
unset DEBEMAIL
unset EMAIL
unset DEBFULLNAME
unset NAME
export DEBFULLNAME="Niemand Nobody"
export EMAIL="npcomplete at example.com<http://example.com>"
# Extract lustre version, replace "_" by "." and remove leading
letter "v".
LUSTRE_VERSION=$(echo `git describe` | sed -e "s/_/\./g" | cut -c2-)
# Add entry into debian/changelog such that packages have proper
version names.
dch --newversion $LUSTRE_VERSION --distribution unstable
--nomultimaint -t "Build from official master upstream."
#
sh ./autogen.sh
# Build debian packages.
dpkg-buildpackage
# Build modules.
#export MODULE_LOC=${PWD}
#cd /usr/src/linux
#make-kpkg modules_image --append-to-version -lustre-my-build
--revision `date +"%Y%m%d"`
Then i got stuck on "checking for external module build target" for
a while, fixed by running "make scripts" in the kernel source folder
It finally compiled !
_______________________________________________
lustre-discuss mailing list
[email protected]<mailto:[email protected]>
http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
--
Jérome BECOT
Administrateur Systèmes et Réseaux
Molécules à visée Thérapeutique par des approches in Silico (MTi)
Univ Paris Diderot, UMRS973 Inserm
Case 013
Bât. Lamarck A, porte 412
35, rue Hélène Brion 75205 Paris Cedex 13
France
Tel : 01 57 27 83 82
_______________________________________________
lustre-discuss mailing list
[email protected]
http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
