Update: seems like the ubuntu installation was to blame. A "brand new" SLES 10 on lustre2 with 2.6.16 kernel actually solved the problem. Thanks, Andriy
On Tue, May 11, 2010 at 10:32 PM, Andriy Chut <[email protected]> wrote: > Sorry, it was just a bad copy-paste my side :( > actual content is lustre2:3334 for pvfs2-server.conf-lustre2 and > lustre1:3334 for pvfs2-server.conf-lustre1 > I even removed pvfs2-server.conf-lustre1 from /etc/pvfs2 @lustre2 and the > same (lustre1 file on lustre2), but seems like it's not the reason, > so I got them both back. > > > On Tue, May 11, 2010 at 9:50 PM, Kevin Harms <[email protected]> wrote: > >> >> This part here looks like your problem. lustre2 should have a different >> host id. When it starts it's using the same handle range as lustre1. >> >> There was also a fix in 2.7.x or 2.8.x for the BDB issue below. I think >> the fix requires adding DBT_USERMEM to all the flags. >> >> kevin >> >> > >> > *pvfs2-server.conf-lustre1* >> > >> > StorageSpace /pvfs2-storage-space >> > HostID "tcp://lustre1:3334" >> > LogFile /tmp/pvfs2-server.log >> > >> > *pvfs2-server.conf-lustre2* >> > >> > StorageSpace /pvfs2-storage-space >> > HostID "tcp://lustre1:3334" >> > LogFile /tmp/pvfs2-server.log >> > >> > MDS + I/O server log: >> > [D 05/11 23:13] PVFS2 Server version 1.5.1 starting. >> > >> > 2nd I/O server log >> > [D 05/11 23:14] PVFS2 Server version 1.5.1 starting. >> > [E 05/11 23:15] TROVE:DBPF:Berkeley DB: DB_THREAD mandates memory >> allocation flag on key DBT >> > [E 05/11 23:15] TROVE:DBPF:Berkeley DB: DB_THREAD mandates memory >> allocation flag on key DBT >> > >> > Berkeley version for lustre1 >> > Name : db4-devel >> > Arch : x86_64 >> > Version : 4.3.29 >> > Release : 10.el5 >> > >> > for lustre2 >> > i libdb-dev 4.7.25.3 >> Berkeley Database Libraries [development] >> > ii libdb4.7 4.7.25-7ubuntu2 >> Berkeley v4.7 Database Libraries [runtime] >> > ii libdb4.7-dev 4.7.25-7ubuntu2 >> Berkeley v4.7 >> > Thanks a lot, >> > Andriy >> > >> > On Tue, May 11, 2010 at 8:47 PM, Kevin Harms <[email protected]> >> wrote: >> > >> > Can you supply your fs.conf file? And perhaps the pvfs2-server.log from >> both nodes? >> > >> > kevin >> > >> > On May 11, 2010, at 10:45 AM, Andriy Chut wrote: >> > >> > > Hello everyone, >> > > >> > > Is there any way to deal with the problem of root handle ownership? >> > > My installation: >> > > MDS+ i/o server >> > > 1) SL 5.0 with 2.6.17-CITI_NFS4_ALL-pnfs-1 from >> > > http://www.citi.umich.edu/projects/asci/pnfs/linux/ and PVFS2 1.5.1 >> > > 2) i/o server: Ubuntu Server 9.04 with 2.6.31-14-server kernel. >> > > PVFS2 1.5.1 (the same version) >> > > >> > > 1 installation with 1 MDS + 1 i/o configuration works just fine, but >> as soon as I add a second node (re-generating configuration, deleting and >> even re-formating the storage), I have a small issue while executing >> pvfs2-ping: >> > > >> > > (7) Verifying that root handle is owned by one server... >> > > >> > > Root handle: 1048576 >> > > Failure: check root handle failed >> > > PVFS_mgmt_setparam_all: Detailed per-server errors are available >> > > >> > > Per-server errors: >> > > Server: tcp://lustre1:3334: Reports ownership of root handle >> > > >> > > >> > > # 2 handles for server 0 >> > > # 0 handles for server 1 >> > > File: <Root> >> > > handle = 1048576, type = Directory, server = 0 >> > > remaining handles: >> > > >> > > Is there (possibly) any way to get a clue on where the problem might >> be? >> > > (I've been checking whether there's just one server instance running >> on each machine). >> > > >> > > I know that is irrelevant for PVFS2 2.8.1 at least, but, >> unfortunately, 1.5.1 seems to be the last (for now) version supporting >> PVFS-pNFS, so I kind of have no option, but to stick to it. >> > > Any suggestions? >> > > Thanks, >> > > Andriy >> > > _______________________________________________ >> > > Pvfs2-users mailing list >> > > [email protected] >> > > http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users >> > >> > >> >> >
_______________________________________________ Pvfs2-users mailing list [email protected] http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users
