Hi Amar, Thanks for the quick reply, but that doesn't work either. I just get more holes and overlaps:
2009-02-06 15:04:04 E [dht-layout.c:460:dht_layout_normalize] dist1: found anomalies in /. holes=3 overlaps=9 2009-02-06 15:04:04 W [dht-common.c:137:dht_lookup_dir_cbk] dist1: fixing assignment on / 2009-02-06 15:04:04 E [dht-selfheal.c:422:dht_selfheal_directory] dist1: the directory is not a virgin 2009-02-06 15:04:04 W [fuse-bridge.c:297:need_fresh_lookup] fuse-bridge: revalidate of / failed (Structure needs cleaning) 2009-02-06 15:04:04 E [dht-layout.c:460:dht_layout_normalize] dist1: found anomalies in /. holes=3 overlaps=9 2009-02-06 15:04:04 W [dht-common.c:137:dht_lookup_dir_cbk] dist1: fixing assignment on / 2009-02-06 15:04:04 E [dht-selfheal.c:422:dht_selfheal_directory] dist1: the directory is not a virgin 2009-02-06 15:04:04 E [fuse-bridge.c:404:fuse_entry_cbk] glusterfs-fuse: 2: LOOKUP() / => -1 (Structure needs cleaning) 2009-02-06 15:04:04 E [dht-layout.c:460:dht_layout_normalize] dist1: found anomalies in /. holes=3 overlaps=9 2009-02-06 15:04:04 W [dht-common.c:137:dht_lookup_dir_cbk] dist1: fixing assignment on / 2009-02-06 15:04:04 E [dht-selfheal.c:422:dht_selfheal_directory] dist1: the directory is not a virgin 2009-02-06 15:04:04 W [fuse-bridge.c:297:need_fresh_lookup] fuse-bridge: revalidate of / failed (Structure needs cleaning) 2009-02-06 15:04:04 E [dht-layout.c:460:dht_layout_normalize] dist1: found anomalies in /. holes=3 overlaps=9 2009-02-06 15:04:04 W [dht-common.c:137:dht_lookup_dir_cbk] dist1: fixing assignment on / 2009-02-06 15:04:04 E [dht-selfheal.c:422:dht_selfheal_directory] dist1: the directory is not a virgin 2009-02-06 15:04:04 E [fuse-bridge.c:404:fuse_entry_cbk] glusterfs-fuse: 3: LOOKUP() / => -1 (Structure needs cleaning) ________________________________ From: Amar Tumballi (bulde) <[email protected]> To: Craig Flockhart <[email protected]> Cc: [email protected] Sent: Friday, February 6, 2009 2:37:08 PM Subject: Re: [Gluster-users] starting 4th node in 4 node dht cluster fails Hi Craig, As you are using 'distribute' (client side) over 'distribute' (server side), this will not be working right now. To get it working right now, you can have 4 export volumes from each server exported, and in client have 4x4 client protocol volumes, which you can aggregate with a single 'cluster/distribute' (which will have 16 subvolumes). To get the below mentioned configuration working as is, you need to wait for a week more IMO. Regards, Amar 2009/2/6 Craig Flockhart <[email protected]> Using dht translator to cluster together 4 nodes each with 4 disks. Starting glusterfs on the 4th causes "Structure needs cleaning" when ls-ing the mount point on any of them. It's fine with 3 nodes started. Using fuse-2.7.4 GlusterFS 2.0.0rc1 Linux 2.6.18-53.el5 kernel Errors from the log: 2009-02-06 15:23:51 E [dht-layout.c:460:dht_layout_normalize] dist1: found anomalies in /. holes=1 overlaps=3 2009-02-06 15:23:51 W [dht-common.c:137:dht_lookup_dir_cbk] dist1: fixing assignment on / 2009-02-06 15:23:51 E [dht-selfheal.c:422:dht_selfheal_directory] dist1: the directory is not a virgin 2009-02-06 15:23:51 W [fuse-bridge.c:297:need_fresh_lookup] fuse-bridge: revalidate of / failed (Structure needs cleaning) 2009-02-06 15:23:51 E [dht-layout.c:460:dht_layout_normalize] dist1: found anomalies in /. holes=1 overlaps=3 2009-02-06 15:23:51 W [dht-common.c:137:dht_lookup_dir_cbk] dist1: fixing assignment on / 2009-02-06 15:23:51 E [dht-selfheal.c:422:dht_selfheal_directory] dist1: the directory is not a virgin 2009-02-06 15:23:51 E [fuse-bridge.c:404:fuse_entry_cbk] glusterfs-fuse: 2: LOOKUP() / => -1 (Structure needs cleaning) Config for one of the machines: volume posix-d1 type storage/posix option directory /mnt/chard1/export end-volume volume locks1 type features/locks subvolumes posix-d1 end-volume volume posix-d2 type storage/posix option directory /mnt/chard2/export end-volume volume locks2 type features/locks subvolumes posix-d2 end-volume volume posix-d3 type storage/posix option directory /mnt/chard3/export end-volume volume locks3 type features/locks subvolumes posix-d3 end-volume volume posix-d4 type storage/posix option directory /mnt/chard4/export end-volume volume locks4 type features/locks subvolumes posix-d4 end-volume volume home-ns type storage/posix option directory /var/local/glusterfs/namespace1 end-volume volume home type cluster/distribute subvolumes locks1 locks2 locks3 locks4 end-volume volume server type protocol/server option transport-type tcp subvolumes home option auth.addr.home.allow * end-volume volume zwei type protocol/client option transport-type tcp option remote-host zwei option remote-subvolume home end-volume volume char type protocol/client option transport-type tcp option remote-host char option remote-subvolume home end-volume volume pente type protocol/client option transport-type tcp option remote-host pente option remote-subvolume home end-volume volume tres type protocol/client option transport-type tcp option remote-host tres option remote-subvolume home end-volume volume dist1 type cluster/distribute subvolumes pente char tres zwei end-volume _______________________________________________ Gluster-users mailing list [email protected] http://zresearch.com/cgi-bin/mailman/listinfo/gluster-users -- Amar Tumballi Gluster/GlusterFS Hacker [bulde on #gluster/irc.gnu.org] http://www.zresearch.com - Commoditizing Super Storage!
_______________________________________________ Gluster-users mailing list [email protected] http://zresearch.com/cgi-bin/mailman/listinfo/gluster-users
