Re: [Gluster-users] Fedora 17 GlusterFS 3.3.0 problmes

Nathan Stratton Fri, 22 Jun 2012 13:49:57 -0700

On Fri, 22 Jun 2012, Harry Mangalam wrote:

Were you automounting your glusterfs?


This is extremely similar to what I described previously:

<http://comments.gmane.org/gmane.comp.file-systems.gluster.user/9241>

 In my case the servers were running SL6.2; clients were running
CentOS 5.7 and were automounting the glusterfs with IPoIB.  When we
switched to hard mounts, the problem seems to have gone away, but I'd
be interested in seeing this resolved.  The "Client lk-version numbers
are not same, reopening the fds" strings in the logs were also
identical, even when I compiled new client versions that matched the
server version.

If this does not have a trivial answer, I'll be happy to file an
official bug report.  One report could be a twit :). second one could
be a bug.

I was automounting, however I get the same problem when I manually mountwith:


mount -t glusterfs localhost:/share /share

<>

Nathan Stratton
nathan at robotics.net
http://www.robotics.net

hjm

On Fri, Jun 22, 2012 at 7:49 AM, Nathan Stratton <[email protected]> wrote:


When I do a NFS mount and do a ls I get:

[root@ovirt share]# ls
ls: reading directory .: Too many levels of symbolic links

[root@ovirt share]# ls -fl
ls: reading directory .: Too many levels of symbolic links
total 3636
drwxr-xr-x   3 root root 16384 Jun 21 19:34 .
dr-xr-xr-x. 21 root root  4096 Jun 21 19:29 ..
drwxr-xr-x   3 root root 16384 Jun 21 19:34 .
dr-xr-xr-x. 21 root root  4096 Jun 21 19:29 ..
drwxr-xr-x   3 root root 16384 Jun 21 19:34 .
dr-xr-xr-x. 21 root root  4096 Jun 21 19:29 ..
drwxr-xr-x   3 root root 16384 Jun 21 19:34 .
dr-xr-xr-x. 21 root root  4096 Jun 21 19:29 ..
drwxr-xr-x   3 root root 16384 Jun 21 19:34 .

{and so on}

When I try a fuse mount I get:
[client-handshake.c:1445:client_setvolume_cbk] 0-share-client-0: Server and
Client lk-version numbers are not same, reopening the fds

I have tried the fedora 16 RPMs and also built new fedora 17 RPMs.

Full Log:

[2012-06-21 19:24:35.633510] I [glusterfsd.c:1666:main]
0-/usr/sbin/glusterfs: Started running /usr/sbin/glusterfs version 3.3.0
[2012-06-21 19:24:35.646832] I [io-cache.c:1549:check_cache_size_ok]
0-share-quick-read: Max cache size is 16825450496
[2012-06-21 19:24:35.646916] I [io-cache.c:1549:check_cache_size_ok]
0-share-io-cache: Max cache size is 16825450496
[2012-06-21 19:24:35.660807] I [client.c:2142:notify] 0-share-client-0:
parent translators are ready, attempting connect on transport
[2012-06-21 19:24:35.664026] I [client.c:2142:notify] 0-share-client-1:
parent translators are ready, attempting connect on transport
[2012-06-21 19:24:35.666801] I [client.c:2142:notify] 0-share-client-2:
parent translators are ready, attempting connect on transport
[2012-06-21 19:24:35.669385] I [client.c:2142:notify] 0-share-client-3:
parent translators are ready, attempting connect on transport
[2012-06-21 19:24:35.671951] I [client.c:2142:notify] 0-share-client-4:
parent translators are ready, attempting connect on transport
[2012-06-21 19:24:35.674514] I [client.c:2142:notify] 0-share-client-5:
parent translators are ready, attempting connect on transport
[2012-06-21 19:24:35.677093] I [client.c:2142:notify] 0-share-client-6:
parent translators are ready, attempting connect on transport
[2012-06-21 19:24:35.679652] I [client.c:2142:notify] 0-share-client-7:
parent translators are ready, attempting connect on transport
Given volfile:
+------------------------------------------------------------------------------+
 1: volume share-client-0
 2:     type protocol/client
 3:     option remote-host virt01.casabi.net
 4:     option remote-subvolume /export
 5:     option transport-type tcp
 6:     option username f0a71b3a-1cb2-4a38-8620-f3853aa90798
 7:     option password 9641cac2-de93-4621-b94b-9320ada20fd7
 8: end-volume
 9:
 10: volume share-client-1
 11:     type protocol/client
 12:     option remote-host virt02.casabi.net
 13:     option remote-subvolume /export
 14:     option transport-type tcp
 15:     option username f0a71b3a-1cb2-4a38-8620-f3853aa90798
 16:     option password 9641cac2-de93-4621-b94b-9320ada20fd7
 17: end-volume
 18:
 19: volume share-client-2
 20:     type protocol/client
 21:     option remote-host virt03.casabi.net
 22:     option remote-subvolume /export
 23:     option transport-type tcp
 24:     option username f0a71b3a-1cb2-4a38-8620-f3853aa90798
 25:     option password 9641cac2-de93-4621-b94b-9320ada20fd7
 26: end-volume
 27:
 28: volume share-client-3
 29:     type protocol/client
 30:     option remote-host virt04.casabi.net
 31:     option remote-subvolume /export
 32:     option transport-type tcp
 33:     option username f0a71b3a-1cb2-4a38-8620-f3853aa90798
 34:     option password 9641cac2-de93-4621-b94b-9320ada20fd7
 35: end-volume
 36:
 37: volume share-client-4
 38:     type protocol/client
 39:     option remote-host virt05.casabi.net
 40:     option remote-subvolume /export
 41:     option transport-type tcp
 42:     option username f0a71b3a-1cb2-4a38-8620-f3853aa90798
 43:     option password 9641cac2-de93-4621-b94b-9320ada20fd7
 44: end-volume
 45:
 46: volume share-client-5
 47:     type protocol/client
 48:     option remote-host virt06.casabi.net
 49:     option remote-subvolume /export
 50:     option transport-type tcp
 51:     option username f0a71b3a-1cb2-4a38-8620-f3853aa90798
 52:     option password 9641cac2-de93-4621-b94b-9320ada20fd7
 53: end-volume
 54:
 55: volume share-client-6
 56:     type protocol/client
 57:     option remote-host virt07.casabi.net
 58:     option remote-subvolume /export
 59:     option transport-type tcp
 60:     option username f0a71b3a-1cb2-4a38-8620-f3853aa90798
 61:     option password 9641cac2-de93-4621-b94b-9320ada20fd7
 62: end-volume
 63:
 64: volume share-client-7
 65:     type protocol/client
 66:     option remote-host virt08.casabi.net
 67:     option remote-subvolume /export
 68:     option transport-type tcp
 69:     option username f0a71b3a-1cb2-4a38-8620-f3853aa90798
 70:     option password 9641cac2-de93-4621-b94b-9320ada20fd7
 71: end-volume
 72:
 73: volume share-replicate-0
 74:     type cluster/replicate
 75:     subvolumes share-client-0 share-client-1
 76: end-volume
 77:
 78: volume share-replicate-1
 79:     type cluster/replicate
 80:     subvolumes share-client-2 share-client-3
 81: end-volume
 82:
 83: volume share-replicate-2
 84:     type cluster/replicate
 85:     subvolumes share-client-4 share-client-5
 86: end-volume
 87:
 88: volume share-replicate-3
 89:     type cluster/replicate
 90:     subvolumes share-client-6 share-client-7
 91: end-volume
 92:
 93: volume share-dht
 94:     type cluster/distribute
 95:     subvolumes share-replicate-0 share-replicate-1 share-replicate-2
share-replicate-3
 96: end-volume
 97:
 98: volume share-write-behind
 99:     type performance/write-behind
100:     subvolumes share-dht
101: end-volume
102:
103: volume share-read-ahead
104:     type performance/read-ahead
105:     subvolumes share-write-behind
106: end-volume
107:
108: volume share-io-cache
109:     type performance/io-cache
110:     subvolumes share-read-ahead
111: end-volume
112:
113: volume share-quick-read
114:     type performance/quick-read
115:     subvolumes share-io-cache
116: end-volume
117:
118: volume share-md-cache
119:     type performance/md-cache
120:     subvolumes share-quick-read
121: end-volume
122:
123: volume share
124:     type debug/io-stats
125:     option latency-measurement off
126:     option count-fop-hits off
127:     subvolumes share-md-cache
128: end-volume

+------------------------------------------------------------------------------+
[2012-06-21 19:24:35.682819] I [rpc-clnt.c:1660:rpc_clnt_reconfig]
0-share-client-0: changing port to 24009 (from 0)
[2012-06-21 19:24:35.682973] I [rpc-clnt.c:1660:rpc_clnt_reconfig]
0-share-client-1: changing port to 24010 (from 0)
[2012-06-21 19:24:35.683014] I [rpc-clnt.c:1660:rpc_clnt_reconfig]
0-share-client-2: changing port to 24010 (from 0)
[2012-06-21 19:24:35.683039] I [rpc-clnt.c:1660:rpc_clnt_reconfig]
0-share-client-3: changing port to 24010 (from 0)
[2012-06-21 19:24:35.683121] I [rpc-clnt.c:1660:rpc_clnt_reconfig]
0-share-client-5: changing port to 24010 (from 0)
[2012-06-21 19:24:35.683162] I [rpc-clnt.c:1660:rpc_clnt_reconfig]
0-share-client-4: changing port to 24010 (from 0)
[2012-06-21 19:24:35.683206] I [rpc-clnt.c:1660:rpc_clnt_reconfig]
0-share-client-7: changing port to 24010 (from 0)
[2012-06-21 19:24:35.683246] I [rpc-clnt.c:1660:rpc_clnt_reconfig]
0-share-client-6: changing port to 24010 (from 0)
[2012-06-21 19:24:39.644198] I
[client-handshake.c:1636:select_server_supported_programs] 0-share-client-0:
Using Program GlusterFS 3.3.0, Num (1298437), Version (330)
[2012-06-21 19:24:39.644493] I
[client-handshake.c:1433:client_setvolume_cbk] 0-share-client-0: Connected
to 10.59.0.11:24009, attached to remote volume '/export'.
[2012-06-21 19:24:39.644512] I
[client-handshake.c:1445:client_setvolume_cbk] 0-share-client-0: Server and
Client lk-version numbers are not same, reopening the fds
[2012-06-21 19:24:39.644579] I [afr-common.c:3627:afr_notify]
0-share-replicate-0: Subvolume 'share-client-0' came back up; going online.
[2012-06-21 19:24:39.644621] I
[client-handshake.c:453:client_set_lk_version_cbk] 0-share-client-0: Server
lk version = 1
[2012-06-21 19:24:39.646923] I
[client-handshake.c:1636:select_server_supported_programs] 0-share-client-1:
Using Program GlusterFS 3.3.0, Num (1298437), Version (330)
[2012-06-21 19:24:39.647207] I
[client-handshake.c:1433:client_setvolume_cbk] 0-share-client-1: Connected
to 10.59.0.12:24010, attached to remote volume '/export'.
[2012-06-21 19:24:39.647227] I
[client-handshake.c:1445:client_setvolume_cbk] 0-share-client-1: Server and
Client lk-version numbers are not same, reopening the fds
[2012-06-21 19:24:39.647371] I
[client-handshake.c:453:client_set_lk_version_cbk] 0-share-client-1: Server
lk version = 1
[2012-06-21 19:24:39.649587] I
[client-handshake.c:1636:select_server_supported_programs] 0-share-client-2:
Using Program GlusterFS 3.3.0, Num (1298437), Version (330)
[2012-06-21 19:24:39.649806] I
[client-handshake.c:1433:client_setvolume_cbk] 0-share-client-2: Connected
to 10.59.0.13:24010, attached to remote volume '/export'.
[2012-06-21 19:24:39.649826] I
[client-handshake.c:1445:client_setvolume_cbk] 0-share-client-2: Server and
Client lk-version numbers are not same, reopening the fds
[2012-06-21 19:24:39.649865] I [afr-common.c:3627:afr_notify]
0-share-replicate-1: Subvolume 'share-client-2' came back up; going online.
[2012-06-21 19:24:39.649935] I
[client-handshake.c:453:client_set_lk_version_cbk] 0-share-client-2: Server
lk version = 1
[2012-06-21 19:24:39.652192] I
[client-handshake.c:1636:select_server_supported_programs] 0-share-client-3:
Using Program GlusterFS 3.3.0, Num (1298437), Version (330)
[2012-06-21 19:24:39.652428] I
[client-handshake.c:1433:client_setvolume_cbk] 0-share-client-3: Connected
to 10.59.0.14:24010, attached to remote volume '/export'.
[2012-06-21 19:24:39.652448] I
[client-handshake.c:1445:client_setvolume_cbk] 0-share-client-3: Server and
Client lk-version numbers are not same, reopening the fds
[2012-06-21 19:24:39.652566] I
[client-handshake.c:453:client_set_lk_version_cbk] 0-share-client-3: Server
lk version = 1
[2012-06-21 19:24:39.654706] I
[client-handshake.c:1636:select_server_supported_programs] 0-share-client-5:
Using Program GlusterFS 3.3.0, Num (1298437), Version (330)
[2012-06-21 19:24:39.654910] I
[client-handshake.c:1433:client_setvolume_cbk] 0-share-client-5: Connected
to 10.59.0.16:24010, attached to remote volume '/export'.
[2012-06-21 19:24:39.654926] I
[client-handshake.c:1445:client_setvolume_cbk] 0-share-client-5: Server and
Client lk-version numbers are not same, reopening the fds
[2012-06-21 19:24:39.654961] I [afr-common.c:3627:afr_notify]
0-share-replicate-2: Subvolume 'share-client-5' came back up; going online.
[2012-06-21 19:24:39.655035] I
[client-handshake.c:453:client_set_lk_version_cbk] 0-share-client-5: Server
lk version = 1
[2012-06-21 19:24:39.657289] I
[client-handshake.c:1636:select_server_supported_programs] 0-share-client-4:
Using Program GlusterFS 3.3.0, Num (1298437), Version (330)
[2012-06-21 19:24:39.657511] I
[client-handshake.c:1433:client_setvolume_cbk] 0-share-client-4: Connected
to 10.59.0.15:24010, attached to remote volume '/export'.
[2012-06-21 19:24:39.657526] I
[client-handshake.c:1445:client_setvolume_cbk] 0-share-client-4: Server and
Client lk-version numbers are not same, reopening the fds
[2012-06-21 19:24:39.657646] I
[client-handshake.c:453:client_set_lk_version_cbk] 0-share-client-4: Server
lk version = 1
[2012-06-21 19:24:39.659884] I
[client-handshake.c:1636:select_server_supported_programs] 0-share-client-7:
Using Program GlusterFS 3.3.0, Num (1298437), Version (330)
[2012-06-21 19:24:39.660075] I
[client-handshake.c:1433:client_setvolume_cbk] 0-share-client-7: Connected
to 10.59.0.18:24010, attached to remote volume '/export'.
[2012-06-21 19:24:39.660091] I
[client-handshake.c:1445:client_setvolume_cbk] 0-share-client-7: Server and
Client lk-version numbers are not same, reopening the fds
[2012-06-21 19:24:39.660127] I [afr-common.c:3627:afr_notify]
0-share-replicate-3: Subvolume 'share-client-7' came back up; going online.
[2012-06-21 19:24:39.660202] I
[client-handshake.c:453:client_set_lk_version_cbk] 0-share-client-7: Server
lk version = 1
[2012-06-21 19:24:39.662579] I
[client-handshake.c:1636:select_server_supported_programs] 0-share-client-6:
Using Program GlusterFS 3.3.0, Num (1298437), Version (330)
[2012-06-21 19:24:39.662908] I
[client-handshake.c:1433:client_setvolume_cbk] 0-share-client-6: Connected
to 10.59.0.17:24010, attached to remote volume '/export'.
[2012-06-21 19:24:39.662931] I
[client-handshake.c:1445:client_setvolume_cbk] 0-share-client-6: Server and
Client lk-version numbers are not same, reopening the fds
[2012-06-21 19:24:39.668028] I [fuse-bridge.c:4193:fuse_graph_setup] 0-fuse:
switched to graph 0
[2012-06-21 19:24:39.668234] I
[client-handshake.c:453:client_set_lk_version_cbk] 0-share-client-6: Server
lk version = 1
[2012-06-21 19:24:39.668287] I [fuse-bridge.c:3376:fuse_init]
0-glusterfs-fuse: FUSE inited with protocol versions: glusterfs 7.13 kernel
7.18
[2012-06-21 19:24:39.668739] I
[afr-common.c:1964:afr_set_root_inode_on_first_lookup] 0-share-replicate-0:
added root inode
[2012-06-21 19:24:39.669232] I
[afr-common.c:1964:afr_set_root_inode_on_first_lookup] 0-share-replicate-1:
added root inode
[2012-06-21 19:24:39.669321] I
[afr-common.c:1964:afr_set_root_inode_on_first_lookup] 0-share-replicate-2:
added root inode
[2012-06-21 19:24:39.669379] I
[afr-common.c:1964:afr_set_root_inode_on_first_lookup] 0-share-replicate-3:
added root inode

<>


Nathan Stratton
nathan at robotics.net
http://www.robotics.net
_______________________________________________
Gluster-users mailing list
[email protected]
http://gluster.org/cgi-bin/mailman/listinfo/gluster-users

_______________________________________________
Gluster-users mailing list
[email protected]
http://gluster.org/cgi-bin/mailman/listinfo/gluster-users

Re: [Gluster-users] Fedora 17 GlusterFS 3.3.0 problmes

Reply via email to