On 11/19/2013 10:49 PM, Alexandre Fournier wrote:
Hello,
We are experiencing strange behavior when writing file on the Gluster
mount point. On some occasion, when writing to the Gluster Mount we
have an Open Stream error. We've looked the gluster logs and found
the following faulty entries :
[From /var/log/glusterfs/mnt-gv0.log]
[2013-11-18 18:18:43.052446] W
[afr-common.c:1411:afr_conflicting_iattrs] 0-gv0-replicate-0:
/aa/aa/aa/aa: filetype differs on subvolumes (0, 1)
[2013-11-18 18:18:43.052468] E
[afr-self-heal-common.c:1409:afr_sh_common_lookup_cbk]
0-gv0-replicate-0: Conflicting entries for /aa/aa/aa/aa
[2013-11-18 18:18:43.052757] E
[afr-self-heal-common.c:2160:afr_self_heal_completion_cbk]
0-gv0-replicate-0: background meta-data data entry missing-entry gfid
self-heal
failed on /aa/aa/aa/aa/aa
[2013-11-18 18:18:43.052780] W [fuse-bridge.c:292:fuse_entry_cbk]
0-glusterfs-fuse: 439382194: LOOKUP() /aa/aa/aa/aa/aa => -1
(Input/output error)
We've looked at the log file etc-glusterfs-glusterd.vol.log but we
found nothing related to this problem. Then, we've looked at the log
From /var/log/glusterfs/bricks/mnt-data.log and we found 70 gig of
logs of the same type :
[2013-11-19 17:13:32.269757] W [posix-handle.c:538:posix_handle_soft]
0-gv0-posix: symlink
../../ab/fe/abfeb61c-501d-4417-b8fb-0accdd57146f/cf ->
/mnt/data/.glusterfs/ab/fe/abfeb61c-501d-4417-b8fb-0accdd57146f/cf
failed (No such file or directory)
[2013-11-19 17:13:32.269978] W [posix-handle.c:538:posix_handle_soft]
0-gv0-posix: symlink
../../c7/8b/c78be78f-cc95-47b2-a27f-4217f1759b67/d2 ->
/mnt/data/.glusterfs/c7/8b/c78be78f-cc95-47b2-a27f-4217f1759b67/d2
failed (No such file or directory)
[2013-11-19 17:13:32.270190] W [posix-handle.c:538:posix_handle_soft]
0-gv0-posix: symlink
../../5a/8f/5a8fa43c-4ccc-4d88-9122-a96bc8ffaebc/f2 ->
/mnt/data/.glusterfs/5a/8f/5a8fa43c-4ccc-4d88-9122-a96bc8ffaebc/f2
failed (No such file or directory)
This looks like a bug, unless there is something wrong with the set-up.
I have copied gluster-devel in this thread as I think they might help.
Just curious, is all your gluster nodes have equal time (i.e. ntp synced).
And it does not stop logging. It seems that the self heal is not
working properly when there are broking symlinks in the gluster. It
is worth saying also that this log is only produce on a single node
but the write fail on several node though. Also, we try to clean the
symlinks manually but it always come back.
Is it possible to recover from broken symlinks?
Configuration :
Gluster Version : 3.3.2
Cluster setup : 4 X 2
OS : Ubuntu
On Fuse
Thanks,
Alexandre
_______________________________________________
Gluster-users mailing list
[email protected]
http://supercolony.gluster.org/mailman/listinfo/gluster-users
_______________________________________________
Gluster-users mailing list
[email protected]
http://supercolony.gluster.org/mailman/listinfo/gluster-users
