RE: [Gluster-devel] atomic operations fails_
--entry-timeout=0 did not make any visible change. Regards, Fredrik -Original Message- From: anand.av...@gmail.com [mailto:anand.av...@gmail.com] On Behalf Of Anand Avati Sent: den 10 februari 2010 05:00 To: Fredrik Widlund Cc: gluster-devel@nongnu.org Subject: Re: [Gluster-devel] atomic operations fails_ On Wed, Feb 10, 2010 at 12:48 AM, Fredrik Widlund fredrik.widl...@qbrick.com wrote: Hi, I'll try to make myself clearer than in the earlier thread, since I need some help here. I'm not sure if I am missing something. I am not able to use atomic operations using glusterfs 3.0.2, on a Arch Linux 2.6.32.7-1 x86_64 server. I've stripped everything down to the most simple configuration Server: a posix store with locks Client: a tcp client I mount the glusterfs on /gfs, and use a writer and a reader as below: [writer]r...@frustration while :; do date /gfs/.file; mv /gfs/.file /gfs/file; done [reader] r...@frustration while :; do cat /gfs/file /dev/null; done Leaving this running, I get a flood of cat: /gfs/file: No such file or directory This should not happen on a filesystem that supports atomic operations?! Please try turning off FUSE entry caching by passing --entry-timeout=0 to your mount command on the client side and let us know what difference that makes. Thanks, Avati ___ Gluster-devel mailing list Gluster-devel@nongnu.org http://lists.nongnu.org/mailman/listinfo/gluster-devel
RE: [Gluster-devel] atomic operations fails_
Can you reproduce this behavior on your end? Regards, Fredrik -Original Message- From: gluster-devel-bounces+fredrik.widlund=qbrick@nongnu.org [mailto:gluster-devel-bounces+fredrik.widlund=qbrick@nongnu.org] On Behalf Of Fredrik Widlund Sent: den 10 februari 2010 10:27 To: Anand Avati Cc: gluster-devel@nongnu.org Subject: RE: [Gluster-devel] atomic operations fails_ --entry-timeout=0 did not make any visible change. Regards, Fredrik -Original Message- From: anand.av...@gmail.com [mailto:anand.av...@gmail.com] On Behalf Of Anand Avati Sent: den 10 februari 2010 05:00 To: Fredrik Widlund Cc: gluster-devel@nongnu.org Subject: Re: [Gluster-devel] atomic operations fails_ On Wed, Feb 10, 2010 at 12:48 AM, Fredrik Widlund fredrik.widl...@qbrick.com wrote: Hi, I'll try to make myself clearer than in the earlier thread, since I need some help here. I'm not sure if I am missing something. I am not able to use atomic operations using glusterfs 3.0.2, on a Arch Linux 2.6.32.7-1 x86_64 server. I've stripped everything down to the most simple configuration Server: a posix store with locks Client: a tcp client I mount the glusterfs on /gfs, and use a writer and a reader as below: [writer]r...@frustration while :; do date /gfs/.file; mv /gfs/.file /gfs/file; done [reader] r...@frustration while :; do cat /gfs/file /dev/null; done Leaving this running, I get a flood of cat: /gfs/file: No such file or directory This should not happen on a filesystem that supports atomic operations?! Please try turning off FUSE entry caching by passing --entry-timeout=0 to your mount command on the client side and let us know what difference that makes. Thanks, Avati ___ Gluster-devel mailing list Gluster-devel@nongnu.org http://lists.nongnu.org/mailman/listinfo/gluster-devel ___ Gluster-devel mailing list Gluster-devel@nongnu.org http://lists.nongnu.org/mailman/listinfo/gluster-devel
Re: [Gluster-devel] atomic operations fails_
Fredrik Widlund wrote: Can you reproduce this behavior on your end? We've been able to reproduce this, and are looking into it. Vikas -- Engineer - Gluster, Inc. ___ Gluster-devel mailing list Gluster-devel@nongnu.org http://lists.nongnu.org/mailman/listinfo/gluster-devel
Re: [Gluster-devel] atomic operations fails_
Haha, I love how you named your server! I think a lot of us feel the same way :) On Tue, Feb 9, 2010 at 11:18 AM, Fredrik Widlund fredrik.widl...@qbrick.com wrote: Hi, I’ll try to make myself clearer than in the earlier thread, since I need some help here. I’m not sure if I am missing something. I am not able to use atomic operations using glusterfs 3.0.2, on a Arch Linux 2.6.32.7-1 x86_64 server. I’ve stripped everything down to the most simple configuration Server: a posix store with locks Client: a tcp client I mount the glusterfs on /gfs, and use a “writer” and a “reader” as below: [writer]r...@frustration while :; do date /gfs/.file; mv /gfs/.file /gfs/file; done [reader] r...@frustration while :; do cat /gfs/file /dev/null; done Leaving this running, I get a flood of cat: /gfs/file: No such file or directory This should not happen on a filesystem that supports atomic operations?! What this means is that I am unable to replace a file as an atomic operation, creating a race-condition where the file is sometimes not present. Furthermore this actually “desyncs” the server in a more complex setup, requiring a restart of the server part to make “desynced” files available again. This reproduces using XFS as well as EXT2 as the underlying fs. [glusterfs.vol] volume media0 type protocol/client option transport-type tcp option remote-host 127.0.0.1 option remote-subvolume media0 end-volume mounted with “glusterfs -f glusterfs.vol /gfs” [glusterfsd.vol] (se below debug log) -x- [2010-02-09 18:18:23] D [glusterfsd.c:424:_get_specfp] glusterfs: loading volume file glusterfsd.vol [2010-02-09 18:18:23] D [xlator.c:739:xlator_set_type] xlator: dlsym(notify) on /usr/lib/glusterfs/3.0.2/xlator/features/locks.so: undefined symbol: notify -- neglecting Version : glusterfs 3.0.2 built on Feb 8 2010 15:23:34 git: v3.0.2 Starting Time: 2010-02-09 18:18:23 Command line : glusterfs -f glusterfsd.vol --debug PID : 19486 System name : Linux Nodename : n100.p1.x0.local Kernel Release : 2.6.32-ARCH Hardware Identifier: x86_64 Given volfile: +--+ 1: volume media0-store 2: type storage/posix 3: option directory /mnt 4: end-volume 5: 6: volume media0 7: type features/locks 8: subvolumes media0-store 9: end-volume 10: 11: volume server 12: type protocol/server 13: option transport-type tcp 14: option auth.addr.media0.allow * 15: subvolumes media0 16: end-volume +--+ [2010-02-09 18:18:23] D [glusterfsd.c:1370:main] glusterfs: running in pid 19486 [2010-02-09 18:18:23] D [transport.c:145:transport_load] transport: attempt to load file /usr/lib/glusterfs/3.0.2/transport/socket.so [2010-02-09 18:18:23] N [glusterfsd.c:1396:main] glusterfs: Successfully started [2010-02-09 18:18:43] D [addr.c:190:gf_auth] media0: allowed = *, received addr = 127.0.0.1 [2010-02-09 18:18:43] N [server-protocol.c:5812:mop_setvolume] server: accepted client from 127.0.0.1:1023 [2010-02-09 18:18:43] D [addr.c:190:gf_auth] media0: allowed = *, received addr = 127.0.0.1 [2010-02-09 18:18:43] N [server-protocol.c:5812:mop_setvolume] server: accepted client from 127.0.0.1:1022 [2010-02-09 18:18:58] D [server-resolve.c:238:resolve_path_deep] media0: RESOLVE RENAME() seeking deep resolution of /file [2010-02-09 18:18:58] D [dict.c:303:dict_get] dict: @this=(nil) @key=0x7f2a38a9f79b [2010-02-09 18:18:58] D [dict.c:303:dict_get] dict: @this=(nil) @key=0x7f2a38a9f7b3 [2010-02-09 18:18:58] D [dict.c:303:dict_get] dict: @this=(nil) @key=0x7f2a38a9f7cb [2010-02-09 18:18:58] D [dict.c:303:dict_get] dict: @this=(nil) @key=0x7f2a38a9f79b [2010-02-09 18:18:58] D [dict.c:303:dict_get] dict: @this=(nil) @key=0x7f2a38a9f7b3 [2010-02-09 18:18:58] D [dict.c:303:dict_get] dict: @this=(nil) @key=0x7f2a38a9f7cb [2010-02-09 18:19:15] D [inode.c:916:inode_path] media0/inode: no dentry for non-root inode 502725 [2010-02-09 18:19:34] D [inode.c:916:inode_path] media0/inode: no dentry for non-root inode 502734 [2010-02-09 18:19:56] D [inode.c:916:inode_path] media0/inode: no dentry for non-root inode 502734 […] repeated many times. Frequency of the “no dentry” entries depends on how aggressive the reader is. ___ Gluster-devel mailing list Gluster-devel@nongnu.org http://lists.nongnu.org/mailman/listinfo/gluster-devel ___ Gluster-devel mailing list Gluster-devel@nongnu.org http://lists.nongnu.org/mailman/listinfo/gluster-devel
Re: [Gluster-devel] atomic operations fails_
On Wed, Feb 10, 2010 at 12:48 AM, Fredrik Widlund fredrik.widl...@qbrick.com wrote: Hi, I’ll try to make myself clearer than in the earlier thread, since I need some help here. I’m not sure if I am missing something. I am not able to use atomic operations using glusterfs 3.0.2, on a Arch Linux 2.6.32.7-1 x86_64 server. I’ve stripped everything down to the most simple configuration Server: a posix store with locks Client: a tcp client I mount the glusterfs on /gfs, and use a “writer” and a “reader” as below: [writer] r...@frustration while :; do date /gfs/.file; mv /gfs/.file /gfs/file; done [reader] r...@frustration while :; do cat /gfs/file /dev/null; done Leaving this running, I get a flood of cat: /gfs/file: No such file or directory This should not happen on a filesystem that supports atomic operations?! Please try turning off FUSE entry caching by passing --entry-timeout=0 to your mount command on the client side and let us know what difference that makes. Thanks, Avati ___ Gluster-devel mailing list Gluster-devel@nongnu.org http://lists.nongnu.org/mailman/listinfo/gluster-devel
