Yes, I just used a text editor i.e. vi to test this out, but my real
application is surveillance camera system. The video was stored in GlusterFS by
FUSE and Samba.
Because of the surveillance camera system 7*24 working mode, it doesn't
have to cache the files. So I added O_DIRECT flag in GlusterFS client protocol
open() and create(). According my test, I wrote a video file into GlusterFS in
windows, then I didn't find any cache in GlusterFS node, and I read a video
file in windows, it also wasn't cached in GlusterFS node, but some frames in
the video file(at the last) was lost, just like my vi test.
IIRC the page-alighed may be the reason of this question. How to solve this
problem? Or in GlusterFS 3.6, 3.7, is there the option to enable io-direct mode
to avoid the video cache in GlusterFS node?
At 2016-07-11 18:51:02, "Krutika Dhananjay" <kdhan...@redhat.com> wrote:
What's the application you are running? Sounds like you're using a text editor
like vi(m) to test this out?
Is the application opening the files with O_DIRECT?
Do you have the strace output of the running application that confirms that it
is open()ing the file with O_DIRECT?
Also, what are the offsets and sizes of the writes on this file by this
application in the strace output?
-Krutika
On Mon, Jul 11, 2016 at 2:44 PM, Keiviw <kei...@163.com> wrote:
I have checked the page-aligned, i.e. the file was larger than one page, a part
of the file(one page size) was saved successfully, and the rest(more than one
page but less than two pages) was lost.
At 2016-07-11 12:53:32, "Pranith Kumar Karampuri" <pkara...@redhat.com> wrote:
Is it possible to share the test you are running? As per your volume, o-direct
is not enabled on your volume, i.e. the file shouldn't be opened with o-direct
but as per the logs it is giving Invalid Argument as if there is something
wrong with the arguments when we do o-direct write with wrong size. so I would
like to test out why exactly is it giving this problem. Please note that for
o-direct write to succeed, both offset and size should be page-aligned,
something like multiple of 512 is one way to check it.
On Sun, Jul 10, 2016 at 5:19 PM, Keiviw <kei...@163.com> wrote:
My volume info:
Volume Name: test
Type: Distribute
Volume ID: 9294b122-d81e-4b12-9b5c-46e89ee0e40b
Status: Started
Number of Bricks: 2
Transport-type: tcp
Bricks:
Brick1: compute2:/home/brick1
Brick2: compute2:/home/brick2
Options Reconfigured:
performance.flush-behind: off
storage.linux-aio: off
My brick logs(I have cleaned up the history log):
[2016-07-10 11:42:50.577683] E [posix.c:2128:posix_writev] 0-test-posix:
write failed: offset 0, Invalid argument
[2016-07-10 11:42:50.577735] I [server3_1-fops.c:1414:server_writev_cbk]
0-test-server: 8569840: WRITEV 5 (526a3118-9994-429e-afc0-4aa063606bde) ==> -1
(Invalid argument)
[2016-07-10 11:42:54.583038] E [posix.c:2128:posix_writev] 0-test-posix:
write failed: offset 0, Invalid argument
[2016-07-10 11:42:54.583080] I [server3_1-fops.c:1414:server_writev_cbk]
0-test-server: 8569870: WRITEV 5 (c3d28f34-8f43-446d-8d0b-80841ae8ec5b) ==> -1
(Invalid argument)
My mnt-test-.logs:
[2016-07-10 11:42:50.577816] W [client3_1-fops.c:876:client3_1_writev_cbk]
0-test-client-1: remote operation failed: Invalid argument
[2016-07-10 11:42:50.578508] W [fuse-bridge.c:968:fuse_err_cbk]
0-glusterfs-fuse: 12398282: FSYNC() ERR => -1 (Invalid argument)
[2016-07-10 11:42:54.583156] W [client3_1-fops.c:876:client3_1_writev_cbk]
0-test-client-1: remote operation failed: Invalid argument
[2016-07-10 11:42:54.583762] W [fuse-bridge.c:968:fuse_err_cbk]
0-glusterfs-fuse: 12398317: FSYNC() ERR => -1 (Invalid argument)
在 2016-07-10 19:18:18，"Krutika Dhananjay" <kdhan...@redhat.com> 写道：
To me it looks like a case of a flush triggering a write() that was cached by
write-behind and because the write buffer
did not meet the page alignment requirement with o-direct write, it was failed
with EINVAL and the trigger fop - i.e., flush() was failed with the 'Invalid
argument' error code.
Could you attach the brick logs as well, so that we can confirm the theory?
-Krutika
On Sat, Jul 9, 2016 at 9:31 PM, Atin Mukherjee <amukh...@redhat.com> wrote:
Pranith/Krutika,
Your inputs please, IIRC we'd need to turn on some o_direct option here?
On Saturday 9 July 2016, Keiviw <kei...@163.com> wrote:
The errors also occured in GlusterFS 3.6.7，I just add the O_DIRECT flag in
client protocol open() and create()! How to explain and solve the problem?
发自 网易邮箱大师
On 07/09/2016 17:58, Atin Mukherjee wrote:
Any specific reason of using 3.3 given that its really quite old? We are at
3.6, 3.7 & 3.8 supportability matrix now.
On Saturday 9 July 2016, Keiviw <kei...@163.com> wrote:
hi,
I have installed GlusterFS 3.3.0, and now I get Fsync failures when saving
files with the O_DIRECT flag in open() and create().
1, I tried to save a flie in vi and got this error:
"test" E667: Fsync failed
2, I see this in the logs:
[2016-07-07 14:20:10.325400] W [fuse-bridge.c:968:fuse_err_cbk]
0-glusterfs-fuse: 102: FSYNC() ERR => -1 (Invalid argument)
[2016-07-07 14:20:13.930384] W [fuse-bridge.c:968:fuse_err_cbk]
0-glusterfs-fuse: 137: FSYNC() ERR => -1 (Invalid argument)
[2016-07-07 14:20:51.199448] W [fuse-bridge.c:968:fuse_err_cbk]
0-glusterfs-fuse: 174: FLUSH() ERR => -1 (Invalid argument)
[2016-07-07 14:21:32.804738] W [fuse-bridge.c:968:fuse_err_cbk]
0-glusterfs-fuse: 206: FLUSH() ERR => -1 (Invalid argument)
[2016-07-07 14:21:43.702146] W [fuse-bridge.c:968:fuse_err_cbk]
0-glusterfs-fuse: 276: FSYNC() ERR => -1 (Invalid argument)
[2016-07-07 14:21:51.296809] W [fuse-bridge.c:968:fuse_err_cbk]
0-glusterfs-fuse: 314: FSYNC() ERR => -1 (Invalid argument)
[2016-07-07 14:21:54.062687] W [fuse-bridge.c:968:fuse_err_cbk]
0-glusterfs-fuse: 349: FSYNC() ERR => -1 (Invalid argument)
[2016-07-07 14:22:54.678960] W [fuse-bridge.c:968:fuse_err_cbk]
0-glusterfs-fuse: 429: FSYNC() ERR => -1 (Invalid argument)
[2016-07-07 14:24:35.546980] W [fuse-bridge.c:968:fuse_err_cbk]
0-glusterfs-fuse: 505: FSYNC() ERR => -1 (Invalid argument)
[2016-07-07 14:24:48.696888] W [fuse-bridge.c:968:fuse_err_cbk]
0-glusterfs-fuse: 538: FSYNC() ERR => -1 (Invalid argument)
[2016-07-07 14:24:55.824752] W [fuse-bridge.c:968:fuse_err_cbk]
0-glusterfs-fuse: 575: FSYNC() ERR => -1 (Invalid argument)
[2016-07-07 14:25:04.364629] W [fuse-bridge.c:968:fuse_err_cbk]
0-glusterfs-fuse: 612: FSYNC() ERR => -1 (Invalid argument)
[2016-07-07 14:27:08.579052] W [fuse-bridge.c:968:fuse_err_cbk]
0-glusterfs-fuse: 713: FSYNC() ERR => -1 (Invalid argument)
[2016-07-07 14:46:44.635169] W [fuse-bridge.c:968:fuse_err_cbk]
0-glusterfs-fuse: 859: FLUSH() ERR => -1 (Invalid argument)
[2016-07-07 14:46:55.892041] W [fuse-bridge.c:968:fuse_err_cbk]
0-glusterfs-fuse: 989: FLUSH() ERR => -1 (Invalid argument)
[2016-07-07 14:51:45.904233] W [fuse-bridge.c:968:fuse_err_cbk]
0-glusterfs-fuse: 148747: FLUSH() ERR => -1 (Invalid argument)
[2016-07-07 14:54:45.603557] W [fuse-bridge.c:968:fuse_err_cbk]
0-glusterfs-fuse: 148986: FLUSH() ERR => -1 (Invalid argument)
[2016-07-07 14:59:18.818972] W [fuse-bridge.c:968:fuse_err_cbk]
0-glusterfs-fuse: 292106: FLUSH() ERR => -1 (Invalid argument)
[2016-07-07 15:48:32.708207] W [fuse-bridge.c:968:fuse_err_cbk]
0-glusterfs-fuse: 6753592: FLUSH() ERR => -1 (Invalid argument)
[2016-07-07 16:24:10.476228] W [fuse-bridge.c:968:fuse_err_cbk]
0-glusterfs-fuse: 10733703: FLUSH() ERR => -1 (Invalid argument)
[2016-07-07 16:30:42.388220] W [fuse-bridge.c:968:fuse_err_cbk]
0-glusterfs-fuse: 11497698: FLUSH() ERR => -1 (Invalid argument)
[2016-07-07 16:44:18.247511] W [fuse-bridge.c:968:fuse_err_cbk]
0-glusterfs-fuse: 12037185: FLUSH() ERR => -1 (Invalid argument)
[2016-07-08 01:00:55.131141] W [fuse-bridge.c:968:fuse_err_cbk]
0-glusterfs-fuse: 12181783: FLUSH() ERR => -1 (Invalid argument)
[2016-07-08 06:54:13.418607] W [fuse-bridge.c:968:fuse_err_cbk]
0-glusterfs-fuse: 12397060: FLUSH() ERR => -1 (Invalid argument)
--
Atin
Sent from iPhone
--
Atin
Sent from iPhone
--
Pranith
_______________________________________________
Gluster-devel mailing list
Gluster-devel@gluster.org
http://www.gluster.org/mailman/listinfo/gluster-devel
