On 11/28/2016 11:37 PM, Karol Mroz wrote:
> Hi Matt,
>
> On Mon, Nov 28, 2016 at 10:40:42PM -0500, Matt W. Benjamin wrote:
> [...]
>>>> For the moment, what you should be doing is mounting with -osync
>>> (Linux) to get consistent results. Can you give that a try?
>>>
>>> Mounting with '-o sync' appears to remove (2ii). A failed, simple
>>> write (echo) on the NFS side
>>> does not trigger IO errors when reading the file. However, the write
>>> failures still persist when
>>> trying to write into the file at an offset, rather than overwriting
>>> the file fully (log snippet
>>> below).
>>
>> I may not be following you completely. Are you saying that there is a way
>> to attempt -whole-file overwrite- which fails? Because that's the only
>> supported write mode, currently. The mechanisms by which nfsv4 and nfs3
>> detect the end of the overwrite cycle differ (nfs3 lacks open), but both
>> should work. I.e, you should be able to write sequentially from offset 0,
>> but not otherwise.
>
> Understood. Let me try to clarify what I'm attempting.
>
> I have a file (/mnt/testbucket-13/foobar-osync-2) that represents an S3 object
> which currently contains:
>
> foobar
> foobar2
> ...
> foobar9
>
> I want to append "foobar10" to the file via an echo:
>
> # echo "foobar10" >> /mnt/testbucket-13/foobar-osync-2
>
> Currently I see one of two cases when tracing ganesha's write op:
>
> 1. The failure case
> -------------------
>
> (gdb) p (WRITE4args)op->nfs_argop4_u
> $9 = {stateid = {seqid = 0, other = "\001\000\000\000V\374<X\001\000\000"},
> offset = 80, stable = FILE_SYNC4,
> data = {data_len = 9, data_val = 0x7fffb4005980 "foobar10\n\v"}}
>
> 2. The success case
> -------------------
> (gdb) p (WRITE4args)op->nfs_argop4_u
> $10 = {stateid = {seqid = 0, other = "\001\000\000\000V\374<X\003\000\000"},
> offset = 0, stable = FILE_SYNC4,
> data = {data_len = 89,
> data_val = 0x7fffb4036f30
> "foobar\nfoobar2\nfoobar3\nfoobar4\nfoobar5\nfoobar6\nfoobar7\nfoobar8\nfoobar9\nfoobar10\nfoobar11\n"}}
>
>
> As you can see, the same append via echo ends up triggering nfs4_write() with
> different offset
> and data values.
>
> I'm trying to understand why _sometimes_ an append behaves as a full
> overwrite in ganesha,
> and other times as a seek+write.
>
> I'm hitting the failure case more often with full debugs enabled on ganesha.
>
> Given what you said above, perhaps this type of echo append to an existing
> file is currently
> a no-no for the RGW FSAL?
>
Naively, I would expect "cat >> foo" to fail for every case except when
"foo" is empty. I can see how, if the file is not in client cache, the
client may read it before appending, but that is not necessary; it can
merely open O_APPEND and write, which is not supported.
This, to me, is clearly a client issue. Sometimes, it reads and
re-writes the entire file (which seems odd to me), and sometimes it's
just appending.
I wonder if it's related to block size in the client cache? If the file
you're appending to is larger than the block size (say, append to a 10M
file), it may always fail.
Daniel
------------------------------------------------------------------------------
_______________________________________________
Nfs-ganesha-devel mailing list
Nfs-ganesha-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/nfs-ganesha-devel
