Hi Karol, ----- "Karol Mroz" <km...@suse.de> wrote:
> Hi Matt, > > Thanks for the input. Please see inline. > > On Sun, Nov 27, 2016 at 10:58:34PM -0500, Matt W. Benjamin wrote: > [...] > > > 1. `echo` or `touch` a new file > > > > > > File creation via `echo` or `touch` (createmode=1, offset=0, > length=N) > > > has > > > so far been successful in my test environment. > > > > > > > > > 2. Append to an existing file using `echo` > > > > > > On an append via `echo`, the behaviour is not consistent. > > > > > > i) offset=0, length=N > > > - A full overwrite via `echo` has shown to be successful. > > > > > > ii) offset=M, length=(N - M) > > > - The librgw:rgw_write() code path ultimately rejects this > write > > > and > > > (-5) bubbles back up to ganesha. This makes sense as RGW > > > objects need > > > to be completely overwritten. > > > > Right, that's the current expected behavior. > > > > > > > > In the case of (2ii), I've seen inconsistency between the S3 > object > > > and the > > > NFS representation. The file will have an updated size as if the > > > append > > > succeeded, however S3 shows that the object has not been modified. > > > > Trying to > > > then read the file from NFS would generate I/O errors. At this > point, > > > a restart > > > of ganesha was needed. > > > > Seems like a bug, sorry. > > > > > > > > Also, it isn't clear to me why nfs4_write() interprets one `echo` > > > invocation as > > > an overwrite, and another identical `echo` as an append? Given > the > > > rgw_write() > > > offset limitations, it seems file modification/appends are not > yet > > > well supported? > > > > For the moment, what you should be doing is mounting with -osync > (Linux) to get consistent results. Can you give that a try? > > Mounting with '-o sync' appears to remove (2ii). A failed, simple > write (echo) on the NFS side > does not trigger IO errors when reading the file. However, the write > failures still persist when > trying to write into the file at an offset, rather than overwriting > the file fully (log snippet > below). I may not be following you completely. Are you saying that there is a way to attempt -whole-file overwrite- which fails? Because that's the only supported write mode, currently. The mechanisms by which nfsv4 and nfs3 detect the end of the overwrite cycle differ (nfs3 lacks open), but both should work. I.e, you should be able to write sequentially from offset 0, but not otherwise. Matt > > What I see is that the first write (new file or append) via an echo > succeeds (offset=0, length=N). > The next write(s) will fail (offset=M, length=(N - M)). _However_, > re-reading the file (cat) and then > issuing an echo will succeed (offset=0, length=N). So re-reading the > file between writes appears to help. > > I'm wondering if it's a client side problem (passing inconsistent > offset and length values to nfs4_write())? > The fact that cat'ing the file seems to reset the offset and length to > an overwrite is interesting... > > Log of failed write: -- Matt Benjamin CohortFS, LLC. 315 West Huron Street, Suite 140A Ann Arbor, Michigan 48103 http://cohortfs.com tel. 734-761-4689 fax. 734-769-8938 cel. 734-216-5309 ------------------------------------------------------------------------------ _______________________________________________ Nfs-ganesha-devel mailing list Nfs-ganesha-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/nfs-ganesha-devel
