Hi Matt, Thanks for the input. Please see inline.
On Sun, Nov 27, 2016 at 10:58:34PM -0500, Matt W. Benjamin wrote:
[...]
> > 1. `echo` or `touch` a new file
> >
> > File creation via `echo` or `touch` (createmode=1, offset=0, length=N)
> > has
> > so far been successful in my test environment.
> >
> >
> > 2. Append to an existing file using `echo`
> >
> > On an append via `echo`, the behaviour is not consistent.
> >
> > i) offset=0, length=N
> > - A full overwrite via `echo` has shown to be successful.
> >
> > ii) offset=M, length=(N - M)
> > - The librgw:rgw_write() code path ultimately rejects this write
> > and
> > (-5) bubbles back up to ganesha. This makes sense as RGW
> > objects need
> > to be completely overwritten.
>
> Right, that's the current expected behavior.
>
> >
> > In the case of (2ii), I've seen inconsistency between the S3 object
> > and the
> > NFS representation. The file will have an updated size as if the
> > append
> > succeeded, however S3 shows that the object has not been modified.
> > Trying to
> > then read the file from NFS would generate I/O errors. At this point,
> > a restart
> > of ganesha was needed.
>
> Seems like a bug, sorry.
>
> >
> > Also, it isn't clear to me why nfs4_write() interprets one `echo`
> > invocation as
> > an overwrite, and another identical `echo` as an append? Given the
> > rgw_write()
> > offset limitations, it seems file modification/appends are not yet
> > well supported?
>
> For the moment, what you should be doing is mounting with -osync (Linux) to
> get consistent results. Can you give that a try?
Mounting with '-o sync' appears to remove (2ii). A failed, simple write (echo)
on the NFS side
does not trigger IO errors when reading the file. However, the write failures
still persist when
trying to write into the file at an offset, rather than overwriting the file
fully (log snippet
below).
What I see is that the first write (new file or append) via an echo succeeds
(offset=0, length=N).
The next write(s) will fail (offset=M, length=(N - M)). _However_, re-reading
the file (cat) and then
issuing an echo will succeed (offset=0, length=N). So re-reading the file
between writes appears to help.
I'm wondering if it's a client side problem (passing inconsistent offset and
length values to nfs4_write())?
The fact that cat'ing the file seems to reset the offset and length to an
overwrite is interesting...
Log of failed write:
...
dec_state_owner_ref :STATE :F_DBG :Decrement refcount now=1
{STATE_OPEN_OWNER_NFSV4 0x7f12b0011450: clientid={0x7f12a40010c0
ClientID={Epoch=0x583cad7b Counter=0x00000001} CONFIRMED Client={0x7f12a40035e0
name=(17:Linux NFSv4.2 elm) refcount=1} t_delta=0 reservations=1 refcount=3}
owner=(24:0x6f70656e2069643a000000290000000000069dd33f49b0f6) confirmed=0
seqid=0 refcount=2}
gsh_dbus_thread :RW LOCK :F_DBG :Released mutex 0x7ce0a0 (&dbus_bcast_lock) at
/home/abuild/rpmbuild/BUILD/nfs-ganesha-2.4.1+git.1480370806.0f51bdb/dbus/dbus_server.c:733
nfs4_write :NFS4 :F_DBG :offset = 71 length = 9 stable = 2
ganesha.nfsd-19055[work-45] nfs4_write :NFS4 :F_DBG :offset = 71 length = 9
rgw_fsal_write2 :FSAL :F_DBG :rgw_fsal_write2 enter obj_hdl 0x7f12bc007560
state 0x7f12b000ebb0
gsh_dbus_thread :DBUS :F_DBG :top of poll loop
gsh_dbus_thread :RW LOCK :F_DBG :Acquired mutex 0x7ce0a0 (&dbus_bcast_lock) at
/home/abuild/rpmbuild/BUILD/nfs-ganesha-2.4.1+git.1480370806.0f51bdb/dbus/dbus_server.c:683
rgw_fsal_write2 :FSAL :F_DBG :rgw_fsal_write2 post obj_hdl 0x7f12bc007560 state
0x7f12b000ebb0 returned -5
gsh_dbus_thread :RW LOCK :F_DBG :Released mutex 0x7ce0a0 (&dbus_bcast_lock) at
/home/abuild/rpmbuild/BUILD/nfs-ganesha-2.4.1+git.1480370806.0f51bdb/dbus/dbus_server.c:733
fsal_write2 :FSAL :F_DBG :FSAL WRITE operation returned I/O error,
asked_size=9, effective_size=0
nfs4_write :NFS4 :DEBUG :write returned I/O error
...
--
Regards,
Karol
signature.asc
Description: Digital signature
------------------------------------------------------------------------------
_______________________________________________ Nfs-ganesha-devel mailing list Nfs-ganesha-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/nfs-ganesha-devel
