Re: Occassional permission denied in the middle of a large transfer over NFS

2012-07-10 Thread Andrey Simonenko 
This message contains only corrections to typos from my previous message.

On Mon, Jul 09, 2012 at 12:44:56PM +0300, Andrey Simonenko wrote:
 
 Here -alldirs means that /cdrom should be a file system.  As I remember
 this even worked before revision 1.85 of mountd/mountd.c, then mountd.c
 began to use nmount(2).  Now if one puts /cdrom in /etc/exports and
   ^^\
  /cdrom -alldirs

 Let's consider the same example but without the -alldirs options:
 
 /usr/local/export -maproot=root 1.2.3.4
 
 3. nfse in the native mode will export /usr if it is or will be a mount point:
 \
  /usr/local/export
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org

Re: Occassional permission denied in the middle of a large transfer over NFS

2012-07-10 Thread Vincent Hoffman 
On 09/07/2012 02:08, Rick Macklem wrote:
 Vincent Hoffman:
 On 08/07/2012 00:26, Rick Macklem wrote:
 Vincent Hoffman wrote:
 Hi Rick,

 I'm afraid this didnt make any real difference for me.
 Since I couldnt test it on the live system I tried it on a test vm.
 on the vm (nfs server) I set a looping mount/umount
 while true ; do mount /dev/md0 /mnt/tmp ; sleep 1 ; umount /mnt/tmp
 ;
 done

 and on the client I set a loop of tars of large directorys to the
 nfs
 mount running under time to see how well it survived. Then
 replicated
 the test with the patch and without.

 Just to confirm, you patched both the kernel and mountd and replaced
 both
 on the server?

 Also, I'm not sure how ZFS handles it's exports. I can't remember if
 you've
 tried an exported UFS volume. It might be something ZFS specific?

 rick
 Hi Rick,

 yes I patched both the kernel and mountd, rebuilt kernel and world (to
 be sure), added the -S flag to mountd in rc.conf and rebooted.
 This is a test VM running -CURRENT and is only exporting a ufs2
 filesystem.
 (11:43:05 ~) 0 $ cat /etc/exports
 /usr/local/export -maproot=root -alldirs XX.XX.XX.XX

 Client is a 8.3-RELEASE box but I see the same with linux clients.
 (I can confirm that it works fine when I am not running the
 mount/umount
 loop)

 Oops, the patch I sent you worked for NFSv4 only. If you also apply the
 attached patch, it seems to work for NFSv3 as well.
 The patch is also at:
   http://people.freebsd.org/~rmacklem/atomic-export2.patch

 You must also run the new/experimental server. (I can't remember if I
 mentioned that before.)

 rick

Hi Rick,
Just wanted to report that this fixed the permission
denied error for my (not terribly exhaustive) testing.
I could transfer all 10G of data by tar to the nfs mount without issue.
I'll try running that a couple more times to be sure but given it was
bombing out in seconds before i'm happy :)

Vince

___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org

Re: Occassional permission denied in the middle of a large transfer over NFS

2012-07-09 Thread Andrey Simonenko 
Hello again,

On Sun, Jul 08, 2012 at 06:35:50PM +0100, Vincent Hoffman wrote:
 Replying to myself just as a record, I have tried nfse and I didnt get
 the permission denied at all.
 The only issue I had with it is that it strictly adheres to the syntax
 in exports(5) while mountd is a little more flexible.
 
 I had
 /usr/local/export -alldirs -maproot=root 85.xx.xx.xx
 
 /usr is the root of that filesystem
 
 mountd - allowed this but actually silently exports /usr (and all dirs
 below)
 
 nfse - didnt allow this, it needed to be the correct
 /usr  -alldirs -maproot=root 85.xx.xx.xx
 
 which is I guess a POLA violation but follows the documentation.
 this was using nfse in mountd compatibility mode. I havent played with
 its more advanced features.

The -alldirs option in the exports(5) file does not mean only
all directories.

Please read these lines from exports(5) DESCRIPTION:

 The second is to specify the pathname of the root of the file system fol-
 lowed by the -alldirs flag; this form allows the host(s) to mount at any
 point within the file system, including regular files if the -r option is
 used on mountd(8).

The -alldirs option specifies that this pathname must be the root (mount
point) of some file system.

And read these lines from EXAMPLES:

 The file system rooted at /cdrom will be exported read-only to the entire
 network 192.168.33.0/24, including all its subdirectories.  Since /cdrom
 is the conventional mountpoint for a CD-ROM device, this export will fail
 if no CD-ROM medium is currently mounted there since that line would then
 attempt to export a subdirectory of the root file system with the
 -alldirs option which is not allowed.  The -quiet option will then sup-
 press the error message for this condition that would normally be sys-
 logged.  As soon as an actual CD-ROM is going to be mounted, mount(8)
 will notify mountd(8) about this situation, and the /cdrom file system
 will be exported as intended.  Note that without using the -alldirs
 option, the export would always succeed.  While there is no CD-ROM medium
 mounted under /cdrom, it would export the (normally empty) directory
 /cdrom of the root file system instead.

Here -alldirs means that /cdrom should be a file system.  As I remember
this even worked before revision 1.85 of mountd/mountd.c, then mountd.c
began to use nmount(2).  Now if one puts /cdrom in /etc/exports and
there is no file system mounted on /cdrom, then the entire / will be
exported.

Just create /etc/exports with one line /cdrom -alldirs and try to
mount server:/ on another system, you will get access to the / of
the server.

The current version of mountd is not little more flexible, it does
not follow traditional -alldirs option's logic and does not follow
description of the -alldirs option in the exports(5) manual page, I guess
this is a POLA violation.

Now let's back to your example where /usr is the root of the file system:

/usr/local/export -alldirs -maproot=root 1.2.3.4

1. mountd exports /usr and all subdirectories under it for NFSv2/3 clients.
   Actually it has to export /usr/local/export only and all subdirectories
   under it only if /usr/local/export is or will be the root of some
   file system.

2. nfse in the compatibility mode sees that there is the -alldirs option
   and will export /usr/local/export and all subdirectories under it
   only if /usr/local/exports is or will be the root of some file system.

If one runs nfse -Ct ... for this file then its output will be:

configure: reading file /etc/exports
configure: converting configuration to native format

Pathname /usr/local/export
Export specifications:
-rw -sec sys -maproot 0:0:5 -host 1.2.3.4
Subdirectories for NFSv2/3:
/usr/local/export
-alldirs -host 1.2.3.4

Warning: subdirectories exports are insecure

This is output in nfse native format.  Using NFSE part in the kernel,
nfse verifies whether /usr/local/export is a mount point, whether file
system supports NFS, and only if everything is correct, it starts to
export it.  When file system is unmounted or when another file system
is mounted, then NFSE part in the kernel is informed by EVENTHANDLERs
vfs_mount_event/vfs_unmount_event and NFS server verifies whether it
still can export this file system (it can be unmounted or it can be
shadowed).  The userland utility nfse got information of VFS events
via kevents VQ_MOUNT/VQ_UNMOUNT and only the verifies using NFSE
part in the kernel whether some file system is still exported or can
become exported.

There is one implementation mistake when nfse -C ... is used.  The
nfse utility follows implementation mistake from mountd and verifies
what is a file system and what is not a file system using the f_mntonname
field in the struct mount{}.  This is wrong since parts of mount point
directory can be renamed and this is not reflected in the f_mntonname
value.  That's why I do not

Re: Occassional permission denied in the middle of a large transfer over NFS

2012-07-09 Thread Andrey Simonenko 
On Sun, Jul 08, 2012 at 07:48:11PM -0400, Rick Macklem wrote:

  Replying to myself just as a record, I have tried nfse and I didnt get
  the permission denied at all.
  The only issue I had with it is that it strictly adheres to the syntax
  in exports(5) while mountd is a little more flexible.
  
  I had
  /usr/local/export -alldirs -maproot=root 85.xx.xx.xx
  
  /usr is the root of that filesystem
  
  mountd - allowed this but actually silently exports /usr (and all dirs
  below)
  
 Not exactly correct. mountd exports the entire file system in the kernel
 for the NFS server, since exports can only be attached to the mount points
 in the kernel. However, when the client's mount protocol requests a mount
 file handle for anything other than /usr/local/export, it will refuse that.
 (Which means that to mount anything other than /usr/local/export, the client
  must maliciously guess the file handle for mounting.)
 
 Put another way, a non-malicious NFSv3 client will only be able to mount
 /usr/local/export. Robert Watson calls this an administrative control and
 feels that it is necessary.

According to the exports(5) manual page and this example (/usr is the mount
point and the -alldir option is given), this example means the following:
export /usr/local/export only if it is or will be a mount point and
administratively export all subdirectories under it for NFSv2/3 clients.
Good description of the -alldirs option is given in the EXAMPLES section
from exports(5) in paragraph about /cdrom -alldirs.

___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org

Re: Occassional permission denied in the middle of a large transfer over NFS

2012-07-09 Thread Vincent Hoffman 
On 09/07/2012 02:08, Rick Macklem wrote:
 Vincent Hoffman:
 On 08/07/2012 00:26, Rick Macklem wrote:
 Vincent Hoffman wrote:
 Hi Rick,

 I'm afraid this didnt make any real difference for me.
 Since I couldnt test it on the live system I tried it on a test vm.
 on the vm (nfs server) I set a looping mount/umount
 while true ; do mount /dev/md0 /mnt/tmp ; sleep 1 ; umount /mnt/tmp
 ;
 done

 and on the client I set a loop of tars of large directorys to the
 nfs
 mount running under time to see how well it survived. Then
 replicated
 the test with the patch and without.

 Just to confirm, you patched both the kernel and mountd and replaced
 both
 on the server?

 Also, I'm not sure how ZFS handles it's exports. I can't remember if
 you've
 tried an exported UFS volume. It might be something ZFS specific?

 rick
 Hi Rick,

 yes I patched both the kernel and mountd, rebuilt kernel and world (to
 be sure), added the -S flag to mountd in rc.conf and rebooted.
 This is a test VM running -CURRENT and is only exporting a ufs2
 filesystem.
 (11:43:05 ~) 0 $ cat /etc/exports
 /usr/local/export -maproot=root -alldirs XX.XX.XX.XX

 Client is a 8.3-RELEASE box but I see the same with linux clients.
 (I can confirm that it works fine when I am not running the
 mount/umount
 loop)

 Oops, the patch I sent you worked for NFSv4 only. If you also apply the
 attached patch, it seems to work for NFSv3 as well.
 The patch is also at:
   http://people.freebsd.org/~rmacklem/atomic-export2.patch

 You must also run the new/experimental server. (I can't remember if I
 mentioned that before.)

 rick

You did mention that :) Thanks I'll give this one a go.

Vince

 The production system has been fine since I removed the SIGHUP call in
 mount.c so thanks for that suggestion.


 Vince
 [root@seaurchin ~]# ministat nopatch.txt atomicpatch.txt
 x nopatch.txt
 + atomicpatch.txt
 +--+
 |
 *
 |
 |
 *
 |
 |
 x*
 |
 |   xx*
 x
 |
 |  +x**
 xx
 |
 |   xxx
 x
 |
 |   xxx +x+
 +
 |
 |  +*xx +x+ x
 +
 |
 |  +*xx + +
 x |
 |  *+*xx+ +++x * x
 x |
 |  x**++*x+***x+ x*+ x ++*+ + x+ +x +
 + +|
 |||___M_M_A__A___|__|
 |
 +--+
 N Min Max Median Avg Stddev
 x 101 1.25 106.8 14.08 21.892178 22.196005
 + 101 1.21 186.93 18.46 27.995842 30.523218
 No difference proven at 95.0% confidence


 (excuse wrapped ascii art)

 I think I'll have a look at the nfse patch set and see how that
 performs.

 Thanks for all your work on NFS on FreeBSD.

 Vince

 Also, you could easily hack mount.c so that it doesn't send a
 SIGHUP
 to mountd (which causes the exports to be reloaded) every time a
 local
 fs is mounted.
 True and I may have to do that for the production NAS for the
 time
 being.
 Thanks for looking at this.

 Vince
 rick

 thanks, Vince
 ___
 freebsd-current@freebsd.org mailing list
 http://lists.freebsd.org/mailman/listinfo/freebsd-current
 To unsubscribe, send any mail to
 freebsd-current-unsubscr...@freebsd.org

 ___
 freebsd-current@freebsd.org mailing list
 http://lists.freebsd.org/mailman/listinfo/freebsd-current
 To unsubscribe, send any mail to
 freebsd-current-unsubscr...@freebsd.org


___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org

Re: Occassional permission denied in the middle of a large transfer over NFS

2012-07-08 Thread Vincent Hoffman 
On 08/07/2012 00:26, Rick Macklem wrote:
 Vincent Hoffman wrote:

 Hi Rick,

 I'm afraid this didnt make any real difference for me.
 Since I couldnt test it on the live system I tried it on a test vm.
 on the vm (nfs server) I set a looping mount/umount
 while true ; do mount /dev/md0 /mnt/tmp ; sleep 1 ; umount /mnt/tmp ;
 done

 and on the client I set a loop of tars of large directorys to the nfs
 mount running under time to see how well it survived. Then replicated
 the test with the patch and without.

 Just to confirm, you patched both the kernel and mountd and replaced both
 on the server?

 Also, I'm not sure how ZFS handles it's exports. I can't remember if you've
 tried an exported UFS volume. It might be something ZFS specific?

 rick

Hi Rick,
  
yes I patched both the kernel and mountd, rebuilt kernel and world (to
be sure), added the -S flag to mountd in rc.conf and rebooted.
This is a test VM running -CURRENT and is only exporting  a ufs2
filesystem.
(11:43:05 ~) 0 $ cat /etc/exports
/usr/local/export -maproot=root -alldirs  XX.XX.XX.XX

Client is a 8.3-RELEASE box but I see the same with linux clients.
(I can confirm that it works fine when I am not running the mount/umount
loop)


The production system has been fine since I removed the SIGHUP call in
mount.c so thanks for that suggestion.


Vince

 [root@seaurchin ~]# ministat nopatch.txt atomicpatch.txt
 x nopatch.txt
 + atomicpatch.txt
 +--+
 |
 *
 |
 |
 *
 |
 |
 x*
 |
 |   xx*
 x
 |
 |  +x**
 xx
 |
 |   xxx
 x
 |
 |   xxx +x+
 +
 |
 |  +*xx +x+ x
 +
 |
 |  +*xx + +
 x |
 |  *+*xx+ +++x * x
 x |
 |  x**++*x+***x+ x*+ x ++*+ + x+ +x +
 + +|
 |||___M_M_A__A___|__|
 |
 +--+
 N Min Max Median Avg Stddev
 x 101 1.25 106.8 14.08 21.892178 22.196005
 + 101 1.21 186.93 18.46 27.995842 30.523218
 No difference proven at 95.0% confidence


 (excuse wrapped ascii art)

 I think I'll have a look at the nfse patch set and see how that
 performs.

 Thanks for all your work on NFS on FreeBSD.

 Vince

 Also, you could easily hack mount.c so that it doesn't send a
 SIGHUP
 to mountd (which causes the exports to be reloaded) every time a
 local
 fs is mounted.
 True and I may have to do that for the production NAS for the time
 being.
 Thanks for looking at this.

 Vince
 rick

 thanks, Vince

 ___
 freebsd-current@freebsd.org mailing list
 http://lists.freebsd.org/mailman/listinfo/freebsd-current
 To unsubscribe, send any mail to
 freebsd-current-unsubscr...@freebsd.org


___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org

Re: Occassional permission denied in the middle of a large transfer over NFS

2012-07-08 Thread Vincent Hoffman 
On 07/07/2012 13:26, Vincent Hoffman wrote:
 On 01/07/2012 12:18, Rick Macklem wrote:
 Vincent Hoffman wrote:
 On 01/07/2012 01:53, Rick Macklem wrote:
 To modify mountd to use the kernel changes is more work than I have
 time for, in part because mountd.c is a very ugly old piece of C
 code, imho.

 I do have a patch that suspends/resumes execution of the nfsd
 threads
 (new, experimental for FreeBSD8.n only) when mountd reloads
 /etc/exports.
 This approach has certain disadvantages vs Andrey's transactional
 load/commit
 model, but it is simple and might be sufficient for your problem.
 If you want to try this patch, it is at:
http://people.freebsd.org/~rmacklem/atomic-export.patch
 (The patch affects both the kernel and mountd.c.)
 Happy to try it, to be certain I have understood, this is for the
 newnfs
 server (experimental in 8.x default for 8
 9+)
 Yes, that is correct. For the old (default on 8.x) it will have no effect.

 I'll apply to my -CUREENT test VM for now. Will fire up a 8.3 vm when
 i
 get a minute.
 Also, to enable it, you must add a -S flag to mountd_flags, after you've
 replaced the mountd executable.

 The patch is pretty straightforward, but has not seen much testing.
 Hopefully, it might be useful for you, rick
 Hi Rick,

 I'm afraid this didnt make any real difference for me.
 Since I couldnt test it on the live system I tried it on a test vm.
 on the vm (nfs server) I set a looping mount/umount
 while true ; do mount /dev/md0 /mnt/tmp ; sleep 1 ; umount /mnt/tmp ;  done

 and on the client I set a loop of tars of large directorys to the nfs
 mount running under time to see how well it survived. Then replicated
 the test with the patch and without.


 [root@seaurchin ~]# ministat nopatch.txt atomicpatch.txt
 x nopatch.txt
 + atomicpatch.txt
 +--+
 |
 * 
   
 |
 |
 * 
   
 |
 |   
 x*

 |
 |   xx* 
 x 

 |
 |  +x** 
 xx

 |
 |   xxx 
 x 

 |
 |   xxx +x+  
 +   
 |
 |  +*xx +x+   x  
 +   
 |
 |  +*xx + +
 x  |
 |  *+*xx+ +++x *  x
 x  |
 |  x**++*x+***x+ x*+ x  ++*+  + x+  +x +  
 +  +|
 |||___M_M_A__A___|__| 
 
 |
 +--+
 N   Min   MaxMedian   AvgStddev
 x 101  1.25 106.8 14.08 21.892178 22.196005
 + 101  1.21186.93 18.46 27.995842 30.523218
 No difference proven at 95.0% confidence


 (excuse wrapped ascii art)

 I think I'll have a look at the nfse patch set and see how that performs.
Replying to myself just as a record, I have tried nfse and I didnt get
the permission denied at all.
The only issue I had with it is that it strictly adheres to the syntax
in exports(5) while mountd is a little more flexible.

I had
/usr/local/export -alldirs -maproot=root 85.xx.xx.xx

/usr is the root of that filesystem

mountd - allowed this but actually silently exports /usr (and all dirs
below)

nfse - didnt allow this, it needed to be the correct
/usr  -alldirs -maproot=root 85.xx.xx.xx

which is I guess a POLA violation but follows the documentation.
this was using nfse in mountd compatibility mode. I havent played with
its more advanced features.

Vince
 Thanks for all your work on NFS on FreeBSD.

 Vince



___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org

Re: Occassional permission denied in the middle of a large transfer over NFS

2012-07-08 Thread Rick Macklem 
Vincent Hoffman wrote:
 On 07/07/2012 13:26, Vincent Hoffman wrote:
  On 01/07/2012 12:18, Rick Macklem wrote:
  Vincent Hoffman wrote:
  On 01/07/2012 01:53, Rick Macklem wrote:
  To modify mountd to use the kernel changes is more work than I
  have
  time for, in part because mountd.c is a very ugly old piece of C
  code, imho.
 
  I do have a patch that suspends/resumes execution of the nfsd
  threads
  (new, experimental for FreeBSD8.n only) when mountd reloads
  /etc/exports.
  This approach has certain disadvantages vs Andrey's transactional
  load/commit
  model, but it is simple and might be sufficient for your problem.
  If you want to try this patch, it is at:
 http://people.freebsd.org/~rmacklem/atomic-export.patch
  (The patch affects both the kernel and mountd.c.)
  Happy to try it, to be certain I have understood, this is for the
  newnfs
  server (experimental in 8.x default for 8
  9+)
  Yes, that is correct. For the old (default on 8.x) it will have no
  effect.
 
  I'll apply to my -CUREENT test VM for now. Will fire up a 8.3 vm
  when
  i
  get a minute.
  Also, to enable it, you must add a -S flag to mountd_flags, after
  you've
  replaced the mountd executable.
 
  The patch is pretty straightforward, but has not seen much testing.
  Hopefully, it might be useful for you, rick
  Hi Rick,
 
  I'm afraid this didnt make any real difference for me.
  Since I couldnt test it on the live system I tried it on a test vm.
  on the vm (nfs server) I set a looping mount/umount
  while true ; do mount /dev/md0 /mnt/tmp ; sleep 1 ; umount /mnt/tmp
  ; done
 
  and on the client I set a loop of tars of large directorys to the
  nfs
  mount running under time to see how well it survived. Then
  replicated
  the test with the patch and without.
 
 
  [root@seaurchin ~]# ministat nopatch.txt atomicpatch.txt
  x nopatch.txt
  + atomicpatch.txt
  +--+
  |
  *
  |
  |
  *
  |
  |
  x*
  |
  |   xx*
  x
  |
  |  +x**
  xx
  |
  |   xxx
  x
  |
  |   xxx +x+
  +
  |
  |  +*xx +x+ x
  +
  |
  |  +*xx + +
  x |
  |  *+*xx+ +++x * x
  x |
  |  x**++*x+***x+ x*+ x ++*+ + x+ +x +
  + +|
  |||___M_M_A__A___|__|
  |
  +--+
  N Min Max Median Avg Stddev
  x 101 1.25 106.8 14.08 21.892178 22.196005
  + 101 1.21 186.93 18.46 27.995842 30.523218
  No difference proven at 95.0% confidence
 
 
  (excuse wrapped ascii art)
 
  I think I'll have a look at the nfse patch set and see how that
  performs.
 Replying to myself just as a record, I have tried nfse and I didnt get
 the permission denied at all.
 The only issue I had with it is that it strictly adheres to the syntax
 in exports(5) while mountd is a little more flexible.
 
 I had
 /usr/local/export -alldirs -maproot=root 85.xx.xx.xx
 
 /usr is the root of that filesystem
 
 mountd - allowed this but actually silently exports /usr (and all dirs
 below)
 
Not exactly correct. mountd exports the entire file system in the kernel
for the NFS server, since exports can only be attached to the mount points
in the kernel. However, when the client's mount protocol requests a mount
file handle for anything other than /usr/local/export, it will refuse that.
(Which means that to mount anything other than /usr/local/export, the client
 must maliciously guess the file handle for mounting.)

Put another way, a non-malicious NFSv3 client will only be able to mount
/usr/local/export. Robert Watson calls this an administrative control and
feels that it is necessary.
Until this is resolved by the collective, I do not see how mountd can be
replaced by nfse, although I'd like to see mountd replaced because of the
above problem + mountd.c is very old, ugly (implies difficult to change) code.

 nfse - didnt allow this, it needed to be the correct
 /usr -alldirs -maproot=root 85.xx.xx.xx
 
 which is I guess a POLA violation but follows the documentation.
 this was using nfse in mountd compatibility mode. I havent played with
 its more advanced features.
 
 Vince
  Thanks for all your work on NFS on FreeBSD.
 
  Vince
 
 
 
 ___
 freebsd-current@freebsd.org mailing list
 http://lists.freebsd.org/mailman/listinfo/freebsd-current
 To unsubscribe, send any mail to
 freebsd-current-unsubscr...@freebsd.org
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org

Re: Occassional permission denied in the middle of a large transfer over NFS

2012-07-08 Thread Rick Macklem 
Vincent Hoffman:
 On 08/07/2012 00:26, Rick Macklem wrote:
  Vincent Hoffman wrote:
 
  Hi Rick,
 
  I'm afraid this didnt make any real difference for me.
  Since I couldnt test it on the live system I tried it on a test vm.
  on the vm (nfs server) I set a looping mount/umount
  while true ; do mount /dev/md0 /mnt/tmp ; sleep 1 ; umount /mnt/tmp
  ;
  done
 
  and on the client I set a loop of tars of large directorys to the
  nfs
  mount running under time to see how well it survived. Then
  replicated
  the test with the patch and without.
 
  Just to confirm, you patched both the kernel and mountd and replaced
  both
  on the server?
 
  Also, I'm not sure how ZFS handles it's exports. I can't remember if
  you've
  tried an exported UFS volume. It might be something ZFS specific?
 
  rick
 
 Hi Rick,
 
 yes I patched both the kernel and mountd, rebuilt kernel and world (to
 be sure), added the -S flag to mountd in rc.conf and rebooted.
 This is a test VM running -CURRENT and is only exporting a ufs2
 filesystem.
 (11:43:05 ~) 0 $ cat /etc/exports
 /usr/local/export -maproot=root -alldirs XX.XX.XX.XX
 
 Client is a 8.3-RELEASE box but I see the same with linux clients.
 (I can confirm that it works fine when I am not running the
 mount/umount
 loop)
 
Oops, the patch I sent you worked for NFSv4 only. If you also apply the
attached patch, it seems to work for NFSv3 as well.
The patch is also at:
  http://people.freebsd.org/~rmacklem/atomic-export2.patch

You must also run the new/experimental server. (I can't remember if I
mentioned that before.)

rick

 
 The production system has been fine since I removed the SIGHUP call in
 mount.c so thanks for that suggestion.
 
 
 Vince
 
  [root@seaurchin ~]# ministat nopatch.txt atomicpatch.txt
  x nopatch.txt
  + atomicpatch.txt
  +--+
  |
  *
  |
  |
  *
  |
  |
  x*
  |
  |   xx*
  x
  |
  |  +x**
  xx
  |
  |   xxx
  x
  |
  |   xxx +x+
  +
  |
  |  +*xx +x+ x
  +
  |
  |  +*xx + +
  x |
  |  *+*xx+ +++x * x
  x |
  |  x**++*x+***x+ x*+ x ++*+ + x+ +x +
  + +|
  |||___M_M_A__A___|__|
  |
  +--+
  N Min Max Median Avg Stddev
  x 101 1.25 106.8 14.08 21.892178 22.196005
  + 101 1.21 186.93 18.46 27.995842 30.523218
  No difference proven at 95.0% confidence
 
 
  (excuse wrapped ascii art)
 
  I think I'll have a look at the nfse patch set and see how that
  performs.
 
  Thanks for all your work on NFS on FreeBSD.
 
  Vince
 
  Also, you could easily hack mount.c so that it doesn't send a
  SIGHUP
  to mountd (which causes the exports to be reloaded) every time a
  local
  fs is mounted.
  True and I may have to do that for the production NAS for the
  time
  being.
  Thanks for looking at this.
 
  Vince
  rick
 
  thanks, Vince
 
  ___
  freebsd-current@freebsd.org mailing list
  http://lists.freebsd.org/mailman/listinfo/freebsd-current
  To unsubscribe, send any mail to
  freebsd-current-unsubscr...@freebsd.org
 
 
 ___
 freebsd-current@freebsd.org mailing list
 http://lists.freebsd.org/mailman/listinfo/freebsd-current
 To unsubscribe, send any mail to
 freebsd-current-unsubscr...@freebsd.org
--- fs/nfsserver/nfs_nfsdsocket.c.sav	2012-07-08 20:32:37.0 -0400
+++ fs/nfsserver/nfs_nfsdsocket.c	2012-07-08 20:42:03.0 -0400
@@ -352,7 +352,7 @@ APPLESTATIC void
 nfsrvd_dorpc(struct nfsrv_descript *nd, int isdgram,
 NFSPROC_T *p)
 {
-	int error = 0, lktype;
+	int error = 0, gotref = 0, lktype;
 	vnode_t vp;
 	mount_t mp = NULL;
 	struct nfsrvfh fh;
@@ -378,6 +378,11 @@ nfsrvd_dorpc(struct nfsrv_descript *nd, 
 nd-nd_repstat = NFSERR_GARBAGE;
 goto out;
 			}
+			gotref = 1;
+			NFSLOCKV4ROOTMUTEX();
+			nfsv4_getref(nfsv4rootfs_lock, NULL,
+			NFSV4ROOTLOCKMUTEXPTR, NULL);
+			NFSUNLOCKV4ROOTMUTEX();
 			if (nd-nd_procnum == NFSPROC_READ ||
 			nd-nd_procnum == NFSPROC_READDIR ||
 			nd-nd_procnum == NFSPROC_READLINK ||
@@ -471,6 +476,11 @@ nfsrvd_dorpc(struct nfsrv_descript *nd, 
 		nd-nd_flag = ~ND_SAVEREPLY;
 
 out:
+	if (gotref != 0) {
+		NFSLOCKV4ROOTMUTEX();
+		nfsv4_relref(nfsv4rootfs_lock);
+		NFSUNLOCKV4ROOTMUTEX();
+	}
 	NFSEXITCODE2(0, nd);
 }
 
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org

Re: Occassional permission denied in the middle of a large transfer over NFS

2012-07-07 Thread Vincent Hoffman 

On 01/07/2012 12:18, Rick Macklem wrote:
 Vincent Hoffman wrote:
 On 01/07/2012 01:53, Rick Macklem wrote:
 To modify mountd to use the kernel changes is more work than I have
 time for, in part because mountd.c is a very ugly old piece of C
 code, imho.

 I do have a patch that suspends/resumes execution of the nfsd
 threads
 (new, experimental for FreeBSD8.n only) when mountd reloads
 /etc/exports.
 This approach has certain disadvantages vs Andrey's transactional
 load/commit
 model, but it is simple and might be sufficient for your problem.
 If you want to try this patch, it is at:
http://people.freebsd.org/~rmacklem/atomic-export.patch
 (The patch affects both the kernel and mountd.c.)
 Happy to try it, to be certain I have understood, this is for the
 newnfs
 server (experimental in 8.x default for 8
 9+)
 Yes, that is correct. For the old (default on 8.x) it will have no effect.

 I'll apply to my -CUREENT test VM for now. Will fire up a 8.3 vm when
 i
 get a minute.
 Also, to enable it, you must add a -S flag to mountd_flags, after you've
 replaced the mountd executable.

 The patch is pretty straightforward, but has not seen much testing.
 Hopefully, it might be useful for you, rick

Hi Rick,

I'm afraid this didnt make any real difference for me.
Since I couldnt test it on the live system I tried it on a test vm.
on the vm (nfs server) I set a looping mount/umount
while true ; do mount /dev/md0 /mnt/tmp ; sleep 1 ; umount /mnt/tmp ;  done

and on the client I set a loop of tars of large directorys to the nfs
mount running under time to see how well it survived. Then replicated
the test with the patch and without.


[root@seaurchin ~]# ministat nopatch.txt atomicpatch.txt
x nopatch.txt
+ atomicpatch.txt
+--+
|
*   

|
|
*   

|
|   
x*  
 
|
|   xx* 
x   
 
|
|  +x** 
xx  
 
|
|   xxx 
x   
 
|
|   xxx +x+  
+   
|
|  +*xx +x+   x  
+   
|
|  +*xx + +
x  |
|  *+*xx+ +++x *  x
x  |
|  x**++*x+***x+ x*+ x  ++*+  + x+  +x +  
+  +|
|||___M_M_A__A___|__|   
  
|
+--+
N   Min   MaxMedian   AvgStddev
x 101  1.25 106.8 14.08 21.892178 22.196005
+ 101  1.21186.93 18.46 27.995842 30.523218
No difference proven at 95.0% confidence


(excuse wrapped ascii art)

I think I'll have a look at the nfse patch set and see how that performs.

Thanks for all your work on NFS on FreeBSD.

Vince


 Also, you could easily hack mount.c so that it doesn't send a SIGHUP
 to mountd (which causes the exports to be reloaded) every time a
 local
 fs is mounted.
 True and I may have to do that for the production NAS for the time
 being.
 Thanks for looking at this.

 Vince
 rick

 thanks, Vince


___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org

Re: Occassional permission denied in the middle of a large transfer over NFS

2012-07-07 Thread Rick Macklem 
Vincent Hoffman wrote:
 On 01/07/2012 12:18, Rick Macklem wrote:
  Vincent Hoffman wrote:
  On 01/07/2012 01:53, Rick Macklem wrote:
  To modify mountd to use the kernel changes is more work than I
  have
  time for, in part because mountd.c is a very ugly old piece of C
  code, imho.
 
  I do have a patch that suspends/resumes execution of the nfsd
  threads
  (new, experimental for FreeBSD8.n only) when mountd reloads
  /etc/exports.
  This approach has certain disadvantages vs Andrey's transactional
  load/commit
  model, but it is simple and might be sufficient for your problem.
  If you want to try this patch, it is at:
 http://people.freebsd.org/~rmacklem/atomic-export.patch
  (The patch affects both the kernel and mountd.c.)
  Happy to try it, to be certain I have understood, this is for the
  newnfs
  server (experimental in 8.x default for 8
  9+)
  Yes, that is correct. For the old (default on 8.x) it will have no
  effect.
 
  I'll apply to my -CUREENT test VM for now. Will fire up a 8.3 vm
  when
  i
  get a minute.
  Also, to enable it, you must add a -S flag to mountd_flags, after
  you've
  replaced the mountd executable.
 
  The patch is pretty straightforward, but has not seen much testing.
  Hopefully, it might be useful for you, rick
 
 Hi Rick,
 
 I'm afraid this didnt make any real difference for me.
 Since I couldnt test it on the live system I tried it on a test vm.
 on the vm (nfs server) I set a looping mount/umount
 while true ; do mount /dev/md0 /mnt/tmp ; sleep 1 ; umount /mnt/tmp ;
 done
 
 and on the client I set a loop of tars of large directorys to the nfs
 mount running under time to see how well it survived. Then replicated
 the test with the patch and without.
 
Just to confirm, you patched both the kernel and mountd and replaced both
on the server?

Also, I'm not sure how ZFS handles it's exports. I can't remember if you've
tried an exported UFS volume. It might be something ZFS specific?

rick

 
 [root@seaurchin ~]# ministat nopatch.txt atomicpatch.txt
 x nopatch.txt
 + atomicpatch.txt
 +--+
 |
 *
 |
 |
 *
 |
 |
 x*
 |
 |   xx*
 x
 |
 |  +x**
 xx
 |
 |   xxx
 x
 |
 |   xxx +x+
 +
 |
 |  +*xx +x+ x
 +
 |
 |  +*xx + +
 x |
 |  *+*xx+ +++x * x
 x |
 |  x**++*x+***x+ x*+ x ++*+ + x+ +x +
 + +|
 |||___M_M_A__A___|__|
 |
 +--+
 N Min Max Median Avg Stddev
 x 101 1.25 106.8 14.08 21.892178 22.196005
 + 101 1.21 186.93 18.46 27.995842 30.523218
 No difference proven at 95.0% confidence
 
 
 (excuse wrapped ascii art)
 
 I think I'll have a look at the nfse patch set and see how that
 performs.
 
 Thanks for all your work on NFS on FreeBSD.
 
 Vince
 
 
  Also, you could easily hack mount.c so that it doesn't send a
  SIGHUP
  to mountd (which causes the exports to be reloaded) every time a
  local
  fs is mounted.
  True and I may have to do that for the production NAS for the time
  being.
  Thanks for looking at this.
 
  Vince
  rick
 
  thanks, Vince
 
 
 ___
 freebsd-current@freebsd.org mailing list
 http://lists.freebsd.org/mailman/listinfo/freebsd-current
 To unsubscribe, send any mail to
 freebsd-current-unsubscr...@freebsd.org
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org

Re: Occassional permission denied in the middle of a large transfer over NFS

2012-07-02 Thread Andrey Simonenko 
On Sun, Jul 01, 2012 at 12:13:30PM +0100, Vincent Hoffman wrote:
 On 01/07/2012 01:53, Rick Macklem wrote:
 
  I haven't looked at Andrey's patch, but conceptually it sounds like
  the best approach. As I understand it, the problem with replacing
  mountd with nfse (at least in the FreeBSD source tree) is that nfse
  is not 100% backwards compatible with /etc/exports and, as such, is
  a POLA violation.
 Understood. Its far from a simple drop in replacement.

List of difference between nfse -C ... (compatible mode with mountd)
and mountd is given here:

http://lists.freebsd.org/pipermail/freebsd-fs/2012-June/014554.html

If we ignore absence of some obsolete options support and some command
line options, the rest of differences visible to a user will occur only
if one does not follow rules of exports(5) file format.

The native mode of nfse (nfs.exports(5) file format) is different
than the logic of mountd, just because using existent exports(5) file
format it is impossible to specify export of not mounted file system,
it is impossible to specify all export settings for one file system in
one line, etc.

Can you verify whether nfse compatible mode with mountd is really
compatible with exports(5) files on your systems using instructions
from this message (no installation or patching is required):

http://lists.freebsd.org/pipermail/freebsd-fs/2010-May/008421.html
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org

Re: Occassional permission denied in the middle of a large transfer over NFS

2012-07-02 Thread Vincent Hoffman 
On 02/07/2012 13:05, Andrey Simonenko wrote:
 On Sun, Jul 01, 2012 at 12:13:30PM +0100, Vincent Hoffman wrote:
 On 01/07/2012 01:53, Rick Macklem wrote:
 I haven't looked at Andrey's patch, but conceptually it sounds like
 the best approach. As I understand it, the problem with replacing
 mountd with nfse (at least in the FreeBSD source tree) is that nfse
 is not 100% backwards compatible with /etc/exports and, as such, is
 a POLA violation.
 Understood. Its far from a simple drop in replacement.
 List of difference between nfse -C ... (compatible mode with mountd)
 and mountd is given here:

 http://lists.freebsd.org/pipermail/freebsd-fs/2012-June/014554.html

 If we ignore absence of some obsolete options support and some command
 line options, the rest of differences visible to a user will occur only
 if one does not follow rules of exports(5) file format.

 The native mode of nfse (nfs.exports(5) file format) is different
 than the logic of mountd, just because using existent exports(5) file
 format it is impossible to specify export of not mounted file system,
 it is impossible to specify all export settings for one file system in
 one line, etc.

 Can you verify whether nfse compatible mode with mountd is really
 compatible with exports(5) files on your systems using instructions
 from this message (no installation or patching is required):

 http://lists.freebsd.org/pipermail/freebsd-fs/2010-May/008421.html
Its certainly compatible for me. I only have simple requirements though.
(Basic NFS exports for servers to dump their backups onto.)
nfse does look very good to me and I'll certainly be trying it in a VM.
Any Ideas as to what would be needed to get this imported?

Vince


___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org

Re: Occassional permission denied in the middle of a large transfer over NFS

2012-07-01 Thread Vincent Hoffman 
On 01/07/2012 01:53, Rick Macklem wrote:
 Vincent Hoffman wrote:
 Just a note to say I have tested this on -CURRENT with the new nfs
 server and it is still the case.

 On the client (FreeBSD seaurchin 8.3-RELEASE-p3 FreeBSD 8.3-RELEASE-p3
 #0: Tue Jun 12 00:39:29 UTC 2012
 r...@amd64-builder.daemonology.net:/usr/obj/usr/src/sys/GENERIC amd64)

 [root@seaurchin /mnt/nfs/vm]# date ; tar -cf foo.tar
 /var/nfsen/profiles-data/ ; date
 Thu Jun 28 20:12:36 BST 2012
 tar: Removing leading '/' from member names
 tar: Write error
 Thu Jun 28 20:12:38 BST 2012
 [root@seaurchin /mnt/nfs/vm]#



 on the Server
 [root@fbsd /var/tmp]# uname -a ; date ; while true ; do mount /dev/md0
 /mnt/tmp/ ; umount /mnt/tmp ; done
 FreeBSD fbsd 10.0-CURRENT FreeBSD 10.0-CURRENT #22 r237646: Wed Jun 27
 19:13:26 BST 2012
 t...@fbsd.bmk.namesco.net:/usr/obj/usr/src/sys/unsane-vm amd64
 Thu Jun 28 20:12:38 BST 2012
 ^C
 [root@fbsd /var/tmp]#

 any suggestions welcome.

 Vince


 On 27/06/2012 16:27, Vincent Hoffman wrote:
 Hi,
 After only one off-list reply from the author of kern/136865 (see
 below)
 after asking -questions, I thought it worth asking -CURRENT.
 Basically:

 I seem to have run into the problems described in this old thread.
 http://lists.freebsd.org/pipermail/freebsd-questions/2004-April/044927.html

 tl:dr mountd may give incorrect permission denied errors when it is
 refreshing the exports list due to non-atomic operations,
 /sbin/mount has code that sends SIGHUP to
 mountd on any mount operation, which implies that any manual mount
 request would cause the problem.

 Currently I have still only tested on 8.3-RELEASE but the svn log
 doesnt
 seem to mention a fix since then. I'm currently taking a VM up to
 -CURRENT to test.

 Looking though old PRs I see the following related.
 kern/131342
 kern/136865 (with patch for 7.2 and links to
 http://nfse.sourceforge.net/ for -CURRENT )

 Does anyone who is qualified (sadly not me) feel like looking at the
 code to see if its suitable for inclusion in part/whole as not
 having
 NFS transfers interrupted by local mount operations on the nfs
 server
 would be very handy :)

 I haven't looked at Andrey's patch, but conceptually it sounds like
 the best approach. As I understand it, the problem with replacing
 mountd with nfse (at least in the FreeBSD source tree) is that nfse
 is not 100% backwards compatible with /etc/exports and, as such, is
 a POLA violation.
Understood. Its far from a simple drop in replacement.

 To modify mountd to use the kernel changes is more work than I have
 time for, in part because mountd.c is a very ugly old piece of C code, imho.

 I do have a patch that suspends/resumes execution of the nfsd threads
 (new, experimental for FreeBSD8.n only) when mountd reloads /etc/exports.
 This approach has certain disadvantages vs Andrey's transactional load/commit
 model, but it is simple and might be sufficient for your problem.
 If you want to try this patch, it is at:
http://people.freebsd.org/~rmacklem/atomic-export.patch
 (The patch affects both the kernel and mountd.c.)
Happy to try it, to be certain I have understood, this is for the newnfs
server (experimental in 8.x default for 8
9+)
I'll apply to my -CUREENT test VM for now. Will fire up a 8.3 vm when i
get a minute.

 Also, you could easily hack mount.c so that it doesn't send a SIGHUP
 to mountd (which causes the exports to be reloaded) every time a local
 fs is mounted.
True and I may have to do that for the production NAS for the time being.
Thanks for looking at this.

Vince

 rick

 thanks, Vince


 ___
 freebsd-current@freebsd.org mailing list
 http://lists.freebsd.org/mailman/listinfo/freebsd-current
 To unsubscribe, send any mail to
 freebsd-current-unsubscr...@freebsd.org

 ___
 freebsd-current@freebsd.org mailing list
 http://lists.freebsd.org/mailman/listinfo/freebsd-current
 To unsubscribe, send any mail to
 freebsd-current-unsubscr...@freebsd.org
 ___
 freebsd-current@freebsd.org mailing list
 http://lists.freebsd.org/mailman/listinfo/freebsd-current
 To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org

Re: Occassional permission denied in the middle of a large transfer over NFS

2012-07-01 Thread Rick Macklem 
Vincent Hoffman wrote:
 On 01/07/2012 01:53, Rick Macklem wrote:
  Vincent Hoffman wrote:
  Just a note to say I have tested this on -CURRENT with the new nfs
  server and it is still the case.
 
  On the client (FreeBSD seaurchin 8.3-RELEASE-p3 FreeBSD
  8.3-RELEASE-p3
  #0: Tue Jun 12 00:39:29 UTC 2012
  r...@amd64-builder.daemonology.net:/usr/obj/usr/src/sys/GENERIC
  amd64)
 
  [root@seaurchin /mnt/nfs/vm]# date ; tar -cf foo.tar
  /var/nfsen/profiles-data/ ; date
  Thu Jun 28 20:12:36 BST 2012
  tar: Removing leading '/' from member names
  tar: Write error
  Thu Jun 28 20:12:38 BST 2012
  [root@seaurchin /mnt/nfs/vm]#
 
 
 
  on the Server
  [root@fbsd /var/tmp]# uname -a ; date ; while true ; do mount
  /dev/md0
  /mnt/tmp/ ; umount /mnt/tmp ; done
  FreeBSD fbsd 10.0-CURRENT FreeBSD 10.0-CURRENT #22 r237646: Wed Jun
  27
  19:13:26 BST 2012
  t...@fbsd.bmk.namesco.net:/usr/obj/usr/src/sys/unsane-vm amd64
  Thu Jun 28 20:12:38 BST 2012
  ^C
  [root@fbsd /var/tmp]#
 
  any suggestions welcome.
 
  Vince
 
 
  On 27/06/2012 16:27, Vincent Hoffman wrote:
  Hi,
  After only one off-list reply from the author of kern/136865 (see
  below)
  after asking -questions, I thought it worth asking -CURRENT.
  Basically:
 
  I seem to have run into the problems described in this old thread.
  http://lists.freebsd.org/pipermail/freebsd-questions/2004-April/044927.html
 
  tl:dr mountd may give incorrect permission denied errors when it
  is
  refreshing the exports list due to non-atomic operations,
  /sbin/mount has code that sends SIGHUP to
  mountd on any mount operation, which implies that any manual mount
  request would cause the problem.
 
  Currently I have still only tested on 8.3-RELEASE but the svn log
  doesnt
  seem to mention a fix since then. I'm currently taking a VM up to
  -CURRENT to test.
 
  Looking though old PRs I see the following related.
  kern/131342
  kern/136865 (with patch for 7.2 and links to
  http://nfse.sourceforge.net/ for -CURRENT )
 
  Does anyone who is qualified (sadly not me) feel like looking at
  the
  code to see if its suitable for inclusion in part/whole as not
  having
  NFS transfers interrupted by local mount operations on the nfs
  server
  would be very handy :)
 
  I haven't looked at Andrey's patch, but conceptually it sounds like
  the best approach. As I understand it, the problem with replacing
  mountd with nfse (at least in the FreeBSD source tree) is that nfse
  is not 100% backwards compatible with /etc/exports and, as such, is
  a POLA violation.
 Understood. Its far from a simple drop in replacement.
 
  To modify mountd to use the kernel changes is more work than I have
  time for, in part because mountd.c is a very ugly old piece of C
  code, imho.
 
  I do have a patch that suspends/resumes execution of the nfsd
  threads
  (new, experimental for FreeBSD8.n only) when mountd reloads
  /etc/exports.
  This approach has certain disadvantages vs Andrey's transactional
  load/commit
  model, but it is simple and might be sufficient for your problem.
  If you want to try this patch, it is at:
 http://people.freebsd.org/~rmacklem/atomic-export.patch
  (The patch affects both the kernel and mountd.c.)
 Happy to try it, to be certain I have understood, this is for the
 newnfs
 server (experimental in 8.x default for 8
 9+)
Yes, that is correct. For the old (default on 8.x) it will have no effect.

 I'll apply to my -CUREENT test VM for now. Will fire up a 8.3 vm when
 i
 get a minute.

Also, to enable it, you must add a -S flag to mountd_flags, after you've
replaced the mountd executable.

The patch is pretty straightforward, but has not seen much testing.
Hopefully, it might be useful for you, rick

 
  Also, you could easily hack mount.c so that it doesn't send a SIGHUP
  to mountd (which causes the exports to be reloaded) every time a
  local
  fs is mounted.
 True and I may have to do that for the production NAS for the time
 being.
 Thanks for looking at this.
 
 Vince
 
  rick
 
  thanks, Vince
 
 
  ___
  freebsd-current@freebsd.org mailing list
  http://lists.freebsd.org/mailman/listinfo/freebsd-current
  To unsubscribe, send any mail to
  freebsd-current-unsubscr...@freebsd.org
 
  ___
  freebsd-current@freebsd.org mailing list
  http://lists.freebsd.org/mailman/listinfo/freebsd-current
  To unsubscribe, send any mail to
  freebsd-current-unsubscr...@freebsd.org
  ___
  freebsd-current@freebsd.org mailing list
  http://lists.freebsd.org/mailman/listinfo/freebsd-current
  To unsubscribe, send any mail to
  freebsd-current-unsubscr...@freebsd.org
 
 
 ___
 freebsd-current@freebsd.org mailing list
 http://lists.freebsd.org/mailman/listinfo/freebsd-current
 To unsubscribe, send any mail to
 freebsd-current-unsubscr...@freebsd.org
___

Re: Occassional permission denied in the middle of a large transfer over NFS

2012-06-30 Thread Rick Macklem 
Vincent Hoffman wrote:
 Just a note to say I have tested this on -CURRENT with the new nfs
 server and it is still the case.
 
 On the client (FreeBSD seaurchin 8.3-RELEASE-p3 FreeBSD 8.3-RELEASE-p3
 #0: Tue Jun 12 00:39:29 UTC 2012
 r...@amd64-builder.daemonology.net:/usr/obj/usr/src/sys/GENERIC amd64)
 
 [root@seaurchin /mnt/nfs/vm]# date ; tar -cf foo.tar
 /var/nfsen/profiles-data/ ; date
 Thu Jun 28 20:12:36 BST 2012
 tar: Removing leading '/' from member names
 tar: Write error
 Thu Jun 28 20:12:38 BST 2012
 [root@seaurchin /mnt/nfs/vm]#
 
 
 
 on the Server
 [root@fbsd /var/tmp]# uname -a ; date ; while true ; do mount /dev/md0
 /mnt/tmp/ ; umount /mnt/tmp ; done
 FreeBSD fbsd 10.0-CURRENT FreeBSD 10.0-CURRENT #22 r237646: Wed Jun 27
 19:13:26 BST 2012
 t...@fbsd.bmk.namesco.net:/usr/obj/usr/src/sys/unsane-vm amd64
 Thu Jun 28 20:12:38 BST 2012
 ^C
 [root@fbsd /var/tmp]#
 
 any suggestions welcome.
 
 Vince
 
 
 On 27/06/2012 16:27, Vincent Hoffman wrote:
  Hi,
  After only one off-list reply from the author of kern/136865 (see
  below)
  after asking -questions, I thought it worth asking -CURRENT.
  Basically:
 
  I seem to have run into the problems described in this old thread.
  http://lists.freebsd.org/pipermail/freebsd-questions/2004-April/044927.html
 
  tl:dr mountd may give incorrect permission denied errors when it is
  refreshing the exports list due to non-atomic operations,
  /sbin/mount has code that sends SIGHUP to
  mountd on any mount operation, which implies that any manual mount
  request would cause the problem.
 
  Currently I have still only tested on 8.3-RELEASE but the svn log
  doesnt
  seem to mention a fix since then. I'm currently taking a VM up to
  -CURRENT to test.
 
  Looking though old PRs I see the following related.
  kern/131342
  kern/136865 (with patch for 7.2 and links to
  http://nfse.sourceforge.net/ for -CURRENT )
 
  Does anyone who is qualified (sadly not me) feel like looking at the
  code to see if its suitable for inclusion in part/whole as not
  having
  NFS transfers interrupted by local mount operations on the nfs
  server
  would be very handy :)
 
I haven't looked at Andrey's patch, but conceptually it sounds like
the best approach. As I understand it, the problem with replacing
mountd with nfse (at least in the FreeBSD source tree) is that nfse
is not 100% backwards compatible with /etc/exports and, as such, is
a POLA violation.

To modify mountd to use the kernel changes is more work than I have
time for, in part because mountd.c is a very ugly old piece of C code, imho.

I do have a patch that suspends/resumes execution of the nfsd threads
(new, experimental for FreeBSD8.n only) when mountd reloads /etc/exports.
This approach has certain disadvantages vs Andrey's transactional load/commit
model, but it is simple and might be sufficient for your problem.
If you want to try this patch, it is at:
   http://people.freebsd.org/~rmacklem/atomic-export.patch
(The patch affects both the kernel and mountd.c.)

Also, you could easily hack mount.c so that it doesn't send a SIGHUP
to mountd (which causes the exports to be reloaded) every time a local
fs is mounted.

rick

 
  thanks, Vince
 
 
  ___
  freebsd-current@freebsd.org mailing list
  http://lists.freebsd.org/mailman/listinfo/freebsd-current
  To unsubscribe, send any mail to
  freebsd-current-unsubscr...@freebsd.org
 
 
 ___
 freebsd-current@freebsd.org mailing list
 http://lists.freebsd.org/mailman/listinfo/freebsd-current
 To unsubscribe, send any mail to
 freebsd-current-unsubscr...@freebsd.org
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org

Re: Occassional permission denied in the middle of a large transfer over NFS

2012-06-28 Thread Vincent Hoffman 
Just a note to say I have tested this on -CURRENT with the new nfs
server and it is still the case.

On the client (FreeBSD seaurchin 8.3-RELEASE-p3 FreeBSD 8.3-RELEASE-p3
#0: Tue Jun 12 00:39:29 UTC 2012
r...@amd64-builder.daemonology.net:/usr/obj/usr/src/sys/GENERIC  amd64)

[root@seaurchin /mnt/nfs/vm]# date ; tar -cf foo.tar 
/var/nfsen/profiles-data/ ; date
Thu Jun 28 20:12:36 BST 2012
tar: Removing leading '/' from member names
tar: Write error
Thu Jun 28 20:12:38 BST 2012
[root@seaurchin /mnt/nfs/vm]#



on the Server
[root@fbsd /var/tmp]# uname -a ; date ; while true ; do mount /dev/md0
/mnt/tmp/ ; umount /mnt/tmp ; done
FreeBSD fbsd 10.0-CURRENT FreeBSD 10.0-CURRENT #22 r237646: Wed Jun 27
19:13:26 BST 2012
t...@fbsd.bmk.namesco.net:/usr/obj/usr/src/sys/unsane-vm  amd64
Thu Jun 28 20:12:38 BST 2012
^C
[root@fbsd /var/tmp]#

any suggestions welcome.

Vince


On 27/06/2012 16:27, Vincent Hoffman wrote:
 Hi,
 After only one off-list reply from the author of kern/136865 (see below)
 after asking -questions, I thought it worth asking -CURRENT.
 Basically:

 I seem to have run into the problems described in this old thread.
 http://lists.freebsd.org/pipermail/freebsd-questions/2004-April/044927.html

 tl:dr mountd may give incorrect permission denied errors when it is
 refreshing the exports list due to non-atomic operations,  /sbin/mount has 
 code that sends SIGHUP to
 mountd on any mount operation, which implies that any manual mount
 request would cause the problem.  

 Currently I have still only tested on 8.3-RELEASE but the svn log doesnt
 seem to mention a fix since then. I'm currently taking a VM up to
 -CURRENT to test.

 Looking though old PRs I see the following related.
 kern/131342
 kern/136865 (with patch for 7.2 and links to
 http://nfse.sourceforge.net/ for -CURRENT )

 Does anyone who is qualified (sadly not me) feel like looking at the
 code to see if its suitable for inclusion in part/whole as not having
 NFS transfers interrupted by local mount operations on the nfs server
 would be very handy :)


 thanks, Vince


 ___
 freebsd-current@freebsd.org mailing list
 http://lists.freebsd.org/mailman/listinfo/freebsd-current
 To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org