[nfs-discuss] NFS service won't stay in the enabled state
Hm. Sorry about the delay, I wanted to make sure that this was really
working before sharing the good news :-p
So the strangest thing happened. By doing these exact steps:
1) Disable ZFS shares
2) Enable /tmp ro share
3) Reboot
4) Enable ZFS shares
5) Remove /tmp share
everything now seems to work. I'm not sure exactly how/why, but it
fixed itself.
Rebooting took over twelve hours. It had to time out on something
while unmounting every zfs filesystem, which was painfully slow.
But it works now, and reboots are down to manageable times.
Thanks for all your help,
Steven
On Apr 10, 2009, at 12:45 PM, Tom Haynes wrote:
> Steven,
>
> I wrote a simple little script to create 3000 zfs filesystems and
> then to
> share the common ancestor:
>
> [th199096 at jhereg ~/scripts]> more tasty.pl
> #!/usr/bin/perl
>
> print "zfs create tank/home\n";
>
> for ($i = 0; $i < 3000; $i++) {
> print "zfs create tank/home/u$i\n";
> }
>
> print "zfs set sharenfs=rw tank/home\n";
>
> And then I executed the output:
>
> [root at jhereg ~]> uname -a
> SunOS jhereg 5.11 snv_111 i86pc i386 i86pc
> [root at jhereg ~]> zfs list | wc -l
> 3004
> [root at jhereg ~]> share | wc -l
> 3002
> [root at jhereg ~]>
>
> No clue as to how long it took - I went out to lunch.
>
> nfsd was not running before I started.
>
>
> So I don't think it is the number of shares and your other
> experiments have supported this.
>
> I can bump down to snv_101b, but I'm not sure that is the problem.
>
> Some questions:
>
> 1) svcs -xv only shows that nfsd is not running, right?
>
> 2) You have rebooted at some point? We all know that makes things
> work, right? :->
>
> 3) Did you change any other properties on the zfs data sets?
>
> 4) Did you change any nfs settings?
> /etc/default/nfs
>
> 5) When you tried Rob's 1 share, are you sure that all of the zfs
> sharenfs properties
> are cleared?
>
> ---
>
> I did a bug search and I saw one where secure-by-default caused nfsd
> not to start. But
> that was not reproducible.
>
> Something to check though.
>
> Thanks,
> Tom
[nfs-discuss] NFS service won't stay in the enabled state
Steven,
I wrote a simple little script to create 3000 zfs filesystems and then to
share the common ancestor:
[th199096 at jhereg ~/scripts]> more tasty.pl
#!/usr/bin/perl
print "zfs create tank/home\n";
for ($i = 0; $i < 3000; $i++) {
print "zfs create tank/home/u$i\n";
}
print "zfs set sharenfs=rw tank/home\n";
And then I executed the output:
[root at jhereg ~]> uname -a
SunOS jhereg 5.11 snv_111 i86pc i386 i86pc
[root at jhereg ~]> zfs list | wc -l
3004
[root at jhereg ~]> share | wc -l
3002
[root at jhereg ~]>
No clue as to how long it took - I went out to lunch.
nfsd was not running before I started.
So I don't think it is the number of shares and your other
experiments have supported this.
I can bump down to snv_101b, but I'm not sure that is the problem.
Some questions:
1) svcs -xv only shows that nfsd is not running, right?
2) You have rebooted at some point? We all know that makes things work,
right? :->
3) Did you change any other properties on the zfs data sets?
4) Did you change any nfs settings?
/etc/default/nfs
5) When you tried Rob's 1 share, are you sure that all of the zfs
sharenfs properties
are cleared?
---
I did a bug search and I saw one where secure-by-default caused nfsd not
to start. But
that was not reproducible.
Something to check though.
Thanks,
Tom
[nfs-discuss] NFS service won't stay in the enabled state
On Apr 10, 2009, at 10:16 AM, Robert Thurlow wrote: > Steven Schlansker wrote: > >> I've set up all my ZFS filesystems (about 3000 of them - I made a >> separate one for each user as I've heard recommended) and I'm >> trying to share them all > > Hi Steven, > > This is the right place, and you've found what looks like a > real problem I haven't seen before. Validation is good, right? :-) At least it means I haven't completely lost it yet! > > > We have had trouble in the past with large numbers of shares, and > we're not done with the work to address it yet. I wondered about > the start method timing out, but that timeout is an hour, and > you're seeing 5 minutes. One time I did see it timeout after an hour - that particular time I was creating a bunch of ZFS filesystems at the same time, and it was going /really/ slowly (one FS about every 30 seconds). I'm guessing some sort of global zfs lock was probably holding up the start script, and it timed out... But either way, I'm not seeing that now. > > > I would try to get things stable with just one share in dfstab; > you do have to have at least one for the server to not disable > itself. If you can get the server functional and then add > shares, we can see if the problem returns and see if it's about > scale. If you just can't get going at all, it's a different > problem. > > Rob T I changed dfstab, ignoring the warning at the top completely, to # Do not modify this file directly. # Use the sharemgr(1m) command for all share management # blah blah blah... share -F nfs -o ro /tmp and then svcadm enable network/nfs/server Same thing. It starts up, complains that there's no shares, then shuts down. Interestingly enough, it rewrote my dfstab to: share -F nfs -o sec=sys,ro /tmp so someone somewhere did see the line and rewrote it. I also tried using a share command directly in my conversation with Tom earlier - same symptoms. Thanks for the help! Hopefully we can get this going soon, I've got a very finicky NFS server which really needs to be retired soon and replaced by this guy ;-) Steven
[nfs-discuss] NFS service won't stay in the enabled state
Steven Schlansker wrote: > I've set up all my ZFS filesystems (about 3000 of them - I made a > separate one for each user as I've heard recommended) and I'm trying to > share them all Hi Steven, This is the right place, and you've found what looks like a real problem I haven't seen before. Validation is good, right? :-) We have had trouble in the past with large numbers of shares, and we're not done with the work to address it yet. I wondered about the start method timing out, but that timeout is an hour, and you're seeing 5 minutes. I would try to get things stable with just one share in dfstab; you do have to have at least one for the server to not disable itself. If you can get the server functional and then add shares, we can see if the problem returns and see if it's about scale. If you just can't get going at all, it's a different problem. Rob T
[nfs-discuss] NFS service won't stay in the enabled state
Steven Schlansker wrote: > > > Then it says: > > svc:/network/nfs/server:default (NFS server) > State: online since Thu Apr 09 20:19:10 2009 >See: man -M /usr/share/man -s 1M nfsd >See: /var/svc/log/network-nfs-server:default.log > Impact: None. > > And when the stop method finishes (according to the log) > > svc:/network/nfs/server:default (NFS server) > State: disabled since Thu Apr 09 20:22:34 2009 > Reason: Temporarily disabled by an administrator. >See: http://sun.com/msg/SMF-8000-1S >See: man -M /usr/share/man -s 1M nfsd >See: /var/svc/log/network-nfs-server:default.log > Impact: This service is not running. Did you follow the link ( http://sun.com/msg/SMF-8000-1S) and try: |svcadm enable -t |svc:/network/nfs/server I don't know the svcs stuff the way I should, but try working with it. Also, what version of OpenSolaris are you running?
[nfs-discuss] NFS service won't stay in the enabled state
(Sorry Tom, didn't reply all - resend!) On Apr 9, 2009, at 10:32 PM, Tom Haynes wrote: > Steven Schlansker wrote: >> >> >> And when the stop method finishes (according to the log) >> >> svc:/network/nfs/server:default (NFS server) >> State: disabled since Thu Apr 09 20:22:34 2009 >> Reason: Temporarily disabled by an administrator. >> See: http://sun.com/msg/SMF-8000-1S >> See: man -M /usr/share/man -s 1M nfsd >> See: /var/svc/log/network-nfs-server:default.log >> Impact: This service is not running. > > Did you follow the link ( http://sun.com/msg/SMF-8000-1S) and try: > > |svcadm enable -t |svc:/network/nfs/server > > I don't know the svcs stuff the way I should, but try working with it. > > Also, what version of OpenSolaris are you running? Sorry, should have mentioned. It does the same thing - enables, complains that there's no filesystems to export, then shuts itself down. SunOS cooler.csua.berkeley.edu 5.11 snv_101b i86pc i386 i86pc Solaris
[nfs-discuss] NFS service won't stay in the enabled state
On Apr 9, 2009, at 6:45 PM, Tom Haynes wrote:
>
> Steven Schlansker wrote:
>>
>>
>> pnunez at coole:/etc/dfs$ pfexec sharemgr show -vp | head
>> default nfs=()
>> zfs
>>zfs/cooler/linux nfs=() nfs:sys=(ro="*")
>> /cooler/linux
>>zfs/cooler/sodahome nfs=() nfs:sys=(ro="@10.1.1")
>
> Just being nit picky, but they all aren't inheriting the share you
> had above. I.e., you've got
> a read-only share to the 10.1.1/24 subnet here.
Yeah, that was intentional. Sorry about my words not matching what is
actually happening - although this doesn't really change the problem :)
>> /cooler/sodahome
>> /cooler/sodahome/Cameron589
>> /cooler/sodahome/GnowAD
>> /cooler/sodahome/RichLan
>> /cooler/sodahome/ThomasEP
>> ...(continues for thousands of lines)
>>
>> and watch the svc logs (with tail -f /var/svc/log/*)
>>
>> ==> /var/svc/log/network-nfs-server:default.log <==
>> [ Apr 9 17:44:52 Enabled. ]
>> [ Apr 9 17:44:52 Executing start method ("/lib/svc/method/nfs-
>> server start"). ]
>> No NFS filesystems are shared
>> [ Apr 9 17:49:19 Method "start" exited with status 0. ]
>> [ Apr 9 17:49:19 Stopping because service disabled. ]
>> [ Apr 9 17:49:20 Executing stop method ("/lib/svc/method/nfs-
>> server stop 142"). ]
>>
>>
>> NFS then proceeds to shut down.
>> First question - why does NFS shut down immediately even though I
>> have configured shares?
>
> What do you get with:
>
>
> svcs -xv svc:/network/nfs/server
>
> You might look at this thread:
While it's starting up:
svc:/network/nfs/server:default (NFS server)
State: offline since Thu Apr 09 20:14:37 2009
Reason: Start method is running.
See: http://sun.com/msg/SMF-8000-C4
See: man -M /usr/share/man -s 1M nfsd
See: /var/svc/log/network-nfs-server:default.log
Impact: This service is not running.
Then it says:
svc:/network/nfs/server:default (NFS server)
State: online since Thu Apr 09 20:19:10 2009
See: man -M /usr/share/man -s 1M nfsd
See: /var/svc/log/network-nfs-server:default.log
Impact: None.
And when the stop method finishes (according to the log)
svc:/network/nfs/server:default (NFS server)
State: disabled since Thu Apr 09 20:22:34 2009
Reason: Temporarily disabled by an administrator.
See: http://sun.com/msg/SMF-8000-1S
See: man -M /usr/share/man -s 1M nfsd
See: /var/svc/log/network-nfs-server:default.log
Impact: This service is not running.
>
>
>
> http://www.unix.com/sun-solaris/80574-nfs-service-offline-after-enabling.html
>
> I'll do some more checking later ...
svcs -xv is empty, and rpcbind is on:
online 12:12:42 svc:/network/rpc/bind:default
>
>
>
>> Second question - is it normal for NFS to take a while five minutes
>> to start up? It seems like it should be faster, even with a few
>> thousand exports...
>>
>
> Look at http://developers.sun.com/solaris/articles/nfs_zfs.html for
> a discussion on what is going on here. We have been working on it. :->
Cool, thanks. Good to know :)
>
>
>
>> Just for fun, i tried doing an export using
>> pfexec share -F nfs -o ro /tmp
>> and the same thing happened (nfs starts then shuts down)
>
> Try removing the zfs shares and doing this again?
Same thing. It shares it, starts nfs/server, and then immediately
disables it again.
>
>
> This would be good because it would eliminate the large number of
> shares being a problem.
>
>>
>> /etc/dfs is:
>> pnunez at cooler:/etc/dfs$ ls -lh
>> total 2.5K
>> -rw-r--r-- 1 root sys 354 2008-11-19 23:43 dfstab
>> -rw-r--r-- 1 root root 89 2008-11-19 23:43 fstypes
>> -rw-r--r-- 1 root root 0 2009-04-09 13:06 sharetab
>
> I.e., nothing is shared here...
Right. For some reason zfs share and share -F nfs aren't sharing
anything. How can I got about fixing this?
Thanks,
Steven
[nfs-discuss] NFS service won't stay in the enabled state
Steven,
Some inline comments:
Steven Schlansker wrote:
> Hello everyone,
> I'm hoping this is the right place to post to - I've just started
> using OpenSolaris and would like to use it as a ZFS/NFS server for
> some Linux/FreeBSD clients.
>
> I do apologize if I'm doing anything particularly dumb - I've cobbled
> this information from so many sources on the Internet that I hardly
> even know what's the right way to do anything anymore :-p
>
> I've set up all my ZFS filesystems (about 3000 of them - I made a
> separate one for each user as I've heard recommended) and I'm trying
> to share them all, using
>
> pfexec zfs set sharenfs=ro pool/dataset
>
> (the rest all inherit the sharenfs property)
>
> I then run
>
> pfexec zfs share -a
>
> and I check sharemgr:
>
> pnunez at coole:/etc/dfs$ pfexec sharemgr show -vp | head
> default nfs=()
> zfs
> zfs/cooler/linux nfs=() nfs:sys=(ro="*")
> /cooler/linux
> zfs/cooler/sodahome nfs=() nfs:sys=(ro="@10.1.1")
Just being nit picky, but they all aren't inheriting the share you had
above. I.e., you've got
a read-only share to the 10.1.1/24 subnet here.
> /cooler/sodahome
> /cooler/sodahome/Cameron589
> /cooler/sodahome/GnowAD
> /cooler/sodahome/RichLan
> /cooler/sodahome/ThomasEP
> ...(continues for thousands of lines)
>
> and watch the svc logs (with tail -f /var/svc/log/*)
>
> ==> /var/svc/log/network-nfs-server:default.log <==
> [ Apr 9 17:44:52 Enabled. ]
> [ Apr 9 17:44:52 Executing start method ("/lib/svc/method/nfs-server
> start"). ]
> No NFS filesystems are shared
> [ Apr 9 17:49:19 Method "start" exited with status 0. ]
> [ Apr 9 17:49:19 Stopping because service disabled. ]
> [ Apr 9 17:49:20 Executing stop method ("/lib/svc/method/nfs-server
> stop 142"). ]
>
>
> NFS then proceeds to shut down.
> First question - why does NFS shut down immediately even though I have
> configured shares?
What do you get with:
svcs -xv svc:/network/nfs/server
You might look at this thread:
http://www.unix.com/sun-solaris/80574-nfs-service-offline-after-enabling.html
I'll do some more checking later ...
> Second question - is it normal for NFS to take a while five minutes to
> start up? It seems like it should be faster, even with a few thousand
> exports...
>
Look at http://developers.sun.com/solaris/articles/nfs_zfs.html for a
discussion on what is going on here. We have been working on it. :->
> Just for fun, i tried doing an export using
> pfexec share -F nfs -o ro /tmp
> and the same thing happened (nfs starts then shuts down)
Try removing the zfs shares and doing this again?
This would be good because it would eliminate the large number of shares
being a problem.
>
> /etc/dfs is:
> pnunez at cooler:/etc/dfs$ ls -lh
> total 2.5K
> -rw-r--r-- 1 root sys 354 2008-11-19 23:43 dfstab
> -rw-r--r-- 1 root root 89 2008-11-19 23:43 fstypes
> -rw-r--r-- 1 root root 0 2009-04-09 13:06 sharetab
I.e., nothing is shared here...
>
> DFStab has only the comments that come with it, fstypes has:
> nfs NFS Utilities
> autofs AUTOFS Utilities
> cachefs CACHEFS Utilities
> smbfs CIFS Utilities
>
>
> and sharetab is empty all the time.
>
>
> Thanks for any tips and/or pointers! I have searched around but
> either this isn't a normal problem or my google-fu is failing me today.
>
> Best,
> Steven Schlansker
> ___
> nfs-discuss mailing list
> nfs-discuss at opensolaris.org
[nfs-discuss] NFS service won't stay in the enabled state
Hello everyone,
I'm hoping this is the right place to post to - I've just started
using OpenSolaris and would like to use it as a ZFS/NFS server for
some Linux/FreeBSD clients.
I do apologize if I'm doing anything particularly dumb - I've cobbled
this information from so many sources on the Internet that I hardly
even know what's the right way to do anything anymore :-p
I've set up all my ZFS filesystems (about 3000 of them - I made a
separate one for each user as I've heard recommended) and I'm trying
to share them all, using
pfexec zfs set sharenfs=ro pool/dataset
(the rest all inherit the sharenfs property)
I then run
pfexec zfs share -a
and I check sharemgr:
pnunez at coole:/etc/dfs$ pfexec sharemgr show -vp | head
default nfs=()
zfs
zfs/cooler/linux nfs=() nfs:sys=(ro="*")
/cooler/linux
zfs/cooler/sodahome nfs=() nfs:sys=(ro="@10.1.1")
/cooler/sodahome
/cooler/sodahome/Cameron589
/cooler/sodahome/GnowAD
/cooler/sodahome/RichLan
/cooler/sodahome/ThomasEP
...(continues for thousands of lines)
and watch the svc logs (with tail -f /var/svc/log/*)
==> /var/svc/log/network-nfs-server:default.log <==
[ Apr 9 17:44:52 Enabled. ]
[ Apr 9 17:44:52 Executing start method ("/lib/svc/method/nfs-server
start"). ]
No NFS filesystems are shared
[ Apr 9 17:49:19 Method "start" exited with status 0. ]
[ Apr 9 17:49:19 Stopping because service disabled. ]
[ Apr 9 17:49:20 Executing stop method ("/lib/svc/method/nfs-server
stop 142"). ]
NFS then proceeds to shut down.
First question - why does NFS shut down immediately even though I have
configured shares?
Second question - is it normal for NFS to take a while five minutes to
start up? It seems like it should be faster, even with a few thousand
exports...
Just for fun, i tried doing an export using
pfexec share -F nfs -o ro /tmp
and the same thing happened (nfs starts then shuts down)
/etc/dfs is:
pnunez at cooler:/etc/dfs$ ls -lh
total 2.5K
-rw-r--r-- 1 root sys 354 2008-11-19 23:43 dfstab
-rw-r--r-- 1 root root 89 2008-11-19 23:43 fstypes
-rw-r--r-- 1 root root 0 2009-04-09 13:06 sharetab
DFStab has only the comments that come with it, fstypes has:
nfs NFS Utilities
autofs AUTOFS Utilities
cachefs CACHEFS Utilities
smbfs CIFS Utilities
and sharetab is empty all the time.
Thanks for any tips and/or pointers! I have searched around but
either this isn't a normal problem or my google-fu is failing me today.
Best,
Steven Schlansker
