Re: [gentoo-user] nfsv4 issues
On Sun, Jul 24, 2016 at 3:37 AM, Adam Carterwrote: > I've added the directory, and after restarting syslog now has new entries; >> >> kernel: [912267.948883] NFSD: Using /var/lib/nfs/v4recovery as the NFSv4 >> state recovery directory >> kernel: NFSD: Using /var/lib/nfs/v4recovery as the NFSv4 state recovery >> directory >> >> I will test shortly and report back - thanks! > > Confirmed - this fixes the 30 second delay. Good. > Should i log a bug for these issues? If I were you, I'd definitely file a bug reports against nfs-utils for: 1) the creation of "/var/lib/nfs/v4recovery/" when systemd is pid 1; 2) the systemd unit compatible envvars.
Re: [gentoo-user] nfsv4 issues
On Fri, Jul 22, 2016 at 8:22 PM, Adam Carterwrote: Does "/var/lib/nfs/v4recovery/" exist? >>> >>> No >>> # ls /var/lib/nfs/ >>> etab export-lock rmtab rpc_pipefs sm sm.bak state xtab >> >> IIRC, it's needed to avoid this delay. I thought that I'd saved a url >> about this but I can't find it. >> >> Do you have a syslog message about "stable storage"? "man nfsdcltrack". > > There's no message about stable storage, but there's this; > kernel: [578030.628415] NFSD: the nfsdcld client tracking upcall will be > removed in 3.10. Please transition to using nfsdcltrack. It's from https://patchwork.kernel.org/patch/1730241/ > # which nfsdcltrack > which: no nfsdcltrack in > (/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/opt/bin:/usr/x86_64-pc-linux-gnu/gcc-bin/5.4.0:/usr/lib64/subversion/bin:/opt/vmware/bin) > # qlist nfs | grep nfsdcltrack > # It depends on the nfs-utils USE settings: # qlist -U nfs-utils net-fs/nfs-utils (libmount nfsdcld nfsidmap nfsv4 nfsv41) # qfile $(which nfsdcltrack) net-fs/nfs-utils (/sbin/nfsdcltrack) >> The openrc script has >> >> >> mkdir_nfsdirs() { >> local d >> for d in v4recovery v4root ; do >> d="/var/lib/nfs/${d}" >> [ ! -d "${d}" ] && mkdir -p "${d}" >> done >> } >> >> >> but systemd doesn't have anything equivalent. On RHEL and Ubuntu, >> "/var/lib/nfs/v4recovery/" is created at installation time. Perhaps >> the Gentoo ebuild should do the same or should ship a >> "/usr/lib/tmpfiles.d/var-lib-nfs.conf" to create it at boot if it >> doesn't exist. > > I've added the directory, and after restarting syslog now has new entries; > kernel: [912267.948883] NFSD: Using /var/lib/nfs/v4recovery as the NFSv4 > state recovery directory > kernel: NFSD: Using /var/lib/nfs/v4recovery as the NFSv4 state recovery > directory > > I will test shortly and report back - thanks! Good luck. You're welcome.
Re: [gentoo-user] nfsv4 issues
I've added the directory, and after restarting syslog now has new entries; > kernel: [912267.948883] NFSD: Using /var/lib/nfs/v4recovery as the NFSv4 > state recovery directory > kernel: NFSD: Using /var/lib/nfs/v4recovery as the NFSv4 state recovery > directory > > I will test shortly and report back - thanks! > Confirmed - this fixes the 30 second delay. Should i log a bug for these issues?
Re: [gentoo-user] nfsv4 issues
> >> I don't use systemd on Gentoo but for the nfs-utils upstream-shipped > >> systemd units that I think that Gentoo's using, you have to re-run > >> nfs-config.service - or run the script that it calls - in order to > >> update the "/run/sysconfig/nfs-utils" environment file that's sourced > >> by the nfs-server.service unit. > > > > In /usr/lib/systemd/system/nfs-server.service > > [Service] > > EnvironmentFile=/etc/conf.d/nfs > > Sorry. Looking at the ebuild, there's: > > > rm "${D}$(systemd_get_unitdir)"/nfs-config.service || die > sed -i -r \ > -e "/^EnvironmentFile=/s:=.*:=${EPREFIX}/etc/conf.d/nfs:" \ > -e '/^(After|Wants)=nfs-config.service$/d' \ > -e 's:/usr/sbin/rpc.statd:/sbin/rpc.statd:' \ > "${D}$(systemd_get_unitdir)"/* || die > > > so the upstream "nfs-config.service" waltz is avoided. > > But that means that the variables in "/etc/conf.d/nfs" aren't renamed. > So the openrc nfs script uses "${OPTS_RPC_NFSD}", which is defined, > and the systemd service uses "$RPCNFSDARGS", which isn't. > I've added $RPCNFSDARGS to /etc/conf.d/nfs, restarted, and the nproc setting works. > > >> Does "/var/lib/nfs/v4recovery/" exist? > > > > No > > # ls /var/lib/nfs/ > > etab export-lock rmtab rpc_pipefs sm sm.bak state xtab > > IIRC, it's needed to avoid this delay. I thought that I'd saved a url > about this but I can't find it. > > Do you have a syslog message about "stable storage"? "man nfsdcltrack". > There's no message about stable storage, but there's this; kernel: [578030.628415] NFSD: the nfsdcld client tracking upcall will be removed in 3.10. Please transition to using nfsdcltrack. # which nfsdcltrack which: no nfsdcltrack in (/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/opt/bin:/usr/x86_64-pc-linux-gnu/gcc-bin/5.4.0:/usr/lib64/subversion/bin:/opt/vmware/bin) # qlist nfs | grep nfsdcltrack # > The openrc script has > > > mkdir_nfsdirs() { > local d > for d in v4recovery v4root ; do > d="/var/lib/nfs/${d}" > [ ! -d "${d}" ] && mkdir -p "${d}" > done > } > > > but systemd doesn't have anything equivalent. On RHEL and Ubuntu, > "/var/lib/nfs/v4recovery/" is created at installation time. Perhaps > the Gentoo ebuild should do the same or should ship a > "/usr/lib/tmpfiles.d/var-lib-nfs.conf" to create it at boot if it > doesn't exist. > > I've added the directory, and after restarting syslog now has new entries; kernel: [912267.948883] NFSD: Using /var/lib/nfs/v4recovery as the NFSv4 state recovery directory kernel: NFSD: Using /var/lib/nfs/v4recovery as the NFSv4 state recovery directory I will test shortly and report back - thanks!
Re: [gentoo-user] nfsv4 issues
On Wed, Jul 20, 2016 at 10:51 PM, Adam Carterwrote: >> I don't use systemd on Gentoo but for the nfs-utils upstream-shipped >> systemd units that I think that Gentoo's using, you have to re-run >> nfs-config.service - or run the script that it calls - in order to >> update the "/run/sysconfig/nfs-utils" environment file that's sourced >> by the nfs-server.service unit. > > In /usr/lib/systemd/system/nfs-server.service > [Service] > EnvironmentFile=/etc/conf.d/nfs Sorry. Looking at the ebuild, there's: rm "${D}$(systemd_get_unitdir)"/nfs-config.service || die sed -i -r \ -e "/^EnvironmentFile=/s:=.*:=${EPREFIX}/etc/conf.d/nfs:" \ -e '/^(After|Wants)=nfs-config.service$/d' \ -e 's:/usr/sbin/rpc.statd:/sbin/rpc.statd:' \ "${D}$(systemd_get_unitdir)"/* || die so the upstream "nfs-config.service" waltz is avoided. But that means that the variables in "/etc/conf.d/nfs" aren't renamed. So the openrc nfs script uses "${OPTS_RPC_NFSD}", which is defined, and the systemd service uses "$RPCNFSDARGS", which isn't. >> Does "/var/lib/nfs/v4recovery/" exist? > > No > # ls /var/lib/nfs/ > etab export-lock rmtab rpc_pipefs sm sm.bak state xtab IIRC, it's needed to avoid this delay. I thought that I'd saved a url about this but I can't find it. Do you have a syslog message about "stable storage"? "man nfsdcltrack". The openrc script has mkdir_nfsdirs() { local d for d in v4recovery v4root ; do d="/var/lib/nfs/${d}" [ ! -d "${d}" ] && mkdir -p "${d}" done } but systemd doesn't have anything equivalent. On RHEL and Ubuntu, "/var/lib/nfs/v4recovery/" is created at installation time. Perhaps the Gentoo ebuild should do the same or should ship a "/usr/lib/tmpfiles.d/var-lib-nfs.conf" to create it at boot if it doesn't exist.
Re: [gentoo-user] nfsv4 issues
Are the nfsd versions that you're setting being respected? You can > check with "rpcinfo -s" or "cat /proc/fs/nfsd/versions". > Yep; # cat /proc/fs/nfsd/versions -2 +3 +4 +4.1 +4.2 > You can change the number of threads on the fly with "echo 1 > > /proc/fs/nfsd/threads". > That works too, but then; # ps -ef | grep nfsd root 1454 1426 0 12:47 pts/000:00:00 grep --colour=auto nfsd root 23546 2 0 Jul19 ?00:00:00 [nfsd4_callbacks] root 23548 2 0 Jul19 ?00:00:00 [nfsd] # strace -p 23548 strace: attach: ptrace(PTRACE_ATTACH, ...): Operation not permitted > I don't use systemd on Gentoo but for the nfs-utils upstream-shipped > systemd units that I think that Gentoo's using, you have to re-run > nfs-config.service - or run the script that it calls - in order to > update the "/run/sysconfig/nfs-utils" environment file that's sourced > by the nfs-server.service unit. > In /usr/lib/systemd/system/nfs-server.service [Service] EnvironmentFile=/etc/conf.d/nfs Does "/var/lib/nfs/v4recovery/" exist? > > No # ls /var/lib/nfs/ etab export-lock rmtab rpc_pipefs sm sm.bak state xtab Does adding the client to "/etc/hosts" - or to your reverse dns zone - > eliminate the delay? > > DNS is setup and both client and server can forward and reverse lookup each other.
Re: [gentoo-user] nfsv4 issues
On Tue, Jul 19, 2016 at 12:49 AM, Adam Carterwrote: > > I'm trying to troubleshoot a newly setup nfs server, which, sometimes has a > 30 second pause (tcpdump shows its server waiting). > > # time touch /usr/portage/distfiles/testfile > > real0m30.088s > user0m0.000s > sys 0m0.001s > > I cant see anything in the nfs server debugging so i want to strace nfsd. > First i tell it to use single thread so i know i;m stracing the correct > thread, but; > > # grep OPTS_RPC_NFSD nfs > #OPTS_RPC_NFSD="8 -N2 -V 3 -V 4 -V 4.1" > OPTS_RPC_NFSD="1 -N2 -V 3 -V 4 -V 4.1" > # systemctl restart nfs-server > # pgrep -lf nfsd > 23546 nfsd4_callbacks > 23548 nfsd > 23549 nfsd > 23550 nfsd > 23551 nfsd > 23552 nfsd > 23553 nfsd > 23554 nfsd > 23555 nfsd > > So its not respecting the nproc setting. Any ideas? I also tried changing > EXPORTFS_TIMEOUT= since its currently set at 30. It didnt help, but perhaps > that's because its being ignored too. Are the nfsd versions that you're setting being respected? You can check with "rpcinfo -s" or "cat /proc/fs/nfsd/versions". You can change the number of threads on the fly with "echo 1 > /proc/fs/nfsd/threads". I don't use systemd on Gentoo but for the nfs-utils upstream-shipped systemd units that I think that Gentoo's using, you have to re-run nfs-config.service - or run the script that it calls - in order to update the "/run/sysconfig/nfs-utils" environment file that's sourced by the nfs-server.service unit. Does "/var/lib/nfs/v4recovery/" exist? Does adding the client to "/etc/hosts" - or to your reverse dns zone - eliminate the delay?
[gentoo-user] nfsv4 issues
I'm trying to troubleshoot a newly setup nfs server, which, sometimes has a 30 second pause (tcpdump shows its server waiting). # time touch /usr/portage/distfiles/testfile real0m30.088s user0m0.000s sys 0m0.001s I cant see anything in the nfs server debugging so i want to strace nfsd. First i tell it to use single thread so i know i;m stracing the correct thread, but; # grep OPTS_RPC_NFSD nfs #OPTS_RPC_NFSD="8 -N2 -V 3 -V 4 -V 4.1" OPTS_RPC_NFSD="1 -N2 -V 3 -V 4 -V 4.1" # systemctl restart nfs-server # pgrep -lf nfsd 23546 nfsd4_callbacks 23548 nfsd 23549 nfsd 23550 nfsd 23551 nfsd 23552 nfsd 23553 nfsd 23554 nfsd 23555 nfsd So its not respecting the nproc setting. Any ideas? I also tried changing EXPORTFS_TIMEOUT= since its currently set at 30. It didnt help, but perhaps that's because its being ignored too.