Re: nfs lockd errors after NetApp software upgrade.

2020-01-09 Thread Daniel Braniss
> On 9 Jan 2020, at 05:24, Rick Macklem wrote: > > The attached patch changes the xid to be a global for all "connections" for > the krpc UDP client. > > You could try it if you'd like. It passed a trivial test, but I don't know why > there is that "misfeature" comment means, so I don't know

Re: nfs lockd errors after NetApp software upgrade.

2020-01-08 Thread Rick Macklem
sday, January 8, 2020 12:08 PM To: Rick Macklem Cc: Richard P Mackerras; Adam McDougall; freebsd-stable@freebsd.org Subject: Re: nfs lockd errors after NetApp software upgrade. top posting NetAPP reply: … Here you can see transaction ID (0x5e15f77a) being used over port 886 and the NF

Re: nfs lockd errors after NetApp software upgrade.

2020-01-08 Thread Rick Macklem
rick From: Daniel Braniss Sent: Wednesday, January 8, 2020 12:08 PM To: Rick Macklem Cc: Richard P Mackerras; Adam McDougall; freebsd-stable@freebsd.org Subject: Re: nfs lockd errors after NetApp software upgrade. top posting NetAPP reply: … Here you can see

Re: nfs lockd errors after NetApp software upgrade.

2020-01-08 Thread Rick Macklem
d-stable@freebsd.org Subject: Re: nfs lockd errors after NetApp software upgrade. top posting NetAPP reply: … Here you can see transaction ID (0x5e15f77a) being used over port 886 and the NFS server successfully responds. 44806952020-01-08 12:20:54 132.65.116.111 13

Re: nfs lockd errors after NetApp software upgrade.

2020-01-08 Thread Daniel Braniss
top posting NetAPP reply: … Here you can see transaction ID (0x5e15f77a) being used over port 886 and the NFS server successfully responds. 44806952020-01-08 12:20:54 132.65.116.111 132.65.60.56 NLM 0x5e15f77a (1578497914) 886

Re: nfs lockd errors after NetApp software upgrade.

2019-12-23 Thread Rick Macklem
Richard P Mackerras wrote: >Hi, > >We had some bully type workloads emerge when we moved a lot of block >storage from old XIV to new all flash 3PAR. I wonder if your IMAP issue >might have emerged just because suddenly there was the opportunity with all >flash. QOS is good on 9.x ONTAP. If anyone

Re: nfs lockd errors after NetApp software upgrade.

2019-12-23 Thread Richard P Mackerras
Hi, We had some bully type workloads emerge when we moved a lot of block storage from old XIV to new all flash 3PAR. I wonder if your IMAP issue might have emerged just because suddenly there was the opportunity with all flash. QOS is good on 9.x ONTAP. If anyone says it’s not then they last

Re: nfs lockd errors after NetApp software upgrade.

2019-12-23 Thread Adam McDougall
On 12/22/19 12:01 PM, Rick Macklem wrote: > Well, I've noted the flawed protocol. Here's an example (from my limited > understanding of these protocols, where there has never been a published > spec) : > - The NLM supports a "blocking lock request" that goes something like this... >- client

Re: nfs lockd errors after NetApp software upgrade.

2019-12-22 Thread Rick Macklem
Daniel Braniss wrote: >> On 21 Dec 2019, at 19:32, Rick Macklem wrote: >> >> Daniel Braniss wrote: On 20 Dec 2019, at 19:19, Rick Macklem >>>mailto:rmack...@uoguelph.ca>> wrote: Adam McDougall wrote: > Try changing bool_t do_tcp = FALSE; to TRUE in >

Re: nfs lockd errors after NetApp software upgrade.

2019-12-21 Thread Daniel Braniss
> On 21 Dec 2019, at 19:32, Rick Macklem wrote: > > Daniel Braniss wrote: >>> On 20 Dec 2019, at 19:19, Rick Macklem >>> >>mailto:rmack...@uoguelph.ca>> wrote: >>> >>> Adam McDougall wrote: Try changing bool_t do_tcp = FALSE; to TRUE in /usr/src/sys/nlm/nlm_prot_impl.c, recompile

Re: nfs lockd errors after NetApp software upgrade.

2019-12-21 Thread Rick Macklem
Daniel Braniss wrote: >>On 20 Dec 2019, at 19:19, Rick Macklem mailto:rmack...@uoguelph.ca>> wrote: >> >>Adam McDougall wrote: >>>Try changing bool_t do_tcp = FALSE; to TRUE in >>>/usr/src/sys/nlm/nlm_prot_impl.c, recompile the kernel and try again. I >>>think this makes it match Linux client

Re: nfs lockd errors after NetApp software upgrade.

2019-12-20 Thread Daniel Braniss
> On 20 Dec 2019, at 19:19, Rick Macklem wrote: > > Adam McDougall wrote: >> Try changing bool_t do_tcp = FALSE; to TRUE in >> /usr/src/sys/nlm/nlm_prot_impl.c, recompile the kernel and try again. I >> think this makes it match Linux client behavior. I suspect I ran into >> the same issue as

Re: nfs lockd errors after NetApp software upgrade.

2019-12-20 Thread Rick Macklem
Adam McDougall wrote: >Try changing bool_t do_tcp = FALSE; to TRUE in >/usr/src/sys/nlm/nlm_prot_impl.c, recompile the kernel and try again. I >think this makes it match Linux client behavior. I suspect I ran into >the same issue as you. I do think I used nolockd is a workaround >temporarily. I

Re: nfs lockd errors after NetApp software upgrade.

2019-12-19 Thread Adam McDougall
Try changing bool_t do_tcp = FALSE; to TRUE in /usr/src/sys/nlm/nlm_prot_impl.c, recompile the kernel and try again. I think this makes it match Linux client behavior. I suspect I ran into the same issue as you. I do think I used nolockd is a workaround temporarily. I can provide some more details

Re: nfs lockd errors after NetApp software upgrade.

2019-12-19 Thread Richard P Mackerras
Hi, At ONTAP 9.3P6 there is a possible LACP group issue after upgrade. Have you checked any LACP groups, These should not be a problem but I assume network interfaces are at the home ports, not on slower ports or something silly. It is marginally better if the traffic goes direct to the node where

Re: nfs lockd errors after NetApp software upgrade.

2019-12-19 Thread Daniel Braniss
> On 19 Dec 2019, at 16:09, Rick Macklem wrote: > > Daniel Braniss wrote: > [stuff snipped] >> all mounts are nfsv3/tcp > This doesn't affect what the NLM code (rpc.lockd) uses. I honestly don't know > when > the NLM uses tcp vs udp. I think rpc.statd still uses IP broadcast at times. can the

Re: nfs lockd errors after NetApp software upgrade.

2019-12-19 Thread Rick Macklem
Daniel Braniss wrote: [stuff snipped] >all mounts are nfsv3/tcp This doesn't affect what the NLM code (rpc.lockd) uses. I honestly don't know when the NLM uses tcp vs udp. I think rpc.statd still uses IP broadcast at times. To me, it looks like a network configuration issue. You could capture

Re: nfs lockd errors after NetApp software upgrade.

2019-12-19 Thread Daniel Braniss
> On 19 Dec 2019, at 02:22, Rick Macklem wrote: > > Richard P Mackerras wrote: > >> Hi, >> What software version is the NetApp using? >> Is the exported volume big? >> Is the vserver configured for 64bit identifiers? >> >> If you enable NFS V4.0 or 4.1 other NFS clients using defaults might

Re: nfs lockd errors after NetApp software upgrade.

2019-12-18 Thread Rick Macklem
Richard P Mackerras wrote: >Hi, >What software version is the NetApp using? >Is the exported volume big? >Is the vserver configured for 64bit identifiers? > >If you enable NFS V4.0 or 4.1 other NFS clients using defaults might mount >NFSv4.x >unexpectedly after a reboot so you need to watch

Re: nfs lockd errors after NetApp software upgrade.

2019-12-18 Thread Richard P Mackerras
Hi, I’m sure the 64 bit identifiers isn’t an issue. Your export isn’t vast. I assume you have restarted statd and lockd on FreeBSD. I did search on the NetApp site earlier and nothing lept out then. Sorry, Richard On Wed, 18 Dec 2019 at 16:06, Daniel Braniss wrote: > > > On 18 Dec 2019, at

Re: nfs lockd errors after NetApp software upgrade.

2019-12-18 Thread Daniel Braniss
> On 18 Dec 2019, at 17:58, Richard P Mackerras wrote: > > Hi, > What software version is the NetApp using? the very latest :-), but will try and find out later. > Is the exported volume big? about 500G, but many files as far as I know, only accessed by one host running the web app - moodle.

Re: nfs lockd errors after NetApp software upgrade.

2019-12-18 Thread Richard P Mackerras
Hi, What software version is the NetApp using? Is the exported volume big? Is the vserver configured for 64bit identifiers? If you enable NFS V4.0 or 4.1 other NFS clients using defaults might mount NFSv4.x unexpectedly after a reboot so you need to watch that. Cheers Richard (NetApp admin) On

Re: nfs lockd errors after NetApp software upgrade.

2019-12-18 Thread Daniel Braniss
> On 18 Dec 2019, at 16:55, Rick Macklem wrote: > > Daniel Braniss wrote: > >> Hi, >> The server with the problems is running FreeBSD 11.1 stable, it was working >> fine for >several months, >> but after a software upgrade of our NetAPP server it’s reporting many lockd >> errors >and

Re: nfs lockd errors after NetApp software upgrade.

2019-12-18 Thread Rick Macklem
Daniel Braniss wrote: >Hi, >The server with the problems is running FreeBSD 11.1 stable, it was working >fine for >several months, >but after a software upgrade of our NetAPP server it’s reporting many lockd >errors >and becomes catatonic, >... >Dec 18 13:11:02 moo-09 kernel: nfs server

nfs lockd errors after NetApp software upgrade.

2019-12-18 Thread Daniel Braniss
Hi, The server with the problems is running FreeBSD 11.1 stable, it was working fine for several months, but after a software upgrade of our NetAPP server it’s reporting many lockd errors and becomes catatonic, ... Dec 18 13:11:02 moo-09 kernel: nfs server fr-06:/web/www: lockd not responding