Re: nfs lockd errors after NetApp software upgrade.
Richard P Mackerras wrote: >Hi, > >We had some bully type workloads emerge when we moved a lot of block >storage from old XIV to new all flash 3PAR. I wonder if your IMAP issue >might have emerged just because suddenly there was the opportunity with all >flash. QOS is good on 9.x ONTAP. If anyone says it’s not then they last >looked on 8.x. So I suggest you QOS the IMAP workload. > > Nobody should be using UDP with NFS unless they have a very specific set >of circumstances. TCP was a real step forward. Well, I can't argue with this, considering I did the first working implementation of NFS over TCP. It was actually Mike Karels that suggested I try doing so, There's a paper in a very old Usenix Conference Proceedings, but it is so old that it isn't on the Usenix web page (around 1988 in Denver, if I recall). I don't even have a copy myself, although I was the author. Now, having said that, I must note that the Network Lock Manager (NLM) and Network Status Monitor (NSM) were not NFS. They were separate stateful protocols (poorly designed imho) that Sun never published. NFS as Sun designed it (NFSv2 and NFSv3) were "stateless server" protocols, so that they could work reliably without server crash recovery. However, the NLM was inherently stateful, since it was dealing with file locks. So, you can't really lump the NLM with NFS (and you should avoid use of the NLM over any transport imho). NFSv4 tackled the difficult problem of having a "stateful server" and crash recovery, which resulted in a much more complex protocol (compare the size of RFC-1813 vs RFC-5661 to get some idea of this). rick Cheers Richard ___ freebsd-stable@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org" ___ freebsd-stable@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
Re: nfs lockd errors after NetApp software upgrade.
Hi, We had some bully type workloads emerge when we moved a lot of block storage from old XIV to new all flash 3PAR. I wonder if your IMAP issue might have emerged just because suddenly there was the opportunity with all flash. QOS is good on 9.x ONTAP. If anyone says it’s not then they last looked on 8.x. So I suggest you QOS the IMAP workload. Nobody should be using UDP with NFS unless they have a very specific set of circumstances. TCP was a real step forward. Cheers Richard ___ freebsd-stable@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
Re: nfs lockd errors after NetApp software upgrade.
On 12/22/19 12:01 PM, Rick Macklem wrote:
> Well, I've noted the flawed protocol. Here's an example (from my limited
> understanding of these protocols, where there has never been a published
> spec) :
> - The NLM supports a "blocking lock request" that goes something like this...
>- client requests lock and is willing to wait for it
>- if server has a conflicting lock on the file, it replies "I'll acquire
> the lock for
> you when I can and let you know".
> --> When the conflicting lock is released, the server acquires the lock
> and does
> a callback (server->client RPC) to tell the client it now has the
> lock.
> You don't have to think about this for long to realize that any network
> unreliability
> or partitioning could result in trouble.
> The kernel RPC layer may do some retries of the RPCs (this is controlled by
> the
> parameters set for the RPC), but at some point the protocol asks the NSM
> (rpc.statd) if the machine is "up" and then uses the NSM's answer to deal
> with it.
> (The NSM basically pokes other systems and notes they are "up" if they get
> replies to these pokes. It uses IP broadcast at some point.)
>
> Now, maybe switching to TCP will make the RPCs reliable enough that it will
> work, or maybe it won't? (It certainly sounds like the Netapp upgrade is
> causing
> some kind of network issue, and the NLM doesn't tolerate that well.)
>
> rick
tl;dr I think netapp effectively nerfed UDP lockd performance in newer
versions, maybe cluster mode.
>From my very un-fun experience after migrating our volumes off an older
netapp onto a new netapp with flash drives (plenty fast) running Ontap
9.x ("cluster mode"), our typical IO load from idle time IMAP
connections was enough to overwhelm the new netapp and drive performance
into the ground. The same IO that was perfectly fine on the old netapp.
Going into a workday in this state was absolutely not possible. I opened
a high priority ticket with netapp, didn't really get anywhere that very
long day and settled on nolockd so I could go home and sleep. Both my
hunch later and netapp support suggested switching lockd traffic to TCP
even though I had no network problems (the old netapp was fine). I think
I still run into occasional load issues but the newer netapp OS seemed
way more capable of this load when using TCP lockd. Of course they also
suggested switching to nfsv4 but I could not seriously entertain
validating that type of change for production in less than a day.
___
freebsd-stable@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
Re: Access to NETMAP from c++ program
On Mon, Nov 25, 2019 at 03:36:21PM -0500, Ryan Stone wrote:
> Remove "using namespace std;" from your program.
I am don't have "using namespace std;".
Example:
===
#include
#include
#include
#include
#include
#include
#include
#include
#include
#include
#include
===
Yes, only includes.
c++ -c tatomic.cc
In file included from tatomic.cc:11:
In file included from /usr/include/net/netmap_user.h:104:
In file included from /usr/include/net/netmap.h:816:
/usr/include/stdatomic.h:186:17: error: unknown type name '_Bool'
typedef _Atomic(_Bool) atomic_bool;
^
/usr/include/stdatomic.h:186:26: error: C++ requires a type specifier for all
declarations
typedef _Atomic(_Bool) atomic_bool;
~~ ^
/usr/include/stdatomic.h:379:17: error: unknown type name '_Bool'
static __inline _Bool
^
/usr/include/stdatomic.h:383:10: error: address argument to atomic operation
must be a pointer to _Atomic type ('volatile atomic_bool *' (aka 'volatile int
*') invalid)
return (atomic_exchange_explicit(&__object->__flag, 1, __order));
^~
/usr/include/stdatomic.h:242:2: note: expanded from macro
'atomic_exchange_explicit'
__c11_atomic_exchange(object, desired, order)
^ ~~
/usr/include/stdatomic.h:390:2: error: address argument to atomic operation
must be a pointer to _Atomic type ('volatile atomic_bool *' (aka 'volatile int
*') invalid)
atomic_store_explicit(&__object->__flag, 0, __order);
^ ~
/usr/include/stdatomic.h:256:2: note: expanded from macro
'atomic_store_explicit'
__c11_atomic_store(object, desired, order)
^ ~~
/usr/include/stdatomic.h:394:17: error: unknown type name '_Bool'
static __inline _Bool
^
6 errors generated.
Ok, try ugly hack for _Bool:
===
#include
#include
#include
#include
#include
#include
#include
#include
#include
#include
typedefint _Bool;
#include
===
No, errors. Now try
===
#include
#include
#include
#include
#include
#include
#include
#include
#include
#include
typedefint _Bool;
#include
#include
#include
===
Many errors:
In file included from tatomic.cc:13:
In file included from /usr/include/c++/v1/memory:668:
/usr/include/c++/v1/atomic:1166:49: error: expected ')'
atomic_is_lock_free(const volatile atomic<_Tp>* __o) _NOEXCEPT
^
/usr/include/c++/v1/atomic:1166:1: note: to match this '('
atomic_is_lock_free(const volatile atomic<_Tp>* __o) _NOEXCEPT
^
/usr/include/stdatomic.h:176:32: note: expanded from macro 'atomic_is_lock_free'
__atomic_is_lock_free(sizeof(*(obj)), obj)
^
In file included from tatomic.cc:13:
In file included from /usr/include/c++/v1/memory:668:
/usr/include/c++/v1/atomic:1166:1: error: expected expression
atomic_is_lock_free(const volatile atomic<_Tp>* __o) _NOEXCEPT
^
/usr/include/stdatomic.h:176:37: note: expanded from macro 'atomic_is_lock_free'
__atomic_is_lock_free(sizeof(*(obj)), obj)
^
In file included from tatomic.cc:13:
In file included from /usr/include/c++/v1/memory:668:
/usr/include/c++/v1/atomic:1166:21: error: expected expression
atomic_is_lock_free(const volatile atomic<_Tp>* __o) _NOEXCEPT
^
/usr/include/c++/v1/atomic:1166:53: error: expected ';' at end of declaration
atomic_is_lock_free(const volatile atomic<_Tp>* __o) _NOEXCEPT
^
/usr/include/c++/v1/atomic:1166:54: error: expected unqualified-id
atomic_is_lock_free(const volatile atomic<_Tp>* __o) _NOEXCEPT
^
/usr/include/c++/v1/__config:839:21: note: expanded from macro '_NOEXCEPT'
# define _NOEXCEPT noexcept
^
In file included from tatomic.cc:13:
In file included from /usr/include/c++/v1/memory:668:
/usr/include/c++/v1/atomic:1174:1: error: redefinition of
'__atomic_is_lock_free'
atomic_is_lock_free(const atomic<_Tp>* __o) _NOEXCEPT
^
/usr/include/stdatomic.h:176:2: note: expanded from macro 'atomic_is_lock_free'
__atomic_is_lock_free(sizeof(*(obj)), obj)
^
/usr/include/c++/v1/atomic:1166:1: note: previous definition is here
atomic_is_lock_free(const volatile atomic<_Tp>* __o) _NOEXCEPT
^
/usr/include/stdatomic.h:176:2: note: expanded from macro 'atomic_is_lock_free'
__atomic_is_lock_free(sizeof(*(obj)), obj)
^
In file included from tatomic.cc:13:
In file included from /usr/include/c++/v1/memory:668:
/usr/include/c++/v1/atomic:1174:40: error: expected ')'
atomic_is_lock_free(const atomic<_Tp>* __o) _NOEXCEPT
^
/usr/include/c++/v1/atomic:1174:1: note: to match this '('
atomic_is_lock_fre
