Hi Bill, There has not been griping. Malahal has done performance measurement.
IIUC (per IRC) Malahal: 1. has empirical evidence that moving the current Ganesha -dispatch queue- bands into lanes measurably improves throughput, when the number of worker threads is large (apparently, 64 and (!) 256) That is expected, so I'm looking to Malahal to send a change (and some other tuning changes) for review. 2. Malahal indicated he found a bug of some kind in the thread fridge, provoked by the shutdown check in the current dispatch queue code, which he says he fixed, so if he hasn't already sent a change, I'm expecting to see one soon which addresses this. 3. Malahal described in IRC an additional change he made to split the ntirpc output ioq into lanes, and believed he saw improvement (as of ~2 weeks ago), but was still benchmarking in order to split out the impact of this change relative to others. I'm surprised this is helpful (but, it would empirically disprove bottom-half serialization by xprt, if true--oh, the measured improvement from reduced xprt->lock contention also seems to disprove it). 4. As I've indicated in discussion (2-3 occasions) with you here, it's been in our medium-term plan to move all of the current dispatch operations back into a new ntirpc run-loop, replacing legacy svc_run(). I consider that completely orthogonal to Ganesha having it's own thread pool. As well, it's very likely that the full change we should be looking at would be using platform async i/o for (at least) all recv and likely send paths. That will require work on decoders and is big enough that I don't see it coming in 2.3, unless RDMA starts working great first. Matt -- Matt Benjamin Red Hat, Inc. 315 West Huron Street, Suite 140A Ann Arbor, Michigan 48103 http://www.redhat.com/en/technologies/storage tel. 734-761-4689 fax. 734-769-8938 cel. 734-216-5309 ----- Original Message ----- > From: "William Allen Simpson" <william.allen.simp...@gmail.com> > To: "NFS Ganesha Developers" <nfs-ganesha-devel@lists.sourceforge.net> > Sent: Wednesday, September 2, 2015 10:35:38 AM > Subject: [Nfs-ganesha-devel] Fridge worker pool to ntirpc? > > There has been some griping (that hasn't yet appeared on > this mailing list) that the ntirpc worker pool is too slow > processing TCP output. But no specific proof. > > There are several possible reasons for that, including that > the work model there would be better per interface, rather > than per connection. I've mentioned fair queuing before. > > I've made a lock-free variant for RDMA, but for TCP that > requires combining the fridge thread pool with the ntirpc > pool. I've had code for that sitting around since May, > although it's rather bit-rotten in the meantime. > > I've been assuming you don't want it until the new RDMA? > > ------------------------------------------------------------------------------ > Monitor Your Dynamic Infrastructure at Any Scale With Datadog! > Get real-time metrics from all of your servers, apps and tools > in one place. > SourceForge users - Click here to start your Free Trial of Datadog now! > http://pubads.g.doubleclick.net/gampad/clk?id=241902991&iu=/4140 > _______________________________________________ > Nfs-ganesha-devel mailing list > Nfs-ganesha-devel@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/nfs-ganesha-devel > ------------------------------------------------------------------------------ Monitor Your Dynamic Infrastructure at Any Scale With Datadog! Get real-time metrics from all of your servers, apps and tools in one place. SourceForge users - Click here to start your Free Trial of Datadog now! http://pubads.g.doubleclick.net/gampad/clk?id=241902991&iu=/4140 _______________________________________________ Nfs-ganesha-devel mailing list Nfs-ganesha-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/nfs-ganesha-devel
