Ganesha currently has 2 phases of dispatch queuing: one for input and decoding, then another for executing/encoding output. (I've fixed the third queue for later sending the output, where the thread should stay hot as long as there's more to process.)
On Monday, Matt told me we were having problems with sequential IO. Wishing that somebody had mentioned this to me sooner. Many (20?) months ago, I'd put together a patch to napalm the second phase that splits into 4 queues: "mount", "call", "low", and "high". For tcp, these don't make much sense. We really want to keep things ordered. For udp, this could make things be processed out of order, and make sequential IO slower. 9P only uses "low", and RDMA bypasses them entirely. My old napalm patch doesn't work with my recently revised locking, but I'm sure that can be updated. What really floored me was looking at the input-side queuing! AFAICT, for tcp it just accepts the request and puts it on a queue that is probably never more than 1 deep, because it doesn't rearm the epoll until there is no more data to read. The OS should be holding any additional data. That should be OK? Maybe? Could cause some TCP stalls, though. But for udp, it puts 1 on the queue, and then drops everything on the floor until that one been "decoded" (put on the second queue), then takes another while the first is executed and encoded. Depending on thread switch latency, that could drop quite a few during the queue handoff. If too many are dropped, it doesn't rearm for 5 ms -- a lifetime! So I can see how udp would be having terrible problems with sequential IO, because back-to-back or parallel requests and re-transmissions will be handled in essentially random order. There must be a brilliant explanation for this design. I need enlightenment.... ------------------------------------------------------------------------------ Announcing the Oxford Dictionaries API! The API offers world-renowned dictionary content that is easy and intuitive to access. Sign up for an account today to start using our lexical data to power your apps and projects. Get started today and enter our developer competition. http://sdm.link/oxford _______________________________________________ Nfs-ganesha-devel mailing list Nfs-ganesha-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/nfs-ganesha-devel
