[bookkeeper] branch branch-4.8 updated: Issue #1791: Read Submission should bypass OSE Threads

[sijie]

This is an automated email from the ASF dual-hosted git repository.

sijie pushed a commit to branch branch-4.8
in repository https://gitbox.apache.org/repos/asf/bookkeeper.git


The following commit(s) were added to refs/heads/branch-4.8 by this push:
     new bb22ad4  Issue #1791: Read Submission should bypass OSE Threads
bb22ad4 is described below

commit bb22ad409bdd0799259af307cbb72fd7521f902f
Author: Nicolas Michael <g...@nmichael.de>
AuthorDate: Thu Nov 8 08:51:10 2018 -0800

    Issue #1791: Read Submission should bypass OSE Threads
    
    Profiling of our Bookkeeper Client code for read requests shows that client 
threads spend half of their time in dispatching requests to OrderedExecutors 
(just the dispatch itself, not the execution inside OSE): 54% of their CPU time 
is spent in OrderedExecutor.executeOrdered() (called by 
LedgerHandle.readEntriesInternalAsync()). The high time spend in request 
submission to OSE is largely caused by Linux scheduling cost, that is the cost 
of dispatching the OSE thread to CPU: 42% of tot [...]
    
    This change executes read submissions (PendingReadOp) on read-only ledger 
handles directly inside the client thread instead of submitting them to Ordered 
Executors.
    
    Tests with a prototype have shown significant improvements in both overall 
CPU consumption as well as read latency. The additional work client threads 
have to do (the dispatch of the read requests to netty) is roughly the same as 
the (saved) dispatch cost to OSE, so the change turns out to be neutral for CPU 
consumption of client threads. In some experiments, the savings even exceed the 
additional work, and client threads consume less cpu even though they "do 
more". It also frees up l [...]
    
    Master Issue: #1791: Read Submission should bypass OSE Threads
    
    Reviewers: Enrico Olivelli <eolive...@gmail.com>, Andrey Yegorov <None>, 
Sijie Guo <si...@apache.org>, Matteo Merli <mme...@apache.org>
    
    This closes #1792 from nicmichael/DirectRead, closes #1791
---
 .../src/main/java/org/apache/bookkeeper/client/LedgerHandle.java | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)

diff --git 
a/bookkeeper-server/src/main/java/org/apache/bookkeeper/client/LedgerHandle.java
 
b/bookkeeper-server/src/main/java/org/apache/bookkeeper/client/LedgerHandle.java
index 6894351..64e0817 100644
--- 
a/bookkeeper-server/src/main/java/org/apache/bookkeeper/client/LedgerHandle.java
+++ 
b/bookkeeper-server/src/main/java/org/apache/bookkeeper/client/LedgerHandle.java
@@ -917,7 +917,14 @@ public class LedgerHandle implements WriteHandle {
                 ws.recycle();
             }
 
-            bk.getMainWorkerPool().executeOrdered(ledgerId, op);
+            if (isHandleWritable()) {
+                // Ledger handle in read/write mode: submit to OSE for ordered 
execution.
+                bk.getMainWorkerPool().executeOrdered(ledgerId, op);
+            } else {
+                // Read-only ledger handle: bypass OSE and execute read 
directly in client thread.
+                // This avoids a context-switch to OSE thread and thus reduces 
latency.
+                op.run();
+            }
         } else {
             
op.future().completeExceptionally(BKException.create(ClientClosedException));
         }