[bookkeeper] branch master updated: Issue #1791: Read Submission should bypass OSE Threads

[sijie]

This is an automated email from the ASF dual-hosted git repository.

sijie pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/bookkeeper.git


The following commit(s) were added to refs/heads/master by this push:
     new 6b99ff7  Issue #1791: Read Submission should bypass OSE Threads
6b99ff7 is described below

commit 6b99ff73278d4e655509f61ea44284e54716f5a8
Author: Nicolas Michael <g...@nmichael.de>
AuthorDate: Thu Nov 8 08:51:10 2018 -0800

    Issue #1791: Read Submission should bypass OSE Threads
    
    ### Motivation
    
    Profiling of our Bookkeeper Client code for read requests shows that client 
threads spend half of their time in dispatching requests to OrderedExecutors 
(just the dispatch itself, not the execution inside OSE): 54% of their CPU time 
is spent in OrderedExecutor.executeOrdered() (called by 
LedgerHandle.readEntriesInternalAsync()). The high time spend in request 
submission to OSE is largely caused by Linux scheduling cost, that is the cost 
of dispatching the OSE thread to CPU: 42% of tot [...]
    
    ### Changes
    
    This change executes read submissions (PendingReadOp) on read-only ledger 
handles directly inside the client thread instead of submitting them to Ordered 
Executors.
    
    Tests with a prototype have shown significant improvements in both overall 
CPU consumption as well as read latency. The additional work client threads 
have to do (the dispatch of the read requests to netty) is roughly the same as 
the (saved) dispatch cost to OSE, so the change turns out to be neutral for CPU 
consumption of client threads. In some experiments, the savings even exceed the 
additional work, and client threads consume less cpu even though they "do 
more". It also frees up l [...]
    
    Master Issue: #1791: Read Submission should bypass OSE Threads
    
    
    Reviewers: Enrico Olivelli <eolive...@gmail.com>, Andrey Yegorov <None>, 
Sijie Guo <si...@apache.org>, Matteo Merli <mme...@apache.org>
    
    This closes #1792 from nicmichael/DirectRead, closes #1791
---
 .../src/main/java/org/apache/bookkeeper/client/LedgerHandle.java | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)

diff --git 
a/bookkeeper-server/src/main/java/org/apache/bookkeeper/client/LedgerHandle.java
 
b/bookkeeper-server/src/main/java/org/apache/bookkeeper/client/LedgerHandle.java
index 6125a3d..12e00e8 100644
--- 
a/bookkeeper-server/src/main/java/org/apache/bookkeeper/client/LedgerHandle.java
+++ 
b/bookkeeper-server/src/main/java/org/apache/bookkeeper/client/LedgerHandle.java
@@ -840,7 +840,14 @@ public class LedgerHandle implements WriteHandle {
                 ws.recycle();
             }
 
-            clientCtx.getMainWorkerPool().executeOrdered(ledgerId, op);
+            if (isHandleWritable()) {
+                // Ledger handle in read/write mode: submit to OSE for ordered 
execution.
+                clientCtx.getMainWorkerPool().executeOrdered(ledgerId, op);
+            } else {
+                // Read-only ledger handle: bypass OSE and execute read 
directly in client thread.
+                // This avoids a context-switch to OSE thread and thus reduces 
latency.
+                op.run();
+            }
         } else {
             
op.future().completeExceptionally(BKException.create(ClientClosedException));
         }