[GitHub] [pulsar] merlimat opened a new pull request #13043: Transparent batching of ZK operations

[GitBox]

merlimat opened a new pull request #13043:
URL: https://github.com/apache/pulsar/pull/13043


   ### Motivation
   
   It is more efficient to do operations to ZK in batch, using the 
`ZooKeeper.multi()` operation instead of individual calls. 
   This reduces the RPC traffic between ZK client and servers and it also 
reduces the number of write transactions, since each `multi()` correspond to a 
single ZK transaction, containing multiple write operations.
   
   ### Modifications
   
    * Added a base class `AbstractBatchedMetadataStore` which contains the 
batching logic
    * Added configurable batching parameters
    * Using 2 different transaction sets, one for read operations and the other 
just for write operations
    * Added code of (very) basic benchmark
   
   ### Preliminary results
   
   This is done with the included basic benchmark, making request to the 
embedded ZK running on a laptop:
   
   <img width="606" alt="image" 
src="https://user-images.githubusercontent.com/62500/144008184-03df9da4-f82b-423b-b817-db3c472b7fda.png";>
   
   <img width="606" alt="image" 
src="https://user-images.githubusercontent.com/62500/144008352-615506c9-c611-4a74-852b-8057f6076f91.png";>
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org