hangc0276 opened a new pull request, #3846: URL: https://github.com/apache/bookkeeper/pull/3846
### Motivation The bookie server process add-entry requests pipeline: - Get one request from the Netty socket channel - Choose one thread to process the written request - Write the entry into the target ledger entry logger's write cache(memory table) - Put the entry into the journal's pending queue - Journal thread takes the entry from the pending queue and writes it into PageCache/Journal Disk - Write the callback response to the Netty buffer and flush to the bookie client side. For every add entry request, the bookie server needs to go through the above steps one by one. It will introduce a lot of thread context switch. We can batch the add requests according to the Netty socket channel, and write a batch of entries into the ledger entry logger and journal disk. ### Modifications The PR will change the add requests processing pipeline into the following steps. - Get a batch of add-entry requests from the socket channel until the socket channel is empty or reached the max capacity (default is 10_000) - Choose one thread to process the batch of add-entry requests. - Write the entries into the target ledger entry logger's write cache one by one - Put the batch of entries into the journal's pending queue - Journal thread drain a batch of entries from the pending queue and write them into PageCache/Journal disk - Write the callback response to the Netty buffer and flush to the bookie client side. With this change, we can save a lot of thread context switch. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
