[
https://issues.apache.org/jira/browse/HBASE-10659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13919818#comment-13919818
]
Liyin Tang commented on HBASE-10659:
------------------------------------
1) IPC writer thread will do all the sanity check as a preparation, such as
figure out which Region and whether it is enabled.
2) IPC writer thread will handoff the Put request, and then start to process
next IPC request. It won't block or wait for the current Put request to finish.
The responder thread will finally return the call to the clients.
3) One HLogSyncer per WAL, and each HLogSyncer has its own concurrent
collections to swap between.
4) I don' fully understand your last question. Since the HLogSyncer thread has
already one the sequencing for each transaction, memstore-update-thread could
just reuse the same sequence id as MVCC.
The basic motivation of this new write path is to reduce the thread
interleaving and synchronizations in the critical write path as much as
possible.
> [89-fb] Optimize the threading model in HBase write path
> --------------------------------------------------------
>
> Key: HBASE-10659
> URL: https://issues.apache.org/jira/browse/HBASE-10659
> Project: HBase
> Issue Type: New Feature
> Reporter: Liyin Tang
>
> Recently, we have done multiple prototypes to optimize the HBase (0.89)write
> path. And based on the simulator results, the following model is able to
> achieve much higher overall throughput with less threads.
> IPC Writer Threads Pool:
> IPC handler threads will prepare all Put requests, and append the WALEdit, as
> one transaction, into a concurrent collection with a read lock. And then just
> return;
> HLogSyncer Thread:
> Each HLogSyncer thread is corresponding to one HLog stream. It swaps the
> concurrent collection with a write lock, and then iterate over all the
> elements in the previous concurrent collection, generate the sequence id for
> each transaction, and write to HLog. After the HLog sync is done, append
> these transactions as a batch into a blocking queue.
> Memstore Update Thread:
> The memstore update thread will poll the blocking queue and update the
> memstore for each transaction by using the sequence id as MVCC. Once the
> memstore update is done, dispatch to the responder thread pool to return to
> the client.
> Responder Thread Pool:
> Responder thread pool will return the RPC call in parallel.
> We are still evaluating this model and will share more results/numbers once
> it is ready. But really appreciate any comments in advance !
--
This message was sent by Atlassian JIRA
(v6.2#6252)
