[
https://issues.apache.org/jira/browse/HBASE-5776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13253939#comment-13253939
]
Kannan Muthukkaruppan commented on HBASE-5776:
----------------------------------------------
Todd:
This is API is user-facing.
The HTables themselves aren't thread safe, so applications end up using a
separate HTable per thread, and the per-RS batching happens only within the
context of a single HTable request. The HTableMultiplexer was intended as a
simple wrapper helper/utility that allows the app to get "deferred put"
functionality, but with batching even the puts coming from different
application threads in most effective manner. It also supports puts to multiple
tables to be grouped in the same batch. Extending HTables which are not thread
safe for this use case didn't make sense, and also painful from backward compat
perspective. In this approach, a single instance of HTableMultiplexer can be
use by all the app threads to shove puts, and let the multiplexer group commits
per RS, and forward them along in batches. This non-blocking multiplexer API
could get more sophisticated in future, e.g., supporting async gets (with
callbacks and so on) and different config knobs (e.g., how much buffering
memory to allow for all queues, retry/failure handling logic via callbacks,
etc.). And maybe, in future, we can gradually deprecate the deferred put
functionality in HTable, and recommend all the non-blocking use cases to go via
the multiplexer.
> HTableMultiplexer
> ------------------
>
> Key: HBASE-5776
> URL: https://issues.apache.org/jira/browse/HBASE-5776
> Project: HBase
> Issue Type: Improvement
> Reporter: Liyin Tang
> Assignee: Liyin Tang
> Attachments: D2775.1.patch, D2775.1.patch, D2775.2.patch,
> D2775.2.patch
>
>
> There is a known issue in HBase client that single slow/dead region server
> could slow down the multiput operations across all the region servers. So the
> HBase client will be as slow as the slowest region server in the cluster.
>
> To solve this problem, HTableMultiplexer will separate the multiput
> submitting threads with the flush threads, which means the multiput operation
> will be a nonblocking operation.
> The submitting thread will shard all the puts into different queues based on
> its destination region server and return immediately. The flush threads will
> flush these puts from each queue to its destination region server.
> Currently the HTableMultiplexer only supports the put operation.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
