[
https://issues.apache.org/jira/browse/HBASE-2782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12881999#action_12881999
]
Todd Lipcon commented on HBASE-2782:
------------------------------------
Here are a couple quick ideas off the top of my head:
1) use a priority queue inside the RPC server, and allow the constructor of the
server to specify a Comparator<Invocation> to decide which things to go first.
We can then introspect the RPCs while they're in the queue to put META requests
first.
2) allow an RPC server to specify a Function<Invocation,Long> which returns a
priority level for any incoming RPC. This would allow not just ordering like
comparator above, but also the ability to reserve pools of handlers for
different RPC tiers. (eg we can say any request to META has a priority 1000,
anything to normal tables is < 500, and we will always reserve a thread pool of
5 handlers for META access)
3) give region servers TWO rpc ports, with separate handler pools - probably a
big mess since we have assumptions that we only have one RPC port per server.
But this is what HDFS is doing now and seems to work.
I kind of like option 2 above, and it doesn't seem like it would be incredibly
difficult.
> QOS for META table access
> -------------------------
>
> Key: HBASE-2782
> URL: https://issues.apache.org/jira/browse/HBASE-2782
> Project: HBase
> Issue Type: Improvement
> Components: regionserver
> Reporter: Todd Lipcon
>
> I'd like to brainstorm some ideas on how we can prioritize reads and writes
> to META above reads and writes to other tables. I've noticed that if the
> regionserver hosting META is under heavy load, then lots of other operations
> take much longer than they should. For example, I'm currently running 120
> threads of YCSB across 3 client nodes hitting a 5-node cluster. Doing a full
> scan of META (only 600 rows) takes upwards of 30 seconds in the shell, since
> all of the handler threads are tied up and there's a long RPC queue.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
