[
https://issues.apache.org/jira/browse/HBASE-4120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13072701#comment-13072701
]
Liu Jia commented on HBASE-4120:
--------------------------------
@stack Hi stack, when design the table priority part, we thought a good design
may need to meet the following four requirements
1, be efficient, don't affect the system throughput.
2, when system is not busy, tables with different priorities no matter high or
low can take full use of the system resources.
3, when system is busy, high priority table costs more resource than low.
4, when system is busy, low priority table will be functioning normal.
Before carrying out this work, we actually thought about managing the block
cache or memstore to achieve the goals, but this is more complex than managing
the RPC queue and meanwhile it’s not a directly approach to solve this
problem. Keep it simple is the principal that we must follow. Due to the LRU
cache replacement strategy of block cache, the more RPC requests processed the
more block cache size will be occupied, so we make this implementation decision.
Stack, would you please send me the issue number which is about the previous
discussion of this function. Other's opinions can help us find defective points
and neglected problem in our design.
> isolation and allocation
> ------------------------
>
> Key: HBASE-4120
> URL: https://issues.apache.org/jira/browse/HBASE-4120
> Project: HBase
> Issue Type: New Feature
> Components: master, regionserver
> Affects Versions: 0.90.2
> Reporter: Liu Jia
> Attachments: Design_document_for_HBase_isolation_and_allocation.pdf,
> Design_document_for_HBase_isolation_and_allocation_Revised.pdf,
> HBase_isolation_and_allocation_user_guide.pdf,
> Performance_of_Table_priority.pdf, System Structure.jpg
>
>
> The HBase isolation and allocation tool is designed to help users manage
> cluster resource among different application and tables.
> When we have a large scale of HBase cluster with many applications running on
> it, there will be lots of problems. In Taobao there is a cluster for many
> departments to test their applications performance, these applications are
> based on HBase. With one cluster which has 12 servers, there will be only one
> application running exclusively on this server, and many other applications
> must wait until the previous test finished.
> After we add allocation manage function to the cluster, applications can
> share the cluster and run concurrently. Also if the Test Engineer wants to
> make sure there is no interference, he/she can move out other tables from
> this group.
> In groups we use table priority to allocate resource, when system is busy; we
> can make sure high-priority tables are not affected lower-priority tables
> Different groups can have different region server configurations, some groups
> optimized for reading can have large block cache size, and others optimized
> for writing can have large memstore size.
> Tables and region servers can be moved easily between groups; after changing
> the configuration, a group can be restarted alone instead of restarting the
> whole cluster.
> git entry : https://github.com/ICT-Ope/HBase_allocation .
> We hope our work is helpful.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
