[jira] [Updated] (HBASE-5169) Group of Region Server, a subtask of issue 4120
[ https://issues.apache.org/jira/browse/HBASE-5169?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liu Jia updated HBASE-5169: --- Attachment: GroupOfRegionServer_v2.patch Update to trunk code. Group of Region Server, a subtask of issue 4120 - Key: HBASE-5169 URL: https://issues.apache.org/jira/browse/HBASE-5169 Project: HBase Issue Type: Sub-task Components: master Reporter: Liu Jia Assignee: Liu Jia Fix For: 0.94.0 Attachments: GroupOfRegionServer_v1.patch, GroupOfRegionServer_v2.patch This is a subtask of issue 4120,this patch provides the region server group feature of HBase. With this patch, region servers can be divided into groups,one table could belong to one or more groups while the region server can only belong to one group. Work load in defferent groups will not affect each other. This patch provides table level and group level load balance,the default load balance and region assignments will consider the group configuration and assign regions to their corresponding groups. More information, please check out the documents of issue 4120. There is a web tool of this patch providing operations of group managements like add/delete group, move in/out servers,change table's group attribute ,balance groups, balance tables. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5169) This is a subtask of issue 4120,this patch provides the region server group feature of HBase.
[ https://issues.apache.org/jira/browse/HBASE-5169?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liu Jia updated HBASE-5169: --- Attachment: GroupOfRegionServer_v1.patch This is a subtask of issue 4120,this patch provides the region server group feature of HBase. -- Key: HBASE-5169 URL: https://issues.apache.org/jira/browse/HBASE-5169 Project: HBase Issue Type: Sub-task Components: master Reporter: Liu Jia Assignee: Liu Jia Fix For: 0.94.0 Attachments: GroupOfRegionServer_v1.patch This is a subtask of issue 4120,this patch provides the region server group feature of HBase. With this patch, region servers can be divided into groups,one table could belong to one or more groups while the region server can only belong to one group. Work load in defferent groups will not affect each other. This patch provides table level and group level load balance,the default load balance and region assignments will consider the group configuration and assign regions to their corresponding groups. More information, please check out the documents of issue 4120. There is a web tool of this patch providing operations of group managements like add/delete group, move in/out servers,change table's group attribute ,balance groups, balance tables. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5169) This is a subtask of issue 4120,this patch provides the region server group feature of HBase.
[ https://issues.apache.org/jira/browse/HBASE-5169?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liu Jia updated HBASE-5169: --- Status: Patch Available (was: Open) The patch of region server group,with test case but don't contain the web pages. This is a subtask of issue 4120,this patch provides the region server group feature of HBase. -- Key: HBASE-5169 URL: https://issues.apache.org/jira/browse/HBASE-5169 Project: HBase Issue Type: Sub-task Components: master Reporter: Liu Jia Assignee: Liu Jia Fix For: 0.94.0 Attachments: GroupOfRegionServer_v1.patch This is a subtask of issue 4120,this patch provides the region server group feature of HBase. With this patch, region servers can be divided into groups,one table could belong to one or more groups while the region server can only belong to one group. Work load in defferent groups will not affect each other. This patch provides table level and group level load balance,the default load balance and region assignments will consider the group configuration and assign regions to their corresponding groups. More information, please check out the documents of issue 4120. There is a web tool of this patch providing operations of group managements like add/delete group, move in/out servers,change table's group attribute ,balance groups, balance tables. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5170) The web tools of region server groups
[ https://issues.apache.org/jira/browse/HBASE-5170?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liu Jia updated HBASE-5170: --- Attachment: GroupOfRegionServerWebTool.patch The web tools of region server groups - Key: HBASE-5170 URL: https://issues.apache.org/jira/browse/HBASE-5170 Project: HBase Issue Type: Sub-task Components: master, regionserver Reporter: Liu Jia Fix For: 0.94.0 Attachments: GroupOfRegionServerWebTool.patch The web pages which allow users to perform some group management operations including add/delete group, move in/out servers,change table's group attribute ,balance groups, balance tables. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5169) Group of Region Server, a subtask of issue 4120
[ https://issues.apache.org/jira/browse/HBASE-5169?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liu Jia updated HBASE-5169: --- Summary: Group of Region Server, a subtask of issue 4120 (was: This is a subtask of issue 4120,this patch provides the region server group feature of HBase.) Group of Region Server, a subtask of issue 4120 - Key: HBASE-5169 URL: https://issues.apache.org/jira/browse/HBASE-5169 Project: HBase Issue Type: Sub-task Components: master Reporter: Liu Jia Assignee: Liu Jia Fix For: 0.94.0 Attachments: GroupOfRegionServer_v1.patch This is a subtask of issue 4120,this patch provides the region server group feature of HBase. With this patch, region servers can be divided into groups,one table could belong to one or more groups while the region server can only belong to one group. Work load in defferent groups will not affect each other. This patch provides table level and group level load balance,the default load balance and region assignments will consider the group configuration and assign regions to their corresponding groups. More information, please check out the documents of issue 4120. There is a web tool of this patch providing operations of group managements like add/delete group, move in/out servers,change table's group attribute ,balance groups, balance tables. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5092) Two adjacent assignments lead region is in PENDING_OPEN state and block table disable and enable actions.
[
https://issues.apache.org/jira/browse/HBASE-5092?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Liu Jia updated HBASE-5092:
---
Attachment: unhandled_PENDING_OPEN_lead_by_two_assignment.patch
Two adjacent assignments lead region is in PENDING_OPEN state and block table
disable and enable actions.
-
Key: HBASE-5092
URL: https://issues.apache.org/jira/browse/HBASE-5092
Project: HBase
Issue Type: Bug
Components: master, regionserver
Affects Versions: 0.92.0
Reporter: Liu Jia
Assignee: Liu Jia
Attachments: unhandled_PENDING_OPEN_lead_by_two_assignment.patch
Region is in PENDING_OPEN state and disable and enable are blocked.
We occasionally find if two assignments which have a short interval time will
lead to a PENDING_OPEN state staying in the regionInTransition map and
blocking the disable and enable table actions.
We found that the second assignment will set the zknode of this region to
M_ZK_REGION_OFFLINE then set the state in assignmentMananger's
regionInTransition map to PENDING_OPEN and abort its further operation
because of finding the the region is already in the regionserver by a
RegionAlreadyInTransitionException.
At the same time the first assignment is tickleOpening and find the version
of the zknode is messed up by the second assignment, so the
OpenRegionHandler print out the following two lines:
{noformat}
2011-12-23 22:12:15,197 WARN [RS_OPEN_REGION-data16,59892,1324649528415-0]
zookeeper.ZKAssign(788): regionserver:59892-0x1346b43b91e0002 Attempt to
transition the unassigned node for 15237599c632752b8cfd3d5a86349768 from
RS_ZK_REGION_OPENING to RS_ZK_REGION_OPENING failed, the node existed but was
version 2 not the expected version 1
2011-12-23 22:12:15,197 WARN [RS_OPEN_REGION-data16,59892,1324649528415-0]
handler.OpenRegionHandler(403): Failed refreshing OPENING;
region=15237599c632752b8cfd3d5a86349768, context=post_region_open
{noformat}
After that it tries to turn the state to FAILED_OPEN, but also failed due to
wrong version,
this is the output:
{noformat}
2011-12-23 22:12:15,199 WARN [RS_OPEN_REGION-data16,59892,1324649528415-0]
zookeeper.ZKAssign(812): regionserver:59892-0x1346b43b91e0002 Attempt to
transition the unassigned node for 15237599c632752b8cfd3d5a86349768 from
RS_ZK_REGION_OPENING to RS_ZK_REGION_FAILED_OPEN failed, the node existed but
was in the state M_ZK_REGION_OFFLINE set by the server
data16,59892,1324649528415
2011-12-23 22:12:15,199 WARN [RS_OPEN_REGION-data16,59892,1324649528415-0]
handler.OpenRegionHandler(307): Unable to mark region {NAME =
'table1,,1324649533045.15237599c632752b8cfd3d5a86349768.', STARTKEY = '',
ENDKEY = '', ENCODED = 15237599c632752b8cfd3d5a86349768,} as FAILED_OPEN.
It's likely that the master already timed out this open attempt, and thus
another RS already has the region.
{noformat}
So after all that, the PENDING_OPEN state is left in the assignmentMananger's
regionInTransition map and none will deal with it further,
This kind of situation will wait until the master find the state out of time.
The following is the test code:
{code:title=test.java|borderStyle=solid}
@Test
public void testDisableTables() throws IOException {
for (int i = 0; i 20; i++) {
HTableDescriptor des = admin.getTableDescriptor(Bytes.toBytes(table1));
ListHRegionInfo hris = TEST_UTIL.getHBaseCluster().getMaster()
.getAssignmentManager().getRegionsOfTable(Bytes.toBytes(table1));
TEST_UTIL.getHBaseCluster().getMaster()
.assign(hris.get(0).getRegionName());
TEST_UTIL.getHBaseCluster().getMaster()
.assign(hris.get(0).getRegionName());
admin.disableTable(Bytes.toBytes(table1));
admin.modifyTable(Bytes.toBytes(table1), des);
admin.enableTable(Bytes.toBytes(table1));
}
}
{code}
To fix this,we add a line to
public static int ZKAssign.transitionNode() to make
endState.RS_ZK_REGION_FAILED_OPEN transition pass.
{code:title=ZKAssign.java|borderStyle=solid}
if((!existingData.getEventType().equals(beginState))
//add the following line to make endState.RS_ZK_REGION_FAILED_OPEN
transition pass.
(!endState.equals(endState.RS_ZK_REGION_FAILED_OPEN))) {
LOG.warn(zkw.prefix(Attempt to transition the +
unassigned node for + encoded +
from + beginState + to + endState + failed, +
the node existed but was in the state +
existingData.getEventType() +
set by the server + serverName));
return -1;
}
{code}
Run the test case again we found that before the first assignment trans the
state from offline to opening, the second assignment could
[jira] [Updated] (HBASE-4120) isolation and allocation
[ https://issues.apache.org/jira/browse/HBASE-4120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liu Jia updated HBASE-4120: --- Attachment: TablePriority_v17.patch Reduce the time used by TestPriorityJobQueue.java. isolation and allocation Key: HBASE-4120 URL: https://issues.apache.org/jira/browse/HBASE-4120 Project: HBase Issue Type: New Feature Components: master, regionserver Affects Versions: 0.90.2, 0.90.3, 0.90.4, 0.92.0 Reporter: Liu Jia Assignee: Liu Jia Fix For: 0.94.0 Attachments: Design_document_for_HBase_isolation_and_allocation.pdf, Design_document_for_HBase_isolation_and_allocation_Revised.pdf, HBase_isolation_and_allocation_user_guide.pdf, Performance_of_Table_priority.pdf, Simple_YCSB_Tests_For_TablePriority_Trunk_and_0.90.4.pdf, System Structure.jpg, TablePriority.patch, TablePriority_v12.patch, TablePriority_v12.patch, TablePriority_v15_with_coprocessor.patch, TablePriority_v16_with_coprocessor.patch, TablePriority_v17.patch, TablePriority_v17.patch, TablePriority_v8.patch, TablePriority_v8.patch, TablePriority_v8_for_trunk.patch, TablePrioriy_v9.patch The HBase isolation and allocation tool is designed to help users manage cluster resource among different application and tables. When we have a large scale of HBase cluster with many applications running on it, there will be lots of problems. In Taobao there is a cluster for many departments to test their applications performance, these applications are based on HBase. With one cluster which has 12 servers, there will be only one application running exclusively on this server, and many other applications must wait until the previous test finished. After we add allocation manage function to the cluster, applications can share the cluster and run concurrently. Also if the Test Engineer wants to make sure there is no interference, he/she can move out other tables from this group. In groups we use table priority to allocate resource, when system is busy; we can make sure high-priority tables are not affected lower-priority tables Different groups can have different region server configurations, some groups optimized for reading can have large block cache size, and others optimized for writing can have large memstore size. Tables and region servers can be moved easily between groups; after changing the configuration, a group can be restarted alone instead of restarting the whole cluster. git entry : https://github.com/ICT-Ope/HBase_allocation . We hope our work is helpful. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4120) isolation and allocation
[ https://issues.apache.org/jira/browse/HBASE-4120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liu Jia updated HBASE-4120: --- Attachment: TablePriority_v17.patch Simplified the design,remove the thread priority and some complex parts. isolation and allocation Key: HBASE-4120 URL: https://issues.apache.org/jira/browse/HBASE-4120 Project: HBase Issue Type: New Feature Components: master, regionserver Affects Versions: 0.90.2, 0.90.3, 0.90.4, 0.92.0 Reporter: Liu Jia Assignee: Liu Jia Fix For: 0.94.0 Attachments: Design_document_for_HBase_isolation_and_allocation.pdf, Design_document_for_HBase_isolation_and_allocation_Revised.pdf, HBase_isolation_and_allocation_user_guide.pdf, Performance_of_Table_priority.pdf, Simple_YCSB_Tests_For_TablePriority_Trunk_and_0.90.4.pdf, System Structure.jpg, TablePriority.patch, TablePriority_v12.patch, TablePriority_v12.patch, TablePriority_v15_with_coprocessor.patch, TablePriority_v16_with_coprocessor.patch, TablePriority_v17.patch, TablePriority_v8.patch, TablePriority_v8.patch, TablePriority_v8_for_trunk.patch, TablePrioriy_v9.patch The HBase isolation and allocation tool is designed to help users manage cluster resource among different application and tables. When we have a large scale of HBase cluster with many applications running on it, there will be lots of problems. In Taobao there is a cluster for many departments to test their applications performance, these applications are based on HBase. With one cluster which has 12 servers, there will be only one application running exclusively on this server, and many other applications must wait until the previous test finished. After we add allocation manage function to the cluster, applications can share the cluster and run concurrently. Also if the Test Engineer wants to make sure there is no interference, he/she can move out other tables from this group. In groups we use table priority to allocate resource, when system is busy; we can make sure high-priority tables are not affected lower-priority tables Different groups can have different region server configurations, some groups optimized for reading can have large block cache size, and others optimized for writing can have large memstore size. Tables and region servers can be moved easily between groups; after changing the configuration, a group can be restarted alone instead of restarting the whole cluster. git entry : https://github.com/ICT-Ope/HBase_allocation . We hope our work is helpful. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5026) Add coprocessor hook to HRegionServer.ScannerListener.leaseExpired()
[ https://issues.apache.org/jira/browse/HBASE-5026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liu Jia updated HBASE-5026: --- Attachment: RegionObserverLeaseExpired.patch Add coprocessor hook to HRegionServer.ScannerListener.leaseExpired() Key: HBASE-5026 URL: https://issues.apache.org/jira/browse/HBASE-5026 Project: HBase Issue Type: Bug Components: coprocessors Affects Versions: 0.92.0 Reporter: Liu Jia Assignee: Liu Jia Fix For: 0.92.0 Attachments: RegionObserverLeaseExpired.patch The RegionObserver's preScannerClose() and postScannerClose() methods should cover the scanner leaseExpired() situation. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5026) Add coprocessor hook to HRegionServer.ScannerListener.leaseExpired()
[ https://issues.apache.org/jira/browse/HBASE-5026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liu Jia updated HBASE-5026: --- Status: Patch Available (was: Open) Add coprocessor hook to HRegionServer.ScannerListener.leaseExpired() Key: HBASE-5026 URL: https://issues.apache.org/jira/browse/HBASE-5026 Project: HBase Issue Type: Bug Components: coprocessors Affects Versions: 0.92.0 Reporter: Liu Jia Assignee: Liu Jia Fix For: 0.92.0 Attachments: RegionObserverLeaseExpired.patch The RegionObserver's preScannerClose() and postScannerClose() methods should cover the scanner leaseExpired() situation. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4120) isolation and allocation
[ https://issues.apache.org/jira/browse/HBASE-4120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liu Jia updated HBASE-4120: --- Attachment: TablePriority_v16_with_coprocessor.patch Modify the two test cases :org.apache.hadoop.hbase.coprocessor.TestMasterCoprocessorExceptionWithAbort org.apache.hadoop.hbase.coprocessor.TestMasterCoprocessorExceptionWithRemove Enable the table priority function by default. isolation and allocation Key: HBASE-4120 URL: https://issues.apache.org/jira/browse/HBASE-4120 Project: HBase Issue Type: New Feature Components: master, regionserver Affects Versions: 0.90.2, 0.90.3, 0.90.4, 0.92.0 Reporter: Liu Jia Assignee: Liu Jia Fix For: 0.94.0 Attachments: Design_document_for_HBase_isolation_and_allocation.pdf, Design_document_for_HBase_isolation_and_allocation_Revised.pdf, HBase_isolation_and_allocation_user_guide.pdf, Performance_of_Table_priority.pdf, System Structure.jpg, TablePriority.patch, TablePriority_v12.patch, TablePriority_v12.patch, TablePriority_v15_with_coprocessor.patch, TablePriority_v16_with_coprocessor.patch, TablePriority_v8.patch, TablePriority_v8.patch, TablePriority_v8_for_trunk.patch, TablePrioriy_v9.patch The HBase isolation and allocation tool is designed to help users manage cluster resource among different application and tables. When we have a large scale of HBase cluster with many applications running on it, there will be lots of problems. In Taobao there is a cluster for many departments to test their applications performance, these applications are based on HBase. With one cluster which has 12 servers, there will be only one application running exclusively on this server, and many other applications must wait until the previous test finished. After we add allocation manage function to the cluster, applications can share the cluster and run concurrently. Also if the Test Engineer wants to make sure there is no interference, he/she can move out other tables from this group. In groups we use table priority to allocate resource, when system is busy; we can make sure high-priority tables are not affected lower-priority tables Different groups can have different region server configurations, some groups optimized for reading can have large block cache size, and others optimized for writing can have large memstore size. Tables and region servers can be moved easily between groups; after changing the configuration, a group can be restarted alone instead of restarting the whole cluster. git entry : https://github.com/ICT-Ope/HBase_allocation . We hope our work is helpful. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4120) isolation and allocation
[ https://issues.apache.org/jira/browse/HBASE-4120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liu Jia updated HBASE-4120: --- Attachment: Simple_YCSB_Tests_For_TablePriority_Trunk_and_0.90.4.pdf isolation and allocation Key: HBASE-4120 URL: https://issues.apache.org/jira/browse/HBASE-4120 Project: HBase Issue Type: New Feature Components: master, regionserver Affects Versions: 0.90.2, 0.90.3, 0.90.4, 0.92.0 Reporter: Liu Jia Assignee: Liu Jia Fix For: 0.94.0 Attachments: Design_document_for_HBase_isolation_and_allocation.pdf, Design_document_for_HBase_isolation_and_allocation_Revised.pdf, HBase_isolation_and_allocation_user_guide.pdf, Performance_of_Table_priority.pdf, Simple_YCSB_Tests_For_TablePriority_Trunk_and_0.90.4.pdf, System Structure.jpg, TablePriority.patch, TablePriority_v12.patch, TablePriority_v12.patch, TablePriority_v15_with_coprocessor.patch, TablePriority_v16_with_coprocessor.patch, TablePriority_v8.patch, TablePriority_v8.patch, TablePriority_v8_for_trunk.patch, TablePrioriy_v9.patch The HBase isolation and allocation tool is designed to help users manage cluster resource among different application and tables. When we have a large scale of HBase cluster with many applications running on it, there will be lots of problems. In Taobao there is a cluster for many departments to test their applications performance, these applications are based on HBase. With one cluster which has 12 servers, there will be only one application running exclusively on this server, and many other applications must wait until the previous test finished. After we add allocation manage function to the cluster, applications can share the cluster and run concurrently. Also if the Test Engineer wants to make sure there is no interference, he/she can move out other tables from this group. In groups we use table priority to allocate resource, when system is busy; we can make sure high-priority tables are not affected lower-priority tables Different groups can have different region server configurations, some groups optimized for reading can have large block cache size, and others optimized for writing can have large memstore size. Tables and region servers can be moved easily between groups; after changing the configuration, a group can be restarted alone instead of restarting the whole cluster. git entry : https://github.com/ICT-Ope/HBase_allocation . We hope our work is helpful. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4120) isolation and allocation
[ https://issues.apache.org/jira/browse/HBASE-4120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liu Jia updated HBASE-4120: --- Attachment: TablePriority_v15_with_coprocessor.patch isolation and allocation Key: HBASE-4120 URL: https://issues.apache.org/jira/browse/HBASE-4120 Project: HBase Issue Type: New Feature Components: master, regionserver Affects Versions: 0.90.2, 0.90.3, 0.90.4, 0.92.0 Reporter: Liu Jia Assignee: Liu Jia Fix For: 0.94.0 Attachments: Design_document_for_HBase_isolation_and_allocation.pdf, Design_document_for_HBase_isolation_and_allocation_Revised.pdf, HBase_isolation_and_allocation_user_guide.pdf, Performance_of_Table_priority.pdf, System Structure.jpg, TablePriority.patch, TablePriority_v12.patch, TablePriority_v12.patch, TablePriority_v15_with_coprocessor.patch, TablePriority_v8.patch, TablePriority_v8.patch, TablePriority_v8_for_trunk.patch, TablePrioriy_v9.patch The HBase isolation and allocation tool is designed to help users manage cluster resource among different application and tables. When we have a large scale of HBase cluster with many applications running on it, there will be lots of problems. In Taobao there is a cluster for many departments to test their applications performance, these applications are based on HBase. With one cluster which has 12 servers, there will be only one application running exclusively on this server, and many other applications must wait until the previous test finished. After we add allocation manage function to the cluster, applications can share the cluster and run concurrently. Also if the Test Engineer wants to make sure there is no interference, he/she can move out other tables from this group. In groups we use table priority to allocate resource, when system is busy; we can make sure high-priority tables are not affected lower-priority tables Different groups can have different region server configurations, some groups optimized for reading can have large block cache size, and others optimized for writing can have large memstore size. Tables and region servers can be moved easily between groups; after changing the configuration, a group can be restarted alone instead of restarting the whole cluster. git entry : https://github.com/ICT-Ope/HBase_allocation . We hope our work is helpful. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4120) isolation and allocation
[ https://issues.apache.org/jira/browse/HBASE-4120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liu Jia updated HBASE-4120: --- Attachment: TablePriority_v12.patch Test cases are more stable in this patch. isolation and allocation Key: HBASE-4120 URL: https://issues.apache.org/jira/browse/HBASE-4120 Project: HBase Issue Type: New Feature Components: master, regionserver Affects Versions: 0.90.2, 0.90.3, 0.90.4, 0.92.0 Reporter: Liu Jia Assignee: Liu Jia Fix For: 0.94.0 Attachments: Design_document_for_HBase_isolation_and_allocation.pdf, Design_document_for_HBase_isolation_and_allocation_Revised.pdf, HBase_isolation_and_allocation_user_guide.pdf, Performance_of_Table_priority.pdf, System Structure.jpg, TablePriority.patch, TablePriority_v12.patch, TablePriority_v12.patch, TablePriority_v8.patch, TablePriority_v8.patch, TablePriority_v8_for_trunk.patch, TablePrioriy_v9.patch The HBase isolation and allocation tool is designed to help users manage cluster resource among different application and tables. When we have a large scale of HBase cluster with many applications running on it, there will be lots of problems. In Taobao there is a cluster for many departments to test their applications performance, these applications are based on HBase. With one cluster which has 12 servers, there will be only one application running exclusively on this server, and many other applications must wait until the previous test finished. After we add allocation manage function to the cluster, applications can share the cluster and run concurrently. Also if the Test Engineer wants to make sure there is no interference, he/she can move out other tables from this group. In groups we use table priority to allocate resource, when system is busy; we can make sure high-priority tables are not affected lower-priority tables Different groups can have different region server configurations, some groups optimized for reading can have large block cache size, and others optimized for writing can have large memstore size. Tables and region servers can be moved easily between groups; after changing the configuration, a group can be restarted alone instead of restarting the whole cluster. git entry : https://github.com/ICT-Ope/HBase_allocation . We hope our work is helpful. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4120) isolation and allocation
[ https://issues.apache.org/jira/browse/HBASE-4120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liu Jia updated HBASE-4120: --- Attachment: TablePriority_v12.patch Test cases are more stable in this patch. isolation and allocation Key: HBASE-4120 URL: https://issues.apache.org/jira/browse/HBASE-4120 Project: HBase Issue Type: New Feature Components: master, regionserver Affects Versions: 0.90.2, 0.90.3, 0.90.4, 0.92.0 Reporter: Liu Jia Assignee: Liu Jia Fix For: 0.94.0 Attachments: Design_document_for_HBase_isolation_and_allocation.pdf, Design_document_for_HBase_isolation_and_allocation_Revised.pdf, HBase_isolation_and_allocation_user_guide.pdf, Performance_of_Table_priority.pdf, System Structure.jpg, TablePriority.patch, TablePriority_v12.patch, TablePriority_v12.patch, TablePriority_v8.patch, TablePriority_v8.patch, TablePriority_v8_for_trunk.patch, TablePrioriy_v9.patch The HBase isolation and allocation tool is designed to help users manage cluster resource among different application and tables. When we have a large scale of HBase cluster with many applications running on it, there will be lots of problems. In Taobao there is a cluster for many departments to test their applications performance, these applications are based on HBase. With one cluster which has 12 servers, there will be only one application running exclusively on this server, and many other applications must wait until the previous test finished. After we add allocation manage function to the cluster, applications can share the cluster and run concurrently. Also if the Test Engineer wants to make sure there is no interference, he/she can move out other tables from this group. In groups we use table priority to allocate resource, when system is busy; we can make sure high-priority tables are not affected lower-priority tables Different groups can have different region server configurations, some groups optimized for reading can have large block cache size, and others optimized for writing can have large memstore size. Tables and region servers can be moved easily between groups; after changing the configuration, a group can be restarted alone instead of restarting the whole cluster. git entry : https://github.com/ICT-Ope/HBase_allocation . We hope our work is helpful. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4120) isolation and allocation
[ https://issues.apache.org/jira/browse/HBASE-4120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liu Jia updated HBASE-4120: --- Attachment: TablePrioriy_v9.patch Change the region priority refresh mode from pull to push, fix code to past some core tests. isolation and allocation Key: HBASE-4120 URL: https://issues.apache.org/jira/browse/HBASE-4120 Project: HBase Issue Type: New Feature Components: master, regionserver Affects Versions: 0.90.2, 0.90.3, 0.90.4, 0.92.0 Reporter: Liu Jia Assignee: Liu Jia Fix For: 0.94.0 Attachments: Design_document_for_HBase_isolation_and_allocation.pdf, Design_document_for_HBase_isolation_and_allocation_Revised.pdf, HBase_isolation_and_allocation_user_guide.pdf, Performance_of_Table_priority.pdf, System Structure.jpg, TablePriority.patch, TablePriority_v8.patch, TablePriority_v8.patch, TablePriority_v8_for_trunk.patch, TablePrioriy_v9.patch The HBase isolation and allocation tool is designed to help users manage cluster resource among different application and tables. When we have a large scale of HBase cluster with many applications running on it, there will be lots of problems. In Taobao there is a cluster for many departments to test their applications performance, these applications are based on HBase. With one cluster which has 12 servers, there will be only one application running exclusively on this server, and many other applications must wait until the previous test finished. After we add allocation manage function to the cluster, applications can share the cluster and run concurrently. Also if the Test Engineer wants to make sure there is no interference, he/she can move out other tables from this group. In groups we use table priority to allocate resource, when system is busy; we can make sure high-priority tables are not affected lower-priority tables Different groups can have different region server configurations, some groups optimized for reading can have large block cache size, and others optimized for writing can have large memstore size. Tables and region servers can be moved easily between groups; after changing the configuration, a group can be restarted alone instead of restarting the whole cluster. git entry : https://github.com/ICT-Ope/HBase_allocation . We hope our work is helpful. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4120) isolation and allocation
[ https://issues.apache.org/jira/browse/HBASE-4120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liu Jia updated HBASE-4120: --- Attachment: TablePriority_v8_for_trunk.patch table priority patch for trunk. isolation and allocation Key: HBASE-4120 URL: https://issues.apache.org/jira/browse/HBASE-4120 Project: HBase Issue Type: New Feature Components: master, regionserver Affects Versions: 0.90.2, 0.90.3, 0.90.4, 0.92.0 Reporter: Liu Jia Assignee: Liu Jia Fix For: 0.94.0 Attachments: Design_document_for_HBase_isolation_and_allocation.pdf, Design_document_for_HBase_isolation_and_allocation_Revised.pdf, HBase_isolation_and_allocation_user_guide.pdf, Performance_of_Table_priority.pdf, System Structure.jpg, TablePriority.patch, TablePriority_v8_for_trunk.patch The HBase isolation and allocation tool is designed to help users manage cluster resource among different application and tables. When we have a large scale of HBase cluster with many applications running on it, there will be lots of problems. In Taobao there is a cluster for many departments to test their applications performance, these applications are based on HBase. With one cluster which has 12 servers, there will be only one application running exclusively on this server, and many other applications must wait until the previous test finished. After we add allocation manage function to the cluster, applications can share the cluster and run concurrently. Also if the Test Engineer wants to make sure there is no interference, he/she can move out other tables from this group. In groups we use table priority to allocate resource, when system is busy; we can make sure high-priority tables are not affected lower-priority tables Different groups can have different region server configurations, some groups optimized for reading can have large block cache size, and others optimized for writing can have large memstore size. Tables and region servers can be moved easily between groups; after changing the configuration, a group can be restarted alone instead of restarting the whole cluster. git entry : https://github.com/ICT-Ope/HBase_allocation . We hope our work is helpful. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
