[jira] [Commented] (HBASE-6143) Make region assignment smarter when regions are re-enabled.
[
https://issues.apache.org/jira/browse/HBASE-6143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14393830#comment-14393830
]
Hadoop QA commented on HBASE-6143:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12709085/HBASE-6143-v2.patch
against master branch at commit 2365932d78180255be7145d8d247ee1f801aed1d.
ATTACHMENT ID: 12709085
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 20 new
or modified tests.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.1 2.5.2 2.6.0)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 protoc{color}. The applied patch does not increase the
total number of protoc compiler warnings.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 2
warning messages.
{color:red}-1 checkstyle{color}. The applied patch generated
1940 checkstyle errors (more than the master's current 1924 errors).
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 lineLengths{color}. The patch introduces the following lines
longer than 100:
+ BulkAssigner ba = new GeneralBulkAssigner(this.server, bulkPlan,
this, bulkAssignWaitTillAllAssigned);
+MapServerName, ServerName noTimeToCurrent = new
HashMapServerName,ServerName(clusterState.size());
+MapHRegionInfo, ServerName initialLocation = new HashMapHRegionInfo,
ServerName(regions.size());
+sn = noTimeToCurrent.get(ServerName.valueOf(old.getHostname(),
old.getPort(), ServerName.NON_STARTCODE));
+MapHRegionInfo, ServerName suggest)
throws HBaseIOException {
+//conf.setClass(hbase.util.ip.to.rack.determiner, MockMapping.class,
DNSToSwitchMapping.class);
+//clusterState.put(servers[5], list1); //servers[5], rack2 hosts
replica_of_region1 and replica_of_region2
+//clusterState.put(servers[10], new ArrayListHRegionInfo());
//servers[10], rack3 hosts no region
+//MapServerName, ListHRegionInfo clusterState = new
HashMapServerName, ListHRegionInfo();
+//assertEquals(10, cluster.numServers); // only 10 servers because they
share the same host + port
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.master.TestMaster
org.apache.hadoop.hbase.master.TestRegionPlacement
org.apache.hadoop.hbase.util.TestHBaseFsck
org.apache.hadoop.hbase.util.TestMergeTable
org.apache.hadoop.hbase.master.TestMasterOperationsForRegionReplicas
org.apache.hadoop.hbase.replication.TestReplicationSyncUpTool
org.apache.hadoop.hbase.util.hbck.TestOfflineMetaRebuildBase
org.apache.hadoop.hbase.master.TestRestartCluster
org.apache.hadoop.hbase.regionserver.TestRSKilledWhenInitializing
org.apache.hadoop.hbase.TestRegionRebalancing
org.apache.hadoop.hbase.client.TestAdmin1
{color:red}-1 core zombie tests{color}. There are 6 zombie test(s):
at
org.apache.hadoop.hbase.client.TestRestoreSnapshotFromClient.testRestoreSnapshot(TestRestoreSnapshotFromClient.java:147)
at
org.apache.hadoop.hbase.client.TestRestoreSnapshotFromClient.testRestoreSnapshot(TestRestoreSnapshotFromClient.java:147)
at
org.apache.hadoop.hbase.client.TestHCM.testConnectionClose(TestHCM.java:327)
at
org.apache.hadoop.hbase.client.TestHCM.testConnectionCloseAllowsInterrupt(TestHCM.java:280)
at
org.apache.hadoop.hbase.snapshot.TestRestoreFlushSnapshotFromClient.testRestoreSnapshot(TestRestoreFlushSnapshotFromClient.java:151)
at
org.apache.hadoop.hbase.security.visibility.TestVisibilityLabels.testVisibilityLabelsInScanThatDoesNotMatchAnyDefinedLabels(TestVisibilityLabels.java:255)
at
org.apache.hadoop.hbase.client.TestSnapshotMetadata.testDescribeOnEmptyTableMatchesAfterMetadataChangeAndRestore(TestSnapshotMetadata.java:248)
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/13545//testReport/
Release Findbugs (version 2.0.3)warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13545//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors:
[jira] [Commented] (HBASE-6143) Make region assignment smarter when regions are re-enabled.
[ https://issues.apache.org/jira/browse/HBASE-6143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14390940#comment-14390940 ] Elliott Clark commented on HBASE-6143: -- Review is up here: https://reviews.facebook.net/D36267 Make region assignment smarter when regions are re-enabled. --- Key: HBASE-6143 URL: https://issues.apache.org/jira/browse/HBASE-6143 Project: HBase Issue Type: Improvement Components: Balancer Affects Versions: 0.98.0, 0.95.1 Reporter: Elliott Clark Assignee: Elliott Clark Priority: Critical Attachments: 6143-v1.txt, 6143-v2.txt, 6143-v3.txt, HBASE-6143-0.patch, HBASE-6143-1.patch, HBASE-6143-v1.patch Right now a random region server is picked when re-enabling a table. This could be much smarter. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-6143) Make region assignment smarter when regions are re-enabled.
[
https://issues.apache.org/jira/browse/HBASE-6143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14391878#comment-14391878
]
Hadoop QA commented on HBASE-6143:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12708729/HBASE-6143-v1.patch
against master branch at commit 485800830a58da1af31beaac859d99b56d823f99.
ATTACHMENT ID: 12708729
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 20 new
or modified tests.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.1 2.5.2 2.6.0)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 protoc{color}. The applied patch does not increase the
total number of protoc compiler warnings.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 6
warning messages.
{color:red}-1 checkstyle{color}. The applied patch generated
1937 checkstyle errors (more than the master's current 1924 errors).
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:red}-1 release audit{color}. The applied patch generated 1 release
audit warnings (more than the master's current 0 warnings).
{color:red}-1 lineLengths{color}. The patch introduces the following lines
longer than 100:
+ BulkAssigner ba = new GeneralBulkAssigner(this.server, bulkPlan,
this, bulkAssignWaitTillAllAssigned);
+MapHRegionInfo, ServerName suggest)
throws HBaseIOException {
+// LOG.debug(testImmediateAssignment with + mock[0] + regions and
+ mock[1] + servers);
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.master.TestMasterOperationsForRegionReplicas
org.apache.hadoop.hbase.master.handler.TestEnableTableHandler
org.apache.hadoop.hbase.TestRegionRebalancing
org.apache.hadoop.hbase.master.TestMaster
org.apache.hadoop.hbase.client.TestAdmin1
org.apache.hadoop.hbase.master.TestRestartCluster
org.apache.hadoop.hbase.master.TestRegionPlacement
{color:red}-1 core zombie tests{color}. There are 2 zombie test(s):
at
org.apache.hadoop.hbase.coprocessor.TestRegionServerObserver.testCoprocessorHooksInRegionsMerge(TestRegionServerObserver.java:100)
at
org.apache.hadoop.hbase.regionserver.TestEncryptionKeyRotation.testMasterKeyRotation(TestEncryptionKeyRotation.java:189)
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/13527//testReport/
Release audit warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13527//artifact/patchprocess/patchReleaseAuditWarnings.txt
Release Findbugs (version 2.0.3)warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13527//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/13527//artifact/patchprocess/checkstyle-aggregate.html
Javadoc warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/13527//artifact/patchprocess/patchJavadocWarnings.txt
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/13527//console
This message is automatically generated.
Make region assignment smarter when regions are re-enabled.
---
Key: HBASE-6143
URL: https://issues.apache.org/jira/browse/HBASE-6143
Project: HBase
Issue Type: Improvement
Components: Balancer
Reporter: Elliott Clark
Assignee: Elliott Clark
Priority: Critical
Attachments: 6143-v1.txt, 6143-v2.txt, 6143-v3.txt,
HBASE-6143-0.patch, HBASE-6143-1.patch, HBASE-6143-v1.patch
Right now a random region server is picked when re-enabling a table. This
could be much smarter.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-6143) Make region assignment smarter when regions are re-enabled.
[
https://issues.apache.org/jira/browse/HBASE-6143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13723465#comment-13723465
]
Vladimir Rodionov commented on HBASE-6143:
--
OK, this is probably not the right place but :
{code}
requestsPerSecond=0, numberOfOnlineRegions=398, numberOfStores=415,
numberOfStorefiles=135, storefileIndexSizeMB=1, rootIndexSizeKB=1865,
totalStaticIndexSizeKB=25208, totalStaticBloomSizeKB=30481, memstoreSizeMB=0,
mbInMemoryWithoutWAL=0, numberOfPutsWithoutWAL=0, readRequestsCount=0,
writeRequestsCount=0, compactionQueueSize=0, flushQueueSize=0, usedHeapMB=118,
maxHeapMB=5987, blockCacheSizeMB=12.28, blockCacheFreeMB=1484.53,
blockCacheCount=0, blockCacheHitCount=0, blockCacheMissCount=0,
blockCacheEvictedCount=0, blockCacheHitRatio=0%, blockCacheHitCachingRatio=0%,
hdfsBlocksLocalityIndex=24, slowHLogAppendCount=0,
fsReadLatencyHistogramMean=0.00, fsReadLatencyHistogramCount=0.00,
fsReadLatencyHistogramMedian=0.00, fsReadLatencyHistogram75th=0.00,
fsReadLatencyHistogram95th=0.00, fsReadLatencyHistogram99th=0.00,
fsReadLatencyHistogram999th=0.00, fsPreadLatencyHistogramMean=0.00,
fsPreadLatencyHistogramCount=0.00, fsPreadLatencyHistogramMedian=0.00,
fsPreadLatencyHistogram75th=0.00, fsPreadLatencyHistogram95th=0.00,
fsPreadLatencyHistogram99th=0.00, fsPreadLatencyHistogram999th=0.00,
fsWriteLatencyHistogramMean=0.00, fsWriteLatencyHistogramCount=0.00,
fsWriteLatencyHistogramMedian=0.00, fsWriteLatencyHistogram75th=0.00,
fsWriteLatencyHistogram95th=0.00, fsWriteLatencyHistogram99th=0.00,
fsWriteLatencyHistogram999th=0.00
{code}
I do have very low hdfsBlocksLocalityIndex ( 25) on ALL RS in my cluster. I
restarted cluster several times but the locality is still very poor. It seems
that smart algorithm of region assignments is not 100% correct?
0.94.6
Make region assignment smarter when regions are re-enabled.
---
Key: HBASE-6143
URL: https://issues.apache.org/jira/browse/HBASE-6143
Project: HBase
Issue Type: Improvement
Reporter: Elliott Clark
Assignee: Ted Yu
Priority: Critical
Fix For: 0.95.2
Attachments: 6143-v1.txt, 6143-v2.txt, 6143-v3.txt,
HBASE-6143-0.patch, HBASE-6143-1.patch
Right now a random region server is picked when re-enabling a table. This
could be much smarter.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6143) Make region assignment smarter when regions are re-enabled.
[ https://issues.apache.org/jira/browse/HBASE-6143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13723470#comment-13723470 ] Lars Hofhansl commented on HBASE-6143: -- Regions are not rebalanced based on locality (currently - but see HBASE-4191). The only way to restore locality (after it was lost) is to trigger compactions, so that data gets rewritten locally. Facebook also worked on HDFS level placement groups in order to improve locality (I do not think this is in any official Hadoop release, though). Make region assignment smarter when regions are re-enabled. --- Key: HBASE-6143 URL: https://issues.apache.org/jira/browse/HBASE-6143 Project: HBase Issue Type: Improvement Reporter: Elliott Clark Assignee: Ted Yu Priority: Critical Fix For: 0.95.2 Attachments: 6143-v1.txt, 6143-v2.txt, 6143-v3.txt, HBASE-6143-0.patch, HBASE-6143-1.patch Right now a random region server is picked when re-enabling a table. This could be much smarter. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6143) Make region assignment smarter when regions are re-enabled.
[ https://issues.apache.org/jira/browse/HBASE-6143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13723475#comment-13723475 ] Elliott Clark commented on HBASE-6143: -- bq.Regions are not rebalanced based on locality Trunk should already have locality awareness built in. Since not every file will be created on the same set of datanodes it will be hard to have 100% until the favored nodes functionality is in a shipped version of HDFS. https://github.com/apache/hbase/blob/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/StochasticLoadBalancer.java#L524 bq.0.94.6 HBASE-9080 was opened up for 0.94 branch, to deal with the full restart case. But like [~lhofhansl] said in 0.94 the balancer doesn't know about locality so the only way to get it for sure is to have compactions run (either minor or major). Make region assignment smarter when regions are re-enabled. --- Key: HBASE-6143 URL: https://issues.apache.org/jira/browse/HBASE-6143 Project: HBase Issue Type: Improvement Reporter: Elliott Clark Assignee: Ted Yu Priority: Critical Fix For: 0.95.2 Attachments: 6143-v1.txt, 6143-v2.txt, 6143-v3.txt, HBASE-6143-0.patch, HBASE-6143-1.patch Right now a random region server is picked when re-enabling a table. This could be much smarter. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6143) Make region assignment smarter when regions are re-enabled.
[ https://issues.apache.org/jira/browse/HBASE-6143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13723211#comment-13723211 ] Lars Hofhansl commented on HBASE-6143: -- Looking at the code again. I think Ted's patch is good. I didn't realize the RegionPlans are cached in the AssignmentManager. So +1 for v3 for 0.94 from me. We should have separate jiras for 0.94 and 0.95+. Should we leave this for 0.95 and make a new one for 0.94 or the other way 'round? Make region assignment smarter when regions are re-enabled. --- Key: HBASE-6143 URL: https://issues.apache.org/jira/browse/HBASE-6143 Project: HBase Issue Type: Improvement Reporter: Elliott Clark Assignee: Ted Yu Priority: Critical Fix For: 0.95.2 Attachments: 6143-v1.txt, 6143-v2.txt, 6143-v3.txt, HBASE-6143-0.patch Right now a random region server is picked when re-enabling a table. This could be much smarter. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6143) Make region assignment smarter when regions are re-enabled.
[ https://issues.apache.org/jira/browse/HBASE-6143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13723215#comment-13723215 ] Ted Yu commented on HBASE-6143: --- I would say open a new JIRA for 0.94 and link that JIRA to this one. The subject of this JIRA implies that more can be done in this subject area. Make region assignment smarter when regions are re-enabled. --- Key: HBASE-6143 URL: https://issues.apache.org/jira/browse/HBASE-6143 Project: HBase Issue Type: Improvement Reporter: Elliott Clark Assignee: Ted Yu Priority: Critical Fix For: 0.95.2 Attachments: 6143-v1.txt, 6143-v2.txt, 6143-v3.txt, HBASE-6143-0.patch Right now a random region server is picked when re-enabling a table. This could be much smarter. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6143) Make region assignment smarter when regions are re-enabled.
[ https://issues.apache.org/jira/browse/HBASE-6143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13723216#comment-13723216 ] Lars Hofhansl commented on HBASE-6143: -- Need to add a test. Make region assignment smarter when regions are re-enabled. --- Key: HBASE-6143 URL: https://issues.apache.org/jira/browse/HBASE-6143 Project: HBase Issue Type: Improvement Reporter: Elliott Clark Assignee: Ted Yu Priority: Critical Fix For: 0.95.2 Attachments: 6143-v1.txt, 6143-v2.txt, 6143-v3.txt, HBASE-6143-0.patch Right now a random region server is picked when re-enabling a table. This could be much smarter. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6143) Make region assignment smarter when regions are re-enabled.
[
https://issues.apache.org/jira/browse/HBASE-6143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13723220#comment-13723220
]
Ted Yu commented on HBASE-6143:
---
@Lars:
{code}
+ public void testEnableTableRetainAssignment() throws IOException,
InterruptedException {
{code}
Can you let me know what other test should be added beyond the one above ?
Thanks
Make region assignment smarter when regions are re-enabled.
---
Key: HBASE-6143
URL: https://issues.apache.org/jira/browse/HBASE-6143
Project: HBase
Issue Type: Improvement
Reporter: Elliott Clark
Assignee: Ted Yu
Priority: Critical
Fix For: 0.95.2
Attachments: 6143-v1.txt, 6143-v2.txt, 6143-v3.txt, HBASE-6143-0.patch
Right now a random region server is picked when re-enabling a table. This
could be much smarter.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6143) Make region assignment smarter when regions are re-enabled.
[ https://issues.apache.org/jira/browse/HBASE-6143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13723227#comment-13723227 ] Lars Hofhansl commented on HBASE-6143: -- [~yuzhih...@gmail.com] Sorry, missed the test namechange. Also turns out my previous comment was wrong again. hbase.master.enabletable.roundrobin is false by default, so the current pass would always use the single assignment in 0.94. Wanna create a 0.94 issue, Ted, or want me to do that? Make region assignment smarter when regions are re-enabled. --- Key: HBASE-6143 URL: https://issues.apache.org/jira/browse/HBASE-6143 Project: HBase Issue Type: Improvement Reporter: Elliott Clark Assignee: Ted Yu Priority: Critical Fix For: 0.95.2 Attachments: 6143-v1.txt, 6143-v2.txt, 6143-v3.txt, HBASE-6143-0.patch, HBASE-6143-1.patch Right now a random region server is picked when re-enabling a table. This could be much smarter. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6143) Make region assignment smarter when regions are re-enabled.
[ https://issues.apache.org/jira/browse/HBASE-6143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13723229#comment-13723229 ] Lars Hofhansl commented on HBASE-6143: -- [~eclark] *Love* all that remove code! Will digest the path a bit more. Make region assignment smarter when regions are re-enabled. --- Key: HBASE-6143 URL: https://issues.apache.org/jira/browse/HBASE-6143 Project: HBase Issue Type: Improvement Reporter: Elliott Clark Assignee: Ted Yu Priority: Critical Fix For: 0.95.2 Attachments: 6143-v1.txt, 6143-v2.txt, 6143-v3.txt, HBASE-6143-0.patch, HBASE-6143-1.patch Right now a random region server is picked when re-enabling a table. This could be much smarter. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6143) Make region assignment smarter when regions are re-enabled.
[ https://issues.apache.org/jira/browse/HBASE-6143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13723230#comment-13723230 ] Ted Yu commented on HBASE-6143: --- @Lars: I logged HBASE-9080 for 0.94 branch. Make region assignment smarter when regions are re-enabled. --- Key: HBASE-6143 URL: https://issues.apache.org/jira/browse/HBASE-6143 Project: HBase Issue Type: Improvement Reporter: Elliott Clark Assignee: Ted Yu Priority: Critical Fix For: 0.95.2 Attachments: 6143-v1.txt, 6143-v2.txt, 6143-v3.txt, HBASE-6143-0.patch, HBASE-6143-1.patch Right now a random region server is picked when re-enabling a table. This could be much smarter. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6143) Make region assignment smarter when regions are re-enabled.
[
https://issues.apache.org/jira/browse/HBASE-6143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13723302#comment-13723302
]
Hadoop QA commented on HBASE-6143:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12594831/HBASE-6143-1.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 20 new
or modified tests.
{color:green}+1 hadoop1.0{color}. The patch compiles against the hadoop
1.0 profile.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 1
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 lineLengths{color}. The patch introduces lines longer than
100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.regionserver.TestRegionMergeTransactionOnCluster
org.apache.hadoop.hbase.client.TestAdmin
org.apache.hadoop.hbase.master.TestAssignmentManagerOnCluster
org.apache.hadoop.hbase.TestDrainingServer
org.apache.hadoop.hbase.snapshot.TestFlushSnapshotFromClient
org.apache.hadoop.hbase.client.TestFromClientSide
org.apache.hadoop.hbase.master.TestRegionPlacement
org.apache.hadoop.hbase.client.TestFromClientSideWithCoprocessor
org.apache.hadoop.hbase.security.access.TestAccessController
org.apache.hadoop.hbase.master.TestDistributedLogSplitting
{color:red}-1 core zombie tests{color}. There are 3 zombie test(s):
at org.apache.hadoop.hbase.client.TestHCM.testMulti(TestHCM.java:771)
at
org.apache.hadoop.hbase.regionserver.wal.TestWALReplay.testReplayEditsAfterRegionMovedWithMultiCF(TestWALReplay.java:190)
at
org.apache.hadoop.hbase.TestZooKeeper.testRegionAssignmentAfterMasterRecoveryDueToZKExpiry(TestZooKeeper.java:488)
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/6516//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6516//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6516//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6516//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6516//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6516//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6516//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6516//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6516//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/6516//console
This message is automatically generated.
Make region assignment smarter when regions are re-enabled.
---
Key: HBASE-6143
URL: https://issues.apache.org/jira/browse/HBASE-6143
Project: HBase
Issue Type: Improvement
Reporter: Elliott Clark
Assignee: Ted Yu
Priority: Critical
Fix For: 0.95.2
Attachments: 6143-v1.txt, 6143-v2.txt, 6143-v3.txt,
HBASE-6143-0.patch, HBASE-6143-1.patch
Right now a random region server is picked when re-enabling a table. This
could be much smarter.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6143) Make region assignment smarter when regions are re-enabled.
[ https://issues.apache.org/jira/browse/HBASE-6143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13722066#comment-13722066 ] Lars Hofhansl commented on HBASE-6143: -- We should probably be doing different patches in 0.94 and 0.95+. Generally, loosing data locality after a simple disable/enable is absolutely unacceptable! Make region assignment smarter when regions are re-enabled. --- Key: HBASE-6143 URL: https://issues.apache.org/jira/browse/HBASE-6143 Project: HBase Issue Type: Improvement Reporter: Elliott Clark Assignee: Ted Yu Priority: Critical Attachments: 6143-v1.txt, 6143-v2.txt, 6143-v3.txt, HBASE-6143-0.patch Right now a random region server is picked when re-enabling a table. This could be much smarter. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6143) Make region assignment smarter when regions are re-enabled.
[ https://issues.apache.org/jira/browse/HBASE-6143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13722074#comment-13722074 ] Lars Hofhansl commented on HBASE-6143: -- In 0.94 I still think it would be best to just do the same that AssignmentManager.joinCluster() does, namely eventually calling AssignmentManager.assign(MapHRegionInfo, ServerName regions). If there's a agreement around that I'll come up with a patch. Make region assignment smarter when regions are re-enabled. --- Key: HBASE-6143 URL: https://issues.apache.org/jira/browse/HBASE-6143 Project: HBase Issue Type: Improvement Reporter: Elliott Clark Assignee: Ted Yu Priority: Critical Fix For: 0.95.2 Attachments: 6143-v1.txt, 6143-v2.txt, 6143-v3.txt, HBASE-6143-0.patch Right now a random region server is picked when re-enabling a table. This could be much smarter. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6143) Make region assignment smarter when regions are re-enabled.
[
https://issues.apache.org/jira/browse/HBASE-6143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13719293#comment-13719293
]
Hadoop QA commented on HBASE-6143:
--
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12594104/6143-v3.txt
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified tests.
{color:green}+1 hadoop1.0{color}. The patch compiles against the hadoop
1.0 profile.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/6460//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6460//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6460//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6460//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6460//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6460//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6460//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6460//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6460//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/6460//console
This message is automatically generated.
Make region assignment smarter when regions are re-enabled.
---
Key: HBASE-6143
URL: https://issues.apache.org/jira/browse/HBASE-6143
Project: HBase
Issue Type: Improvement
Reporter: Elliott Clark
Assignee: Ted Yu
Attachments: 6143-v1.txt, 6143-v2.txt, 6143-v3.txt
Right now a random region server is picked when re-enabling a table. This
could be much smarter.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6143) Make region assignment smarter when regions are re-enabled.
[ https://issues.apache.org/jira/browse/HBASE-6143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13719771#comment-13719771 ] Jimmy Xiang commented on HBASE-6143: [~saint@gmail.com], in trunk and 0.95, bulk assigner is safe. Make region assignment smarter when regions are re-enabled. --- Key: HBASE-6143 URL: https://issues.apache.org/jira/browse/HBASE-6143 Project: HBase Issue Type: Improvement Reporter: Elliott Clark Assignee: Ted Yu Attachments: 6143-v1.txt, 6143-v2.txt, 6143-v3.txt Right now a random region server is picked when re-enabling a table. This could be much smarter. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6143) Make region assignment smarter when regions are re-enabled.
[ https://issues.apache.org/jira/browse/HBASE-6143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13720264#comment-13720264 ] Elliott Clark commented on HBASE-6143: -- It's really raw and I doubt it even runs yet. But I think the simplification of the code is worth it. Make region assignment smarter when regions are re-enabled. --- Key: HBASE-6143 URL: https://issues.apache.org/jira/browse/HBASE-6143 Project: HBase Issue Type: Improvement Reporter: Elliott Clark Assignee: Ted Yu Attachments: 6143-v1.txt, 6143-v2.txt, 6143-v3.txt, HBASE-6143-0.patch Right now a random region server is picked when re-enabling a table. This could be much smarter. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6143) Make region assignment smarter when regions are re-enabled.
[ https://issues.apache.org/jira/browse/HBASE-6143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13720268#comment-13720268 ] stack commented on HBASE-6143: -- Did a skim. I like all the removes and the emergence of a simple pattern all follow when they want to do region moving. Let me know when you want a closer review. Make region assignment smarter when regions are re-enabled. --- Key: HBASE-6143 URL: https://issues.apache.org/jira/browse/HBASE-6143 Project: HBase Issue Type: Improvement Reporter: Elliott Clark Assignee: Ted Yu Attachments: 6143-v1.txt, 6143-v2.txt, 6143-v3.txt, HBASE-6143-0.patch Right now a random region server is picked when re-enabling a table. This could be much smarter. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6143) Make region assignment smarter when regions are re-enabled.
[
https://issues.apache.org/jira/browse/HBASE-6143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13720366#comment-13720366
]
Hadoop QA commented on HBASE-6143:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12594295/HBASE-6143-0.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 20 new
or modified tests.
{color:green}+1 hadoop1.0{color}. The patch compiles against the hadoop
1.0 profile.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 1
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 lineLengths{color}. The patch introduces lines longer than
100
{color:red}-1 site{color}. The patch appears to cause mvn site goal to
fail.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.coprocessor.TestMasterObserver
org.apache.hadoop.hbase.regionserver.TestRegionMergeTransactionOnCluster
org.apache.hadoop.hbase.security.access.TestAccessController
org.apache.hadoop.hbase.master.TestRegionPlacement
org.apache.hadoop.hbase.TestRegionRebalancing
org.apache.hadoop.hbase.master.TestAssignmentManagerOnCluster
org.apache.hadoop.hbase.regionserver.TestHRegionOnCluster
org.apache.hadoop.hbase.snapshot.TestFlushSnapshotFromClient
org.apache.hadoop.hbase.regionserver.TestSplitTransactionOnCluster
org.apache.hadoop.hbase.master.TestRollingRestart
{color:red}-1 core zombie tests{color}. There are 10 zombie test(s):
at
org.apache.hadoop.hbase.client.TestFromClientSide.testScanMetrics(TestFromClientSide.java:4740)
at
org.apache.hadoop.hbase.util.TestHBaseFsck.testValidLingeringSplitParent(TestHBaseFsck.java:1283)
at
org.apache.hadoop.hbase.client.TestScannersFromClientSide.testScanBatch(TestScannersFromClientSide.java:131)
at
org.apache.hadoop.hbase.client.TestHCM.testRegionCaching(TestHCM.java:365)
at
org.apache.hadoop.hbase.regionserver.wal.TestWALReplay.testReplayEditsAfterRegionMovedWithMultiCF(TestWALReplay.java:190)
at
org.apache.hadoop.hbase.TestZooKeeper.testRegionAssignmentAfterMasterRecoveryDueToZKExpiry(TestZooKeeper.java:488)
at
org.apache.hadoop.hbase.client.TestFromClientSide.testScanMetrics(TestFromClientSide.java:4740)
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/6473//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6473//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6473//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6473//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6473//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6473//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6473//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6473//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6473//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/6473//console
This message is automatically generated.
Make region assignment smarter when regions are re-enabled.
---
Key: HBASE-6143
URL: https://issues.apache.org/jira/browse/HBASE-6143
Project: HBase
Issue Type: Improvement
Reporter: Elliott Clark
Assignee: Ted Yu
Attachments: 6143-v1.txt, 6143-v2.txt, 6143-v3.txt,
[jira] [Commented] (HBASE-6143) Make region assignment smarter when regions are re-enabled.
[ https://issues.apache.org/jira/browse/HBASE-6143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13718436#comment-13718436 ] Ted Yu commented on HBASE-6143: --- Note on patch v2: EnableTableHandler.retainAssignment was improperly named. I renamed it this.skipTableStateCheck to be consistent with the name of parameter passed to ctor. Make region assignment smarter when regions are re-enabled. --- Key: HBASE-6143 URL: https://issues.apache.org/jira/browse/HBASE-6143 Project: HBase Issue Type: Improvement Reporter: Elliott Clark Assignee: Elliott Clark Attachments: 6143-v1.txt, 6143-v2.txt Right now a random region server is picked when re-enabling a table. This could be much smarter. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6143) Make region assignment smarter when regions are re-enabled.
[
https://issues.apache.org/jira/browse/HBASE-6143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13718520#comment-13718520
]
Hadoop QA commented on HBASE-6143:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12593947/6143-v2.txt
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 hadoop1.0{color}. The patch compiles against the hadoop
1.0 profile.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.TestIOFencing
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/6450//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6450//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6450//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6450//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6450//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6450//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6450//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6450//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6450//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/6450//console
This message is automatically generated.
Make region assignment smarter when regions are re-enabled.
---
Key: HBASE-6143
URL: https://issues.apache.org/jira/browse/HBASE-6143
Project: HBase
Issue Type: Improvement
Reporter: Elliott Clark
Assignee: Ted Yu
Attachments: 6143-v1.txt, 6143-v2.txt
Right now a random region server is picked when re-enabling a table. This
could be much smarter.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6143) Make region assignment smarter when regions are re-enabled.
[
https://issues.apache.org/jira/browse/HBASE-6143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13719201#comment-13719201
]
Lars Hofhansl commented on HBASE-6143:
--
v2 *could* work. The code path for assignment after enabling is now quite
different, though.
Might still be better to call {{AssignmentManager.assign(MapHRegionInfo,
ServerName regions)}} {{BulkEnabler.populatePool}}, I think we can build the
map ahead of time.
This is the same call used by AssignmentManager.joinCluster(), so we'd use the
same code.
Make region assignment smarter when regions are re-enabled.
---
Key: HBASE-6143
URL: https://issues.apache.org/jira/browse/HBASE-6143
Project: HBase
Issue Type: Improvement
Reporter: Elliott Clark
Assignee: Ted Yu
Attachments: 6143-v1.txt, 6143-v2.txt
Right now a random region server is picked when re-enabling a table. This
could be much smarter.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6143) Make region assignment smarter when regions are re-enabled.
[ https://issues.apache.org/jira/browse/HBASE-6143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13719209#comment-13719209 ] stack commented on HBASE-6143: -- Unit test? Is bulk assigner safe? ([~jxiang]?) In the past, bulk assigner was known to not be safe and if we failed, because we were using it on startup only, then we'd just exit so user would retry. Jimmy may have fixed this. Make region assignment smarter when regions are re-enabled. --- Key: HBASE-6143 URL: https://issues.apache.org/jira/browse/HBASE-6143 Project: HBase Issue Type: Improvement Reporter: Elliott Clark Assignee: Ted Yu Attachments: 6143-v1.txt, 6143-v2.txt Right now a random region server is picked when re-enabling a table. This could be much smarter. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6143) Make region assignment smarter when regions are re-enabled.
[ https://issues.apache.org/jira/browse/HBASE-6143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13719226#comment-13719226 ] Lars Hofhansl commented on HBASE-6143: -- If we use AssignmentManager.assign(MapHRegionInfo, ServerName regions) we're using the same code as used in startup, hopefully that is safe. We absolutely would need a test for this. Make region assignment smarter when regions are re-enabled. --- Key: HBASE-6143 URL: https://issues.apache.org/jira/browse/HBASE-6143 Project: HBase Issue Type: Improvement Reporter: Elliott Clark Assignee: Ted Yu Attachments: 6143-v1.txt, 6143-v2.txt Right now a random region server is picked when re-enabling a table. This could be much smarter. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6143) Make region assignment smarter when regions are re-enabled.
[ https://issues.apache.org/jira/browse/HBASE-6143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13717576#comment-13717576 ] Vladimir Rodionov commented on HBASE-6143: -- This is Ted's quick analysis on this issue: http://mail-archives.apache.org/mod_mbox/hbase-dev/201307.mbox/%3CCALte62yCN7RKcgHxr6nNy6A7UyC8g-uztt7epu%3D9OevmbSQE1A%40mail.gmail.com%3E The full thread starts here: http://mail-archives.apache.org/mod_mbox/hbase-dev/201307.mbox/%3cdc5ebe7f3610eb4ca5c7e92d76873e1517ecf1d...@exchange2007.carrieriq.com%3E Make region assignment smarter when regions are re-enabled. --- Key: HBASE-6143 URL: https://issues.apache.org/jira/browse/HBASE-6143 Project: HBase Issue Type: Improvement Reporter: Elliott Clark Assignee: Elliott Clark Right now a random region server is picked when re-enabling a table. This could be much smarter. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6143) Make region assignment smarter when regions are re-enabled.
[ https://issues.apache.org/jira/browse/HBASE-6143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13717583#comment-13717583 ] Elliott Clark commented on HBASE-6143: -- I pretty vehemently disagree that we need to add more apis on the balancer. What we really need to do is simplify and make sure that everything uses the same code path (that code path will include locality and placement). Make region assignment smarter when regions are re-enabled. --- Key: HBASE-6143 URL: https://issues.apache.org/jira/browse/HBASE-6143 Project: HBase Issue Type: Improvement Reporter: Elliott Clark Assignee: Elliott Clark Right now a random region server is picked when re-enabling a table. This could be much smarter. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6143) Make region assignment smarter when regions are re-enabled.
[ https://issues.apache.org/jira/browse/HBASE-6143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13717951#comment-13717951 ] Lars Hofhansl commented on HBASE-6143: -- I agree. We should just default to attempting to keep the previous region assignment. No need to make that optional via a new API. Make region assignment smarter when regions are re-enabled. --- Key: HBASE-6143 URL: https://issues.apache.org/jira/browse/HBASE-6143 Project: HBase Issue Type: Improvement Reporter: Elliott Clark Assignee: Elliott Clark Right now a random region server is picked when re-enabling a table. This could be much smarter. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6143) Make region assignment smarter when regions are re-enabled.
[ https://issues.apache.org/jira/browse/HBASE-6143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13717962#comment-13717962 ] Lars Hofhansl commented on HBASE-6143: -- Both I and Vladimir said that much on the email-thread. Ted's analysis is only 1/2 the story, though. The retainAssignment flag is tied to skipTableStateCheck right now, and only enabled during recovery. It is a shame that we go through some length to keep assignments during a full cluster bounce, but drop the ball for a simple disable/enable cycle. Make region assignment smarter when regions are re-enabled. --- Key: HBASE-6143 URL: https://issues.apache.org/jira/browse/HBASE-6143 Project: HBase Issue Type: Improvement Reporter: Elliott Clark Assignee: Elliott Clark Attachments: 6143-v1.txt Right now a random region server is picked when re-enabling a table. This could be much smarter. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6143) Make region assignment smarter when regions are re-enabled.
[ https://issues.apache.org/jira/browse/HBASE-6143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13717964#comment-13717964 ] Lars Hofhansl commented on HBASE-6143: -- I do not think it is going to be that simple, Ted. Simply changing that flag has other implication, too. For example we'd no longer check for concurrent enable request neither would we now check for non-existent tables (among others). Make region assignment smarter when regions are re-enabled. --- Key: HBASE-6143 URL: https://issues.apache.org/jira/browse/HBASE-6143 Project: HBase Issue Type: Improvement Reporter: Elliott Clark Assignee: Elliott Clark Attachments: 6143-v1.txt Right now a random region server is picked when re-enabling a table. This could be much smarter. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6143) Make region assignment smarter when regions are re-enabled.
[ https://issues.apache.org/jira/browse/HBASE-6143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13294618#comment-13294618 ] Nicolas Spiegelberg commented on HBASE-6143: The default behavior should most likely take load balancing into account (See HBASE-4191). In FB, our master has a concept of 'favored nodes' that are assigned upon Region creation and map directly to the HDFS replication pipeline. Make region assignment smarter when regions are re-enabled. --- Key: HBASE-6143 URL: https://issues.apache.org/jira/browse/HBASE-6143 Project: HBase Issue Type: Improvement Reporter: Elliott Clark Assignee: Elliott Clark Right now a random region server is picked when re-enabling a table. This could be much smarter. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6143) Make region assignment smarter when regions are re-enabled.
[ https://issues.apache.org/jira/browse/HBASE-6143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13294648#comment-13294648 ] Elliott Clark commented on HBASE-6143: -- I agree. Though sometimes choosing a balance of the regions can be an expensive operation so the balancer should provide an api for a cheap balance. Make region assignment smarter when regions are re-enabled. --- Key: HBASE-6143 URL: https://issues.apache.org/jira/browse/HBASE-6143 Project: HBase Issue Type: Improvement Reporter: Elliott Clark Assignee: Elliott Clark Right now a random region server is picked when re-enabling a table. This could be much smarter. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
