[jira] [Commented] (HBASE-14197) TestRegionServerHostname#testInvalidRegionServerHostnameAbortsServer fails in Jenkins
[
https://issues.apache.org/jira/browse/HBASE-14197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14662824#comment-14662824
]
Hudson commented on HBASE-14197:
SUCCESS: Integrated in HBase-1.3 #96 (See
[https://builds.apache.org/job/HBase-1.3/96/])
HBASE-14197
TestRegionServerHostname#testInvalidRegionServerHostnameAbortsServer fails in
Jenkins (tedyu: rev 05a9638085bb05811b8f805667c3bca478144c25)
*
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionServerHostname.java
TestRegionServerHostname#testInvalidRegionServerHostnameAbortsServer fails in
Jenkins
-
Key: HBASE-14197
URL: https://issues.apache.org/jira/browse/HBASE-14197
Project: HBase
Issue Type: Test
Reporter: Ted Yu
Assignee: Ted Yu
Priority: Minor
Fix For: 2.0.0, 1.3.0
Attachments: 14197-v1.txt, 14197-v2.txt
The following test failure can be observed in various recent Jenkins builds:
{code}
testInvalidRegionServerHostnameAbortsServer(org.apache.hadoop.hbase.regionserver.TestRegionServerHostname)
Time elapsed: 9.344 sec FAILURE!
java.lang.AssertionError: null
at org.junit.Assert.fail(Assert.java:86)
at org.junit.Assert.assertTrue(Assert.java:41)
at org.junit.Assert.assertTrue(Assert.java:52)
at
org.apache.hadoop.hbase.regionserver.TestRegionServerHostname.testInvalidRegionServerHostnameAbortsServer(TestRegionServerHostname.java:65)
{code}
The test inspects exception message and looks for specific sentence, making
it vulnerable to environment changes.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-14144) Bloomfilter path to work with Byte buffered cells
[ https://issues.apache.org/jira/browse/HBASE-14144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-14144: --- Status: Open (was: Patch Available) Bloomfilter path to work with Byte buffered cells - Key: HBASE-14144 URL: https://issues.apache.org/jira/browse/HBASE-14144 Project: HBase Issue Type: Sub-task Components: regionserver, Scanners Reporter: ramkrishna.s.vasudevan Assignee: ramkrishna.s.vasudevan Fix For: 2.0.0 Attachments: HBASE-14144.patch, HBASE-14144_1.patch This JIRA is to check if there will be a need to make the bloom filters to work with ByteBuffer cells. During POC this path created lot of duplicated code but considering other refactorings done in this path may lead to less duplication. This JIRA is a placeholder. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14144) Bloomfilter path to work with Byte buffered cells
[ https://issues.apache.org/jira/browse/HBASE-14144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-14144: --- Attachment: HBASE-14144_2.patch Updated patch with rename. For QA. Bloomfilter path to work with Byte buffered cells - Key: HBASE-14144 URL: https://issues.apache.org/jira/browse/HBASE-14144 Project: HBase Issue Type: Sub-task Components: regionserver, Scanners Reporter: ramkrishna.s.vasudevan Assignee: ramkrishna.s.vasudevan Fix For: 2.0.0 Attachments: HBASE-14144.patch, HBASE-14144_1.patch, HBASE-14144_2.patch This JIRA is to check if there will be a need to make the bloom filters to work with ByteBuffer cells. During POC this path created lot of duplicated code but considering other refactorings done in this path may lead to less duplication. This JIRA is a placeholder. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14197) TestRegionServerHostname#testInvalidRegionServerHostnameAbortsServer fails in Jenkins
[
https://issues.apache.org/jira/browse/HBASE-14197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14662839#comment-14662839
]
Hudson commented on HBASE-14197:
FAILURE: Integrated in HBase-1.3-IT #78 (See
[https://builds.apache.org/job/HBase-1.3-IT/78/])
HBASE-14197
TestRegionServerHostname#testInvalidRegionServerHostnameAbortsServer fails in
Jenkins (tedyu: rev 05a9638085bb05811b8f805667c3bca478144c25)
*
hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionServerHostname.java
TestRegionServerHostname#testInvalidRegionServerHostnameAbortsServer fails in
Jenkins
-
Key: HBASE-14197
URL: https://issues.apache.org/jira/browse/HBASE-14197
Project: HBase
Issue Type: Test
Reporter: Ted Yu
Assignee: Ted Yu
Priority: Minor
Fix For: 2.0.0, 1.3.0
Attachments: 14197-v1.txt, 14197-v2.txt
The following test failure can be observed in various recent Jenkins builds:
{code}
testInvalidRegionServerHostnameAbortsServer(org.apache.hadoop.hbase.regionserver.TestRegionServerHostname)
Time elapsed: 9.344 sec FAILURE!
java.lang.AssertionError: null
at org.junit.Assert.fail(Assert.java:86)
at org.junit.Assert.assertTrue(Assert.java:41)
at org.junit.Assert.assertTrue(Assert.java:52)
at
org.apache.hadoop.hbase.regionserver.TestRegionServerHostname.testInvalidRegionServerHostnameAbortsServer(TestRegionServerHostname.java:65)
{code}
The test inspects exception message and looks for specific sentence, making
it vulnerable to environment changes.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13706) CoprocessorClassLoader should not exempt Hive classes
[
https://issues.apache.org/jira/browse/HBASE-13706?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14662886#comment-14662886
]
Hadoop QA commented on HBASE-13706:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12749405/HBASE-13706-branch-1.patch
against branch-1 branch at commit d4fc7cf2ba0d06b040dad435ab72366c4fd44116.
ATTACHMENT ID: 12749405
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.7.0)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 protoc{color}. The applied patch does not increase the
total number of protoc compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 checkstyle{color}. The applied patch does not increase the
total number of checkstyle errors
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn post-site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/15010//testReport/
Release Findbugs (version 2.0.3)warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/15010//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/15010//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/15010//console
This message is automatically generated.
CoprocessorClassLoader should not exempt Hive classes
-
Key: HBASE-13706
URL: https://issues.apache.org/jira/browse/HBASE-13706
Project: HBase
Issue Type: Bug
Components: Coprocessors
Affects Versions: 2.0.0, 1.0.1, 1.1.0, 0.98.12
Reporter: Jerry He
Assignee: Jerry He
Priority: Minor
Fix For: 2.0.0, 1.2.0, 1.3.0
Attachments: HBASE-13706-branch-1.patch, HBASE-13706-master-v2.patch,
HBASE-13706-master-v2.patch, HBASE-13706.patch
CoprocessorClassLoader is used to load classes from the coprocessor jar.
Certain classes are exempt from being loaded by this ClassLoader, which means
they will be ignored in the coprocessor jar, but loaded from parent classpath
instead.
One problem is that we categorically exempt org.apache.hadoop.
But it happens that Hive packages start with org.apache.hadoop.
There is no reason to exclude hive classes from theCoprocessorClassLoader.
HBase does not even include Hive jars.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14144) Bloomfilter path to work with Byte buffered cells
[
https://issues.apache.org/jira/browse/HBASE-14144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14662893#comment-14662893
]
Hadoop QA commented on HBASE-14144:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12749407/HBASE-14144_2.patch
against master branch at commit d4fc7cf2ba0d06b040dad435ab72366c4fd44116.
ATTACHMENT ID: 12749407
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.7.0)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 protoc{color}. The applied patch does not increase the
total number of protoc compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 checkstyle{color}. The applied patch does not increase the
total number of checkstyle errors
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn post-site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/15011//testReport/
Release Findbugs (version 2.0.3)warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/15011//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/15011//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/15011//console
This message is automatically generated.
Bloomfilter path to work with Byte buffered cells
-
Key: HBASE-14144
URL: https://issues.apache.org/jira/browse/HBASE-14144
Project: HBase
Issue Type: Sub-task
Components: regionserver, Scanners
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan
Fix For: 2.0.0
Attachments: HBASE-14144.patch, HBASE-14144_1.patch,
HBASE-14144_2.patch
This JIRA is to check if there will be a need to make the bloom filters to
work with ByteBuffer cells. During POC this path created lot of duplicated
code but considering other refactorings done in this path may lead to less
duplication. This JIRA is a placeholder.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-13376) Improvements to Stochastic load balancer
[ https://issues.apache.org/jira/browse/HBASE-13376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-13376: --- Status: Patch Available (was: Open) Improvements to Stochastic load balancer Key: HBASE-13376 URL: https://issues.apache.org/jira/browse/HBASE-13376 Project: HBase Issue Type: Improvement Components: Balancer Affects Versions: 0.98.12, 1.0.0 Reporter: Vandana Ayyalasomayajula Assignee: Vandana Ayyalasomayajula Priority: Minor Attachments: 13376-v2.txt, HBASE-13376.patch, HBASE-13376_0.98.txt, HBASE-13376_0.txt, HBASE-13376_1.txt, HBASE-13376_1_1.txt, HBASE-13376_2.patch, HBASE-13376_2_branch-1.patch, HBASE-13376_98.patch, HBASE-13376_branch-1.patch There are two things this jira tries to address: 1. The locality picker in the stochastic balancer does not pick regions with least locality as candidates for swap/move. So when any user configures locality cost in the configs, the balancer does not always seems to move regions with bad locality. 2. When a cluster has equal number of loaded regions, it always picks the first one. It should pick a random region on one of the equally loaded servers. This improves a chance of finding a good candidate, when load picker is invoked several times. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14144) Bloomfilter path to work with Byte buffered cells
[ https://issues.apache.org/jira/browse/HBASE-14144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14663034#comment-14663034 ] Ted Yu commented on HBASE-14144: +1 Bloomfilter path to work with Byte buffered cells - Key: HBASE-14144 URL: https://issues.apache.org/jira/browse/HBASE-14144 Project: HBase Issue Type: Sub-task Components: regionserver, Scanners Reporter: ramkrishna.s.vasudevan Assignee: ramkrishna.s.vasudevan Fix For: 2.0.0 Attachments: HBASE-14144.patch, HBASE-14144_1.patch, HBASE-14144_2.patch This JIRA is to check if there will be a need to make the bloom filters to work with ByteBuffer cells. During POC this path created lot of duplicated code but considering other refactorings done in this path may lead to less duplication. This JIRA is a placeholder. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14190) Assign system tables ahead of user region assignment
[
https://issues.apache.org/jira/browse/HBASE-14190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-14190:
---
Attachment: 14190-v4.txt
Patch v4 calls UserProvider.isHBaseSecurityEnabled() to see if ACL and labels
table need to be assigned.
This is to avoid unnecessary calls to MetaTableAccessor.tableExists().
Assign system tables ahead of user region assignment
Key: HBASE-14190
URL: https://issues.apache.org/jira/browse/HBASE-14190
Project: HBase
Issue Type: Bug
Reporter: Ted Yu
Assignee: Ted Yu
Priority: Critical
Attachments: 14190-v1.txt, 14190-v2.txt, 14190-v3.txt, 14190-v3.txt,
14190-v3.txt, 14190-v4.txt
Currently the namespace table region is assigned like user regions.
I spent several hours working with a customer where master couldn't finish
initialization.
Even though master was restarted quite a few times, it went down with the
following:
{code}
2015-08-05 17:16:57,530 FATAL [hdpmaster1:6.activeMasterManager]
master.HMaster: Master server abort: loaded coprocessors are: []
2015-08-05 17:16:57,530 FATAL [hdpmaster1:6.activeMasterManager]
master.HMaster: Unhandled exception. Starting shutdown.
java.io.IOException: Timedout 30ms waiting for namespace table to be
assigned
at
org.apache.hadoop.hbase.master.TableNamespaceManager.start(TableNamespaceManager.java:104)
at org.apache.hadoop.hbase.master.HMaster.initNamespace(HMaster.java:985)
at
org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:779)
at org.apache.hadoop.hbase.master.HMaster.access$500(HMaster.java:182)
at org.apache.hadoop.hbase.master.HMaster$1.run(HMaster.java:1646)
at java.lang.Thread.run(Thread.java:744)
{code}
During previous run(s), namespace table was created, hence leaving an entry
in hbase:meta.
The following if block in TableNamespaceManager#start() was skipped:
{code}
if (!MetaTableAccessor.tableExists(masterServices.getConnection(),
TableName.NAMESPACE_TABLE_NAME)) {
{code}
TableNamespaceManager#start() spins, waiting for namespace region to be
assigned.
There was issue in master assigning user regions.
We tried issuing 'assign' command from hbase shell which didn't work because
of the following check in MasterRpcServices#assignRegion():
{code}
master.checkInitialized();
{code}
This scenario can be avoided if we assign hbase:namespace table after
hbase:meta is assigned but before user table region assignment.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13376) Improvements to Stochastic load balancer
[
https://issues.apache.org/jira/browse/HBASE-13376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14663042#comment-14663042
]
Hadoop QA commented on HBASE-13376:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12749386/HBASE-13376_2.patch
against master branch at commit d4fc7cf2ba0d06b040dad435ab72366c4fd44116.
ATTACHMENT ID: 12749386
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 6 new
or modified tests.
{color:red}-1 javac{color}. The patch appears to cause mvn compile goal to
fail with Hadoop version 2.4.0.
Compilation errors resume:
[ERROR] COMPILATION ERROR :
[ERROR]
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/StochasticLoadBalancer.java:[708,14]
cannot find symbol
[ERROR]
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/StochasticLoadBalancer.java:[725,32]
cannot find symbol
[ERROR]
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/StochasticLoadBalancer.java:[738,21]
cannot find symbol
[ERROR]
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/StochasticLoadBalancer.java:[742,21]
cannot find symbol
[ERROR]
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/StochasticLoadBalancer.java:[1188,31]
cannot find symbol
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-compiler-plugin:3.2:compile (default-compile) on
project hbase-server: Compilation failure: Compilation failure:
[ERROR]
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/StochasticLoadBalancer.java:[708,14]
cannot find symbol
[ERROR] symbol: method calculateRegionServerLocalities()
[ERROR] location: variable cluster of type
org.apache.hadoop.hbase.master.balancer.BaseLoadBalancer.Cluster
[ERROR]
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/StochasticLoadBalancer.java:[725,32]
cannot find symbol
[ERROR] symbol: method getLeastLoadedTopServerForRegion(int)
[ERROR] location: variable cluster of type
org.apache.hadoop.hbase.master.balancer.BaseLoadBalancer.Cluster
[ERROR]
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/StochasticLoadBalancer.java:[738,21]
cannot find symbol
[ERROR] symbol: method getLowestLocalityRegionServer()
[ERROR] location: variable cluster of type
org.apache.hadoop.hbase.master.balancer.BaseLoadBalancer.Cluster
[ERROR]
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/StochasticLoadBalancer.java:[742,21]
cannot find symbol
[ERROR] symbol: method getLowestLocalityRegionOnServer(int)
[ERROR] location: variable cluster of type
org.apache.hadoop.hbase.master.balancer.BaseLoadBalancer.Cluster
[ERROR]
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/StochasticLoadBalancer.java:[1188,31]
cannot find symbol
[ERROR] symbol: method getLocalityOfRegion(int,int)
[ERROR] location: variable cluster of type
org.apache.hadoop.hbase.master.balancer.BaseLoadBalancer.Cluster
[ERROR] - [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e
switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please
read the following articles:
[ERROR] [Help 1]
http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR] mvn goals -rf :hbase-server
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/15012//console
This message is automatically generated.
Improvements to Stochastic load balancer
Key: HBASE-13376
URL: https://issues.apache.org/jira/browse/HBASE-13376
Project: HBase
Issue Type: Improvement
Components: Balancer
Affects Versions: 1.0.0, 0.98.12
Reporter: Vandana Ayyalasomayajula
Assignee: Vandana Ayyalasomayajula
Priority: Minor
Attachments: 13376-v2.txt, HBASE-13376.patch, HBASE-13376_0.98.txt,
[jira] [Commented] (HBASE-14199) maven-remote-resources-plugin failure processing NOTICE.vm in hbase-assembly
[
https://issues.apache.org/jira/browse/HBASE-14199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14663056#comment-14663056
]
Sean Busbey commented on HBASE-14199:
-
My apologies that neither of my emailed comments came to this ticket. Also that
my notes in the velocity templates didn't make it more obvious that the issue
was a need to update supplemental models.
I'd like to find a way to fail the build with a meaningful message when a
license is missing so we don't slide so far again. Maybe use a property to
determine if it's a failure condition and the set it to true on Jenkins and in
the release profile?
maven-remote-resources-plugin failure processing NOTICE.vm in hbase-assembly
Key: HBASE-14199
URL: https://issues.apache.org/jira/browse/HBASE-14199
Project: HBase
Issue Type: Bug
Affects Versions: 0.98.14
Reporter: Andrew Purtell
Assignee: Andrew Purtell
Priority: Blocker
Fix For: 2.0.0, 0.98.14, 1.0.2, 1.2.0, 1.1.2, 1.3.0
Attachments: HBASE-14199-0.98.patch, HBASE-14199.patch
Only seen when building 0.98 with -Dhadoop.profile=1.1. Happens with both JDK
6 and 7.
{noformat}
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-remote-resources-plugin:1.5:process
(default) on project hbase-assembly: Error rendering velocity resource. Error
invoking method
'get(java.lang.Integer)' in java.util.ArrayList at META-INF/NOTICE.vm[line
275, column 22]:
InvocationTargetException: Index: 0, Size: 0 - [Help 1]
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HBASE-14200) Separate some subtests of TestStochasticLoadBalancer into TestStochasticLoadBalancer2
Ted Yu created HBASE-14200: -- Summary: Separate some subtests of TestStochasticLoadBalancer into TestStochasticLoadBalancer2 Key: HBASE-14200 URL: https://issues.apache.org/jira/browse/HBASE-14200 Project: HBase Issue Type: Test Reporter: Ted Yu Assignee: Ted Yu Priority: Minor More and more functionality is added to StochasticLoadBalancer , making TestStochasticLoadBalancer run longer. From https://builds.apache.org/job/PreCommit-HBASE-Build/15011/testReport/org.apache.hadoop.hbase.master.balancer/TestStochasticLoadBalancer/ where total runtime was 14 min, here are the longest subtests: testRegionReplicasOnLargeCluster: 1 min 34 sec testRegionReplicasOnMidCluster: 1 min 31 sec testRegionReplicasOnMidClusterHighReplication: 2 min testRegionReplicationOnMidClusterReplicationGreaterThanNumNodes: 2 min 25 sec This issue is to separate out the above subtests into TestStochasticLoadBalancer2, giving each of the tests around 7 min runtime. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14200) Separate some subtests of TestStochasticLoadBalancer into TestStochasticLoadBalancer2
[ https://issues.apache.org/jira/browse/HBASE-14200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-14200: --- Status: Patch Available (was: Open) Separate some subtests of TestStochasticLoadBalancer into TestStochasticLoadBalancer2 - Key: HBASE-14200 URL: https://issues.apache.org/jira/browse/HBASE-14200 Project: HBase Issue Type: Test Reporter: Ted Yu Assignee: Ted Yu Priority: Minor Attachments: 14200-v1.txt More and more functionality is added to StochasticLoadBalancer , making TestStochasticLoadBalancer run longer. From https://builds.apache.org/job/PreCommit-HBASE-Build/15011/testReport/org.apache.hadoop.hbase.master.balancer/TestStochasticLoadBalancer/ where total runtime was 14 min, here are the longest subtests: testRegionReplicasOnLargeCluster: 1 min 34 sec testRegionReplicasOnMidCluster: 1 min 31 sec testRegionReplicasOnMidClusterHighReplication: 2 min testRegionReplicationOnMidClusterReplicationGreaterThanNumNodes: 2 min 25 sec This issue is to separate out the above subtests into TestStochasticLoadBalancer2, giving each of the tests around 7 min runtime. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14200) Separate some subtests of TestStochasticLoadBalancer into TestStochasticLoadBalancer2
[ https://issues.apache.org/jira/browse/HBASE-14200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-14200: --- Attachment: 14200-v1.txt Separate some subtests of TestStochasticLoadBalancer into TestStochasticLoadBalancer2 - Key: HBASE-14200 URL: https://issues.apache.org/jira/browse/HBASE-14200 Project: HBase Issue Type: Test Reporter: Ted Yu Assignee: Ted Yu Priority: Minor Attachments: 14200-v1.txt More and more functionality is added to StochasticLoadBalancer , making TestStochasticLoadBalancer run longer. From https://builds.apache.org/job/PreCommit-HBASE-Build/15011/testReport/org.apache.hadoop.hbase.master.balancer/TestStochasticLoadBalancer/ where total runtime was 14 min, here are the longest subtests: testRegionReplicasOnLargeCluster: 1 min 34 sec testRegionReplicasOnMidCluster: 1 min 31 sec testRegionReplicasOnMidClusterHighReplication: 2 min testRegionReplicationOnMidClusterReplicationGreaterThanNumNodes: 2 min 25 sec This issue is to separate out the above subtests into TestStochasticLoadBalancer2, giving each of the tests around 7 min runtime. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14190) Assign system tables ahead of user region assignment
[
https://issues.apache.org/jira/browse/HBASE-14190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14663121#comment-14663121
]
Hadoop QA commented on HBASE-14190:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12749421/14190-v4.txt
against master branch at commit d4fc7cf2ba0d06b040dad435ab72366c4fd44116.
ATTACHMENT ID: 12749421
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.7.0)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 protoc{color}. The applied patch does not increase the
total number of protoc compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 checkstyle{color}. The applied patch does not increase the
total number of checkstyle errors
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn post-site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
{color:red}-1 core zombie tests{color}. There are 2 zombie test(s):
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/15013//testReport/
Release Findbugs (version 2.0.3)warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/15013//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/15013//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/15013//console
This message is automatically generated.
Assign system tables ahead of user region assignment
Key: HBASE-14190
URL: https://issues.apache.org/jira/browse/HBASE-14190
Project: HBase
Issue Type: Bug
Reporter: Ted Yu
Assignee: Ted Yu
Priority: Critical
Attachments: 14190-v1.txt, 14190-v2.txt, 14190-v3.txt, 14190-v3.txt,
14190-v3.txt, 14190-v4.txt
Currently the namespace table region is assigned like user regions.
I spent several hours working with a customer where master couldn't finish
initialization.
Even though master was restarted quite a few times, it went down with the
following:
{code}
2015-08-05 17:16:57,530 FATAL [hdpmaster1:6.activeMasterManager]
master.HMaster: Master server abort: loaded coprocessors are: []
2015-08-05 17:16:57,530 FATAL [hdpmaster1:6.activeMasterManager]
master.HMaster: Unhandled exception. Starting shutdown.
java.io.IOException: Timedout 30ms waiting for namespace table to be
assigned
at
org.apache.hadoop.hbase.master.TableNamespaceManager.start(TableNamespaceManager.java:104)
at org.apache.hadoop.hbase.master.HMaster.initNamespace(HMaster.java:985)
at
org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:779)
at org.apache.hadoop.hbase.master.HMaster.access$500(HMaster.java:182)
at org.apache.hadoop.hbase.master.HMaster$1.run(HMaster.java:1646)
at java.lang.Thread.run(Thread.java:744)
{code}
During previous run(s), namespace table was created, hence leaving an entry
in hbase:meta.
The following if block in TableNamespaceManager#start() was skipped:
{code}
if (!MetaTableAccessor.tableExists(masterServices.getConnection(),
TableName.NAMESPACE_TABLE_NAME)) {
{code}
TableNamespaceManager#start() spins, waiting for namespace region to be
assigned.
There was issue in master assigning user regions.
We tried issuing 'assign' command from hbase shell which didn't work because
of the following check in MasterRpcServices#assignRegion():
{code}
master.checkInitialized();
{code}
This scenario can be avoided if we assign hbase:namespace table after
hbase:meta is assigned but before user table region assignment.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14190) Assign system tables ahead of user region assignment
[
https://issues.apache.org/jira/browse/HBASE-14190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14663144#comment-14663144
]
Ted Yu commented on HBASE-14190:
{code}
Fetching the console output from the URL
Printing hanging tests
Printing Failing tests
Failing test :
org.apache.hadoop.hbase.master.procedure.TestWALProcedureStoreOnHDFS
{code}
TestWALProcedureStoreOnHDFS is known flaky.
Assign system tables ahead of user region assignment
Key: HBASE-14190
URL: https://issues.apache.org/jira/browse/HBASE-14190
Project: HBase
Issue Type: Bug
Reporter: Ted Yu
Assignee: Ted Yu
Priority: Critical
Attachments: 14190-v1.txt, 14190-v2.txt, 14190-v3.txt, 14190-v3.txt,
14190-v3.txt, 14190-v4.txt
Currently the namespace table region is assigned like user regions.
I spent several hours working with a customer where master couldn't finish
initialization.
Even though master was restarted quite a few times, it went down with the
following:
{code}
2015-08-05 17:16:57,530 FATAL [hdpmaster1:6.activeMasterManager]
master.HMaster: Master server abort: loaded coprocessors are: []
2015-08-05 17:16:57,530 FATAL [hdpmaster1:6.activeMasterManager]
master.HMaster: Unhandled exception. Starting shutdown.
java.io.IOException: Timedout 30ms waiting for namespace table to be
assigned
at
org.apache.hadoop.hbase.master.TableNamespaceManager.start(TableNamespaceManager.java:104)
at org.apache.hadoop.hbase.master.HMaster.initNamespace(HMaster.java:985)
at
org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:779)
at org.apache.hadoop.hbase.master.HMaster.access$500(HMaster.java:182)
at org.apache.hadoop.hbase.master.HMaster$1.run(HMaster.java:1646)
at java.lang.Thread.run(Thread.java:744)
{code}
During previous run(s), namespace table was created, hence leaving an entry
in hbase:meta.
The following if block in TableNamespaceManager#start() was skipped:
{code}
if (!MetaTableAccessor.tableExists(masterServices.getConnection(),
TableName.NAMESPACE_TABLE_NAME)) {
{code}
TableNamespaceManager#start() spins, waiting for namespace region to be
assigned.
There was issue in master assigning user regions.
We tried issuing 'assign' command from hbase shell which didn't work because
of the following check in MasterRpcServices#assignRegion():
{code}
master.checkInitialized();
{code}
This scenario can be avoided if we assign hbase:namespace table after
hbase:meta is assigned but before user table region assignment.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14200) Separate some subtests of TestStochasticLoadBalancer into TestStochasticLoadBalancer2
[
https://issues.apache.org/jira/browse/HBASE-14200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14663163#comment-14663163
]
Ted Yu commented on HBASE-14200:
From https://builds.apache.org/job/PreCommit-HBASE-Build/15014/console :
{code}
Tests run: 19, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 326.818 sec -
in org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancer
...
Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 453.443 sec -
in org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancer2
{code}
Separate some subtests of TestStochasticLoadBalancer into
TestStochasticLoadBalancer2
-
Key: HBASE-14200
URL: https://issues.apache.org/jira/browse/HBASE-14200
Project: HBase
Issue Type: Test
Reporter: Ted Yu
Assignee: Ted Yu
Priority: Minor
Fix For: 2.0.0, 1.3.0
Attachments: 14200-v1.txt
More and more functionality is added to StochasticLoadBalancer , making
TestStochasticLoadBalancer run longer.
From
https://builds.apache.org/job/PreCommit-HBASE-Build/15011/testReport/org.apache.hadoop.hbase.master.balancer/TestStochasticLoadBalancer/
where total runtime was 14 min, here are the longest subtests:
testRegionReplicasOnLargeCluster: 1 min 34 sec
testRegionReplicasOnMidCluster: 1 min 31 sec
testRegionReplicasOnMidClusterHighReplication: 2 min
testRegionReplicationOnMidClusterReplicationGreaterThanNumNodes: 2 min 25 sec
This issue is to separate out the above subtests into
TestStochasticLoadBalancer2, giving each of the tests around 7 min runtime.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-14200) Separate some subtests of TestStochasticLoadBalancer into TestStochasticLoadBalancer2
[ https://issues.apache.org/jira/browse/HBASE-14200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-14200: --- Fix Version/s: 1.3.0 2.0.0 Separate some subtests of TestStochasticLoadBalancer into TestStochasticLoadBalancer2 - Key: HBASE-14200 URL: https://issues.apache.org/jira/browse/HBASE-14200 Project: HBase Issue Type: Test Reporter: Ted Yu Assignee: Ted Yu Priority: Minor Fix For: 2.0.0, 1.3.0 Attachments: 14200-v1.txt More and more functionality is added to StochasticLoadBalancer , making TestStochasticLoadBalancer run longer. From https://builds.apache.org/job/PreCommit-HBASE-Build/15011/testReport/org.apache.hadoop.hbase.master.balancer/TestStochasticLoadBalancer/ where total runtime was 14 min, here are the longest subtests: testRegionReplicasOnLargeCluster: 1 min 34 sec testRegionReplicasOnMidCluster: 1 min 31 sec testRegionReplicasOnMidClusterHighReplication: 2 min testRegionReplicationOnMidClusterReplicationGreaterThanNumNodes: 2 min 25 sec This issue is to separate out the above subtests into TestStochasticLoadBalancer2, giving each of the tests around 7 min runtime. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14200) Separate some subtests of TestStochasticLoadBalancer into TestStochasticLoadBalancer2
[
https://issues.apache.org/jira/browse/HBASE-14200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14663171#comment-14663171
]
Hadoop QA commented on HBASE-14200:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12749427/14200-v1.txt
against master branch at commit d4fc7cf2ba0d06b040dad435ab72366c4fd44116.
ATTACHMENT ID: 12749427
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 5 new
or modified tests.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.7.0)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 protoc{color}. The applied patch does not increase the
total number of protoc compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 checkstyle{color}. The applied patch does not increase the
total number of checkstyle errors
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 lineLengths{color}. The patch introduces the following lines
longer than 100:
+ int[] largeCluster = new int[] { 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+ RackManager rackManager, boolean assertFullyBalanced, boolean
assertFullyBalancedForReplicas) {
{color:green}+1 site{color}. The mvn post-site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/15014//testReport/
Release Findbugs (version 2.0.3)warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/15014//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/15014//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/15014//console
This message is automatically generated.
Separate some subtests of TestStochasticLoadBalancer into
TestStochasticLoadBalancer2
-
Key: HBASE-14200
URL: https://issues.apache.org/jira/browse/HBASE-14200
Project: HBase
Issue Type: Test
Reporter: Ted Yu
Assignee: Ted Yu
Priority: Minor
Fix For: 2.0.0, 1.3.0
Attachments: 14200-v1.txt
More and more functionality is added to StochasticLoadBalancer , making
TestStochasticLoadBalancer run longer.
From
https://builds.apache.org/job/PreCommit-HBASE-Build/15011/testReport/org.apache.hadoop.hbase.master.balancer/TestStochasticLoadBalancer/
where total runtime was 14 min, here are the longest subtests:
testRegionReplicasOnLargeCluster: 1 min 34 sec
testRegionReplicasOnMidCluster: 1 min 31 sec
testRegionReplicasOnMidClusterHighReplication: 2 min
testRegionReplicationOnMidClusterReplicationGreaterThanNumNodes: 2 min 25 sec
This issue is to separate out the above subtests into
TestStochasticLoadBalancer2, giving each of the tests around 7 min runtime.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13706) CoprocessorClassLoader should not exempt Hive classes
[ https://issues.apache.org/jira/browse/HBASE-13706?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14663265#comment-14663265 ] Jerry He commented on HBASE-13706: -- master-v2 patch has been committed to master branch. I also committed branch-1 version of the patch to branch-1 and branch-1.2. I will add a release note. Thanks for the review, [~apurtell]. Do you want it in 0.98? It will be the same as the branch-1 version of the patch. CoprocessorClassLoader should not exempt Hive classes - Key: HBASE-13706 URL: https://issues.apache.org/jira/browse/HBASE-13706 Project: HBase Issue Type: Bug Components: Coprocessors Affects Versions: 2.0.0, 1.0.1, 1.1.0, 0.98.12 Reporter: Jerry He Assignee: Jerry He Priority: Minor Fix For: 2.0.0, 1.2.0, 1.3.0 Attachments: HBASE-13706-branch-1.patch, HBASE-13706-master-v2.patch, HBASE-13706-master-v2.patch, HBASE-13706.patch CoprocessorClassLoader is used to load classes from the coprocessor jar. Certain classes are exempt from being loaded by this ClassLoader, which means they will be ignored in the coprocessor jar, but loaded from parent classpath instead. One problem is that we categorically exempt org.apache.hadoop. But it happens that Hive packages start with org.apache.hadoop. There is no reason to exclude hive classes from theCoprocessorClassLoader. HBase does not even include Hive jars. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-13706) CoprocessorClassLoader should not exempt Hive classes
[ https://issues.apache.org/jira/browse/HBASE-13706?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jerry He updated HBASE-13706: - Hadoop Flags: Incompatible change,Reviewed Release Note: Starting from HBase 2.0, CoprocessorClassLoader will not exempt hadoop classes or zookeeper classes. This means that if the custom coprocessor jar contains hadoop or zookeeper packages and classes, they will be loaded by the CoprocessorClassLoader. Only hbase packages and classes are exempted from the CoprocessorClassLoader. They (and their dependencies) are loaded by the parent server class loader. CoprocessorClassLoader should not exempt Hive classes - Key: HBASE-13706 URL: https://issues.apache.org/jira/browse/HBASE-13706 Project: HBase Issue Type: Bug Components: Coprocessors Affects Versions: 2.0.0, 1.0.1, 1.1.0, 0.98.12 Reporter: Jerry He Assignee: Jerry He Priority: Minor Fix For: 2.0.0, 1.2.0, 1.3.0 Attachments: HBASE-13706-branch-1.patch, HBASE-13706-master-v2.patch, HBASE-13706-master-v2.patch, HBASE-13706.patch CoprocessorClassLoader is used to load classes from the coprocessor jar. Certain classes are exempt from being loaded by this ClassLoader, which means they will be ignored in the coprocessor jar, but loaded from parent classpath instead. One problem is that we categorically exempt org.apache.hadoop. But it happens that Hive packages start with org.apache.hadoop. There is no reason to exclude hive classes from theCoprocessorClassLoader. HBase does not even include Hive jars. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
