[jira] [Commented] (HBASE-11489) ClassNotFoundException while running IT tests in trunk using 'mvn verify'
[ https://issues.apache.org/jira/browse/HBASE-11489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14061723#comment-14061723 ] ramkrishna.s.vasudevan commented on HBASE-11489: The patch does not work {code} error message=org/jboss/netty/channel/ChannelPipelineFactory type=java.lang.NoClassDefFoundErrorjava.lang.NoClassDefFoundError: org/jboss/netty/channel/ChannelPipelineFactory at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:270) at org.apache.hadoop.conf.Configuration.getClassByNameOrNull(Configuration.java:1834) at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:1799) at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1893) at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1919) at org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServices.serviceInit(AuxServices.java:105) at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163) at org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:107) at org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl.serviceInit(ContainerManagerImpl.java:220) at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163) at org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:107) at org.apache.hadoop.yarn.server.nodemanager.NodeManager.serviceInit(NodeManager.java:186) at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163) at org.apache.hadoop.yarn.server.MiniYARNCluster$NodeManagerWrapper.serviceInit(MiniYARNCluster.java:533) at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163) at org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:107) at org.apache.hadoop.yarn.server.MiniYARNCluster.serviceInit(MiniYARNCluster.java:265) at org.apache.hadoop.mapreduce.v2.MiniMRYarnCluster.serviceInit(MiniMRYarnCluster.java:179) at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163) at org.apache.hadoop.mapred.MiniMRClientClusterFactory.create(MiniMRClientClusterFactory.java:79) at org.apache.hadoop.mapred.MiniMRCluster.lt;initgt;(MiniMRCluster.java:183) at org.apache.hadoop.mapred.MiniMRCluster.lt;initgt;(MiniMRCluster.java:171) at org.apache.hadoop.mapred.MiniMRCluster.lt;initgt;(MiniMRCluster.java:163) at org.apache.hadoop.mapred.MiniMRCluster.lt;initgt;(MiniMRCluster.java:124) at org.apache.hadoop.hbase.HBaseTestingUtility.startMiniMapReduceCluster(HBaseTestingUtility.java:2249) at org.apache.hadoop.hbase.HBaseTestingUtility.startMiniMapReduceCluster(HBaseTestingUtility.java:2190) at org.apache.hadoop.hbase.test.IntegrationTestLoadAndVerify.setUpCluster(IntegrationTestLoadAndVerify.java:130) at org.apache.hadoop.hbase.test.IntegrationTestWithCellVisibilityLoadAndVerify.setUpCluster(IntegrationTestWithCellVisibilityLoadAndVerify.java:123) {code} This ChannelFactory belongs to org.jboss.netty package and not the io.netty package that we have. There is some indirect dependency from the MR. ClassNotFoundException while running IT tests in trunk using 'mvn verify' - Key: HBASE-11489 URL: https://issues.apache.org/jira/browse/HBASE-11489 Project: HBase Issue Type: Bug Reporter: ramkrishna.s.vasudevan Assignee: ramkrishna.s.vasudevan Fix For: 0.99.0, 2.0.0 Attachments: HBASE-11489.patch Trying to run {code} mvn verify -Dit.test=IntegrationTestBigLinkedList -Dtest.output.tofile=false {code} causes this ClassNotFoundException issue bq.testContinuousIngest(org.apache.hadoop.hbase.test.IntegrationTestBigLinkedList): org/jboss/netty/channel/ChannelFactory -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11118) non environment variable solution for IllegalAccessError: class com.google.protobuf.ZeroCopyLiteralByteString cannot access its superclass com.google.protobuf.Literal
[ https://issues.apache.org/jira/browse/HBASE-8?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14061768#comment-14061768 ] Hudson commented on HBASE-8: FAILURE: Integrated in HBase-TRUNK #5304 (See [https://builds.apache.org/job/HBase-TRUNK/5304/]) Add note on HBASE-8 and HBASE-10877 HBaseZeroCopyByteString woes to troubleshooting section (stack: rev bf2933e08ab65af3643bdadbc3d59c943d40602f) * src/main/docbkx/troubleshooting.xml non environment variable solution for IllegalAccessError: class com.google.protobuf.ZeroCopyLiteralByteString cannot access its superclass com.google.protobuf.LiteralByteString -- Key: HBASE-8 URL: https://issues.apache.org/jira/browse/HBASE-8 Project: HBase Issue Type: Bug Components: mapreduce Affects Versions: 0.98.2 Reporter: André Kelpe Assignee: stack Priority: Blocker Fix For: 0.99.0, 0.98.4, 2.0.0 Attachments: 8.098-0.txt, 8.098.txt, 8.bytestringer.txt, 1118.suggested.undoing.optimization.on.clientside.txt, 1118.suggested.undoing.optimization.on.clientside.txt, HBASE-8-0.98.00.patch, HBASE-8-0.98.01.patch, HBASE-8-0.98.02.patch, HBASE-8-0.98.03.patch, HBASE-8-0.98.patch.gz, HBASE-8-trunk.patch.gz, HBASE-8.00.patch, HBASE-8.01.patch, HBASE-8.02.patch, HBASE-8_0.98_addendum.patch, HBASE-8_master_addendum.patch, shade_attempt.patch I am running into the problem described in https://issues.apache.org/jira/browse/HBASE-10304, while trying to use a newer version within cascading.hbase (https://github.com/cascading/cascading.hbase). One of the features of cascading.hbase is that you can use it from lingual (http://www.cascading.org/projects/lingual/), our SQL layer for hadoop. lingual has a notion of providers, which are fat jars that we pull down dynamically at runtime. Those jars give users the ability to talk to any system or format from SQL. They are added to the classpath programmatically before we submit jobs to a hadoop cluster. Since lingual does not know upfront , which providers are going to be used in a given run, the HADOOP_CLASSPATH trick proposed in the JIRA above is really clunky and breaks the ease of use we had before. No other provider requires this right now. It would be great to have a programmatical way to fix this, when using fat jars. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-10877) HBase non-retriable exception list should be expanded
[ https://issues.apache.org/jira/browse/HBASE-10877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14061769#comment-14061769 ] Hudson commented on HBASE-10877: FAILURE: Integrated in HBase-TRUNK #5304 (See [https://builds.apache.org/job/HBase-TRUNK/5304/]) Add note on HBASE-8 and HBASE-10877 HBaseZeroCopyByteString woes to troubleshooting section (stack: rev bf2933e08ab65af3643bdadbc3d59c943d40602f) * src/main/docbkx/troubleshooting.xml HBase non-retriable exception list should be expanded - Key: HBASE-10877 URL: https://issues.apache.org/jira/browse/HBASE-10877 Project: HBase Issue Type: Improvement Reporter: Sergey Shelukhin Priority: Minor Example where retries do not make sense: {noformat} 2014-03-31 20:54:27,765 WARN [InputInitializer [Map 1] #0] org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation: Encountered problems when prefetch hbase:meta table: org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after attempts=35, exceptions: Mon Mar 31 20:45:17 UTC 2014, org.apache.hadoop.hbase.client.RpcRetryingCaller@343d511e, java.lang.IllegalAccessError: class com.google.protobuf.HBaseZeroCopyByteString cannot access its superclass com.google.protobuf.LiteralByteString Mon Mar 31 20:45:17 UTC 2014, org.apache.hadoop.hbase.client.RpcRetryingCaller@343d511e, java.lang.IllegalAccessError: com/google/protobuf/HBaseZeroCopyByteString Mon Mar 31 20:45:17 UTC 2014, org.apache.hadoop.hbase.client.RpcRetryingCaller@343d511e, java.lang.IllegalAccessError: com/google/protobuf/HBaseZeroCopyByteString Mon Mar 31 20:45:18 UTC 2014, org.apache.hadoop.hbase.client.RpcRetryingCaller@343d511e, java.lang.IllegalAccessError: com/google/protobuf/HBaseZeroCopyByteString Mon Mar 31 20:45:20 UTC 2014, org.apache.hadoop.hbase.client.RpcRetryingCaller@343d511e, java.lang.IllegalAccessError: com/google/protobuf/HBaseZeroCopyByteString Mon Mar 31 20:45:24 UTC 2014, org.apache.hadoop.hbase.client.RpcRetryingCaller@343d511e, java.lang.IllegalAccessError: com/google/protobuf/HBaseZeroCopyByteString Mon Mar 31 20:45:34 UTC 2014, org.apache.hadoop.hbase.client.RpcRetryingCaller@343d511e, java.lang.IllegalAccessError: com/google/protobuf/HBaseZeroCopyByteString Mon Mar 31 20:45:45 UTC 2014, org.apache.hadoop.hbase.client.RpcRetryingCaller@343d511e, java.lang.IllegalAccessError: com/google/protobuf/HBaseZeroCopyByteString Mon Mar 31 20:45:55 UTC 2014, org.apache.hadoop.hbase.client.RpcRetryingCaller@343d511e, java.lang.IllegalAccessError: com/google/protobuf/HBaseZeroCopyByteString Mon Mar 31 20:46:05 UTC 2014, org.apache.hadoop.hbase.client.RpcRetryingCaller@343d511e, java.lang.IllegalAccessError: com/google/protobuf/HBaseZeroCopyByteString Mon Mar 31 20:46:25 UTC 2014, org.apache.hadoop.hbase.client.RpcRetryingCaller@343d511e, java.lang.IllegalAccessError: com/google/protobuf/HBaseZeroCopyByteString Mon Mar 31 20:46:45 UTC 2014, org.apache.hadoop.hbase.client.RpcRetryingCaller@343d511e, java.lang.IllegalAccessError: com/google/protobuf/HBaseZeroCopyByteString Mon Mar 31 20:47:05 UTC 2014, org.apache.hadoop.hbase.client.RpcRetryingCaller@343d511e, java.lang.IllegalAccessError: com/google/protobuf/HBaseZeroCopyByteString Mon Mar 31 20:47:25 UTC 2014, org.apache.hadoop.hbase.client.RpcRetryingCaller@343d511e, java.lang.IllegalAccessError: com/google/protobuf/HBaseZeroCopyByteString Mon Mar 31 20:47:45 UTC 2014, org.apache.hadoop.hbase.client.RpcRetryingCaller@343d511e, java.lang.IllegalAccessError: com/google/protobuf/HBaseZeroCopyByteString Mon Mar 31 20:48:05 UTC 2014, org.apache.hadoop.hbase.client.RpcRetryingCaller@343d511e, java.lang.IllegalAccessError: com/google/protobuf/HBaseZeroCopyByteString Mon Mar 31 20:48:25 UTC 2014, org.apache.hadoop.hbase.client.RpcRetryingCaller@343d511e, java.lang.IllegalAccessError: com/google/protobuf/HBaseZeroCopyByteString Mon Mar 31 20:48:46 UTC 2014, org.apache.hadoop.hbase.client.RpcRetryingCaller@343d511e, java.lang.IllegalAccessError: com/google/protobuf/HBaseZeroCopyByteString Mon Mar 31 20:49:06 UTC 2014, org.apache.hadoop.hbase.client.RpcRetryingCaller@343d511e, java.lang.IllegalAccessError: com/google/protobuf/HBaseZeroCopyByteString Mon Mar 31 20:49:26 UTC 2014, org.apache.hadoop.hbase.client.RpcRetryingCaller@343d511e, java.lang.IllegalAccessError: com/google/protobuf/HBaseZeroCopyByteString Mon Mar 31 20:49:46 UTC 2014, org.apache.hadoop.hbase.client.RpcRetryingCaller@343d511e, java.lang.IllegalAccessError: com/google/protobuf/HBaseZeroCopyByteString Mon Mar 31 20:50:06 UTC 2014, org.apache.hadoop.hbase.client.RpcRetryingCaller@343d511e, java.lang.IllegalAccessError: com/google/protobuf/HBaseZeroCopyByteString
[jira] [Updated] (HBASE-11489) ClassNotFoundException while running IT tests in trunk using 'mvn verify'
[ https://issues.apache.org/jira/browse/HBASE-11489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-11489: --- Status: Open (was: Patch Available) ClassNotFoundException while running IT tests in trunk using 'mvn verify' - Key: HBASE-11489 URL: https://issues.apache.org/jira/browse/HBASE-11489 Project: HBase Issue Type: Bug Reporter: ramkrishna.s.vasudevan Assignee: ramkrishna.s.vasudevan Fix For: 0.99.0, 2.0.0 Attachments: HBASE-11489.patch Trying to run {code} mvn verify -Dit.test=IntegrationTestBigLinkedList -Dtest.output.tofile=false {code} causes this ClassNotFoundException issue bq.testContinuousIngest(org.apache.hadoop.hbase.test.IntegrationTestBigLinkedList): org/jboss/netty/channel/ChannelFactory -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11489) ClassNotFoundException while running IT tests in trunk using 'mvn verify'
[ https://issues.apache.org/jira/browse/HBASE-11489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-11489: --- Status: Patch Available (was: Open) ClassNotFoundException while running IT tests in trunk using 'mvn verify' - Key: HBASE-11489 URL: https://issues.apache.org/jira/browse/HBASE-11489 Project: HBase Issue Type: Bug Reporter: ramkrishna.s.vasudevan Assignee: ramkrishna.s.vasudevan Fix For: 0.99.0, 2.0.0 Attachments: HBASE-11489.patch, HBASE-11489_1.patch Trying to run {code} mvn verify -Dit.test=IntegrationTestBigLinkedList -Dtest.output.tofile=false {code} causes this ClassNotFoundException issue bq.testContinuousIngest(org.apache.hadoop.hbase.test.IntegrationTestBigLinkedList): org/jboss/netty/channel/ChannelFactory -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11489) ClassNotFoundException while running IT tests in trunk using 'mvn verify'
[ https://issues.apache.org/jira/browse/HBASE-11489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-11489: --- Attachment: HBASE-11489_1.patch This patch works. ClassNotFoundException while running IT tests in trunk using 'mvn verify' - Key: HBASE-11489 URL: https://issues.apache.org/jira/browse/HBASE-11489 Project: HBase Issue Type: Bug Reporter: ramkrishna.s.vasudevan Assignee: ramkrishna.s.vasudevan Fix For: 0.99.0, 2.0.0 Attachments: HBASE-11489.patch, HBASE-11489_1.patch Trying to run {code} mvn verify -Dit.test=IntegrationTestBigLinkedList -Dtest.output.tofile=false {code} causes this ClassNotFoundException issue bq.testContinuousIngest(org.apache.hadoop.hbase.test.IntegrationTestBigLinkedList): org/jboss/netty/channel/ChannelFactory -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11480) ClientScanner might not close the HConnection created in construction
[ https://issues.apache.org/jira/browse/HBASE-11480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14061788#comment-14061788 ] cuijianwei commented on HBASE-11480: I checked the source code for 0.94, 0.96 and 0.98, this constructor is not used by other classes in hbase. Then, we can deprecate this constructor and avoid using it by user. ClientScanner might not close the HConnection created in construction - Key: HBASE-11480 URL: https://issues.apache.org/jira/browse/HBASE-11480 Project: HBase Issue Type: Improvement Components: Client, Scanners Affects Versions: 0.96.2, 0.94.20, 0.98.3 Reporter: cuijianwei Priority: Trivial ClientScanner will create HConnection in its construction: {code} public ClientScanner(final Configuration conf, final Scan scan, final byte[] tableName) throws IOException { this(conf, scan, tableName, HConnectionManager.getConnection(conf)); } {code} However, this connection won't be closed in ClientScanner.close(). Is it better to deprecate this construction? -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11480) ClientScanner might not close the HConnection created in construction
[ https://issues.apache.org/jira/browse/HBASE-11480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] cuijianwei updated HBASE-11480: --- Attachment: HBASE-11480-0.98-v1.patch HBASE-11480-0.96-v1.patch HBASE-11480-0.95-v1.patch HBASE-11480-0.94-v1.patch ClientScanner might not close the HConnection created in construction - Key: HBASE-11480 URL: https://issues.apache.org/jira/browse/HBASE-11480 Project: HBase Issue Type: Improvement Components: Client, Scanners Affects Versions: 0.96.2, 0.94.20, 0.98.3 Reporter: cuijianwei Priority: Trivial Attachments: HBASE-11480-0.94-v1.patch, HBASE-11480-0.95-v1.patch, HBASE-11480-0.96-v1.patch, HBASE-11480-0.98-v1.patch ClientScanner will create HConnection in its construction: {code} public ClientScanner(final Configuration conf, final Scan scan, final byte[] tableName) throws IOException { this(conf, scan, tableName, HConnectionManager.getConnection(conf)); } {code} However, this connection won't be closed in ClientScanner.close(). Is it better to deprecate this construction? -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11339) HBase MOB
[ https://issues.apache.org/jira/browse/HBASE-11339?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jingcheng Du updated HBASE-11339: - Attachment: (was: HBase MOB Design.pdf) HBase MOB - Key: HBASE-11339 URL: https://issues.apache.org/jira/browse/HBASE-11339 Project: HBase Issue Type: New Feature Components: regionserver, Scanners Reporter: Jingcheng Du Assignee: Jingcheng Du Attachments: HBase MOB Design.pdf It's quite useful to save the medium binary data like images, documents into Apache HBase. Unfortunately directly saving the binary MOB(medium object) to HBase leads to a worse performance since the frequent split and compaction. In this design, the MOB data are stored in an more efficient way, which keeps a high write/read performance and guarantees the data consistency in Apache HBase. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11339) HBase MOB
[ https://issues.apache.org/jira/browse/HBASE-11339?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jingcheng Du updated HBASE-11339: - Attachment: HBase MOB Design.pdf HBase MOB - Key: HBASE-11339 URL: https://issues.apache.org/jira/browse/HBASE-11339 Project: HBase Issue Type: New Feature Components: regionserver, Scanners Reporter: Jingcheng Du Assignee: Jingcheng Du Attachments: HBase MOB Design.pdf It's quite useful to save the medium binary data like images, documents into Apache HBase. Unfortunately directly saving the binary MOB(medium object) to HBase leads to a worse performance since the frequent split and compaction. In this design, the MOB data are stored in an more efficient way, which keeps a high write/read performance and guarantees the data consistency in Apache HBase. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11339) HBase MOB
[ https://issues.apache.org/jira/browse/HBASE-11339?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14061795#comment-14061795 ] Jingcheng Du commented on HBASE-11339: -- Upload the latest design document. Refine the design and description. HBase MOB - Key: HBASE-11339 URL: https://issues.apache.org/jira/browse/HBASE-11339 Project: HBase Issue Type: New Feature Components: regionserver, Scanners Reporter: Jingcheng Du Assignee: Jingcheng Du Attachments: HBase MOB Design.pdf It's quite useful to save the medium binary data like images, documents into Apache HBase. Unfortunately directly saving the binary MOB(medium object) to HBase leads to a worse performance since the frequent split and compaction. In this design, the MOB data are stored in an more efficient way, which keeps a high write/read performance and guarantees the data consistency in Apache HBase. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11489) ClassNotFoundException while running IT tests in trunk using 'mvn verify'
[ https://issues.apache.org/jira/browse/HBASE-11489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14061833#comment-14061833 ] Hadoop QA commented on HBASE-11489: --- {color:red}-1 overall{color}. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12655706/HBASE-11489_1.patch against trunk revision . ATTACHMENT ID: 12655706 {color:green}+1 @author{color}. The patch does not contain any @author tags. {color:red}-1 tests included{color}. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. {color:green}+1 javac{color}. The applied patch does not increase the total number of javac compiler warnings. {color:green}+1 javac{color}. The applied patch does not increase the total number of javac compiler warnings. {color:green}+1 javadoc{color}. The javadoc tool did not generate any warning messages. {color:red}-1 findbugs{color}. The patch appears to introduce 1 new Findbugs (version 2.0.3) warnings. {color:green}+1 release audit{color}. The applied patch does not increase the total number of release audit warnings. {color:green}+1 lineLengths{color}. The patch does not introduce lines longer than 100 {color:green}+1 site{color}. The mvn site goal succeeds with this patch. {color:red}-1 core tests{color}. The patch failed these unit tests: {color:red}-1 core zombie tests{color}. There are 1 zombie test(s): at org.apache.hadoop.hbase.client.TestReplicaWithCluster.testCreateDeleteTable(TestReplicaWithCluster.java:138) Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/10067//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10067//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10067//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10067//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10067//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10067//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10067//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10067//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10067//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10067//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/10067//console This message is automatically generated. ClassNotFoundException while running IT tests in trunk using 'mvn verify' - Key: HBASE-11489 URL: https://issues.apache.org/jira/browse/HBASE-11489 Project: HBase Issue Type: Bug Reporter: ramkrishna.s.vasudevan Assignee: ramkrishna.s.vasudevan Fix For: 0.99.0, 2.0.0 Attachments: HBASE-11489.patch, HBASE-11489_1.patch Trying to run {code} mvn verify -Dit.test=IntegrationTestBigLinkedList -Dtest.output.tofile=false {code} causes this ClassNotFoundException issue bq.testContinuousIngest(org.apache.hadoop.hbase.test.IntegrationTestBigLinkedList): org/jboss/netty/channel/ChannelFactory -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11489) ClassNotFoundException while running IT tests in trunk using 'mvn verify'
[ https://issues.apache.org/jira/browse/HBASE-11489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14061838#comment-14061838 ] Ted Yu commented on HBASE-11489: +1 ClassNotFoundException while running IT tests in trunk using 'mvn verify' - Key: HBASE-11489 URL: https://issues.apache.org/jira/browse/HBASE-11489 Project: HBase Issue Type: Bug Reporter: ramkrishna.s.vasudevan Assignee: ramkrishna.s.vasudevan Fix For: 0.99.0, 2.0.0 Attachments: HBASE-11489.patch, HBASE-11489_1.patch Trying to run {code} mvn verify -Dit.test=IntegrationTestBigLinkedList -Dtest.output.tofile=false {code} causes this ClassNotFoundException issue bq.testContinuousIngest(org.apache.hadoop.hbase.test.IntegrationTestBigLinkedList): org/jboss/netty/channel/ChannelFactory -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11064) Odd behaviors of TableName for empty namespace
[ https://issues.apache.org/jira/browse/HBASE-11064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rekha Joshi updated HBASE-11064: Attachment: HBASE-11064.2.patch [~enis] [~apurtell]: just saw ping.attached patch with test added/tested.thanks Odd behaviors of TableName for empty namespace -- Key: HBASE-11064 URL: https://issues.apache.org/jira/browse/HBASE-11064 Project: HBase Issue Type: Bug Affects Versions: 0.98.3 Reporter: Hiroshi Ikeda Assignee: Rekha Joshi Priority: Trivial Fix For: 0.99.0, 1.0.0, 0.98.5 Attachments: HBASE-11064.1.patch, HBASE-11064.2.patch In the class TableName, {code} public static byte [] isLegalFullyQualifiedTableName(final byte[] tableName) { ... int namespaceDelimIndex = ... if (namespaceDelimIndex == 0 || namespaceDelimIndex == -1){ isLegalTableQualifierName(tableName); } else { ... {code} That means, for example, giving :a as the argument throws an exception which says invalid qualifier, instead of invalid namespace. Also, TableName.valueOf(String) and valueOf(byte[]) can create an instance with empty namespace, which is inconsistent. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-10398) HBase book updates for Replication after HBASE-10322
[ https://issues.apache.org/jira/browse/HBASE-10398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14061878#comment-14061878 ] Anoop Sam John commented on HBASE-10398: The normal codec which is used btw client and server will strip the tags. By default the replication will also use the same codec to pass Cells from server to server. So when tags are part of the Cells (ie. When one uses cell level visibility labels or cell level acl or any other feature which uses cell tags) we need to configure a Codec which supports shipping of tags also, along with Cells. We have introduced a config for this hbase.replication.rpc.codec Configure this with a Tag supported codec. org.apache.hadoop.hbase.codec.KeyValueCodecWithTags Also make sure this is configured in RS side in source and sink clusters of the Replication. Does this explain things well Misty? HBase book updates for Replication after HBASE-10322 Key: HBASE-10398 URL: https://issues.apache.org/jira/browse/HBASE-10398 Project: HBase Issue Type: Task Components: documentation Affects Versions: 0.98.0 Reporter: Anoop Sam John Assignee: Anoop Sam John -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11064) Odd behaviors of TableName for empty namespace
[ https://issues.apache.org/jira/browse/HBASE-11064?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14061879#comment-14061879 ] Hadoop QA commented on HBASE-11064: --- {color:red}-1 overall{color}. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12655727/HBASE-11064.2.patch against trunk revision . ATTACHMENT ID: 12655727 {color:green}+1 @author{color}. The patch does not contain any @author tags. {color:green}+1 tests included{color}. The patch appears to include 3 new or modified tests. {color:red}-1 patch{color}. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/10068//console This message is automatically generated. Odd behaviors of TableName for empty namespace -- Key: HBASE-11064 URL: https://issues.apache.org/jira/browse/HBASE-11064 Project: HBase Issue Type: Bug Affects Versions: 0.98.3 Reporter: Hiroshi Ikeda Assignee: Rekha Joshi Priority: Trivial Fix For: 0.99.0, 1.0.0, 0.98.5 Attachments: HBASE-11064.1.patch, HBASE-11064.2.patch In the class TableName, {code} public static byte [] isLegalFullyQualifiedTableName(final byte[] tableName) { ... int namespaceDelimIndex = ... if (namespaceDelimIndex == 0 || namespaceDelimIndex == -1){ isLegalTableQualifierName(tableName); } else { ... {code} That means, for example, giving :a as the argument throws an exception which says invalid qualifier, instead of invalid namespace. Also, TableName.valueOf(String) and valueOf(byte[]) can create an instance with empty namespace, which is inconsistent. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11064) Odd behaviors of TableName for empty namespace
[ https://issues.apache.org/jira/browse/HBASE-11064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rekha Joshi updated HBASE-11064: Attachment: HBASE-11064.2.patch Odd behaviors of TableName for empty namespace -- Key: HBASE-11064 URL: https://issues.apache.org/jira/browse/HBASE-11064 Project: HBase Issue Type: Bug Affects Versions: 0.98.3 Reporter: Hiroshi Ikeda Assignee: Rekha Joshi Priority: Trivial Fix For: 0.99.0, 1.0.0, 0.98.5 Attachments: HBASE-11064.1.patch, HBASE-11064.2.patch In the class TableName, {code} public static byte [] isLegalFullyQualifiedTableName(final byte[] tableName) { ... int namespaceDelimIndex = ... if (namespaceDelimIndex == 0 || namespaceDelimIndex == -1){ isLegalTableQualifierName(tableName); } else { ... {code} That means, for example, giving :a as the argument throws an exception which says invalid qualifier, instead of invalid namespace. Also, TableName.valueOf(String) and valueOf(byte[]) can create an instance with empty namespace, which is inconsistent. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11064) Odd behaviors of TableName for empty namespace
[ https://issues.apache.org/jira/browse/HBASE-11064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rekha Joshi updated HBASE-11064: Attachment: (was: HBASE-11064.2.patch) Odd behaviors of TableName for empty namespace -- Key: HBASE-11064 URL: https://issues.apache.org/jira/browse/HBASE-11064 Project: HBase Issue Type: Bug Affects Versions: 0.98.3 Reporter: Hiroshi Ikeda Assignee: Rekha Joshi Priority: Trivial Fix For: 0.99.0, 1.0.0, 0.98.5 Attachments: HBASE-11064.1.patch, HBASE-11064.2.patch In the class TableName, {code} public static byte [] isLegalFullyQualifiedTableName(final byte[] tableName) { ... int namespaceDelimIndex = ... if (namespaceDelimIndex == 0 || namespaceDelimIndex == -1){ isLegalTableQualifierName(tableName); } else { ... {code} That means, for example, giving :a as the argument throws an exception which says invalid qualifier, instead of invalid namespace. Also, TableName.valueOf(String) and valueOf(byte[]) can create an instance with empty namespace, which is inconsistent. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11240) Print hdfs pipeline when hlog's sync is slow
[ https://issues.apache.org/jira/browse/HBASE-11240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liu Shaohui updated HBASE-11240: Attachment: HBASE-11240-addendum.diff addendum to fix the bug in transforming ns to ms Print hdfs pipeline when hlog's sync is slow Key: HBASE-11240 URL: https://issues.apache.org/jira/browse/HBASE-11240 Project: HBase Issue Type: Improvement Components: Operability, wal Reporter: Liu Shaohui Assignee: Liu Shaohui Fix For: 2.0.0 Attachments: HBASE-11240-addendum.diff, HBASE-11240-trunk-v1.diff, HBASE-11240-trunk-v2.diff Sometimes the slow sync of hlog writer is because there is an abnormal datanode in the pipeline. So it will be helpful to print the pipeline of slow sync to diagnose those problems. The ultimate solution is to join the trace of HBase and HDFS. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11240) Print hdfs pipeline when hlog's sync is slow
[ https://issues.apache.org/jira/browse/HBASE-11240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14061897#comment-14061897 ] Liu Shaohui commented on HBASE-11240: - [~enis] Would you help to review the addendum? Thanks Print hdfs pipeline when hlog's sync is slow Key: HBASE-11240 URL: https://issues.apache.org/jira/browse/HBASE-11240 Project: HBase Issue Type: Improvement Components: Operability, wal Reporter: Liu Shaohui Assignee: Liu Shaohui Fix For: 2.0.0 Attachments: HBASE-11240-addendum.diff, HBASE-11240-trunk-v1.diff, HBASE-11240-trunk-v2.diff Sometimes the slow sync of hlog writer is because there is an abnormal datanode in the pipeline. So it will be helpful to print the pipeline of slow sync to diagnose those problems. The ultimate solution is to join the trace of HBase and HDFS. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11064) Odd behaviors of TableName for empty namespace
[ https://issues.apache.org/jira/browse/HBASE-11064?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14061898#comment-14061898 ] Hadoop QA commented on HBASE-11064: --- {color:red}-1 overall{color}. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12655730/HBASE-11064.2.patch against trunk revision . ATTACHMENT ID: 12655730 {color:green}+1 @author{color}. The patch does not contain any @author tags. {color:green}+1 tests included{color}. The patch appears to include 3 new or modified tests. {color:red}-1 patch{color}. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/10069//console This message is automatically generated. Odd behaviors of TableName for empty namespace -- Key: HBASE-11064 URL: https://issues.apache.org/jira/browse/HBASE-11064 Project: HBase Issue Type: Bug Affects Versions: 0.98.3 Reporter: Hiroshi Ikeda Assignee: Rekha Joshi Priority: Trivial Fix For: 0.99.0, 1.0.0, 0.98.5 Attachments: HBASE-11064.1.patch, HBASE-11064.2.patch In the class TableName, {code} public static byte [] isLegalFullyQualifiedTableName(final byte[] tableName) { ... int namespaceDelimIndex = ... if (namespaceDelimIndex == 0 || namespaceDelimIndex == -1){ isLegalTableQualifierName(tableName); } else { ... {code} That means, for example, giving :a as the argument throws an exception which says invalid qualifier, instead of invalid namespace. Also, TableName.valueOf(String) and valueOf(byte[]) can create an instance with empty namespace, which is inconsistent. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (HBASE-11518) doc update for how to create non-shared HConnection
Qiang Tian created HBASE-11518: -- Summary: doc update for how to create non-shared HConnection Key: HBASE-11518 URL: https://issues.apache.org/jira/browse/HBASE-11518 Project: HBase Issue Type: Bug Components: documentation Affects Versions: 0.98.4, 0.94.21 Reporter: Qiang Tian Assignee: Qiang Tian Priority: Minor creation for non-shared connections has changed since hbase-3777, but the doc still remains the same... a simple test program: public class testHbase2{ public static void main(String[] args) throws Exception { Configuration conf = HBaseConfiguration.create(); conf.set(hbase.zookeeper.quorum,localhost); conf.set(hbase.zookeeper.property.clientPort, 2181); conf.set(hbase.client.instance.id, 2); HBaseAdmin admin = new HBaseAdmin(conf); conf.set(hbase.client.instance.id, 3); HBaseAdmin admin2 = new HBaseAdmin(conf); } } public static HConnection getConnection(final Configuration conf) throws IOException { HConnectionKey connectionKey = new HConnectionKey(conf); LOG.info(###create new connectionkey: + connectionKey); synchronized (CONNECTION_INSTANCES) { 14/07/15 18:06:08 INFO client.HConnectionManager: ###create new connectionkey: HConnectionKey{properties={hbase.rpc.timeout=60, hbase.client.instance.id=2, hbase.zookeeper.quorum=localhost, hbase.client.pause=100, hbase.zookeeper.property.clientPort=2181, zookeeper.znode.parent=/hbase, hbase.client.retries.number=35}, username='tianq'} 14/07/15 18:06:08 INFO client.HConnectionManager: ###create new connection### 14/07/15 18:06:08 INFO zookeeper.RecoverableZooKeeper: Process identifier=hconnection-0x3d7c3d7c connecting to ZooKeeper ensemble=localhost:2181 14/07/15 18:06:09 INFO client.HConnectionManager: ###create new connectionkey: HConnectionKey{properties={hbase.rpc.timeout=60, hbase.client.instance.id=3, hbase.zookeeper.quorum=localhost, hbase.client.pause=100, hbase.zookeeper.property.clientPort=2181, zookeeper.znode.parent=/hbase, hbase.client.retries.number=35}, username='tianq'} 14/07/15 18:06:09 INFO client.HConnectionManager: ###create new connection### 14/07/15 18:06:09 INFO zookeeper.RecoverableZooKeeper: Process identifier=hconnection-0xd460d46 connecting to ZooKeeper ensemble=localhost:2181 -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11518) doc update for how to create non-shared HConnection
[ https://issues.apache.org/jira/browse/HBASE-11518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Qiang Tian updated HBASE-11518: --- Attachment: hbase-11518-master.patch doc update for how to create non-shared HConnection --- Key: HBASE-11518 URL: https://issues.apache.org/jira/browse/HBASE-11518 Project: HBase Issue Type: Bug Components: documentation Affects Versions: 0.94.21, 0.98.4 Reporter: Qiang Tian Assignee: Qiang Tian Priority: Minor Attachments: hbase-11518-master.patch creation for non-shared connections has changed since hbase-3777, but the doc still remains the same... a simple test program: public class testHbase2{ public static void main(String[] args) throws Exception { Configuration conf = HBaseConfiguration.create(); conf.set(hbase.zookeeper.quorum,localhost); conf.set(hbase.zookeeper.property.clientPort, 2181); conf.set(hbase.client.instance.id, 2); HBaseAdmin admin = new HBaseAdmin(conf); conf.set(hbase.client.instance.id, 3); HBaseAdmin admin2 = new HBaseAdmin(conf); } } public static HConnection getConnection(final Configuration conf) throws IOException { HConnectionKey connectionKey = new HConnectionKey(conf); LOG.info(###create new connectionkey: + connectionKey); synchronized (CONNECTION_INSTANCES) { 14/07/15 18:06:08 INFO client.HConnectionManager: ###create new connectionkey: HConnectionKey{properties={hbase.rpc.timeout=60, hbase.client.instance.id=2, hbase.zookeeper.quorum=localhost, hbase.client.pause=100, hbase.zookeeper.property.clientPort=2181, zookeeper.znode.parent=/hbase, hbase.client.retries.number=35}, username='tianq'} 14/07/15 18:06:08 INFO client.HConnectionManager: ###create new connection### 14/07/15 18:06:08 INFO zookeeper.RecoverableZooKeeper: Process identifier=hconnection-0x3d7c3d7c connecting to ZooKeeper ensemble=localhost:2181 14/07/15 18:06:09 INFO client.HConnectionManager: ###create new connectionkey: HConnectionKey{properties={hbase.rpc.timeout=60, hbase.client.instance.id=3, hbase.zookeeper.quorum=localhost, hbase.client.pause=100, hbase.zookeeper.property.clientPort=2181, zookeeper.znode.parent=/hbase, hbase.client.retries.number=35}, username='tianq'} 14/07/15 18:06:09 INFO client.HConnectionManager: ###create new connection### 14/07/15 18:06:09 INFO zookeeper.RecoverableZooKeeper: Process identifier=hconnection-0xd460d46 connecting to ZooKeeper ensemble=localhost:2181 -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11518) doc update for how to create non-shared HConnection
[ https://issues.apache.org/jira/browse/HBASE-11518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14061906#comment-14061906 ] Zhang Jingpeng commented on HBASE-11518: for creation non-shared connections ,why not use new Configuration() ? doc update for how to create non-shared HConnection --- Key: HBASE-11518 URL: https://issues.apache.org/jira/browse/HBASE-11518 Project: HBase Issue Type: Bug Components: documentation Affects Versions: 0.94.21, 0.98.4 Reporter: Qiang Tian Assignee: Qiang Tian Priority: Minor Attachments: hbase-11518-master.patch creation for non-shared connections has changed since hbase-3777, but the doc still remains the same... a simple test program: public class testHbase2{ public static void main(String[] args) throws Exception { Configuration conf = HBaseConfiguration.create(); conf.set(hbase.zookeeper.quorum,localhost); conf.set(hbase.zookeeper.property.clientPort, 2181); conf.set(hbase.client.instance.id, 2); HBaseAdmin admin = new HBaseAdmin(conf); conf.set(hbase.client.instance.id, 3); HBaseAdmin admin2 = new HBaseAdmin(conf); } } public static HConnection getConnection(final Configuration conf) throws IOException { HConnectionKey connectionKey = new HConnectionKey(conf); LOG.info(###create new connectionkey: + connectionKey); synchronized (CONNECTION_INSTANCES) { 14/07/15 18:06:08 INFO client.HConnectionManager: ###create new connectionkey: HConnectionKey{properties={hbase.rpc.timeout=60, hbase.client.instance.id=2, hbase.zookeeper.quorum=localhost, hbase.client.pause=100, hbase.zookeeper.property.clientPort=2181, zookeeper.znode.parent=/hbase, hbase.client.retries.number=35}, username='tianq'} 14/07/15 18:06:08 INFO client.HConnectionManager: ###create new connection### 14/07/15 18:06:08 INFO zookeeper.RecoverableZooKeeper: Process identifier=hconnection-0x3d7c3d7c connecting to ZooKeeper ensemble=localhost:2181 14/07/15 18:06:09 INFO client.HConnectionManager: ###create new connectionkey: HConnectionKey{properties={hbase.rpc.timeout=60, hbase.client.instance.id=3, hbase.zookeeper.quorum=localhost, hbase.client.pause=100, hbase.zookeeper.property.clientPort=2181, zookeeper.znode.parent=/hbase, hbase.client.retries.number=35}, username='tianq'} 14/07/15 18:06:09 INFO client.HConnectionManager: ###create new connection### 14/07/15 18:06:09 INFO zookeeper.RecoverableZooKeeper: Process identifier=hconnection-0xd460d46 connecting to ZooKeeper ensemble=localhost:2181 -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11517) TestReplicaWithCluster turns zombie
[ https://issues.apache.org/jira/browse/HBASE-11517?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14062100#comment-14062100 ] stack commented on HBASE-11517: --- This https://builds.apache.org/job/HBase-TRUNK/5304/changes build of trunk has the hang TestReplicaWithCluster turns zombie --- Key: HBASE-11517 URL: https://issues.apache.org/jira/browse/HBASE-11517 Project: HBase Issue Type: Bug Reporter: stack Assignee: stack Attachments: 10930v4.txt, 11517.timeouts.txt Happened a few times for me fixing unrelated findbugs. Here is example: https://builds.apache.org/job/PreCommit-HBASE-Build/10065//consoleFull See how it is hanging creating a table: pool-1-thread-1 prio=10 tid=0x7f1714657000 nid=0x4b7f waiting on condition [0x7f16e9f8] java.lang.Thread.State: TIMED_WAITING (sleeping) at java.lang.Thread.sleep(Native Method) at org.apache.hadoop.hbase.client.HBaseAdmin.createTable(HBaseAdmin.java:539) at org.apache.hadoop.hbase.client.HBaseAdmin.createTable(HBaseAdmin.java:424) at org.apache.hadoop.hbase.HBaseTestingUtility.createTable(HBaseTestingUtility.java:1185) at org.apache.hadoop.hbase.client.TestReplicaWithCluster.testCreateDeleteTable(TestReplicaWithCluster.java:138) -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11514) Fix findbugs warnings in blockcache
[ https://issues.apache.org/jira/browse/HBASE-11514?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-11514: -- Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Committed to master and branch-1. I see the test failure in builds where this patch is not present: e.g.https://builds.apache.org/job/HBase-TRUNK/5304/changes. Will look at test failure over in HBASE-11517 Fix findbugs warnings in blockcache --- Key: HBASE-11514 URL: https://issues.apache.org/jira/browse/HBASE-11514 Project: HBase Issue Type: Bug Reporter: stack Assignee: stack Fix For: 0.99.0, 2.0.0 Attachments: 10930.txt, 10930v2.txt, 10930v3.txt, 10930v3.txt Fixing a few findbugs issues around blockcache. Adding equals when we have compareTo. Noticed that hbase.bucketcache.percentage.in.combinedcache is NOT used anywhere though we doc it as working. Rather than implement, I'm removing it. Makes config of cache cleaner (do the L1 and then do the L2 independent of each other rather than have independent config and then this odd hbase.bucketcache.percentage.in.combinedcache that spans them). Changed the package doc to remove reference to hbase.bucketcache.percentage.in.combinedcache (did an edit too removing slabcache mentions in package-info). -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11240) Print hdfs pipeline when hlog's sync is slow
[ https://issues.apache.org/jira/browse/HBASE-11240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-11240: -- Fix Version/s: 0.99.0 Release Note: Adds emission of a log message that lists datanodes in HDFS pipeline when sync is slow. Set when to log with hbase.regionserver.hlog.slowsync.ms. Defaults logging if sync takes 100ms. Committed to branch-1, both original patch and the addendum. Enis asked that we do something about too much log when slow. I see this happening in recent runs. Should be ameliorated by fix to timing calc but my guess is that we will still have 'too much'. Can address in a follow on patch. Thanks for the nice addition [~liushaohui] Print hdfs pipeline when hlog's sync is slow Key: HBASE-11240 URL: https://issues.apache.org/jira/browse/HBASE-11240 Project: HBase Issue Type: Improvement Components: Operability, wal Reporter: Liu Shaohui Assignee: Liu Shaohui Fix For: 0.99.0, 2.0.0 Attachments: HBASE-11240-addendum.diff, HBASE-11240-trunk-v1.diff, HBASE-11240-trunk-v2.diff Sometimes the slow sync of hlog writer is because there is an abnormal datanode in the pipeline. So it will be helpful to print the pipeline of slow sync to diagnose those problems. The ultimate solution is to join the trace of HBase and HDFS. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11240) Print hdfs pipeline when hlog's sync is slow
[ https://issues.apache.org/jira/browse/HBASE-11240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14062179#comment-14062179 ] stack commented on HBASE-11240: --- Oh, I also applied the ADDENDUM to master. Print hdfs pipeline when hlog's sync is slow Key: HBASE-11240 URL: https://issues.apache.org/jira/browse/HBASE-11240 Project: HBase Issue Type: Improvement Components: Operability, wal Reporter: Liu Shaohui Assignee: Liu Shaohui Fix For: 0.99.0, 2.0.0 Attachments: HBASE-11240-addendum.diff, HBASE-11240-trunk-v1.diff, HBASE-11240-trunk-v2.diff Sometimes the slow sync of hlog writer is because there is an abnormal datanode in the pipeline. So it will be helpful to print the pipeline of slow sync to diagnose those problems. The ultimate solution is to join the trace of HBase and HDFS. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11240) Print hdfs pipeline when hlog's sync is slow
[ https://issues.apache.org/jira/browse/HBASE-11240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14062230#comment-14062230 ] Hudson commented on HBASE-11240: FAILURE: Integrated in HBase-1.0 #43 (See [https://builds.apache.org/job/HBase-1.0/43/]) HBASE-11240 Print hdfs pipeline when hlog's sync is slow (Original patch + ADDENDUM) (stack: rev a6d271201f132b254613b1aa839a503bb61be4bc) * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/FSHLog.java Print hdfs pipeline when hlog's sync is slow Key: HBASE-11240 URL: https://issues.apache.org/jira/browse/HBASE-11240 Project: HBase Issue Type: Improvement Components: Operability, wal Reporter: Liu Shaohui Assignee: Liu Shaohui Fix For: 0.99.0, 2.0.0 Attachments: HBASE-11240-addendum.diff, HBASE-11240-trunk-v1.diff, HBASE-11240-trunk-v2.diff Sometimes the slow sync of hlog writer is because there is an abnormal datanode in the pipeline. So it will be helpful to print the pipeline of slow sync to diagnose those problems. The ultimate solution is to join the trace of HBase and HDFS. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11514) Fix findbugs warnings in blockcache
[ https://issues.apache.org/jira/browse/HBASE-11514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14062231#comment-14062231 ] Hudson commented on HBASE-11514: FAILURE: Integrated in HBase-1.0 #43 (See [https://builds.apache.org/job/HBase-1.0/43/]) HBASE-11514 Fix findbugs warnings in blockcache (stack: rev 2a20143f72c32ededd90bcdbb2e162174498914c) * hbase-server/src/test/java/org/apache/hadoop/hbase/io/hfile/TestCacheConfig.java * hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/CacheConfig.java * hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/package-info.java * hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/bucket/BucketCache.java * src/main/docbkx/book.xml * hbase-server/src/test/java/org/apache/hadoop/hbase/io/hfile/TestBlockCacheReporting.java * hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/LruBlockCache.java Fix findbugs warnings in blockcache --- Key: HBASE-11514 URL: https://issues.apache.org/jira/browse/HBASE-11514 Project: HBase Issue Type: Bug Reporter: stack Assignee: stack Fix For: 0.99.0, 2.0.0 Attachments: 10930.txt, 10930v2.txt, 10930v3.txt, 10930v3.txt Fixing a few findbugs issues around blockcache. Adding equals when we have compareTo. Noticed that hbase.bucketcache.percentage.in.combinedcache is NOT used anywhere though we doc it as working. Rather than implement, I'm removing it. Makes config of cache cleaner (do the L1 and then do the L2 independent of each other rather than have independent config and then this odd hbase.bucketcache.percentage.in.combinedcache that spans them). Changed the package doc to remove reference to hbase.bucketcache.percentage.in.combinedcache (did an edit too removing slabcache mentions in package-info). -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-10877) HBase non-retriable exception list should be expanded
[ https://issues.apache.org/jira/browse/HBASE-10877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14062296#comment-14062296 ] Nick Dimiduk commented on HBASE-10877: -- Sounds like you folks are experiencing the issue from HBASE-10304. As of HBASE-8 we have a solution such that adding hbase-protocol.jar to the launch classpath is no longer necessary. This fix will be shipped in 0.98.4. I was working with another user on that ticket to verify that the solution resolves the issue for them. Maybe have a look at the comments over there and verify for us? If things are not working, we should get it resolved now, before 0.98.4 is official. [~xiemeilong], [~CodingCat], [~ted.m]: please have a look at HBASE-8 and continue the conversation over there. HBase non-retriable exception list should be expanded - Key: HBASE-10877 URL: https://issues.apache.org/jira/browse/HBASE-10877 Project: HBase Issue Type: Improvement Reporter: Sergey Shelukhin Priority: Minor Example where retries do not make sense: {noformat} 2014-03-31 20:54:27,765 WARN [InputInitializer [Map 1] #0] org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation: Encountered problems when prefetch hbase:meta table: org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after attempts=35, exceptions: Mon Mar 31 20:45:17 UTC 2014, org.apache.hadoop.hbase.client.RpcRetryingCaller@343d511e, java.lang.IllegalAccessError: class com.google.protobuf.HBaseZeroCopyByteString cannot access its superclass com.google.protobuf.LiteralByteString Mon Mar 31 20:45:17 UTC 2014, org.apache.hadoop.hbase.client.RpcRetryingCaller@343d511e, java.lang.IllegalAccessError: com/google/protobuf/HBaseZeroCopyByteString Mon Mar 31 20:45:17 UTC 2014, org.apache.hadoop.hbase.client.RpcRetryingCaller@343d511e, java.lang.IllegalAccessError: com/google/protobuf/HBaseZeroCopyByteString Mon Mar 31 20:45:18 UTC 2014, org.apache.hadoop.hbase.client.RpcRetryingCaller@343d511e, java.lang.IllegalAccessError: com/google/protobuf/HBaseZeroCopyByteString Mon Mar 31 20:45:20 UTC 2014, org.apache.hadoop.hbase.client.RpcRetryingCaller@343d511e, java.lang.IllegalAccessError: com/google/protobuf/HBaseZeroCopyByteString Mon Mar 31 20:45:24 UTC 2014, org.apache.hadoop.hbase.client.RpcRetryingCaller@343d511e, java.lang.IllegalAccessError: com/google/protobuf/HBaseZeroCopyByteString Mon Mar 31 20:45:34 UTC 2014, org.apache.hadoop.hbase.client.RpcRetryingCaller@343d511e, java.lang.IllegalAccessError: com/google/protobuf/HBaseZeroCopyByteString Mon Mar 31 20:45:45 UTC 2014, org.apache.hadoop.hbase.client.RpcRetryingCaller@343d511e, java.lang.IllegalAccessError: com/google/protobuf/HBaseZeroCopyByteString Mon Mar 31 20:45:55 UTC 2014, org.apache.hadoop.hbase.client.RpcRetryingCaller@343d511e, java.lang.IllegalAccessError: com/google/protobuf/HBaseZeroCopyByteString Mon Mar 31 20:46:05 UTC 2014, org.apache.hadoop.hbase.client.RpcRetryingCaller@343d511e, java.lang.IllegalAccessError: com/google/protobuf/HBaseZeroCopyByteString Mon Mar 31 20:46:25 UTC 2014, org.apache.hadoop.hbase.client.RpcRetryingCaller@343d511e, java.lang.IllegalAccessError: com/google/protobuf/HBaseZeroCopyByteString Mon Mar 31 20:46:45 UTC 2014, org.apache.hadoop.hbase.client.RpcRetryingCaller@343d511e, java.lang.IllegalAccessError: com/google/protobuf/HBaseZeroCopyByteString Mon Mar 31 20:47:05 UTC 2014, org.apache.hadoop.hbase.client.RpcRetryingCaller@343d511e, java.lang.IllegalAccessError: com/google/protobuf/HBaseZeroCopyByteString Mon Mar 31 20:47:25 UTC 2014, org.apache.hadoop.hbase.client.RpcRetryingCaller@343d511e, java.lang.IllegalAccessError: com/google/protobuf/HBaseZeroCopyByteString Mon Mar 31 20:47:45 UTC 2014, org.apache.hadoop.hbase.client.RpcRetryingCaller@343d511e, java.lang.IllegalAccessError: com/google/protobuf/HBaseZeroCopyByteString Mon Mar 31 20:48:05 UTC 2014, org.apache.hadoop.hbase.client.RpcRetryingCaller@343d511e, java.lang.IllegalAccessError: com/google/protobuf/HBaseZeroCopyByteString Mon Mar 31 20:48:25 UTC 2014, org.apache.hadoop.hbase.client.RpcRetryingCaller@343d511e, java.lang.IllegalAccessError: com/google/protobuf/HBaseZeroCopyByteString Mon Mar 31 20:48:46 UTC 2014, org.apache.hadoop.hbase.client.RpcRetryingCaller@343d511e, java.lang.IllegalAccessError: com/google/protobuf/HBaseZeroCopyByteString Mon Mar 31 20:49:06 UTC 2014, org.apache.hadoop.hbase.client.RpcRetryingCaller@343d511e, java.lang.IllegalAccessError: com/google/protobuf/HBaseZeroCopyByteString Mon Mar 31 20:49:26 UTC 2014, org.apache.hadoop.hbase.client.RpcRetryingCaller@343d511e, java.lang.IllegalAccessError: com/google/protobuf/HBaseZeroCopyByteString Mon Mar 31
[jira] [Updated] (HBASE-10930) Change Filters and GetClosestRowBeforeTracker to work with Cells
[ https://issues.apache.org/jira/browse/HBASE-10930?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-10930: --- Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Committed to trunk. Thanks for the review Stack. Change Filters and GetClosestRowBeforeTracker to work with Cells Key: HBASE-10930 URL: https://issues.apache.org/jira/browse/HBASE-10930 Project: HBase Issue Type: Sub-task Reporter: ramkrishna.s.vasudevan Assignee: ramkrishna.s.vasudevan Fix For: 0.99.0 Attachments: HBASE-10930.patch, HBASE-10930_1.patch, HBASE-10930_2.patch -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11240) Print hdfs pipeline when hlog's sync is slow
[ https://issues.apache.org/jira/browse/HBASE-11240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14062307#comment-14062307 ] Enis Soztutar commented on HBASE-11240: --- Thanks Liu, Stack. bq. Should be ameliorated by fix to timing calc but my guess is that we will still have 'too much'. Can address in a follow on patch. Yes, it would be awesome if we fix this as well. Print hdfs pipeline when hlog's sync is slow Key: HBASE-11240 URL: https://issues.apache.org/jira/browse/HBASE-11240 Project: HBase Issue Type: Improvement Components: Operability, wal Reporter: Liu Shaohui Assignee: Liu Shaohui Fix For: 0.99.0, 2.0.0 Attachments: HBASE-11240-addendum.diff, HBASE-11240-trunk-v1.diff, HBASE-11240-trunk-v2.diff Sometimes the slow sync of hlog writer is because there is an abnormal datanode in the pipeline. So it will be helpful to print the pipeline of slow sync to diagnose those problems. The ultimate solution is to join the trace of HBase and HDFS. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-10930) Change Filters and GetClosestRowBeforeTracker to work with Cells
[ https://issues.apache.org/jira/browse/HBASE-10930?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-10930: --- Attachment: HBASE-10930_3.patch Change Filters and GetClosestRowBeforeTracker to work with Cells Key: HBASE-10930 URL: https://issues.apache.org/jira/browse/HBASE-10930 Project: HBase Issue Type: Sub-task Reporter: ramkrishna.s.vasudevan Assignee: ramkrishna.s.vasudevan Fix For: 0.99.0 Attachments: HBASE-10930.patch, HBASE-10930_1.patch, HBASE-10930_2.patch, HBASE-10930_3.patch -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11489) ClassNotFoundException while running IT tests in trunk using 'mvn verify'
[ https://issues.apache.org/jira/browse/HBASE-11489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14062312#comment-14062312 ] Enis Soztutar commented on HBASE-11489: --- +1 for branch-1 as well. Does this affect 0.98 ? ClassNotFoundException while running IT tests in trunk using 'mvn verify' - Key: HBASE-11489 URL: https://issues.apache.org/jira/browse/HBASE-11489 Project: HBase Issue Type: Bug Reporter: ramkrishna.s.vasudevan Assignee: ramkrishna.s.vasudevan Fix For: 0.99.0, 2.0.0 Attachments: HBASE-11489.patch, HBASE-11489_1.patch Trying to run {code} mvn verify -Dit.test=IntegrationTestBigLinkedList -Dtest.output.tofile=false {code} causes this ClassNotFoundException issue bq.testContinuousIngest(org.apache.hadoop.hbase.test.IntegrationTestBigLinkedList): org/jboss/netty/channel/ChannelFactory -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11517) TestReplicaWithCluster turns zombie
[ https://issues.apache.org/jira/browse/HBASE-11517?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14062317#comment-14062317 ] Enis Soztutar commented on HBASE-11517: --- Thanks Stack. I've noticed this too. I'll keep an eye out after this patch. +1. TestReplicaWithCluster turns zombie --- Key: HBASE-11517 URL: https://issues.apache.org/jira/browse/HBASE-11517 Project: HBase Issue Type: Bug Reporter: stack Assignee: stack Attachments: 10930v4.txt, 11517.timeouts.txt Happened a few times for me fixing unrelated findbugs. Here is example: https://builds.apache.org/job/PreCommit-HBASE-Build/10065//consoleFull See how it is hanging creating a table: pool-1-thread-1 prio=10 tid=0x7f1714657000 nid=0x4b7f waiting on condition [0x7f16e9f8] java.lang.Thread.State: TIMED_WAITING (sleeping) at java.lang.Thread.sleep(Native Method) at org.apache.hadoop.hbase.client.HBaseAdmin.createTable(HBaseAdmin.java:539) at org.apache.hadoop.hbase.client.HBaseAdmin.createTable(HBaseAdmin.java:424) at org.apache.hadoop.hbase.HBaseTestingUtility.createTable(HBaseTestingUtility.java:1185) at org.apache.hadoop.hbase.client.TestReplicaWithCluster.testCreateDeleteTable(TestReplicaWithCluster.java:138) -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-10877) HBase non-retriable exception list should be expanded
[ https://issues.apache.org/jira/browse/HBASE-10877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14062318#comment-14062318 ] stack commented on HBASE-10877: --- I added note to refguide troubleshooting section highlighting this exception point folks at HBASE-8, etc. HBase non-retriable exception list should be expanded - Key: HBASE-10877 URL: https://issues.apache.org/jira/browse/HBASE-10877 Project: HBase Issue Type: Improvement Reporter: Sergey Shelukhin Priority: Minor Example where retries do not make sense: {noformat} 2014-03-31 20:54:27,765 WARN [InputInitializer [Map 1] #0] org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation: Encountered problems when prefetch hbase:meta table: org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after attempts=35, exceptions: Mon Mar 31 20:45:17 UTC 2014, org.apache.hadoop.hbase.client.RpcRetryingCaller@343d511e, java.lang.IllegalAccessError: class com.google.protobuf.HBaseZeroCopyByteString cannot access its superclass com.google.protobuf.LiteralByteString Mon Mar 31 20:45:17 UTC 2014, org.apache.hadoop.hbase.client.RpcRetryingCaller@343d511e, java.lang.IllegalAccessError: com/google/protobuf/HBaseZeroCopyByteString Mon Mar 31 20:45:17 UTC 2014, org.apache.hadoop.hbase.client.RpcRetryingCaller@343d511e, java.lang.IllegalAccessError: com/google/protobuf/HBaseZeroCopyByteString Mon Mar 31 20:45:18 UTC 2014, org.apache.hadoop.hbase.client.RpcRetryingCaller@343d511e, java.lang.IllegalAccessError: com/google/protobuf/HBaseZeroCopyByteString Mon Mar 31 20:45:20 UTC 2014, org.apache.hadoop.hbase.client.RpcRetryingCaller@343d511e, java.lang.IllegalAccessError: com/google/protobuf/HBaseZeroCopyByteString Mon Mar 31 20:45:24 UTC 2014, org.apache.hadoop.hbase.client.RpcRetryingCaller@343d511e, java.lang.IllegalAccessError: com/google/protobuf/HBaseZeroCopyByteString Mon Mar 31 20:45:34 UTC 2014, org.apache.hadoop.hbase.client.RpcRetryingCaller@343d511e, java.lang.IllegalAccessError: com/google/protobuf/HBaseZeroCopyByteString Mon Mar 31 20:45:45 UTC 2014, org.apache.hadoop.hbase.client.RpcRetryingCaller@343d511e, java.lang.IllegalAccessError: com/google/protobuf/HBaseZeroCopyByteString Mon Mar 31 20:45:55 UTC 2014, org.apache.hadoop.hbase.client.RpcRetryingCaller@343d511e, java.lang.IllegalAccessError: com/google/protobuf/HBaseZeroCopyByteString Mon Mar 31 20:46:05 UTC 2014, org.apache.hadoop.hbase.client.RpcRetryingCaller@343d511e, java.lang.IllegalAccessError: com/google/protobuf/HBaseZeroCopyByteString Mon Mar 31 20:46:25 UTC 2014, org.apache.hadoop.hbase.client.RpcRetryingCaller@343d511e, java.lang.IllegalAccessError: com/google/protobuf/HBaseZeroCopyByteString Mon Mar 31 20:46:45 UTC 2014, org.apache.hadoop.hbase.client.RpcRetryingCaller@343d511e, java.lang.IllegalAccessError: com/google/protobuf/HBaseZeroCopyByteString Mon Mar 31 20:47:05 UTC 2014, org.apache.hadoop.hbase.client.RpcRetryingCaller@343d511e, java.lang.IllegalAccessError: com/google/protobuf/HBaseZeroCopyByteString Mon Mar 31 20:47:25 UTC 2014, org.apache.hadoop.hbase.client.RpcRetryingCaller@343d511e, java.lang.IllegalAccessError: com/google/protobuf/HBaseZeroCopyByteString Mon Mar 31 20:47:45 UTC 2014, org.apache.hadoop.hbase.client.RpcRetryingCaller@343d511e, java.lang.IllegalAccessError: com/google/protobuf/HBaseZeroCopyByteString Mon Mar 31 20:48:05 UTC 2014, org.apache.hadoop.hbase.client.RpcRetryingCaller@343d511e, java.lang.IllegalAccessError: com/google/protobuf/HBaseZeroCopyByteString Mon Mar 31 20:48:25 UTC 2014, org.apache.hadoop.hbase.client.RpcRetryingCaller@343d511e, java.lang.IllegalAccessError: com/google/protobuf/HBaseZeroCopyByteString Mon Mar 31 20:48:46 UTC 2014, org.apache.hadoop.hbase.client.RpcRetryingCaller@343d511e, java.lang.IllegalAccessError: com/google/protobuf/HBaseZeroCopyByteString Mon Mar 31 20:49:06 UTC 2014, org.apache.hadoop.hbase.client.RpcRetryingCaller@343d511e, java.lang.IllegalAccessError: com/google/protobuf/HBaseZeroCopyByteString Mon Mar 31 20:49:26 UTC 2014, org.apache.hadoop.hbase.client.RpcRetryingCaller@343d511e, java.lang.IllegalAccessError: com/google/protobuf/HBaseZeroCopyByteString Mon Mar 31 20:49:46 UTC 2014, org.apache.hadoop.hbase.client.RpcRetryingCaller@343d511e, java.lang.IllegalAccessError: com/google/protobuf/HBaseZeroCopyByteString Mon Mar 31 20:50:06 UTC 2014, org.apache.hadoop.hbase.client.RpcRetryingCaller@343d511e, java.lang.IllegalAccessError: com/google/protobuf/HBaseZeroCopyByteString Mon Mar 31 20:50:26 UTC 2014, org.apache.hadoop.hbase.client.RpcRetryingCaller@343d511e, java.lang.IllegalAccessError: com/google/protobuf/HBaseZeroCopyByteString Mon Mar 31
[jira] [Commented] (HBASE-11517) TestReplicaWithCluster turns zombie
[ https://issues.apache.org/jira/browse/HBASE-11517?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14062324#comment-14062324 ] stack commented on HBASE-11517: --- Ok. I applied the attached timeouts.txt 'debugging' patch to see if we can get more info on why its hanging up. TestReplicaWithCluster turns zombie --- Key: HBASE-11517 URL: https://issues.apache.org/jira/browse/HBASE-11517 Project: HBase Issue Type: Bug Reporter: stack Assignee: stack Attachments: 10930v4.txt, 11517.timeouts.txt Happened a few times for me fixing unrelated findbugs. Here is example: https://builds.apache.org/job/PreCommit-HBASE-Build/10065//consoleFull See how it is hanging creating a table: pool-1-thread-1 prio=10 tid=0x7f1714657000 nid=0x4b7f waiting on condition [0x7f16e9f8] java.lang.Thread.State: TIMED_WAITING (sleeping) at java.lang.Thread.sleep(Native Method) at org.apache.hadoop.hbase.client.HBaseAdmin.createTable(HBaseAdmin.java:539) at org.apache.hadoop.hbase.client.HBaseAdmin.createTable(HBaseAdmin.java:424) at org.apache.hadoop.hbase.HBaseTestingUtility.createTable(HBaseTestingUtility.java:1185) at org.apache.hadoop.hbase.client.TestReplicaWithCluster.testCreateDeleteTable(TestReplicaWithCluster.java:138) -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-10877) HBase non-retriable exception list should be expanded
[ https://issues.apache.org/jira/browse/HBASE-10877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14062341#comment-14062341 ] Nick Dimiduk commented on HBASE-10877: -- Thanks [~stack]. HBase non-retriable exception list should be expanded - Key: HBASE-10877 URL: https://issues.apache.org/jira/browse/HBASE-10877 Project: HBase Issue Type: Improvement Reporter: Sergey Shelukhin Priority: Minor Example where retries do not make sense: {noformat} 2014-03-31 20:54:27,765 WARN [InputInitializer [Map 1] #0] org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation: Encountered problems when prefetch hbase:meta table: org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after attempts=35, exceptions: Mon Mar 31 20:45:17 UTC 2014, org.apache.hadoop.hbase.client.RpcRetryingCaller@343d511e, java.lang.IllegalAccessError: class com.google.protobuf.HBaseZeroCopyByteString cannot access its superclass com.google.protobuf.LiteralByteString Mon Mar 31 20:45:17 UTC 2014, org.apache.hadoop.hbase.client.RpcRetryingCaller@343d511e, java.lang.IllegalAccessError: com/google/protobuf/HBaseZeroCopyByteString Mon Mar 31 20:45:17 UTC 2014, org.apache.hadoop.hbase.client.RpcRetryingCaller@343d511e, java.lang.IllegalAccessError: com/google/protobuf/HBaseZeroCopyByteString Mon Mar 31 20:45:18 UTC 2014, org.apache.hadoop.hbase.client.RpcRetryingCaller@343d511e, java.lang.IllegalAccessError: com/google/protobuf/HBaseZeroCopyByteString Mon Mar 31 20:45:20 UTC 2014, org.apache.hadoop.hbase.client.RpcRetryingCaller@343d511e, java.lang.IllegalAccessError: com/google/protobuf/HBaseZeroCopyByteString Mon Mar 31 20:45:24 UTC 2014, org.apache.hadoop.hbase.client.RpcRetryingCaller@343d511e, java.lang.IllegalAccessError: com/google/protobuf/HBaseZeroCopyByteString Mon Mar 31 20:45:34 UTC 2014, org.apache.hadoop.hbase.client.RpcRetryingCaller@343d511e, java.lang.IllegalAccessError: com/google/protobuf/HBaseZeroCopyByteString Mon Mar 31 20:45:45 UTC 2014, org.apache.hadoop.hbase.client.RpcRetryingCaller@343d511e, java.lang.IllegalAccessError: com/google/protobuf/HBaseZeroCopyByteString Mon Mar 31 20:45:55 UTC 2014, org.apache.hadoop.hbase.client.RpcRetryingCaller@343d511e, java.lang.IllegalAccessError: com/google/protobuf/HBaseZeroCopyByteString Mon Mar 31 20:46:05 UTC 2014, org.apache.hadoop.hbase.client.RpcRetryingCaller@343d511e, java.lang.IllegalAccessError: com/google/protobuf/HBaseZeroCopyByteString Mon Mar 31 20:46:25 UTC 2014, org.apache.hadoop.hbase.client.RpcRetryingCaller@343d511e, java.lang.IllegalAccessError: com/google/protobuf/HBaseZeroCopyByteString Mon Mar 31 20:46:45 UTC 2014, org.apache.hadoop.hbase.client.RpcRetryingCaller@343d511e, java.lang.IllegalAccessError: com/google/protobuf/HBaseZeroCopyByteString Mon Mar 31 20:47:05 UTC 2014, org.apache.hadoop.hbase.client.RpcRetryingCaller@343d511e, java.lang.IllegalAccessError: com/google/protobuf/HBaseZeroCopyByteString Mon Mar 31 20:47:25 UTC 2014, org.apache.hadoop.hbase.client.RpcRetryingCaller@343d511e, java.lang.IllegalAccessError: com/google/protobuf/HBaseZeroCopyByteString Mon Mar 31 20:47:45 UTC 2014, org.apache.hadoop.hbase.client.RpcRetryingCaller@343d511e, java.lang.IllegalAccessError: com/google/protobuf/HBaseZeroCopyByteString Mon Mar 31 20:48:05 UTC 2014, org.apache.hadoop.hbase.client.RpcRetryingCaller@343d511e, java.lang.IllegalAccessError: com/google/protobuf/HBaseZeroCopyByteString Mon Mar 31 20:48:25 UTC 2014, org.apache.hadoop.hbase.client.RpcRetryingCaller@343d511e, java.lang.IllegalAccessError: com/google/protobuf/HBaseZeroCopyByteString Mon Mar 31 20:48:46 UTC 2014, org.apache.hadoop.hbase.client.RpcRetryingCaller@343d511e, java.lang.IllegalAccessError: com/google/protobuf/HBaseZeroCopyByteString Mon Mar 31 20:49:06 UTC 2014, org.apache.hadoop.hbase.client.RpcRetryingCaller@343d511e, java.lang.IllegalAccessError: com/google/protobuf/HBaseZeroCopyByteString Mon Mar 31 20:49:26 UTC 2014, org.apache.hadoop.hbase.client.RpcRetryingCaller@343d511e, java.lang.IllegalAccessError: com/google/protobuf/HBaseZeroCopyByteString Mon Mar 31 20:49:46 UTC 2014, org.apache.hadoop.hbase.client.RpcRetryingCaller@343d511e, java.lang.IllegalAccessError: com/google/protobuf/HBaseZeroCopyByteString Mon Mar 31 20:50:06 UTC 2014, org.apache.hadoop.hbase.client.RpcRetryingCaller@343d511e, java.lang.IllegalAccessError: com/google/protobuf/HBaseZeroCopyByteString Mon Mar 31 20:50:26 UTC 2014, org.apache.hadoop.hbase.client.RpcRetryingCaller@343d511e, java.lang.IllegalAccessError: com/google/protobuf/HBaseZeroCopyByteString Mon Mar 31 20:50:46 UTC 2014, org.apache.hadoop.hbase.client.RpcRetryingCaller@343d511e,
[jira] [Resolved] (HBASE-11480) ClientScanner might not close the HConnection created in construction
[ https://issues.apache.org/jira/browse/HBASE-11480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack resolved HBASE-11480. --- Resolution: Fixed Fix Version/s: 0.94.22 0.98.5 0.96.3 Assignee: cuijianwei Release Note: Deprecate a (unused) ClientScanner constructor because can leave a connection open. Hadoop Flags: Reviewed Committed to 0.94, 0.96, and 0.98. Thanks for the patch [~cuijianwei] ClientScanner might not close the HConnection created in construction - Key: HBASE-11480 URL: https://issues.apache.org/jira/browse/HBASE-11480 Project: HBase Issue Type: Improvement Components: Client, Scanners Affects Versions: 0.96.2, 0.94.20, 0.98.3 Reporter: cuijianwei Assignee: cuijianwei Priority: Trivial Fix For: 0.96.3, 0.98.5, 0.94.22 Attachments: HBASE-11480-0.94-v1.patch, HBASE-11480-0.95-v1.patch, HBASE-11480-0.96-v1.patch, HBASE-11480-0.98-v1.patch ClientScanner will create HConnection in its construction: {code} public ClientScanner(final Configuration conf, final Scan scan, final byte[] tableName) throws IOException { this(conf, scan, tableName, HConnectionManager.getConnection(conf)); } {code} However, this connection won't be closed in ClientScanner.close(). Is it better to deprecate this construction? -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11438) [Visibility Controller] Support UTF8 character as Visibility Labels
[ https://issues.apache.org/jira/browse/HBASE-11438?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-11438: --- Description: This would be an action item that we would be addressing so that the visibility labels could have UTF8 characters in them. Also allow the user to use a client supplied API that allows to specify the visibility labels inside double quotes such that UTF8 characters and cases like , |, ! and double quotes itself could be specified with proper escape sequence. Accumulo too provides one such API in the client side. (was: This would be an action item that we would be addressing so that the visibility labels could have UTF8 characters in them.) [Visibility Controller] Support UTF8 character as Visibility Labels --- Key: HBASE-11438 URL: https://issues.apache.org/jira/browse/HBASE-11438 Project: HBase Issue Type: Improvement Components: security Affects Versions: 0.98.4 Reporter: ramkrishna.s.vasudevan Assignee: ramkrishna.s.vasudevan Fix For: 0.98.5 This would be an action item that we would be addressing so that the visibility labels could have UTF8 characters in them. Also allow the user to use a client supplied API that allows to specify the visibility labels inside double quotes such that UTF8 characters and cases like , |, ! and double quotes itself could be specified with proper escape sequence. Accumulo too provides one such API in the client side. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11240) Print hdfs pipeline when hlog's sync is slow
[ https://issues.apache.org/jira/browse/HBASE-11240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14062353#comment-14062353 ] Hudson commented on HBASE-11240: FAILURE: Integrated in HBase-TRUNK #5305 (See [https://builds.apache.org/job/HBase-TRUNK/5305/]) HBASE-11240 Print hdfs pipeline when hlog's sync is slow ADDENDUM (stack: rev c694ec11df3437b2ad24f365e3318dc7e931ddfc) * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/FSHLog.java Print hdfs pipeline when hlog's sync is slow Key: HBASE-11240 URL: https://issues.apache.org/jira/browse/HBASE-11240 Project: HBase Issue Type: Improvement Components: Operability, wal Reporter: Liu Shaohui Assignee: Liu Shaohui Fix For: 0.99.0, 2.0.0 Attachments: HBASE-11240-addendum.diff, HBASE-11240-trunk-v1.diff, HBASE-11240-trunk-v2.diff Sometimes the slow sync of hlog writer is because there is an abnormal datanode in the pipeline. So it will be helpful to print the pipeline of slow sync to diagnose those problems. The ultimate solution is to join the trace of HBase and HDFS. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-7336) HFileBlock.readAtOffset does not work well with multiple threads
[ https://issues.apache.org/jira/browse/HBASE-7336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14062356#comment-14062356 ] Vladimir Rodionov commented on HBASE-7336: -- h3. Effect of compaction on large scan operations and vice verse The compaction scanner and user scanner will compete for the same input stream (DFSInputStream). This results in a sub optimal performance for both of them, becuase *there is no guarantee that next call to read HFile block from the lucky scanner will use the same streaming API and pre-cached data will still be valid*. Yep? Both scanners, periodically, switch between stream/pread API calls, hdfs cache can not be used (?), performance of both of them is defined by positional read performance (which is low for scan mode operation). Is this correct assessment? HFileBlock.readAtOffset does not work well with multiple threads Key: HBASE-7336 URL: https://issues.apache.org/jira/browse/HBASE-7336 Project: HBase Issue Type: Sub-task Components: Performance Reporter: Lars Hofhansl Assignee: Lars Hofhansl Priority: Critical Fix For: 0.94.4, 0.95.0 Attachments: 7336-0.94.txt, 7336-0.96.txt HBase grinds to a halt when many threads scan along the same set of blocks and neither read short circuit is nor block caching is enabled for the dfs client ... disabling the block cache makes sense on very large scans. It turns out that synchronizing in istream in HFileBlock.readAtOffset is the culprit. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11489) ClassNotFoundException while running IT tests in trunk using 'mvn verify'
[ https://issues.apache.org/jira/browse/HBASE-11489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-11489: --- Resolution: Fixed Fix Version/s: (was: 2.0.0) 1.0.0 Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Committed to branch-1 and trunk. In 0.98 did not face these issues. Thanks for the reviews. ClassNotFoundException while running IT tests in trunk using 'mvn verify' - Key: HBASE-11489 URL: https://issues.apache.org/jira/browse/HBASE-11489 Project: HBase Issue Type: Bug Reporter: ramkrishna.s.vasudevan Assignee: ramkrishna.s.vasudevan Fix For: 0.99.0, 1.0.0 Attachments: HBASE-11489.patch, HBASE-11489_1.patch Trying to run {code} mvn verify -Dit.test=IntegrationTestBigLinkedList -Dtest.output.tofile=false {code} causes this ClassNotFoundException issue bq.testContinuousIngest(org.apache.hadoop.hbase.test.IntegrationTestBigLinkedList): org/jboss/netty/channel/ChannelFactory -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Assigned] (HBASE-11468) MasterAddressTracker needs to be moved to hbase-server
[ https://issues.apache.org/jira/browse/HBASE-11468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Soldatov reassigned HBASE-11468: --- Assignee: Sergey Soldatov MasterAddressTracker needs to be moved to hbase-server -- Key: HBASE-11468 URL: https://issues.apache.org/jira/browse/HBASE-11468 Project: HBase Issue Type: Sub-task Components: Client, Consensus, Zookeeper Reporter: Mikhail Antonov Assignee: Sergey Soldatov Fix For: 2.0.0 Later we should get rid of it. For now, for the purpose to abstract client from ZK, we don't use in in hbase-client (where we read master location elsewhere), but on the server side we can use it for now, and still keep current znode tracking location of current active master. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11516) Track time spent in executing coprocessors in each region.
[ https://issues.apache.org/jira/browse/HBASE-11516?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14062364#comment-14062364 ] Andrew Purtell commented on HBASE-11516: I like the idea but not the proposed changes. Suggestions 1) Expose this as a metric instead of changes to ClusterStatus. or 2) When changing ClusterStatus make it easy to add more metrics like this over time, something like: {noformat} repeated Metric metrics = 16; message Metric { required string name = 1; required uint64 value = 2; } {noformat} I'd prefer option 1. I don't think we should be passing metrics through ClusterStatus. However, if consensus opinion is we can and should do that here, then I prefer option 2 over the current patch. Track time spent in executing coprocessors in each region. -- Key: HBASE-11516 URL: https://issues.apache.org/jira/browse/HBASE-11516 Project: HBase Issue Type: Improvement Components: Coprocessors Affects Versions: 0.98.4 Reporter: Srikanth Srungarapu Assignee: Srikanth Srungarapu Priority: Minor Fix For: 0.98.5 Attachments: HBASE-11516.patch Currently, the time spent in executing coprocessors is not yet being tracked. This feature can be handy for debugging coprocessors in case of any trouble. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11514) Fix findbugs warnings in blockcache
[ https://issues.apache.org/jira/browse/HBASE-11514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14062354#comment-14062354 ] Hudson commented on HBASE-11514: FAILURE: Integrated in HBase-TRUNK #5305 (See [https://builds.apache.org/job/HBase-TRUNK/5305/]) HBASE-11514 Fix findbugs warnings in blockcache (stack: rev 96dcd67f565413c0efba38b32ba9d0f277e90fd9) * hbase-server/src/test/java/org/apache/hadoop/hbase/io/hfile/TestBlockCacheReporting.java * hbase-server/src/test/java/org/apache/hadoop/hbase/io/hfile/TestCacheConfig.java * hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/CacheConfig.java * hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/package-info.java * hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/bucket/BucketCache.java * hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/LruBlockCache.java * src/main/docbkx/book.xml Fix findbugs warnings in blockcache --- Key: HBASE-11514 URL: https://issues.apache.org/jira/browse/HBASE-11514 Project: HBase Issue Type: Bug Reporter: stack Assignee: stack Fix For: 0.99.0, 2.0.0 Attachments: 10930.txt, 10930v2.txt, 10930v3.txt, 10930v3.txt Fixing a few findbugs issues around blockcache. Adding equals when we have compareTo. Noticed that hbase.bucketcache.percentage.in.combinedcache is NOT used anywhere though we doc it as working. Rather than implement, I'm removing it. Makes config of cache cleaner (do the L1 and then do the L2 independent of each other rather than have independent config and then this odd hbase.bucketcache.percentage.in.combinedcache that spans them). Changed the package doc to remove reference to hbase.bucketcache.percentage.in.combinedcache (did an edit too removing slabcache mentions in package-info). -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11516) Track time spent in executing coprocessors in each region.
[ https://issues.apache.org/jira/browse/HBASE-11516?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14062395#comment-14062395 ] stack commented on HBASE-11516: --- Agree w/ Andy on not using ClusterStatus. We are making the info twice, once to emit via metrics and then again to put on the ClusterStatus message. ClusterStatus should die and master should consume metrics. Track time spent in executing coprocessors in each region. -- Key: HBASE-11516 URL: https://issues.apache.org/jira/browse/HBASE-11516 Project: HBase Issue Type: Improvement Components: Coprocessors Affects Versions: 0.98.4 Reporter: Srikanth Srungarapu Assignee: Srikanth Srungarapu Priority: Minor Fix For: 0.98.5 Attachments: HBASE-11516.patch Currently, the time spent in executing coprocessors is not yet being tracked. This feature can be handy for debugging coprocessors in case of any trouble. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11480) ClientScanner might not close the HConnection created in construction
[ https://issues.apache.org/jira/browse/HBASE-11480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14062399#comment-14062399 ] Hudson commented on HBASE-11480: FAILURE: Integrated in hbase-0.96 #408 (See [https://builds.apache.org/job/hbase-0.96/408/]) HBASE-11480 ClientScanner might not close the HConnection created in construction (cuijianwei) (stack: rev f58b45cae73b89b43b26bfa8872dbaedbba80dc3) * hbase-client/src/main/java/org/apache/hadoop/hbase/client/ClientScanner.java ClientScanner might not close the HConnection created in construction - Key: HBASE-11480 URL: https://issues.apache.org/jira/browse/HBASE-11480 Project: HBase Issue Type: Improvement Components: Client, Scanners Affects Versions: 0.96.2, 0.94.20, 0.98.3 Reporter: cuijianwei Assignee: cuijianwei Priority: Trivial Fix For: 0.96.3, 0.98.5, 0.94.22 Attachments: HBASE-11480-0.94-v1.patch, HBASE-11480-0.95-v1.patch, HBASE-11480-0.96-v1.patch, HBASE-11480-0.98-v1.patch ClientScanner will create HConnection in its construction: {code} public ClientScanner(final Configuration conf, final Scan scan, final byte[] tableName) throws IOException { this(conf, scan, tableName, HConnectionManager.getConnection(conf)); } {code} However, this connection won't be closed in ClientScanner.close(). Is it better to deprecate this construction? -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11516) Track time spent in executing coprocessors in each region.
[ https://issues.apache.org/jira/browse/HBASE-11516?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Srikanth Srungarapu updated HBASE-11516: Status: In Progress (was: Patch Available) Thanks folks for taking a look. Sure, I would upload a new patch as per first option suggested by [~andrew.purt...@gmail.com]. Track time spent in executing coprocessors in each region. -- Key: HBASE-11516 URL: https://issues.apache.org/jira/browse/HBASE-11516 Project: HBase Issue Type: Improvement Components: Coprocessors Affects Versions: 0.98.4 Reporter: Srikanth Srungarapu Assignee: Srikanth Srungarapu Priority: Minor Fix For: 0.98.5 Attachments: HBASE-11516.patch Currently, the time spent in executing coprocessors is not yet being tracked. This feature can be handy for debugging coprocessors in case of any trouble. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (HBASE-11519) ClusterStatus should die and master should consume metrics
Andrew Purtell created HBASE-11519: -- Summary: ClusterStatus should die and master should consume metrics Key: HBASE-11519 URL: https://issues.apache.org/jira/browse/HBASE-11519 Project: HBase Issue Type: Task Reporter: Andrew Purtell -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11516) Track time spent in executing coprocessors in each region.
[ https://issues.apache.org/jira/browse/HBASE-11516?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14062422#comment-14062422 ] Andrew Purtell commented on HBASE-11516: bq. ClusterStatus should die and master should consume metrics. That's great, used it to file HBASE-11519 Track time spent in executing coprocessors in each region. -- Key: HBASE-11516 URL: https://issues.apache.org/jira/browse/HBASE-11516 Project: HBase Issue Type: Improvement Components: Coprocessors Affects Versions: 0.98.4 Reporter: Srikanth Srungarapu Assignee: Srikanth Srungarapu Priority: Minor Fix For: 0.98.5 Attachments: HBASE-11516.patch Currently, the time spent in executing coprocessors is not yet being tracked. This feature can be handy for debugging coprocessors in case of any trouble. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11519) ClusterStatus should die and master should consume metrics
[ https://issues.apache.org/jira/browse/HBASE-11519?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-11519: --- Description: The master (or a separate monitoring UI) should consume metrics emitted by cluster daemons and watch ZooKeeper to determine cluster status rather than require daemons send ClusterStatus reports. ClusterStatus should die and master should consume metrics -- Key: HBASE-11519 URL: https://issues.apache.org/jira/browse/HBASE-11519 Project: HBase Issue Type: Task Reporter: Andrew Purtell The master (or a separate monitoring UI) should consume metrics emitted by cluster daemons and watch ZooKeeper to determine cluster status rather than require daemons send ClusterStatus reports. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11480) ClientScanner might not close the HConnection created in construction
[ https://issues.apache.org/jira/browse/HBASE-11480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14062443#comment-14062443 ] Hudson commented on HBASE-11480: FAILURE: Integrated in HBase-0.94-security #499 (See [https://builds.apache.org/job/HBase-0.94-security/499/]) HBASE-11480 ClientScanner might not close the HConnection created in construction (cuijianwei) (stack: rev 4453e9cf6b6306a6c49359f1fe87263a72ef1c30) * src/main/java/org/apache/hadoop/hbase/client/ClientScanner.java ClientScanner might not close the HConnection created in construction - Key: HBASE-11480 URL: https://issues.apache.org/jira/browse/HBASE-11480 Project: HBase Issue Type: Improvement Components: Client, Scanners Affects Versions: 0.96.2, 0.94.20, 0.98.3 Reporter: cuijianwei Assignee: cuijianwei Priority: Trivial Fix For: 0.96.3, 0.98.5, 0.94.22 Attachments: HBASE-11480-0.94-v1.patch, HBASE-11480-0.95-v1.patch, HBASE-11480-0.96-v1.patch, HBASE-11480-0.98-v1.patch ClientScanner will create HConnection in its construction: {code} public ClientScanner(final Configuration conf, final Scan scan, final byte[] tableName) throws IOException { this(conf, scan, tableName, HConnectionManager.getConnection(conf)); } {code} However, this connection won't be closed in ClientScanner.close(). Is it better to deprecate this construction? -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Assigned] (HBASE-11470) Move Server interface to hbase-server
[ https://issues.apache.org/jira/browse/HBASE-11470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikhail Antonov reassigned HBASE-11470: --- Assignee: Alex Newman Move Server interface to hbase-server - Key: HBASE-11470 URL: https://issues.apache.org/jira/browse/HBASE-11470 Project: HBase Issue Type: Sub-task Components: Client, Consensus, Zookeeper Reporter: Mikhail Antonov Assignee: Alex Newman Fix For: 2.0.0 Looks like it's not being used on client side anyway. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Work started] (HBASE-11470) Move Server interface to hbase-server
[ https://issues.apache.org/jira/browse/HBASE-11470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HBASE-11470 started by Alex Newman. Move Server interface to hbase-server - Key: HBASE-11470 URL: https://issues.apache.org/jira/browse/HBASE-11470 Project: HBase Issue Type: Sub-task Components: Client, Consensus, Zookeeper Reporter: Mikhail Antonov Assignee: Alex Newman Fix For: 2.0.0 Looks like it's not being used on client side anyway. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11471) Move TableStateManager and ZkTableStateManager and Server to hbase-server
[ https://issues.apache.org/jira/browse/HBASE-11471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Newman updated HBASE-11471: Summary: Move TableStateManager and ZkTableStateManager and Server to hbase-server (was: Move TableStateManager and ZkTableStateManager to hbase-server) Move TableStateManager and ZkTableStateManager and Server to hbase-server - Key: HBASE-11471 URL: https://issues.apache.org/jira/browse/HBASE-11471 Project: HBase Issue Type: Sub-task Components: Client, Consensus, Zookeeper Reporter: Mikhail Antonov Assignee: Alex Newman Fix For: 2.0.0 -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11480) ClientScanner might not close the HConnection created in construction
[ https://issues.apache.org/jira/browse/HBASE-11480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14062491#comment-14062491 ] Hudson commented on HBASE-11480: SUCCESS: Integrated in HBase-0.94-JDK7 #154 (See [https://builds.apache.org/job/HBase-0.94-JDK7/154/]) HBASE-11480 ClientScanner might not close the HConnection created in construction (cuijianwei) (stack: rev 4453e9cf6b6306a6c49359f1fe87263a72ef1c30) * src/main/java/org/apache/hadoop/hbase/client/ClientScanner.java ClientScanner might not close the HConnection created in construction - Key: HBASE-11480 URL: https://issues.apache.org/jira/browse/HBASE-11480 Project: HBase Issue Type: Improvement Components: Client, Scanners Affects Versions: 0.96.2, 0.94.20, 0.98.3 Reporter: cuijianwei Assignee: cuijianwei Priority: Trivial Fix For: 0.96.3, 0.98.5, 0.94.22 Attachments: HBASE-11480-0.94-v1.patch, HBASE-11480-0.95-v1.patch, HBASE-11480-0.96-v1.patch, HBASE-11480-0.98-v1.patch ClientScanner will create HConnection in its construction: {code} public ClientScanner(final Configuration conf, final Scan scan, final byte[] tableName) throws IOException { this(conf, scan, tableName, HConnectionManager.getConnection(conf)); } {code} However, this connection won't be closed in ClientScanner.close(). Is it better to deprecate this construction? -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Resolved] (HBASE-11470) Move Server interface to hbase-server
[ https://issues.apache.org/jira/browse/HBASE-11470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Newman resolved HBASE-11470. - Resolution: Duplicate dup of HBASE-11471 Move Server interface to hbase-server - Key: HBASE-11470 URL: https://issues.apache.org/jira/browse/HBASE-11470 Project: HBase Issue Type: Sub-task Components: Client, Consensus, Zookeeper Reporter: Mikhail Antonov Assignee: Alex Newman Fix For: 2.0.0 Looks like it's not being used on client side anyway. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11489) ClassNotFoundException while running IT tests in trunk using 'mvn verify'
[ https://issues.apache.org/jira/browse/HBASE-11489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14062524#comment-14062524 ] Hudson commented on HBASE-11489: FAILURE: Integrated in HBase-TRUNK #5306 (See [https://builds.apache.org/job/HBase-TRUNK/5306/]) HBASE-11489 ClassNotFoundException while running IT tests in trunk using (ramkrishna: rev fe50c6d366092baf2232b8649b63dd714e4064b8) * hbase-it/pom.xml ClassNotFoundException while running IT tests in trunk using 'mvn verify' - Key: HBASE-11489 URL: https://issues.apache.org/jira/browse/HBASE-11489 Project: HBase Issue Type: Bug Reporter: ramkrishna.s.vasudevan Assignee: ramkrishna.s.vasudevan Fix For: 0.99.0, 1.0.0 Attachments: HBASE-11489.patch, HBASE-11489_1.patch Trying to run {code} mvn verify -Dit.test=IntegrationTestBigLinkedList -Dtest.output.tofile=false {code} causes this ClassNotFoundException issue bq.testContinuousIngest(org.apache.hadoop.hbase.test.IntegrationTestBigLinkedList): org/jboss/netty/channel/ChannelFactory -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11517) TestReplicaWithCluster turns zombie
[ https://issues.apache.org/jira/browse/HBASE-11517?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14062523#comment-14062523 ] Hudson commented on HBASE-11517: FAILURE: Integrated in HBase-TRUNK #5306 (See [https://builds.apache.org/job/HBase-TRUNK/5306/]) HBASE-11517 TestReplicaWithCluster turns zombie -- ADDS TIMEOUTS SO CAN DEBUG ZOMBIE (stack: rev bb73791dade15f389dfc1bacd20ca8b177bd39e7) * hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestReplicaWithCluster.java TestReplicaWithCluster turns zombie --- Key: HBASE-11517 URL: https://issues.apache.org/jira/browse/HBASE-11517 Project: HBase Issue Type: Bug Reporter: stack Assignee: stack Attachments: 10930v4.txt, 11517.timeouts.txt Happened a few times for me fixing unrelated findbugs. Here is example: https://builds.apache.org/job/PreCommit-HBASE-Build/10065//consoleFull See how it is hanging creating a table: pool-1-thread-1 prio=10 tid=0x7f1714657000 nid=0x4b7f waiting on condition [0x7f16e9f8] java.lang.Thread.State: TIMED_WAITING (sleeping) at java.lang.Thread.sleep(Native Method) at org.apache.hadoop.hbase.client.HBaseAdmin.createTable(HBaseAdmin.java:539) at org.apache.hadoop.hbase.client.HBaseAdmin.createTable(HBaseAdmin.java:424) at org.apache.hadoop.hbase.HBaseTestingUtility.createTable(HBaseTestingUtility.java:1185) at org.apache.hadoop.hbase.client.TestReplicaWithCluster.testCreateDeleteTable(TestReplicaWithCluster.java:138) -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-10930) Change Filters and GetClosestRowBeforeTracker to work with Cells
[ https://issues.apache.org/jira/browse/HBASE-10930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14062526#comment-14062526 ] Hudson commented on HBASE-10930: FAILURE: Integrated in HBase-TRUNK #5306 (See [https://builds.apache.org/job/HBase-TRUNK/5306/]) HBASE-10930 Change Filters and GetClosestRowBeforeTracker to work with (ramkrishna: rev 995a5a6c6838164f5baa5ec96f4395edd1c50d56) * hbase-server/src/test/java/org/apache/hadoop/hbase/filter/TestFilterList.java * hbase-client/src/main/java/org/apache/hadoop/hbase/filter/Filter.java * hbase-client/src/main/java/org/apache/hadoop/hbase/filter/FilterWrapper.java * hbase-client/src/main/java/org/apache/hadoop/hbase/filter/KeyOnlyFilter.java * hbase-client/src/main/java/org/apache/hadoop/hbase/filter/ColumnPaginationFilter.java * hbase-server/src/main/java/org/apache/hadoop/hbase/security/visibility/VisibilityController.java * hbase-client/src/main/java/org/apache/hadoop/hbase/filter/SingleColumnValueExcludeFilter.java * hbase-common/src/main/java/org/apache/hadoop/hbase/CellComparator.java * hbase-client/src/main/java/org/apache/hadoop/hbase/filter/FilterList.java * hbase-client/src/main/java/org/apache/hadoop/hbase/filter/ColumnRangeFilter.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/GetClosestRowBeforeTracker.java * hbase-client/src/main/java/org/apache/hadoop/hbase/filter/ColumnPrefixFilter.java * hbase-client/src/main/java/org/apache/hadoop/hbase/filter/SingleColumnValueFilter.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java * hbase-client/src/main/java/org/apache/hadoop/hbase/filter/DependentColumnFilter.java * hbase-client/src/main/java/org/apache/hadoop/hbase/filter/FilterBase.java * hbase-client/src/main/java/org/apache/hadoop/hbase/filter/MultipleColumnPrefixFilter.java * hbase-client/src/main/java/org/apache/hadoop/hbase/filter/FuzzyRowFilter.java Change Filters and GetClosestRowBeforeTracker to work with Cells Key: HBASE-10930 URL: https://issues.apache.org/jira/browse/HBASE-10930 Project: HBase Issue Type: Sub-task Reporter: ramkrishna.s.vasudevan Assignee: ramkrishna.s.vasudevan Fix For: 0.99.0 Attachments: HBASE-10930.patch, HBASE-10930_1.patch, HBASE-10930_2.patch, HBASE-10930_3.patch -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11480) ClientScanner might not close the HConnection created in construction
[ https://issues.apache.org/jira/browse/HBASE-11480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14062536#comment-14062536 ] Hudson commented on HBASE-11480: SUCCESS: Integrated in HBase-0.98-on-Hadoop-1.1 #381 (See [https://builds.apache.org/job/HBase-0.98-on-Hadoop-1.1/381/]) HBASE-11480 ClientScanner might not close the HConnection created in construction (cuijianwei) (stack: rev 790190c2b949af6239d9971a2e002e74760f5bd2) * hbase-client/src/main/java/org/apache/hadoop/hbase/client/ClientScanner.java ClientScanner might not close the HConnection created in construction - Key: HBASE-11480 URL: https://issues.apache.org/jira/browse/HBASE-11480 Project: HBase Issue Type: Improvement Components: Client, Scanners Affects Versions: 0.96.2, 0.94.20, 0.98.3 Reporter: cuijianwei Assignee: cuijianwei Priority: Trivial Fix For: 0.96.3, 0.98.5, 0.94.22 Attachments: HBASE-11480-0.94-v1.patch, HBASE-11480-0.95-v1.patch, HBASE-11480-0.96-v1.patch, HBASE-11480-0.98-v1.patch ClientScanner will create HConnection in its construction: {code} public ClientScanner(final Configuration conf, final Scan scan, final byte[] tableName) throws IOException { this(conf, scan, tableName, HConnectionManager.getConnection(conf)); } {code} However, this connection won't be closed in ClientScanner.close(). Is it better to deprecate this construction? -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11480) ClientScanner might not close the HConnection created in construction
[ https://issues.apache.org/jira/browse/HBASE-11480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14062545#comment-14062545 ] Hudson commented on HBASE-11480: SUCCESS: Integrated in HBase-0.94 #1386 (See [https://builds.apache.org/job/HBase-0.94/1386/]) HBASE-11480 ClientScanner might not close the HConnection created in construction (cuijianwei) (stack: rev 4453e9cf6b6306a6c49359f1fe87263a72ef1c30) * src/main/java/org/apache/hadoop/hbase/client/ClientScanner.java ClientScanner might not close the HConnection created in construction - Key: HBASE-11480 URL: https://issues.apache.org/jira/browse/HBASE-11480 Project: HBase Issue Type: Improvement Components: Client, Scanners Affects Versions: 0.96.2, 0.94.20, 0.98.3 Reporter: cuijianwei Assignee: cuijianwei Priority: Trivial Fix For: 0.96.3, 0.98.5, 0.94.22 Attachments: HBASE-11480-0.94-v1.patch, HBASE-11480-0.95-v1.patch, HBASE-11480-0.96-v1.patch, HBASE-11480-0.98-v1.patch ClientScanner will create HConnection in its construction: {code} public ClientScanner(final Configuration conf, final Scan scan, final byte[] tableName) throws IOException { this(conf, scan, tableName, HConnectionManager.getConnection(conf)); } {code} However, this connection won't be closed in ClientScanner.close(). Is it better to deprecate this construction? -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11480) ClientScanner might not close the HConnection created in construction
[ https://issues.apache.org/jira/browse/HBASE-11480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14062549#comment-14062549 ] Hudson commented on HBASE-11480: SUCCESS: Integrated in HBase-0.98 #401 (See [https://builds.apache.org/job/HBase-0.98/401/]) HBASE-11480 ClientScanner might not close the HConnection created in construction (cuijianwei) (stack: rev 790190c2b949af6239d9971a2e002e74760f5bd2) * hbase-client/src/main/java/org/apache/hadoop/hbase/client/ClientScanner.java ClientScanner might not close the HConnection created in construction - Key: HBASE-11480 URL: https://issues.apache.org/jira/browse/HBASE-11480 Project: HBase Issue Type: Improvement Components: Client, Scanners Affects Versions: 0.96.2, 0.94.20, 0.98.3 Reporter: cuijianwei Assignee: cuijianwei Priority: Trivial Fix For: 0.96.3, 0.98.5, 0.94.22 Attachments: HBASE-11480-0.94-v1.patch, HBASE-11480-0.95-v1.patch, HBASE-11480-0.96-v1.patch, HBASE-11480-0.98-v1.patch ClientScanner will create HConnection in its construction: {code} public ClientScanner(final Configuration conf, final Scan scan, final byte[] tableName) throws IOException { this(conf, scan, tableName, HConnectionManager.getConnection(conf)); } {code} However, this connection won't be closed in ClientScanner.close(). Is it better to deprecate this construction? -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11489) ClassNotFoundException while running IT tests in trunk using 'mvn verify'
[ https://issues.apache.org/jira/browse/HBASE-11489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14062552#comment-14062552 ] Hudson commented on HBASE-11489: FAILURE: Integrated in HBase-1.0 #44 (See [https://builds.apache.org/job/HBase-1.0/44/]) HBASE-11489 ClassNotFoundException while running IT tests in trunk using (ramkrishna: rev 063247bf3b3e1b2890d7e02c1e5426936b66ebba) * hbase-it/pom.xml ClassNotFoundException while running IT tests in trunk using 'mvn verify' - Key: HBASE-11489 URL: https://issues.apache.org/jira/browse/HBASE-11489 Project: HBase Issue Type: Bug Reporter: ramkrishna.s.vasudevan Assignee: ramkrishna.s.vasudevan Fix For: 0.99.0, 1.0.0 Attachments: HBASE-11489.patch, HBASE-11489_1.patch Trying to run {code} mvn verify -Dit.test=IntegrationTestBigLinkedList -Dtest.output.tofile=false {code} causes this ClassNotFoundException issue bq.testContinuousIngest(org.apache.hadoop.hbase.test.IntegrationTestBigLinkedList): org/jboss/netty/channel/ChannelFactory -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (HBASE-11520) Simplify offheap cache config by removing the confusing hbase.bucketcache.percentage.in.combinedcache
stack created HBASE-11520: - Summary: Simplify offheap cache config by removing the confusing hbase.bucketcache.percentage.in.combinedcache Key: HBASE-11520 URL: https://issues.apache.org/jira/browse/HBASE-11520 Project: HBase Issue Type: Sub-task Affects Versions: 0.99.0 Reporter: stack Assignee: stack Fix For: 0.99.0 Remove hbase.bucketcache.percentage.in.combinedcache. It is unnecessary complication of block cache config. Let L1 config setup be as it is whether a L2 present or not, just set hfile.block.cache.size (not hbase.bucketcache.size * (1.0 - hbase.bucketcache.percentage.in.combinedcache)). For L2, let hbase.bucketcache.size be the actual size of the bucket cache, not hbase.bucketcache.size * hbase.bucketcache.percentage.in.combinedcache. Attached patch removes the config. and updates docs. Adds tests to confirm configs are as expected whether a CombinedBlockCache deploy or a strict L1+L2 deploy. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11520) Simplify offheap cache config by removing the confusing hbase.bucketcache.percentage.in.combinedcache
[ https://issues.apache.org/jira/browse/HBASE-11520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14062630#comment-14062630 ] stack commented on HBASE-11520: --- This patch relates to the parent in that it is a less ambitious simplification, something we could get into 1.0 (IMO). See release note for repercussions. Simplify offheap cache config by removing the confusing hbase.bucketcache.percentage.in.combinedcache --- Key: HBASE-11520 URL: https://issues.apache.org/jira/browse/HBASE-11520 Project: HBase Issue Type: Sub-task Components: io Affects Versions: 0.99.0 Reporter: stack Assignee: stack Fix For: 0.99.0 Attachments: 11520.txt Remove hbase.bucketcache.percentage.in.combinedcache. It is unnecessary complication of block cache config. Let L1 config setup be as it is whether a L2 present or not, just set hfile.block.cache.size (not hbase.bucketcache.size * (1.0 - hbase.bucketcache.percentage.in.combinedcache)). For L2, let hbase.bucketcache.size be the actual size of the bucket cache, not hbase.bucketcache.size * hbase.bucketcache.percentage.in.combinedcache. Attached patch removes the config. and updates docs. Adds tests to confirm configs are as expected whether a CombinedBlockCache deploy or a strict L1+L2 deploy. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11520) Simplify offheap cache config by removing the confusing hbase.bucketcache.percentage.in.combinedcache
[ https://issues.apache.org/jira/browse/HBASE-11520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-11520: -- Attachment: 11520.txt Patch is big mostly because of doc changes and test code. Simplify offheap cache config by removing the confusing hbase.bucketcache.percentage.in.combinedcache --- Key: HBASE-11520 URL: https://issues.apache.org/jira/browse/HBASE-11520 Project: HBase Issue Type: Sub-task Components: io Affects Versions: 0.99.0 Reporter: stack Assignee: stack Fix For: 0.99.0 Attachments: 11520.txt Remove hbase.bucketcache.percentage.in.combinedcache. It is unnecessary complication of block cache config. Let L1 config setup be as it is whether a L2 present or not, just set hfile.block.cache.size (not hbase.bucketcache.size * (1.0 - hbase.bucketcache.percentage.in.combinedcache)). For L2, let hbase.bucketcache.size be the actual size of the bucket cache, not hbase.bucketcache.size * hbase.bucketcache.percentage.in.combinedcache. Attached patch removes the config. and updates docs. Adds tests to confirm configs are as expected whether a CombinedBlockCache deploy or a strict L1+L2 deploy. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11520) Simplify offheap cache config by removing the confusing hbase.bucketcache.percentage.in.combinedcache
[ https://issues.apache.org/jira/browse/HBASE-11520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-11520: -- Status: Patch Available (was: Open) Simplify offheap cache config by removing the confusing hbase.bucketcache.percentage.in.combinedcache --- Key: HBASE-11520 URL: https://issues.apache.org/jira/browse/HBASE-11520 Project: HBase Issue Type: Sub-task Components: io Affects Versions: 0.99.0 Reporter: stack Assignee: stack Fix For: 0.99.0 Attachments: 11520.txt Remove hbase.bucketcache.percentage.in.combinedcache. It is unnecessary complication of block cache config. Let L1 config setup be as it is whether a L2 present or not, just set hfile.block.cache.size (not hbase.bucketcache.size * (1.0 - hbase.bucketcache.percentage.in.combinedcache)). For L2, let hbase.bucketcache.size be the actual size of the bucket cache, not hbase.bucketcache.size * hbase.bucketcache.percentage.in.combinedcache. Attached patch removes the config. and updates docs. Adds tests to confirm configs are as expected whether a CombinedBlockCache deploy or a strict L1+L2 deploy. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11520) Simplify offheap cache config by removing the confusing hbase.bucketcache.percentage.in.combinedcache
[ https://issues.apache.org/jira/browse/HBASE-11520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14062700#comment-14062700 ] Nick Dimiduk commented on HBASE-11520: -- Sorry for the lag boss. This sounds like the right approach, with one caveat that configs just got harder for anyone using onheap BucketCache. I would have thought this a minority, but I think you were saying this is more common that I thought. Will give the patch a proper look this afternoon. Simplify offheap cache config by removing the confusing hbase.bucketcache.percentage.in.combinedcache --- Key: HBASE-11520 URL: https://issues.apache.org/jira/browse/HBASE-11520 Project: HBase Issue Type: Sub-task Components: io Affects Versions: 0.99.0 Reporter: stack Assignee: stack Fix For: 0.99.0 Attachments: 11520.txt Remove hbase.bucketcache.percentage.in.combinedcache. It is unnecessary complication of block cache config. Let L1 config setup be as it is whether a L2 present or not, just set hfile.block.cache.size (not hbase.bucketcache.size * (1.0 - hbase.bucketcache.percentage.in.combinedcache)). For L2, let hbase.bucketcache.size be the actual size of the bucket cache, not hbase.bucketcache.size * hbase.bucketcache.percentage.in.combinedcache. Attached patch removes the config. and updates docs. Adds tests to confirm configs are as expected whether a CombinedBlockCache deploy or a strict L1+L2 deploy. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11511) Write flush events to WAL
[ https://issues.apache.org/jira/browse/HBASE-11511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Enis Soztutar updated HBASE-11511: -- Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Committed this to master and branch-1. Thanks Stack for review. Write flush events to WAL - Key: HBASE-11511 URL: https://issues.apache.org/jira/browse/HBASE-11511 Project: HBase Issue Type: Sub-task Reporter: Enis Soztutar Assignee: Enis Soztutar Fix For: 0.99.0, 2.0.0 Attachments: hbase-11511_v1.patch, hbase-11511_v1.patch, hbase-11511_v2.patch, hbase-11511_v3.patch, hbase-11511_v4.patch We used to write COMPLETE_FLUSH event to WAL until it got removed in 0.96 in issue HBASE-7329. For secondary region replicas, it is important to share the data files with the primary region. So we should reintroduce the flush wal markers so that the secondary region replicas can pick up the newly flushed files from the WAL and start serving data from those. A design doc which explains the context a bit better can be found in HBASE-11183. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11511) Write flush events to WAL
[ https://issues.apache.org/jira/browse/HBASE-11511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Enis Soztutar updated HBASE-11511: -- Attachment: hbase-11511_v4.patch Attaching the final version that has been committed. Minor rebase. Write flush events to WAL - Key: HBASE-11511 URL: https://issues.apache.org/jira/browse/HBASE-11511 Project: HBase Issue Type: Sub-task Reporter: Enis Soztutar Assignee: Enis Soztutar Fix For: 0.99.0, 2.0.0 Attachments: hbase-11511_v1.patch, hbase-11511_v1.patch, hbase-11511_v2.patch, hbase-11511_v3.patch, hbase-11511_v4.patch We used to write COMPLETE_FLUSH event to WAL until it got removed in 0.96 in issue HBASE-7329. For secondary region replicas, it is important to share the data files with the primary region. So we should reintroduce the flush wal markers so that the secondary region replicas can pick up the newly flushed files from the WAL and start serving data from those. A design doc which explains the context a bit better can be found in HBASE-11183. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11512) Write region open/close events to WAL
[ https://issues.apache.org/jira/browse/HBASE-11512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Enis Soztutar updated HBASE-11512: -- Attachment: hbase-11512_v1.patch Attaching a patch in the same line as HBASE-11511. Both region open and region close events are persisted to WAL. Write region open/close events to WAL - Key: HBASE-11512 URL: https://issues.apache.org/jira/browse/HBASE-11512 Project: HBase Issue Type: Sub-task Reporter: Enis Soztutar Assignee: Enis Soztutar Attachments: hbase-11512_v1.patch Similar to writing flush events to WAL (HBASE-11511) and compaction events to WAL (HBASE-2231), we should write region open and close events to WAL. This is especially important for secondary region replicas, since we can use this information to pick up primary regions' files from secondary replicas. However, we may need this for regular inter cluster replication as well, see issues HBASE-10343 and HBASE-9465. A design doc for secondary replica replication can be found at HBASE-11183. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11520) Simplify offheap cache config by removing the confusing hbase.bucketcache.percentage.in.combinedcache
[ https://issues.apache.org/jira/browse/HBASE-11520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14062722#comment-14062722 ] Andrew Purtell commented on HBASE-11520: +1 Simplify offheap cache config by removing the confusing hbase.bucketcache.percentage.in.combinedcache --- Key: HBASE-11520 URL: https://issues.apache.org/jira/browse/HBASE-11520 Project: HBase Issue Type: Sub-task Components: io Affects Versions: 0.99.0 Reporter: stack Assignee: stack Fix For: 0.99.0 Attachments: 11520.txt Remove hbase.bucketcache.percentage.in.combinedcache. It is unnecessary complication of block cache config. Let L1 config setup be as it is whether a L2 present or not, just set hfile.block.cache.size (not hbase.bucketcache.size * (1.0 - hbase.bucketcache.percentage.in.combinedcache)). For L2, let hbase.bucketcache.size be the actual size of the bucket cache, not hbase.bucketcache.size * hbase.bucketcache.percentage.in.combinedcache. Attached patch removes the config. and updates docs. Adds tests to confirm configs are as expected whether a CombinedBlockCache deploy or a strict L1+L2 deploy. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11520) Simplify offheap cache config by removing the confusing hbase.bucketcache.percentage.in.combinedcache
[ https://issues.apache.org/jira/browse/HBASE-11520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14062725#comment-14062725 ] Enis Soztutar commented on HBASE-11520: --- Skimmed the patch. Looks good for branch-1. Simplify offheap cache config by removing the confusing hbase.bucketcache.percentage.in.combinedcache --- Key: HBASE-11520 URL: https://issues.apache.org/jira/browse/HBASE-11520 Project: HBase Issue Type: Sub-task Components: io Affects Versions: 0.99.0 Reporter: stack Assignee: stack Fix For: 0.99.0, 2.0.0 Attachments: 11520.txt Remove hbase.bucketcache.percentage.in.combinedcache. It is unnecessary complication of block cache config. Let L1 config setup be as it is whether a L2 present or not, just set hfile.block.cache.size (not hbase.bucketcache.size * (1.0 - hbase.bucketcache.percentage.in.combinedcache)). For L2, let hbase.bucketcache.size be the actual size of the bucket cache, not hbase.bucketcache.size * hbase.bucketcache.percentage.in.combinedcache. Attached patch removes the config. and updates docs. Adds tests to confirm configs are as expected whether a CombinedBlockCache deploy or a strict L1+L2 deploy. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11520) Simplify offheap cache config by removing the confusing hbase.bucketcache.percentage.in.combinedcache
[ https://issues.apache.org/jira/browse/HBASE-11520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Enis Soztutar updated HBASE-11520: -- Fix Version/s: 2.0.0 Simplify offheap cache config by removing the confusing hbase.bucketcache.percentage.in.combinedcache --- Key: HBASE-11520 URL: https://issues.apache.org/jira/browse/HBASE-11520 Project: HBase Issue Type: Sub-task Components: io Affects Versions: 0.99.0 Reporter: stack Assignee: stack Fix For: 0.99.0, 2.0.0 Attachments: 11520.txt Remove hbase.bucketcache.percentage.in.combinedcache. It is unnecessary complication of block cache config. Let L1 config setup be as it is whether a L2 present or not, just set hfile.block.cache.size (not hbase.bucketcache.size * (1.0 - hbase.bucketcache.percentage.in.combinedcache)). For L2, let hbase.bucketcache.size be the actual size of the bucket cache, not hbase.bucketcache.size * hbase.bucketcache.percentage.in.combinedcache. Attached patch removes the config. and updates docs. Adds tests to confirm configs are as expected whether a CombinedBlockCache deploy or a strict L1+L2 deploy. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11064) Odd behaviors of TableName for empty namespace
[ https://issues.apache.org/jira/browse/HBASE-11064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Enis Soztutar updated HBASE-11064: -- Attachment: HBASE-11064.2.patch Our hadoopqa infra seems broken recently for -p0 patches. Reattaching -p1 patch. Will commit if tests pass. Odd behaviors of TableName for empty namespace -- Key: HBASE-11064 URL: https://issues.apache.org/jira/browse/HBASE-11064 Project: HBase Issue Type: Bug Affects Versions: 0.98.3 Reporter: Hiroshi Ikeda Assignee: Rekha Joshi Priority: Trivial Fix For: 0.99.0, 1.0.0, 0.98.5 Attachments: HBASE-11064.1.patch, HBASE-11064.2.patch, HBASE-11064.2.patch In the class TableName, {code} public static byte [] isLegalFullyQualifiedTableName(final byte[] tableName) { ... int namespaceDelimIndex = ... if (namespaceDelimIndex == 0 || namespaceDelimIndex == -1){ isLegalTableQualifierName(tableName); } else { ... {code} That means, for example, giving :a as the argument throws an exception which says invalid qualifier, instead of invalid namespace. Also, TableName.valueOf(String) and valueOf(byte[]) can create an instance with empty namespace, which is inconsistent. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-7336) HFileBlock.readAtOffset does not work well with multiple threads
[ https://issues.apache.org/jira/browse/HBASE-7336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14062733#comment-14062733 ] Vladimir Rodionov commented on HBASE-7336: -- DFSInputStream class is heavily synchronized (at least in HDFS 2.2) and regardless of a read op type (stream, positional) all readers will be waiting on a single lock eventually. This is what I see in my local tests. 1 scanner - 14 sec 2 scanners - 36 sec (!!!) 4 scanners - too long to be true. I have no explanation yet, but something is wrong here. HFileBlock.readAtOffset does not work well with multiple threads Key: HBASE-7336 URL: https://issues.apache.org/jira/browse/HBASE-7336 Project: HBase Issue Type: Sub-task Components: Performance Reporter: Lars Hofhansl Assignee: Lars Hofhansl Priority: Critical Fix For: 0.94.4, 0.95.0 Attachments: 7336-0.94.txt, 7336-0.96.txt HBase grinds to a halt when many threads scan along the same set of blocks and neither read short circuit is nor block caching is enabled for the dfs client ... disabling the block cache makes sense on very large scans. It turns out that synchronizing in istream in HFileBlock.readAtOffset is the culprit. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11502) Track down broken images in Ref Guide
[ https://issues.apache.org/jira/browse/HBASE-11502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14062737#comment-14062737 ] Enis Soztutar commented on HBASE-11502: --- I've fixed the typo in the image name. [~misty] is there anything else to do here? Thanks. Track down broken images in Ref Guide - Key: HBASE-11502 URL: https://issues.apache.org/jira/browse/HBASE-11502 Project: HBase Issue Type: Bug Components: documentation Reporter: Misty Stanley-Jones Assignee: Misty Stanley-Jones I realized that image support was broken in the Ref Guide. I fixed it by making some changes to the pom.xml (in HBASE-11400). I need to track down what images are broken besides the new ones I added, find out where they are, and make them work again. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11520) Simplify offheap cache config by removing the confusing hbase.bucketcache.percentage.in.combinedcache
[ https://issues.apache.org/jira/browse/HBASE-11520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-11520: -- Attachment: 11520v2.txt Thanks for taking a look [~ndimiduk]. v2 has tests for 'heap' and 'file' added. Or were you thinking of something else? If 'heap', yeah, could OOME if we assign too much to onheap bucketcache but that should happen on startup and be pretty plain as to what is going on. Chunhui Shen uses BucketCache onheap. Its way slower but heap won't fragment because of BC operations. Simplify offheap cache config by removing the confusing hbase.bucketcache.percentage.in.combinedcache --- Key: HBASE-11520 URL: https://issues.apache.org/jira/browse/HBASE-11520 Project: HBase Issue Type: Sub-task Components: io Affects Versions: 0.99.0 Reporter: stack Assignee: stack Fix For: 0.99.0, 2.0.0 Attachments: 11520.txt, 11520v2.txt Remove hbase.bucketcache.percentage.in.combinedcache. It is unnecessary complication of block cache config. Let L1 config setup be as it is whether a L2 present or not, just set hfile.block.cache.size (not hbase.bucketcache.size * (1.0 - hbase.bucketcache.percentage.in.combinedcache)). For L2, let hbase.bucketcache.size be the actual size of the bucket cache, not hbase.bucketcache.size * hbase.bucketcache.percentage.in.combinedcache. Attached patch removes the config. and updates docs. Adds tests to confirm configs are as expected whether a CombinedBlockCache deploy or a strict L1+L2 deploy. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-7336) HFileBlock.readAtOffset does not work well with multiple threads
[ https://issues.apache.org/jira/browse/HBASE-7336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14062740#comment-14062740 ] Vladimir Rodionov commented on HBASE-7336: -- Forgot to mention: HBase 0.98.3 hadoop2. All tests are in a local mode with HBase mini cluster. HFileBlock.readAtOffset does not work well with multiple threads Key: HBASE-7336 URL: https://issues.apache.org/jira/browse/HBASE-7336 Project: HBase Issue Type: Sub-task Components: Performance Reporter: Lars Hofhansl Assignee: Lars Hofhansl Priority: Critical Fix For: 0.94.4, 0.95.0 Attachments: 7336-0.94.txt, 7336-0.96.txt HBase grinds to a halt when many threads scan along the same set of blocks and neither read short circuit is nor block caching is enabled for the dfs client ... disabling the block cache makes sense on very large scans. It turns out that synchronizing in istream in HFileBlock.readAtOffset is the culprit. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-7336) HFileBlock.readAtOffset does not work well with multiple threads
[ https://issues.apache.org/jira/browse/HBASE-7336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14062749#comment-14062749 ] stack commented on HBASE-7336: -- [~vrodionov] Can you try on hdfs? HFileBlock.readAtOffset does not work well with multiple threads Key: HBASE-7336 URL: https://issues.apache.org/jira/browse/HBASE-7336 Project: HBase Issue Type: Sub-task Components: Performance Reporter: Lars Hofhansl Assignee: Lars Hofhansl Priority: Critical Fix For: 0.94.4, 0.95.0 Attachments: 7336-0.94.txt, 7336-0.96.txt HBase grinds to a halt when many threads scan along the same set of blocks and neither read short circuit is nor block caching is enabled for the dfs client ... disabling the block cache makes sense on very large scans. It turns out that synchronizing in istream in HFileBlock.readAtOffset is the culprit. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11520) Simplify offheap cache config by removing the confusing hbase.bucketcache.percentage.in.combinedcache
[ https://issues.apache.org/jira/browse/HBASE-11520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14062757#comment-14062757 ] Hadoop QA commented on HBASE-11520: --- {color:red}-1 overall{color}. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12655861/11520.txt against trunk revision . ATTACHMENT ID: 12655861 {color:green}+1 @author{color}. The patch does not contain any @author tags. {color:green}+1 tests included{color}. The patch appears to include 3 new or modified tests. {color:green}+1 javac{color}. The applied patch does not increase the total number of javac compiler warnings. {color:green}+1 javac{color}. The applied patch does not increase the total number of javac compiler warnings. {color:green}+1 javadoc{color}. The javadoc tool did not generate any warning messages. {color:green}+1 findbugs{color}. The patch does not introduce any new Findbugs (version 2.0.3) warnings. {color:green}+1 release audit{color}. The applied patch does not increase the total number of release audit warnings. {color:green}+1 lineLengths{color}. The patch does not introduce lines longer than 100 {color:green}+1 site{color}. The mvn site goal succeeds with this patch. {color:red}-1 core tests{color}. The patch failed these unit tests: org.apache.hadoop.hbase.client.TestReplicaWithCluster Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/10070//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10070//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10070//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10070//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10070//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10070//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10070//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10070//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10070//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10070//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/10070//console This message is automatically generated. Simplify offheap cache config by removing the confusing hbase.bucketcache.percentage.in.combinedcache --- Key: HBASE-11520 URL: https://issues.apache.org/jira/browse/HBASE-11520 Project: HBase Issue Type: Sub-task Components: io Affects Versions: 0.99.0 Reporter: stack Assignee: stack Fix For: 0.99.0, 2.0.0 Attachments: 11520.txt, 11520v2.txt Remove hbase.bucketcache.percentage.in.combinedcache. It is unnecessary complication of block cache config. Let L1 config setup be as it is whether a L2 present or not, just set hfile.block.cache.size (not hbase.bucketcache.size * (1.0 - hbase.bucketcache.percentage.in.combinedcache)). For L2, let hbase.bucketcache.size be the actual size of the bucket cache, not hbase.bucketcache.size * hbase.bucketcache.percentage.in.combinedcache. Attached patch removes the config. and updates docs. Adds tests to confirm configs are as expected whether a CombinedBlockCache deploy or a strict L1+L2 deploy. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11502) Track down broken images in Ref Guide
[ https://issues.apache.org/jira/browse/HBASE-11502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14062767#comment-14062767 ] Misty Stanley-Jones commented on HBASE-11502: - It looks fine now, thank you! I am waiting on a commit from HBASE-11400 to fix some problems with building the book itself, and I have just discovered another problem with CSS that I will fix there as well. Track down broken images in Ref Guide - Key: HBASE-11502 URL: https://issues.apache.org/jira/browse/HBASE-11502 Project: HBase Issue Type: Bug Components: documentation Reporter: Misty Stanley-Jones Assignee: Misty Stanley-Jones I realized that image support was broken in the Ref Guide. I fixed it by making some changes to the pom.xml (in HBASE-11400). I need to track down what images are broken besides the new ones I added, find out where they are, and make them work again. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11064) Odd behaviors of TableName for empty namespace
[ https://issues.apache.org/jira/browse/HBASE-11064?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14062781#comment-14062781 ] Hadoop QA commented on HBASE-11064: --- {color:red}-1 overall{color}. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12655891/HBASE-11064.2.patch against trunk revision . ATTACHMENT ID: 12655891 {color:green}+1 @author{color}. The patch does not contain any @author tags. {color:green}+1 tests included{color}. The patch appears to include 3 new or modified tests. {color:green}+1 javac{color}. The applied patch does not increase the total number of javac compiler warnings. {color:green}+1 javac{color}. The applied patch does not increase the total number of javac compiler warnings. {color:green}+1 javadoc{color}. The javadoc tool did not generate any warning messages. {color:green}+1 findbugs{color}. The patch does not introduce any new Findbugs (version 2.0.3) warnings. {color:green}+1 release audit{color}. The applied patch does not increase the total number of release audit warnings. {color:green}+1 lineLengths{color}. The patch does not introduce lines longer than 100 {color:green}+1 site{color}. The mvn site goal succeeds with this patch. {color:red}-1 core tests{color}. The patch failed these unit tests: org.apache.hadoop.hbase.TestCheckTestClasses Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/10071//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10071//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10071//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10071//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10071//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10071//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10071//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10071//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10071//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10071//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/10071//console This message is automatically generated. Odd behaviors of TableName for empty namespace -- Key: HBASE-11064 URL: https://issues.apache.org/jira/browse/HBASE-11064 Project: HBase Issue Type: Bug Affects Versions: 0.98.3 Reporter: Hiroshi Ikeda Assignee: Rekha Joshi Priority: Trivial Fix For: 0.99.0, 1.0.0, 0.98.5 Attachments: HBASE-11064.1.patch, HBASE-11064.2.patch, HBASE-11064.2.patch In the class TableName, {code} public static byte [] isLegalFullyQualifiedTableName(final byte[] tableName) { ... int namespaceDelimIndex = ... if (namespaceDelimIndex == 0 || namespaceDelimIndex == -1){ isLegalTableQualifierName(tableName); } else { ... {code} That means, for example, giving :a as the argument throws an exception which says invalid qualifier, instead of invalid namespace. Also, TableName.valueOf(String) and valueOf(byte[]) can create an instance with empty namespace, which is inconsistent. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11517) TestReplicaWithCluster turns zombie
[ https://issues.apache.org/jira/browse/HBASE-11517?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14062789#comment-14062789 ] stack commented on HBASE-11517: --- Test times out now and we are getting logs: https://builds.apache.org/job/PreCommit-HBASE-Build/10070//testReport/org.apache.hadoop.hbase.client/TestReplicaWithCluster/testCreateDeleteTable/. I see this: {code} 2014-07-15 22:02:28,553 WARN [PostOpenDeployTasks:235204d3d299a5047091fb4aed3d9b56] handler.OpenRegionHandler$PostOpenDeployTasksThread(326): Exception running postOpenDeployTasks; region=235204d3d299a5047091fb4aed3d9b56 java.lang.NullPointerException: No connection at org.apache.hadoop.hbase.MetaTableAccessor.getHTable(MetaTableAccessor.java:180) at org.apache.hadoop.hbase.MetaTableAccessor.getMetaHTable(MetaTableAccessor.java:193) at org.apache.hadoop.hbase.MetaTableAccessor.putToMetaTable(MetaTableAccessor.java:941) at org.apache.hadoop.hbase.MetaTableAccessor.updateLocation(MetaTableAccessor.java:1300) at org.apache.hadoop.hbase.MetaTableAccessor.updateRegionLocation(MetaTableAccessor.java:1278) at org.apache.hadoop.hbase.regionserver.HRegionServer.postOpenDeployTasks(HRegionServer.java:1724) at org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler$PostOpenDeployTasksThread.run(OpenRegionHandler.java:321) {code} How is this supposed to work might [~mantonov]? (Presuming the above came in w/ your recent change herein?) Thanks. TestReplicaWithCluster turns zombie --- Key: HBASE-11517 URL: https://issues.apache.org/jira/browse/HBASE-11517 Project: HBase Issue Type: Bug Reporter: stack Assignee: stack Attachments: 10930v4.txt, 11517.timeouts.txt Happened a few times for me fixing unrelated findbugs. Here is example: https://builds.apache.org/job/PreCommit-HBASE-Build/10065//consoleFull See how it is hanging creating a table: pool-1-thread-1 prio=10 tid=0x7f1714657000 nid=0x4b7f waiting on condition [0x7f16e9f8] java.lang.Thread.State: TIMED_WAITING (sleeping) at java.lang.Thread.sleep(Native Method) at org.apache.hadoop.hbase.client.HBaseAdmin.createTable(HBaseAdmin.java:539) at org.apache.hadoop.hbase.client.HBaseAdmin.createTable(HBaseAdmin.java:424) at org.apache.hadoop.hbase.HBaseTestingUtility.createTable(HBaseTestingUtility.java:1185) at org.apache.hadoop.hbase.client.TestReplicaWithCluster.testCreateDeleteTable(TestReplicaWithCluster.java:138) -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-4495) CatalogTracker has an identity crisis; needs to be cut-back in scope
[ https://issues.apache.org/jira/browse/HBASE-4495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14062812#comment-14062812 ] Mikhail Antonov commented on HBASE-4495: [~stack] could you take a look at addendum-2 patch? Or it may may sense to file a separate cleanup jira for that. CatalogTracker has an identity crisis; needs to be cut-back in scope Key: HBASE-4495 URL: https://issues.apache.org/jira/browse/HBASE-4495 Project: HBase Issue Type: Improvement Affects Versions: 0.94.0 Reporter: stack Assignee: Mikhail Antonov Labels: 1.0.0 Fix For: 0.99.0, 2.0.0 Attachments: HBASE-4495-v2.patch, HBASE-4495-v3.patch, HBASE-4495-v5.patch, HBASE-4495-v5.patch, HBASE-4495-v6.patch, HBASE-4495-v7.patch, HBASE-4495-v7.patch, HBASE-4495.patch, HBASE-4495.patch, HBASE-4495.patch, HBASE-4495.patch, HBASE-4495_(ADDENDUM-2).patch, hbase-4495_addendum.patch CT needs a good reworking. I'd suggest its scope be cut way down to only deal in zk transactions rather than zk and reading meta location in hbase (over an HConnection) and being a purveyor of HRegionInterfaces on meta and root servers and being an Abortable and a verifier of catalog locations. Once this is done, I would suggest it then better belongs over under the zk package and that the Meta* classes then move to client package. Here's some messy notes I added to head of CT class in hbase-3446 where I spent some time trying to make out what it was CT did. {code} // TODO: This class needs a rethink. The original intent was that it would be // the one-stop-shop for root and meta locations and that it would get this // info from reading and watching zk state. The class was to be used by // servers when they needed to know of root and meta movement but also by // client-side (inside in HTable) so rather than figure root and meta // locations on fault, the client would instead get notifications out of zk. // // But this original intent is frustrated by the fact that this class has to // read an hbase table, the -ROOT- table, to figure out the .META. region // location which means we depend on an HConnection. HConnection will do // retrying but also, it has its own mechanism for finding root and meta // locations (and for 'verifying'; it tries the location and if it fails, does // new lookup, etc.). So, at least for now, HConnection (or HTable) can't // have a CT since CT needs a HConnection (Even then, do want HT to have a CT? // For HT keep up a session with ZK? Rather, shouldn't we do like asynchbase // where we'd open a connection to zk, read what we need then let the // connection go?). The 'fix' is make it so both root and meta addresses // are wholey up in zk -- not in zk (root) -- and in an hbase table (meta). // // But even then, this class does 'verification' of the location and it does // this by making a call over an HConnection (which will do its own root // and meta lookups). Isn't this verification 'useless' since when we // return, whatever is dependent on the result of this call then needs to // use HConnection; what we have verified may change in meantime (HConnection // uses the CT primitives, the root and meta trackers finding root locations). // // When meta is moved to zk, this class may make more sense. In the // meantime, it does not cohere. It should just watch meta and root and // NOT do verification -- let that be out in HConnection since its going to // be done there ultimately anyways. // // This class has spread throughout the codebase. It needs to be reigned in. // This class should be used server-side only, even if we move meta location // up into zk. Currently its used over in the client package. Its used in // MetaReader and MetaEditor classes usually just to get the Configuration // its using (It does this indirectly by asking its HConnection for its // Configuration and even then this is just used to get an HConnection out on // the other end). St.Ack 10/23/2011. // {code} -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Resolved] (HBASE-11502) Track down broken images in Ref Guide
[ https://issues.apache.org/jira/browse/HBASE-11502?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Enis Soztutar resolved HBASE-11502. --- Resolution: Fixed Fix Version/s: 2.0.0 0.99.0 Thanks Misty. Resolving this issue then. Track down broken images in Ref Guide - Key: HBASE-11502 URL: https://issues.apache.org/jira/browse/HBASE-11502 Project: HBase Issue Type: Bug Components: documentation Reporter: Misty Stanley-Jones Assignee: Misty Stanley-Jones Fix For: 0.99.0, 2.0.0 I realized that image support was broken in the Ref Guide. I fixed it by making some changes to the pom.xml (in HBASE-11400). I need to track down what images are broken besides the new ones I added, find out where they are, and make them work again. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-7336) HFileBlock.readAtOffset does not work well with multiple threads
[ https://issues.apache.org/jira/browse/HBASE-7336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14062814#comment-14062814 ] Vladimir Rodionov commented on HBASE-7336: -- I monitor thread stack traces during test run. Usually, just one thread (Scanner) is running, all others are waiting on DFSInputStream in some places (as I said, too many synchronized methods). This is HDFS, not HBase. HFileBlock.readAtOffset does not work well with multiple threads Key: HBASE-7336 URL: https://issues.apache.org/jira/browse/HBASE-7336 Project: HBase Issue Type: Sub-task Components: Performance Reporter: Lars Hofhansl Assignee: Lars Hofhansl Priority: Critical Fix For: 0.94.4, 0.95.0 Attachments: 7336-0.94.txt, 7336-0.96.txt HBase grinds to a halt when many threads scan along the same set of blocks and neither read short circuit is nor block caching is enabled for the dfs client ... disabling the block cache makes sense on very large scans. It turns out that synchronizing in istream in HFileBlock.readAtOffset is the culprit. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11517) TestReplicaWithCluster turns zombie
[ https://issues.apache.org/jira/browse/HBASE-11517?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14062820#comment-14062820 ] Mikhail Antonov commented on HBASE-11517: - Let me take a look. TestReplicaWithCluster turns zombie --- Key: HBASE-11517 URL: https://issues.apache.org/jira/browse/HBASE-11517 Project: HBase Issue Type: Bug Reporter: stack Assignee: stack Attachments: 10930v4.txt, 11517.timeouts.txt Happened a few times for me fixing unrelated findbugs. Here is example: https://builds.apache.org/job/PreCommit-HBASE-Build/10065//consoleFull See how it is hanging creating a table: pool-1-thread-1 prio=10 tid=0x7f1714657000 nid=0x4b7f waiting on condition [0x7f16e9f8] java.lang.Thread.State: TIMED_WAITING (sleeping) at java.lang.Thread.sleep(Native Method) at org.apache.hadoop.hbase.client.HBaseAdmin.createTable(HBaseAdmin.java:539) at org.apache.hadoop.hbase.client.HBaseAdmin.createTable(HBaseAdmin.java:424) at org.apache.hadoop.hbase.HBaseTestingUtility.createTable(HBaseTestingUtility.java:1185) at org.apache.hadoop.hbase.client.TestReplicaWithCluster.testCreateDeleteTable(TestReplicaWithCluster.java:138) -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11261) Handle splitting/merging of regions that have region_replication greater than one
[ https://issues.apache.org/jira/browse/HBASE-11261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Devaraj Das updated HBASE-11261: Summary: Handle splitting/merging of regions that have region_replication greater than one (was: Handle splitting of regions that have region_replication greater than one) Handle splitting/merging of regions that have region_replication greater than one - Key: HBASE-11261 URL: https://issues.apache.org/jira/browse/HBASE-11261 Project: HBase Issue Type: Sub-task Reporter: Devaraj Das Assignee: Devaraj Das Fix For: 0.99.0 Attachments: 11261-1.1.txt, 11261-1.2.txt, 11261-2.txt, 11261-3.txt -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11520) Simplify offheap cache config by removing the confusing hbase.bucketcache.percentage.in.combinedcache
[ https://issues.apache.org/jira/browse/HBASE-11520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14062836#comment-14062836 ] Nick Dimiduk commented on HBASE-11520: -- v2 looks good to me. I'm still confused by CacheConfig though. There's two places where {{GLOBAL_BLOCK_CACHE_INSTANCE = new CombinedBlockCache(lruCache, bucketCache);}} This should not be the case. Can we rework the logic such that if {{conf.get(BUCKET_CACHE_IOENGINE_KEY, null) == null}} we just create the LruBlockCache instance and return? Additional confusion comes from the updated test where you have {{conf.set(CacheConfig.BUCKET_CACHE_IOENGINE_KEY, offheap)}} and {{conf.setBoolean(CacheConfig.BUCKET_CACHE_COMBINED_KEY, false)}}. That should be an invalid configuration, right? I want two caches, but I want them to be independent of each other? That does make sense. From my read, this should result in just an Lru instance, right? Isn't {{BUCKET_CACHE_COMBINED_KEY=true}} implied by specifying {{BUCKET_CACHE_IOENGINE_KEY}}? Maybe this is all for yet another follow-on patch? Anyway, +1 for v2. Simplify offheap cache config by removing the confusing hbase.bucketcache.percentage.in.combinedcache --- Key: HBASE-11520 URL: https://issues.apache.org/jira/browse/HBASE-11520 Project: HBase Issue Type: Sub-task Components: io Affects Versions: 0.99.0 Reporter: stack Assignee: stack Fix For: 0.99.0, 2.0.0 Attachments: 11520.txt, 11520v2.txt Remove hbase.bucketcache.percentage.in.combinedcache. It is unnecessary complication of block cache config. Let L1 config setup be as it is whether a L2 present or not, just set hfile.block.cache.size (not hbase.bucketcache.size * (1.0 - hbase.bucketcache.percentage.in.combinedcache)). For L2, let hbase.bucketcache.size be the actual size of the bucket cache, not hbase.bucketcache.size * hbase.bucketcache.percentage.in.combinedcache. Attached patch removes the config. and updates docs. Adds tests to confirm configs are as expected whether a CombinedBlockCache deploy or a strict L1+L2 deploy. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11261) Handle splitting/merging of regions that have region_replication greater than one
[ https://issues.apache.org/jira/browse/HBASE-11261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Devaraj Das updated HBASE-11261: Attachment: 11261-with-merge-2.txt The patch submitted on RB. Handle splitting/merging of regions that have region_replication greater than one - Key: HBASE-11261 URL: https://issues.apache.org/jira/browse/HBASE-11261 Project: HBase Issue Type: Sub-task Reporter: Devaraj Das Assignee: Devaraj Das Fix For: 0.99.0 Attachments: 11261-1.1.txt, 11261-1.2.txt, 11261-2.txt, 11261-3.txt, 11261-with-merge-2.txt -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11517) TestReplicaWithCluster turns zombie
[ https://issues.apache.org/jira/browse/HBASE-11517?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14062839#comment-14062839 ] stack commented on HBASE-11517: --- It seems reproducible locally now taking a look too. TestReplicaWithCluster turns zombie --- Key: HBASE-11517 URL: https://issues.apache.org/jira/browse/HBASE-11517 Project: HBase Issue Type: Bug Reporter: stack Assignee: stack Attachments: 10930v4.txt, 11517.timeouts.txt Happened a few times for me fixing unrelated findbugs. Here is example: https://builds.apache.org/job/PreCommit-HBASE-Build/10065//consoleFull See how it is hanging creating a table: pool-1-thread-1 prio=10 tid=0x7f1714657000 nid=0x4b7f waiting on condition [0x7f16e9f8] java.lang.Thread.State: TIMED_WAITING (sleeping) at java.lang.Thread.sleep(Native Method) at org.apache.hadoop.hbase.client.HBaseAdmin.createTable(HBaseAdmin.java:539) at org.apache.hadoop.hbase.client.HBaseAdmin.createTable(HBaseAdmin.java:424) at org.apache.hadoop.hbase.HBaseTestingUtility.createTable(HBaseTestingUtility.java:1185) at org.apache.hadoop.hbase.client.TestReplicaWithCluster.testCreateDeleteTable(TestReplicaWithCluster.java:138) -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11400) Edit, consolidate, and update Compression and data encoding docs
[ https://issues.apache.org/jira/browse/HBASE-11400?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misty Stanley-Jones updated HBASE-11400: Attachment: HBASE-11400-6.patch I removed the changes from pom.xml and realized I had put the images in the wrong place in the build, so fixed those things in this patch. I will open a separate JIRA to handle the pom.xml changes as there is another change needed for CSS. Edit, consolidate, and update Compression and data encoding docs Key: HBASE-11400 URL: https://issues.apache.org/jira/browse/HBASE-11400 Project: HBase Issue Type: Improvement Components: documentation Reporter: Misty Stanley-Jones Assignee: Misty Stanley-Jones Priority: Minor Attachments: HBASE-11400-1.patch, HBASE-11400-2.patch, HBASE-11400-3.patch, HBASE-11400-4.patch, HBASE-11400-5.patch, HBASE-11400-6.patch, HBASE-11400.patch, data_block_diff_encoding.png, data_block_no_encoding.png, data_block_prefix_encoding.png Current docs are here: http://hbase.apache.org/book.html#compression.test It could use some editing and expansion. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (HBASE-11521) Modify pom.xml to copy the images/ and css/ directories to the right location for the Ref Guide to see them correctly
Misty Stanley-Jones created HBASE-11521: --- Summary: Modify pom.xml to copy the images/ and css/ directories to the right location for the Ref Guide to see them correctly Key: HBASE-11521 URL: https://issues.apache.org/jira/browse/HBASE-11521 Project: HBase Issue Type: Bug Components: documentation Reporter: Misty Stanley-Jones Assignee: Misty Stanley-Jones Priority: Critical Currently, images are broken in the html-single version of the Ref Guide and a CSS file is missing from it too. This change fixes those issues. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11521) Modify pom.xml to copy the images/ and css/ directories to the right location for the Ref Guide to see them correctly
[ https://issues.apache.org/jira/browse/HBASE-11521?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misty Stanley-Jones updated HBASE-11521: Attachment: HBASE-11521.patch Modify pom.xml to copy the images/ and css/ directories to the right location for the Ref Guide to see them correctly - Key: HBASE-11521 URL: https://issues.apache.org/jira/browse/HBASE-11521 Project: HBase Issue Type: Bug Components: documentation Reporter: Misty Stanley-Jones Assignee: Misty Stanley-Jones Priority: Critical Currently, images are broken in the html-single version of the Ref Guide and a CSS file is missing from it too. This change fixes those issues. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11521) Modify pom.xml to copy the images/ and css/ directories to the right location for the Ref Guide to see them correctly
[ https://issues.apache.org/jira/browse/HBASE-11521?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misty Stanley-Jones updated HBASE-11521: Attachment: (was: HBASE-11521.patch) Modify pom.xml to copy the images/ and css/ directories to the right location for the Ref Guide to see them correctly - Key: HBASE-11521 URL: https://issues.apache.org/jira/browse/HBASE-11521 Project: HBase Issue Type: Bug Components: documentation Reporter: Misty Stanley-Jones Assignee: Misty Stanley-Jones Priority: Critical Currently, images are broken in the html-single version of the Ref Guide and a CSS file is missing from it too. This change fixes those issues. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11521) Modify pom.xml to copy the images/ and css/ directories to the right location for the Ref Guide to see them correctly
[ https://issues.apache.org/jira/browse/HBASE-11521?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misty Stanley-Jones updated HBASE-11521: Attachment: HBASE-11521.patch Modify pom.xml to copy the images/ and css/ directories to the right location for the Ref Guide to see them correctly - Key: HBASE-11521 URL: https://issues.apache.org/jira/browse/HBASE-11521 Project: HBase Issue Type: Bug Components: documentation Reporter: Misty Stanley-Jones Assignee: Misty Stanley-Jones Priority: Critical Attachments: HBASE-11521.patch Currently, images are broken in the html-single version of the Ref Guide and a CSS file is missing from it too. This change fixes those issues. -- This message was sent by Atlassian JIRA (v6.2#6252)