[jira] [Commented] (HBASE-6627) TestMultiVersions.testGetRowVersions is flaky
[ https://issues.apache.org/jira/browse/HBASE-6627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13440073#comment-13440073 ] nkeywal commented on HBASE-6627: I don't, but as I don't reproduce the issue anymore my guess is that my fix is unrelated. :-) I will look more into this/ TestMultiVersions.testGetRowVersions is flaky - Key: HBASE-6627 URL: https://issues.apache.org/jira/browse/HBASE-6627 Project: HBase Issue Type: Improvement Components: test Affects Versions: 0.96.0 Environment: hadoop-qa mainly, seems to happen tests in parallel; difficult to reproduce on a single test. Reporter: nkeywal Assignee: nkeywal Attachments: 6627.v1.patch org.apache.hadoop.hbase.TestMultiVersions.testGetRowVersions Shutting down Stacktrace java.io.IOException: Shutting down at org.apache.hadoop.hbase.MiniHBaseCluster.init(MiniHBaseCluster.java:229) at org.apache.hadoop.hbase.MiniHBaseCluster.init(MiniHBaseCluster.java:92) at org.apache.hadoop.hbase.HBaseTestingUtility.startMiniHBaseCluster(HBaseTestingUtility.java:688) at org.apache.hadoop.hbase.HBaseTestingUtility.startMiniHBaseCluster(HBaseTestingUtility.java:661) at org.apache.hadoop.hbase.TestMultiVersions.testGetRowVersions(TestMultiVersions.java:143) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:30) at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:47) at org.junit.rules.RunRules.evaluate(RunRules.java:18) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:68) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47) at org.junit.runners.ParentRunner$3.run -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5549) Master can fail if ZooKeeper session expires
[ https://issues.apache.org/jira/browse/HBASE-5549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13440074#comment-13440074 ] stack commented on HBASE-5549: -- @Himanshu A thread dump when its hung? Might tell us something? @Lars Done I'm triggering and watching 0.94 builds... will make sure I've not added a regression. Master can fail if ZooKeeper session expires Key: HBASE-5549 URL: https://issues.apache.org/jira/browse/HBASE-5549 Project: HBase Issue Type: Bug Components: master, zookeeper Affects Versions: 0.96.0 Environment: all Reporter: nkeywal Assignee: nkeywal Priority: Minor Fix For: 0.92.2, 0.96.0, 0.94.2 Attachments: 5549_092.txt, 5549_094.txt, 5549.v10.patch, 5549.v11.patch, 5549.v6.patch, 5549.v7.patch, 5549.v8.patch, 5549.v9.patch, nochange.patch There is a retry mechanism in RecoverableZooKeeper, but when the session expires, the whole ZooKeeperWatcher is recreated, hence the retry mechanism does not work in this case. This is why a sleep is needed in TestZooKeeper#testMasterSessionExpired: we need to wait for ZooKeeperWatcher to be recreated before using the connection. This can happen in real life, it can happen when: - master zookeeper starts - zookeeper connection is cut - master enters the retry loop - in the meantime the session expires - the network comes back, the session is recreated - the retries continues, but on the wrong object, hence fails. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5549) Master can fail if ZooKeeper session expires
[ https://issues.apache.org/jira/browse/HBASE-5549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13440081#comment-13440081 ] Lars Hofhansl commented on HBASE-5549: -- Thanks Stack! I was going to do the 0.94 patch tomorrow... But this is better :) Master can fail if ZooKeeper session expires Key: HBASE-5549 URL: https://issues.apache.org/jira/browse/HBASE-5549 Project: HBase Issue Type: Bug Components: master, zookeeper Affects Versions: 0.96.0 Environment: all Reporter: nkeywal Assignee: nkeywal Priority: Minor Fix For: 0.92.2, 0.96.0, 0.94.2 Attachments: 5549_092.txt, 5549_094.txt, 5549.v10.patch, 5549.v11.patch, 5549.v6.patch, 5549.v7.patch, 5549.v8.patch, 5549.v9.patch, nochange.patch There is a retry mechanism in RecoverableZooKeeper, but when the session expires, the whole ZooKeeperWatcher is recreated, hence the retry mechanism does not work in this case. This is why a sleep is needed in TestZooKeeper#testMasterSessionExpired: we need to wait for ZooKeeperWatcher to be recreated before using the connection. This can happen in real life, it can happen when: - master zookeeper starts - zookeeper connection is cut - master enters the retry loop - in the meantime the session expires - the network comes back, the session is recreated - the retries continues, but on the wrong object, hence fails. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6586) Quarantine Corrupted HFiles
[
https://issues.apache.org/jira/browse/HBASE-6586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13440090#comment-13440090
]
Hadoop QA commented on HBASE-6586:
--
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12542089/hbase-6586-trunk-v8.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 12 new or modified tests.
+1 hadoop2.0. The patch compiles against the hadoop 2.0 profile.
-1 javadoc. The javadoc tool appears to have generated 5 warning messages.
-1 javac. The applied patch generated 5 javac compiler warnings (more than
the trunk's current 4 warnings).
-1 findbugs. The patch appears to introduce 8 new Findbugs (version 1.3.9)
warnings.
+1 release audit. The applied patch does not increase the total number of
release audit warnings.
-1 core tests. The patch failed these unit tests:
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/2656//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/2656//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/2656//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/2656//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/2656//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/2656//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/2656//console
This message is automatically generated.
Quarantine Corrupted HFiles
---
Key: HBASE-6586
URL: https://issues.apache.org/jira/browse/HBASE-6586
Project: HBase
Issue Type: Improvement
Reporter: Jonathan Hsieh
Assignee: Jonathan Hsieh
Attachments: 0001-hbase-6568-hbck-quarantine-v6.patch,
hbase-6586-92-v3.patch, hbase-6586-94-v3.patch, hbase-6586.patch,
hbase-6586-trunk-v3.patch, hbase-6586-trunk-v4.patch,
hbase-6586-trunk-v5.patch, hbase-6586-trunk-v6.patch,
hbase-6586-trunk-v7.patch, hbase-6586-trunk-v8.patch
We've encountered a few upgrades from 0.90 hbases + 20.2/1.x hdfs to 0.92
hbases + hdfs 2.x that get stuck. I haven't been able to duplicate the
problem in my dev environment but we suspect this may be related to
HDFS-3731. On the HBase side, it seems reasonable to quarantine what are
most likely truncated hfiles, so that can could later be recovered.
Here's an example of the exception we've encountered:
{code}
2012-07-18 05:55:01,152 ERROR handler.OpenRegionHandler
(OpenRegionHandler.java:openRegion(346)) - Failed open of
region=user_mappings,080112102AA76EF98197605D341B9E6C5824D2BC|1001,1317824890618.eaed0e7abc6d27d28ff0e5a9b49c4c
0d.
java.io.IOException: java.lang.IllegalArgumentException: Invalid HFile
version: 842220600 (expected to be between 1 and 2)
at
org.apache.hadoop.hbase.io.hfile.FixedFileTrailer.readFromStream(FixedFileTrailer.java:306)
at org.apache.hadoop.hbase.io.hfile.HFile.pickReaderVersion(HFile.java:371)
at org.apache.hadoop.hbase.io.hfile.HFile.createReader(HFile.java:387)
at
org.apache.hadoop.hbase.regionserver.StoreFile$Reader.init(StoreFile.java:1026)
at org.apache.hadoop.hbase.regionserver.StoreFile.open(StoreFile.java:485)
at
org.apache.hadoop.hbase.regionserver.StoreFile.createReader(StoreFile.java:566)
at org.apache.hadoop.hbase.regionserver.Store.loadStoreFiles(Store.java:286)
at org.apache.hadoop.hbase.regionserver.Store.init(Store.java:223)
at
org.apache.hadoop.hbase.regionserver.HRegion.instantiateHStore(HRegion.java:2534)
at org.apache.hadoop.hbase.regionserver.HRegion.initialize(HRegion.java:454)
at
org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:3282)
at
org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:3230)
at
org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.openRegion(OpenRegionHandler.java:331)
at
org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.process(OpenRegionHandler.java:107)
at org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:169)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:619)
Caused by:
[jira] [Commented] (HBASE-5549) Master can fail if ZooKeeper session expires
[ https://issues.apache.org/jira/browse/HBASE-5549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13440097#comment-13440097 ] Hudson commented on HBASE-5549: --- Integrated in HBase-0.94 #417 (See [https://builds.apache.org/job/HBase-0.94/417/]) HBASE-5549 Master can fail if ZooKeeper session expires (Revision 1376374) Result = SUCCESS stack : Files : * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/ActiveMasterManager.java * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/HMaster.java * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/zookeeper/RecoverableZooKeeper.java * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/zookeeper/ZooKeeperWatcher.java * /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java * /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/TestZooKeeper.java * /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/master/TestDistributedLogSplitting.java * /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/master/TestMasterZKSessionRecovery.java * /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/regionserver/TestSplitTransactionOnCluster.java * /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/replication/TestReplication.java Master can fail if ZooKeeper session expires Key: HBASE-5549 URL: https://issues.apache.org/jira/browse/HBASE-5549 Project: HBase Issue Type: Bug Components: master, zookeeper Affects Versions: 0.96.0 Environment: all Reporter: nkeywal Assignee: nkeywal Priority: Minor Fix For: 0.92.2, 0.96.0, 0.94.2 Attachments: 5549_092.txt, 5549_094.txt, 5549.v10.patch, 5549.v11.patch, 5549.v6.patch, 5549.v7.patch, 5549.v8.patch, 5549.v9.patch, nochange.patch There is a retry mechanism in RecoverableZooKeeper, but when the session expires, the whole ZooKeeperWatcher is recreated, hence the retry mechanism does not work in this case. This is why a sleep is needed in TestZooKeeper#testMasterSessionExpired: we need to wait for ZooKeeperWatcher to be recreated before using the connection. This can happen in real life, it can happen when: - master zookeeper starts - zookeeper connection is cut - master enters the retry loop - in the meantime the session expires - the network comes back, the session is recreated - the retries continues, but on the wrong object, hence fails. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6524) Hooks for hbase tracing
[ https://issues.apache.org/jira/browse/HBASE-6524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13440100#comment-13440100 ] Hudson commented on HBASE-6524: --- Integrated in HBase-TRUNK #3254 (See [https://builds.apache.org/job/HBase-TRUNK/3254/]) HBASE-6524 revert due to new test failures against hadoop 2.0 (Revision 1376365) Result = FAILURE tedyu : Files : * /hbase/trunk/hbase-server/pom.xml * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/executor/EventHandler.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/HBaseClient.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/HBaseServer.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/handler/DisableTableHandler.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/handler/EnableTableHandler.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/protobuf/generated/FilterProtos.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/protobuf/generated/RPCProtos.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/protobuf/generated/Tracing.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/trace/HBaseLocalFileSpanReceiver.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/trace/SpanReceiverHost.java * /hbase/trunk/hbase-server/src/main/protobuf/RPC.proto * /hbase/trunk/hbase-server/src/main/protobuf/Tracing.proto * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/trace/TestHTraceHooks.java Hooks for hbase tracing --- Key: HBASE-6524 URL: https://issues.apache.org/jira/browse/HBASE-6524 Project: HBase Issue Type: Sub-task Reporter: Jonathan Leavitt Fix For: 0.96.0 Attachments: 6524.addendum, createTableTrace.png, hbase-6524.diff Includes the hooks that use [htrace|http://www.github.com/cloudera/htrace] library to add dapper-like tracing to hbase. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3271) Allow .META. table to be exported
[
https://issues.apache.org/jira/browse/HBASE-3271?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13440105#comment-13440105
]
Hadoop QA commented on HBASE-3271:
--
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12542086/HBASE-3271-v2.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 3 new or modified tests.
+1 hadoop2.0. The patch compiles against the hadoop 2.0 profile.
+1 javadoc. The javadoc tool did not generate any warning messages.
-1 javac. The applied patch generated 5 javac compiler warnings (more than
the trunk's current 4 warnings).
-1 findbugs. The patch appears to introduce 7 new Findbugs (version 1.3.9)
warnings.
+1 release audit. The applied patch does not increase the total number of
release audit warnings.
-1 core tests. The patch failed these unit tests:
org.apache.hadoop.hbase.io.encoding.TestUpgradeFromHFileV1ToEncoding
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/2657//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/2657//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/2657//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/2657//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/2657//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/2657//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/2657//console
This message is automatically generated.
Allow .META. table to be exported
-
Key: HBASE-3271
URL: https://issues.apache.org/jira/browse/HBASE-3271
Project: HBase
Issue Type: Improvement
Components: util
Affects Versions: 0.20.6
Reporter: Ted Yu
Fix For: 0.96.0
Attachments: HBASE-3271.patch, HBASE-3271-v2.patch
I tried to export .META. table in 0.20.6 and got:
[hadoop@us01-ciqps1-name01 hbase]$ bin/hbase
org.apache.hadoop.hbase.mapreduce.Export .META. h-meta 1 0 0
10/11/23 20:59:05 INFO jvm.JvmMetrics: Initializing JVM Metrics with
processName=JobTracker, sessionId=
2010-11-23 20:59:05.255::INFO: Logging to STDERR via
org.mortbay.log.StdErrLog
2010-11-23 20:59:05.255::INFO: verisons=1, starttime=0,
endtime=9223372036854775807
10/11/23 20:59:05 INFO zookeeper.ZooKeeper: Client
environment:zookeeper.version=3.2.2-888565, built on 12/08/2009 21:51 GMT
10/11/23 20:59:05 INFO zookeeper.ZooKeeper: Client
environment:host.name=us01-ciqps1-name01.carrieriq.com
10/11/23 20:59:05 INFO zookeeper.ZooKeeper: Client
environment:java.version=1.6.0_21
10/11/23 20:59:05 INFO zookeeper.ZooKeeper: Client
environment:java.vendor=Sun Microsystems Inc.
...
10/11/23 20:59:05 INFO zookeeper.ClientCnxn: Server connection successful
10/11/23 20:59:05 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode
/hbase/root-region-server got 10.202.50.112:60020
10/11/23 20:59:05 DEBUG client.HConnectionManager$TableServers: Found ROOT at
10.202.50.112:60020
10/11/23 20:59:05 DEBUG client.HConnectionManager$TableServers: Cached
location for .META.,,1 is us01-ciqps1-grid02.carrieriq.com:60020
Exception in thread main java.io.IOException: Expecting at least one region.
at
org.apache.hadoop.hbase.mapreduce.TableInputFormatBase.getSplits(TableInputFormatBase.java:281)
at
org.apache.hadoop.mapred.JobClient.writeNewSplits(JobClient.java:885)
at
org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:779)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:432)
at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:447)
at org.apache.hadoop.hbase.mapreduce.Export.main(Export.java:146)
Related code is:
if (keys == null || keys.getFirst() == null ||
keys.getFirst().length == 0) {
throw new IOException(Expecting at least one region.);
}
My intention was to save the dangling rows in .META. (for future
investigation) which prevented a table from being created.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see:
[jira] [Commented] (HBASE-5329) addRowLock() may allocate duplicate lock id, causing the client to be blocked
[
https://issues.apache.org/jira/browse/HBASE-5329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13440110#comment-13440110
]
Hadoop QA commented on HBASE-5329:
--
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12542080/5329-v2.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified
tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
+1 hadoop2.0. The patch compiles against the hadoop 2.0 profile.
+1 javadoc. The javadoc tool did not generate any warning messages.
-1 javac. The applied patch generated 5 javac compiler warnings (more than
the trunk's current 4 warnings).
-1 findbugs. The patch appears to introduce 7 new Findbugs (version 1.3.9)
warnings.
+1 release audit. The applied patch does not increase the total number of
release audit warnings.
-1 core tests. The patch failed these unit tests:
org.apache.hadoop.hbase.io.hfile.TestForceCacheImportantBlocks
org.apache.hadoop.hbase.backup.example.TestZooKeeperTableArchiveClient
org.apache.hadoop.hbase.master.TestSplitLogManager
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/2658//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/2658//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/2658//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/2658//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/2658//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/2658//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/2658//console
This message is automatically generated.
addRowLock() may allocate duplicate lock id, causing the client to be blocked
-
Key: HBASE-5329
URL: https://issues.apache.org/jira/browse/HBASE-5329
Project: HBase
Issue Type: Bug
Components: regionserver
Affects Versions: 0.90.3
Environment: Red Hat Enterprise Linux Server release 5.4
Reporter: liaoxiangui
Assignee: Ian Varley
Priority: Minor
Fix For: 0.96.0
Attachments: 5329-v2.patch, HBASE-5329.patch
{code}
protected long addRowLock(Integer r, HRegion region) throws
LeaseStillHeldException
{
long lockId = -1L;
lockId = rand.nextLong(); //!!!may generate duplicated
id，bug?
String lockName = String.valueOf(lockId);
rowlocks.put(lockName, r);
this.leases.createLease(lockName, new RowLockListener(lockName,
region));
return lockId;
}
{code}
In addRowLock(),rand may generate duplicated lock id, it may cause
regionserver throw exception(Leases$LeaseStillHeldException).The client will
be blocked until old rowlock is released.
{code}
2012-02-03 15:21:50,084 ERROR
org.apache.hadoop.hbase.regionserver.HRegionServer: Error obtaining row lock
(fsOk: true)
org.apache.hadoop.hbase.regionserver.Leases$LeaseStillHeldException
at
org.apache.hadoop.hbase.regionserver.Leases.createLease(Leases.java:150)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.addRowLock(HRegionServer.java:1986)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.lockRow(HRegionServer.java:1963)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:570)
at
org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1039)
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6598) Combine Master Metrics into a single class.
[ https://issues.apache.org/jira/browse/HBASE-6598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13440130#comment-13440130 ] Hadoop QA commented on HBASE-6598: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12542079/HBASE-6598-3.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 16 new or modified tests. +1 hadoop2.0. The patch compiles against the hadoop 2.0 profile. +1 javadoc. The javadoc tool did not generate any warning messages. -1 javac. The applied patch generated 5 javac compiler warnings (more than the trunk's current 4 warnings). -1 findbugs. The patch appears to introduce 6 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hbase.io.encoding.TestUpgradeFromHFileV1ToEncoding Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/2659//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2659//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2659//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2659//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2659//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2659//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/2659//console This message is automatically generated. Combine Master Metrics into a single class. --- Key: HBASE-6598 URL: https://issues.apache.org/jira/browse/HBASE-6598 Project: HBase Issue Type: Sub-task Reporter: Elliott Clark Assignee: Elliott Clark Attachments: HBASE-6598-0.patch, HBASE-6598-1.patch, HBASE-6598-2.patch, HBASE-6598-3.patch Rather than an MBean and a dynamic source. Lets use just one. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6052) Convert .META. and -ROOT- content to pb
[ https://issues.apache.org/jira/browse/HBASE-6052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13440131#comment-13440131 ] Hadoop QA commented on HBASE-6052: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12542066/HBASE-6052_v7.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 67 new or modified tests. -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/2661//console This message is automatically generated. Convert .META. and -ROOT- content to pb --- Key: HBASE-6052 URL: https://issues.apache.org/jira/browse/HBASE-6052 Project: HBase Issue Type: Sub-task Reporter: stack Assignee: Enis Soztutar Priority: Blocker Fix For: 0.96.0 Attachments: 6052-v5.txt, HBASE-6052_v1.patch, HBASE-6052_v2.patch, HBASE-6052_v3.patch, HBASE-6052_v4.patch, HBASE-6052_v4.patch, HBASE-6052_v7.patch, TestMetaMigrationConvertToPB.tgz -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6606) Test for reconnecting with HBaseAdmin using unmanaged HConnection
[ https://issues.apache.org/jira/browse/HBASE-6606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13440132#comment-13440132 ] Hadoop QA commented on HBASE-6606: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12542064/HBASE-6606.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/2662//console This message is automatically generated. Test for reconnecting with HBaseAdmin using unmanaged HConnection - Key: HBASE-6606 URL: https://issues.apache.org/jira/browse/HBASE-6606 Project: HBase Issue Type: Task Components: client, test Reporter: Gregory Chanan Assignee: Gregory Chanan Priority: Minor Fix For: 0.96.0 Attachments: HBASE-6606.patch HBASE-5058 allowed HBaseAdmin to work with an existing and unmanaged HConnection. The retry semantics of managed vs unmanaged connections are different. From the JIRA: For an HConnection that is passed from the outside, it has to be possible to try again. So if the HConnection is managed we retain the old behavior (i.e. only try once, give up after that, even if that failed). For an unmanaged connection we try again unless we actually found a master. . I couldn't find any test of this behavior, only that the HBaseAdmin works with an unmanaged connection, i.e. no retry testing. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6638) Move DaemonThreadFactory into Threads (0.94)
[ https://issues.apache.org/jira/browse/HBASE-6638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13440133#comment-13440133 ] Hadoop QA commented on HBASE-6638: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12542063/hbase-6638-v0.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 6 new or modified tests. -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/2663//console This message is automatically generated. Move DaemonThreadFactory into Threads (0.94) Key: HBASE-6638 URL: https://issues.apache.org/jira/browse/HBASE-6638 Project: HBase Issue Type: Bug Affects Versions: 0.94.2 Reporter: Jesse Yates Assignee: Jesse Yates Fix For: 0.94.2 Attachments: hbase-6638-v0.patch Move DaemonThreadFactory out of HTable and into Threads since its a generally useful class. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6477) Use PB filter definitions in RPC
[ https://issues.apache.org/jira/browse/HBASE-6477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13440135#comment-13440135 ] Hadoop QA commented on HBASE-6477: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12542062/HBASE-6477-nonrb.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 46 new or modified tests. -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/2665//console This message is automatically generated. Use PB filter definitions in RPC Key: HBASE-6477 URL: https://issues.apache.org/jira/browse/HBASE-6477 Project: HBase Issue Type: Task Components: ipc, migration Reporter: Gregory Chanan Assignee: Gregory Chanan Fix For: 0.96.0 Attachments: HBASE-6477-nonrb.patch Use the filters introduced in HBASE-6454 in the rpc so they are extensible -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-2155) Add the option to bind to a specific IP address to the Nonblocking Thrift servers
[ https://issues.apache.org/jira/browse/HBASE-2155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13440137#comment-13440137 ] Hadoop QA commented on HBASE-2155: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12542078/HBASE-2155.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 hadoop2.0. The patch compiles against the hadoop 2.0 profile. +1 javadoc. The javadoc tool did not generate any warning messages. -1 javac. The applied patch generated 5 javac compiler warnings (more than the trunk's current 4 warnings). -1 findbugs. The patch appears to introduce 7 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hbase.replication.TestReplication org.apache.hadoop.hbase.master.TestSplitLogManager Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/2660//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2660//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2660//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2660//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2660//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2660//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/2660//console This message is automatically generated. Add the option to bind to a specific IP address to the Nonblocking Thrift servers - Key: HBASE-2155 URL: https://issues.apache.org/jira/browse/HBASE-2155 Project: HBase Issue Type: Improvement Components: thrift Reporter: Lars Francke Assignee: liang xie Priority: Minor Fix For: 0.96.0 Attachments: HBASE-2155.patch This is not possible in Thrift 0.2.0 so we'll have to wait until the next version is released (which includes THRIFT-684). After that is released this is an easy and quick fix. For a few more details see HBASE-1373 and HBASE-65. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6632) [0.92 UNIT TESTS] testCreateTableRPCTimeOut sets rpc timeout to 1500ms and leaves it (testHundredsOfTable fails w/ 1500ms timeout)
[ https://issues.apache.org/jira/browse/HBASE-6632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13440145#comment-13440145 ] Hudson commented on HBASE-6632: --- Integrated in HBase-0.92-security #116 (See [https://builds.apache.org/job/HBase-0.92-security/116/]) HBASE-6632 [0.92 UNIT TESTS] testCreateTableRPCTimeOut sets rpc timeout to 1500ms and leaves it (testHundredsOfTable fails w/ 1500ms timeout) (Revision 1375899) Result = FAILURE stack : Files : * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/client/TestAdmin.java [0.92 UNIT TESTS] testCreateTableRPCTimeOut sets rpc timeout to 1500ms and leaves it (testHundredsOfTable fails w/ 1500ms timeout) -- Key: HBASE-6632 URL: https://issues.apache.org/jira/browse/HBASE-6632 Project: HBase Issue Type: Bug Reporter: stack Assignee: stack Fix For: 0.92.2, 0.94.2 Attachments: 6632-trunk.txt, 6632.txt I see that in 0.92 #502 and #501 that TestAdmin.testHundredsOfTable fails because socket times out after 1500ms. I see in TestAdmin that before this test runs, testCreateTableRPCTimeOut sets the socket timeout to 1500 and then does not set it back. Maybe the obnoxious testHundredsOfTable will pass more often if it has the default rpc timeout. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6308) Coprocessors should be loaded in a custom ClassLoader to prevent dependency conflicts with HBase
[ https://issues.apache.org/jira/browse/HBASE-6308?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13440144#comment-13440144 ] Hudson commented on HBASE-6308: --- Integrated in HBase-0.92-security #116 (See [https://builds.apache.org/job/HBase-0.92-security/116/]) HBASE-6308. Coprocessors should be loaded in a custom ClassLoader (James Baldassari) (Revision 1372562) Result = FAILURE apurtell : Files : * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/coprocessor/CoprocessorClassLoader.java * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/coprocessor/CoprocessorHost.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/coprocessor/TestClassLoading.java Coprocessors should be loaded in a custom ClassLoader to prevent dependency conflicts with HBase Key: HBASE-6308 URL: https://issues.apache.org/jira/browse/HBASE-6308 Project: HBase Issue Type: Improvement Components: coprocessors Affects Versions: 0.92.1, 0.94.0 Reporter: James Baldassari Assignee: Andrew Purtell Fix For: 0.92.2, 0.96.0, 0.94.2 Attachments: 6308-v2.txt, HBASE-6308-0.92.patch, HBASE-6308-0.92-with-test.patch, HBASE-6308-0.94-with-test.patch, HBASE-6308-trunk.patch, HBASE-6308-trunk-with-test.patch Currently each coprocessor is loaded with a URLClassLoader that puts the coprocessor's jar at the beginning of the classpath. The URLClassLoader always tries to load classes from the parent ClassLoader first and only attempts to load from its own configured URLs if the class was not found by the parent. This class loading behavior can be problematic for coprocessors that have common dependencies with HBase but whose versions are incompatible. For example, I have a coprocessor that depends on a different version of Avro than the version used by HBase. The current class loading behavior results in NoSuchMethodErrors in my coprocessor because some Avro classes have already been loaded by HBase, and the ClassLoader for my coprocessor picks up HBase's loaded classes first. My proposed solution to this problem is to use a custom ClassLoader when instantiating coprocessor instances. This custom ClassLoader would always attempt to load classes from the coprocessor's jar first and would only delegate to the parent ClassLoader if the class were not found in the coprocessor jar. However, certain classes would need to be exempt from this behavior. As an example, if the Copcoessor interface were loaded by both the region server's ClassLoader and the coprocessor's custom ClassLoader, then the region server would get a ClassCastException when attempting to cast the coprocessor instance to the Coprocessor interface. This problem can be avoided by defining a set of class name prefixes that would be exempt from loading by the custom ClassLoader. When loading a class, if the class starts with any of these prefixes (e.g. org.apache.hadoop), then the ClassLoader would delegate immediately to the parent ClassLoader. I've already implemented a patch to provide this functionality which I'll attach shortly. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6565) Coprocessor exec result Map is not thread safe
[ https://issues.apache.org/jira/browse/HBASE-6565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13440151#comment-13440151 ] Hudson commented on HBASE-6565: --- Integrated in HBase-0.92-security #116 (See [https://builds.apache.org/job/HBase-0.92-security/116/]) HBASE-6565. Coprocessor exec result Map is not thread safe (Yuan Kang) (Revision 1373976) Result = FAILURE apurtell : Files : * /hbase/branches/0.92/CHANGES.txt * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/client/HTable.java Coprocessor exec result Map is not thread safe -- Key: HBASE-6565 URL: https://issues.apache.org/jira/browse/HBASE-6565 Project: HBase Issue Type: Bug Components: client, coprocessors Affects Versions: 0.92.2, 0.94.0, 0.96.0 Environment: hadoop1.0.2,hbase0.94,jdk1.6 Reporter: Yuan Kang Assignee: Yuan Kang Labels: coprocessors, patch Fix For: 0.92.2, 0.96.0, 0.94.2 Attachments: Coprocessor-result-thread unsafe-bug-fix.patch Original Estimate: 168h Remaining Estimate: 168h I develop a coprocessor program ,but found some different results in repeated tests.for example,normally,the result's size is 10.but sometimes it appears 9. I read the HTable.java code,found a TreeMap(thread-unsafe) be used in multithreading environment.It cause the bug happened -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6576) HBaseAdmin.createTable should wait until the table is enabled
[
https://issues.apache.org/jira/browse/HBASE-6576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13440155#comment-13440155
]
Hudson commented on HBASE-6576:
---
Integrated in HBase-0.92-security #116 (See
[https://builds.apache.org/job/HBase-0.92-security/116/])
HBASE-6576 HBaseAdmin.createTable should wait until the table is enabled
(Gregory) (Revision 1373833)
Result = FAILURE
tedyu :
Files :
*
/hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java
HBaseAdmin.createTable should wait until the table is enabled
-
Key: HBASE-6576
URL: https://issues.apache.org/jira/browse/HBASE-6576
Project: HBase
Issue Type: Bug
Components: client, test
Reporter: Gregory Chanan
Assignee: Gregory Chanan
Fix For: 0.92.2, 0.96.0, 0.94.2
Attachments: HBASE-6576-92.patch, HBASE-6576-94.patch,
HBASE-6576-trunk.patch
The function:
{code}
public void createTable(final HTableDescriptor desc, byte [][] splitKeys)
{code}
in HBaseAdmin is synchronous and returns once all the regions of the table
are online, but does not wait for the table to be enabled, which is the last
step of table creation (see CreateTableHandler).
This is confusing and leads to racy code because users do not realize that
this is the case. For example, I saw the following test failure in 0.92 when
I ran:
mvn test
-Dtest=org.apache.hadoop.hbase.client.TestAdmin#testEnableDisableAddColumnDeleteColumn
{code}
Error Message
org.apache.hadoop.hbase.TableNotEnabledException: testMasterAdmin at
org.apache.hadoop.hbase.master.handler.DisableTableHandler.init(DisableTableHandler.java:75)
at org.apache.hadoop.hbase.master.HMaster.disableTable(HMaster.java:1154) at
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597) at
org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364)
at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1336)
Stacktrace
org.apache.hadoop.hbase.TableNotEnabledException:
org.apache.hadoop.hbase.TableNotEnabledException: testMasterAdmin
at
org.apache.hadoop.hbase.master.handler.DisableTableHandler.init(DisableTableHandler.java:75)
at org.apache.hadoop.hbase.master.HMaster.disableTable(HMaster.java:1154)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364)
at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1336)
{code}
The issue is that code will create and table and immediately disable it in
order to do some testing, for example, to test an operation that only works
when the table is disabled. If the table has not been enabled yet, they will
get back a TableNotEnabledException.
The specific test above was fixed in HBASE-5206, but other examples exist in
the code, for example the following:
{code}
hbase org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat newtable asdf14
{code}
The code in question is:
{code}
byte[] tname = args[1].getBytes();
HTable table = util.createTable(tname, FAMILIES);
HBaseAdmin admin = new HBaseAdmin(conf);
admin.disableTable(tname);
{code}
It would be better if createTable just waited until the table was enabled, or
threw a TableNotEnabledException if it exhausted the configured number of
retries.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6631) TestHMasterRPCException in 0.92 failed twice on socket timeout
[ https://issues.apache.org/jira/browse/HBASE-6631?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13440146#comment-13440146 ] Hudson commented on HBASE-6631: --- Integrated in HBase-0.92-security #116 (See [https://builds.apache.org/job/HBase-0.92-security/116/]) HBASE-6631 TestHMasterRPCException in 0.92 failed twice on socket timeout (Revision 1375844) Result = FAILURE stack : Files : * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/master/TestHMasterRPCException.java TestHMasterRPCException in 0.92 failed twice on socket timeout -- Key: HBASE-6631 URL: https://issues.apache.org/jira/browse/HBASE-6631 Project: HBase Issue Type: Bug Reporter: stack Assignee: stack Fix For: 0.92.2, 0.94.2 Attachments: 6631-trunk.txt, 6631-trunk.txt, 6631.txt #502 and #498 0.92 builds have TestHMasterRPCException failing because of socket timeout when servernotrunning is expected. Socket timeout is 100ms only. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6503) HBase Shell Documentation For DROP Is Outdated
[ https://issues.apache.org/jira/browse/HBASE-6503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13440148#comment-13440148 ] Hudson commented on HBASE-6503: --- Integrated in HBase-0.92-security #116 (See [https://builds.apache.org/job/HBase-0.92-security/116/]) HBASE-6503 HBase Shell Documentation For DROP Is Outdated (Revision 1375207) Result = FAILURE stack : Files : * /hbase/branches/0.92/src/main/ruby/shell/commands/drop.rb HBase Shell Documentation For DROP Is Outdated -- Key: HBASE-6503 URL: https://issues.apache.org/jira/browse/HBASE-6503 Project: HBase Issue Type: Bug Reporter: Paul Cavallaro Assignee: Paul Cavallaro Priority: Trivial Fix For: 0.92.2, 0.94.2 Attachments: HBASE-6503-example.patch, HBASE-6503.patch HBase Shell help documentation for the drop command says: If table has more than one region, run a major compaction on .META. According to JD this is old news: jdcryans: back in the days when hadoop didn't support durability it was possible to lose .META. data so we were force flushing .META. and major compacting it all the time also we used to have consistency issues that major compacting was solving ahhh the good old days -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6608) Fix for HBASE-6160, META entries from daughters can be deleted before parent entries, shouldn't compare HRegionInfo's
[
https://issues.apache.org/jira/browse/HBASE-6608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13440156#comment-13440156
]
Hudson commented on HBASE-6608:
---
Integrated in HBase-0.92-security #116 (See
[https://builds.apache.org/job/HBase-0.92-security/116/])
HBASE-6608 Fix for HBASE-6160, META entries from daughters can be deleted
before parent entries, shouldn't compare HRegionInfo's (Enis) (Revision 1375159)
Result = FAILURE
tedyu :
Files :
*
/hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/master/CatalogJanitor.java
*
/hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java
Fix for HBASE-6160, META entries from daughters can be deleted before parent
entries, shouldn't compare HRegionInfo's
-
Key: HBASE-6608
URL: https://issues.apache.org/jira/browse/HBASE-6608
Project: HBase
Issue Type: Bug
Components: client, regionserver
Affects Versions: 0.92.1, 0.96.0, 0.94.2
Reporter: Enis Soztutar
Assignee: Enis Soztutar
Fix For: 0.92.2, 0.96.0, 0.94.2
Attachments: 6608-v2.patch, hbase-6608_v1-0.92+0.94.patch,
hbase-6608_v1.patch
Our nightlies discovered that the patch for HBASE-6160 did not actually fix
the issue of META entries from daughters can be deleted before parent
entries. Instead of reopening the HBASE-6160, it is cleaner to track it
here.
The original issue is:
{quote}
HBASE-5986 fixed and issue, where the client sees the META entry for the
parent, but not the children. However, after the fix, we have seen the
following issue in tests:
Region A is split to - B, C
Region B is split to - D, E
After some time, META entry for B is deleted since it is not needed anymore,
but META entry for Region A stays in META (C still refers it). In this case,
the client throws RegionOfflineException for B.
{quote}
The problem with the fix seems to be that we keep and compare HRegionInfo's
in the HashSet at CatalogJanitor.java#scan(), but HRI that are compared are
not equal.
{code}
HashSetHRegionInfo parentNotCleaned = new HashSetHRegionInfo(); //regions
whose parents are still around
for (Map.EntryHRegionInfo, Result e : splitParents.entrySet()) {
if (!parentNotCleaned.contains(e.getKey()) cleanParent(e.getKey(),
e.getValue())) {
cleaned++;
} else {
...
{code}
In the above case, Meta row for region A will contain a serialized version of
B that is not offline. However Meta row for region B will contain a
serialized version of B that is offline (MetaEditor.offlineParentInMeta()
does that). So the deserialized version we put to HashSet and the
deserialized version we query contains() from HashSet are different in the
offline field, thus HRI.equals() fail.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5714) Add write permissions check before any hbck run that modifies hdfs.
[ https://issues.apache.org/jira/browse/HBASE-5714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13440154#comment-13440154 ] Hudson commented on HBASE-5714: --- Integrated in HBase-0.92-security #116 (See [https://builds.apache.org/job/HBase-0.92-security/116/]) HBASE-5714 Add write permissions check before any hbck run that modifies hdfs (Liang Xie) (Revision 1375229) Result = FAILURE jmhsieh : Files : * /hbase/branches/0.92/CHANGES.txt * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/util/FSUtils.java * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java Add write permissions check before any hbck run that modifies hdfs. --- Key: HBASE-5714 URL: https://issues.apache.org/jira/browse/HBASE-5714 Project: HBase Issue Type: Improvement Components: hbck Affects Versions: 0.90.6, 0.92.2, 0.94.0, 0.96.0 Reporter: Jonathan Hsieh Assignee: liang xie Fix For: 0.90.7, 0.92.2, 0.96.0, 0.94.2 Attachments: HBASE-5628.patch, HBASE-5628.patch.v2, hbase-5714-90.patch, hbase-5714-92.patch, hbase-5714-94.patch We encoutered a situation where hbck was run by an under-privileged user that was unable to write/modify/merge regions due to hdfs perms. Unfortunately, this user was alerted of this after several minutes of read-only operations. hbck should fail early by having a write perm check and providing actionable advice to the hbase admin. Maybe something like: Current user yy does not have write perms to hbase home. Please run hbck as hdfs user xxx -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6538) Remove copy_table.rb script
[ https://issues.apache.org/jira/browse/HBASE-6538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13440149#comment-13440149 ] Hudson commented on HBASE-6538: --- Integrated in HBase-0.92-security #116 (See [https://builds.apache.org/job/HBase-0.92-security/116/]) HBASE-6538 Remove copy_table.rb script (Revision 1374373) Result = FAILURE stack : Files : * /hbase/branches/0.92/bin/copy_table.rb Remove copy_table.rb script --- Key: HBASE-6538 URL: https://issues.apache.org/jira/browse/HBASE-6538 Project: HBase Issue Type: Task Components: scripts Affects Versions: 0.96.0 Reporter: David S. Wang Assignee: David S. Wang Priority: Minor Labels: noob Fix For: 0.92.2, 0.94.2 Attachments: hbase-6583-1.patch Remove copy_table.rb script as per mailing list discussion. It hasn't been maintained in a while and does not run against any recent HBase release. There is also an MR job to do the same thing that does work. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6160) META entries from daughters can be deleted before parent entries
[ https://issues.apache.org/jira/browse/HBASE-6160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13440147#comment-13440147 ] Hudson commented on HBASE-6160: --- Integrated in HBase-0.92-security #116 (See [https://builds.apache.org/job/HBase-0.92-security/116/]) HBASE-6608 Fix for HBASE-6160, META entries from daughters can be deleted before parent entries, shouldn't compare HRegionInfo's (Enis) (Revision 1375159) Result = FAILURE tedyu : Files : * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/master/CatalogJanitor.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java META entries from daughters can be deleted before parent entries Key: HBASE-6160 URL: https://issues.apache.org/jira/browse/HBASE-6160 Project: HBase Issue Type: Bug Components: client, regionserver Affects Versions: 0.92.2, 0.94.0, 0.96.0 Reporter: Enis Soztutar Assignee: Enis Soztutar Fix For: 0.92.2, 0.94.1 Attachments: HBASE-6160_v1.patch, HBASE-6160v2092.txt, HBASE-6160_v2.patch, HBASE-6160_v2.patch HBASE-5986 fixed and issue, where the client sees the META entry for the parent, but not the children. However, after the fix, we have seen the following issue in tests: Region A is split to - B, C Region B is split to - D, E After some time, META entry for B is deleted since it is not needed anymore, but META entry for Region A stays in META (C still refers it). In this case, the client throws RegionOfflineException for B. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6512) Incorrect OfflineMetaRepair log class name
[ https://issues.apache.org/jira/browse/HBASE-6512?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13440150#comment-13440150 ] Hudson commented on HBASE-6512: --- Integrated in HBase-0.92-security #116 (See [https://builds.apache.org/job/HBase-0.92-security/116/]) HBASE-6512 Incorrect OfflineMetaRepair log class name (Liang Xie) (Revision 1371522) Result = FAILURE jmhsieh : Files : * /hbase/branches/0.92/CHANGES.txt * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/util/hbck/OfflineMetaRepair.java Incorrect OfflineMetaRepair log class name -- Key: HBASE-6512 URL: https://issues.apache.org/jira/browse/HBASE-6512 Project: HBase Issue Type: Bug Components: hbck Affects Versions: 0.94.0, 0.96.0, 0.94.1, 0.94.2 Reporter: liang xie Assignee: liang xie Fix For: 0.92.2, 0.96.0, 0.94.2 Attachments: HBASE-6512.diff At the beginning of OfflineMetaRepair.java, we can observe: private static final Log LOG = LogFactory.getLog(HBaseFsck.class.getName()); It would be better change to : private static final Log LOG = LogFactory.getLog(OfflineMetaRepair.class.getName()); -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5549) Master can fail if ZooKeeper session expires
[ https://issues.apache.org/jira/browse/HBASE-5549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13440153#comment-13440153 ] Hudson commented on HBASE-5549: --- Integrated in HBase-0.92-security #116 (See [https://builds.apache.org/job/HBase-0.92-security/116/]) HBASE-5549 Master can fail if ZooKeeper session expires (Revision 1376369) Result = FAILURE stack : Files : * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/master/ActiveMasterManager.java * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/master/HMaster.java * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/zookeeper/RecoverableZooKeeper.java * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/zookeeper/ZooKeeperWatcher.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/TestZooKeeper.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/master/TestDistributedLogSplitting.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/master/TestMasterZKSessionRecovery.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/regionserver/TestSplitTransactionOnCluster.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/replication/TestReplication.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/replication/TestReplicationPeer.java Master can fail if ZooKeeper session expires Key: HBASE-5549 URL: https://issues.apache.org/jira/browse/HBASE-5549 Project: HBase Issue Type: Bug Components: master, zookeeper Affects Versions: 0.96.0 Environment: all Reporter: nkeywal Assignee: nkeywal Priority: Minor Fix For: 0.92.2, 0.96.0, 0.94.2 Attachments: 5549_092.txt, 5549_094.txt, 5549.v10.patch, 5549.v11.patch, 5549.v6.patch, 5549.v7.patch, 5549.v8.patch, 5549.v9.patch, nochange.patch There is a retry mechanism in RecoverableZooKeeper, but when the session expires, the whole ZooKeeperWatcher is recreated, hence the retry mechanism does not work in this case. This is why a sleep is needed in TestZooKeeper#testMasterSessionExpired: we need to wait for ZooKeeperWatcher to be recreated before using the connection. This can happen in real life, it can happen when: - master zookeeper starts - zookeeper connection is cut - master enters the retry loop - in the meantime the session expires - the network comes back, the session is recreated - the retries continues, but on the wrong object, hence fails. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6552) TestAcidGuarantees system test should flush more aggressively
[ https://issues.apache.org/jira/browse/HBASE-6552?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13440152#comment-13440152 ] Hudson commented on HBASE-6552: --- Integrated in HBase-0.92-security #116 (See [https://builds.apache.org/job/HBase-0.92-security/116/]) HBASE-6552 TestAcidGuarantees system test should flush more aggresively (Gregory Chanan) (Revision 1371503) Result = FAILURE jmhsieh : Files : * /hbase/branches/0.92/CHANGES.txt * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/TestAcidGuarantees.java TestAcidGuarantees system test should flush more aggressively - Key: HBASE-6552 URL: https://issues.apache.org/jira/browse/HBASE-6552 Project: HBase Issue Type: Bug Components: test Affects Versions: 0.92.2, 0.96.0, 0.94.1 Reporter: Gregory Chanan Assignee: Gregory Chanan Fix For: 0.92.2, 0.96.0, 0.94.2 Attachments: HBASE-6552-94-92.patch, HBASE-6552-trunk.patch HBASE-5887 allowed TestAcidGuarantees to be run as a system test by avoiding the call to util.flush(). It would be better to go through the HBaseAdmin interface to force flushes. This would unify the code path between the unit test and the system test, as well as forcing more frequent flushes, which have previously been the source of ACID guarantee problems, e.g. HBASE-2856. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6469) Failure on enable/disable table will cause table state in zk to be left as enabling/disabling until master is restart
[
https://issues.apache.org/jira/browse/HBASE-6469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13440174#comment-13440174
]
ramkrishna.s.vasudevan commented on HBASE-6469:
---
HBCK also does not handle this case. It only tries to assign or unassign based
on ENABLING/DISABLING but does not set the state of the table.
Failure on enable/disable table will cause table state in zk to be left as
enabling/disabling until master is restart
-
Key: HBASE-6469
URL: https://issues.apache.org/jira/browse/HBASE-6469
Project: HBase
Issue Type: Bug
Affects Versions: 0.96.0, 0.94.2
Reporter: Enis Soztutar
Assignee: Enis Soztutar
Fix For: 0.94.2
In Enable/DisableTableHandler code, if something goes wrong in handling, the
table state in zk is left as ENABLING / DISABLING. After that we cannot force
any more action from the API or CLI, and the only recovery path is restarting
the master.
{code}
if (done) {
// Flip the table to enabled.
this.assignmentManager.getZKTable().setEnabledTable(
this.tableNameStr);
LOG.info(Table ' + this.tableNameStr
+ ' was successfully enabled. Status: done= + done);
} else {
LOG.warn(Table ' + this.tableNameStr
+ ' wasn't successfully enabled. Status: done= + done);
}
{code}
Here, if done is false, the table state is not changed. There is also no way
to set skipTableStateCheck from cli / api.
We have run into this issue a couple of times before.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5322) RetriesExhaustedException: Trying to contact region server
[ https://issues.apache.org/jira/browse/HBASE-5322?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13440201#comment-13440201 ] Ian Varley commented on HBASE-5322: --- A couple things: - Have you tried it with caching turned off entirely? If you're caching 300 rows, it may be that it's constantly replacing those 300 rows as you scan billions of rows - It's also possible that your full table scan simply takes longer than the timeout; if so, that's not a bug that needs fixing in HBase, that's just how it is. I know that it's possible to scan large numbers of rows, but I also know that avoiding timeouts can be a bit of an art when you're working with large sets. Generally speaking, in an interactive environment, HBase is really designed for small scans gets among a large data set, rather than scanning an entire data set (which is more the province of Map/Reduce). Lacking a specific stack trace, this issue is too general to follow up on. I propose we close this, and then if you run into an identifiable stack trace in the future, post it to the list and / or open a new ticket. Sound OK? RetriesExhaustedException: Trying to contact region server -- Key: HBASE-5322 URL: https://issues.apache.org/jira/browse/HBASE-5322 Project: HBase Issue Type: Bug Affects Versions: 0.90.4 Reporter: Karthik Pandian I have a hbase table which holds data for more than 10GB. Now I used the same client scanner to scan which fails and reports, Could not seek StoreFileScanner[HFileScanner for reader reader=hdfs. This issue occurs only for the table which holds huge data and not for tables holding small data. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6364) Powering down the server host holding the .META. table causes HBase Client to take excessively long to recover and connect to reassigned .META. table
[ https://issues.apache.org/jira/browse/HBASE-6364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13440213#comment-13440213 ] Hudson commented on HBASE-6364: --- Integrated in HBase-0.94-security #49 (See [https://builds.apache.org/job/HBase-0.94-security/49/]) HBASE-6364 Powering down the server host holding the .META. table causes HBase Client to take excessively long to recover and connect to reassigned .META. table - security addendum (Revision 1376136) Result = SUCCESS nkeywal : Files : * /hbase/branches/0.94/security/src/main/java/org/apache/hadoop/hbase/ipc/SecureClient.java Powering down the server host holding the .META. table causes HBase Client to take excessively long to recover and connect to reassigned .META. table - Key: HBASE-6364 URL: https://issues.apache.org/jira/browse/HBASE-6364 Project: HBase Issue Type: Bug Components: client Affects Versions: 0.90.6, 0.92.1, 0.94.0 Reporter: Suraj Varma Assignee: nkeywal Labels: client Fix For: 0.96.0, 0.94.2 Attachments: 6364.94.v2.nolargetest.patch, 6364.94.v2.nolargetest.security-addendum.patch, 6364-host-serving-META.v1.patch, 6364.v11.nolargetest.patch, 6364.v1.patch, 6364.v1.patch, 6364.v2.patch, 6364.v3.patch, 6364.v3.patch, 6364.v5.patch, 6364.v5.withtests.patch, 6364.v6.patch, 6364.v6.withtests.patch, 6364.v7.withtests.patch, 6364.v8.withtests.patch, 6364.v9.patch, stacktrace.txt When a server host with a Region Server holding the .META. table is powered down on a live cluster, while the HBase cluster itself detects and reassigns the .META. table, connected HBase Client's take an excessively long time to detect this and re-discover the reassigned .META. Workaround: Decrease the ipc.socket.timeout on HBase Client side to a low value (default is 20s leading to 35 minute recovery time; we were able to get acceptable results with 100ms getting a 3 minute recovery) This was found during some hardware failure testing scenarios. Test Case: 1) Apply load via client app on HBase cluster for several minutes 2) Power down the region server holding the .META. server (i.e. power off ... and keep it off) 3) Measure how long it takes for cluster to reassign META table and for client threads to re-lookup and re-orient to the lesser cluster (minus the RS and DN on that host). Observation: 1) Client threads spike up to maxThreads size ... and take over 35 mins to recover (i.e. for the thread count to go back to normal) - no client calls are serviced - they just back up on a synchronized method (see #2 below) 2) All the client app threads queue up behind the oahh.ipc.HBaseClient#setupIOStreams method http://tinyurl.com/7js53dj After taking several thread dumps we found that the thread within this synchronized method was blocked on NetUtils.connect(this.socket, remoteId.getAddress(), getSocketTimeout(conf)); The client thread that gets the synchronized lock would try to connect to the dead RS (till socket times out after 20s), retries, and then the next thread gets in and so forth in a serial manner. Workaround: --- Default ipc.socket.timeout is set to 20s. We dropped this to a low number (1000 ms, 100 ms, etc) on the client side hbase-site.xml. With this setting, the client threads recovered in a couple of minutes by failing fast and re-discovering the .META. table on a reassigned RS. Assumption: This ipc.socket.timeout is only ever used during the initial HConnection setup via the NetUtils.connect and should only ever be used when connectivity to a region server is lost and needs to be re-established. i.e it does not affect the normal RPC actiivity as this is just the connect timeout. During RS GC periods, any _new_ clients trying to connect will fail and will require .META. table re-lookups. This above timeout workaround is only for the HBase client side. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5549) Master can fail if ZooKeeper session expires
[ https://issues.apache.org/jira/browse/HBASE-5549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13440214#comment-13440214 ] Hudson commented on HBASE-5549: --- Integrated in HBase-0.94-security #49 (See [https://builds.apache.org/job/HBase-0.94-security/49/]) HBASE-5549 Master can fail if ZooKeeper session expires (Revision 1376374) Result = SUCCESS stack : Files : * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/ActiveMasterManager.java * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/HMaster.java * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/zookeeper/RecoverableZooKeeper.java * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/zookeeper/ZooKeeperWatcher.java * /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java * /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/TestZooKeeper.java * /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/master/TestDistributedLogSplitting.java * /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/master/TestMasterZKSessionRecovery.java * /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/regionserver/TestSplitTransactionOnCluster.java * /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/replication/TestReplication.java Master can fail if ZooKeeper session expires Key: HBASE-5549 URL: https://issues.apache.org/jira/browse/HBASE-5549 Project: HBase Issue Type: Bug Components: master, zookeeper Affects Versions: 0.96.0 Environment: all Reporter: nkeywal Assignee: nkeywal Priority: Minor Fix For: 0.92.2, 0.96.0, 0.94.2 Attachments: 5549_092.txt, 5549_094.txt, 5549.v10.patch, 5549.v11.patch, 5549.v6.patch, 5549.v7.patch, 5549.v8.patch, 5549.v9.patch, nochange.patch There is a retry mechanism in RecoverableZooKeeper, but when the session expires, the whole ZooKeeperWatcher is recreated, hence the retry mechanism does not work in this case. This is why a sleep is needed in TestZooKeeper#testMasterSessionExpired: we need to wait for ZooKeeperWatcher to be recreated before using the connection. This can happen in real life, it can happen when: - master zookeeper starts - zookeeper connection is cut - master enters the retry loop - in the meantime the session expires - the network comes back, the session is recreated - the retries continues, but on the wrong object, hence fails. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6524) Hooks for hbase tracing
[ https://issues.apache.org/jira/browse/HBASE-6524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13440223#comment-13440223 ] Hudson commented on HBASE-6524: --- Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #143 (See [https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/143/]) HBASE-6524 revert due to new test failures against hadoop 2.0 (Revision 1376365) Result = FAILURE tedyu : Files : * /hbase/trunk/hbase-server/pom.xml * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/executor/EventHandler.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/HBaseClient.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/HBaseServer.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/handler/DisableTableHandler.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/handler/EnableTableHandler.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/protobuf/generated/FilterProtos.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/protobuf/generated/RPCProtos.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/protobuf/generated/Tracing.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/trace/HBaseLocalFileSpanReceiver.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/trace/SpanReceiverHost.java * /hbase/trunk/hbase-server/src/main/protobuf/RPC.proto * /hbase/trunk/hbase-server/src/main/protobuf/Tracing.proto * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/trace/TestHTraceHooks.java Hooks for hbase tracing --- Key: HBASE-6524 URL: https://issues.apache.org/jira/browse/HBASE-6524 Project: HBase Issue Type: Sub-task Reporter: Jonathan Leavitt Fix For: 0.96.0 Attachments: 6524.addendum, createTableTrace.png, hbase-6524.diff Includes the hooks that use [htrace|http://www.github.com/cloudera/htrace] library to add dapper-like tracing to hbase. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-6641) more message with DoNotRetryIOException in client
Zhou wenjian created HBASE-6641: --- Summary: more message with DoNotRetryIOException in client Key: HBASE-6641 URL: https://issues.apache.org/jira/browse/HBASE-6641 Project: HBase Issue Type: Bug Components: client Affects Versions: 0.94.0 Reporter: Zhou wenjian Assignee: Zhou wenjian Fix For: 0.94.2 when write a row with wrong or unexist family into a table , we will get message below org.apache.hadoop.hbase.client.RetriesExhaustedWithDetailsException: Failed 2000 actions: DoNotRetryIOException: 2000 times, servers with issues: dw82.kgb.sqa.cm4:64020, at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatchCallback(HConnectionManager.java:1591) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatch(HConnectionManager.java:1367) at org.apache.hadoop.hbase.client.HTable.flushCommits(HTable.java:945) at org.apache.hadoop.hbase.client.HTable.doPut(HTable.java:801) at org.apache.hadoop.hbase.client.HTable.put(HTable.java:784) at zookeeper.WriteMultiThread.doInsert(WriteToTable.java:101) at zookeeper.WriteMultiThread.run(WriteToTable.java:80) it not friendly to the client. Need to show the client more details about the exception. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6641) more message with DoNotRetryIOException in client
[ https://issues.apache.org/jira/browse/HBASE-6641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhou wenjian updated HBASE-6641: Attachment: HBASE-6641.patch more message with DoNotRetryIOException in client - Key: HBASE-6641 URL: https://issues.apache.org/jira/browse/HBASE-6641 Project: HBase Issue Type: Bug Components: client Affects Versions: 0.94.0 Reporter: Zhou wenjian Assignee: Zhou wenjian Fix For: 0.94.2 Attachments: HBASE-6641.patch when write a row with wrong or unexist family into a table , we will get message below org.apache.hadoop.hbase.client.RetriesExhaustedWithDetailsException: Failed 2000 actions: DoNotRetryIOException: 2000 times, servers with issues: dw82.kgb.sqa.cm4:64020, at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatchCallback(HConnectionManager.java:1591) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatch(HConnectionManager.java:1367) at org.apache.hadoop.hbase.client.HTable.flushCommits(HTable.java:945) at org.apache.hadoop.hbase.client.HTable.doPut(HTable.java:801) at org.apache.hadoop.hbase.client.HTable.put(HTable.java:784) at zookeeper.WriteMultiThread.doInsert(WriteToTable.java:101) at zookeeper.WriteMultiThread.run(WriteToTable.java:80) it not friendly to the client. Need to show the client more details about the exception. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6641) more message with DoNotRetryIOException in client
[
https://issues.apache.org/jira/browse/HBASE-6641?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13440230#comment-13440230
]
Zhou wenjian commented on HBASE-6641:
-
After applying the patch,we could get what happened
org.apache.hadoop.hbase.client.RetriesExhaustedWithDetailsException: Failed
2000 actions: org.apache.hadoop.hbase.DoNotRetryIOException: Column family a
does not exist in region
logTable,8LK27P38EYG15H7JDEB5YT6TK1MADYZYGFXFE330L3MB29G5Z7,1345183514575.0c8bf995b1e70d6f26fdcaa03c701035.
in table {NAME = 'logTable', FAMILIES = [{NAME = 'cf', DATA_BLOCK_ENCODING
= 'NONE', BLOOMFILTER = 'NONE', REPLICATION_SCOPE = '0', COMPRESSION =
'NONE', VERSIONS = '3', TTL = '2147483647', MIN_VERSIONS = '0',
KEEP_DELETED_CELLS = 'false', BLOCKSIZE = '65536', ENCODE_ON_DISK = 'true',
IN_MEMORY = 'false', BLOCKCACHE = 'true'}]}
at
org.apache.hadoop.hbase.regionserver.HRegionServer.multi(HRegionServer.java:3773)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364)
at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1401)
more message with DoNotRetryIOException in client
-
Key: HBASE-6641
URL: https://issues.apache.org/jira/browse/HBASE-6641
Project: HBase
Issue Type: Bug
Components: client
Affects Versions: 0.94.0
Reporter: Zhou wenjian
Assignee: Zhou wenjian
Fix For: 0.94.2
Attachments: HBASE-6641.patch
when write a row with wrong or unexist family into a table , we will get
message below
org.apache.hadoop.hbase.client.RetriesExhaustedWithDetailsException: Failed
2000 actions: DoNotRetryIOException: 2000 times, servers with issues:
dw82.kgb.sqa.cm4:64020, at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatchCallback(HConnectionManager.java:1591)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatch(HConnectionManager.java:1367)
at org.apache.hadoop.hbase.client.HTable.flushCommits(HTable.java:945)
at org.apache.hadoop.hbase.client.HTable.doPut(HTable.java:801)
at org.apache.hadoop.hbase.client.HTable.put(HTable.java:784)
at zookeeper.WriteMultiThread.doInsert(WriteToTable.java:101)
at zookeeper.WriteMultiThread.run(WriteToTable.java:80)
it not friendly to the client. Need to show the client more details about the
exception.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6473) deleted table is not deleted completely, some region may be still online
[
https://issues.apache.org/jira/browse/HBASE-6473?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhou wenjian updated HBASE-6473:
Attachment: HBASE-6473-trunk-v4.patch
deleted table is not deleted completely, some region may be still online
Key: HBASE-6473
URL: https://issues.apache.org/jira/browse/HBASE-6473
Project: HBase
Issue Type: Bug
Affects Versions: 0.94.0
Reporter: zhou wenjian
Assignee: Zhou wenjian
Fix For: 0.96.0, 0.94.2
Attachments: HBASE-6473-trunk.patch, HBASE-6473-trunk-v2.patch,
HBASE-6473-trunk-v3.patch, HBASE-6473-trunk-v4.patch
consider such Scenario:
we have a table called T1, which has 1 regions: A
1. move A from rs1 to rs2,and A is now closed
2. disable T1,
3. delete T1.
when we disable T1, disable handler will just set the zk to disabled and A
will still be assigned. when Ais opened, A in transition will be clean out.
At that time, Deletetable found it is safe to delete all regions and table in
meta and fs , it will also delete the zk node of T1.
{code}
while (System.currentTimeMillis() done) {
AssignmentManager.RegionState rs = am.isRegionInTransition(region);
if (rs == null) break;
Threads.sleep(waitingTimeForEvents);
LOG.debug(Waiting on region to clear regions in transition; + rs);
}
if (am.isRegionInTransition(region) != null) {
throw new IOException(Waited hbase.master.wait.on.region ( +
waitTime + ms) for region to leave region +
region.getRegionNameAsString() + in transitions);
}
{code}
however A is still being unassigned, when it finished closed the A,it finds
that the disabled state in zk is deleted, and then A will be assigned again.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6537) Balancer compete with disable table will lead to cluster inconsistent
[ https://issues.apache.org/jira/browse/HBASE-6537?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhou wenjian updated HBASE-6537: Attachment: (was: HBASE-6537-94.patch) Balancer compete with disable table will lead to cluster inconsistent -- Key: HBASE-6537 URL: https://issues.apache.org/jira/browse/HBASE-6537 Project: HBase Issue Type: Bug Components: master Affects Versions: 0.94.0 Reporter: Zhou wenjian Assignee: Zhou wenjian Fix For: 0.94.2 Attachments: HBASE-6537-trunk.patch Appear in 94. trunk is ok for the issue Balancer will collect the regionplans to move(unassign and then assign). before unassign, disable table appears, after close the region in rs, master will delete the znode, romove region from RIT, and then clean the region from the online regions. During romoving region from RIT and cleaning out the region from the online regions. balancer begins to unassign, it will get a NotServingRegionException and if the table is disabling, it will deal with the state in master and delete the znode . However the table is disabled now, so the RIT and znode will remain. TimeoutMonitor draws a blank on it. It will hold back enabling the table or balancer unless restart -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HBASE-6537) Balancer compete with disable table will lead to cluster inconsistent
[ https://issues.apache.org/jira/browse/HBASE-6537?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhou wenjian reassigned HBASE-6537: --- Assignee: Zhou wenjian Balancer compete with disable table will lead to cluster inconsistent -- Key: HBASE-6537 URL: https://issues.apache.org/jira/browse/HBASE-6537 Project: HBase Issue Type: Bug Components: master Affects Versions: 0.94.0 Reporter: Zhou wenjian Assignee: Zhou wenjian Fix For: 0.94.2 Attachments: HBASE-6537-trunk.patch Appear in 94. trunk is ok for the issue Balancer will collect the regionplans to move(unassign and then assign). before unassign, disable table appears, after close the region in rs, master will delete the znode, romove region from RIT, and then clean the region from the online regions. During romoving region from RIT and cleaning out the region from the online regions. balancer begins to unassign, it will get a NotServingRegionException and if the table is disabling, it will deal with the state in master and delete the znode . However the table is disabled now, so the RIT and znode will remain. TimeoutMonitor draws a blank on it. It will hold back enabling the table or balancer unless restart -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6537) Balancer compete with disable table will lead to cluster inconsistent
[ https://issues.apache.org/jira/browse/HBASE-6537?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhou wenjian updated HBASE-6537: Attachment: (was: HBASE-6537-94-v2.patch) Balancer compete with disable table will lead to cluster inconsistent -- Key: HBASE-6537 URL: https://issues.apache.org/jira/browse/HBASE-6537 Project: HBase Issue Type: Bug Components: master Affects Versions: 0.94.0 Reporter: Zhou wenjian Assignee: Zhou wenjian Fix For: 0.94.2 Attachments: HBASE-6537-trunk.patch Appear in 94. trunk is ok for the issue Balancer will collect the regionplans to move(unassign and then assign). before unassign, disable table appears, after close the region in rs, master will delete the znode, romove region from RIT, and then clean the region from the online regions. During romoving region from RIT and cleaning out the region from the online regions. balancer begins to unassign, it will get a NotServingRegionException and if the table is disabling, it will deal with the state in master and delete the znode . However the table is disabled now, so the RIT and znode will remain. TimeoutMonitor draws a blank on it. It will hold back enabling the table or balancer unless restart -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6537) Balancer compete with disable table will lead to cluster inconsistent
[ https://issues.apache.org/jira/browse/HBASE-6537?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhou wenjian updated HBASE-6537: Attachment: HBASE-6537-trunk-v2.patch Balancer compete with disable table will lead to cluster inconsistent -- Key: HBASE-6537 URL: https://issues.apache.org/jira/browse/HBASE-6537 Project: HBase Issue Type: Bug Components: master Affects Versions: 0.94.0 Reporter: Zhou wenjian Assignee: Zhou wenjian Fix For: 0.94.2 Attachments: HBASE-6537-trunk.patch, HBASE-6537-trunk-v2.patch Appear in 94. trunk is ok for the issue Balancer will collect the regionplans to move(unassign and then assign). before unassign, disable table appears, after close the region in rs, master will delete the znode, romove region from RIT, and then clean the region from the online regions. During romoving region from RIT and cleaning out the region from the online regions. balancer begins to unassign, it will get a NotServingRegionException and if the table is disabling, it will deal with the state in master and delete the znode . However the table is disabled now, so the RIT and znode will remain. TimeoutMonitor draws a blank on it. It will hold back enabling the table or balancer unless restart -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HBASE-6563) s.isMajorCompaction() throws npe will cause current major Compaction checking abort
[ https://issues.apache.org/jira/browse/HBASE-6563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhou wenjian reassigned HBASE-6563: --- Assignee: Zhou wenjian s.isMajorCompaction() throws npe will cause current major Compaction checking abort --- Key: HBASE-6563 URL: https://issues.apache.org/jira/browse/HBASE-6563 Project: HBase Issue Type: Bug Components: regionserver Reporter: Zhou wenjian Assignee: Zhou wenjian Fix For: 0.94.1 Attachments: HBASE-6563-trunk.patch, HBASE-6563-trunk-v2.patch, HBASE-6563-trunk-v3.patch 2012-05-05 00:49:43,265 ERROR org.apache.hadoop.hbase.regionserver.HRegionServer$MajorCompactionChecker: Caught exception java.lang.NullPointerException at org.apache.hadoop.hbase.regionserver.Store.isMajorCompaction(Store.java:938) at org.apache.hadoop.hbase.regionserver.Store.isMajorCompaction(Store.java:917) at org.apache.hadoop.hbase.regionserver.HRegion.isMajorCompaction(HRegion.java:3250) at org.apache.hadoop.hbase.regionserver.HRegionServer$MajorCompactionChecker.chore(HRegionServer.java:1222) at org.apache.hadoop.hbase.Chore.run(Chore.java:66) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6563) s.isMajorCompaction() throws npe will cause current major Compaction checking abort
[ https://issues.apache.org/jira/browse/HBASE-6563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhou wenjian updated HBASE-6563: Attachment: HBASE-6563-trunk-v3.patch s.isMajorCompaction() throws npe will cause current major Compaction checking abort --- Key: HBASE-6563 URL: https://issues.apache.org/jira/browse/HBASE-6563 Project: HBase Issue Type: Bug Components: regionserver Reporter: Zhou wenjian Assignee: Zhou wenjian Fix For: 0.94.1 Attachments: HBASE-6563-trunk.patch, HBASE-6563-trunk-v2.patch, HBASE-6563-trunk-v3.patch 2012-05-05 00:49:43,265 ERROR org.apache.hadoop.hbase.regionserver.HRegionServer$MajorCompactionChecker: Caught exception java.lang.NullPointerException at org.apache.hadoop.hbase.regionserver.Store.isMajorCompaction(Store.java:938) at org.apache.hadoop.hbase.regionserver.Store.isMajorCompaction(Store.java:917) at org.apache.hadoop.hbase.regionserver.HRegion.isMajorCompaction(HRegion.java:3250) at org.apache.hadoop.hbase.regionserver.HRegionServer$MajorCompactionChecker.chore(HRegionServer.java:1222) at org.apache.hadoop.hbase.Chore.run(Chore.java:66) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6618) Implement FuzzyRowFilter with ranges support
[ https://issues.apache.org/jira/browse/HBASE-6618?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13440242#comment-13440242 ] Alex Baranau commented on HBASE-6618: - Ah, sorry, haven't said anything about that. For toInc - we may not change it at every step, so if there's a missing arrow, that means nothing should be changed. Thanx for checking out! One thing that I'm not 100% sure about - is it better to adjust current FuzzyRowFilter and this functionality to it or add new. I'm leaning towards adjusting FuzzyRowFilter as this new feature fits naturally in it. Thoughts? Implement FuzzyRowFilter with ranges support Key: HBASE-6618 URL: https://issues.apache.org/jira/browse/HBASE-6618 Project: HBase Issue Type: New Feature Components: filters Reporter: Alex Baranau Priority: Minor Attachments: HBASE-6618-algo-desc-bits.png, HBASE-6618-algo.patch Apart from current ability to specify fuzzy row filter e.g. for userId_actionId format as _0004 (where 0004 - actionId) it would be great to also have ability to specify the fuzzy range , e.g. _0004, ..., _0099. See initial discussion here: http://search-hadoop.com/m/WVLJdX0Z65 Note: currently it is possible to provide multiple fuzzy row rules to existing FuzzyRowFilter, but in case when the range is big (contains thousands of values) it is not efficient. Filter should perform efficient fast-forwarding during the scan (this is what distinguishes it from regex row filter). While such functionality may seem like a proper fit for custom filter (i.e. not including into standard filter set) it looks like the filter may be very re-useable. We may judge based on the implementation that will hopefully be added. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-6642) disable_all '*' is not performing disable operation.
Y. SREENIVASULU REDDY created HBASE-6642:
Summary: disable_all '*' is not performing disable operation.
Key: HBASE-6642
URL: https://issues.apache.org/jira/browse/HBASE-6642
Project: HBase
Issue Type: Bug
Components: shell
Reporter: Y. SREENIVASULU REDDY
created few tables. then performing disable_all operation in shell prompt.
but it is not performing operation successfully.
{noformat}
hbase(main):043:0 disable_all '*'
table12
zk0113
zk0114
Disable the above 3 tables (y/n)?
y/
3 tables successfully disabled
just it is showing the message but operation is not success.
but the following way only performing successfully
hbase(main):043:0 disable_all '*.*'
table12
zk0113
zk0114
Disable the above 3 tables (y/n)?
y
3 tables successfully disabled
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6618) Implement FuzzyRowFilter with ranges support
[ https://issues.apache.org/jira/browse/HBASE-6618?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13440268#comment-13440268 ] Zhihong Ted Yu commented on HBASE-6618: --- Enhancing existing class is fine. Implement FuzzyRowFilter with ranges support Key: HBASE-6618 URL: https://issues.apache.org/jira/browse/HBASE-6618 Project: HBase Issue Type: New Feature Components: filters Reporter: Alex Baranau Priority: Minor Attachments: HBASE-6618-algo-desc-bits.png, HBASE-6618-algo.patch Apart from current ability to specify fuzzy row filter e.g. for userId_actionId format as _0004 (where 0004 - actionId) it would be great to also have ability to specify the fuzzy range , e.g. _0004, ..., _0099. See initial discussion here: http://search-hadoop.com/m/WVLJdX0Z65 Note: currently it is possible to provide multiple fuzzy row rules to existing FuzzyRowFilter, but in case when the range is big (contains thousands of values) it is not efficient. Filter should perform efficient fast-forwarding during the scan (this is what distinguishes it from regex row filter). While such functionality may seem like a proper fit for custom filter (i.e. not including into standard filter set) it looks like the filter may be very re-useable. We may judge based on the implementation that will hopefully be added. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3271) Allow .META. table to be exported
[
https://issues.apache.org/jira/browse/HBASE-3271?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13440289#comment-13440289
]
Zhihong Ted Yu commented on HBASE-3271:
---
{code}
Running org.apache.hadoop.hbase.mapreduce.TestImportExport
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 61.517 sec
{code}
Failed test is not related to this patch.
Patch integrated to trunk.
Thanks for the patch, Liang.
Allow .META. table to be exported
-
Key: HBASE-3271
URL: https://issues.apache.org/jira/browse/HBASE-3271
Project: HBase
Issue Type: Improvement
Components: util
Affects Versions: 0.20.6
Reporter: Ted Yu
Fix For: 0.96.0
Attachments: HBASE-3271.patch, HBASE-3271-v2.patch
I tried to export .META. table in 0.20.6 and got:
[hadoop@us01-ciqps1-name01 hbase]$ bin/hbase
org.apache.hadoop.hbase.mapreduce.Export .META. h-meta 1 0 0
10/11/23 20:59:05 INFO jvm.JvmMetrics: Initializing JVM Metrics with
processName=JobTracker, sessionId=
2010-11-23 20:59:05.255::INFO: Logging to STDERR via
org.mortbay.log.StdErrLog
2010-11-23 20:59:05.255::INFO: verisons=1, starttime=0,
endtime=9223372036854775807
10/11/23 20:59:05 INFO zookeeper.ZooKeeper: Client
environment:zookeeper.version=3.2.2-888565, built on 12/08/2009 21:51 GMT
10/11/23 20:59:05 INFO zookeeper.ZooKeeper: Client
environment:host.name=us01-ciqps1-name01.carrieriq.com
10/11/23 20:59:05 INFO zookeeper.ZooKeeper: Client
environment:java.version=1.6.0_21
10/11/23 20:59:05 INFO zookeeper.ZooKeeper: Client
environment:java.vendor=Sun Microsystems Inc.
...
10/11/23 20:59:05 INFO zookeeper.ClientCnxn: Server connection successful
10/11/23 20:59:05 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode
/hbase/root-region-server got 10.202.50.112:60020
10/11/23 20:59:05 DEBUG client.HConnectionManager$TableServers: Found ROOT at
10.202.50.112:60020
10/11/23 20:59:05 DEBUG client.HConnectionManager$TableServers: Cached
location for .META.,,1 is us01-ciqps1-grid02.carrieriq.com:60020
Exception in thread main java.io.IOException: Expecting at least one region.
at
org.apache.hadoop.hbase.mapreduce.TableInputFormatBase.getSplits(TableInputFormatBase.java:281)
at
org.apache.hadoop.mapred.JobClient.writeNewSplits(JobClient.java:885)
at
org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:779)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:432)
at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:447)
at org.apache.hadoop.hbase.mapreduce.Export.main(Export.java:146)
Related code is:
if (keys == null || keys.getFirst() == null ||
keys.getFirst().length == 0) {
throw new IOException(Expecting at least one region.);
}
My intention was to save the dangling rows in .META. (for future
investigation) which prevented a table from being created.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5329) addRowLock() may allocate duplicate lock id, causing the client to be blocked
[
https://issues.apache.org/jira/browse/HBASE-5329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13440295#comment-13440295
]
Zhihong Ted Yu commented on HBASE-5329:
---
I ran TestSplitLogManager#testOrphanTaskAcquisition manually and it passed.
Integrated to trunk.
Thanks for the patch, Ian.
addRowLock() may allocate duplicate lock id, causing the client to be blocked
-
Key: HBASE-5329
URL: https://issues.apache.org/jira/browse/HBASE-5329
Project: HBase
Issue Type: Bug
Components: regionserver
Affects Versions: 0.90.3
Environment: Red Hat Enterprise Linux Server release 5.4
Reporter: liaoxiangui
Assignee: Ian Varley
Priority: Minor
Fix For: 0.96.0
Attachments: 5329-v2.patch, HBASE-5329.patch
{code}
protected long addRowLock(Integer r, HRegion region) throws
LeaseStillHeldException
{
long lockId = -1L;
lockId = rand.nextLong(); //!!!may generate duplicated
id，bug?
String lockName = String.valueOf(lockId);
rowlocks.put(lockName, r);
this.leases.createLease(lockName, new RowLockListener(lockName,
region));
return lockId;
}
{code}
In addRowLock(),rand may generate duplicated lock id, it may cause
regionserver throw exception(Leases$LeaseStillHeldException).The client will
be blocked until old rowlock is released.
{code}
2012-02-03 15:21:50,084 ERROR
org.apache.hadoop.hbase.regionserver.HRegionServer: Error obtaining row lock
(fsOk: true)
org.apache.hadoop.hbase.regionserver.Leases$LeaseStillHeldException
at
org.apache.hadoop.hbase.regionserver.Leases.createLease(Leases.java:150)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.addRowLock(HRegionServer.java:1986)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.lockRow(HRegionServer.java:1963)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:570)
at
org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1039)
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-2155) Add the option to bind to a specific IP address to the Nonblocking Thrift servers
[ https://issues.apache.org/jira/browse/HBASE-2155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13440298#comment-13440298 ] Zhihong Ted Yu commented on HBASE-2155: --- Test failure is not related to thrift. Integrated to trunk. Thanks for the patch, Liang. Add the option to bind to a specific IP address to the Nonblocking Thrift servers - Key: HBASE-2155 URL: https://issues.apache.org/jira/browse/HBASE-2155 Project: HBase Issue Type: Improvement Components: thrift Reporter: Lars Francke Assignee: liang xie Priority: Minor Fix For: 0.96.0 Attachments: HBASE-2155.patch This is not possible in Thrift 0.2.0 so we'll have to wait until the next version is released (which includes THRIFT-684). After that is released this is an easy and quick fix. For a few more details see HBASE-1373 and HBASE-65. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6563) s.isMajorCompaction() throws npe will cause current major Compaction checking abort
[
https://issues.apache.org/jira/browse/HBASE-6563?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13440299#comment-13440299
]
Zhihong Ted Yu commented on HBASE-6563:
---
{code}
+ } catch (Exception e) {
+// Ignore the npe, see HBASE-6563
{code}
I would expect the exception caught to match comment.
s.isMajorCompaction() throws npe will cause current major Compaction checking
abort
---
Key: HBASE-6563
URL: https://issues.apache.org/jira/browse/HBASE-6563
Project: HBase
Issue Type: Bug
Components: regionserver
Reporter: Zhou wenjian
Assignee: Zhou wenjian
Fix For: 0.94.1
Attachments: HBASE-6563-trunk.patch, HBASE-6563-trunk-v2.patch,
HBASE-6563-trunk-v3.patch
2012-05-05 00:49:43,265 ERROR
org.apache.hadoop.hbase.regionserver.HRegionServer$MajorCompactionChecker:
Caught exception
java.lang.NullPointerException
at
org.apache.hadoop.hbase.regionserver.Store.isMajorCompaction(Store.java:938)
at
org.apache.hadoop.hbase.regionserver.Store.isMajorCompaction(Store.java:917)
at
org.apache.hadoop.hbase.regionserver.HRegion.isMajorCompaction(HRegion.java:3250)
at
org.apache.hadoop.hbase.regionserver.HRegionServer$MajorCompactionChecker.chore(HRegionServer.java:1222)
at org.apache.hadoop.hbase.Chore.run(Chore.java:66)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6581) Build with hadoop.profile=3.0
[
https://issues.apache.org/jira/browse/HBASE-6581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13440303#comment-13440303
]
Zhihong Ted Yu commented on HBASE-6581:
---
160 test failures.
Here is partial list:
{code}
testDelayedDeleteOnFailure(org.apache.hadoop.hbase.master.TestDistributedLogSplitting):
test timed out after 25000 milliseconds
org.apache.hadoop.hbase.master.TestZKBasedOpenCloseRegion: Shutting down
testMasterShutdown(org.apache.hadoop.hbase.master.TestMasterShutdown):
Shutting down
testMasterFailoverWithMockedRIT(org.apache.hadoop.hbase.master.TestMasterFailover):
test timed out after 18 milliseconds
testMasterFailoverWithMockedRITOnDeadRS(org.apache.hadoop.hbase.master.TestMasterFailover):
test timed out after 18 milliseconds
testShouldCheckMasterFailOverWhenMETAIsInOpenedState(org.apache.hadoop.hbase.master.TestMasterFailover):
test timed out after 18 milliseconds
testSimpleMasterFailover(org.apache.hadoop.hbase.master.TestMasterFailover):
Shutting down
testRestartClusterAfterKill(org.apache.hadoop.hbase.master.TestRestartCluster):
Shutting down
testClusterRestart(org.apache.hadoop.hbase.master.TestRestartCluster):
Shutting down
org.apache.hadoop.hbase.master.TestMaster: Shutting down
testOpenedRegionHandlerOnMasterRestart(org.apache.hadoop.hbase.master.TestOpenedRegionHandler):
Shutting down
org.apache.hadoop.hbase.master.TestMasterTransitions: Shutting down
org.apache.hadoop.hbase.master.TestAssignmentManagerOnCluster: Shutting down
org.apache.hadoop.hbase.rest.TestTableResource: Shutting down
org.apache.hadoop.hbase.rest.client.TestRemoteAdmin: Shutting down
org.apache.hadoop.hbase.rest.client.TestRemoteTable: Shutting down
org.apache.hadoop.hbase.rest.client.TestRemoteTable
org.apache.hadoop.hbase.rest.TestScannerResource: Shutting down
org.apache.hadoop.hbase.rest.TestVersionResource: Shutting down
org.apache.hadoop.hbase.rest.TestGzipFilter: Shutting down
org.apache.hadoop.hbase.rest.TestScannersWithFilters: Shutting down
org.apache.hadoop.hbase.rest.TestSchemaResource: Shutting down
org.apache.hadoop.hbase.rest.TestRowResource: Shutting down
org.apache.hadoop.hbase.rest.TestStatusResource: Shutting down
org.apache.hadoop.hbase.rest.TestMultiRowResource: Shutting down
testMultiClusters(org.apache.hadoop.hbase.TestHBaseTestingUtility): test
timed out after 18 milliseconds
testMiniCluster(org.apache.hadoop.hbase.TestHBaseTestingUtility): Shutting
down
testMultipleStartStop(org.apache.hadoop.hbase.TestHBaseTestingUtility):
Shutting down
testMiniDFSCluster(org.apache.hadoop.hbase.TestHBaseTestingUtility): Port in
use: localhost:0
testSetupClusterTestBuildDir(org.apache.hadoop.hbase.TestHBaseTestingUtility):
Port in use: localhost:0
Tests run: 1210, Failures: 0, Errors: 160, Skipped: 2
{code}
Build with hadoop.profile=3.0
-
Key: HBASE-6581
URL: https://issues.apache.org/jira/browse/HBASE-6581
Project: HBase
Issue Type: Bug
Reporter: Eric Charles
Attachments: HBASE-6581-1.patch, HBASE-6581-2.patch, HBASE-6581.diff
Building trunk with hadoop.profile=3.0 gives exceptions (see [1]) due to
change in the hadoop maven modules naming (and also usage of 3.0-SNAPSHOT
instead of 3.0.0-SNAPSHOT in hbase-common).
I can provide a patch that would move most of hadoop dependencies in their
respective profiles and will define the correct hadoop deps in the 3.0
profile.
Please tell me if that's ok to go this way.
Thx, Eric
[1]
$ mvn clean install -Dhadoop.profile=3.0
[INFO] Scanning for projects...
[ERROR] The build could not read 3 projects - [Help 1]
[ERROR]
[ERROR] The project org.apache.hbase:hbase-server:0.95-SNAPSHOT
(/d/hbase.svn/hbase-server/pom.xml) has 3 errors
[ERROR] 'dependencies.dependency.version' for
org.apache.hadoop:hadoop-common:jar is missing. @ line 655, column 21
[ERROR] 'dependencies.dependency.version' for
org.apache.hadoop:hadoop-annotations:jar is missing. @ line 659, column 21
[ERROR] 'dependencies.dependency.version' for
org.apache.hadoop:hadoop-minicluster:jar is missing. @ line 663, column 21
[ERROR]
[ERROR] The project org.apache.hbase:hbase-common:0.95-SNAPSHOT
(/d/hbase.svn/hbase-common/pom.xml) has 3 errors
[ERROR] 'dependencies.dependency.version' for
org.apache.hadoop:hadoop-common:jar is missing. @ line 170, column 21
[ERROR] 'dependencies.dependency.version' for
org.apache.hadoop:hadoop-annotations:jar is missing. @ line 174, column 21
[ERROR] 'dependencies.dependency.version' for
org.apache.hadoop:hadoop-minicluster:jar is missing. @ line 178, column 21
[ERROR]
[ERROR] The project org.apache.hbase:hbase-it:0.95-SNAPSHOT
(/d/hbase.svn/hbase-it/pom.xml) has 3 errors
[ERROR]
[jira] [Commented] (HBASE-5549) Master can fail if ZooKeeper session expires
[ https://issues.apache.org/jira/browse/HBASE-5549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13440351#comment-13440351 ] Himanshu Vashishtha commented on HBASE-5549: @Stack: So, currently (in this patch also), we don't wait for the Expired event, which is not correct. In 6354, v2 patch also doesn't look for it. I added this check (as mentioned earlier) but sometimes, the only event received is synConnected, and not the Expired event. So, it waits on the AtomicBoolean sessionClosed, to become true, which will happen only when the watcher received the Expired event. Thinking today, it looks like we should invoke the close method only *after* the monitorwatcher is properly initialized. Hmmm, I will look at this today again. Master can fail if ZooKeeper session expires Key: HBASE-5549 URL: https://issues.apache.org/jira/browse/HBASE-5549 Project: HBase Issue Type: Bug Components: master, zookeeper Affects Versions: 0.96.0 Environment: all Reporter: nkeywal Assignee: nkeywal Priority: Minor Fix For: 0.92.2, 0.96.0, 0.94.2 Attachments: 5549_092.txt, 5549_094.txt, 5549.v10.patch, 5549.v11.patch, 5549.v6.patch, 5549.v7.patch, 5549.v8.patch, 5549.v9.patch, nochange.patch There is a retry mechanism in RecoverableZooKeeper, but when the session expires, the whole ZooKeeperWatcher is recreated, hence the retry mechanism does not work in this case. This is why a sleep is needed in TestZooKeeper#testMasterSessionExpired: we need to wait for ZooKeeperWatcher to be recreated before using the connection. This can happen in real life, it can happen when: - master zookeeper starts - zookeeper connection is cut - master enters the retry loop - in the meantime the session expires - the network comes back, the session is recreated - the retries continues, but on the wrong object, hence fails. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-3271) Allow .META. table to be exported
[
https://issues.apache.org/jira/browse/HBASE-3271?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13440355#comment-13440355
]
Hudson commented on HBASE-3271:
---
Integrated in HBase-TRUNK #3257 (See
[https://builds.apache.org/job/HBase-TRUNK/3257/])
HBASE-3271 Allow .META. table to be exported (Liang Xie) (Revision 1376487)
Result = SUCCESS
Tedyu :
Files :
*
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/TableInputFormatBase.java
*
/hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/mapreduce/TestImportExport.java
Allow .META. table to be exported
-
Key: HBASE-3271
URL: https://issues.apache.org/jira/browse/HBASE-3271
Project: HBase
Issue Type: Improvement
Components: util
Affects Versions: 0.20.6
Reporter: Ted Yu
Fix For: 0.96.0
Attachments: HBASE-3271.patch, HBASE-3271-v2.patch
I tried to export .META. table in 0.20.6 and got:
[hadoop@us01-ciqps1-name01 hbase]$ bin/hbase
org.apache.hadoop.hbase.mapreduce.Export .META. h-meta 1 0 0
10/11/23 20:59:05 INFO jvm.JvmMetrics: Initializing JVM Metrics with
processName=JobTracker, sessionId=
2010-11-23 20:59:05.255::INFO: Logging to STDERR via
org.mortbay.log.StdErrLog
2010-11-23 20:59:05.255::INFO: verisons=1, starttime=0,
endtime=9223372036854775807
10/11/23 20:59:05 INFO zookeeper.ZooKeeper: Client
environment:zookeeper.version=3.2.2-888565, built on 12/08/2009 21:51 GMT
10/11/23 20:59:05 INFO zookeeper.ZooKeeper: Client
environment:host.name=us01-ciqps1-name01.carrieriq.com
10/11/23 20:59:05 INFO zookeeper.ZooKeeper: Client
environment:java.version=1.6.0_21
10/11/23 20:59:05 INFO zookeeper.ZooKeeper: Client
environment:java.vendor=Sun Microsystems Inc.
...
10/11/23 20:59:05 INFO zookeeper.ClientCnxn: Server connection successful
10/11/23 20:59:05 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode
/hbase/root-region-server got 10.202.50.112:60020
10/11/23 20:59:05 DEBUG client.HConnectionManager$TableServers: Found ROOT at
10.202.50.112:60020
10/11/23 20:59:05 DEBUG client.HConnectionManager$TableServers: Cached
location for .META.,,1 is us01-ciqps1-grid02.carrieriq.com:60020
Exception in thread main java.io.IOException: Expecting at least one region.
at
org.apache.hadoop.hbase.mapreduce.TableInputFormatBase.getSplits(TableInputFormatBase.java:281)
at
org.apache.hadoop.mapred.JobClient.writeNewSplits(JobClient.java:885)
at
org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:779)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:432)
at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:447)
at org.apache.hadoop.hbase.mapreduce.Export.main(Export.java:146)
Related code is:
if (keys == null || keys.getFirst() == null ||
keys.getFirst().length == 0) {
throw new IOException(Expecting at least one region.);
}
My intention was to save the dangling rows in .META. (for future
investigation) which prevented a table from being created.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5329) addRowLock() may allocate duplicate lock id, causing the client to be blocked
[
https://issues.apache.org/jira/browse/HBASE-5329?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13440354#comment-13440354
]
Hudson commented on HBASE-5329:
---
Integrated in HBase-TRUNK #3257 (See
[https://builds.apache.org/job/HBase-TRUNK/3257/])
HBASE-5329 addRowLock() may allocate duplicate lock id, causing the client
to be blocked (Ian Varley) (Revision 1376489)
Result = SUCCESS
Tedyu :
Files :
*
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
addRowLock() may allocate duplicate lock id, causing the client to be blocked
-
Key: HBASE-5329
URL: https://issues.apache.org/jira/browse/HBASE-5329
Project: HBase
Issue Type: Bug
Components: regionserver
Affects Versions: 0.90.3
Environment: Red Hat Enterprise Linux Server release 5.4
Reporter: liaoxiangui
Assignee: Ian Varley
Priority: Minor
Fix For: 0.96.0
Attachments: 5329-v2.patch, HBASE-5329.patch
{code}
protected long addRowLock(Integer r, HRegion region) throws
LeaseStillHeldException
{
long lockId = -1L;
lockId = rand.nextLong(); //!!!may generate duplicated
id，bug?
String lockName = String.valueOf(lockId);
rowlocks.put(lockName, r);
this.leases.createLease(lockName, new RowLockListener(lockName,
region));
return lockId;
}
{code}
In addRowLock(),rand may generate duplicated lock id, it may cause
regionserver throw exception(Leases$LeaseStillHeldException).The client will
be blocked until old rowlock is released.
{code}
2012-02-03 15:21:50,084 ERROR
org.apache.hadoop.hbase.regionserver.HRegionServer: Error obtaining row lock
(fsOk: true)
org.apache.hadoop.hbase.regionserver.Leases$LeaseStillHeldException
at
org.apache.hadoop.hbase.regionserver.Leases.createLease(Leases.java:150)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.addRowLock(HRegionServer.java:1986)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.lockRow(HRegionServer.java:1963)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:570)
at
org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1039)
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-2155) Add the option to bind to a specific IP address to the Nonblocking Thrift servers
[ https://issues.apache.org/jira/browse/HBASE-2155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13440356#comment-13440356 ] Hudson commented on HBASE-2155: --- Integrated in HBase-TRUNK #3257 (See [https://builds.apache.org/job/HBase-TRUNK/3257/]) HBASE-2155 Add the option to bind to a specific IP address to the Nonblocking Thrift servers (Liang Xie) (Revision 1376490) Result = SUCCESS Tedyu : Files : * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/thrift/ThriftServer.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/thrift/ThriftServerRunner.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/thrift2/ThriftServer.java Add the option to bind to a specific IP address to the Nonblocking Thrift servers - Key: HBASE-2155 URL: https://issues.apache.org/jira/browse/HBASE-2155 Project: HBase Issue Type: Improvement Components: thrift Reporter: Lars Francke Assignee: liang xie Priority: Minor Fix For: 0.96.0 Attachments: HBASE-2155.patch This is not possible in Thrift 0.2.0 so we'll have to wait until the next version is released (which includes THRIFT-684). After that is released this is an easy and quick fix. For a few more details see HBASE-1373 and HBASE-65. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5549) Master can fail if ZooKeeper session expires
[ https://issues.apache.org/jira/browse/HBASE-5549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13440378#comment-13440378 ] stack commented on HBASE-5549: -- @Himanshu Where is the issue which adds waiting on expire if its not being done over in hbase-6354? I backported this patch because it removed a test purportedly 'useless' that was messing up a subsequent test. I also backported it to get the other cleanups around zk interaction that this patch adds and to make it so there is parity here from 0.92 through to trunk. If there is an issue w/ fix for wait on expire, I can backport that too. Suggest you look at Alex's countdown latch over in his watcher in the this patch https://reviews.facebook.net/D4605 It does something like what you paste above for the expiration monitor. Master can fail if ZooKeeper session expires Key: HBASE-5549 URL: https://issues.apache.org/jira/browse/HBASE-5549 Project: HBase Issue Type: Bug Components: master, zookeeper Affects Versions: 0.96.0 Environment: all Reporter: nkeywal Assignee: nkeywal Priority: Minor Fix For: 0.92.2, 0.96.0, 0.94.2 Attachments: 5549_092.txt, 5549_094.txt, 5549.v10.patch, 5549.v11.patch, 5549.v6.patch, 5549.v7.patch, 5549.v8.patch, 5549.v9.patch, nochange.patch There is a retry mechanism in RecoverableZooKeeper, but when the session expires, the whole ZooKeeperWatcher is recreated, hence the retry mechanism does not work in this case. This is why a sleep is needed in TestZooKeeper#testMasterSessionExpired: we need to wait for ZooKeeperWatcher to be recreated before using the connection. This can happen in real life, it can happen when: - master zookeeper starts - zookeeper connection is cut - master enters the retry loop - in the meantime the session expires - the network comes back, the session is recreated - the retries continues, but on the wrong object, hence fails. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6524) Hooks for hbase tracing
[
https://issues.apache.org/jira/browse/HBASE-6524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13440412#comment-13440412
]
Todd Lipcon commented on HBASE-6524:
{quote}
I think the intent of putting htrace into maven repo is for wider adoption.
I wonder if the above namespace (involving cloudera which should not be an org)
would serve that purpose well.
{quote}
Hey Ted. We use {{org.cloudera}} here to distinguish that this is a fully open
source component, distinct from other software like Cloudera Manager which uses
{{com.cloudera}}. The project is open on github and we fully anticipate that,
if some community springs up around contributions, we'll accept pull requests
and eventually move it to the Apache Incubator. At this point, though, it would
be inappropriate to publish under {{org.apache}}
Consider it the same as a project like Guava which is under Google's namespace
but is still open source.
Hooks for hbase tracing
---
Key: HBASE-6524
URL: https://issues.apache.org/jira/browse/HBASE-6524
Project: HBase
Issue Type: Sub-task
Reporter: Jonathan Leavitt
Fix For: 0.96.0
Attachments: 6524.addendum, createTableTrace.png, hbase-6524.diff
Includes the hooks that use [htrace|http://www.github.com/cloudera/htrace]
library to add dapper-like tracing to hbase.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-6643) Accept encoded region name in compacting/spliting region from shell
Jimmy Xiang created HBASE-6643: -- Summary: Accept encoded region name in compacting/spliting region from shell Key: HBASE-6643 URL: https://issues.apache.org/jira/browse/HBASE-6643 Project: HBase Issue Type: Improvement Components: shell Reporter: Jimmy Xiang Sometimes, the region name has binary characters. When compacting/splitting it from shell, the region name is not recognized. If we can support encoded region name, it will make things easier. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6524) Hooks for hbase tracing
[
https://issues.apache.org/jira/browse/HBASE-6524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13440421#comment-13440421
]
Zhihong Ted Yu commented on HBASE-6524:
---
Thanks Todd for the explanation. I understand the support of open source from
Cloudera.
http://cloudera.org is redirected to cloudera.com
As for Google code, it is under com.google:
{code}
import com.google.common.base.Function;
import com.google.protobuf.ByteString;
{code}
Hooks for hbase tracing
---
Key: HBASE-6524
URL: https://issues.apache.org/jira/browse/HBASE-6524
Project: HBase
Issue Type: Sub-task
Reporter: Jonathan Leavitt
Fix For: 0.96.0
Attachments: 6524.addendum, createTableTrace.png, hbase-6524.diff
Includes the hooks that use [htrace|http://www.github.com/cloudera/htrace]
library to add dapper-like tracing to hbase.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6564) HDFS space is not reclaimed when a column family is deleted
[ https://issues.apache.org/jira/browse/HBASE-6564?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] J Mohamed Zahoor updated HBASE-6564: Attachment: HBASE-6564-v3.patch HDFS space is not reclaimed when a column family is deleted --- Key: HBASE-6564 URL: https://issues.apache.org/jira/browse/HBASE-6564 Project: HBase Issue Type: Bug Components: master Affects Versions: 0.94.1 Reporter: J Mohamed Zahoor Assignee: J Mohamed Zahoor Priority: Minor Attachments: HBASE-6564-trunk.patch, HBASE-6564-v2.patch, HBASE-6564-v3.patch When a column family of a table is deleted, the HDFS space of the column family does not seem to be reclaimed even after a major compaction. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6564) HDFS space is not reclaimed when a column family is deleted
[ https://issues.apache.org/jira/browse/HBASE-6564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13440432#comment-13440432 ] J Mohamed Zahoor commented on HBASE-6564: - Attached a V3 patch incorporating the above comments. HDFS space is not reclaimed when a column family is deleted --- Key: HBASE-6564 URL: https://issues.apache.org/jira/browse/HBASE-6564 Project: HBase Issue Type: Bug Components: master Affects Versions: 0.94.1 Reporter: J Mohamed Zahoor Assignee: J Mohamed Zahoor Priority: Minor Attachments: HBASE-6564-trunk.patch, HBASE-6564-v2.patch, HBASE-6564-v3.patch When a column family of a table is deleted, the HDFS space of the column family does not seem to be reclaimed even after a major compaction. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HBASE-6524) Hooks for hbase tracing
[ https://issues.apache.org/jira/browse/HBASE-6524?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihong Ted Yu reassigned HBASE-6524: - Assignee: Jonathan Leavitt Hooks for hbase tracing --- Key: HBASE-6524 URL: https://issues.apache.org/jira/browse/HBASE-6524 Project: HBase Issue Type: Sub-task Reporter: Jonathan Leavitt Assignee: Jonathan Leavitt Fix For: 0.96.0 Attachments: 6524.addendum, createTableTrace.png, hbase-6524.diff Includes the hooks that use [htrace|http://www.github.com/cloudera/htrace] library to add dapper-like tracing to hbase. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6564) HDFS space is not reclaimed when a column family is deleted
[ https://issues.apache.org/jira/browse/HBASE-6564?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] J Mohamed Zahoor updated HBASE-6564: Attachment: HBASE-6564-v4.patch Corrected some spelling mistakes :-) HDFS space is not reclaimed when a column family is deleted --- Key: HBASE-6564 URL: https://issues.apache.org/jira/browse/HBASE-6564 Project: HBase Issue Type: Bug Components: master Affects Versions: 0.94.1 Reporter: J Mohamed Zahoor Assignee: J Mohamed Zahoor Priority: Minor Attachments: HBASE-6564-trunk.patch, HBASE-6564-v2.patch, HBASE-6564-v3.patch, HBASE-6564-v4.patch When a column family of a table is deleted, the HDFS space of the column family does not seem to be reclaimed even after a major compaction. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-6644) HBaseAdmin.createTable should wait more till table is enabled.
Jimmy Xiang created HBASE-6644: -- Summary: HBaseAdmin.createTable should wait more till table is enabled. Key: HBASE-6644 URL: https://issues.apache.org/jira/browse/HBASE-6644 Project: HBase Issue Type: Improvement Components: client, test Reporter: Jimmy Xiang Assignee: Jimmy Xiang Priority: Minor As mentioned by Jon in HBASE-6576, the issue is not completely fixed. We need to wait longer. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6564) HDFS space is not reclaimed when a column family is deleted
[
https://issues.apache.org/jira/browse/HBASE-6564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13440486#comment-13440486
]
Zhihong Ted Yu commented on HBASE-6564:
---
TestHRegion failed. See
https://builds.apache.org/job/PreCommit-HBASE-Build/2669/console
{code}
+ public void deleteFamily(HRegionInfo region, byte[] familyName)
{code}
Please rename the above method deleteFamilyFromFS().
{code}
+ if (fs.delete(delDir, true) == false) {
+ throw new IOException(Could not delete family
{code}
Indent the throw statement to the right by two spaces.
{code}
+ + Bytes.toString(familyName) + from FileSystem for region
+ + region.getTableNameAsString());
{code}
I think getTableNameAsString() shouldn't be used because you want to tell user
the name of the region.
For TestTableDeleteFamilyHandler.java:
{code}
+ * Copyright 2012 The Apache Software Foundation
{code}
Year is not needed in license header.
HDFS space is not reclaimed when a column family is deleted
---
Key: HBASE-6564
URL: https://issues.apache.org/jira/browse/HBASE-6564
Project: HBase
Issue Type: Bug
Components: master
Affects Versions: 0.94.1
Reporter: J Mohamed Zahoor
Assignee: J Mohamed Zahoor
Priority: Minor
Attachments: HBASE-6564-trunk.patch, HBASE-6564-v2.patch,
HBASE-6564-v3.patch, HBASE-6564-v4.patch
When a column family of a table is deleted, the HDFS space of the column
family does not seem to be reclaimed even after a major compaction.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6644) HBaseAdmin.createTable should wait more till table is enabled.
[ https://issues.apache.org/jira/browse/HBASE-6644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jimmy Xiang updated HBASE-6644: --- Status: Patch Available (was: Open) HBaseAdmin.createTable should wait more till table is enabled. -- Key: HBASE-6644 URL: https://issues.apache.org/jira/browse/HBASE-6644 Project: HBase Issue Type: Improvement Components: client, test Reporter: Jimmy Xiang Assignee: Jimmy Xiang Priority: Minor Attachments: trunk-6644.patch As mentioned by Jon in HBASE-6576, the issue is not completely fixed. We need to wait longer. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6644) HBaseAdmin.createTable should wait more till table is enabled.
[ https://issues.apache.org/jira/browse/HBASE-6644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jimmy Xiang updated HBASE-6644: --- Attachment: trunk-6644.patch HBaseAdmin.createTable should wait more till table is enabled. -- Key: HBASE-6644 URL: https://issues.apache.org/jira/browse/HBASE-6644 Project: HBase Issue Type: Improvement Components: client, test Reporter: Jimmy Xiang Assignee: Jimmy Xiang Priority: Minor Attachments: trunk-6644.patch As mentioned by Jon in HBASE-6576, the issue is not completely fixed. We need to wait longer. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4955) Use the official versions of surefire junit
[ https://issues.apache.org/jira/browse/HBASE-4955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13440491#comment-13440491 ] nkeywal commented on HBASE-4955: 2.12.2 is now available, and there are some activity on surefire, so we could have a 2.13 soon. I tested the snapshot version on HBase trunk, the only issue I found is #800. I proposed a reasonable (imho) fix for it in surefire. It covers the case for forkMode=perthread (i.e. medium large tests). I don't have a non hacky fix for small tests (they are on a different code path in surefire), so we will have either to accept output on the console for small tests either to change the verbose small tests to medium. The former option would likely put us with very simple tests that we should be able to parallelize within a single jvm, so it's reasonable. Use the official versions of surefire junit - Key: HBASE-4955 URL: https://issues.apache.org/jira/browse/HBASE-4955 Project: HBase Issue Type: Improvement Components: test Affects Versions: 0.94.0 Environment: all Reporter: nkeywal Assignee: nkeywal Priority: Minor We currently use private versions for Surefire JUnit since HBASE-4763. This JIRA traks what we need to move to official versions. Surefire 2.11 is just out, but, after some tests, it does not contain all what we need. JUnit. Could be for JUnit 4.11. Issue to monitor: https://github.com/KentBeck/junit/issues/359: fixed in our version, no feedback for an integration on trunk Surefire: Could be for Surefire 2.12. Issues to monitor are: 329 (category support): fixed, we use the official implementation from the trunk 786 (@Category with forkMode=always): fixed, we use the official implementation from the trunk 791 (incorrect elapsed time on test failure): fixed, we use the official implementation from the trunk 793 (incorrect time in the XML report): Not fixed (reopen) on trunk, fixed on our version. 760 (does not take into account the test method): fixed in trunk, not fixed in our version 798 (print immediately the test class name): not fixed in trunk, not fixed in our version 799 (Allow test parallelization when forkMode=always): not fixed in trunk, not fixed in our version 800 (redirectTestOutputToFile not taken into account): not yet fix on trunk, fixed on our version 800 793 are the more important to monitor, it's the only ones that are fixed in our version but not on trunk. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6052) Convert .META. and -ROOT- content to pb
[ https://issues.apache.org/jira/browse/HBASE-6052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13440492#comment-13440492 ] stack commented on HBASE-6052: -- @Enis can you put up something that applies? You might address a few of the comments Ted raises over in RB... they are small... in the next patch you post. Lets get this in. Thanks boss. Convert .META. and -ROOT- content to pb --- Key: HBASE-6052 URL: https://issues.apache.org/jira/browse/HBASE-6052 Project: HBase Issue Type: Sub-task Reporter: stack Assignee: Enis Soztutar Priority: Blocker Fix For: 0.96.0 Attachments: 6052-v5.txt, HBASE-6052_v1.patch, HBASE-6052_v2.patch, HBASE-6052_v3.patch, HBASE-6052_v4.patch, HBASE-6052_v4.patch, HBASE-6052_v7.patch, TestMetaMigrationConvertToPB.tgz -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4955) Use the official versions of surefire junit
[ https://issues.apache.org/jira/browse/HBASE-4955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13440495#comment-13440495 ] stack commented on HBASE-4955: -- Thanks for update. I'd say if 2.13 works even though we have to move a good few tests to medium, lets use it so we can undo our dependency on custom surefire. Good on you N. Use the official versions of surefire junit - Key: HBASE-4955 URL: https://issues.apache.org/jira/browse/HBASE-4955 Project: HBase Issue Type: Improvement Components: test Affects Versions: 0.94.0 Environment: all Reporter: nkeywal Assignee: nkeywal Priority: Minor We currently use private versions for Surefire JUnit since HBASE-4763. This JIRA traks what we need to move to official versions. Surefire 2.11 is just out, but, after some tests, it does not contain all what we need. JUnit. Could be for JUnit 4.11. Issue to monitor: https://github.com/KentBeck/junit/issues/359: fixed in our version, no feedback for an integration on trunk Surefire: Could be for Surefire 2.12. Issues to monitor are: 329 (category support): fixed, we use the official implementation from the trunk 786 (@Category with forkMode=always): fixed, we use the official implementation from the trunk 791 (incorrect elapsed time on test failure): fixed, we use the official implementation from the trunk 793 (incorrect time in the XML report): Not fixed (reopen) on trunk, fixed on our version. 760 (does not take into account the test method): fixed in trunk, not fixed in our version 798 (print immediately the test class name): not fixed in trunk, not fixed in our version 799 (Allow test parallelization when forkMode=always): not fixed in trunk, not fixed in our version 800 (redirectTestOutputToFile not taken into account): not yet fix on trunk, fixed on our version 800 793 are the more important to monitor, it's the only ones that are fixed in our version but not on trunk. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6644) HBaseAdmin.createTable should wait more till table is enabled.
[ https://issues.apache.org/jira/browse/HBASE-6644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13440497#comment-13440497 ] stack commented on HBASE-6644: -- +1 on patch. HBaseAdmin.createTable should wait more till table is enabled. -- Key: HBASE-6644 URL: https://issues.apache.org/jira/browse/HBASE-6644 Project: HBase Issue Type: Improvement Components: client, test Reporter: Jimmy Xiang Assignee: Jimmy Xiang Priority: Minor Attachments: trunk-6644.patch As mentioned by Jon in HBASE-6576, the issue is not completely fixed. We need to wait longer. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-6645) Quarantine invalid reference hfiles.
Jonathan Hsieh created HBASE-6645: - Summary: Quarantine invalid reference hfiles. Key: HBASE-6645 URL: https://issues.apache.org/jira/browse/HBASE-6645 Project: HBase Issue Type: New Feature Reporter: Jonathan Hsieh Similar to the quarantining feature introduced to hbck in HBASE-6586, we've encountered cases of bad reference files. We should add a feature to quarantine reference files if their parents are not present or valid. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6564) HDFS space is not reclaimed when a column family is deleted
[ https://issues.apache.org/jira/browse/HBASE-6564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13440520#comment-13440520 ] Hadoop QA commented on HBASE-6564: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12542154/HBASE-6564-v4.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 7 new or modified tests. +1 hadoop2.0. The patch compiles against the hadoop 2.0 profile. +1 javadoc. The javadoc tool did not generate any warning messages. -1 javac. The applied patch generated 5 javac compiler warnings (more than the trunk's current 4 warnings). -1 findbugs. The patch appears to introduce 7 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hbase.backup.example.TestZooKeeperTableArchiveClient org.apache.hadoop.hbase.regionserver.TestHRegion org.apache.hadoop.hbase.replication.TestReplication org.apache.hadoop.hbase.io.encoding.TestUpgradeFromHFileV1ToEncoding Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/2669//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2669//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2669//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2669//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2669//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2669//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/2669//console This message is automatically generated. HDFS space is not reclaimed when a column family is deleted --- Key: HBASE-6564 URL: https://issues.apache.org/jira/browse/HBASE-6564 Project: HBase Issue Type: Bug Components: master Affects Versions: 0.94.1 Reporter: J Mohamed Zahoor Assignee: J Mohamed Zahoor Priority: Minor Attachments: HBASE-6564-trunk.patch, HBASE-6564-v2.patch, HBASE-6564-v3.patch, HBASE-6564-v4.patch When a column family of a table is deleted, the HDFS space of the column family does not seem to be reclaimed even after a major compaction. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6633) Adding new hooks to the split flow - For roll backs and one final hook after split is completed either successfully or failed
[ https://issues.apache.org/jira/browse/HBASE-6633?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-6633: -- Attachment: HBASE-6633.patch Patch for trunk. Added new preSplit hook that accepts splitRow. Also two hooks for rollback and one postCompleteSplit hook. Pls review and provide your comments. Adding new hooks to the split flow - For roll backs and one final hook after split is completed either successfully or failed - Key: HBASE-6633 URL: https://issues.apache.org/jira/browse/HBASE-6633 Project: HBase Issue Type: Improvement Components: coprocessors Reporter: ramkrishna.s.vasudevan Assignee: ramkrishna.s.vasudevan Labels: coprocessors Attachments: HBASE-6633.patch Currently we have two hooks in the split flow of a region. PreSplit() and postSplit(). But not always these are helpful in case i have a problem in preSplit() or postSplit() i need to do a rollback of the current region or the region that i am handling thro the hooks. So its better if we have a hook in the rollback code and also one final hook say postCompleteSplit() so that CP can take any corrective action. Pls do suggest if i can provide a patch for this. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4216) IllegalArgumentException prefetching from META
[ https://issues.apache.org/jira/browse/HBASE-4216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13440560#comment-13440560 ] Lars Hofhansl commented on HBASE-4216: -- Is this still an issue? If not, let's close it. IllegalArgumentException prefetching from META -- Key: HBASE-4216 URL: https://issues.apache.org/jira/browse/HBASE-4216 Project: HBase Issue Type: Bug Components: client Affects Versions: 0.92.0 Reporter: Todd Lipcon Priority: Critical Received one of these while doing a YCSB test on 26 nodes on trunk: java.io.IOException: java.lang.IllegalArgumentException: hostname can't be null -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4223) Support the ability to return a set of rows using Coprocessors
[ https://issues.apache.org/jira/browse/HBASE-4223?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13440562#comment-13440562 ] Lars Hofhansl commented on HBASE-4223: -- This is specific to the AggregationClient, right? Coprocessor can already intercept Scan request and return custom results (that is how we aggregate at Salesforce) Support the ability to return a set of rows using Coprocessors -- Key: HBASE-4223 URL: https://issues.apache.org/jira/browse/HBASE-4223 Project: HBase Issue Type: Improvement Components: coprocessors Affects Versions: 0.92.0 Reporter: Nichole Treadway Priority: Minor Attachments: HBASE-4223.patch Currently HBase supports returning the results of aggregation operations using coprocessors with the AggregationClient. It would be useful to include a client and implementation which would return a set of rows which match a certain criteria using coprocessors as well. We have a use case in our business process for this. We have an initial implementation of this, which I've attached. The only limitation that we've found is that it cannot be used to return very large sets of rows. If the result set is very large, it would probably require some sort of pagination. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6633) Adding new hooks to the split flow - For roll backs and one final hook after split is completed either successfully or failed
[ https://issues.apache.org/jira/browse/HBASE-6633?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-6633: -- Status: Patch Available (was: Open) Adding new hooks to the split flow - For roll backs and one final hook after split is completed either successfully or failed - Key: HBASE-6633 URL: https://issues.apache.org/jira/browse/HBASE-6633 Project: HBase Issue Type: Improvement Components: coprocessors Reporter: ramkrishna.s.vasudevan Assignee: ramkrishna.s.vasudevan Labels: coprocessors Attachments: HBASE-6633.patch Currently we have two hooks in the split flow of a region. PreSplit() and postSplit(). But not always these are helpful in case i have a problem in preSplit() or postSplit() i need to do a rollback of the current region or the region that i am handling thro the hooks. So its better if we have a hook in the rollback code and also one final hook say postCompleteSplit() so that CP can take any corrective action. Pls do suggest if i can provide a patch for this. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4214) Per-region request counters should be clearer about scope
[ https://issues.apache.org/jira/browse/HBASE-4214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13440564#comment-13440564 ] Lars Hofhansl commented on HBASE-4214: -- This is true for a bunch of other counters as well. Did we tackle this already with Metrics2? Per-region request counters should be clearer about scope - Key: HBASE-4214 URL: https://issues.apache.org/jira/browse/HBASE-4214 Project: HBase Issue Type: Bug Components: metrics, regionserver Affects Versions: 0.92.0 Reporter: Todd Lipcon Fix For: 0.96.0 In testing trunk, I noticed that per-region request counters shown on table.jsp are lifetime-scoped, rather than per-second or some other time range. However, I'm pretty sure they reset when the region is moved. So, it's hard to use them to judge relative hotness of regions from the web UI without hooking it up to something lik OpenTSDB -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4207) Run test suite in parallel, multiple concurrent test instances.
[ https://issues.apache.org/jira/browse/HBASE-4207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13440566#comment-13440566 ] Lars Hofhansl commented on HBASE-4207: -- N did this, no? I think we can close this one. Run test suite in parallel, multiple concurrent test instances. --- Key: HBASE-4207 URL: https://issues.apache.org/jira/browse/HBASE-4207 Project: HBase Issue Type: Task Components: test Reporter: stack Attachments: parallel.build.txt From a suggestion by Lohit up on the list, surefire allows running unit tests in parallel. I'm trying it. I'll attach the patch to do classes in parallel (as opposed to methods) with four threads per core max. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4210) Allow coprocessor to interact with batches per region sent from a client(?)
[
https://issues.apache.org/jira/browse/HBASE-4210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13440570#comment-13440570
]
Lars Hofhansl commented on HBASE-4210:
--
An example is to keep a 2ndary index up to date. In that case we definitely
want to make use of the batching. (and we cannot collect in the single hooks,
since we have no indication about when we're done).
Allow coprocessor to interact with batches per region sent from a client(?)
---
Key: HBASE-4210
URL: https://issues.apache.org/jira/browse/HBASE-4210
Project: HBase
Issue Type: New Feature
Reporter: Lars Hofhansl
Priority: Minor
Currently the coprocessor write hooks - {pre|post}{Put|Delete} - are strictly
one row|cell operations.
It might be a good idea to allow a coprocessor to deal with batches of puts
and deletes as they arrive from the client.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4202) Check filesystem permissions on startup
[ https://issues.apache.org/jira/browse/HBASE-4202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13440573#comment-13440573 ] Lars Hofhansl commented on HBASE-4202: -- @Ram: are you still doing this, or should we close? Check filesystem permissions on startup --- Key: HBASE-4202 URL: https://issues.apache.org/jira/browse/HBASE-4202 Project: HBase Issue Type: Bug Components: regionserver Affects Versions: 0.20.4 Environment: debian squeeze Reporter: Matthias Hofschen Assignee: ramkrishna.s.vasudevan Priority: Trivial Labels: noob We added a new node to a 44 node cluster starting the datanode, mapred and regionserver processes on it. The Unix filesystem was configured incorrectly, i.e. /tmp was not writable to processes. All three processes had issues with this. Datanode and mapred shutdown on exception. Regionserver did not stop, in fact reported to master that its up without regions. So master assigned regions to it. Regionserver would not accept them, resulting in a constant assign, reject, reassign cycle, that put many regions into a state of not being available. There are no logs about this, but we could observer the regioncount fluctuate by hundredths of regions and the application throwing many NotServingRegion exceptions. In fact to the master process the regionserver looked fine, so it was trying to send regions its way. Regionserver rejected them. So the master/balancer was going into a assign/reassign cycle destabilizing the cluster. Many puts and gets simply failed with NotServingRegionExceptions and took a long time to complete. Exception from regionserver: 2011-08-06 23:57:13,953 INFO org.apache.hadoop.hbase.regionserver.HRegionServer: Got ZooKeeper event, state: SyncConnected, type: NodeCreated, path: /hbase/master 2011-08-06 23:57:13,957 INFO org.apache.hadoop.hbase.regionserver.HRegionServer: Telling master at 17.1.0.1:6 that we are up 2011-08-06 23:57:13,957 INFO org.apache.hadoop.hbase.regionserver.HRegionServer: Telling master at 17.1.0.1:6 that we are up 2011-08-07 00:07:39.648::INFO: Logging to STDERR via org.mortbay.log.StdErrLog 2011-08-07 00:07:39.712::INFO: jetty-6.1.14 2011-08-07 00:07:39.742::WARN: tmpdir java.io.IOException: Permission denied at java.io.UnixFileSystem.createFileExclusively(Native Method) at java.io.File.checkAndCreate(File.java:1704) at java.io.File.createTempFile(File.java:1792) at java.io.File.createTempFile(File.java:1828) at org.mortbay.jetty.webapp.WebAppContext.getTempDirectory(WebAppContext.java:745) at org.mortbay.jetty.webapp.WebAppContext.doStart(WebAppContext.java:458) at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50) at org.mortbay.jetty.handler.HandlerCollection.doStart(HandlerCollection.java:152) at org.mortbay.jetty.handler.ContextHandlerCollection.doStart(ContextHandlerCollection.java:156) at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50) at org.mortbay.jetty.handler.HandlerWrapper.doStart(HandlerWrapper.java:130) at org.mortbay.jetty.Server.doStart(Server.java:222) at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50) at org.apache.hadoop.http.HttpServer.start(HttpServer.java:461) at org.apache.hadoop.hbase.regionserver.HRegionServer.startServiceThreads(HRegionServer.java:1168) at org.apache.hadoop.hbase.regionserver.HRegionServer.init(HRegionServer.java:792) at org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:430) at java.lang.Thread.run(Thread.java:619) Exception from datanode: 2011-08-06 23:37:20,444 INFO org.apache.hadoop.http.HttpServer: Jetty bound to port 50075 2011-08-06 23:37:20,444 INFO org.mortbay.log: jetty-6.1.14 2011-08-06 23:37:20,469 WARN org.mortbay.log: tmpdir java.io.IOException: Permission denied at java.io.UnixFileSystem.createFileExclusively(Native Method) at java.io.File.checkAndCreate(File.java:1704) at java.io.File.createTempFile(File.java:1792) at java.io.File.createTempFile(File.java:1828) at org.mortbay.jetty.webapp.WebAppContext.getTempDirectory(WebAppContext.java:745) at org.mortbay.jetty.webapp.WebAppContext.doStart(WebAppContext.java:458) at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50) at org.mortbay.jetty.handler.HandlerCollection.doStart(HandlerCollection.java:152) at org.mortbay.jetty.handler.ContextHandlerCollection.doStart(ContextHandlerCollection.java:156) at
[jira] [Resolved] (HBASE-4200) blockCache summary - web UI
[ https://issues.apache.org/jira/browse/HBASE-4200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl resolved HBASE-4200. -- Resolution: Duplicate blockCache summary - web UI --- Key: HBASE-4200 URL: https://issues.apache.org/jira/browse/HBASE-4200 Project: HBase Issue Type: Sub-task Reporter: Doug Meil Priority: Minor Web UI for block cache summary report. This will most likely be a new web-page linked from the RegionServer detail page. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4183) FSUtils checkFileSystem() should not close the FileSystem by default
[ https://issues.apache.org/jira/browse/HBASE-4183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13440577#comment-13440577 ] Lars Hofhansl commented on HBASE-4183: -- So is this a problem? FSUtils checkFileSystem() should not close the FileSystem by default Key: HBASE-4183 URL: https://issues.apache.org/jira/browse/HBASE-4183 Project: HBase Issue Type: Bug Reporter: Pritam Damania The checkFileSystem() function in FSUtils closes down the FileSystem for the HRegionServer by default if the FileSystem is not available. Ideally we should let the the HRegionServer threads exit and then shutdown the FileSystem. The checkFileSystem() function should not by default kill the FileSystem. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4177) Handling read failures during recovery - when HMaster calls Namenode recovery, recovery may be a failure leading to read failure while splitting logs
[ https://issues.apache.org/jira/browse/HBASE-4177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13440578#comment-13440578 ] Lars Hofhansl commented on HBASE-4177: -- That is superseded by all of N's work, correct? Handling read failures during recovery - when HMaster calls Namenode recovery, recovery may be a failure leading to read failure while splitting logs -- Key: HBASE-4177 URL: https://issues.apache.org/jira/browse/HBASE-4177 Project: HBase Issue Type: Bug Components: master Reporter: ramkrishna.s.vasudevan Assignee: ramkrishna.s.vasudevan Priority: Critical As per the mailing thread with the heading 'Handling read failures during recovery' we found this problem. As part of split Logs the HMaster calls Namenode recovery. The recovery is an asynchronous process. In HDFS === Even though client is getting the updated block info from Namenode on first read failure, client is discarding the new info and using the old info only to retrieve the data from datanode. So, all the read retries are failing. [Method parameter reassignment - Not reflected in caller]. In HBASE === In HMaster code we tend to wait for 1sec. But if the recovery had some failure then split log may not happen and may lead to dataloss. So may be we need to decide upon the actual delay that needs to be introduced once Hmaster calls NN recovery. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4155) the problem in hbase thrift client when scan/get rows by timestamp
[
https://issues.apache.org/jira/browse/HBASE-4155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13440580#comment-13440580
]
Lars Hofhansl commented on HBASE-4155:
--
Where are we with this? Can we close?
the problem in hbase thrift client when scan/get rows by timestamp
--
Key: HBASE-4155
URL: https://issues.apache.org/jira/browse/HBASE-4155
Project: HBase
Issue Type: Bug
Components: thrift
Affects Versions: 0.90.0
Reporter: zezhou
Attachments: 4155.txt, patch.txt, patch.txt.svn
Original Estimate: 1m
Remaining Estimate: 1m
I want to scan rows by specified timestamp. I use following hbase shell
command :
scan 'testcrawl',{TIMESTAMP=1312268202071}
ROW COLUMN+CELL
put1.com column=crawl:data,
timestamp=1312268202071, value=htmlput1/html
put1.com column=crawl:type,
timestamp=1312268202071, value=html
put1.com column=links:outlinks,
timestamp=1312268202071, value=www.163.com;www.sina.com
As I expected, I can get the rows which timestamp is 1312268202071.
But when I use thift client to do the same thing ,the return data is the rows
which time before specified timestamp , not the same as hbase
shell.following is timestamp of return data:
131217917
1312268202059
I look up the source in
hbase/src/main/java/org/apache/hadoop/hbase/thrift/ThriftServer.java, it use
following code to set time parameter .
scan.setTimeRange(Long.MIN_VALUE, timestamp);
This cause thrift client return rows before specified row ,not the rows
timestamp specified.
But in hbase client and avro client ，it use following code to set time
parameter.
scan.setTimeStamp(timestamp);
this will return rows timestamp specified.
Is this a feature or a bug in thrift client ?
if this is a feature, which method in thrift client can get the rows by
specified timestamp?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6644) HBaseAdmin.createTable should wait more till table is enabled.
[ https://issues.apache.org/jira/browse/HBASE-6644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13440591#comment-13440591 ] Hadoop QA commented on HBASE-6644: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12542162/trunk-6644.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 hadoop2.0. The patch compiles against the hadoop 2.0 profile. +1 javadoc. The javadoc tool did not generate any warning messages. -1 javac. The applied patch generated 5 javac compiler warnings (more than the trunk's current 4 warnings). -1 findbugs. The patch appears to introduce 7 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hbase.io.encoding.TestUpgradeFromHFileV1ToEncoding Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/2670//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2670//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2670//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2670//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2670//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2670//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/2670//console This message is automatically generated. HBaseAdmin.createTable should wait more till table is enabled. -- Key: HBASE-6644 URL: https://issues.apache.org/jira/browse/HBASE-6644 Project: HBase Issue Type: Improvement Components: client, test Reporter: Jimmy Xiang Assignee: Jimmy Xiang Priority: Minor Attachments: trunk-6644.patch As mentioned by Jon in HBASE-6576, the issue is not completely fixed. We need to wait longer. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4210) Allow coprocessor to interact with batches per region sent from a client(?)
[
https://issues.apache.org/jira/browse/HBASE-4210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13440593#comment-13440593
]
Andrew Purtell commented on HBASE-4210:
---
+1
Allow coprocessor to interact with batches per region sent from a client(?)
---
Key: HBASE-4210
URL: https://issues.apache.org/jira/browse/HBASE-4210
Project: HBase
Issue Type: New Feature
Reporter: Lars Hofhansl
Priority: Minor
Currently the coprocessor write hooks - {pre|post}{Put|Delete} - are strictly
one row|cell operations.
It might be a good idea to allow a coprocessor to deal with batches of puts
and deletes as they arrive from the client.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6606) Test for reconnecting with HBaseAdmin using unmanaged HConnection
[ https://issues.apache.org/jira/browse/HBASE-6606?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gregory Chanan updated HBASE-6606: -- Attachment: HBASE-6606-v2.patch New version of patch, old patch was reversed. Test for reconnecting with HBaseAdmin using unmanaged HConnection - Key: HBASE-6606 URL: https://issues.apache.org/jira/browse/HBASE-6606 Project: HBase Issue Type: Task Components: client, test Reporter: Gregory Chanan Assignee: Gregory Chanan Priority: Minor Fix For: 0.96.0 Attachments: HBASE-6606.patch, HBASE-6606-v2.patch HBASE-5058 allowed HBaseAdmin to work with an existing and unmanaged HConnection. The retry semantics of managed vs unmanaged connections are different. From the JIRA: For an HConnection that is passed from the outside, it has to be possible to try again. So if the HConnection is managed we retain the old behavior (i.e. only try once, give up after that, even if that failed). For an unmanaged connection we try again unless we actually found a master. . I couldn't find any test of this behavior, only that the HBaseAdmin works with an unmanaged connection, i.e. no retry testing. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-6646) Upgrade to 0.96 section in the book
Enis Soztutar created HBASE-6646: Summary: Upgrade to 0.96 section in the book Key: HBASE-6646 URL: https://issues.apache.org/jira/browse/HBASE-6646 Project: HBase Issue Type: Improvement Components: documentation Affects Versions: 0.96.0 Reporter: Enis Soztutar Priority: Blocker We should have an upgrade section in the book for 0.96. Raising this as blocker. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6524) Hooks for hbase tracing
[
https://issues.apache.org/jira/browse/HBASE-6524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13440631#comment-13440631
]
Jonathan Leavitt commented on HBASE-6524:
-
I'm running the test suite with hadoop 2.0 profile
{code}mvn clean -Dhadoop.profile=2.0 -P localTests test {code}
with and without patch. So far, the same test failures are affecting both.
Will update again once the whole suite is done running.
Hooks for hbase tracing
---
Key: HBASE-6524
URL: https://issues.apache.org/jira/browse/HBASE-6524
Project: HBase
Issue Type: Sub-task
Reporter: Jonathan Leavitt
Assignee: Jonathan Leavitt
Fix For: 0.96.0
Attachments: 6524.addendum, createTableTrace.png, hbase-6524.diff
Includes the hooks that use [htrace|http://www.github.com/cloudera/htrace]
library to add dapper-like tracing to hbase.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6644) HBaseAdmin.createTable should wait more till table is enabled.
[ https://issues.apache.org/jira/browse/HBASE-6644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jimmy Xiang updated HBASE-6644: --- Resolution: Fixed Fix Version/s: 0.94.2 0.96.0 0.92.2 Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Integrated into 0.92, 0,94 and trunk. Thanks Stack for the reviewing. HBaseAdmin.createTable should wait more till table is enabled. -- Key: HBASE-6644 URL: https://issues.apache.org/jira/browse/HBASE-6644 Project: HBase Issue Type: Improvement Components: client, test Reporter: Jimmy Xiang Assignee: Jimmy Xiang Priority: Minor Fix For: 0.92.2, 0.96.0, 0.94.2 Attachments: trunk-6644.patch As mentioned by Jon in HBASE-6576, the issue is not completely fixed. We need to wait longer. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6052) Convert .META. and -ROOT- content to pb
[ https://issues.apache.org/jira/browse/HBASE-6052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Enis Soztutar updated HBASE-6052: - Release Note: Changed .META. and ROOT tables' content to be Protobuf serialization of HRegionInfo's instead of Writable serialization. When upgrading from 0.92 and 0.94 clusters, after master deploys META and ROOT, and before it deploys user level tables, it does a migration for the data in the catalog files, rewriting them in the new format. Before upgrading, you can take an export of the catalog tables using HBASE-3271 for backup. Convert .META. and -ROOT- content to pb --- Key: HBASE-6052 URL: https://issues.apache.org/jira/browse/HBASE-6052 Project: HBase Issue Type: Sub-task Reporter: stack Assignee: Enis Soztutar Priority: Blocker Fix For: 0.96.0 Attachments: 6052-v5.txt, HBASE-6052_v1.patch, HBASE-6052_v2.patch, HBASE-6052_v3.patch, HBASE-6052_v4.patch, HBASE-6052_v4.patch, HBASE-6052_v7.patch, TestMetaMigrationConvertToPB.tgz -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6633) Adding new hooks to the split flow - For roll backs and one final hook after split is completed either successfully or failed
[ https://issues.apache.org/jira/browse/HBASE-6633?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13440645#comment-13440645 ] Hadoop QA commented on HBASE-6633: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12542171/HBASE-6633.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 9 new or modified tests. +1 hadoop2.0. The patch compiles against the hadoop 2.0 profile. +1 javadoc. The javadoc tool did not generate any warning messages. -1 javac. The applied patch generated 5 javac compiler warnings (more than the trunk's current 4 warnings). -1 findbugs. The patch appears to introduce 7 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hbase.io.encoding.TestUpgradeFromHFileV1ToEncoding Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/2671//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2671//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2671//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2671//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2671//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/2671//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/2671//console This message is automatically generated. Adding new hooks to the split flow - For roll backs and one final hook after split is completed either successfully or failed - Key: HBASE-6633 URL: https://issues.apache.org/jira/browse/HBASE-6633 Project: HBase Issue Type: Improvement Components: coprocessors Reporter: ramkrishna.s.vasudevan Assignee: ramkrishna.s.vasudevan Labels: coprocessors Attachments: HBASE-6633.patch Currently we have two hooks in the split flow of a region. PreSplit() and postSplit(). But not always these are helpful in case i have a problem in preSplit() or postSplit() i need to do a rollback of the current region or the region that i am handling thro the hooks. So its better if we have a hook in the rollback code and also one final hook say postCompleteSplit() so that CP can take any corrective action. Pls do suggest if i can provide a patch for this. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HBASE-4207) Run test suite in parallel, multiple concurrent test instances.
[ https://issues.apache.org/jira/browse/HBASE-4207?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] nkeywal resolved HBASE-4207. Resolution: Duplicate @lars Yep. There are still stuff to do in this area, but it better to have new jiras. Run test suite in parallel, multiple concurrent test instances. --- Key: HBASE-4207 URL: https://issues.apache.org/jira/browse/HBASE-4207 Project: HBase Issue Type: Task Components: test Reporter: stack Attachments: parallel.build.txt From a suggestion by Lohit up on the list, surefire allows running unit tests in parallel. I'm trying it. I'll attach the patch to do classes in parallel (as opposed to methods) with four threads per core max. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4177) Handling read failures during recovery - when HMaster calls Namenode recovery, recovery may be a failure leading to read failure while splitting logs
[ https://issues.apache.org/jira/browse/HBASE-4177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13440652#comment-13440652 ] nkeywal commented on HBASE-4177: hum, it's really closed to what I've done, but this problem may still be there. Ram, what do you think? If you don't have the time, I can give it a try. Handling read failures during recovery - when HMaster calls Namenode recovery, recovery may be a failure leading to read failure while splitting logs -- Key: HBASE-4177 URL: https://issues.apache.org/jira/browse/HBASE-4177 Project: HBase Issue Type: Bug Components: master Reporter: ramkrishna.s.vasudevan Assignee: ramkrishna.s.vasudevan Priority: Critical As per the mailing thread with the heading 'Handling read failures during recovery' we found this problem. As part of split Logs the HMaster calls Namenode recovery. The recovery is an asynchronous process. In HDFS === Even though client is getting the updated block info from Namenode on first read failure, client is discarding the new info and using the old info only to retrieve the data from datanode. So, all the read retries are failing. [Method parameter reassignment - Not reflected in caller]. In HBASE === In HMaster code we tend to wait for 1sec. But if the recovery had some failure then split log may not happen and may lead to dataloss. So may be we need to decide upon the actual delay that needs to be introduced once Hmaster calls NN recovery. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6524) Hooks for hbase tracing
[
https://issues.apache.org/jira/browse/HBASE-6524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13440664#comment-13440664
]
Jonathan Leavitt commented on HBASE-6524:
-
Update after running
{code}mvn clean -Dhadoop.profile=2.0 -P localTests test{code}
with and without the patch. Many of the failed tests fail on both, and there
are some failures unique to each. I would bet the results would be different if
I ran the tests again. I believe that some of the tests are not completely
stable for hadoop 2.0.
Hooks for hbase tracing
---
Key: HBASE-6524
URL: https://issues.apache.org/jira/browse/HBASE-6524
Project: HBase
Issue Type: Sub-task
Reporter: Jonathan Leavitt
Assignee: Jonathan Leavitt
Fix For: 0.96.0
Attachments: 6524.addendum, createTableTrace.png, hbase-6524.diff
Includes the hooks that use [htrace|http://www.github.com/cloudera/htrace]
library to add dapper-like tracing to hbase.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6612) Hbase command line improvements
[ https://issues.apache.org/jira/browse/HBASE-6612?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13440673#comment-13440673 ] Ionut Ignatescu commented on HBASE-6612: I think it not about adding data types, it's more about adding SerDes support. Behind a common model(interface), it should be support for any standard data type: long, double, strings with different encoding, etc. Also, in this way we could add support for any custom model(Ex: a table with a composed key, where firsts 4 bytes are an integer value, next eight are a long representing a date and the rest are a result of a hash function). In this way is much more easier to perform operations using shell. Hbase command line improvements --- Key: HBASE-6612 URL: https://issues.apache.org/jira/browse/HBASE-6612 Project: HBase Issue Type: New Feature Components: scripts, shell Affects Versions: 0.94.1 Reporter: Ionut Ignatescu Priority: Minor Currently, if the row key or any column value is something different than a string, when a scan is performed via command line, the value extracted are not decoded to a human-readable format. It would be nice to have support to some standard data types(long,double,etc..) or to specify some custom decoders(this would be extremely useful for tables having composed keys). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6524) Hooks for hbase tracing
[ https://issues.apache.org/jira/browse/HBASE-6524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13440680#comment-13440680 ] stack commented on HBASE-6524: -- That could indeed be the case. If you don't mind Jonathan, do some more runs. If the set of flakey tests goes roughly unchanged when this patch is added, we can put it back and work on the flakeyness in other JIRAs. Hooks for hbase tracing --- Key: HBASE-6524 URL: https://issues.apache.org/jira/browse/HBASE-6524 Project: HBase Issue Type: Sub-task Reporter: Jonathan Leavitt Assignee: Jonathan Leavitt Fix For: 0.96.0 Attachments: 6524.addendum, createTableTrace.png, hbase-6524.diff Includes the hooks that use [htrace|http://www.github.com/cloudera/htrace] library to add dapper-like tracing to hbase. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6524) Hooks for hbase tracing
[
https://issues.apache.org/jira/browse/HBASE-6524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13440687#comment-13440687
]
Zhihong Ted Yu commented on HBASE-6524:
---
I received the following from Jonathan:
{code}
with htrace:
Failed tests: testWALPlayer(org.apache.hadoop.hbase.mapreduce.TestWALPlayer):
expected:0 but was:1
testMultiRegionTable(org.apache.hadoop.hbase.mapreduce.TestTableMapReduce)
testMROnTableWithTimestamp(org.apache.hadoop.hbase.mapreduce.TestImportTsv)
testMROnTableWithCustomMapper(org.apache.hadoop.hbase.mapreduce.TestImportTsv)
testRowCounterExclusiveColumn(org.apache.hadoop.hbase.mapreduce.TestRowCounter)
testRowCounterHiddenColumn(org.apache.hadoop.hbase.mapreduce.TestRowCounter)
Tests in error:
testMRIncrementalLoad(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat)
testMRIncrementalLoadWithSplit(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat)
testExcludeMinorCompaction(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat)
testMROnTable(org.apache.hadoop.hbase.mapreduce.TestImportTsv)
testBulkOutputWithoutAnExistingTable(org.apache.hadoop.hbase.mapreduce.TestImportTsv)
testRowCounterNoColumn(org.apache.hadoop.hbase.mapreduce.TestRowCounter)
testMultithreadedTableMapper(org.apache.hadoop.hbase.mapreduce.TestMultithreadedTableMapper)
testSimpleCase(org.apache.hadoop.hbase.mapreduce.TestImportExport)
testMetaExport(org.apache.hadoop.hbase.mapreduce.TestImportExport)
testWithDeletes(org.apache.hadoop.hbase.mapreduce.TestImportExport)
testScanOBBToOPP(org.apache.hadoop.hbase.mapreduce.TestTableInputFormatScan)
testGetRowVersions(org.apache.hadoop.hbase.TestMultiVersions): Shutting down
testScanMultipleVersions(org.apache.hadoop.hbase.TestMultiVersions):
org.apache.hadoop.hbase.MasterNotRunningException: Can create a proxy to
master, but it is not running
testMultiRegionTable(org.apache.hadoop.hbase.mapred.TestTableMapReduce): Job
failed!
testUpgrade(org.apache.hadoop.hbase.io.encoding.TestUpgradeFromHFileV1ToEncoding):
Shutting down
{code}
As you can see, the count was much higher than 3
Please refer to
https://builds.apache.org/view/G-L/view/HBase/job/HBase-TRUNK-on-Hadoop-2.0.0/143/testReport/
Hooks for hbase tracing
---
Key: HBASE-6524
URL: https://issues.apache.org/jira/browse/HBASE-6524
Project: HBase
Issue Type: Sub-task
Reporter: Jonathan Leavitt
Assignee: Jonathan Leavitt
Fix For: 0.96.0
Attachments: 6524.addendum, createTableTrace.png, hbase-6524.diff
Includes the hooks that use [htrace|http://www.github.com/cloudera/htrace]
library to add dapper-like tracing to hbase.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6633) Adding new hooks to the split flow - For roll backs and one final hook after split is completed either successfully or failed
[ https://issues.apache.org/jira/browse/HBASE-6633?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13440688#comment-13440688 ] stack commented on HBASE-6633: -- +1 on commit Adding new hooks to the split flow - For roll backs and one final hook after split is completed either successfully or failed - Key: HBASE-6633 URL: https://issues.apache.org/jira/browse/HBASE-6633 Project: HBase Issue Type: Improvement Components: coprocessors Reporter: ramkrishna.s.vasudevan Assignee: ramkrishna.s.vasudevan Labels: coprocessors Attachments: HBASE-6633.patch Currently we have two hooks in the split flow of a region. PreSplit() and postSplit(). But not always these are helpful in case i have a problem in preSplit() or postSplit() i need to do a rollback of the current region or the region that i am handling thro the hooks. So its better if we have a hook in the rollback code and also one final hook say postCompleteSplit() so that CP can take any corrective action. Pls do suggest if i can provide a patch for this. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6646) Upgrade to 0.96 section in the book
[ https://issues.apache.org/jira/browse/HBASE-6646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13440691#comment-13440691 ] stack commented on HBASE-6646: -- We have an upgrade chapter in the refguide currently: http://hbase.apache.org/book.html#upgrading +1 on section on going to 0.96 Upgrade to 0.96 section in the book --- Key: HBASE-6646 URL: https://issues.apache.org/jira/browse/HBASE-6646 Project: HBase Issue Type: Improvement Components: documentation Affects Versions: 0.96.0 Reporter: Enis Soztutar Priority: Blocker We should have an upgrade section in the book for 0.96. Raising this as blocker. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
