[jira] [Commented] (HBASE-4729) Race between online altering and splitting kills the master
[
https://issues.apache.org/jira/browse/HBASE-4729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13159143#comment-13159143
]
Hadoop QA commented on HBASE-4729:
--
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12505460/4729-v4.txt
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 2 new or modified tests.
-1 javadoc. The javadoc tool appears to have generated -162 warning
messages.
+1 javac. The applied patch does not increase the total number of javac
compiler warnings.
-1 findbugs. The patch appears to introduce 67 new Findbugs (version
1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of
release audit warnings.
-1 core tests. The patch failed these unit tests:
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/402//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/402//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/402//console
This message is automatically generated.
> Race between online altering and splitting kills the master
> ---
>
> Key: HBASE-4729
> URL: https://issues.apache.org/jira/browse/HBASE-4729
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.92.0
>Reporter: Jean-Daniel Cryans
>Assignee: ramkrishna.s.vasudevan
>Priority: Critical
> Fix For: 0.92.0, 0.94.0
>
> Attachments: 4729-v2.txt, 4729-v3.txt, 4729-v4.txt, 4729.txt
>
>
> I was running an online alter while regions were splitting, and suddenly the
> master died and left my table half-altered (haven't restarted the master yet).
> What killed the master:
> {quote}
> 2011-11-02 17:06:44,428 FATAL org.apache.hadoop.hbase.master.HMaster:
> Unexpected ZK exception creating node CLOSING
> org.apache.zookeeper.KeeperException$NodeExistsException: KeeperErrorCode =
> NodeExists for /hbase/unassigned/f7e1783e65ea8d621a4bc96ad310f101
> at
> org.apache.zookeeper.KeeperException.create(KeeperException.java:110)
> at
> org.apache.zookeeper.KeeperException.create(KeeperException.java:42)
> at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:637)
> at
> org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.createNonSequential(RecoverableZooKeeper.java:459)
> at
> org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.create(RecoverableZooKeeper.java:441)
> at
> org.apache.hadoop.hbase.zookeeper.ZKUtil.createAndWatch(ZKUtil.java:769)
> at
> org.apache.hadoop.hbase.zookeeper.ZKAssign.createNodeClosing(ZKAssign.java:568)
> at
> org.apache.hadoop.hbase.master.AssignmentManager.unassign(AssignmentManager.java:1722)
> at
> org.apache.hadoop.hbase.master.AssignmentManager.unassign(AssignmentManager.java:1661)
> at org.apache.hadoop.hbase.master.BulkReOpen$1.run(BulkReOpen.java:69)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:662)
> {quote}
> A znode was created because the region server was splitting the region 4
> seconds before:
> {quote}
> 2011-11-02 17:06:40,704 INFO
> org.apache.hadoop.hbase.regionserver.SplitTransaction: Starting split of
> region TestTable,0012469153,1320253135043.f7e1783e65ea8d621a4bc96ad310f101.
> 2011-11-02 17:06:40,704 DEBUG
> org.apache.hadoop.hbase.regionserver.SplitTransaction:
> regionserver:62023-0x132f043bbde0710 Creating ephemeral node for
> f7e1783e65ea8d621a4bc96ad310f101 in SPLITTING state
> 2011-11-02 17:06:40,751 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign:
> regionserver:62023-0x132f043bbde0710 Attempting to transition node
> f7e1783e65ea8d621a4bc96ad310f101 from RS_ZK_REGION_SPLITTING to
> RS_ZK_REGION_SPLITTING
> ...
> 2011-11-02 17:06:44,061 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign:
> regionserver:62023-0x132f043bbde0710 Successfully transitioned node
> f7e1783e65ea8d621a4bc96ad310f101 from RS_ZK_REGION_SPLITTING to
> RS_ZK_REGION_SPLIT
> 2011-11-02 17:06:44,061 INFO
> org.apache.hadoop.hbase.regionserver.SplitTransaction: Still waiting on the
> master to process the split for f7e1783e65ea8d621a4bc96ad310f101
> {quote}
> Now that the master is dead the region server is spewing those last two lines
> like mad.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.o
[jira] [Commented] (HBASE-4729) Race between online altering and splitting kills the master
[
https://issues.apache.org/jira/browse/HBASE-4729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13159138#comment-13159138
]
stack commented on HBASE-4729:
--
I'll make new JIRA for TODO on commit and for discussing 'How about introducing
a new state, such as RS_ZK_PARENT_REGION_CLOSE_THRU_SPLIT, so that rolling back
failed split can decide what to do ?'
I think this the last issue for 0.92 -- or at least, once this goes in I'm
going to RC now I can do a build w/o tests failing.
> Race between online altering and splitting kills the master
> ---
>
> Key: HBASE-4729
> URL: https://issues.apache.org/jira/browse/HBASE-4729
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.92.0
>Reporter: Jean-Daniel Cryans
>Assignee: ramkrishna.s.vasudevan
>Priority: Critical
> Fix For: 0.92.0, 0.94.0
>
> Attachments: 4729-v2.txt, 4729-v3.txt, 4729-v4.txt, 4729.txt
>
>
> I was running an online alter while regions were splitting, and suddenly the
> master died and left my table half-altered (haven't restarted the master yet).
> What killed the master:
> {quote}
> 2011-11-02 17:06:44,428 FATAL org.apache.hadoop.hbase.master.HMaster:
> Unexpected ZK exception creating node CLOSING
> org.apache.zookeeper.KeeperException$NodeExistsException: KeeperErrorCode =
> NodeExists for /hbase/unassigned/f7e1783e65ea8d621a4bc96ad310f101
> at
> org.apache.zookeeper.KeeperException.create(KeeperException.java:110)
> at
> org.apache.zookeeper.KeeperException.create(KeeperException.java:42)
> at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:637)
> at
> org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.createNonSequential(RecoverableZooKeeper.java:459)
> at
> org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.create(RecoverableZooKeeper.java:441)
> at
> org.apache.hadoop.hbase.zookeeper.ZKUtil.createAndWatch(ZKUtil.java:769)
> at
> org.apache.hadoop.hbase.zookeeper.ZKAssign.createNodeClosing(ZKAssign.java:568)
> at
> org.apache.hadoop.hbase.master.AssignmentManager.unassign(AssignmentManager.java:1722)
> at
> org.apache.hadoop.hbase.master.AssignmentManager.unassign(AssignmentManager.java:1661)
> at org.apache.hadoop.hbase.master.BulkReOpen$1.run(BulkReOpen.java:69)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:662)
> {quote}
> A znode was created because the region server was splitting the region 4
> seconds before:
> {quote}
> 2011-11-02 17:06:40,704 INFO
> org.apache.hadoop.hbase.regionserver.SplitTransaction: Starting split of
> region TestTable,0012469153,1320253135043.f7e1783e65ea8d621a4bc96ad310f101.
> 2011-11-02 17:06:40,704 DEBUG
> org.apache.hadoop.hbase.regionserver.SplitTransaction:
> regionserver:62023-0x132f043bbde0710 Creating ephemeral node for
> f7e1783e65ea8d621a4bc96ad310f101 in SPLITTING state
> 2011-11-02 17:06:40,751 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign:
> regionserver:62023-0x132f043bbde0710 Attempting to transition node
> f7e1783e65ea8d621a4bc96ad310f101 from RS_ZK_REGION_SPLITTING to
> RS_ZK_REGION_SPLITTING
> ...
> 2011-11-02 17:06:44,061 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign:
> regionserver:62023-0x132f043bbde0710 Successfully transitioned node
> f7e1783e65ea8d621a4bc96ad310f101 from RS_ZK_REGION_SPLITTING to
> RS_ZK_REGION_SPLIT
> 2011-11-02 17:06:44,061 INFO
> org.apache.hadoop.hbase.regionserver.SplitTransaction: Still waiting on the
> master to process the split for f7e1783e65ea8d621a4bc96ad310f101
> {quote}
> Now that the master is dead the region server is spewing those last two lines
> like mad.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4729) Race between online altering and splitting kills the master
[
https://issues.apache.org/jira/browse/HBASE-4729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-4729:
-
Attachment: 4729-v4.txt
Address Ted comments (I like the one where we need to verify NoNodeException).
> Race between online altering and splitting kills the master
> ---
>
> Key: HBASE-4729
> URL: https://issues.apache.org/jira/browse/HBASE-4729
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.92.0
>Reporter: Jean-Daniel Cryans
>Assignee: ramkrishna.s.vasudevan
>Priority: Critical
> Fix For: 0.92.0, 0.94.0
>
> Attachments: 4729-v2.txt, 4729-v3.txt, 4729-v4.txt, 4729.txt
>
>
> I was running an online alter while regions were splitting, and suddenly the
> master died and left my table half-altered (haven't restarted the master yet).
> What killed the master:
> {quote}
> 2011-11-02 17:06:44,428 FATAL org.apache.hadoop.hbase.master.HMaster:
> Unexpected ZK exception creating node CLOSING
> org.apache.zookeeper.KeeperException$NodeExistsException: KeeperErrorCode =
> NodeExists for /hbase/unassigned/f7e1783e65ea8d621a4bc96ad310f101
> at
> org.apache.zookeeper.KeeperException.create(KeeperException.java:110)
> at
> org.apache.zookeeper.KeeperException.create(KeeperException.java:42)
> at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:637)
> at
> org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.createNonSequential(RecoverableZooKeeper.java:459)
> at
> org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.create(RecoverableZooKeeper.java:441)
> at
> org.apache.hadoop.hbase.zookeeper.ZKUtil.createAndWatch(ZKUtil.java:769)
> at
> org.apache.hadoop.hbase.zookeeper.ZKAssign.createNodeClosing(ZKAssign.java:568)
> at
> org.apache.hadoop.hbase.master.AssignmentManager.unassign(AssignmentManager.java:1722)
> at
> org.apache.hadoop.hbase.master.AssignmentManager.unassign(AssignmentManager.java:1661)
> at org.apache.hadoop.hbase.master.BulkReOpen$1.run(BulkReOpen.java:69)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:662)
> {quote}
> A znode was created because the region server was splitting the region 4
> seconds before:
> {quote}
> 2011-11-02 17:06:40,704 INFO
> org.apache.hadoop.hbase.regionserver.SplitTransaction: Starting split of
> region TestTable,0012469153,1320253135043.f7e1783e65ea8d621a4bc96ad310f101.
> 2011-11-02 17:06:40,704 DEBUG
> org.apache.hadoop.hbase.regionserver.SplitTransaction:
> regionserver:62023-0x132f043bbde0710 Creating ephemeral node for
> f7e1783e65ea8d621a4bc96ad310f101 in SPLITTING state
> 2011-11-02 17:06:40,751 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign:
> regionserver:62023-0x132f043bbde0710 Attempting to transition node
> f7e1783e65ea8d621a4bc96ad310f101 from RS_ZK_REGION_SPLITTING to
> RS_ZK_REGION_SPLITTING
> ...
> 2011-11-02 17:06:44,061 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign:
> regionserver:62023-0x132f043bbde0710 Successfully transitioned node
> f7e1783e65ea8d621a4bc96ad310f101 from RS_ZK_REGION_SPLITTING to
> RS_ZK_REGION_SPLIT
> 2011-11-02 17:06:44,061 INFO
> org.apache.hadoop.hbase.regionserver.SplitTransaction: Still waiting on the
> master to process the split for f7e1783e65ea8d621a4bc96ad310f101
> {quote}
> Now that the master is dead the region server is spewing those last two lines
> like mad.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4773) HBaseAdmin may leak ZooKeeper connections
[
https://issues.apache.org/jira/browse/HBASE-4773?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13159130#comment-13159130
]
Hudson commented on HBASE-4773:
---
Integrated in HBase-TRUNK-security #13 (See
[https://builds.apache.org/job/HBase-TRUNK-security/13/])
HBASE-4773 HBaseAdmin may leak ZooKeeper connections (Xufeng)
tedyu :
Files :
* /hbase/trunk/CHANGES.txt
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java
> HBaseAdmin may leak ZooKeeper connections
> -
>
> Key: HBASE-4773
> URL: https://issues.apache.org/jira/browse/HBASE-4773
> Project: HBase
> Issue Type: Bug
> Components: client
>Affects Versions: 0.90.4
>Reporter: gaojinchao
>Assignee: xufeng
>Priority: Critical
> Fix For: 0.90.5
>
> Attachments: 4773.patch, branches_4773.patch, trunk_4773_patch.patch
>
>
> When master crashs, HBaseAdmin will leaks ZooKeeper connections
> I think we should close the zk connetion when throw MasterNotRunningException
> public HBaseAdmin(Configuration c)
> throws MasterNotRunningException, ZooKeeperConnectionException {
> this.conf = HBaseConfiguration.create(c);
> this.connection = HConnectionManager.getConnection(this.conf);
> this.pause = this.conf.getLong("hbase.client.pause", 1000);
> this.numRetries = this.conf.getInt("hbase.client.retries.number", 10);
> this.retryLongerMultiplier =
> this.conf.getInt("hbase.client.retries.longer.multiplier", 10);
> //we should add this code and close the zk connection
> try{
> this.connection.getMaster();
> }catch(MasterNotRunningException e){
> HConnectionManager.deleteConnection(conf, false);
> throw e;
> }
> }
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4883) TestCatalogTracker failing for me on ubuntu
[
https://issues.apache.org/jira/browse/HBASE-4883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13159131#comment-13159131
]
Hudson commented on HBASE-4883:
---
Integrated in HBase-TRUNK-security #13 (See
[https://builds.apache.org/job/HBase-TRUNK-security/13/])
HBASE-4883 TestCatalogTracker failing for me on ubuntu
stack :
Files :
*
/hbase/trunk/src/test/java/org/apache/hadoop/hbase/catalog/TestCatalogTracker.java
> TestCatalogTracker failing for me on ubuntu
> ---
>
> Key: HBASE-4883
> URL: https://issues.apache.org/jira/browse/HBASE-4883
> Project: HBase
> Issue Type: Bug
>Reporter: stack
> Attachments: 4883.addendum, 4883.txt, tct.txt
>
>
> {code}
> ---
> Test set: org.apache.hadoop.hbase.catalog.TestCatalogTracker
> ---
> Tests run: 9, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 5.276 sec <<<
> FAILURE!
> testNoTimeoutWaitForMeta(org.apache.hadoop.hbase.catalog.TestCatalogTracker)
> Time elapsed: 1.051 sec <<< ERROR!
> org.mockito.exceptions.misusing.WrongTypeOfReturnValue:
> Result cannot be returned by getConfiguration()
> getConfiguration() should return Configuration
> at
> org.apache.hadoop.hbase.catalog.TestCatalogTracker.testNoTimeoutWaitForMeta(TestCatalogTracker.java:378)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:30)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:68)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47)
> {code}
> The above is strange since it seems to pass on jenkins and on macosx.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4878) Master crash when splitting hlog may cause data loss
[
https://issues.apache.org/jira/browse/HBASE-4878?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13159129#comment-13159129
]
Hudson commented on HBASE-4878:
---
Integrated in HBase-TRUNK-security #13 (See
[https://builds.apache.org/job/HBase-TRUNK-security/13/])
HBASE-4878 Master crash when splitting hlog may cause data loss (Chunhui
Shen)
tedyu :
Files :
* /hbase/trunk/CHANGES.txt
*
/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLogSplitter.java
> Master crash when splitting hlog may cause data loss
>
>
> Key: HBASE-4878
> URL: https://issues.apache.org/jira/browse/HBASE-4878
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.92.0
>Reporter: chunhui shen
>Assignee: chunhui shen
> Fix For: 0.92.0, 0.94.0
>
> Attachments: hbase-4878.diff, hbase-4878v2.patch
>
>
> Let's see the code of HlogSplitter#splitLog(final FileStatus[] logfiles)
> {code}
> private List splitLog(final FileStatus[] logfiles) throws IOException {
> try {
> for (FileStatus log : logfiles) {
> parseHLog(in, logPath, entryBuffers, fs, conf, skipErrors);
> }
> archiveLogs(srcDir, corruptedLogs, processedLogs, oldLogDir, fs, conf);
> } finally {
> status.setStatus("Finishing writing output logs and closing down.");
> splits = outputSink.finishWritingAndClose();
> }
> }
> {code}
> If master is killed, after finishing archiveLogs(srcDir, corruptedLogs,
> processedLogs, oldLogDir, fs, conf),
> but before finishing splits = outputSink.finishWritingAndClose();
> Log date would loss!
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4886) truncate fails in HBase shell
[
https://issues.apache.org/jira/browse/HBASE-4886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13159132#comment-13159132
]
Hudson commented on HBASE-4886:
---
Integrated in HBase-TRUNK-security #13 (See
[https://builds.apache.org/job/HBase-TRUNK-security/13/])
HBASE-4886 - truncate fails in HBase shell (Lars H)
larsh :
Files :
* /hbase/trunk/src/main/ruby/hbase/admin.rb
> truncate fails in HBase shell
> -
>
> Key: HBASE-4886
> URL: https://issues.apache.org/jira/browse/HBASE-4886
> Project: HBase
> Issue Type: Bug
> Components: shell
>Affects Versions: 0.94.0
>Reporter: Lars Hofhansl
>Assignee: Lars Hofhansl
>Priority: Minor
> Fix For: 0.94.0
>
> Attachments: 4886.txt
>
>
> Seeing this in trunk:
> {noformat}
> hbase(main):001:0> truncate 'table'
> Truncating 'table' table (it may take a while):
> ERROR: wrong number of arguments (1 for 3)
> Here is some help for this command:
> Disables, drops and recreates the specified table.
> {noformat}
> ... caused by the removal of the HTable(String) constructor.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4877) TestHCM failing sporadically on jenkins and always for me on an ubuntu machine
[ https://issues.apache.org/jira/browse/HBASE-4877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13159128#comment-13159128 ] Hudson commented on HBASE-4877: --- Integrated in HBase-TRUNK-security #13 (See [https://builds.apache.org/job/HBase-TRUNK-security/13/]) HBASE-4877 TestHCM failing sporadically on jenkins and always for me on an ubuntu machine stack : Files : * /hbase/trunk/CHANGES.txt * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/client/TestHCM.java > TestHCM failing sporadically on jenkins and always for me on an ubuntu machine > -- > > Key: HBASE-4877 > URL: https://issues.apache.org/jira/browse/HBASE-4877 > Project: HBase > Issue Type: Bug >Reporter: stack >Assignee: stack > Fix For: 0.92.0 > > Attachments: 4877.txt > > > TestHCM takes 13 minutes for me on ubuntu and fails in testClosing. It runs > fine on a mac. The problem test is not testClosing as I thought originally, > its the test just previous, testConnectionUniqueness. > testConnectionUniqueness creates the maximum cached HConnections + 10 to > verify each is unique if the passed in Configuration has a unique hash. > Problem comes when zk enforces its default max from single host of 30 > connections which is < (max cached + 10). The max does not seem to be > enforced on mac for me. The max connections runs up to max of 31 -- zk max + > 1 -- and works fine until we do the +10. On ubuntu, when we hit the zk max > of 30, we'll then go into a fail mode where we cannot set up a zk session... > each attempt takes a while. Test passes, it just takes a while. > Only, the uniqueness test does not clean up after itself and so all sessions > to zk are outstanding so then when the subsequent testClosing runs, it can't > set up connections successfully so fails. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4888) Not close ResultScanner cause the cluster abnormal ( RS memory increase)
[ https://issues.apache.org/jira/browse/HBASE-4888?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuan Kang updated HBASE-4888: - Description: A ResultScanner is created in client side, If the user doesn't invoke the "ResultScanner.close()" ,it happens that the memory of RegionServer increase rapidly and hold for a long time. Finally,the cluster goes to an abnormal status was: A ResultScanner is created in client side, If the user doesn't invoke the "ResultScanner.close()" ,it happens that the memory of RegionServer increase rapidly and hold for a long time. Finally,the cluster go to an abnormal status > Not close ResultScanner cause the cluster abnormal ( RS memory increase) > > > Key: HBASE-4888 > URL: https://issues.apache.org/jira/browse/HBASE-4888 > Project: HBase > Issue Type: Bug > Components: client >Affects Versions: 0.90.3 > Environment: CentOS 5.5 final, > hadoop-0.20.2-cdh3u1,hbase-0.20.2-cdh3u1 >Reporter: Yuan Kang > Labels: ResultScanner > Original Estimate: 672h > Remaining Estimate: 672h > > A ResultScanner is created in client side, If the user doesn't invoke the > "ResultScanner.close()" ,it happens that the memory of RegionServer increase > rapidly and hold for a long time. Finally,the cluster goes to an abnormal > status > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-4888) Not close ResultScanner cause the cluster abnormal ( RS memory increase)
Not close ResultScanner cause the cluster abnormal ( RS memory increase) Key: HBASE-4888 URL: https://issues.apache.org/jira/browse/HBASE-4888 Project: HBase Issue Type: Bug Components: client Affects Versions: 0.90.3 Environment: CentOS 5.5 final, hadoop-0.20.2-cdh3u1,hbase-0.20.2-cdh3u1 Reporter: Yuan Kang A ResultScanner is created in client side, If the user doesn't invoke the "ResultScanner.close()" ,it happens that the memory of RegionServer increase rapidly and hold for a long time. Finally,the cluster go to an abnormal status -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4729) Race between online altering and splitting kills the master
[
https://issues.apache.org/jira/browse/HBASE-4729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13159113#comment-13159113
]
Ted Yu commented on HBASE-4729:
---
{code}
+ * Updates the RegionState and sends the CLOSE RPC unless regions is being
{code}
The above should read '... region is being'
{code}
+ * of existence). TODO: What to do if split fails and is rolled back and
+ * parent is revivified?
{code}
The above would be handled in another JIRA. How about introducing a new state,
such as RS_ZK_PARENT_REGION_CLOSE_THRU_SPLIT, so that rolling back failed split
can decide what to do ?
{code}
+} catch (KeeperException ke) {
+ LOG.warn("Presuming failed getData on " + path + "; presuming " +
+"split and that the region to unassign, " + encodedName +
+", no longer exists -- confirm", ke);
+ return;
+}
{code}
We should verify the above assumption by checking that ke is NoNodeException.
If ke is not NoNodeException, we should abort.
> Race between online altering and splitting kills the master
> ---
>
> Key: HBASE-4729
> URL: https://issues.apache.org/jira/browse/HBASE-4729
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.92.0
>Reporter: Jean-Daniel Cryans
>Assignee: ramkrishna.s.vasudevan
>Priority: Critical
> Fix For: 0.92.0, 0.94.0
>
> Attachments: 4729-v2.txt, 4729-v3.txt, 4729.txt
>
>
> I was running an online alter while regions were splitting, and suddenly the
> master died and left my table half-altered (haven't restarted the master yet).
> What killed the master:
> {quote}
> 2011-11-02 17:06:44,428 FATAL org.apache.hadoop.hbase.master.HMaster:
> Unexpected ZK exception creating node CLOSING
> org.apache.zookeeper.KeeperException$NodeExistsException: KeeperErrorCode =
> NodeExists for /hbase/unassigned/f7e1783e65ea8d621a4bc96ad310f101
> at
> org.apache.zookeeper.KeeperException.create(KeeperException.java:110)
> at
> org.apache.zookeeper.KeeperException.create(KeeperException.java:42)
> at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:637)
> at
> org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.createNonSequential(RecoverableZooKeeper.java:459)
> at
> org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.create(RecoverableZooKeeper.java:441)
> at
> org.apache.hadoop.hbase.zookeeper.ZKUtil.createAndWatch(ZKUtil.java:769)
> at
> org.apache.hadoop.hbase.zookeeper.ZKAssign.createNodeClosing(ZKAssign.java:568)
> at
> org.apache.hadoop.hbase.master.AssignmentManager.unassign(AssignmentManager.java:1722)
> at
> org.apache.hadoop.hbase.master.AssignmentManager.unassign(AssignmentManager.java:1661)
> at org.apache.hadoop.hbase.master.BulkReOpen$1.run(BulkReOpen.java:69)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:662)
> {quote}
> A znode was created because the region server was splitting the region 4
> seconds before:
> {quote}
> 2011-11-02 17:06:40,704 INFO
> org.apache.hadoop.hbase.regionserver.SplitTransaction: Starting split of
> region TestTable,0012469153,1320253135043.f7e1783e65ea8d621a4bc96ad310f101.
> 2011-11-02 17:06:40,704 DEBUG
> org.apache.hadoop.hbase.regionserver.SplitTransaction:
> regionserver:62023-0x132f043bbde0710 Creating ephemeral node for
> f7e1783e65ea8d621a4bc96ad310f101 in SPLITTING state
> 2011-11-02 17:06:40,751 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign:
> regionserver:62023-0x132f043bbde0710 Attempting to transition node
> f7e1783e65ea8d621a4bc96ad310f101 from RS_ZK_REGION_SPLITTING to
> RS_ZK_REGION_SPLITTING
> ...
> 2011-11-02 17:06:44,061 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign:
> regionserver:62023-0x132f043bbde0710 Successfully transitioned node
> f7e1783e65ea8d621a4bc96ad310f101 from RS_ZK_REGION_SPLITTING to
> RS_ZK_REGION_SPLIT
> 2011-11-02 17:06:44,061 INFO
> org.apache.hadoop.hbase.regionserver.SplitTransaction: Still waiting on the
> master to process the split for f7e1783e65ea8d621a4bc96ad310f101
> {quote}
> Now that the master is dead the region server is spewing those last two lines
> like mad.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4120) isolation and allocation
[ https://issues.apache.org/jira/browse/HBASE-4120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13159111#comment-13159111 ] jirapos...@reviews.apache.org commented on HBASE-4120: -- --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/1421/ --- (Updated 2011-11-29 06:06:22.278826) Review request for hbase. Changes --- code formated Summary --- Patch used for table priority alone,In this patch, not only tables can have different priorities but also the different actions like "get","scan","put" and "delete" can have priorities. This addresses bug HBase-4120. https://issues.apache.org/jira/browse/HBase-4120 Diffs (updated) - http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/Delete.java 1207113 http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/Get.java 1207113 http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/MultiPut.java 1207113 http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/Mutation.java 1207113 http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/Operation.java 1207113 http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/Scan.java 1207113 http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/ipc/HBaseRPC.java 1207113 http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/ipc/HBaseServer.java 1207113 http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/ipc/PriorityFunction.java PRE-CREATION http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/ipc/PriorityHBaseServer.java PRE-CREATION http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/ipc/PriorityJobQueue.java PRE-CREATION http://svn.apache.org/repos/asf/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java 1207113 http://svn.apache.org/repos/asf/hbase/trunk/src/test/java/org/apache/hadoop/hbase/allocation/test/TestActionPriority.java PRE-CREATION http://svn.apache.org/repos/asf/hbase/trunk/src/test/java/org/apache/hadoop/hbase/allocation/test/TestPriorityJobQueue.java PRE-CREATION http://svn.apache.org/repos/asf/hbase/trunk/src/test/java/org/apache/hadoop/hbase/allocation/test/TestTablePriority.java PRE-CREATION Diff: https://reviews.apache.org/r/1421/diff Testing --- Tested with test cases in TestCase_For_TablePriority_trunk_v1.patch please apply the patch of HBASE-4181 first,in some circumstances this bug will affect the performance of client. Thanks, Jia > isolation and allocation > > > Key: HBASE-4120 > URL: https://issues.apache.org/jira/browse/HBASE-4120 > Project: HBase > Issue Type: New Feature > Components: master, regionserver >Affects Versions: 0.90.2, 0.90.3, 0.90.4, 0.92.0 >Reporter: Liu Jia >Assignee: Liu Jia > Fix For: 0.94.0 > > Attachments: Design_document_for_HBase_isolation_and_allocation.pdf, > Design_document_for_HBase_isolation_and_allocation_Revised.pdf, > HBase_isolation_and_allocation_user_guide.pdf, > Performance_of_Table_priority.pdf, System Structure.jpg, TablePriority.patch, > TablePriority_v8.patch, TablePriority_v8.patch, > TablePriority_v8_for_trunk.patch, TablePrioriy_v9.patch > > > The HBase isolation and allocation tool is designed to help users manage > cluster resource among different application and tables. > When we have a large scale of HBase cluster with many applications running on > it, there will be lots of problems. In Taobao there is a cluster for many > departments to test their applications performance, these applications are > based on HBase. With one cluster which has 12 servers, there will be only one > application running exclusively on this server, and many other applications > must wait until the previous test finished. > After we add allocation manage function to the cluster, applications can > share the cluster and run concurrently. Also if the Test Engineer wants to > make sure there is no interference, he/she can move out other tables from > this group. > In groups we use table priority to allocate resource, when system is busy; we > can make sure high-priority tables are not affected lower-priority tables > Different groups can have different region server configurations, some groups > optimized for reading can have large block cache size, and others optimized > for
[jira] [Updated] (HBASE-4885) Building against Hadoop 0.23 uses out-of-date MapReduce artifacts
[ https://issues.apache.org/jira/browse/HBASE-4885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-4885: - Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Committed to trunk. Thanks for patch Tom (and review Ted). > Building against Hadoop 0.23 uses out-of-date MapReduce artifacts > - > > Key: HBASE-4885 > URL: https://issues.apache.org/jira/browse/HBASE-4885 > Project: HBase > Issue Type: Bug > Components: build >Reporter: Tom White >Assignee: Tom White > Fix For: 0.94.0 > > Attachments: HBASE-4885.patch > > > The "hadoop-mapred" artifacts have been replaced by "hadoop-mapreduce-*" > artifacts in 0.23 onwards. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4729) Race between online altering and splitting kills the master
[
https://issues.apache.org/jira/browse/HBASE-4729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13159105#comment-13159105
]
Hadoop QA commented on HBASE-4729:
--
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12505452/4729-v3.txt
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 2 new or modified tests.
-1 javadoc. The javadoc tool appears to have generated -162 warning
messages.
+1 javac. The applied patch does not increase the total number of javac
compiler warnings.
-1 findbugs. The patch appears to introduce 67 new Findbugs (version
1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of
release audit warnings.
-1 core tests. The patch failed these unit tests:
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/401//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/401//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/401//console
This message is automatically generated.
> Race between online altering and splitting kills the master
> ---
>
> Key: HBASE-4729
> URL: https://issues.apache.org/jira/browse/HBASE-4729
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.92.0
>Reporter: Jean-Daniel Cryans
>Assignee: ramkrishna.s.vasudevan
>Priority: Critical
> Fix For: 0.92.0, 0.94.0
>
> Attachments: 4729-v2.txt, 4729-v3.txt, 4729.txt
>
>
> I was running an online alter while regions were splitting, and suddenly the
> master died and left my table half-altered (haven't restarted the master yet).
> What killed the master:
> {quote}
> 2011-11-02 17:06:44,428 FATAL org.apache.hadoop.hbase.master.HMaster:
> Unexpected ZK exception creating node CLOSING
> org.apache.zookeeper.KeeperException$NodeExistsException: KeeperErrorCode =
> NodeExists for /hbase/unassigned/f7e1783e65ea8d621a4bc96ad310f101
> at
> org.apache.zookeeper.KeeperException.create(KeeperException.java:110)
> at
> org.apache.zookeeper.KeeperException.create(KeeperException.java:42)
> at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:637)
> at
> org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.createNonSequential(RecoverableZooKeeper.java:459)
> at
> org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.create(RecoverableZooKeeper.java:441)
> at
> org.apache.hadoop.hbase.zookeeper.ZKUtil.createAndWatch(ZKUtil.java:769)
> at
> org.apache.hadoop.hbase.zookeeper.ZKAssign.createNodeClosing(ZKAssign.java:568)
> at
> org.apache.hadoop.hbase.master.AssignmentManager.unassign(AssignmentManager.java:1722)
> at
> org.apache.hadoop.hbase.master.AssignmentManager.unassign(AssignmentManager.java:1661)
> at org.apache.hadoop.hbase.master.BulkReOpen$1.run(BulkReOpen.java:69)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:662)
> {quote}
> A znode was created because the region server was splitting the region 4
> seconds before:
> {quote}
> 2011-11-02 17:06:40,704 INFO
> org.apache.hadoop.hbase.regionserver.SplitTransaction: Starting split of
> region TestTable,0012469153,1320253135043.f7e1783e65ea8d621a4bc96ad310f101.
> 2011-11-02 17:06:40,704 DEBUG
> org.apache.hadoop.hbase.regionserver.SplitTransaction:
> regionserver:62023-0x132f043bbde0710 Creating ephemeral node for
> f7e1783e65ea8d621a4bc96ad310f101 in SPLITTING state
> 2011-11-02 17:06:40,751 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign:
> regionserver:62023-0x132f043bbde0710 Attempting to transition node
> f7e1783e65ea8d621a4bc96ad310f101 from RS_ZK_REGION_SPLITTING to
> RS_ZK_REGION_SPLITTING
> ...
> 2011-11-02 17:06:44,061 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign:
> regionserver:62023-0x132f043bbde0710 Successfully transitioned node
> f7e1783e65ea8d621a4bc96ad310f101 from RS_ZK_REGION_SPLITTING to
> RS_ZK_REGION_SPLIT
> 2011-11-02 17:06:44,061 INFO
> org.apache.hadoop.hbase.regionserver.SplitTransaction: Still waiting on the
> master to process the split for f7e1783e65ea8d621a4bc96ad310f101
> {quote}
> Now that the master is dead the region server is spewing those last two lines
> like mad.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secur
[jira] [Updated] (HBASE-4884) Allow environment overrides for various HBase processes
[ https://issues.apache.org/jira/browse/HBASE-4884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-4884: - Resolution: Fixed Fix Version/s: 0.94.0 Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Committed to TRUNK. Thanks for the patch Ryan. (The three failed tests are not because of your patch -- they come of complaints about 'too many open files' which we are trying to figure...) > Allow environment overrides for various HBase processes > --- > > Key: HBASE-4884 > URL: https://issues.apache.org/jira/browse/HBASE-4884 > Project: HBase > Issue Type: Improvement > Components: shell >Reporter: Ryan Thiessen >Priority: Minor > Fix For: 0.94.0 > > Attachments: hbase-4884.txt > > Original Estimate: 0h > Remaining Estimate: 0h > > The current shell scripts have no mechanism for granting different > environments to various HBase subcomponents. Sometimes we want to customize > these, for example to run the thrift command with a lower HBASE_HEAPSIZE than > what we grant the regionservers. > Checking for the presence of an override file and then sourcing it if present > allows me to override this heapsize or any other variable. > Test process: > * tested with file missing. no problem, used default conf/hbase-env.sh > settings. > * tested with conf/hbase-env-thrift.sh file with heap size of 1234. ps axww > showed -Xmx1234m > * started regionserver using bin/hadoop-daemon.sh > * started hbase shell > * ran some jruby scripts via bin/hbase org.jruby.Main > * running in production for approx 2 weeks. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4729) Race between online altering and splitting kills the master
[
https://issues.apache.org/jira/browse/HBASE-4729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-4729:
-
Attachment: 4729-v3.txt
Add some more assertion about internal state of AssignmentManager after
unassign while SPLITTING in place.
> Race between online altering and splitting kills the master
> ---
>
> Key: HBASE-4729
> URL: https://issues.apache.org/jira/browse/HBASE-4729
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.92.0
>Reporter: Jean-Daniel Cryans
>Assignee: ramkrishna.s.vasudevan
>Priority: Critical
> Fix For: 0.92.0, 0.94.0
>
> Attachments: 4729-v2.txt, 4729-v3.txt, 4729.txt
>
>
> I was running an online alter while regions were splitting, and suddenly the
> master died and left my table half-altered (haven't restarted the master yet).
> What killed the master:
> {quote}
> 2011-11-02 17:06:44,428 FATAL org.apache.hadoop.hbase.master.HMaster:
> Unexpected ZK exception creating node CLOSING
> org.apache.zookeeper.KeeperException$NodeExistsException: KeeperErrorCode =
> NodeExists for /hbase/unassigned/f7e1783e65ea8d621a4bc96ad310f101
> at
> org.apache.zookeeper.KeeperException.create(KeeperException.java:110)
> at
> org.apache.zookeeper.KeeperException.create(KeeperException.java:42)
> at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:637)
> at
> org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.createNonSequential(RecoverableZooKeeper.java:459)
> at
> org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.create(RecoverableZooKeeper.java:441)
> at
> org.apache.hadoop.hbase.zookeeper.ZKUtil.createAndWatch(ZKUtil.java:769)
> at
> org.apache.hadoop.hbase.zookeeper.ZKAssign.createNodeClosing(ZKAssign.java:568)
> at
> org.apache.hadoop.hbase.master.AssignmentManager.unassign(AssignmentManager.java:1722)
> at
> org.apache.hadoop.hbase.master.AssignmentManager.unassign(AssignmentManager.java:1661)
> at org.apache.hadoop.hbase.master.BulkReOpen$1.run(BulkReOpen.java:69)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:662)
> {quote}
> A znode was created because the region server was splitting the region 4
> seconds before:
> {quote}
> 2011-11-02 17:06:40,704 INFO
> org.apache.hadoop.hbase.regionserver.SplitTransaction: Starting split of
> region TestTable,0012469153,1320253135043.f7e1783e65ea8d621a4bc96ad310f101.
> 2011-11-02 17:06:40,704 DEBUG
> org.apache.hadoop.hbase.regionserver.SplitTransaction:
> regionserver:62023-0x132f043bbde0710 Creating ephemeral node for
> f7e1783e65ea8d621a4bc96ad310f101 in SPLITTING state
> 2011-11-02 17:06:40,751 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign:
> regionserver:62023-0x132f043bbde0710 Attempting to transition node
> f7e1783e65ea8d621a4bc96ad310f101 from RS_ZK_REGION_SPLITTING to
> RS_ZK_REGION_SPLITTING
> ...
> 2011-11-02 17:06:44,061 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign:
> regionserver:62023-0x132f043bbde0710 Successfully transitioned node
> f7e1783e65ea8d621a4bc96ad310f101 from RS_ZK_REGION_SPLITTING to
> RS_ZK_REGION_SPLIT
> 2011-11-02 17:06:44,061 INFO
> org.apache.hadoop.hbase.regionserver.SplitTransaction: Still waiting on the
> master to process the split for f7e1783e65ea8d621a4bc96ad310f101
> {quote}
> Now that the master is dead the region server is spewing those last two lines
> like mad.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4729) Race between online altering and splitting kills the master
[
https://issues.apache.org/jira/browse/HBASE-4729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-4729:
-
Attachment: 4729-v2.txt
This is Ram's patch with a Test
M src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java
Remove debugLog method. Seems odd to me that the message log level would
vary by the region.
(regionOnline): Changed from public to package private.
(unassign): Allow that a region may be SPLITTING or in SPLIT state when
we go to put up the CLOSING state; just fail with a log warn message.
M src/main/java/org/apache/hadoop/hbase/regionserver/SplitTransaction.java
Minor javadoc fix.
M src/main/java/org/apache/hadoop/hbase/zookeeper/ZKTable.java
Allow that enable/disable dir may not exist (fine for test context).
A src/test/java/org/apache/hadoop/hbase/master/TestAssignmentManager.java
Added.
> Race between online altering and splitting kills the master
> ---
>
> Key: HBASE-4729
> URL: https://issues.apache.org/jira/browse/HBASE-4729
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.92.0
>Reporter: Jean-Daniel Cryans
>Assignee: ramkrishna.s.vasudevan
>Priority: Critical
> Fix For: 0.92.0, 0.94.0
>
> Attachments: 4729-v2.txt, 4729.txt
>
>
> I was running an online alter while regions were splitting, and suddenly the
> master died and left my table half-altered (haven't restarted the master yet).
> What killed the master:
> {quote}
> 2011-11-02 17:06:44,428 FATAL org.apache.hadoop.hbase.master.HMaster:
> Unexpected ZK exception creating node CLOSING
> org.apache.zookeeper.KeeperException$NodeExistsException: KeeperErrorCode =
> NodeExists for /hbase/unassigned/f7e1783e65ea8d621a4bc96ad310f101
> at
> org.apache.zookeeper.KeeperException.create(KeeperException.java:110)
> at
> org.apache.zookeeper.KeeperException.create(KeeperException.java:42)
> at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:637)
> at
> org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.createNonSequential(RecoverableZooKeeper.java:459)
> at
> org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.create(RecoverableZooKeeper.java:441)
> at
> org.apache.hadoop.hbase.zookeeper.ZKUtil.createAndWatch(ZKUtil.java:769)
> at
> org.apache.hadoop.hbase.zookeeper.ZKAssign.createNodeClosing(ZKAssign.java:568)
> at
> org.apache.hadoop.hbase.master.AssignmentManager.unassign(AssignmentManager.java:1722)
> at
> org.apache.hadoop.hbase.master.AssignmentManager.unassign(AssignmentManager.java:1661)
> at org.apache.hadoop.hbase.master.BulkReOpen$1.run(BulkReOpen.java:69)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:662)
> {quote}
> A znode was created because the region server was splitting the region 4
> seconds before:
> {quote}
> 2011-11-02 17:06:40,704 INFO
> org.apache.hadoop.hbase.regionserver.SplitTransaction: Starting split of
> region TestTable,0012469153,1320253135043.f7e1783e65ea8d621a4bc96ad310f101.
> 2011-11-02 17:06:40,704 DEBUG
> org.apache.hadoop.hbase.regionserver.SplitTransaction:
> regionserver:62023-0x132f043bbde0710 Creating ephemeral node for
> f7e1783e65ea8d621a4bc96ad310f101 in SPLITTING state
> 2011-11-02 17:06:40,751 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign:
> regionserver:62023-0x132f043bbde0710 Attempting to transition node
> f7e1783e65ea8d621a4bc96ad310f101 from RS_ZK_REGION_SPLITTING to
> RS_ZK_REGION_SPLITTING
> ...
> 2011-11-02 17:06:44,061 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign:
> regionserver:62023-0x132f043bbde0710 Successfully transitioned node
> f7e1783e65ea8d621a4bc96ad310f101 from RS_ZK_REGION_SPLITTING to
> RS_ZK_REGION_SPLIT
> 2011-11-02 17:06:44,061 INFO
> org.apache.hadoop.hbase.regionserver.SplitTransaction: Still waiting on the
> master to process the split for f7e1783e65ea8d621a4bc96ad310f101
> {quote}
> Now that the master is dead the region server is spewing those last two lines
> like mad.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4729) Race between online altering and splitting kills the master
[
https://issues.apache.org/jira/browse/HBASE-4729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13159099#comment-13159099
]
stack commented on HBASE-4729:
--
May I have some review please?
> Race between online altering and splitting kills the master
> ---
>
> Key: HBASE-4729
> URL: https://issues.apache.org/jira/browse/HBASE-4729
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.92.0
>Reporter: Jean-Daniel Cryans
>Assignee: ramkrishna.s.vasudevan
>Priority: Critical
> Fix For: 0.92.0, 0.94.0
>
> Attachments: 4729-v2.txt, 4729.txt
>
>
> I was running an online alter while regions were splitting, and suddenly the
> master died and left my table half-altered (haven't restarted the master yet).
> What killed the master:
> {quote}
> 2011-11-02 17:06:44,428 FATAL org.apache.hadoop.hbase.master.HMaster:
> Unexpected ZK exception creating node CLOSING
> org.apache.zookeeper.KeeperException$NodeExistsException: KeeperErrorCode =
> NodeExists for /hbase/unassigned/f7e1783e65ea8d621a4bc96ad310f101
> at
> org.apache.zookeeper.KeeperException.create(KeeperException.java:110)
> at
> org.apache.zookeeper.KeeperException.create(KeeperException.java:42)
> at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:637)
> at
> org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.createNonSequential(RecoverableZooKeeper.java:459)
> at
> org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.create(RecoverableZooKeeper.java:441)
> at
> org.apache.hadoop.hbase.zookeeper.ZKUtil.createAndWatch(ZKUtil.java:769)
> at
> org.apache.hadoop.hbase.zookeeper.ZKAssign.createNodeClosing(ZKAssign.java:568)
> at
> org.apache.hadoop.hbase.master.AssignmentManager.unassign(AssignmentManager.java:1722)
> at
> org.apache.hadoop.hbase.master.AssignmentManager.unassign(AssignmentManager.java:1661)
> at org.apache.hadoop.hbase.master.BulkReOpen$1.run(BulkReOpen.java:69)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:662)
> {quote}
> A znode was created because the region server was splitting the region 4
> seconds before:
> {quote}
> 2011-11-02 17:06:40,704 INFO
> org.apache.hadoop.hbase.regionserver.SplitTransaction: Starting split of
> region TestTable,0012469153,1320253135043.f7e1783e65ea8d621a4bc96ad310f101.
> 2011-11-02 17:06:40,704 DEBUG
> org.apache.hadoop.hbase.regionserver.SplitTransaction:
> regionserver:62023-0x132f043bbde0710 Creating ephemeral node for
> f7e1783e65ea8d621a4bc96ad310f101 in SPLITTING state
> 2011-11-02 17:06:40,751 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign:
> regionserver:62023-0x132f043bbde0710 Attempting to transition node
> f7e1783e65ea8d621a4bc96ad310f101 from RS_ZK_REGION_SPLITTING to
> RS_ZK_REGION_SPLITTING
> ...
> 2011-11-02 17:06:44,061 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign:
> regionserver:62023-0x132f043bbde0710 Successfully transitioned node
> f7e1783e65ea8d621a4bc96ad310f101 from RS_ZK_REGION_SPLITTING to
> RS_ZK_REGION_SPLIT
> 2011-11-02 17:06:44,061 INFO
> org.apache.hadoop.hbase.regionserver.SplitTransaction: Still waiting on the
> master to process the split for f7e1783e65ea8d621a4bc96ad310f101
> {quote}
> Now that the master is dead the region server is spewing those last two lines
> like mad.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4883) TestCatalogTracker failing for me on ubuntu
[
https://issues.apache.org/jira/browse/HBASE-4883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13159085#comment-13159085
]
Hudson commented on HBASE-4883:
---
Integrated in HBase-0.92-security #23 (See
[https://builds.apache.org/job/HBase-0.92-security/23/])
HBASE-4883 Addendum fixes compilation error
tedyu :
Files :
*
/hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/catalog/TestCatalogTracker.java
> TestCatalogTracker failing for me on ubuntu
> ---
>
> Key: HBASE-4883
> URL: https://issues.apache.org/jira/browse/HBASE-4883
> Project: HBase
> Issue Type: Bug
>Reporter: stack
> Attachments: 4883.addendum, 4883.txt, tct.txt
>
>
> {code}
> ---
> Test set: org.apache.hadoop.hbase.catalog.TestCatalogTracker
> ---
> Tests run: 9, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 5.276 sec <<<
> FAILURE!
> testNoTimeoutWaitForMeta(org.apache.hadoop.hbase.catalog.TestCatalogTracker)
> Time elapsed: 1.051 sec <<< ERROR!
> org.mockito.exceptions.misusing.WrongTypeOfReturnValue:
> Result cannot be returned by getConfiguration()
> getConfiguration() should return Configuration
> at
> org.apache.hadoop.hbase.catalog.TestCatalogTracker.testNoTimeoutWaitForMeta(TestCatalogTracker.java:378)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:30)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:68)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47)
> {code}
> The above is strange since it seems to pass on jenkins and on macosx.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4884) Allow environment overrides for various HBase processes
[ https://issues.apache.org/jira/browse/HBASE-4884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13159081#comment-13159081 ] Ryan Thiessen commented on HBASE-4884: -- I don't know what this means in the context of my patch. I am pretty sure that I didn't generate 162 warning messages, or 67 new bugs, or make three core tests fail. > Allow environment overrides for various HBase processes > --- > > Key: HBASE-4884 > URL: https://issues.apache.org/jira/browse/HBASE-4884 > Project: HBase > Issue Type: Improvement > Components: shell >Reporter: Ryan Thiessen >Priority: Minor > Attachments: hbase-4884.txt > > Original Estimate: 0h > Remaining Estimate: 0h > > The current shell scripts have no mechanism for granting different > environments to various HBase subcomponents. Sometimes we want to customize > these, for example to run the thrift command with a lower HBASE_HEAPSIZE than > what we grant the regionservers. > Checking for the presence of an override file and then sourcing it if present > allows me to override this heapsize or any other variable. > Test process: > * tested with file missing. no problem, used default conf/hbase-env.sh > settings. > * tested with conf/hbase-env-thrift.sh file with heap size of 1234. ps axww > showed -Xmx1234m > * started regionserver using bin/hadoop-daemon.sh > * started hbase shell > * ran some jruby scripts via bin/hbase org.jruby.Main > * running in production for approx 2 weeks. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4883) TestCatalogTracker failing for me on ubuntu
[
https://issues.apache.org/jira/browse/HBASE-4883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13159056#comment-13159056
]
Hadoop QA commented on HBASE-4883:
--
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12505439/4883.addendum
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 3 new or modified tests.
-1 patch. The patch command could not apply the patch.
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/400//console
This message is automatically generated.
> TestCatalogTracker failing for me on ubuntu
> ---
>
> Key: HBASE-4883
> URL: https://issues.apache.org/jira/browse/HBASE-4883
> Project: HBase
> Issue Type: Bug
>Reporter: stack
> Attachments: 4883.addendum, 4883.txt, tct.txt
>
>
> {code}
> ---
> Test set: org.apache.hadoop.hbase.catalog.TestCatalogTracker
> ---
> Tests run: 9, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 5.276 sec <<<
> FAILURE!
> testNoTimeoutWaitForMeta(org.apache.hadoop.hbase.catalog.TestCatalogTracker)
> Time elapsed: 1.051 sec <<< ERROR!
> org.mockito.exceptions.misusing.WrongTypeOfReturnValue:
> Result cannot be returned by getConfiguration()
> getConfiguration() should return Configuration
> at
> org.apache.hadoop.hbase.catalog.TestCatalogTracker.testNoTimeoutWaitForMeta(TestCatalogTracker.java:378)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:30)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:68)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47)
> {code}
> The above is strange since it seems to pass on jenkins and on macosx.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4883) TestCatalogTracker failing for me on ubuntu
[
https://issues.apache.org/jira/browse/HBASE-4883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13159055#comment-13159055
]
Ted Yu commented on HBASE-4883:
---
Applied addendum to 0.92 branch.
> TestCatalogTracker failing for me on ubuntu
> ---
>
> Key: HBASE-4883
> URL: https://issues.apache.org/jira/browse/HBASE-4883
> Project: HBase
> Issue Type: Bug
>Reporter: stack
> Attachments: 4883.addendum, 4883.txt, tct.txt
>
>
> {code}
> ---
> Test set: org.apache.hadoop.hbase.catalog.TestCatalogTracker
> ---
> Tests run: 9, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 5.276 sec <<<
> FAILURE!
> testNoTimeoutWaitForMeta(org.apache.hadoop.hbase.catalog.TestCatalogTracker)
> Time elapsed: 1.051 sec <<< ERROR!
> org.mockito.exceptions.misusing.WrongTypeOfReturnValue:
> Result cannot be returned by getConfiguration()
> getConfiguration() should return Configuration
> at
> org.apache.hadoop.hbase.catalog.TestCatalogTracker.testNoTimeoutWaitForMeta(TestCatalogTracker.java:378)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:30)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:68)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47)
> {code}
> The above is strange since it seems to pass on jenkins and on macosx.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4883) TestCatalogTracker failing for me on ubuntu
[
https://issues.apache.org/jira/browse/HBASE-4883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-4883:
--
Attachment: 4883.addendum
> TestCatalogTracker failing for me on ubuntu
> ---
>
> Key: HBASE-4883
> URL: https://issues.apache.org/jira/browse/HBASE-4883
> Project: HBase
> Issue Type: Bug
>Reporter: stack
> Attachments: 4883.addendum, 4883.txt, tct.txt
>
>
> {code}
> ---
> Test set: org.apache.hadoop.hbase.catalog.TestCatalogTracker
> ---
> Tests run: 9, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 5.276 sec <<<
> FAILURE!
> testNoTimeoutWaitForMeta(org.apache.hadoop.hbase.catalog.TestCatalogTracker)
> Time elapsed: 1.051 sec <<< ERROR!
> org.mockito.exceptions.misusing.WrongTypeOfReturnValue:
> Result cannot be returned by getConfiguration()
> getConfiguration() should return Configuration
> at
> org.apache.hadoop.hbase.catalog.TestCatalogTracker.testNoTimeoutWaitForMeta(TestCatalogTracker.java:378)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:30)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:68)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47)
> {code}
> The above is strange since it seems to pass on jenkins and on macosx.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4218) Delta Encoding of KeyValues (aka prefix compression)
[ https://issues.apache.org/jira/browse/HBASE-4218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13159048#comment-13159048 ] Ted Yu commented on HBASE-4218: --- HadoopQA isn't functioning as usual. Manual execution of test suite is needed. > Delta Encoding of KeyValues (aka prefix compression) > - > > Key: HBASE-4218 > URL: https://issues.apache.org/jira/browse/HBASE-4218 > Project: HBase > Issue Type: Improvement > Components: io >Affects Versions: 0.94.0 >Reporter: Jacek Migdal >Assignee: Mikhail Bautin > Labels: compression > Attachments: D447.1.patch, D447.2.patch, D447.3.patch, D447.4.patch, > D447.5.patch, Delta_encoding_with_memstore_TS.patch, open-source.diff > > > A compression for keys. Keys are sorted in HFile and they are usually very > similar. Because of that, it is possible to design better compression than > general purpose algorithms, > It is an additional step designed to be used in memory. It aims to save > memory in cache as well as speeding seeks within HFileBlocks. It should > improve performance a lot, if key lengths are larger than value lengths. For > example, it makes a lot of sense to use it when value is a counter. > Initial tests on real data (key length = ~ 90 bytes , value length = 8 bytes) > shows that I could achieve decent level of compression: > key compression ratio: 92% > total compression ratio: 85% > LZO on the same data: 85% > LZO after delta encoding: 91% > While having much better performance (20-80% faster decompression ratio than > LZO). Moreover, it should allow far more efficient seeking which should > improve performance a bit. > It seems that a simple compression algorithms are good enough. Most of the > savings are due to prefix compression, int128 encoding, timestamp diffs and > bitfields to avoid duplication. That way, comparisons of compressed data can > be much faster than a byte comparator (thanks to prefix compression and > bitfields). > In order to implement it in HBase two important changes in design will be > needed: > -solidify interface to HFileBlock / HFileReader Scanner to provide seeking > and iterating; access to uncompressed buffer in HFileBlock will have bad > performance > -extend comparators to support comparison assuming that N first bytes are > equal (or some fields are equal) > Link to a discussion about something similar: > http://search-hadoop.com/m/5aqGXJEnaD1/hbase+windows&subj=Re+prefix+compression -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4875) ZKLeaderManager.handleLeaderChange() doesn't handle KeeperException$SessionExpiredException
[
https://issues.apache.org/jira/browse/HBASE-4875?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13159047#comment-13159047
]
Ted Yu commented on HBASE-4875:
---
In HMaster, this.zooKeeper is replenished:
{code}
private boolean tryRecoveringExpiredZKSession() throws InterruptedException,
IOException, KeeperException {
this.zooKeeper = new ZooKeeperWatcher(conf, MASTER + ":"
+ this.serverName.getPort(), this, true);
{code}
To handle expired session in a general way, we need to update the
ZooKeeperWatcher of the caller.
> ZKLeaderManager.handleLeaderChange() doesn't handle
> KeeperException$SessionExpiredException
> ---
>
> Key: HBASE-4875
> URL: https://issues.apache.org/jira/browse/HBASE-4875
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.92.0
>Reporter: Ted Yu
>
> TestMasterFailover#testSimpleMasterFailover has failed twice in a row for
> builds 15 and 16.
> From
> https://builds.apache.org/view/G-L/view/HBase/job/HBase-0.92-security/16/testReport/org.apache.hadoop.hbase.master/TestMasterFailover/testSimpleMasterFailover/:
> {code}
> 2011-11-26 01:34:49,217 DEBUG
> [RegionServer:0;hemera.apache.org,57516,1322271278190-EventThread]
> zookeeper.ZooKeeperWatcher(257): regionserver:57516-0x133dd828133
> Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected,
> path=/hbase/tokenauth/keymaster
> 2011-11-26 01:34:49,217 WARN [Thread-1-EventThread] zookeeper.ZKUtil(234):
> master:52934-0x133dd828131 Unable to set watcher on znode
> /hbase/tokenauth/keymaster
> org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode
> = Session expired for /hbase/tokenauth/keymaster
> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
> at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1003)
> at
> org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.exists(RecoverableZooKeeper.java:154)
> at
> org.apache.hadoop.hbase.zookeeper.ZKUtil.watchAndCheckExists(ZKUtil.java:225)
> at
> org.apache.hadoop.hbase.zookeeper.ZKLeaderManager.handleLeaderChange(ZKLeaderManager.java:85)
> at
> org.apache.hadoop.hbase.zookeeper.ZKLeaderManager.nodeDeleted(ZKLeaderManager.java:78)
> at
> org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.process(ZooKeeperWatcher.java:281)
> at
> org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:521)
> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:497)
> 2011-11-26 01:34:49,218 ERROR [Thread-1-EventThread]
> zookeeper.ZooKeeperWatcher(403): master:52934-0x133dd828131 Received
> unexpected KeeperException, re-throwing exception
> org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode
> = Session expired for /hbase/tokenauth/keymaster
> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
> at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1003)
> at
> org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.exists(RecoverableZooKeeper.java:154)
> at
> org.apache.hadoop.hbase.zookeeper.ZKUtil.watchAndCheckExists(ZKUtil.java:225)
> at
> org.apache.hadoop.hbase.zookeeper.ZKLeaderManager.handleLeaderChange(ZKLeaderManager.java:85)
> at
> org.apache.hadoop.hbase.zookeeper.ZKLeaderManager.nodeDeleted(ZKLeaderManager.java:78)
> at
> org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.process(ZooKeeperWatcher.java:281)
> at
> org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:521)
> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:497)
> 2011-11-26 01:34:49,216 DEBUG
> [RegionServer:2;hemera.apache.org,44702,1322271278232-EventThread]
> zookeeper.ZKUtil(230): hconnection-0x133dd828139 /hbase/master does not
> exist. Watcher is set.
> 2011-11-26 01:34:49,215 DEBUG [Thread-1-EventThread] zookeeper.ZKUtil(230):
> master:44883-0x133dd828132 /hbase/master does not exist. Watcher is set.
> 2011-11-26 01:34:49,219 DEBUG [Thread-1-EventThread]
> master.ActiveMasterManager(104): No master available. Notifying waiting
> threads
> 2011-11-26 01:34:49,215 INFO
> [Master:1;hemera.apache.org,52934,1322271278115] master.HMaster(338): HMaster
> main thread exiting
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4218) Delta Encoding of KeyValues (aka prefix compression)
[ https://issues.apache.org/jira/browse/HBASE-4218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13159045#comment-13159045 ] Hadoop QA commented on HBASE-4218: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12505438/Delta_encoding_with_memstore_TS.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 81 new or modified tests. -1 javadoc. The javadoc tool appears to have generated -145 warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 72 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/399//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/399//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/399//console This message is automatically generated. > Delta Encoding of KeyValues (aka prefix compression) > - > > Key: HBASE-4218 > URL: https://issues.apache.org/jira/browse/HBASE-4218 > Project: HBase > Issue Type: Improvement > Components: io >Affects Versions: 0.94.0 >Reporter: Jacek Migdal >Assignee: Mikhail Bautin > Labels: compression > Attachments: D447.1.patch, D447.2.patch, D447.3.patch, D447.4.patch, > D447.5.patch, Delta_encoding_with_memstore_TS.patch, open-source.diff > > > A compression for keys. Keys are sorted in HFile and they are usually very > similar. Because of that, it is possible to design better compression than > general purpose algorithms, > It is an additional step designed to be used in memory. It aims to save > memory in cache as well as speeding seeks within HFileBlocks. It should > improve performance a lot, if key lengths are larger than value lengths. For > example, it makes a lot of sense to use it when value is a counter. > Initial tests on real data (key length = ~ 90 bytes , value length = 8 bytes) > shows that I could achieve decent level of compression: > key compression ratio: 92% > total compression ratio: 85% > LZO on the same data: 85% > LZO after delta encoding: 91% > While having much better performance (20-80% faster decompression ratio than > LZO). Moreover, it should allow far more efficient seeking which should > improve performance a bit. > It seems that a simple compression algorithms are good enough. Most of the > savings are due to prefix compression, int128 encoding, timestamp diffs and > bitfields to avoid duplication. That way, comparisons of compressed data can > be much faster than a byte comparator (thanks to prefix compression and > bitfields). > In order to implement it in HBase two important changes in design will be > needed: > -solidify interface to HFileBlock / HFileReader Scanner to provide seeking > and iterating; access to uncompressed buffer in HFileBlock will have bad > performance > -extend comparators to support comparison assuming that N first bytes are > equal (or some fields are equal) > Link to a discussion about something similar: > http://search-hadoop.com/m/5aqGXJEnaD1/hbase+windows&subj=Re+prefix+compression -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4218) Delta Encoding of KeyValues (aka prefix compression)
[ https://issues.apache.org/jira/browse/HBASE-4218?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikhail Bautin updated HBASE-4218: -- Status: Patch Available (was: Open) > Delta Encoding of KeyValues (aka prefix compression) > - > > Key: HBASE-4218 > URL: https://issues.apache.org/jira/browse/HBASE-4218 > Project: HBase > Issue Type: Improvement > Components: io >Affects Versions: 0.94.0 >Reporter: Jacek Migdal >Assignee: Mikhail Bautin > Labels: compression > Attachments: D447.1.patch, D447.2.patch, D447.3.patch, D447.4.patch, > D447.5.patch, Delta_encoding_with_memstore_TS.patch, open-source.diff > > > A compression for keys. Keys are sorted in HFile and they are usually very > similar. Because of that, it is possible to design better compression than > general purpose algorithms, > It is an additional step designed to be used in memory. It aims to save > memory in cache as well as speeding seeks within HFileBlocks. It should > improve performance a lot, if key lengths are larger than value lengths. For > example, it makes a lot of sense to use it when value is a counter. > Initial tests on real data (key length = ~ 90 bytes , value length = 8 bytes) > shows that I could achieve decent level of compression: > key compression ratio: 92% > total compression ratio: 85% > LZO on the same data: 85% > LZO after delta encoding: 91% > While having much better performance (20-80% faster decompression ratio than > LZO). Moreover, it should allow far more efficient seeking which should > improve performance a bit. > It seems that a simple compression algorithms are good enough. Most of the > savings are due to prefix compression, int128 encoding, timestamp diffs and > bitfields to avoid duplication. That way, comparisons of compressed data can > be much faster than a byte comparator (thanks to prefix compression and > bitfields). > In order to implement it in HBase two important changes in design will be > needed: > -solidify interface to HFileBlock / HFileReader Scanner to provide seeking > and iterating; access to uncompressed buffer in HFileBlock will have bad > performance > -extend comparators to support comparison assuming that N first bytes are > equal (or some fields are equal) > Link to a discussion about something similar: > http://search-hadoop.com/m/5aqGXJEnaD1/hbase+windows&subj=Re+prefix+compression -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4218) Delta Encoding of KeyValues (aka prefix compression)
[ https://issues.apache.org/jira/browse/HBASE-4218?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikhail Bautin updated HBASE-4218: -- Attachment: Delta_encoding_with_memstore_TS.patch Attaching the most recent patch for testing on Jenkins. This is still pending cluster testing. > Delta Encoding of KeyValues (aka prefix compression) > - > > Key: HBASE-4218 > URL: https://issues.apache.org/jira/browse/HBASE-4218 > Project: HBase > Issue Type: Improvement > Components: io >Affects Versions: 0.94.0 >Reporter: Jacek Migdal > Labels: compression > Attachments: D447.1.patch, D447.2.patch, D447.3.patch, D447.4.patch, > D447.5.patch, Delta_encoding_with_memstore_TS.patch, open-source.diff > > > A compression for keys. Keys are sorted in HFile and they are usually very > similar. Because of that, it is possible to design better compression than > general purpose algorithms, > It is an additional step designed to be used in memory. It aims to save > memory in cache as well as speeding seeks within HFileBlocks. It should > improve performance a lot, if key lengths are larger than value lengths. For > example, it makes a lot of sense to use it when value is a counter. > Initial tests on real data (key length = ~ 90 bytes , value length = 8 bytes) > shows that I could achieve decent level of compression: > key compression ratio: 92% > total compression ratio: 85% > LZO on the same data: 85% > LZO after delta encoding: 91% > While having much better performance (20-80% faster decompression ratio than > LZO). Moreover, it should allow far more efficient seeking which should > improve performance a bit. > It seems that a simple compression algorithms are good enough. Most of the > savings are due to prefix compression, int128 encoding, timestamp diffs and > bitfields to avoid duplication. That way, comparisons of compressed data can > be much faster than a byte comparator (thanks to prefix compression and > bitfields). > In order to implement it in HBase two important changes in design will be > needed: > -solidify interface to HFileBlock / HFileReader Scanner to provide seeking > and iterating; access to uncompressed buffer in HFileBlock will have bad > performance > -extend comparators to support comparison assuming that N first bytes are > equal (or some fields are equal) > Link to a discussion about something similar: > http://search-hadoop.com/m/5aqGXJEnaD1/hbase+windows&subj=Re+prefix+compression -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HBASE-4218) Delta Encoding of KeyValues (aka prefix compression)
[ https://issues.apache.org/jira/browse/HBASE-4218?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikhail Bautin reassigned HBASE-4218: - Assignee: Mikhail Bautin > Delta Encoding of KeyValues (aka prefix compression) > - > > Key: HBASE-4218 > URL: https://issues.apache.org/jira/browse/HBASE-4218 > Project: HBase > Issue Type: Improvement > Components: io >Affects Versions: 0.94.0 >Reporter: Jacek Migdal >Assignee: Mikhail Bautin > Labels: compression > Attachments: D447.1.patch, D447.2.patch, D447.3.patch, D447.4.patch, > D447.5.patch, Delta_encoding_with_memstore_TS.patch, open-source.diff > > > A compression for keys. Keys are sorted in HFile and they are usually very > similar. Because of that, it is possible to design better compression than > general purpose algorithms, > It is an additional step designed to be used in memory. It aims to save > memory in cache as well as speeding seeks within HFileBlocks. It should > improve performance a lot, if key lengths are larger than value lengths. For > example, it makes a lot of sense to use it when value is a counter. > Initial tests on real data (key length = ~ 90 bytes , value length = 8 bytes) > shows that I could achieve decent level of compression: > key compression ratio: 92% > total compression ratio: 85% > LZO on the same data: 85% > LZO after delta encoding: 91% > While having much better performance (20-80% faster decompression ratio than > LZO). Moreover, it should allow far more efficient seeking which should > improve performance a bit. > It seems that a simple compression algorithms are good enough. Most of the > savings are due to prefix compression, int128 encoding, timestamp diffs and > bitfields to avoid duplication. That way, comparisons of compressed data can > be much faster than a byte comparator (thanks to prefix compression and > bitfields). > In order to implement it in HBase two important changes in design will be > needed: > -solidify interface to HFileBlock / HFileReader Scanner to provide seeking > and iterating; access to uncompressed buffer in HFileBlock will have bad > performance > -extend comparators to support comparison assuming that N first bytes are > equal (or some fields are equal) > Link to a discussion about something similar: > http://search-hadoop.com/m/5aqGXJEnaD1/hbase+windows&subj=Re+prefix+compression -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4218) Delta Encoding of KeyValues (aka prefix compression)
[ https://issues.apache.org/jira/browse/HBASE-4218?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phabricator updated HBASE-4218: --- Attachment: D447.5.patch mbautin updated the revision "[jira] [HBASE-4218] Delta encoding for keys in HFile". Reviewers: JIRA, tedyu, stack, nspiegelberg, Kannan Addressed most of Ted's and my own review comments from https://reviews.apache.org/r/2308/diff/. Resolved conflicts with memstoreTS storage, and all unit tests but one pass (TestRollingRestart fails in a weird way, investigating). Also, reading/writing VLongs from/to ByteBuffers (for memstore timestamp serialization) is currently implemented inefficiently. REVISION DETAIL https://reviews.facebook.net/D447 AFFECTED FILES src/main/java/org/apache/hadoop/hbase/HColumnDescriptor.java src/main/java/org/apache/hadoop/hbase/KeyValue.java src/main/java/org/apache/hadoop/hbase/io/HalfStoreFileReader.java src/main/java/org/apache/hadoop/hbase/io/deltaencoder/BitsetKeyDeltaEncoder.java src/main/java/org/apache/hadoop/hbase/io/deltaencoder/BufferedDeltaEncoder.java src/main/java/org/apache/hadoop/hbase/io/deltaencoder/CompressionState.java src/main/java/org/apache/hadoop/hbase/io/deltaencoder/CopyKeyDeltaEncoder.java src/main/java/org/apache/hadoop/hbase/io/deltaencoder/DeltaEncodedBlock.java src/main/java/org/apache/hadoop/hbase/io/deltaencoder/DeltaEncoder.java src/main/java/org/apache/hadoop/hbase/io/deltaencoder/DeltaEncoderAlgorithms.java src/main/java/org/apache/hadoop/hbase/io/deltaencoder/DeltaEncoderBufferTooSmallException.java src/main/java/org/apache/hadoop/hbase/io/deltaencoder/DiffKeyDeltaEncoder.java src/main/java/org/apache/hadoop/hbase/io/deltaencoder/FastDiffDeltaEncoder.java src/main/java/org/apache/hadoop/hbase/io/deltaencoder/PrefixKeyDeltaEncoder.java src/main/java/org/apache/hadoop/hbase/io/hfile/AbstractHFileReader.java src/main/java/org/apache/hadoop/hbase/io/hfile/AbstractHFileWriter.java src/main/java/org/apache/hadoop/hbase/io/hfile/BlockDeltaEncoder.java src/main/java/org/apache/hadoop/hbase/io/hfile/BlockType.java src/main/java/org/apache/hadoop/hbase/io/hfile/EmptyBlockDeltaEncoder.java src/main/java/org/apache/hadoop/hbase/io/hfile/HFile.java src/main/java/org/apache/hadoop/hbase/io/hfile/HFileBlock.java src/main/java/org/apache/hadoop/hbase/io/hfile/HFileBlockDeltaEncoder.java src/main/java/org/apache/hadoop/hbase/io/hfile/HFileBlockIndex.java src/main/java/org/apache/hadoop/hbase/io/hfile/HFileReaderV1.java src/main/java/org/apache/hadoop/hbase/io/hfile/HFileReaderV2.java src/main/java/org/apache/hadoop/hbase/io/hfile/HFileWriterV1.java src/main/java/org/apache/hadoop/hbase/io/hfile/HFileWriterV2.java src/main/java/org/apache/hadoop/hbase/mapreduce/LoadIncrementalHFiles.java src/main/java/org/apache/hadoop/hbase/regionserver/MemStore.java src/main/java/org/apache/hadoop/hbase/regionserver/Store.java src/main/java/org/apache/hadoop/hbase/regionserver/StoreFile.java src/main/java/org/apache/hadoop/hbase/regionserver/StoreFileScanner.java src/main/java/org/apache/hadoop/hbase/regionserver/metrics/SchemaConfigured.java src/main/ruby/hbase/admin.rb src/test/java/org/apache/hadoop/hbase/HBaseTestCase.java src/test/java/org/apache/hadoop/hbase/HFilePerformanceEvaluation.java src/test/java/org/apache/hadoop/hbase/client/TestFromClientSide.java src/test/java/org/apache/hadoop/hbase/io/TestHalfStoreFileReader.java src/test/java/org/apache/hadoop/hbase/io/TestHeapSize.java src/test/java/org/apache/hadoop/hbase/io/deltaencoder/RedundantKVGenerator.java src/test/java/org/apache/hadoop/hbase/io/deltaencoder/TestBufferedDeltaEncoder.java src/test/java/org/apache/hadoop/hbase/io/deltaencoder/TestDeltaEncoders.java src/test/java/org/apache/hadoop/hbase/io/hfile/CacheTestUtils.java src/test/java/org/apache/hadoop/hbase/io/hfile/TestCacheOnWrite.java src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFileBlock.java src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFileBlockDeltaEncoder.java src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFileBlockIndex.java src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFileWriterV2.java src/test/java/org/apache/hadoop/hbase/regionserver/CreateRandomStoreFile.java src/test/java/org/apache/hadoop/hbase/regionserver/DeltaEncodingSeekPerformance.java src/test/java/org/apache/hadoop/hbase/regionserver/DeltaEncodingUtil.java src/test/java/org/apache/hadoop/hbase/regionserver/TestCompactSelection.java src/test/java/org/apache/hadoop/hbase/regionserver/TestCompaction.java src/test/java/org/apache/hadoop/hbase/regionserver/TestCompoundBloomFilter.java src/test/java/org/apache/hadoop/hbase/regionserver/TestFSErrorsExposed.java src/test/java/org/apache/hadoop/hbase/regionserver/TestStoreFile.java > Delta Encoding of KeyValues (aka prefix compression) >
[jira] [Commented] (HBASE-4875) ZKLeaderManager.handleLeaderChange() doesn't handle KeeperException$SessionExpiredException
[
https://issues.apache.org/jira/browse/HBASE-4875?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13159017#comment-13159017
]
Jimmy Xiang commented on HBASE-4875:
Can we just retry if it is session expired exception? Currently, it just retry
in case of (RecoverableZooKeeper):
case CONNECTIONLOSS:
case OPERATIONTIMEOUT:
> ZKLeaderManager.handleLeaderChange() doesn't handle
> KeeperException$SessionExpiredException
> ---
>
> Key: HBASE-4875
> URL: https://issues.apache.org/jira/browse/HBASE-4875
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.92.0
>Reporter: Ted Yu
>
> TestMasterFailover#testSimpleMasterFailover has failed twice in a row for
> builds 15 and 16.
> From
> https://builds.apache.org/view/G-L/view/HBase/job/HBase-0.92-security/16/testReport/org.apache.hadoop.hbase.master/TestMasterFailover/testSimpleMasterFailover/:
> {code}
> 2011-11-26 01:34:49,217 DEBUG
> [RegionServer:0;hemera.apache.org,57516,1322271278190-EventThread]
> zookeeper.ZooKeeperWatcher(257): regionserver:57516-0x133dd828133
> Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected,
> path=/hbase/tokenauth/keymaster
> 2011-11-26 01:34:49,217 WARN [Thread-1-EventThread] zookeeper.ZKUtil(234):
> master:52934-0x133dd828131 Unable to set watcher on znode
> /hbase/tokenauth/keymaster
> org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode
> = Session expired for /hbase/tokenauth/keymaster
> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
> at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1003)
> at
> org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.exists(RecoverableZooKeeper.java:154)
> at
> org.apache.hadoop.hbase.zookeeper.ZKUtil.watchAndCheckExists(ZKUtil.java:225)
> at
> org.apache.hadoop.hbase.zookeeper.ZKLeaderManager.handleLeaderChange(ZKLeaderManager.java:85)
> at
> org.apache.hadoop.hbase.zookeeper.ZKLeaderManager.nodeDeleted(ZKLeaderManager.java:78)
> at
> org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.process(ZooKeeperWatcher.java:281)
> at
> org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:521)
> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:497)
> 2011-11-26 01:34:49,218 ERROR [Thread-1-EventThread]
> zookeeper.ZooKeeperWatcher(403): master:52934-0x133dd828131 Received
> unexpected KeeperException, re-throwing exception
> org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode
> = Session expired for /hbase/tokenauth/keymaster
> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
> at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1003)
> at
> org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.exists(RecoverableZooKeeper.java:154)
> at
> org.apache.hadoop.hbase.zookeeper.ZKUtil.watchAndCheckExists(ZKUtil.java:225)
> at
> org.apache.hadoop.hbase.zookeeper.ZKLeaderManager.handleLeaderChange(ZKLeaderManager.java:85)
> at
> org.apache.hadoop.hbase.zookeeper.ZKLeaderManager.nodeDeleted(ZKLeaderManager.java:78)
> at
> org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.process(ZooKeeperWatcher.java:281)
> at
> org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:521)
> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:497)
> 2011-11-26 01:34:49,216 DEBUG
> [RegionServer:2;hemera.apache.org,44702,1322271278232-EventThread]
> zookeeper.ZKUtil(230): hconnection-0x133dd828139 /hbase/master does not
> exist. Watcher is set.
> 2011-11-26 01:34:49,215 DEBUG [Thread-1-EventThread] zookeeper.ZKUtil(230):
> master:44883-0x133dd828132 /hbase/master does not exist. Watcher is set.
> 2011-11-26 01:34:49,219 DEBUG [Thread-1-EventThread]
> master.ActiveMasterManager(104): No master available. Notifying waiting
> threads
> 2011-11-26 01:34:49,215 INFO
> [Master:1;hemera.apache.org,52934,1322271278115] master.HMaster(338): HMaster
> main thread exiting
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4887) Write full enum name of Compression.Algorithm into HFile
[ https://issues.apache.org/jira/browse/HBASE-4887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13159011#comment-13159011 ] Jimmy Xiang commented on HBASE-4887: Using full enum name is great. Since the ordinal in the file is an integer, to use enum name which is a string, we may need to increase the file version, or using some trick (for example, writing -1, then the enum name). How about a different solution? Introducing an Id to the enum, for those existing Compression.Algorithm, we use their current ordinal as their Ids. For new compress algorithms, their ordinal will not matter any more. > Write full enum name of Compression.Algorithm into HFile > > > Key: HBASE-4887 > URL: https://issues.apache.org/jira/browse/HBASE-4887 > Project: HBase > Issue Type: Improvement >Reporter: Ted Yu > > Currently the ordinal of compression algorithms is used. > This places unnecessary constraint when new compress algorithm is added. > We should write full enum name of Compression.Algorithm into HFile -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4883) TestCatalogTracker failing for me on ubuntu
[
https://issues.apache.org/jira/browse/HBASE-4883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13159001#comment-13159001
]
Ted Yu commented on HBASE-4883:
---
Looks like the following was missing under 0.92 branch:
{code}
import org.junit.Ignore;
{code}
> TestCatalogTracker failing for me on ubuntu
> ---
>
> Key: HBASE-4883
> URL: https://issues.apache.org/jira/browse/HBASE-4883
> Project: HBase
> Issue Type: Bug
>Reporter: stack
> Attachments: 4883.txt, tct.txt
>
>
> {code}
> ---
> Test set: org.apache.hadoop.hbase.catalog.TestCatalogTracker
> ---
> Tests run: 9, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 5.276 sec <<<
> FAILURE!
> testNoTimeoutWaitForMeta(org.apache.hadoop.hbase.catalog.TestCatalogTracker)
> Time elapsed: 1.051 sec <<< ERROR!
> org.mockito.exceptions.misusing.WrongTypeOfReturnValue:
> Result cannot be returned by getConfiguration()
> getConfiguration() should return Configuration
> at
> org.apache.hadoop.hbase.catalog.TestCatalogTracker.testNoTimeoutWaitForMeta(TestCatalogTracker.java:378)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:30)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:68)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47)
> {code}
> The above is strange since it seems to pass on jenkins and on macosx.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4797) [availability] Skip recovered.edits files with edits we know older than what region currently has
[ https://issues.apache.org/jira/browse/HBASE-4797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13158997#comment-13158997 ] Hudson commented on HBASE-4797: --- Integrated in HBase-0.92-security #22 (See [https://builds.apache.org/job/HBase-0.92-security/22/]) HBASE-4869 Backport to 0.92: HBASE-4797 [availability] Skip recovered.edits files with edits older than what region currently has (Jimmy Xiang) tedyu : Files : * /hbase/branches/0.92/CHANGES.txt * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegion.java > [availability] Skip recovered.edits files with edits we know older than what > region currently has > - > > Key: HBASE-4797 > URL: https://issues.apache.org/jira/browse/HBASE-4797 > Project: HBase > Issue Type: Bug > Components: performance >Reporter: stack >Assignee: Jimmy Xiang >Priority: Critical > Labels: noob > Fix For: 0.94.0 > > Attachments: 0001-HBASE-4797-[availability]-skip-older-edits.patch, > 0001-HBASE-4797-[availability]-skip-older-edits.patch, > 0001-HBASE-4797-availability-skip-files-with-edits-we-kno.patch, > 0001-HBASE-4797-availability-skip-files-with-edits-we-kno.patch > > > Testing 0.92, I crashed all servers out. Another bug makes it so WALs are > not getting cleaned so I had 7000 regions to replay. The distributed split > code did a nice job and cluster came back but interesting is that some hot > regions ended up having loads of recovered.edits files -- tens if not > hundreds -- to replay against the region (can we bulk load recovered.edits > instead of replaying them?). Each recovered.edits file is taking about a > second to process (though only about 30 odd edits per file it seems). The > region is unavailable during this time. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4883) TestCatalogTracker failing for me on ubuntu
[
https://issues.apache.org/jira/browse/HBASE-4883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13158995#comment-13158995
]
Hudson commented on HBASE-4883:
---
Integrated in HBase-0.92-security #22 (See
[https://builds.apache.org/job/HBase-0.92-security/22/])
HBASE-4883 TestCatalogTracker failing for me on ubuntu
stack :
Files :
*
/hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/catalog/TestCatalogTracker.java
> TestCatalogTracker failing for me on ubuntu
> ---
>
> Key: HBASE-4883
> URL: https://issues.apache.org/jira/browse/HBASE-4883
> Project: HBase
> Issue Type: Bug
>Reporter: stack
> Attachments: 4883.txt, tct.txt
>
>
> {code}
> ---
> Test set: org.apache.hadoop.hbase.catalog.TestCatalogTracker
> ---
> Tests run: 9, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 5.276 sec <<<
> FAILURE!
> testNoTimeoutWaitForMeta(org.apache.hadoop.hbase.catalog.TestCatalogTracker)
> Time elapsed: 1.051 sec <<< ERROR!
> org.mockito.exceptions.misusing.WrongTypeOfReturnValue:
> Result cannot be returned by getConfiguration()
> getConfiguration() should return Configuration
> at
> org.apache.hadoop.hbase.catalog.TestCatalogTracker.testNoTimeoutWaitForMeta(TestCatalogTracker.java:378)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:30)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:68)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47)
> {code}
> The above is strange since it seems to pass on jenkins and on macosx.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4869) Backport to 0.92: HBASE-4797 [availability] Skip recovered.edits files with edits older than what region currently has
[ https://issues.apache.org/jira/browse/HBASE-4869?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13158996#comment-13158996 ] Hudson commented on HBASE-4869: --- Integrated in HBase-0.92-security #22 (See [https://builds.apache.org/job/HBase-0.92-security/22/]) HBASE-4869 Backport to 0.92: HBASE-4797 [availability] Skip recovered.edits files with edits older than what region currently has (Jimmy Xiang) tedyu : Files : * /hbase/branches/0.92/CHANGES.txt * /hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegion.java > Backport to 0.92: HBASE-4797 [availability] Skip recovered.edits files with > edits older than what region currently has > -- > > Key: HBASE-4869 > URL: https://issues.apache.org/jira/browse/HBASE-4869 > Project: HBase > Issue Type: Improvement > Components: wal >Affects Versions: 0.90.2 >Reporter: Jimmy Xiang >Assignee: Jimmy Xiang >Priority: Minor > Fix For: 0.92.0 > > Attachments: > 0001-HBASE-4869-Backport-to-0.92-HBASE-4797-availability-.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4878) Master crash when splitting hlog may cause data loss
[
https://issues.apache.org/jira/browse/HBASE-4878?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13158993#comment-13158993
]
Hudson commented on HBASE-4878:
---
Integrated in HBase-0.92-security #22 (See
[https://builds.apache.org/job/HBase-0.92-security/22/])
HBASE-4878 Master crash when splitting hlog may cause data loss (Chunhui
Shen)
tedyu :
Files :
* /hbase/branches/0.92/CHANGES.txt
*
/hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLogSplitter.java
> Master crash when splitting hlog may cause data loss
>
>
> Key: HBASE-4878
> URL: https://issues.apache.org/jira/browse/HBASE-4878
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.92.0
>Reporter: chunhui shen
>Assignee: chunhui shen
> Fix For: 0.92.0, 0.94.0
>
> Attachments: hbase-4878.diff, hbase-4878v2.patch
>
>
> Let's see the code of HlogSplitter#splitLog(final FileStatus[] logfiles)
> {code}
> private List splitLog(final FileStatus[] logfiles) throws IOException {
> try {
> for (FileStatus log : logfiles) {
> parseHLog(in, logPath, entryBuffers, fs, conf, skipErrors);
> }
> archiveLogs(srcDir, corruptedLogs, processedLogs, oldLogDir, fs, conf);
> } finally {
> status.setStatus("Finishing writing output logs and closing down.");
> splits = outputSink.finishWritingAndClose();
> }
> }
> {code}
> If master is killed, after finishing archiveLogs(srcDir, corruptedLogs,
> processedLogs, oldLogDir, fs, conf),
> but before finishing splits = outputSink.finishWritingAndClose();
> Log date would loss!
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4773) HBaseAdmin may leak ZooKeeper connections
[
https://issues.apache.org/jira/browse/HBASE-4773?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13158994#comment-13158994
]
Hudson commented on HBASE-4773:
---
Integrated in HBase-0.92-security #22 (See
[https://builds.apache.org/job/HBase-0.92-security/22/])
HBASE-4773 HBaseAdmin may leak ZooKeeper connections (Xufeng)
tedyu :
Files :
* /hbase/branches/0.92/CHANGES.txt
*
/hbase/branches/0.92/src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java
> HBaseAdmin may leak ZooKeeper connections
> -
>
> Key: HBASE-4773
> URL: https://issues.apache.org/jira/browse/HBASE-4773
> Project: HBase
> Issue Type: Bug
> Components: client
>Affects Versions: 0.90.4
>Reporter: gaojinchao
>Assignee: xufeng
>Priority: Critical
> Fix For: 0.90.5
>
> Attachments: 4773.patch, branches_4773.patch, trunk_4773_patch.patch
>
>
> When master crashs, HBaseAdmin will leaks ZooKeeper connections
> I think we should close the zk connetion when throw MasterNotRunningException
> public HBaseAdmin(Configuration c)
> throws MasterNotRunningException, ZooKeeperConnectionException {
> this.conf = HBaseConfiguration.create(c);
> this.connection = HConnectionManager.getConnection(this.conf);
> this.pause = this.conf.getLong("hbase.client.pause", 1000);
> this.numRetries = this.conf.getInt("hbase.client.retries.number", 10);
> this.retryLongerMultiplier =
> this.conf.getInt("hbase.client.retries.longer.multiplier", 10);
> //we should add this code and close the zk connection
> try{
> this.connection.getMaster();
> }catch(MasterNotRunningException e){
> HConnectionManager.deleteConnection(conf, false);
> throw e;
> }
> }
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4877) TestHCM failing sporadically on jenkins and always for me on an ubuntu machine
[ https://issues.apache.org/jira/browse/HBASE-4877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13158992#comment-13158992 ] Hudson commented on HBASE-4877: --- Integrated in HBase-0.92-security #22 (See [https://builds.apache.org/job/HBase-0.92-security/22/]) HBASE-4877 TestHCM failing sporadically on jenkins and always for me on an ubuntu machine stack : Files : * /hbase/branches/0.92/CHANGES.txt * /hbase/branches/0.92/src/test/java/org/apache/hadoop/hbase/client/TestHCM.java > TestHCM failing sporadically on jenkins and always for me on an ubuntu machine > -- > > Key: HBASE-4877 > URL: https://issues.apache.org/jira/browse/HBASE-4877 > Project: HBase > Issue Type: Bug >Reporter: stack >Assignee: stack > Fix For: 0.92.0 > > Attachments: 4877.txt > > > TestHCM takes 13 minutes for me on ubuntu and fails in testClosing. It runs > fine on a mac. The problem test is not testClosing as I thought originally, > its the test just previous, testConnectionUniqueness. > testConnectionUniqueness creates the maximum cached HConnections + 10 to > verify each is unique if the passed in Configuration has a unique hash. > Problem comes when zk enforces its default max from single host of 30 > connections which is < (max cached + 10). The max does not seem to be > enforced on mac for me. The max connections runs up to max of 31 -- zk max + > 1 -- and works fine until we do the +10. On ubuntu, when we hit the zk max > of 30, we'll then go into a fail mode where we cannot set up a zk session... > each attempt takes a while. Test passes, it just takes a while. > Only, the uniqueness test does not clean up after itself and so all sessions > to zk are outstanding so then when the subsequent testClosing runs, it can't > set up connections successfully so fails. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4878) Master crash when splitting hlog may cause data loss
[
https://issues.apache.org/jira/browse/HBASE-4878?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-4878:
--
Resolution: Fixed
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
> Master crash when splitting hlog may cause data loss
>
>
> Key: HBASE-4878
> URL: https://issues.apache.org/jira/browse/HBASE-4878
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.92.0
>Reporter: chunhui shen
>Assignee: chunhui shen
> Fix For: 0.92.0, 0.94.0
>
> Attachments: hbase-4878.diff, hbase-4878v2.patch
>
>
> Let's see the code of HlogSplitter#splitLog(final FileStatus[] logfiles)
> {code}
> private List splitLog(final FileStatus[] logfiles) throws IOException {
> try {
> for (FileStatus log : logfiles) {
> parseHLog(in, logPath, entryBuffers, fs, conf, skipErrors);
> }
> archiveLogs(srcDir, corruptedLogs, processedLogs, oldLogDir, fs, conf);
> } finally {
> status.setStatus("Finishing writing output logs and closing down.");
> splits = outputSink.finishWritingAndClose();
> }
> }
> {code}
> If master is killed, after finishing archiveLogs(srcDir, corruptedLogs,
> processedLogs, oldLogDir, fs, conf),
> but before finishing splits = outputSink.finishWritingAndClose();
> Log date would loss!
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4886) truncate fails in HBase shell
[
https://issues.apache.org/jira/browse/HBASE-4886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13158975#comment-13158975
]
Hudson commented on HBASE-4886:
---
Integrated in HBase-TRUNK #2495 (See
[https://builds.apache.org/job/HBase-TRUNK/2495/])
HBASE-4886 - truncate fails in HBase shell (Lars H)
larsh :
Files :
* /hbase/trunk/src/main/ruby/hbase/admin.rb
> truncate fails in HBase shell
> -
>
> Key: HBASE-4886
> URL: https://issues.apache.org/jira/browse/HBASE-4886
> Project: HBase
> Issue Type: Bug
> Components: shell
>Affects Versions: 0.94.0
>Reporter: Lars Hofhansl
>Assignee: Lars Hofhansl
>Priority: Minor
> Fix For: 0.94.0
>
> Attachments: 4886.txt
>
>
> Seeing this in trunk:
> {noformat}
> hbase(main):001:0> truncate 'table'
> Truncating 'table' table (it may take a while):
> ERROR: wrong number of arguments (1 for 3)
> Here is some help for this command:
> Disables, drops and recreates the specified table.
> {noformat}
> ... caused by the removal of the HTable(String) constructor.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4883) TestCatalogTracker failing for me on ubuntu
[
https://issues.apache.org/jira/browse/HBASE-4883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13158974#comment-13158974
]
Hudson commented on HBASE-4883:
---
Integrated in HBase-TRUNK #2495 (See
[https://builds.apache.org/job/HBase-TRUNK/2495/])
HBASE-4883 TestCatalogTracker failing for me on ubuntu
stack :
Files :
*
/hbase/trunk/src/test/java/org/apache/hadoop/hbase/catalog/TestCatalogTracker.java
> TestCatalogTracker failing for me on ubuntu
> ---
>
> Key: HBASE-4883
> URL: https://issues.apache.org/jira/browse/HBASE-4883
> Project: HBase
> Issue Type: Bug
>Reporter: stack
> Attachments: 4883.txt, tct.txt
>
>
> {code}
> ---
> Test set: org.apache.hadoop.hbase.catalog.TestCatalogTracker
> ---
> Tests run: 9, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 5.276 sec <<<
> FAILURE!
> testNoTimeoutWaitForMeta(org.apache.hadoop.hbase.catalog.TestCatalogTracker)
> Time elapsed: 1.051 sec <<< ERROR!
> org.mockito.exceptions.misusing.WrongTypeOfReturnValue:
> Result cannot be returned by getConfiguration()
> getConfiguration() should return Configuration
> at
> org.apache.hadoop.hbase.catalog.TestCatalogTracker.testNoTimeoutWaitForMeta(TestCatalogTracker.java:378)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:30)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:68)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47)
> {code}
> The above is strange since it seems to pass on jenkins and on macosx.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-4887) Write full enum name of Compression.Algorithm into HFile
Write full enum name of Compression.Algorithm into HFile Key: HBASE-4887 URL: https://issues.apache.org/jira/browse/HBASE-4887 Project: HBase Issue Type: Improvement Reporter: Ted Yu Currently the ordinal of compression algorithms is used. This places unnecessary constraint when new compress algorithm is added. We should write full enum name of Compression.Algorithm into HFile -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Issue Comment Edited] (HBASE-4605) Constraints
[ https://issues.apache.org/jira/browse/HBASE-4605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13158139#comment-13158139 ] Ted Yu edited comment on HBASE-4605 at 11/28/11 11:47 PM: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12505280/java_HBASE-4605_v3.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 18 new or modified tests. -1 javadoc. The javadoc tool appears to have generated -162 warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 69 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hbase.master.TestRollingRestart org.apache.hadoop.hbase.util.TestRegionSplitter org.apache.hadoop.hbase.io.hfile.TestLruBlockCache Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/386//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/386//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/386//console This message is automatically generated. was (Author: hadoopqa): -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12505280/java_HBASE-4605_v3.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 18 new or modified tests. -1 javadoc. The javadoc tool appears to have generated -162 warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 69 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hbase.master.TestRollingRestart org.apache.hadoop.hbase.util.TestRegionSplitter org.apache.hadoop.hbase.io.hfile.TestLruBlockCache org.apache.hadoop.hbase.client.TestMultiParallel org.apache.hadoop.hbase.master.TestRestartCluster org.apache.hadoop.hbase.thrift2.TestThriftHBaseServiceHandler org.apache.hadoop.hbase.client.TestInstantSchemaChange org.apache.hadoop.hbase.regionserver.TestStore org.apache.hadoop.hbase.regionserver.wal.TestHLogBench org.apache.hadoop.hbase.rest.TestGzipFilter org.apache.hadoop.hbase.client.TestMetaMigrationRemovingHTD org.apache.hadoop.hbase.regionserver.TestAtomicOperation org.apache.hadoop.hbase.rest.TestScannersWithFilters org.apache.hadoop.hbase.TestInfoServers org.apache.hadoop.hbase.regionserver.TestParallelPut org.apache.hadoop.hbase.coprocessor.TestClassLoading org.apache.hadoop.hbase.client.TestAdmin org.apache.hadoop.hbase.regionserver.wal.TestLogRolling org.apache.hadoop.hbase.filter.TestColumnRangeFilter org.apache.hadoop.hbase.mapred.TestTableInputFormat org.apache.hadoop.hbase.client.TestHCM org.apache.hadoop.hbase.regionserver.TestStoreFileBlockCacheSummary org.apache.hadoop.hbase.util.hbck.TestOfflineMetaRebuildHole org.apache.hadoop.hbase.coprocessor.TestMasterObserver org.apache.hadoop.hbase.rest.TestStatusResource org.apache.hadoop.hbase.TestRegionRebalancing org.apache.hadoop.hbase.regionserver.wal.TestLogRollAbort org.apache.hadoop.hbase.rest.TestVersionResource org.apache.hadoop.hbase.client.TestScannerTimeout org.apache.hadoop.hbase.client.TestFromClientSide org.apache.hadoop.hbase.regionserver.TestFSErrorsExposed org.apache.hadoop.hbase.coprocessor.TestAggregateProtocol org.apache.hadoop.hbase.regionserver.TestSplitTransaction org.apache.hadoop.hbase.rest.TestRowResource org.apache.hadoop.hbase.rest.TestScannerResource org.apache.hadoop.hbase.ipc.TestDelayedRpc org.apache.hadoop.hbase.rest.client.TestRemoteA
[jira] [Commented] (HBASE-4605) Constraints
[
https://issues.apache.org/jira/browse/HBASE-4605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13158947#comment-13158947
]
Ted Yu commented on HBASE-4605:
---
About the changes to book.xml:
{code}
+Constraints could be also be used to ...
{code}
The above should read 'Constraints could also be used to'
{code}
where that checking enabled
{code}
The above should read 'where integrity checking is enabled'
> Constraints
> ---
>
> Key: HBASE-4605
> URL: https://issues.apache.org/jira/browse/HBASE-4605
> Project: HBase
> Issue Type: Improvement
> Components: client, coprocessors
>Affects Versions: 0.94.0
>Reporter: Jesse Yates
>Assignee: Jesse Yates
> Attachments: 4605.v7, constraint_as_cp.txt, java_Constraint_v2.patch,
> java_HBASE-4605_v1.patch, java_HBASE-4605_v2.patch, java_HBASE-4605_v3.patch
>
>
> From Jesse's comment on dev:
> {quote}
> What I would like to propose is a simple interface that people can use to
> implement a 'constraint' (matching the classic database definition). This
> would help ease of adoption by helping HBase more easily check that box, help
> minimize code duplication across organizations, and lead to easier adoption.
> Essentially, people would implement a 'Constraint' interface for checking
> keys before they are put into a table. Puts that are valid get written to the
> table, but if not people can will throw an exception that gets propagated
> back to the client explaining why the put was invalid.
> Constraints would be set on a per-table basis and the user would be expected
> to ensure the jars containing the constraint are present on the machines
> serving that table.
> Yes, people could roll their own mechanism for doing this via coprocessors
> each time, but this would make it easier to do so, so you only have to
> implement a very minimal interface and not worry about the specifics.
> {quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4120) isolation and allocation
[
https://issues.apache.org/jira/browse/HBASE-4120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13158928#comment-13158928
]
Ted Yu commented on HBASE-4120:
---
In PriorityJobQueue.java:
{code}
public void setPutPlus(int putPlus) {
{code}
Maybe putPlus supersedes previous version of this feature at your company. But
for other HBase users, such name may be confusing.
I think the method can be named setPutPriority and parameter can be named
priority.
Please remove plus from method and parameter names in PriorityJobQueue.java
For Mutation.java, I see:
{code}
public int getPriority(ActionPriorities actionPriority) {
return actionPriority.getGetPlus();
}
{code}
Is getPutPriority() be called above ? All mutations involve some kind of write.
> isolation and allocation
>
>
> Key: HBASE-4120
> URL: https://issues.apache.org/jira/browse/HBASE-4120
> Project: HBase
> Issue Type: New Feature
> Components: master, regionserver
>Affects Versions: 0.90.2, 0.90.3, 0.90.4, 0.92.0
>Reporter: Liu Jia
>Assignee: Liu Jia
> Fix For: 0.94.0
>
> Attachments: Design_document_for_HBase_isolation_and_allocation.pdf,
> Design_document_for_HBase_isolation_and_allocation_Revised.pdf,
> HBase_isolation_and_allocation_user_guide.pdf,
> Performance_of_Table_priority.pdf, System Structure.jpg, TablePriority.patch,
> TablePriority_v8.patch, TablePriority_v8.patch,
> TablePriority_v8_for_trunk.patch, TablePrioriy_v9.patch
>
>
> The HBase isolation and allocation tool is designed to help users manage
> cluster resource among different application and tables.
> When we have a large scale of HBase cluster with many applications running on
> it, there will be lots of problems. In Taobao there is a cluster for many
> departments to test their applications performance, these applications are
> based on HBase. With one cluster which has 12 servers, there will be only one
> application running exclusively on this server, and many other applications
> must wait until the previous test finished.
> After we add allocation manage function to the cluster, applications can
> share the cluster and run concurrently. Also if the Test Engineer wants to
> make sure there is no interference, he/she can move out other tables from
> this group.
> In groups we use table priority to allocate resource, when system is busy; we
> can make sure high-priority tables are not affected lower-priority tables
> Different groups can have different region server configurations, some groups
> optimized for reading can have large block cache size, and others optimized
> for writing can have large memstore size.
> Tables and region servers can be moved easily between groups; after changing
> the configuration, a group can be restarted alone instead of restarting the
> whole cluster.
> git entry : https://github.com/ICT-Ope/HBase_allocation .
> We hope our work is helpful.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4820) Distributed log splitting coding enhancement to make it easier to understand, no semantics change
[ https://issues.apache.org/jira/browse/HBASE-4820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13158926#comment-13158926 ] jirapos...@reviews.apache.org commented on HBASE-4820: -- --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2895/#review3546 --- Ship it! - Todd On 2011-11-28 19:25:01, Jimmy Xiang wrote: bq. bq. --- bq. This is an automatically generated e-mail. To reply, visit: bq. https://reviews.apache.org/r/2895/ bq. --- bq. bq. (Updated 2011-11-28 19:25:01) bq. bq. bq. Review request for hbase, Todd Lipcon and Jonathan Robie. bq. bq. bq. Summary bq. --- bq. bq. Distributed log splitting coding enhancement to make it easier to understand, no semantics change. bq. It is some issue raised during the code review in back porting this feature to CDH. bq. bq. bq. This addresses bug HBASE-4820. bq. https://issues.apache.org/jira/browse/HBASE-4820 bq. bq. bq. Diffs bq. - bq. bq.src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java 2101054 bq.src/main/java/org/apache/hadoop/hbase/master/MasterFileSystem.java d7a648d bq.src/main/java/org/apache/hadoop/hbase/master/SplitLogManager.java 7dd67e9 bq.src/main/java/org/apache/hadoop/hbase/regionserver/SplitLogWorker.java 1d329b0 bq.src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLogSplitter.java 21747b1 bq. src/test/java/org/apache/hadoop/hbase/master/TestDistributedLogSplitting.java 51daa1f bq.src/test/java/org/apache/hadoop/hbase/master/TestSplitLogManager.java c8684ec bq. bq. Diff: https://reviews.apache.org/r/2895/diff bq. bq. bq. Testing bq. --- bq. bq. Ran unit tests. Non-flaky tests are green. Two client tests didn't pass, which are not related to this change. bq. bq. bq. Thanks, bq. bq. Jimmy bq. bq. > Distributed log splitting coding enhancement to make it easier to understand, > no semantics change > - > > Key: HBASE-4820 > URL: https://issues.apache.org/jira/browse/HBASE-4820 > Project: HBase > Issue Type: Improvement > Components: wal >Affects Versions: 0.92.0, 0.94.0 >Reporter: Jimmy Xiang >Assignee: Jimmy Xiang >Priority: Minor > Labels: newbie > Fix For: 0.92.0, 0.94.0 > > Attachments: > 0001-HBASE-4820-Distributed-log-splitting-coding-enhancement-to-makeit-easier-to-understand,-no-semantics-change..patch, > > 0001-HBASE-4820-Distributed-log-splitting-coding-enhancement-to-makeit-easier-to-understand,-no-semantics-change..patch, > 0001-HBASE-4820-Minor-distributed-log-splitting-enhanceme.patch, > 0001-HBASE-4820-Minor-distributed-log-splitting-enhanceme.patch, > 0001-HBASE-4820-Minor-distributed-log-splitting-enhanceme_new.patch > > > In reviewing distributed log splitting feature, we found some cosmetic > issues. They make the code hard to understand. > It will be great to fix them. For this issue, there should be no semantic > change. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4869) Backport to 0.92: HBASE-4797 [availability] Skip recovered.edits files with edits older than what region currently has
[ https://issues.apache.org/jira/browse/HBASE-4869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jimmy Xiang updated HBASE-4869: --- Resolution: Fixed Status: Resolved (was: Patch Available) > Backport to 0.92: HBASE-4797 [availability] Skip recovered.edits files with > edits older than what region currently has > -- > > Key: HBASE-4869 > URL: https://issues.apache.org/jira/browse/HBASE-4869 > Project: HBase > Issue Type: Improvement > Components: wal >Affects Versions: 0.90.2 >Reporter: Jimmy Xiang >Assignee: Jimmy Xiang >Priority: Minor > Fix For: 0.92.0 > > Attachments: > 0001-HBASE-4869-Backport-to-0.92-HBASE-4797-availability-.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4884) Allow environment overrides for various HBase processes
[ https://issues.apache.org/jira/browse/HBASE-4884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13158910#comment-13158910 ] Hadoop QA commented on HBASE-4884: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12505391/hbase-4884.txt against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. -1 javadoc. The javadoc tool appears to have generated -162 warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 67 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hbase.client.TestAdmin org.apache.hadoop.hbase.client.TestInstantSchemaChange org.apache.hadoop.hbase.master.TestDistributedLogSplitting Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/396//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/396//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/396//console This message is automatically generated. > Allow environment overrides for various HBase processes > --- > > Key: HBASE-4884 > URL: https://issues.apache.org/jira/browse/HBASE-4884 > Project: HBase > Issue Type: Improvement > Components: shell >Reporter: Ryan Thiessen >Priority: Minor > Attachments: hbase-4884.txt > > Original Estimate: 0h > Remaining Estimate: 0h > > The current shell scripts have no mechanism for granting different > environments to various HBase subcomponents. Sometimes we want to customize > these, for example to run the thrift command with a lower HBASE_HEAPSIZE than > what we grant the regionservers. > Checking for the presence of an override file and then sourcing it if present > allows me to override this heapsize or any other variable. > Test process: > * tested with file missing. no problem, used default conf/hbase-env.sh > settings. > * tested with conf/hbase-env-thrift.sh file with heap size of 1234. ps axww > showed -Xmx1234m > * started regionserver using bin/hadoop-daemon.sh > * started hbase shell > * ran some jruby scripts via bin/hbase org.jruby.Main > * running in production for approx 2 weeks. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4878) Master crash when splitting hlog may cause data loss
[
https://issues.apache.org/jira/browse/HBASE-4878?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13158879#comment-13158879
]
Hudson commented on HBASE-4878:
---
Integrated in HBase-TRUNK #2494 (See
[https://builds.apache.org/job/HBase-TRUNK/2494/])
HBASE-4878 Master crash when splitting hlog may cause data loss (Chunhui
Shen)
tedyu :
Files :
* /hbase/trunk/CHANGES.txt
*
/hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLogSplitter.java
> Master crash when splitting hlog may cause data loss
>
>
> Key: HBASE-4878
> URL: https://issues.apache.org/jira/browse/HBASE-4878
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.92.0
>Reporter: chunhui shen
>Assignee: chunhui shen
> Fix For: 0.92.0, 0.94.0
>
> Attachments: hbase-4878.diff, hbase-4878v2.patch
>
>
> Let's see the code of HlogSplitter#splitLog(final FileStatus[] logfiles)
> {code}
> private List splitLog(final FileStatus[] logfiles) throws IOException {
> try {
> for (FileStatus log : logfiles) {
> parseHLog(in, logPath, entryBuffers, fs, conf, skipErrors);
> }
> archiveLogs(srcDir, corruptedLogs, processedLogs, oldLogDir, fs, conf);
> } finally {
> status.setStatus("Finishing writing output logs and closing down.");
> splits = outputSink.finishWritingAndClose();
> }
> }
> {code}
> If master is killed, after finishing archiveLogs(srcDir, corruptedLogs,
> processedLogs, oldLogDir, fs, conf),
> but before finishing splits = outputSink.finishWritingAndClose();
> Log date would loss!
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4885) Building against Hadoop 0.23 uses out-of-date MapReduce artifacts
[ https://issues.apache.org/jira/browse/HBASE-4885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13158881#comment-13158881 ] Hadoop QA commented on HBASE-4885: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12505401/HBASE-4885.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. -1 javadoc. The javadoc tool appears to have generated -162 warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 67 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/398//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/398//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/398//console This message is automatically generated. > Building against Hadoop 0.23 uses out-of-date MapReduce artifacts > - > > Key: HBASE-4885 > URL: https://issues.apache.org/jira/browse/HBASE-4885 > Project: HBase > Issue Type: Bug > Components: build >Reporter: Tom White >Assignee: Tom White > Fix For: 0.94.0 > > Attachments: HBASE-4885.patch > > > The "hadoop-mapred" artifacts have been replaced by "hadoop-mapreduce-*" > artifacts in 0.23 onwards. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HBASE-4886) truncate fails in HBase shell
[
https://issues.apache.org/jira/browse/HBASE-4886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lars Hofhansl resolved HBASE-4886.
--
Resolution: Fixed
Hadoop Flags: Reviewed
Committed to trunk
> truncate fails in HBase shell
> -
>
> Key: HBASE-4886
> URL: https://issues.apache.org/jira/browse/HBASE-4886
> Project: HBase
> Issue Type: Bug
> Components: shell
>Affects Versions: 0.94.0
>Reporter: Lars Hofhansl
>Assignee: Lars Hofhansl
>Priority: Minor
> Fix For: 0.94.0
>
> Attachments: 4886.txt
>
>
> Seeing this in trunk:
> {noformat}
> hbase(main):001:0> truncate 'table'
> Truncating 'table' table (it may take a while):
> ERROR: wrong number of arguments (1 for 3)
> Here is some help for this command:
> Disables, drops and recreates the specified table.
> {noformat}
> ... caused by the removal of the HTable(String) constructor.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4886) truncate fails in HBase shell
[
https://issues.apache.org/jira/browse/HBASE-4886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13158847#comment-13158847
]
Ted Yu commented on HBASE-4886:
---
+1 on patch.
> truncate fails in HBase shell
> -
>
> Key: HBASE-4886
> URL: https://issues.apache.org/jira/browse/HBASE-4886
> Project: HBase
> Issue Type: Bug
> Components: shell
>Affects Versions: 0.94.0
>Reporter: Lars Hofhansl
>Assignee: Lars Hofhansl
>Priority: Minor
> Fix For: 0.94.0
>
> Attachments: 4886.txt
>
>
> Seeing this in trunk:
> {noformat}
> hbase(main):001:0> truncate 'table'
> Truncating 'table' table (it may take a while):
> ERROR: wrong number of arguments (1 for 3)
> Here is some help for this command:
> Disables, drops and recreates the specified table.
> {noformat}
> ... caused by the removal of the HTable(String) constructor.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4885) Building against Hadoop 0.23 uses out-of-date MapReduce artifacts
[ https://issues.apache.org/jira/browse/HBASE-4885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-4885: -- Status: Patch Available (was: Open) I tried the patch on TRUNK codebase. Tests ran smoothly (I didn't finish all tests). Submit for patch testing. > Building against Hadoop 0.23 uses out-of-date MapReduce artifacts > - > > Key: HBASE-4885 > URL: https://issues.apache.org/jira/browse/HBASE-4885 > Project: HBase > Issue Type: Bug > Components: build >Reporter: Tom White >Assignee: Tom White > Fix For: 0.94.0 > > Attachments: HBASE-4885.patch > > > The "hadoop-mapred" artifacts have been replaced by "hadoop-mapreduce-*" > artifacts in 0.23 onwards. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4885) Building against Hadoop 0.23 uses out-of-date MapReduce artifacts
[ https://issues.apache.org/jira/browse/HBASE-4885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-4885: -- Fix Version/s: 0.94.0 > Building against Hadoop 0.23 uses out-of-date MapReduce artifacts > - > > Key: HBASE-4885 > URL: https://issues.apache.org/jira/browse/HBASE-4885 > Project: HBase > Issue Type: Bug > Components: build >Reporter: Tom White >Assignee: Tom White > Fix For: 0.94.0 > > Attachments: HBASE-4885.patch > > > The "hadoop-mapred" artifacts have been replaced by "hadoop-mapreduce-*" > artifacts in 0.23 onwards. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HBASE-4885) Building against Hadoop 0.23 uses out-of-date MapReduce artifacts
[ https://issues.apache.org/jira/browse/HBASE-4885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu reassigned HBASE-4885: - Assignee: Tom White > Building against Hadoop 0.23 uses out-of-date MapReduce artifacts > - > > Key: HBASE-4885 > URL: https://issues.apache.org/jira/browse/HBASE-4885 > Project: HBase > Issue Type: Bug > Components: build >Reporter: Tom White >Assignee: Tom White > Fix For: 0.94.0 > > Attachments: HBASE-4885.patch > > > The "hadoop-mapred" artifacts have been replaced by "hadoop-mapreduce-*" > artifacts in 0.23 onwards. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4862) Splitting hlog and opening region concurrently may cause data loss
[ https://issues.apache.org/jira/browse/HBASE-4862?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-4862: -- Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) > Splitting hlog and opening region concurrently may cause data loss > -- > > Key: HBASE-4862 > URL: https://issues.apache.org/jira/browse/HBASE-4862 > Project: HBase > Issue Type: Bug >Affects Versions: 0.90.2 >Reporter: chunhui shen >Assignee: chunhui shen >Priority: Critical > Fix For: 0.92.0, 0.94.0, 0.90.5 > > Attachments: 4862-0.92.txt, 4862-v6-90.txt, 4862-v6-trunk.patch, > 4862.patch, 4862.txt, hbase-4862v1 for 0.90.diff, hbase-4862v1 for 0.90.diff, > hbase-4862v1 for trunk.diff, hbase-4862v1 for trunk.diff, > hbase-4862v2for0.90.diff, hbase-4862v2fortrunk.diff, > hbase-4862v3for0.90.diff, hbase-4862v3fortrunk.diff, > hbase-4862v5for0.90.diff, hbase-4862v5fortrunk.diff, > hbase-4862v7for0.90.patch, hbase-4862v7fortrunk.patch > > > Case Description: > 1.Split hlog thread creat writer for the file region A/recoverd.edits/123456 > and is appending log entry > 2.Regionserver is opening region A now, and in the process > replayRecoveredEditsIfAny() ,it will delete the file region > A/recoverd.edits/123456 > 3.Split hlog thread catches the io exception, and stop parse this log file > and if skipError = true , add it to the corrupt logsHowever, data in > other regions in this log file will loss > 4.Or if skipError = false, it will check filesystem.Of course, the file > system is ok , and it only prints a error log, continue assigning regions. > Therefore, data in other log files will also loss!! > The case may happen in the following: > 1.Move region from server A to server B > 2.kill server A and Server B > 3.restart server A and Server B > We could prevent this exception throuth forbiding deleting recover.edits > file > which is appending by split hlog thread -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-4886) truncate fails in HBase shell
truncate fails in HBase shell
-
Key: HBASE-4886
URL: https://issues.apache.org/jira/browse/HBASE-4886
Project: HBase
Issue Type: Bug
Components: shell
Affects Versions: 0.94.0
Reporter: Lars Hofhansl
Assignee: Lars Hofhansl
Priority: Minor
Fix For: 0.94.0
Attachments: 4886.txt
Seeing this in trunk:
{noformat}
hbase(main):001:0> truncate 'table'
Truncating 'table' table (it may take a while):
ERROR: wrong number of arguments (1 for 3)
Here is some help for this command:
Disables, drops and recreates the specified table.
{noformat}
... caused by the removal of the HTable(String) constructor.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4886) truncate fails in HBase shell
[
https://issues.apache.org/jira/browse/HBASE-4886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lars Hofhansl updated HBASE-4886:
-
Attachment: 4886.txt
Simple patch to fix the problem...
> truncate fails in HBase shell
> -
>
> Key: HBASE-4886
> URL: https://issues.apache.org/jira/browse/HBASE-4886
> Project: HBase
> Issue Type: Bug
> Components: shell
>Affects Versions: 0.94.0
>Reporter: Lars Hofhansl
>Assignee: Lars Hofhansl
>Priority: Minor
> Fix For: 0.94.0
>
> Attachments: 4886.txt
>
>
> Seeing this in trunk:
> {noformat}
> hbase(main):001:0> truncate 'table'
> Truncating 'table' table (it may take a while):
> ERROR: wrong number of arguments (1 for 3)
> Here is some help for this command:
> Disables, drops and recreates the specified table.
> {noformat}
> ... caused by the removal of the HTable(String) constructor.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4862) Splitting hlog and opening region concurrently may cause data loss
[ https://issues.apache.org/jira/browse/HBASE-4862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13158835#comment-13158835 ] stack commented on HBASE-4862: -- This is integrated. Can we close it? > Splitting hlog and opening region concurrently may cause data loss > -- > > Key: HBASE-4862 > URL: https://issues.apache.org/jira/browse/HBASE-4862 > Project: HBase > Issue Type: Bug >Affects Versions: 0.90.2 >Reporter: chunhui shen >Assignee: chunhui shen >Priority: Critical > Fix For: 0.92.0, 0.94.0, 0.90.5 > > Attachments: 4862-0.92.txt, 4862-v6-90.txt, 4862-v6-trunk.patch, > 4862.patch, 4862.txt, hbase-4862v1 for 0.90.diff, hbase-4862v1 for 0.90.diff, > hbase-4862v1 for trunk.diff, hbase-4862v1 for trunk.diff, > hbase-4862v2for0.90.diff, hbase-4862v2fortrunk.diff, > hbase-4862v3for0.90.diff, hbase-4862v3fortrunk.diff, > hbase-4862v5for0.90.diff, hbase-4862v5fortrunk.diff, > hbase-4862v7for0.90.patch, hbase-4862v7fortrunk.patch > > > Case Description: > 1.Split hlog thread creat writer for the file region A/recoverd.edits/123456 > and is appending log entry > 2.Regionserver is opening region A now, and in the process > replayRecoveredEditsIfAny() ,it will delete the file region > A/recoverd.edits/123456 > 3.Split hlog thread catches the io exception, and stop parse this log file > and if skipError = true , add it to the corrupt logsHowever, data in > other regions in this log file will loss > 4.Or if skipError = false, it will check filesystem.Of course, the file > system is ok , and it only prints a error log, continue assigning regions. > Therefore, data in other log files will also loss!! > The case may happen in the following: > 1.Move region from server A to server B > 2.kill server A and Server B > 3.restart server A and Server B > We could prevent this exception throuth forbiding deleting recover.edits > file > which is appending by split hlog thread -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4883) TestCatalogTracker failing for me on ubuntu
[
https://issues.apache.org/jira/browse/HBASE-4883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13158830#comment-13158830
]
Hadoop QA commented on HBASE-4883:
--
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12505400/tct.txt
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 3 new or modified tests.
-1 patch. The patch command could not apply the patch.
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/397//console
This message is automatically generated.
> TestCatalogTracker failing for me on ubuntu
> ---
>
> Key: HBASE-4883
> URL: https://issues.apache.org/jira/browse/HBASE-4883
> Project: HBase
> Issue Type: Bug
>Reporter: stack
> Attachments: 4883.txt, tct.txt
>
>
> {code}
> ---
> Test set: org.apache.hadoop.hbase.catalog.TestCatalogTracker
> ---
> Tests run: 9, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 5.276 sec <<<
> FAILURE!
> testNoTimeoutWaitForMeta(org.apache.hadoop.hbase.catalog.TestCatalogTracker)
> Time elapsed: 1.051 sec <<< ERROR!
> org.mockito.exceptions.misusing.WrongTypeOfReturnValue:
> Result cannot be returned by getConfiguration()
> getConfiguration() should return Configuration
> at
> org.apache.hadoop.hbase.catalog.TestCatalogTracker.testNoTimeoutWaitForMeta(TestCatalogTracker.java:378)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:30)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:68)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47)
> {code}
> The above is strange since it seems to pass on jenkins and on macosx.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4820) Distributed log splitting coding enhancement to make it easier to understand, no semantics change
[ https://issues.apache.org/jira/browse/HBASE-4820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jimmy Xiang updated HBASE-4820: --- Affects Version/s: 0.92.0 Fix Version/s: 0.92.0 This patch works for both 0.94 and 0.92. TestSplitLogManager,TestDistributedLogSplitting works for both. Please commit the patch to both 0.94 and 0.92. > Distributed log splitting coding enhancement to make it easier to understand, > no semantics change > - > > Key: HBASE-4820 > URL: https://issues.apache.org/jira/browse/HBASE-4820 > Project: HBase > Issue Type: Improvement > Components: wal >Affects Versions: 0.92.0, 0.94.0 >Reporter: Jimmy Xiang >Assignee: Jimmy Xiang >Priority: Minor > Labels: newbie > Fix For: 0.92.0, 0.94.0 > > Attachments: > 0001-HBASE-4820-Distributed-log-splitting-coding-enhancement-to-makeit-easier-to-understand,-no-semantics-change..patch, > > 0001-HBASE-4820-Distributed-log-splitting-coding-enhancement-to-makeit-easier-to-understand,-no-semantics-change..patch, > 0001-HBASE-4820-Minor-distributed-log-splitting-enhanceme.patch, > 0001-HBASE-4820-Minor-distributed-log-splitting-enhanceme.patch, > 0001-HBASE-4820-Minor-distributed-log-splitting-enhanceme_new.patch > > > In reviewing distributed log splitting feature, we found some cosmetic > issues. They make the code hard to understand. > It will be great to fix them. For this issue, there should be no semantic > change. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4883) TestCatalogTracker failing for me on ubuntu
[
https://issues.apache.org/jira/browse/HBASE-4883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13158826#comment-13158826
]
stack commented on HBASE-4883:
--
I committed tct.txt, the patch that adds @Ignore to testNoTimeoutWaitForMeta
for now. Not resolving issue because test needs to be rewritten or
mockito-usage needs fixing.
> TestCatalogTracker failing for me on ubuntu
> ---
>
> Key: HBASE-4883
> URL: https://issues.apache.org/jira/browse/HBASE-4883
> Project: HBase
> Issue Type: Bug
>Reporter: stack
> Attachments: 4883.txt, tct.txt
>
>
> {code}
> ---
> Test set: org.apache.hadoop.hbase.catalog.TestCatalogTracker
> ---
> Tests run: 9, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 5.276 sec <<<
> FAILURE!
> testNoTimeoutWaitForMeta(org.apache.hadoop.hbase.catalog.TestCatalogTracker)
> Time elapsed: 1.051 sec <<< ERROR!
> org.mockito.exceptions.misusing.WrongTypeOfReturnValue:
> Result cannot be returned by getConfiguration()
> getConfiguration() should return Configuration
> at
> org.apache.hadoop.hbase.catalog.TestCatalogTracker.testNoTimeoutWaitForMeta(TestCatalogTracker.java:378)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:30)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:68)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47)
> {code}
> The above is strange since it seems to pass on jenkins and on macosx.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4885) Building against Hadoop 0.23 uses out-of-date MapReduce artifacts
[ https://issues.apache.org/jira/browse/HBASE-4885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tom White updated HBASE-4885: - Attachment: HBASE-4885.patch This patch changes the artifacts so that HBase compiles against the correct 0.23 jars. I still need to run the tests. > Building against Hadoop 0.23 uses out-of-date MapReduce artifacts > - > > Key: HBASE-4885 > URL: https://issues.apache.org/jira/browse/HBASE-4885 > Project: HBase > Issue Type: Bug > Components: build >Reporter: Tom White > Attachments: HBASE-4885.patch > > > The "hadoop-mapred" artifacts have been replaced by "hadoop-mapreduce-*" > artifacts in 0.23 onwards. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4883) TestCatalogTracker failing for me on ubuntu
[
https://issues.apache.org/jira/browse/HBASE-4883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-4883:
-
Attachment: tct.txt
Am going to disable the test that fails. I can't make it work reliably. It
passes if I run it 20 times in a row on my ubuntu box but then when I run it as
part of general build it fails again complaining about a new method being
incorrectly mocked.
Its like I have to mock all in the members of the Interface; i can't rely on
default behavior. It more seems that inserting mocks after test starts is
destabilizing the mocked Connection.
I could spend time trying to figure this but looks like a bit of a rat hole so
putting it aside for now by adding @Ignore to this one failing test
(testWaitOnMeta).
> TestCatalogTracker failing for me on ubuntu
> ---
>
> Key: HBASE-4883
> URL: https://issues.apache.org/jira/browse/HBASE-4883
> Project: HBase
> Issue Type: Bug
>Reporter: stack
> Attachments: 4883.txt, tct.txt
>
>
> {code}
> ---
> Test set: org.apache.hadoop.hbase.catalog.TestCatalogTracker
> ---
> Tests run: 9, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 5.276 sec <<<
> FAILURE!
> testNoTimeoutWaitForMeta(org.apache.hadoop.hbase.catalog.TestCatalogTracker)
> Time elapsed: 1.051 sec <<< ERROR!
> org.mockito.exceptions.misusing.WrongTypeOfReturnValue:
> Result cannot be returned by getConfiguration()
> getConfiguration() should return Configuration
> at
> org.apache.hadoop.hbase.catalog.TestCatalogTracker.testNoTimeoutWaitForMeta(TestCatalogTracker.java:378)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:30)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:68)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47)
> {code}
> The above is strange since it seems to pass on jenkins and on macosx.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-4885) Building against Hadoop 0.23 uses out-of-date MapReduce artifacts
Building against Hadoop 0.23 uses out-of-date MapReduce artifacts - Key: HBASE-4885 URL: https://issues.apache.org/jira/browse/HBASE-4885 Project: HBase Issue Type: Bug Components: build Reporter: Tom White The "hadoop-mapred" artifacts have been replaced by "hadoop-mapreduce-*" artifacts in 0.23 onwards. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-4884) Allow environment overrides for various HBase processes
Allow environment overrides for various HBase processes --- Key: HBASE-4884 URL: https://issues.apache.org/jira/browse/HBASE-4884 Project: HBase Issue Type: Improvement Components: shell Reporter: Ryan Thiessen Priority: Minor Attachments: hbase-4884.txt The current shell scripts have no mechanism for granting different environments to various HBase subcomponents. Sometimes we want to customize these, for example to run the thrift command with a lower HBASE_HEAPSIZE than what we grant the regionservers. Checking for the presence of an override file and then sourcing it if present allows me to override this heapsize or any other variable. Test process: * tested with file missing. no problem, used default conf/hbase-env.sh settings. * tested with conf/hbase-env-thrift.sh file with heap size of 1234. ps axww showed -Xmx1234m * started regionserver using bin/hadoop-daemon.sh * started hbase shell * ran some jruby scripts via bin/hbase org.jruby.Main * running in production for approx 2 weeks. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4884) Allow environment overrides for various HBase processes
[ https://issues.apache.org/jira/browse/HBASE-4884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ryan Thiessen updated HBASE-4884: - Status: Patch Available (was: Open) > Allow environment overrides for various HBase processes > --- > > Key: HBASE-4884 > URL: https://issues.apache.org/jira/browse/HBASE-4884 > Project: HBase > Issue Type: Improvement > Components: shell >Reporter: Ryan Thiessen >Priority: Minor > Attachments: hbase-4884.txt > > Original Estimate: 0h > Remaining Estimate: 0h > > The current shell scripts have no mechanism for granting different > environments to various HBase subcomponents. Sometimes we want to customize > these, for example to run the thrift command with a lower HBASE_HEAPSIZE than > what we grant the regionservers. > Checking for the presence of an override file and then sourcing it if present > allows me to override this heapsize or any other variable. > Test process: > * tested with file missing. no problem, used default conf/hbase-env.sh > settings. > * tested with conf/hbase-env-thrift.sh file with heap size of 1234. ps axww > showed -Xmx1234m > * started regionserver using bin/hadoop-daemon.sh > * started hbase shell > * ran some jruby scripts via bin/hbase org.jruby.Main > * running in production for approx 2 weeks. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4877) TestHCM failing sporadically on jenkins and always for me on an ubuntu machine
[ https://issues.apache.org/jira/browse/HBASE-4877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13158759#comment-13158759 ] Hudson commented on HBASE-4877: --- Integrated in HBase-TRUNK #2493 (See [https://builds.apache.org/job/HBase-TRUNK/2493/]) HBASE-4877 TestHCM failing sporadically on jenkins and always for me on an ubuntu machine stack : Files : * /hbase/trunk/CHANGES.txt * /hbase/trunk/src/test/java/org/apache/hadoop/hbase/client/TestHCM.java > TestHCM failing sporadically on jenkins and always for me on an ubuntu machine > -- > > Key: HBASE-4877 > URL: https://issues.apache.org/jira/browse/HBASE-4877 > Project: HBase > Issue Type: Bug >Reporter: stack >Assignee: stack > Fix For: 0.92.0 > > Attachments: 4877.txt > > > TestHCM takes 13 minutes for me on ubuntu and fails in testClosing. It runs > fine on a mac. The problem test is not testClosing as I thought originally, > its the test just previous, testConnectionUniqueness. > testConnectionUniqueness creates the maximum cached HConnections + 10 to > verify each is unique if the passed in Configuration has a unique hash. > Problem comes when zk enforces its default max from single host of 30 > connections which is < (max cached + 10). The max does not seem to be > enforced on mac for me. The max connections runs up to max of 31 -- zk max + > 1 -- and works fine until we do the +10. On ubuntu, when we hit the zk max > of 30, we'll then go into a fail mode where we cannot set up a zk session... > each attempt takes a while. Test passes, it just takes a while. > Only, the uniqueness test does not clean up after itself and so all sessions > to zk are outstanding so then when the subsequent testClosing runs, it can't > set up connections successfully so fails. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4884) Allow environment overrides for various HBase processes
[ https://issues.apache.org/jira/browse/HBASE-4884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ryan Thiessen updated HBASE-4884: - Attachment: hbase-4884.txt > Allow environment overrides for various HBase processes > --- > > Key: HBASE-4884 > URL: https://issues.apache.org/jira/browse/HBASE-4884 > Project: HBase > Issue Type: Improvement > Components: shell >Reporter: Ryan Thiessen >Priority: Minor > Attachments: hbase-4884.txt > > Original Estimate: 0h > Remaining Estimate: 0h > > The current shell scripts have no mechanism for granting different > environments to various HBase subcomponents. Sometimes we want to customize > these, for example to run the thrift command with a lower HBASE_HEAPSIZE than > what we grant the regionservers. > Checking for the presence of an override file and then sourcing it if present > allows me to override this heapsize or any other variable. > Test process: > * tested with file missing. no problem, used default conf/hbase-env.sh > settings. > * tested with conf/hbase-env-thrift.sh file with heap size of 1234. ps axww > showed -Xmx1234m > * started regionserver using bin/hadoop-daemon.sh > * started hbase shell > * ran some jruby scripts via bin/hbase org.jruby.Main > * running in production for approx 2 weeks. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4869) Backport to 0.92: HBASE-4797 [availability] Skip recovered.edits files with edits older than what region currently has
[ https://issues.apache.org/jira/browse/HBASE-4869?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13158748#comment-13158748 ] Ted Yu commented on HBASE-4869: --- Patch applied to 0.92 branch. Thanks for the patch, Jimmy. Thanks for the review Lars and Stack. > Backport to 0.92: HBASE-4797 [availability] Skip recovered.edits files with > edits older than what region currently has > -- > > Key: HBASE-4869 > URL: https://issues.apache.org/jira/browse/HBASE-4869 > Project: HBase > Issue Type: Improvement > Components: wal >Affects Versions: 0.90.2 >Reporter: Jimmy Xiang >Assignee: Jimmy Xiang >Priority: Minor > Fix For: 0.92.0 > > Attachments: > 0001-HBASE-4869-Backport-to-0.92-HBASE-4797-availability-.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4869) Backport to 0.92: HBASE-4797 [availability] Skip recovered.edits files with edits older than what region currently has
[ https://issues.apache.org/jira/browse/HBASE-4869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-4869: -- Summary: Backport to 0.92: HBASE-4797 [availability] Skip recovered.edits files with edits older than what region currently has (was: Backport to 0.92: HBASE-4797 [availability] Skip recovered.edits files with edits we know older than what region currently has.) > Backport to 0.92: HBASE-4797 [availability] Skip recovered.edits files with > edits older than what region currently has > -- > > Key: HBASE-4869 > URL: https://issues.apache.org/jira/browse/HBASE-4869 > Project: HBase > Issue Type: Improvement > Components: wal >Affects Versions: 0.90.2 >Reporter: Jimmy Xiang >Assignee: Jimmy Xiang >Priority: Minor > Fix For: 0.92.0 > > Attachments: > 0001-HBASE-4869-Backport-to-0.92-HBASE-4797-availability-.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4869) Backport to 0.92: HBASE-4797 [availability] Skip recovered.edits files with edits we know older than what region currently has.
[ https://issues.apache.org/jira/browse/HBASE-4869?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13158739#comment-13158739 ] Jimmy Xiang commented on HBASE-4869: With the hostname change, TestHRegion is green. @Ted, could you please commit it? Thanks. > Backport to 0.92: HBASE-4797 [availability] Skip recovered.edits files with > edits we know older than what region currently has. > --- > > Key: HBASE-4869 > URL: https://issues.apache.org/jira/browse/HBASE-4869 > Project: HBase > Issue Type: Improvement > Components: wal >Affects Versions: 0.90.2 >Reporter: Jimmy Xiang >Assignee: Jimmy Xiang >Priority: Minor > Fix For: 0.92.0 > > Attachments: > 0001-HBASE-4869-Backport-to-0.92-HBASE-4797-availability-.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4878) Master crash when splitting hlog may cause data loss
[
https://issues.apache.org/jira/browse/HBASE-4878?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13158711#comment-13158711
]
Ted Yu commented on HBASE-4878:
---
I ran TestHLogSplit and TestHLog which passed.
Integrated to 0.92 and TRUNK.
Thanks for the patch Chunhui.
Thanks for the review Ramkrishna.
> Master crash when splitting hlog may cause data loss
>
>
> Key: HBASE-4878
> URL: https://issues.apache.org/jira/browse/HBASE-4878
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.92.0
>Reporter: chunhui shen
>Assignee: chunhui shen
> Fix For: 0.92.0, 0.94.0
>
> Attachments: hbase-4878.diff, hbase-4878v2.patch
>
>
> Let's see the code of HlogSplitter#splitLog(final FileStatus[] logfiles)
> {code}
> private List splitLog(final FileStatus[] logfiles) throws IOException {
> try {
> for (FileStatus log : logfiles) {
> parseHLog(in, logPath, entryBuffers, fs, conf, skipErrors);
> }
> archiveLogs(srcDir, corruptedLogs, processedLogs, oldLogDir, fs, conf);
> } finally {
> status.setStatus("Finishing writing output logs and closing down.");
> splits = outputSink.finishWritingAndClose();
> }
> }
> {code}
> If master is killed, after finishing archiveLogs(srcDir, corruptedLogs,
> processedLogs, oldLogDir, fs, conf),
> but before finishing splits = outputSink.finishWritingAndClose();
> Log date would loss!
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4879) Add Constraint support to shell
[
https://issues.apache.org/jira/browse/HBASE-4879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13158708#comment-13158708
]
Jesse Yates commented on HBASE-4879:
As a follow-on ticket to 4605, this means adding the support for altering a
table more easily. Specifically, this means doing something like:
{code}
alter("table").addCoprocessor(…)
alter("table").addColumnFamily(…)
…
{code}
and then for constraints, something like:
{code}
alter("table").addConstraint(…)
alter("table").enableConstraints
…
{code}
with those methods being added dynamically by a 'constraint module' in the
jruby code. This would also serve as an example for how further shell extension
can be done in a modular manner.
This ticket may also become a more major revamp for the how the shell works,
e.g. dropping into a table context, etc as described in the discussion.
> Add Constraint support to shell
> ---
>
> Key: HBASE-4879
> URL: https://issues.apache.org/jira/browse/HBASE-4879
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 0.94.0
>Reporter: Jesse Yates
>
> Follow-on ticket to HBASE-4605. Extend the shell functionality to include
> altering a table to add a Constraint.
> Discussion around this can be found at:
> http://search-hadoop.com/m/EeYb3dezMM
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4878) Master crash when splitting hlog may cause data loss
[
https://issues.apache.org/jira/browse/HBASE-4878?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-4878:
--
Summary: Master crash when splitting hlog may cause data loss (was: Master
crash when spliting hlog may cause data loss)
> Master crash when splitting hlog may cause data loss
>
>
> Key: HBASE-4878
> URL: https://issues.apache.org/jira/browse/HBASE-4878
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.92.0
>Reporter: chunhui shen
>Assignee: chunhui shen
> Fix For: 0.92.0, 0.94.0
>
> Attachments: hbase-4878.diff, hbase-4878v2.patch
>
>
> Let's see the code of HlogSplitter#splitLog(final FileStatus[] logfiles)
> {code}
> private List splitLog(final FileStatus[] logfiles) throws IOException {
> try {
> for (FileStatus log : logfiles) {
> parseHLog(in, logPath, entryBuffers, fs, conf, skipErrors);
> }
> archiveLogs(srcDir, corruptedLogs, processedLogs, oldLogDir, fs, conf);
> } finally {
> status.setStatus("Finishing writing output logs and closing down.");
> splits = outputSink.finishWritingAndClose();
> }
> }
> {code}
> If master is killed, after finishing archiveLogs(srcDir, corruptedLogs,
> processedLogs, oldLogDir, fs, conf),
> but before finishing splits = outputSink.finishWritingAndClose();
> Log date would loss!
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4869) Backport to 0.92: HBASE-4797 [availability] Skip recovered.edits files with edits we know older than what region currently has.
[ https://issues.apache.org/jira/browse/HBASE-4869?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13158705#comment-13158705 ] Jimmy Xiang commented on HBASE-4869: I found out the root cause: HBASE-4671 > Backport to 0.92: HBASE-4797 [availability] Skip recovered.edits files with > edits we know older than what region currently has. > --- > > Key: HBASE-4869 > URL: https://issues.apache.org/jira/browse/HBASE-4869 > Project: HBase > Issue Type: Improvement > Components: wal >Affects Versions: 0.90.2 >Reporter: Jimmy Xiang >Assignee: Jimmy Xiang >Priority: Minor > Fix For: 0.92.0 > > Attachments: > 0001-HBASE-4869-Backport-to-0.92-HBASE-4797-availability-.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4883) TestCatalogTracker failing for me on ubuntu
[
https://issues.apache.org/jira/browse/HBASE-4883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13158690#comment-13158690
]
Hadoop QA commented on HBASE-4883:
--
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12505382/4883.txt
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 3 new or modified tests.
-1 patch. The patch command could not apply the patch.
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/395//console
This message is automatically generated.
> TestCatalogTracker failing for me on ubuntu
> ---
>
> Key: HBASE-4883
> URL: https://issues.apache.org/jira/browse/HBASE-4883
> Project: HBase
> Issue Type: Bug
>Reporter: stack
> Attachments: 4883.txt
>
>
> {code}
> ---
> Test set: org.apache.hadoop.hbase.catalog.TestCatalogTracker
> ---
> Tests run: 9, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 5.276 sec <<<
> FAILURE!
> testNoTimeoutWaitForMeta(org.apache.hadoop.hbase.catalog.TestCatalogTracker)
> Time elapsed: 1.051 sec <<< ERROR!
> org.mockito.exceptions.misusing.WrongTypeOfReturnValue:
> Result cannot be returned by getConfiguration()
> getConfiguration() should return Configuration
> at
> org.apache.hadoop.hbase.catalog.TestCatalogTracker.testNoTimeoutWaitForMeta(TestCatalogTracker.java:378)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:30)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:68)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47)
> {code}
> The above is strange since it seems to pass on jenkins and on macosx.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4883) TestCatalogTracker failing for me on ubuntu
[
https://issues.apache.org/jira/browse/HBASE-4883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-4883:
-
Status: Patch Available (was: Open)
> TestCatalogTracker failing for me on ubuntu
> ---
>
> Key: HBASE-4883
> URL: https://issues.apache.org/jira/browse/HBASE-4883
> Project: HBase
> Issue Type: Bug
>Reporter: stack
> Attachments: 4883.txt
>
>
> {code}
> ---
> Test set: org.apache.hadoop.hbase.catalog.TestCatalogTracker
> ---
> Tests run: 9, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 5.276 sec <<<
> FAILURE!
> testNoTimeoutWaitForMeta(org.apache.hadoop.hbase.catalog.TestCatalogTracker)
> Time elapsed: 1.051 sec <<< ERROR!
> org.mockito.exceptions.misusing.WrongTypeOfReturnValue:
> Result cannot be returned by getConfiguration()
> getConfiguration() should return Configuration
> at
> org.apache.hadoop.hbase.catalog.TestCatalogTracker.testNoTimeoutWaitForMeta(TestCatalogTracker.java:378)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:30)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:68)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47)
> {code}
> The above is strange since it seems to pass on jenkins and on macosx.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-4883) TestCatalogTracker failing for me on ubuntu
[
https://issues.apache.org/jira/browse/HBASE-4883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-4883:
-
Attachment: 4883.txt
This patch passes when I run it ten times (will run more now in a minute).
It cleans up our mock of HConnection using same pattern other tests uses and
then it adds mocks for close and locateRegion, two methods mockito was tripping
over up on this machine for some reason.
> TestCatalogTracker failing for me on ubuntu
> ---
>
> Key: HBASE-4883
> URL: https://issues.apache.org/jira/browse/HBASE-4883
> Project: HBase
> Issue Type: Bug
>Reporter: stack
> Attachments: 4883.txt
>
>
> {code}
> ---
> Test set: org.apache.hadoop.hbase.catalog.TestCatalogTracker
> ---
> Tests run: 9, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 5.276 sec <<<
> FAILURE!
> testNoTimeoutWaitForMeta(org.apache.hadoop.hbase.catalog.TestCatalogTracker)
> Time elapsed: 1.051 sec <<< ERROR!
> org.mockito.exceptions.misusing.WrongTypeOfReturnValue:
> Result cannot be returned by getConfiguration()
> getConfiguration() should return Configuration
> at
> org.apache.hadoop.hbase.catalog.TestCatalogTracker.testNoTimeoutWaitForMeta(TestCatalogTracker.java:378)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:30)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:68)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47)
> {code}
> The above is strange since it seems to pass on jenkins and on macosx.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4878) Master crash when spliting hlog may cause data loss
[
https://issues.apache.org/jira/browse/HBASE-4878?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13158685#comment-13158685
]
Ted Yu commented on HBASE-4878:
---
Looks like asf000.sp2.ygridcore.net was able to perform test suite validation.
The failed tests were due to 'Too many open files'
> Master crash when spliting hlog may cause data loss
> ---
>
> Key: HBASE-4878
> URL: https://issues.apache.org/jira/browse/HBASE-4878
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.92.0
>Reporter: chunhui shen
>Assignee: chunhui shen
> Fix For: 0.92.0, 0.94.0
>
> Attachments: hbase-4878.diff, hbase-4878v2.patch
>
>
> Let's see the code of HlogSplitter#splitLog(final FileStatus[] logfiles)
> {code}
> private List splitLog(final FileStatus[] logfiles) throws IOException {
> try {
> for (FileStatus log : logfiles) {
> parseHLog(in, logPath, entryBuffers, fs, conf, skipErrors);
> }
> archiveLogs(srcDir, corruptedLogs, processedLogs, oldLogDir, fs, conf);
> } finally {
> status.setStatus("Finishing writing output logs and closing down.");
> splits = outputSink.finishWritingAndClose();
> }
> }
> {code}
> If master is killed, after finishing archiveLogs(srcDir, corruptedLogs,
> processedLogs, oldLogDir, fs, conf),
> but before finishing splits = outputSink.finishWritingAndClose();
> Log date would loss!
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4883) TestCatalogTracker failing for me on ubuntu
[
https://issues.apache.org/jira/browse/HBASE-4883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13158681#comment-13158681
]
Ted Yu commented on HBASE-4883:
---
This line looks suspicious:
{code}
Mockito.when(connection.getRegionServerWithRetries((ServerCallable)Mockito.any())).
thenReturn(result);
{code}
There is actually a compiler warning:
{code}
Type safety: unchecked cast from Object to ServerCallable
{code}
> TestCatalogTracker failing for me on ubuntu
> ---
>
> Key: HBASE-4883
> URL: https://issues.apache.org/jira/browse/HBASE-4883
> Project: HBase
> Issue Type: Bug
>Reporter: stack
>
> {code}
> ---
> Test set: org.apache.hadoop.hbase.catalog.TestCatalogTracker
> ---
> Tests run: 9, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 5.276 sec <<<
> FAILURE!
> testNoTimeoutWaitForMeta(org.apache.hadoop.hbase.catalog.TestCatalogTracker)
> Time elapsed: 1.051 sec <<< ERROR!
> org.mockito.exceptions.misusing.WrongTypeOfReturnValue:
> Result cannot be returned by getConfiguration()
> getConfiguration() should return Configuration
> at
> org.apache.hadoop.hbase.catalog.TestCatalogTracker.testNoTimeoutWaitForMeta(TestCatalogTracker.java:378)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:30)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:68)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47)
> {code}
> The above is strange since it seems to pass on jenkins and on macosx.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4820) Distributed log splitting coding enhancement to make it easier to understand, no semantics change
[ https://issues.apache.org/jira/browse/HBASE-4820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13158664#comment-13158664 ] jirapos...@reviews.apache.org commented on HBASE-4820: -- --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/2895/ --- (Updated 2011-11-28 19:25:01.275541) Review request for hbase, Todd Lipcon and Jonathan Robie. Changes --- Latest diff, upload it again. Summary --- Distributed log splitting coding enhancement to make it easier to understand, no semantics change. It is some issue raised during the code review in back porting this feature to CDH. This addresses bug HBASE-4820. https://issues.apache.org/jira/browse/HBASE-4820 Diffs (updated) - src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java 2101054 src/main/java/org/apache/hadoop/hbase/master/MasterFileSystem.java d7a648d src/main/java/org/apache/hadoop/hbase/master/SplitLogManager.java 7dd67e9 src/main/java/org/apache/hadoop/hbase/regionserver/SplitLogWorker.java 1d329b0 src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLogSplitter.java 21747b1 src/test/java/org/apache/hadoop/hbase/master/TestDistributedLogSplitting.java 51daa1f src/test/java/org/apache/hadoop/hbase/master/TestSplitLogManager.java c8684ec Diff: https://reviews.apache.org/r/2895/diff Testing --- Ran unit tests. Non-flaky tests are green. Two client tests didn't pass, which are not related to this change. Thanks, Jimmy > Distributed log splitting coding enhancement to make it easier to understand, > no semantics change > - > > Key: HBASE-4820 > URL: https://issues.apache.org/jira/browse/HBASE-4820 > Project: HBase > Issue Type: Improvement > Components: wal >Affects Versions: 0.94.0 >Reporter: Jimmy Xiang >Assignee: Jimmy Xiang >Priority: Minor > Labels: newbie > Fix For: 0.94.0 > > Attachments: > 0001-HBASE-4820-Distributed-log-splitting-coding-enhancement-to-makeit-easier-to-understand,-no-semantics-change..patch, > > 0001-HBASE-4820-Distributed-log-splitting-coding-enhancement-to-makeit-easier-to-understand,-no-semantics-change..patch, > 0001-HBASE-4820-Minor-distributed-log-splitting-enhanceme.patch, > 0001-HBASE-4820-Minor-distributed-log-splitting-enhanceme.patch, > 0001-HBASE-4820-Minor-distributed-log-splitting-enhanceme_new.patch > > > In reviewing distributed log splitting feature, we found some cosmetic > issues. They make the code hard to understand. > It will be great to fix them. For this issue, there should be no semantic > change. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4820) Distributed log splitting coding enhancement to make it easier to understand, no semantics change
[ https://issues.apache.org/jira/browse/HBASE-4820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13158648#comment-13158648 ] Ted Yu commented on HBASE-4820: --- @Jimmy: Please upload latest patch to review board. https://builds.apache.org/job/PreCommit-HBASE-Build/367//testReport/ was incomplete. Please run log splitting related tests manually. Let's see what Kannan has to say. > Distributed log splitting coding enhancement to make it easier to understand, > no semantics change > - > > Key: HBASE-4820 > URL: https://issues.apache.org/jira/browse/HBASE-4820 > Project: HBase > Issue Type: Improvement > Components: wal >Affects Versions: 0.94.0 >Reporter: Jimmy Xiang >Assignee: Jimmy Xiang >Priority: Minor > Labels: newbie > Fix For: 0.94.0 > > Attachments: > 0001-HBASE-4820-Distributed-log-splitting-coding-enhancement-to-makeit-easier-to-understand,-no-semantics-change..patch, > > 0001-HBASE-4820-Distributed-log-splitting-coding-enhancement-to-makeit-easier-to-understand,-no-semantics-change..patch, > 0001-HBASE-4820-Minor-distributed-log-splitting-enhanceme.patch, > 0001-HBASE-4820-Minor-distributed-log-splitting-enhanceme.patch, > 0001-HBASE-4820-Minor-distributed-log-splitting-enhanceme_new.patch > > > In reviewing distributed log splitting feature, we found some cosmetic > issues. They make the code hard to understand. > It will be great to fix them. For this issue, there should be no semantic > change. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4883) TestCatalogTracker failing for me on ubuntu
[
https://issues.apache.org/jira/browse/HBASE-4883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13158647#comment-13158647
]
stack commented on HBASE-4883:
--
hmm... maybe more that 50%. Here is another failure type:
{code}
---
Test set: org.apache.hadoop.hbase.catalog.TestCatalogTracker
---
Tests run: 9, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 4.658 sec <<<
FAILURE!
testNoTimeoutWaitForMeta(org.apache.hadoop.hbase.catalog.TestCatalogTracker)
Time elapsed: 1.038 sec <<< ERROR!
org.mockito.exceptions.base.MockitoException:
'processBatch' is a *void method* and it *cannot* be stubbed with a *return
value*!
Voids are usually stubbed with Throwables:
doThrow(exception).when(mock).someVoidMethod();
If the method you are trying to stub is *overloaded* then make sure you are
calling the right overloaded version.
at
org.apache.hadoop.hbase.catalog.TestCatalogTracker.testNoTimeoutWaitForMeta(TestCatalogTracker.java:380)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
...
{code}
> TestCatalogTracker failing for me on ubuntu
> ---
>
> Key: HBASE-4883
> URL: https://issues.apache.org/jira/browse/HBASE-4883
> Project: HBase
> Issue Type: Bug
>Reporter: stack
>
> {code}
> ---
> Test set: org.apache.hadoop.hbase.catalog.TestCatalogTracker
> ---
> Tests run: 9, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 5.276 sec <<<
> FAILURE!
> testNoTimeoutWaitForMeta(org.apache.hadoop.hbase.catalog.TestCatalogTracker)
> Time elapsed: 1.051 sec <<< ERROR!
> org.mockito.exceptions.misusing.WrongTypeOfReturnValue:
> Result cannot be returned by getConfiguration()
> getConfiguration() should return Configuration
> at
> org.apache.hadoop.hbase.catalog.TestCatalogTracker.testNoTimeoutWaitForMeta(TestCatalogTracker.java:378)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:30)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:68)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47)
> {code}
> The above is strange since it seems to pass on jenkins and on macosx.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-4883) TestCatalogTracker failing for me on ubuntu
TestCatalogTracker failing for me on ubuntu
---
Key: HBASE-4883
URL: https://issues.apache.org/jira/browse/HBASE-4883
Project: HBase
Issue Type: Bug
Reporter: stack
{code}
---
Test set: org.apache.hadoop.hbase.catalog.TestCatalogTracker
---
Tests run: 9, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 5.276 sec <<<
FAILURE!
testNoTimeoutWaitForMeta(org.apache.hadoop.hbase.catalog.TestCatalogTracker)
Time elapsed: 1.051 sec <<< ERROR!
org.mockito.exceptions.misusing.WrongTypeOfReturnValue:
Result cannot be returned by getConfiguration()
getConfiguration() should return Configuration
at
org.apache.hadoop.hbase.catalog.TestCatalogTracker.testNoTimeoutWaitForMeta(TestCatalogTracker.java:378)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:30)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:68)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47)
{code}
The above is strange since it seems to pass on jenkins and on macosx.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4883) TestCatalogTracker failing for me on ubuntu
[
https://issues.apache.org/jira/browse/HBASE-4883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13158646#comment-13158646
]
stack commented on HBASE-4883:
--
On this ubuntu machine, it fails 50% of the time.
> TestCatalogTracker failing for me on ubuntu
> ---
>
> Key: HBASE-4883
> URL: https://issues.apache.org/jira/browse/HBASE-4883
> Project: HBase
> Issue Type: Bug
>Reporter: stack
>
> {code}
> ---
> Test set: org.apache.hadoop.hbase.catalog.TestCatalogTracker
> ---
> Tests run: 9, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 5.276 sec <<<
> FAILURE!
> testNoTimeoutWaitForMeta(org.apache.hadoop.hbase.catalog.TestCatalogTracker)
> Time elapsed: 1.051 sec <<< ERROR!
> org.mockito.exceptions.misusing.WrongTypeOfReturnValue:
> Result cannot be returned by getConfiguration()
> getConfiguration() should return Configuration
> at
> org.apache.hadoop.hbase.catalog.TestCatalogTracker.testNoTimeoutWaitForMeta(TestCatalogTracker.java:378)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:30)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:68)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47)
> {code}
> The above is strange since it seems to pass on jenkins and on macosx.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4120) isolation and allocation
[
https://issues.apache.org/jira/browse/HBASE-4120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13158626#comment-13158626
]
Ted Yu commented on HBASE-4120:
---
Indentation in certain classes was off.
e.g.
{code}
public static int initRegionPriority(HRegion region) {
{code}
Please use Eclipse formatter Nicloas attached to HBASE-3678 to format every
file.
{code}
private static int initRegionPriority(String region, boolean force) {
...
if (ret != null)
return ret;
{code}
Please enclose the return statement in curly braces.
> isolation and allocation
>
>
> Key: HBASE-4120
> URL: https://issues.apache.org/jira/browse/HBASE-4120
> Project: HBase
> Issue Type: New Feature
> Components: master, regionserver
>Affects Versions: 0.90.2, 0.90.3, 0.90.4, 0.92.0
>Reporter: Liu Jia
>Assignee: Liu Jia
> Fix For: 0.94.0
>
> Attachments: Design_document_for_HBase_isolation_and_allocation.pdf,
> Design_document_for_HBase_isolation_and_allocation_Revised.pdf,
> HBase_isolation_and_allocation_user_guide.pdf,
> Performance_of_Table_priority.pdf, System Structure.jpg, TablePriority.patch,
> TablePriority_v8.patch, TablePriority_v8.patch,
> TablePriority_v8_for_trunk.patch, TablePrioriy_v9.patch
>
>
> The HBase isolation and allocation tool is designed to help users manage
> cluster resource among different application and tables.
> When we have a large scale of HBase cluster with many applications running on
> it, there will be lots of problems. In Taobao there is a cluster for many
> departments to test their applications performance, these applications are
> based on HBase. With one cluster which has 12 servers, there will be only one
> application running exclusively on this server, and many other applications
> must wait until the previous test finished.
> After we add allocation manage function to the cluster, applications can
> share the cluster and run concurrently. Also if the Test Engineer wants to
> make sure there is no interference, he/she can move out other tables from
> this group.
> In groups we use table priority to allocate resource, when system is busy; we
> can make sure high-priority tables are not affected lower-priority tables
> Different groups can have different region server configurations, some groups
> optimized for reading can have large block cache size, and others optimized
> for writing can have large memstore size.
> Tables and region servers can be moved easily between groups; after changing
> the configuration, a group can be restarted alone instead of restarting the
> whole cluster.
> git entry : https://github.com/ICT-Ope/HBase_allocation .
> We hope our work is helpful.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4869) Backport to 0.92: HBASE-4797 [availability] Skip recovered.edits files with edits we know older than what region currently has.
[ https://issues.apache.org/jira/browse/HBASE-4869?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13158618#comment-13158618 ] stack commented on HBASE-4869: -- I'd think jvm version shouldn't matter. I tried this patch on ubuntu machine and it passes for me. I'm +1 on committing it. > Backport to 0.92: HBASE-4797 [availability] Skip recovered.edits files with > edits we know older than what region currently has. > --- > > Key: HBASE-4869 > URL: https://issues.apache.org/jira/browse/HBASE-4869 > Project: HBase > Issue Type: Improvement > Components: wal >Affects Versions: 0.90.2 >Reporter: Jimmy Xiang >Assignee: Jimmy Xiang >Priority: Minor > Fix For: 0.92.0 > > Attachments: > 0001-HBASE-4869-Backport-to-0.92-HBASE-4797-availability-.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HBASE-4877) TestHCM failing sporadically on jenkins and always for me on an ubuntu machine
[ https://issues.apache.org/jira/browse/HBASE-4877?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack resolved HBASE-4877. -- Resolution: Fixed Assignee: stack Hadoop Flags: Reviewed Committed trunk and branch. Thanks for reviews lads. > TestHCM failing sporadically on jenkins and always for me on an ubuntu machine > -- > > Key: HBASE-4877 > URL: https://issues.apache.org/jira/browse/HBASE-4877 > Project: HBase > Issue Type: Bug >Reporter: stack >Assignee: stack > Fix For: 0.92.0 > > Attachments: 4877.txt > > > TestHCM takes 13 minutes for me on ubuntu and fails in testClosing. It runs > fine on a mac. The problem test is not testClosing as I thought originally, > its the test just previous, testConnectionUniqueness. > testConnectionUniqueness creates the maximum cached HConnections + 10 to > verify each is unique if the passed in Configuration has a unique hash. > Problem comes when zk enforces its default max from single host of 30 > connections which is < (max cached + 10). The max does not seem to be > enforced on mac for me. The max connections runs up to max of 31 -- zk max + > 1 -- and works fine until we do the +10. On ubuntu, when we hit the zk max > of 30, we'll then go into a fail mode where we cannot set up a zk session... > each attempt takes a while. Test passes, it just takes a while. > Only, the uniqueness test does not clean up after itself and so all sessions > to zk are outstanding so then when the subsequent testClosing runs, it can't > set up connections successfully so fails. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4712) Document rules for writing tests
[ https://issues.apache.org/jira/browse/HBASE-4712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13158608#comment-13158608 ] nkeywal commented on HBASE-4712: Udpated doc, taking into account the possibility to use mvn verify for large test. Will send it to the list after a sucessful pre-integration (hadoop qa fails often these days). - 1) Running tests HBase tests are divided in three categories: small, medium and large, with corresponding JUnit categories: SmallTests, MediumTests, LargeTests. - Small tests are executed in a shared JVM. We put in this category all the tests that can be executed quickly (the maximum execution time for a test is 15 seconds, and they do not use a cluster) in a shared jvm. - Medium tests represents tests that must be executed before proposing a patch. They are designed to run in less than 30 minutes altogether, and are quite stable in their results. They're designed to last less than 50 seconds individually. They can use a cluster, and each of them is executed in a separate JVM. - Large tests are everything else. They are typically integration tests, regression tests for specific bugs, timeout tests, performance tests. Some of them can be flaky. They are executed before a commit on the pre-integration machines. They can be run on the developer machine as well. Commands are: 1) mvn test - execute small tests in a single JVM and medium tests in a separate JVM for each test - medium tests are NOT executed if there is an error in a small test - large tests are NOT executed - there is one report for small tests, and one report for medium tests -if they are executed- 2) mvn verify - execute small tests in a single JVM then medium tests in a separate JVM for each test, then large tests in a separate JVM as well. - medium tests are NOT executed if there is an error in a small test - large tests are NOT executed if there is an error in a small or medium test - there is one report by test category, small, medium and large 3) mvn test -P localTests -Dtest=myTests - remove any category effect (without this specific profile, the profiles are taken into account) - use actually the official release of surefire & the old connector to junit - tests are executed in separated JVM - You will see a new message at the end of the report: "[INFO] Tests are skipped". It's harmless. 4) mvn test -P runAllTests - execute small tests in a single JVM then medium & large tests in a separate JVM for each test - medium and large tests are NOT executed if there is an error in a small test - large tests are NOT executed if there is an error in a small or medium test - there are one report for small tests, and one report for medium & large tests -if they are executed- 5) Various other profiles - mvn test -P runSmallTests - execute small tests only, in a single JVM. - mvn test -P runMediumTests - execute medium tests in a single JVM. - mvn test -P runLargeTests - execute medium tests in a single JVM. It's as well possible to use the script 'hbasetests.sh'. This script runs the medium and large tests in parallel with two maven instances, and provide a single report. It must be executed from the directory which contains the pom.xml. Commands are: ./dev-support/hbasetests.sh - execute small and medium tests ./dev-support/hbasetests.sh runAllTests - execute all tests ./dev-support/hbasetests.sh replayFailed - rerun the failed tests a second time, in a separate jvm and without parallelisation. 2) Writing tests Tests rules & hints are: - As most as possible, tests should be written as small tests. - All tests must be written to support parallel execution on the same machine, hence should not use shared resources as fixed ports or fixed file names. - Tests should not overlog. More than 100 lines/second makes the logs complex to read and use i/o that are hence not available for the other tests. - Tests can be written with HBaseTestingUtility . This class offers helper functions to create a temp directory and do the cleanup, or to start a cluster. - Categories and execution time - All tests must be categorized, if not they could be skipped. - All tests should be written to be as fast as possible. - Small tests should last less than 15 seconds, and must not have any side effect. - Medium tests should last less than 45 seconds. - large tests should last less than 3 minutes, this ensure a good parallelization for the ones using it, and ease the analysis when the test fails. - Sleeps: - Whenever possible, tests should not use sleep, but rather waiting for the real event. This is faster and clearer for the reader. - Tests should not do a 'Thread.sleep' without testing an ending condition. This allows understanding what the test is waiting for. Moreover, the test will work whatever the machine performances. - Sleep should be minimal to be as fast as
[jira] [Commented] (HBASE-4869) Backport to 0.92: HBASE-4797 [availability] Skip recovered.edits files with edits we know older than what region currently has.
[ https://issues.apache.org/jira/browse/HBASE-4869?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13158610#comment-13158610 ] Jimmy Xiang commented on HBASE-4869: I tried HBASE-4874. It doesn't work for me. Which JDK version do you use? What's the recommend dev env? > Backport to 0.92: HBASE-4797 [availability] Skip recovered.edits files with > edits we know older than what region currently has. > --- > > Key: HBASE-4869 > URL: https://issues.apache.org/jira/browse/HBASE-4869 > Project: HBase > Issue Type: Improvement > Components: wal >Affects Versions: 0.90.2 >Reporter: Jimmy Xiang >Assignee: Jimmy Xiang >Priority: Minor > Fix For: 0.92.0 > > Attachments: > 0001-HBASE-4869-Backport-to-0.92-HBASE-4797-availability-.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4120) isolation and allocation
[ https://issues.apache.org/jira/browse/HBASE-4120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13158598#comment-13158598 ] Ted Yu commented on HBASE-4120: --- Please also categorize the new tests. See N's email on d...@hbase.apache.org, entitled 'unit tests - pom.xml & surefire changed - categories are available', for guideline. > isolation and allocation > > > Key: HBASE-4120 > URL: https://issues.apache.org/jira/browse/HBASE-4120 > Project: HBase > Issue Type: New Feature > Components: master, regionserver >Affects Versions: 0.90.2, 0.90.3, 0.90.4, 0.92.0 >Reporter: Liu Jia >Assignee: Liu Jia > Fix For: 0.94.0 > > Attachments: Design_document_for_HBase_isolation_and_allocation.pdf, > Design_document_for_HBase_isolation_and_allocation_Revised.pdf, > HBase_isolation_and_allocation_user_guide.pdf, > Performance_of_Table_priority.pdf, System Structure.jpg, TablePriority.patch, > TablePriority_v8.patch, TablePriority_v8.patch, > TablePriority_v8_for_trunk.patch, TablePrioriy_v9.patch > > > The HBase isolation and allocation tool is designed to help users manage > cluster resource among different application and tables. > When we have a large scale of HBase cluster with many applications running on > it, there will be lots of problems. In Taobao there is a cluster for many > departments to test their applications performance, these applications are > based on HBase. With one cluster which has 12 servers, there will be only one > application running exclusively on this server, and many other applications > must wait until the previous test finished. > After we add allocation manage function to the cluster, applications can > share the cluster and run concurrently. Also if the Test Engineer wants to > make sure there is no interference, he/she can move out other tables from > this group. > In groups we use table priority to allocate resource, when system is busy; we > can make sure high-priority tables are not affected lower-priority tables > Different groups can have different region server configurations, some groups > optimized for reading can have large block cache size, and others optimized > for writing can have large memstore size. > Tables and region servers can be moved easily between groups; after changing > the configuration, a group can be restarted alone instead of restarting the > whole cluster. > git entry : https://github.com/ICT-Ope/HBase_allocation . > We hope our work is helpful. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4820) Distributed log splitting coding enhancement to make it easier to understand, no semantics change
[ https://issues.apache.org/jira/browse/HBASE-4820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13158592#comment-13158592 ] Jimmy Xiang commented on HBASE-4820: Hi Kannan and Ted, Do you have any concerns with the new patch? > Distributed log splitting coding enhancement to make it easier to understand, > no semantics change > - > > Key: HBASE-4820 > URL: https://issues.apache.org/jira/browse/HBASE-4820 > Project: HBase > Issue Type: Improvement > Components: wal >Affects Versions: 0.94.0 >Reporter: Jimmy Xiang >Assignee: Jimmy Xiang >Priority: Minor > Labels: newbie > Fix For: 0.94.0 > > Attachments: > 0001-HBASE-4820-Distributed-log-splitting-coding-enhancement-to-makeit-easier-to-understand,-no-semantics-change..patch, > > 0001-HBASE-4820-Distributed-log-splitting-coding-enhancement-to-makeit-easier-to-understand,-no-semantics-change..patch, > 0001-HBASE-4820-Minor-distributed-log-splitting-enhanceme.patch, > 0001-HBASE-4820-Minor-distributed-log-splitting-enhanceme.patch, > 0001-HBASE-4820-Minor-distributed-log-splitting-enhanceme_new.patch > > > In reviewing distributed log splitting feature, we found some cosmetic > issues. They make the code hard to understand. > It will be great to fix them. For this issue, there should be no semantic > change. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4120) isolation and allocation
[ https://issues.apache.org/jira/browse/HBASE-4120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13158589#comment-13158589 ] Ted Yu commented on HBASE-4120: --- The new tests should be placed under src/test/java/org/apache/hadoop/hbase/allocation/, instead of src/test/java/org/apache/hadoop/hbase/allocation/test > isolation and allocation > > > Key: HBASE-4120 > URL: https://issues.apache.org/jira/browse/HBASE-4120 > Project: HBase > Issue Type: New Feature > Components: master, regionserver >Affects Versions: 0.90.2, 0.90.3, 0.90.4, 0.92.0 >Reporter: Liu Jia >Assignee: Liu Jia > Fix For: 0.94.0 > > Attachments: Design_document_for_HBase_isolation_and_allocation.pdf, > Design_document_for_HBase_isolation_and_allocation_Revised.pdf, > HBase_isolation_and_allocation_user_guide.pdf, > Performance_of_Table_priority.pdf, System Structure.jpg, TablePriority.patch, > TablePriority_v8.patch, TablePriority_v8.patch, > TablePriority_v8_for_trunk.patch, TablePrioriy_v9.patch > > > The HBase isolation and allocation tool is designed to help users manage > cluster resource among different application and tables. > When we have a large scale of HBase cluster with many applications running on > it, there will be lots of problems. In Taobao there is a cluster for many > departments to test their applications performance, these applications are > based on HBase. With one cluster which has 12 servers, there will be only one > application running exclusively on this server, and many other applications > must wait until the previous test finished. > After we add allocation manage function to the cluster, applications can > share the cluster and run concurrently. Also if the Test Engineer wants to > make sure there is no interference, he/she can move out other tables from > this group. > In groups we use table priority to allocate resource, when system is busy; we > can make sure high-priority tables are not affected lower-priority tables > Different groups can have different region server configurations, some groups > optimized for reading can have large block cache size, and others optimized > for writing can have large memstore size. > Tables and region servers can be moved easily between groups; after changing > the configuration, a group can be restarted alone instead of restarting the > whole cluster. > git entry : https://github.com/ICT-Ope/HBase_allocation . > We hope our work is helpful. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4773) HBaseAdmin may leak ZooKeeper connections
[
https://issues.apache.org/jira/browse/HBASE-4773?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13158581#comment-13158581
]
Hudson commented on HBASE-4773:
---
Integrated in HBase-TRUNK #2492 (See
[https://builds.apache.org/job/HBase-TRUNK/2492/])
HBASE-4773 HBaseAdmin may leak ZooKeeper connections (Xufeng)
tedyu :
Files :
* /hbase/trunk/CHANGES.txt
* /hbase/trunk/src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java
> HBaseAdmin may leak ZooKeeper connections
> -
>
> Key: HBASE-4773
> URL: https://issues.apache.org/jira/browse/HBASE-4773
> Project: HBase
> Issue Type: Bug
> Components: client
>Affects Versions: 0.90.4
>Reporter: gaojinchao
>Assignee: xufeng
>Priority: Critical
> Fix For: 0.90.5
>
> Attachments: 4773.patch, branches_4773.patch, trunk_4773_patch.patch
>
>
> When master crashs, HBaseAdmin will leaks ZooKeeper connections
> I think we should close the zk connetion when throw MasterNotRunningException
> public HBaseAdmin(Configuration c)
> throws MasterNotRunningException, ZooKeeperConnectionException {
> this.conf = HBaseConfiguration.create(c);
> this.connection = HConnectionManager.getConnection(this.conf);
> this.pause = this.conf.getLong("hbase.client.pause", 1000);
> this.numRetries = this.conf.getInt("hbase.client.retries.number", 10);
> this.retryLongerMultiplier =
> this.conf.getInt("hbase.client.retries.longer.multiplier", 10);
> //we should add this code and close the zk connection
> try{
> this.connection.getMaster();
> }catch(MasterNotRunningException e){
> HConnectionManager.deleteConnection(conf, false);
> throw e;
> }
> }
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-4882) Shell enhancements - getting description of row (list 'T1','R1')
Shell enhancements - getting description of row (list 'T1','R1') Key: HBASE-4882 URL: https://issues.apache.org/jira/browse/HBASE-4882 Project: HBase Issue Type: Improvement Components: shell Affects Versions: 0.90.2 Reporter: Mikael Sitruk Fix For: 0.90.2 In case a row keep a lot of columns, invoking the get operation on the key will display too much information (without scrolling possibility) The idea here is not to display the content of the cells, but the name of each columns. for example list 'T1','R1' will display the columns of R1 of T1 C1, C2, C3, C4, C5. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HBASE-4773) HBaseAdmin may leak ZooKeeper connections
[
https://issues.apache.org/jira/browse/HBASE-4773?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu reassigned HBASE-4773:
-
Assignee: xufeng
> HBaseAdmin may leak ZooKeeper connections
> -
>
> Key: HBASE-4773
> URL: https://issues.apache.org/jira/browse/HBASE-4773
> Project: HBase
> Issue Type: Bug
> Components: client
>Affects Versions: 0.90.4
>Reporter: gaojinchao
>Assignee: xufeng
>Priority: Critical
> Fix For: 0.90.5
>
> Attachments: 4773.patch, branches_4773.patch, trunk_4773_patch.patch
>
>
> When master crashs, HBaseAdmin will leaks ZooKeeper connections
> I think we should close the zk connetion when throw MasterNotRunningException
> public HBaseAdmin(Configuration c)
> throws MasterNotRunningException, ZooKeeperConnectionException {
> this.conf = HBaseConfiguration.create(c);
> this.connection = HConnectionManager.getConnection(this.conf);
> this.pause = this.conf.getLong("hbase.client.pause", 1000);
> this.numRetries = this.conf.getInt("hbase.client.retries.number", 10);
> this.retryLongerMultiplier =
> this.conf.getInt("hbase.client.retries.longer.multiplier", 10);
> //we should add this code and close the zk connection
> try{
> this.connection.getMaster();
> }catch(MasterNotRunningException e){
> HConnectionManager.deleteConnection(conf, false);
> throw e;
> }
> }
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4773) HBaseAdmin may leak ZooKeeper connections
[
https://issues.apache.org/jira/browse/HBASE-4773?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13158569#comment-13158569
]
Ted Yu commented on HBASE-4773:
---
Integrated to 0.90, 0.92 and TRUNK.
Thanks for the patch Xufeng.
Thanks for the review Jinchao and Ramkrishna.
> HBaseAdmin may leak ZooKeeper connections
> -
>
> Key: HBASE-4773
> URL: https://issues.apache.org/jira/browse/HBASE-4773
> Project: HBase
> Issue Type: Bug
> Components: client
>Affects Versions: 0.90.4
>Reporter: gaojinchao
>Priority: Critical
> Fix For: 0.90.5
>
> Attachments: 4773.patch, branches_4773.patch, trunk_4773_patch.patch
>
>
> When master crashs, HBaseAdmin will leaks ZooKeeper connections
> I think we should close the zk connetion when throw MasterNotRunningException
> public HBaseAdmin(Configuration c)
> throws MasterNotRunningException, ZooKeeperConnectionException {
> this.conf = HBaseConfiguration.create(c);
> this.connection = HConnectionManager.getConnection(this.conf);
> this.pause = this.conf.getLong("hbase.client.pause", 1000);
> this.numRetries = this.conf.getInt("hbase.client.retries.number", 10);
> this.retryLongerMultiplier =
> this.conf.getInt("hbase.client.retries.longer.multiplier", 10);
> //we should add this code and close the zk connection
> try{
> this.connection.getMaster();
> }catch(MasterNotRunningException e){
> HConnectionManager.deleteConnection(conf, false);
> throw e;
> }
> }
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4847) Activate single jvm for small tests on jenkins
[ https://issues.apache.org/jira/browse/HBASE-4847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13158565#comment-13158565 ] Hadoop QA commented on HBASE-4847: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12505349/4847_all.v5.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 9 new or modified tests. -1 javadoc. The javadoc tool appears to have generated -162 warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 67 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/394//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/394//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/394//console This message is automatically generated. > Activate single jvm for small tests on jenkins > -- > > Key: HBASE-4847 > URL: https://issues.apache.org/jira/browse/HBASE-4847 > Project: HBase > Issue Type: Improvement > Components: build, test >Affects Versions: 0.94.0 > Environment: build >Reporter: nkeywal >Assignee: nkeywal >Priority: Minor > Attachments: 4847_all.v4.patch, 4847_all.v5.patch, 4847_pom.patch, > 4847_pom.v2.patch, 4847_pom.v2.patch, 4847_pom.v2.patch, 4847_pom.v3.patch > > > This will not revolutionate performances alone. We will win between 1 to 4 > minutes. > But we win as well: > - it's a step for parallelizing the tests > - new tests are less expensive as they do not create a new jvm: it's a > continuous win > - it will allow to push it on dev env while having the same env on dev & on > build, and 3 minutes are 10% of small + medium tests execution time. > I will do a few "submit patch" to see if it works well before asking for the > real commit. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
