[jira] [Updated] (HBASE-9002) TestDistributedLogSplitting.testRecoveredEdits fails
[
https://issues.apache.org/jira/browse/HBASE-9002?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jeffrey Zhong updated HBASE-9002:
-
Attachment: hbase-9002.patch
TestDistributedLogSplitting.testRecoveredEdits fails
Key: HBASE-9002
URL: https://issues.apache.org/jira/browse/HBASE-9002
Project: HBase
Issue Type: Bug
Components: test
Reporter: stack
Fix For: 0.95.2
Attachments: hbase-9002.patch
https://builds.apache.org/job/hbase-0.95/342/testReport/org.apache.hadoop.hbase.master/TestDistributedLogSplitting/testRecoveredEdits/
{code}
java.lang.AssertionError: expected:1000 but was:0
at org.junit.Assert.fail(Assert.java:88)
at org.junit.Assert.failNotEquals(Assert.java:743)
at org.junit.Assert.assertEquals(Assert.java:118)
at org.junit.Assert.assertEquals(Assert.java:555)
at org.junit.Assert.assertEquals(Assert.java:542)
at
org.apache.hadoop.hbase.master.TestDistributedLogSplitting.testRecoveredEdits(TestDistributedLogSplitting.java:209)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at
org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74)
{code}
[~jxiang] or [~jeffreyz] want to take a look at this one?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-9002) TestDistributedLogSplitting.testRecoveredEdits fails
[
https://issues.apache.org/jira/browse/HBASE-9002?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jeffrey Zhong updated HBASE-9002:
-
Assignee: Jeffrey Zhong
Status: Patch Available (was: Open)
TestDistributedLogSplitting.testRecoveredEdits fails
Key: HBASE-9002
URL: https://issues.apache.org/jira/browse/HBASE-9002
Project: HBase
Issue Type: Bug
Components: test
Reporter: stack
Assignee: Jeffrey Zhong
Fix For: 0.95.2
Attachments: hbase-9002.patch
https://builds.apache.org/job/hbase-0.95/342/testReport/org.apache.hadoop.hbase.master/TestDistributedLogSplitting/testRecoveredEdits/
{code}
java.lang.AssertionError: expected:1000 but was:0
at org.junit.Assert.fail(Assert.java:88)
at org.junit.Assert.failNotEquals(Assert.java:743)
at org.junit.Assert.assertEquals(Assert.java:118)
at org.junit.Assert.assertEquals(Assert.java:555)
at org.junit.Assert.assertEquals(Assert.java:542)
at
org.apache.hadoop.hbase.master.TestDistributedLogSplitting.testRecoveredEdits(TestDistributedLogSplitting.java:209)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at
org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74)
{code}
[~jxiang] or [~jeffreyz] want to take a look at this one?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-4955) Use the official versions of surefire junit
[ https://issues.apache.org/jira/browse/HBASE-4955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13714945#comment-13714945 ] Nicolas Liochon commented on HBASE-4955: On the precommit env, there is only one application tested at a time. So the load should be limited. Or more exactly, this load is suspicious. IIRC there are some improvements in the last surefire versions, but I can't say if they will work for us. Use the official versions of surefire junit - Key: HBASE-4955 URL: https://issues.apache.org/jira/browse/HBASE-4955 Project: HBase Issue Type: Improvement Components: test Affects Versions: 0.94.0 Environment: all Reporter: Nicolas Liochon Assignee: Nicolas Liochon Priority: Critical Attachments: 4955.v1.patch, 4955.v2.patch, 4955.v2.patch, 4955.v2.patch, 4955.v2.patch, 4955.v3.patch, 4955.v3.patch, 4955.v3.patch, 4955.v4.patch, 4955.v4.patch, 4955.v4.patch, 4955.v4.patch, 4955.v4.patch, 4955.v4.patch, 4955.v5.patch, 4955.v6.patch, 8204.v4.patch We currently use private versions for Surefire JUnit since HBASE-4763. This JIRA traks what we need to move to official versions. Surefire 2.11 is just out, but, after some tests, it does not contain all what we need. JUnit. Could be for JUnit 4.11. Issue to monitor: https://github.com/KentBeck/junit/issues/359: fixed in our version, no feedback for an integration on trunk Surefire: Could be for Surefire 2.12. Issues to monitor are: 329 (category support): fixed, we use the official implementation from the trunk 786 (@Category with forkMode=always): fixed, we use the official implementation from the trunk 791 (incorrect elapsed time on test failure): fixed, we use the official implementation from the trunk 793 (incorrect time in the XML report): Not fixed (reopen) on trunk, fixed on our version. 760 (does not take into account the test method): fixed in trunk, not fixed in our version 798 (print immediately the test class name): not fixed in trunk, not fixed in our version 799 (Allow test parallelization when forkMode=always): not fixed in trunk, not fixed in our version 800 (redirectTestOutputToFile not taken into account): not yet fix on trunk, fixed on our version 800 793 are the more important to monitor, it's the only ones that are fixed in our version but not on trunk. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8705) RS holding META when restarted in a single node setup may hang infinitely without META assignment
[
https://issues.apache.org/jira/browse/HBASE-8705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13714954#comment-13714954
]
rajeshbabu commented on HBASE-8705:
---
[~ram_krish],[~jxiang]
if we stop all region servers by hbase-deamon.sh stop command then .META. is
never online with 10 seconds delay in starting a region server.
In single node cluster, many times .META. in transition after stopping the RS.
Can we retry meta assignment even after max attempts until we find some online
region server. Otherwise we need to run hbck everytime in this common case.
What do you say?
RS holding META when restarted in a single node setup may hang infinitely
without META assignment
-
Key: HBASE-8705
URL: https://issues.apache.org/jira/browse/HBASE-8705
Project: HBase
Issue Type: Bug
Affects Versions: 0.95.1
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan
Priority: Minor
Fix For: 0.98.0, 0.95.2
Attachments: HBASE-8705_1.patch, HBASE-8705_2.patch, HBASE-8705.patch
This bug may be minor as it likely to happen in a single node setup.
I restarted the RS holding META. The master tried assigning META using
MetaSSH. But tried this before the new RS came up.
So as not region plan is found
{code}
if (plan == null) {
LOG.warn(Unable to determine a plan to assign + region);
if (tomActivated){
this.timeoutMonitor.setAllRegionServersOffline(true);
} else {
regionStates.updateRegionState(region,
RegionState.State.FAILED_OPEN);
}
return;
}
{code}
we just return without assigment. And this being the META the small cluster
just hangs.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8414) Hbck still refers to -ROOT- table to locate .META.
[
https://issues.apache.org/jira/browse/HBASE-8414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13714957#comment-13714957
]
rajeshbabu commented on HBASE-8414:
---
[~himan...@cloudera.com]
I have recently faced this problem and prepared patch with test case.
If you are not working on this I will upload.
Hbck still refers to -ROOT- table to locate .META.
--
Key: HBASE-8414
URL: https://issues.apache.org/jira/browse/HBASE-8414
Project: HBase
Issue Type: Bug
Components: hbck
Affects Versions: 0.95.0
Reporter: Himanshu Vashishtha
Assignee: Himanshu Vashishtha
In the current ROOT-less trunk, hbck still tries to fix meta by looking its
location in the .ROOT. table. This happens if there is no .META. assigned
when hbck is ran.
HbaseFsck.java:
{code}
boolean checkMetaRegion() {
...
HRegionLocation rootLocation = connection.locateRegion(
HConstants.ROOT_TABLE_NAME, HConstants.EMPTY_START_ROW);
...
}
{code}
Running hbck while meta is in transition:
{code}
bin/hbase hbck
Version: 0.95.0-SNAPSHOT
ERROR: META region or some of its attributes are null.
ERROR: Fatal error: unable to get root region location. Exiting...
Summary:
2 inconsistencies detected.
Status: INCONSISTENT
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8414) Hbck still refers to -ROOT- table to locate .META.
[
https://issues.apache.org/jira/browse/HBASE-8414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13714963#comment-13714963
]
Anoop Sam John commented on HBASE-8414:
---
Rajesh
Have a look at HBASE-8627. I have uploaded a patch there which solves
this issue also I guess. Can u pls test that once?
Hbck still refers to -ROOT- table to locate .META.
--
Key: HBASE-8414
URL: https://issues.apache.org/jira/browse/HBASE-8414
Project: HBase
Issue Type: Bug
Components: hbck
Affects Versions: 0.95.0
Reporter: Himanshu Vashishtha
Assignee: Himanshu Vashishtha
In the current ROOT-less trunk, hbck still tries to fix meta by looking its
location in the .ROOT. table. This happens if there is no .META. assigned
when hbck is ran.
HbaseFsck.java:
{code}
boolean checkMetaRegion() {
...
HRegionLocation rootLocation = connection.locateRegion(
HConstants.ROOT_TABLE_NAME, HConstants.EMPTY_START_ROW);
...
}
{code}
Running hbck while meta is in transition:
{code}
bin/hbase hbck
Version: 0.95.0-SNAPSHOT
ERROR: META region or some of its attributes are null.
ERROR: Fatal error: unable to get root region location. Exiting...
Summary:
2 inconsistencies detected.
Status: INCONSISTENT
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8627) HBCK can not fix meta not assigned issue
[ https://issues.apache.org/jira/browse/HBASE-8627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13714966#comment-13714966 ] Anoop Sam John commented on HBASE-8627: --- You mean test cases for that scenarios also [~jxiang]? Sorry for the late reply. May be I need to remove those code in deleteMetaRegion() which handles regionInfoOnly and hdfs. What do you say? Is this test case enough for this issue? This patch solves what is mentioned in HBASE-8414 also HBCK can not fix meta not assigned issue Key: HBASE-8627 URL: https://issues.apache.org/jira/browse/HBASE-8627 Project: HBase Issue Type: Bug Components: hbck Affects Versions: 0.95.0 Reporter: Anoop Sam John Assignee: Anoop Sam John Attachments: HBASE-8627_Trunk.patch, HBASE-8627_Trunk-V2.patch When meta table region is not assigned to any RS, HBCK run will get exception. I can see code added in checkMetaRegion() to solve this issue but it wont work. It still refers to ROOT region! -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8780) a column Family can have VERSIONS less than zero
[
https://issues.apache.org/jira/browse/HBASE-8780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13714968#comment-13714968
]
Anoop Sam John commented on HBASE-8780:
---
V2 lgtm. Test cases are passing now.
a column Family can have VERSIONS less than zero
-
Key: HBASE-8780
URL: https://issues.apache.org/jira/browse/HBASE-8780
Project: HBase
Issue Type: Bug
Components: Client
Affects Versions: 0.94.8
Reporter: Demai Ni
Assignee: Demai Ni
Priority: Trivial
Attachments: HBASE-8780-0.94.8-v0.patch, HBASE-8780-test.txt,
HBASE-8780-trunk.patch, HBASE-8780-trunk.patch, HBASE-8780-trunk-v1.patch,
HBASE-8780-trunk-v2.patch
User can create/alter a columnfam and set its VERSION(aka maxVERSIONS) to a
negative or zero value. Although there is a checking in
HColumnDesciptor#construtor, hbase shell command will invoke the
setter(setMaxVersions and setMinVersions) directly, hence by pass the
checking. For example:
{code:title=set VERSIONS = -1}
hbase(main):016:0 create 't5_dn',{NAME='cf1',VERSIONS=-1}
0 row(s) in 1.0420 seconds
hbase(main):017:0 put 't5_dn','row1','cf1:q1','row1cf1_v1'
0 row(s) in 0.0700 seconds
hbase(main):018:0 scan 't5_dn'
ROW COLUMN+CELL
0 row(s) in 0.0090 seconds
hbase(main):019:0 describe 't5_dn'
DESCRIPTION ENABLED
't5_dn', {NAME = 'cf1', REPLICATION_SCOPE = '0', true
KEEP_DELETED_CELLS = 'false', COMPRESSION = 'NONE
', ENCODE_ON_DISK = 'true', BLOCKCACHE = 'true',
MIN_VERSIONS = '0', DATA_BLOCK_ENCODING = 'NONE',
IN_MEMORY = 'false', BLOOMFILTER = 'NONE', TTL =
'2147483647', VERSIONS = '-1', BLOCKSIZE = '655
36'}
1 row(s) in 0.0410 seconds
{code}
above example shows VERSIONS = '-1', and put/scan doesn't keep the data
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8414) Hbck still refers to -ROOT- table to locate .META.
[
https://issues.apache.org/jira/browse/HBASE-8414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13714969#comment-13714969
]
rajeshbabu commented on HBASE-8414:
---
[~anoop.hbase]
Sorry I didnt see that. I will test it. Thanks.
Hbck still refers to -ROOT- table to locate .META.
--
Key: HBASE-8414
URL: https://issues.apache.org/jira/browse/HBASE-8414
Project: HBase
Issue Type: Bug
Components: hbck
Affects Versions: 0.95.0
Reporter: Himanshu Vashishtha
Assignee: Himanshu Vashishtha
In the current ROOT-less trunk, hbck still tries to fix meta by looking its
location in the .ROOT. table. This happens if there is no .META. assigned
when hbck is ran.
HbaseFsck.java:
{code}
boolean checkMetaRegion() {
...
HRegionLocation rootLocation = connection.locateRegion(
HConstants.ROOT_TABLE_NAME, HConstants.EMPTY_START_ROW);
...
}
{code}
Running hbck while meta is in transition:
{code}
bin/hbase hbck
Version: 0.95.0-SNAPSHOT
ERROR: META region or some of its attributes are null.
ERROR: Fatal error: unable to get root region location. Exiting...
Summary:
2 inconsistencies detected.
Status: INCONSISTENT
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8947) Thrift 2 : Replace bool writeToWAL with TDurability durability
[
https://issues.apache.org/jira/browse/HBASE-8947?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13715001#comment-13715001
]
Hamed Madani commented on HBASE-8947:
-
For 0.94 we can drop the default for writeToWAL to avoid sending an extra
field. I just double checked and for 0.94
{code}
private boolean writeToWAL = true;
{code}
writeToWAL get initialized to true , so I suggest we drop default value for
writeToWAL, add Durability to Mutation (Apparently Increment doesn't support it
since it doesn't extend Mutation in 0.94), then as you suggested we check for
isSetDurability, if not set then isSetwriteToWAL and if not set then just don't
set writeToWAL since it gets initialized to true inside Mutation class. What do
you think ? Also do we need a test for this ? I wasn't sure how to test it.
Thrift 2 : Replace bool writeToWAL with TDurability durability
---
Key: HBASE-8947
URL: https://issues.apache.org/jira/browse/HBASE-8947
Project: HBase
Issue Type: Sub-task
Components: Thrift
Reporter: Hamed Madani
Assignee: Hamed Madani
Attachments: HBASE-8947.patch, HBASE-8947-v2.patch
Introduce new enum *TDurability* to expose more options for *Write To Wal.*
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9002) TestDistributedLogSplitting.testRecoveredEdits fails
[
https://issues.apache.org/jira/browse/HBASE-9002?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13715006#comment-13715006
]
Hadoop QA commented on HBASE-9002:
--
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12593455/hbase-9002.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified tests.
{color:green}+1 hadoop1.0{color}. The patch compiles against the hadoop
1.0 profile.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/6422//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6422//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6422//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6422//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6422//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6422//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6422//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6422//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6422//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/6422//console
This message is automatically generated.
TestDistributedLogSplitting.testRecoveredEdits fails
Key: HBASE-9002
URL: https://issues.apache.org/jira/browse/HBASE-9002
Project: HBase
Issue Type: Bug
Components: test
Reporter: stack
Assignee: Jeffrey Zhong
Fix For: 0.95.2
Attachments: hbase-9002.patch
https://builds.apache.org/job/hbase-0.95/342/testReport/org.apache.hadoop.hbase.master/TestDistributedLogSplitting/testRecoveredEdits/
{code}
java.lang.AssertionError: expected:1000 but was:0
at org.junit.Assert.fail(Assert.java:88)
at org.junit.Assert.failNotEquals(Assert.java:743)
at org.junit.Assert.assertEquals(Assert.java:118)
at org.junit.Assert.assertEquals(Assert.java:555)
at org.junit.Assert.assertEquals(Assert.java:542)
at
org.apache.hadoop.hbase.master.TestDistributedLogSplitting.testRecoveredEdits(TestDistributedLogSplitting.java:209)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at
org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74)
{code}
[~jxiang] or [~jeffreyz] want to take a look at this one?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA,
[jira] [Commented] (HBASE-8627) HBCK can not fix meta not assigned issue
[
https://issues.apache.org/jira/browse/HBASE-8627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13715035#comment-13715035
]
rajeshbabu commented on HBASE-8627:
---
[~anoop.hbase]
If meta server znode is deleted(in metaSSH), then patch is not able to fix this
issue.
{code}
public void assignMeta() throws KeeperException {
MetaRegionTracker.deleteMetaLocation(this.watcher);
assign(HRegionInfo.FIRST_META_REGIONINFO, true);
}
{code}
In recordMetaRegion we will read meta location from zk, in this case we will
get false and returing without fixing.
{code}
// get regions according to what is online on each RegionServer
loadDeployedRegions();
// check whether .META. is deployed and online
if (!recordMetaRegion()) {
// Will remove later if we can fix it
errors.reportError(Fatal error: unable to get .META. region location.
Exiting...);
return -2;
}
{code}
I think in checkMetaRegion only we need to read from zk and deployed regions
list. If it is not present in any case we need to deploy it.
Here is the test to reproduce the scenario. FYI
{code}
@Test
public void testFixAssignmentsWhenMETAinTransition() throws Exception {
MiniHBaseCluster cluster = TEST_UTIL.getHBaseCluster();
HBaseAdmin admin = null;
try {
admin = new HBaseAdmin(TEST_UTIL.getConfiguration());
admin.closeRegion(cluster.getServerHoldingMeta(),
HRegionInfo.FIRST_META_REGIONINFO);
} finally {
if (admin != null) {
admin.close();
}
}
regionStates.regionOffline(HRegionInfo.FIRST_META_REGIONINFO);
MetaRegionTracker.deleteMetaLocation(cluster.getMaster().getZooKeeper());
assertFalse(regionStates
.isRegionAssigned(HRegionInfo.FIRST_META_REGIONINFO));
HBaseFsck hbck = doFsck(conf, true);
assertErrors(hbck, new ERROR_CODE[] { ERROR_CODE.NULL_META_REGION,
ERROR_CODE.UNKNOWN });
assertNoErrors(doFsck(conf, false));
}
{code}
HBCK can not fix meta not assigned issue
Key: HBASE-8627
URL: https://issues.apache.org/jira/browse/HBASE-8627
Project: HBase
Issue Type: Bug
Components: hbck
Affects Versions: 0.95.0
Reporter: Anoop Sam John
Assignee: Anoop Sam John
Attachments: HBASE-8627_Trunk.patch, HBASE-8627_Trunk-V2.patch
When meta table region is not assigned to any RS, HBCK run will get
exception. I can see code added in checkMetaRegion() to solve this issue but
it wont work. It still refers to ROOT region!
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8705) RS holding META when restarted in a single node setup may hang infinitely without META assignment
[
https://issues.apache.org/jira/browse/HBASE-8705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13715064#comment-13715064
]
ramkrishna.s.vasudevan commented on HBASE-8705:
---
In single node cluster, many times .META. in transition after stopping the RS.
Is there any easy way to identify a single node cluster? Or you mean to say if
online servers are 0 keep waiting till it becomes atleast 1?
RS holding META when restarted in a single node setup may hang infinitely
without META assignment
-
Key: HBASE-8705
URL: https://issues.apache.org/jira/browse/HBASE-8705
Project: HBase
Issue Type: Bug
Affects Versions: 0.95.1
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan
Priority: Minor
Fix For: 0.98.0, 0.95.2
Attachments: HBASE-8705_1.patch, HBASE-8705_2.patch, HBASE-8705.patch
This bug may be minor as it likely to happen in a single node setup.
I restarted the RS holding META. The master tried assigning META using
MetaSSH. But tried this before the new RS came up.
So as not region plan is found
{code}
if (plan == null) {
LOG.warn(Unable to determine a plan to assign + region);
if (tomActivated){
this.timeoutMonitor.setAllRegionServersOffline(true);
} else {
regionStates.updateRegionState(region,
RegionState.State.FAILED_OPEN);
}
return;
}
{code}
we just return without assigment. And this being the META the small cluster
just hangs.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8705) RS holding META when restarted in a single node setup may hang infinitely without META assignment
[
https://issues.apache.org/jira/browse/HBASE-8705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13715076#comment-13715076
]
rajeshbabu commented on HBASE-8705:
---
I mean we can retry until we find one online server to assign META.
After reaching max attempts if we reset i value to 0(only for META region),
then we can check for online servers even after max attempts. If any one RS is
online then META will be assinged.
{code}
if (plan == null) {
LOG.warn(Unable to determine a plan to assign + region);
if (tomActivated){
this.timeoutMonitor.setAllRegionServersOffline(true);
} else {
if (region.isMetaRegion()) {
try {
if (i == maximumAttempts) {
i = 0;
}
Thread.sleep(this.sleepTimeBeforeRetryingMetaAssignment);
continue;
} catch (InterruptedException e) {
LOG.error(Got exception while waiting for META assignment);
Thread.currentThread().interrupt();
}
}
regionStates.updateRegionState(region, RegionState.State.FAILED_OPEN);
}
return;
}
{code}
RS holding META when restarted in a single node setup may hang infinitely
without META assignment
-
Key: HBASE-8705
URL: https://issues.apache.org/jira/browse/HBASE-8705
Project: HBase
Issue Type: Bug
Affects Versions: 0.95.1
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan
Priority: Minor
Fix For: 0.98.0, 0.95.2
Attachments: HBASE-8705_1.patch, HBASE-8705_2.patch, HBASE-8705.patch
This bug may be minor as it likely to happen in a single node setup.
I restarted the RS holding META. The master tried assigning META using
MetaSSH. But tried this before the new RS came up.
So as not region plan is found
{code}
if (plan == null) {
LOG.warn(Unable to determine a plan to assign + region);
if (tomActivated){
this.timeoutMonitor.setAllRegionServersOffline(true);
} else {
regionStates.updateRegionState(region,
RegionState.State.FAILED_OPEN);
}
return;
}
{code}
we just return without assigment. And this being the META the small cluster
just hangs.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8015) Support for Namespaces
[ https://issues.apache.org/jira/browse/HBASE-8015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13715085#comment-13715085 ] Francis Liu commented on HBASE-8015: @enis it looks like ':' is not in this list of not-allowed characters in ZK. I naively tried it and it works. Just curious I'm not familiar with the HBase on windows effort. They won't be running HBase on HDFS? But on a windows DFS? @stack I believe enis is suggesting on using a different delimiter when storing FQTN as part of filenames on hdfs. Support for Namespaces -- Key: HBASE-8015 URL: https://issues.apache.org/jira/browse/HBASE-8015 Project: HBase Issue Type: New Feature Reporter: Francis Liu Assignee: Francis Liu Attachments: HBASE-8015_draft_94.patch, Namespace Design.pdf -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8662) [rest] support impersonation
[ https://issues.apache.org/jira/browse/HBASE-8662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13715090#comment-13715090 ] Francis Liu commented on HBASE-8662: I believe the 0.94 sample patch I provided is backwards compatible (including the rpc). I only need to update it with the minor differences in the trunk patch. [rest] support impersonation Key: HBASE-8662 URL: https://issues.apache.org/jira/browse/HBASE-8662 Project: HBase Issue Type: Sub-task Components: REST, security Reporter: Jimmy Xiang Assignee: Jimmy Xiang Fix For: 0.98.0, 0.95.2 Attachments: method_doas.patch, secure_rest.patch, trunk-8662.patch, trunk-8662_v2.patch, trunk-8662_v3.patch, trunk-8662_v4.patch, trunk-8662_v5.patch, trunk-8662_v6.patch, trunk-8662_v7.1.patch, trunk-8662_v7.2.patch, trunk-8662_v7.3.patch, trunk-8662_v7.patch Currently, our client API uses a fixed user: the current user. It should accept a user passed in, if authenticated. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8705) RS holding META when restarted in a single node setup may hang infinitely without META assignment
[
https://issues.apache.org/jira/browse/HBASE-8705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13715094#comment-13715094
]
ramkrishna.s.vasudevan commented on HBASE-8705:
---
I got your point. If you are running a single cluster setup
{code}
sleepTimeBeforeRetryingMetaAssignment
{code}
Can you increase this config and if needed the maxTries also. Should a fix in
the code needed here?
RS holding META when restarted in a single node setup may hang infinitely
without META assignment
-
Key: HBASE-8705
URL: https://issues.apache.org/jira/browse/HBASE-8705
Project: HBase
Issue Type: Bug
Affects Versions: 0.95.1
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan
Priority: Minor
Fix For: 0.98.0, 0.95.2
Attachments: HBASE-8705_1.patch, HBASE-8705_2.patch, HBASE-8705.patch
This bug may be minor as it likely to happen in a single node setup.
I restarted the RS holding META. The master tried assigning META using
MetaSSH. But tried this before the new RS came up.
So as not region plan is found
{code}
if (plan == null) {
LOG.warn(Unable to determine a plan to assign + region);
if (tomActivated){
this.timeoutMonitor.setAllRegionServersOffline(true);
} else {
regionStates.updateRegionState(region,
RegionState.State.FAILED_OPEN);
}
return;
}
{code}
we just return without assigment. And this being the META the small cluster
just hangs.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8705) RS holding META when restarted in a single node setup may hang infinitely without META assignment
[
https://issues.apache.org/jira/browse/HBASE-8705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13715100#comment-13715100
]
rajeshbabu commented on HBASE-8705:
---
bq. Can you increase this config and if needed the maxTries also.
We can increase these values, but may not know the delay before starting new RS.
bq. Should a fix in the code needed here?
It will be better I feel.
RS holding META when restarted in a single node setup may hang infinitely
without META assignment
-
Key: HBASE-8705
URL: https://issues.apache.org/jira/browse/HBASE-8705
Project: HBase
Issue Type: Bug
Affects Versions: 0.95.1
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan
Priority: Minor
Fix For: 0.98.0, 0.95.2
Attachments: HBASE-8705_1.patch, HBASE-8705_2.patch, HBASE-8705.patch
This bug may be minor as it likely to happen in a single node setup.
I restarted the RS holding META. The master tried assigning META using
MetaSSH. But tried this before the new RS came up.
So as not region plan is found
{code}
if (plan == null) {
LOG.warn(Unable to determine a plan to assign + region);
if (tomActivated){
this.timeoutMonitor.setAllRegionServersOffline(true);
} else {
regionStates.updateRegionState(region,
RegionState.State.FAILED_OPEN);
}
return;
}
{code}
we just return without assigment. And this being the META the small cluster
just hangs.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8705) RS holding META when restarted in a single node setup may hang infinitely without META assignment
[
https://issues.apache.org/jira/browse/HBASE-8705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13715151#comment-13715151
]
ramkrishna.s.vasudevan commented on HBASE-8705:
---
Okie.. Come up with a patch. If others agree we can commit it.
RS holding META when restarted in a single node setup may hang infinitely
without META assignment
-
Key: HBASE-8705
URL: https://issues.apache.org/jira/browse/HBASE-8705
Project: HBase
Issue Type: Bug
Affects Versions: 0.95.1
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan
Priority: Minor
Fix For: 0.98.0, 0.95.2
Attachments: HBASE-8705_1.patch, HBASE-8705_2.patch, HBASE-8705.patch
This bug may be minor as it likely to happen in a single node setup.
I restarted the RS holding META. The master tried assigning META using
MetaSSH. But tried this before the new RS came up.
So as not region plan is found
{code}
if (plan == null) {
LOG.warn(Unable to determine a plan to assign + region);
if (tomActivated){
this.timeoutMonitor.setAllRegionServersOffline(true);
} else {
regionStates.updateRegionState(region,
RegionState.State.FAILED_OPEN);
}
return;
}
{code}
we just return without assigment. And this being the META the small cluster
just hangs.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8900) TestRSKilledWhenMasterInitializing.testCorrectnessWhenMasterFailOver is flakey
[
https://issues.apache.org/jira/browse/HBASE-8900?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13715150#comment-13715150
]
ramkrishna.s.vasudevan commented on HBASE-8900:
---
Actually here we have got an HDFS exception
{code}
Could not obtain block:
BP-1276573177-67.195.138.60-1373240501128:blk_-2451087782628280101_1034
file=/user/jenkins/hbase/testCorrectnessWhenMasterFailOver/fbf43cbd6a7509dbedc9f0fa410c46a9/recovered.edits/002
{code}
also AccessControlException which has prevented the HlogReader to read the file
after failover. Hence once of the regions continues to be in RIT and the
testcase timed out.
TestRSKilledWhenMasterInitializing.testCorrectnessWhenMasterFailOver is flakey
--
Key: HBASE-8900
URL: https://issues.apache.org/jira/browse/HBASE-8900
Project: HBase
Issue Type: Bug
Components: test
Reporter: stack
Assignee: ramkrishna.s.vasudevan
Attachments: 8900.txt
Failed here:
https://builds.apache.org/job/hbase-0.95-on-hadoop2/169/testReport/junit/org.apache.hadoop.hbase.regionserver/TestRSKilledWhenMasterInitializing/testCorrectnessWhenMasterFailOver/
and
http://54.241.6.143/job/HBase-0.95-Hadoop-2/579/org.apache.hbase$hbase-server/testReport/junit/org.apache.hadoop.hbase.regionserver/TestRSKilledWhenMasterInitializing/org_apache_hadoop_hbase_regionserver_TestRSKilledWhenMasterInitializing/
{code}
java.lang.Exception: test timed out after 12 milliseconds
at java.lang.Thread.sleep(Native Method)
at
org.apache.hadoop.hbase.zookeeper.ZKAssign.blockUntilNoRIT(ZKAssign.java:1002)
at
org.apache.hadoop.hbase.regionserver.TestRSKilledWhenMasterInitializing.testCorrectnessWhenMasterFailOver(TestRSKilledWhenMasterInitializing.java:177)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at
org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74)
{code}
and with this:
{code}
java.lang.NullPointerException
at
org.apache.hadoop.hbase.regionserver.TestRSKilledWhenMasterInitializing.tearDownAfterClass(TestRSKilledWhenMasterInitializing.java:83)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33)
at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
at org.junit.runners.Suite.runChild(Suite.java:127)
at org.junit.runners.Suite.runChild(Suite.java:26)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
at java.lang.Thread.run(Thread.java:662)
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8353) -ROOT-/.META. regions are hanging if master restarted while closing -ROOT-/.META. regions on dead RS
[
https://issues.apache.org/jira/browse/HBASE-8353?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13715152#comment-13715152
]
ramkrishna.s.vasudevan commented on HBASE-8353:
---
I will test all the scenarios including rolling restart tomorrow morning IST
and let you know.
Have you tried this down in a cluster.
-ROOT-/.META. regions are hanging if master restarted while closing
-ROOT-/.META. regions on dead RS
Key: HBASE-8353
URL: https://issues.apache.org/jira/browse/HBASE-8353
Project: HBase
Issue Type: Bug
Components: Region Assignment
Affects Versions: 0.94.6
Reporter: rajeshbabu
Assignee: rajeshbabu
Fix For: 0.94.11
Attachments: HBASE-8353_94_2.patch, HBASE-8353_94_3.patch,
HBASE-8353_94_4.patch, HBASE-8353_94.patch
ROOT/META are not getting assigned if master restarted while closing
ROOT/META.
Lets suppose catalog table regions in M_ZK_REGION_CLOSING state during master
initialization and then just we are adding the them to RIT and waiting for
TM. {code}
if (isOnDeadServer(regionInfo, deadServers)
(data.getOrigin() == null ||
!serverManager.isServerOnline(data.getOrigin( {
// If was on dead server, its closed now. Force to OFFLINE and this
// will get it reassigned if appropriate
forceOffline(regionInfo, data);
} else {
// Just insert region into RIT.
// If this never updates the timeout will trigger new assignment
regionsInTransition.put(encodedRegionName, new RegionState(
regionInfo, RegionState.State.CLOSING,
data.getStamp(), data.getOrigin()));
}
{code}
isOnDeadServer always return false to ROOT/META because deadServers is null.
Even TM cannot close them properly because its not available in online
regions since its not yet assigned.
{code}
synchronized (this.regions) {
// Check if this region is currently assigned
if (!regions.containsKey(region)) {
LOG.debug(Attempted to unassign region +
region.getRegionNameAsString() + but it is not +
currently assigned anywhere);
return;
}
}
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8947) Thrift 2 : Replace bool writeToWAL with TDurability durability
[ https://issues.apache.org/jira/browse/HBASE-8947?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13715271#comment-13715271 ] Lars George commented on HBASE-8947: If possible, you could somehow emulate the various settings and check if the resulting object reacts the way it should? Somehow we need to make sure that the above strategy works. Thrift 2 : Replace bool writeToWAL with TDurability durability --- Key: HBASE-8947 URL: https://issues.apache.org/jira/browse/HBASE-8947 Project: HBase Issue Type: Sub-task Components: Thrift Reporter: Hamed Madani Assignee: Hamed Madani Attachments: HBASE-8947.patch, HBASE-8947-v2.patch Introduce new enum *TDurability* to expose more options for *Write To Wal.* -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9002) TestDistributedLogSplitting.testRecoveredEdits fails
[
https://issues.apache.org/jira/browse/HBASE-9002?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13715310#comment-13715310
]
stack commented on HBASE-9002:
--
+1 Makes sense. Thanks [~jeffreyz]
TestDistributedLogSplitting.testRecoveredEdits fails
Key: HBASE-9002
URL: https://issues.apache.org/jira/browse/HBASE-9002
Project: HBase
Issue Type: Bug
Components: test
Reporter: stack
Assignee: Jeffrey Zhong
Fix For: 0.95.2
Attachments: hbase-9002.patch
https://builds.apache.org/job/hbase-0.95/342/testReport/org.apache.hadoop.hbase.master/TestDistributedLogSplitting/testRecoveredEdits/
{code}
java.lang.AssertionError: expected:1000 but was:0
at org.junit.Assert.fail(Assert.java:88)
at org.junit.Assert.failNotEquals(Assert.java:743)
at org.junit.Assert.assertEquals(Assert.java:118)
at org.junit.Assert.assertEquals(Assert.java:555)
at org.junit.Assert.assertEquals(Assert.java:542)
at
org.apache.hadoop.hbase.master.TestDistributedLogSplitting.testRecoveredEdits(TestDistributedLogSplitting.java:209)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at
org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74)
{code}
[~jxiang] or [~jeffreyz] want to take a look at this one?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8984) Test online snapshots with online merge
[ https://issues.apache.org/jira/browse/HBASE-8984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13715320#comment-13715320 ] Ted Yu commented on HBASE-8984: --- Overall looks good. I don't see where the admin is closed. Please add the close() call. Test online snapshots with online merge --- Key: HBASE-8984 URL: https://issues.apache.org/jira/browse/HBASE-8984 Project: HBase Issue Type: Test Components: snapshots Affects Versions: 0.95.1 Reporter: Matteo Bertozzi Assignee: Matteo Bertozzi Priority: Minor Fix For: 0.95.2 Attachments: HBASE-8984.patch Add unit tests to verify that after an online merge: * taking a snapshot still works * snapshots and cloned tables are still in a good state. Also on the same line, fix the unit test to have more than one region, and test multi RS/multi regions online snapshot. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8900) TestRSKilledWhenMasterInitializing.testCorrectnessWhenMasterFailOver is flakey
[
https://issues.apache.org/jira/browse/HBASE-8900?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13715346#comment-13715346
]
stack commented on HBASE-8900:
--
I am going to remove from trunk too.
Here we have one of those silent failures. If I compare a list of tests that
passed on successful run to those that show on this fail I get this difference:
{code}
durruti:trunk stack$ diff /tmp/bad_trunk.txt /tmp/good_trunk.txt
91a92
Running org.apache.hadoop.hbase.mapreduce.TestMultiTableInputFormat
159a161
Running
org.apache.hadoop.hbase.regionserver.TestRSKilledWhenMasterInitializing
176a179,180
Running org.apache.hadoop.hbase.regionserver.wal.TestHLogSplitCompressed
Running org.apache.hadoop.hbase.regionserver.wal.TestLogRollAbort
183,185c187,188
Running org.apache.hadoop.hbase.replication.TestReplicationKillMasterRS
Running
org.apache.hadoop.hbase.replication.TestReplicationKillMasterRSCompressed
Running org.apache.hadoop.hbase.replication.TestReplicationKillSlaveRS
---
Running org.apache.hadoop.hbase.replication.TestReplicationQueueFailover
Running
org.apache.hadoop.hbase.replication.TestReplicationQueueFailoverCompressed
200a204
Running org.apache.hadoop.hbase.rest.client.TestRemoteAdmin
{code}
TestMultiTableInputFormat has been removed already.
TestReplicationKillMasterRS is likely new since the good run. etc.
TestRSKilledWhenMasterInitializing is in the list. I'm going to remove it for
now until it has been fixed. It is already removed from 0.95. Doing same for
trunk so can get clean builds.
TestRSKilledWhenMasterInitializing.testCorrectnessWhenMasterFailOver is flakey
--
Key: HBASE-8900
URL: https://issues.apache.org/jira/browse/HBASE-8900
Project: HBase
Issue Type: Bug
Components: test
Reporter: stack
Assignee: ramkrishna.s.vasudevan
Attachments: 8900.txt
Failed here:
https://builds.apache.org/job/hbase-0.95-on-hadoop2/169/testReport/junit/org.apache.hadoop.hbase.regionserver/TestRSKilledWhenMasterInitializing/testCorrectnessWhenMasterFailOver/
and
http://54.241.6.143/job/HBase-0.95-Hadoop-2/579/org.apache.hbase$hbase-server/testReport/junit/org.apache.hadoop.hbase.regionserver/TestRSKilledWhenMasterInitializing/org_apache_hadoop_hbase_regionserver_TestRSKilledWhenMasterInitializing/
{code}
java.lang.Exception: test timed out after 12 milliseconds
at java.lang.Thread.sleep(Native Method)
at
org.apache.hadoop.hbase.zookeeper.ZKAssign.blockUntilNoRIT(ZKAssign.java:1002)
at
org.apache.hadoop.hbase.regionserver.TestRSKilledWhenMasterInitializing.testCorrectnessWhenMasterFailOver(TestRSKilledWhenMasterInitializing.java:177)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at
org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74)
{code}
and with this:
{code}
java.lang.NullPointerException
at
org.apache.hadoop.hbase.regionserver.TestRSKilledWhenMasterInitializing.tearDownAfterClass(TestRSKilledWhenMasterInitializing.java:83)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33)
at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
at org.junit.runners.Suite.runChild(Suite.java:127)
at org.junit.runners.Suite.runChild(Suite.java:26)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439)
at
[jira] [Commented] (HBASE-8662) [rest] support impersonation
[ https://issues.apache.org/jira/browse/HBASE-8662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13715348#comment-13715348 ] Jimmy Xiang commented on HBASE-8662: That wound be great. It is fine with me. [rest] support impersonation Key: HBASE-8662 URL: https://issues.apache.org/jira/browse/HBASE-8662 Project: HBase Issue Type: Sub-task Components: REST, security Reporter: Jimmy Xiang Assignee: Jimmy Xiang Fix For: 0.98.0, 0.95.2 Attachments: method_doas.patch, secure_rest.patch, trunk-8662.patch, trunk-8662_v2.patch, trunk-8662_v3.patch, trunk-8662_v4.patch, trunk-8662_v5.patch, trunk-8662_v6.patch, trunk-8662_v7.1.patch, trunk-8662_v7.2.patch, trunk-8662_v7.3.patch, trunk-8662_v7.patch Currently, our client API uses a fixed user: the current user. It should accept a user passed in, if authenticated. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8900) TestRSKilledWhenMasterInitializing.testCorrectnessWhenMasterFailOver is flakey
[
https://issues.apache.org/jira/browse/HBASE-8900?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13715347#comment-13715347
]
stack commented on HBASE-8900:
--
Hmm... nvm. Already removed above. I was comparing old builds.
TestRSKilledWhenMasterInitializing.testCorrectnessWhenMasterFailOver is flakey
--
Key: HBASE-8900
URL: https://issues.apache.org/jira/browse/HBASE-8900
Project: HBase
Issue Type: Bug
Components: test
Reporter: stack
Assignee: ramkrishna.s.vasudevan
Attachments: 8900.txt
Failed here:
https://builds.apache.org/job/hbase-0.95-on-hadoop2/169/testReport/junit/org.apache.hadoop.hbase.regionserver/TestRSKilledWhenMasterInitializing/testCorrectnessWhenMasterFailOver/
and
http://54.241.6.143/job/HBase-0.95-Hadoop-2/579/org.apache.hbase$hbase-server/testReport/junit/org.apache.hadoop.hbase.regionserver/TestRSKilledWhenMasterInitializing/org_apache_hadoop_hbase_regionserver_TestRSKilledWhenMasterInitializing/
{code}
java.lang.Exception: test timed out after 12 milliseconds
at java.lang.Thread.sleep(Native Method)
at
org.apache.hadoop.hbase.zookeeper.ZKAssign.blockUntilNoRIT(ZKAssign.java:1002)
at
org.apache.hadoop.hbase.regionserver.TestRSKilledWhenMasterInitializing.testCorrectnessWhenMasterFailOver(TestRSKilledWhenMasterInitializing.java:177)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at
org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74)
{code}
and with this:
{code}
java.lang.NullPointerException
at
org.apache.hadoop.hbase.regionserver.TestRSKilledWhenMasterInitializing.tearDownAfterClass(TestRSKilledWhenMasterInitializing.java:83)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33)
at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
at org.junit.runners.Suite.runChild(Suite.java:127)
at org.junit.runners.Suite.runChild(Suite.java:26)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
at java.lang.Thread.run(Thread.java:662)
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8984) Test online snapshots with online merge
[ https://issues.apache.org/jira/browse/HBASE-8984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13715338#comment-13715338 ] Matteo Bertozzi commented on HBASE-8984: the admin instance is owned by the HBaseTestingUtility and closed by shutdownMiniHBaseCluster() part of the @AfterClass cleanupTest() that contains UTIL.shutdownMiniCluster(); Test online snapshots with online merge --- Key: HBASE-8984 URL: https://issues.apache.org/jira/browse/HBASE-8984 Project: HBase Issue Type: Test Components: snapshots Affects Versions: 0.95.1 Reporter: Matteo Bertozzi Assignee: Matteo Bertozzi Priority: Minor Fix For: 0.95.2 Attachments: HBASE-8984.patch Add unit tests to verify that after an online merge: * taking a snapshot still works * snapshots and cloned tables are still in a good state. Also on the same line, fix the unit test to have more than one region, and test multi RS/multi regions online snapshot. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-9012) TestBlockReorder.testBlockLocationReorder fails
stack created HBASE-9012: Summary: TestBlockReorder.testBlockLocationReorder fails Key: HBASE-9012 URL: https://issues.apache.org/jira/browse/HBASE-9012 Project: HBase Issue Type: Bug Components: test Reporter: stack http://54.241.6.143/job/HBase-0.95/669/org.apache.hbase$hbase-server/testReport/junit/org.apache.hadoop.hbase.fs/TestBlockReorder/testBlockLocationReorder/ java.net.BindException: Address already in use at java.net.PlainSocketImpl.socketBind(Native Method) at java.net.PlainSocketImpl.bind(PlainSocketImpl.java:383) at java.net.ServerSocket.bind(ServerSocket.java:328) at java.net.ServerSocket.init(ServerSocket.java:194) at java.net.ServerSocket.init(ServerSocket.java:106) at org.apache.hadoop.hbase.fs.TestBlockReorder.testBlockLocationReorder(TestBlockReorder.java:182) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8900) TestRSKilledWhenMasterInitializing.testCorrectnessWhenMasterFailOver is flakey
[
https://issues.apache.org/jira/browse/HBASE-8900?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13715306#comment-13715306
]
stack commented on HBASE-8900:
--
[~ram_krish] Anything in the log on what happened to the block? On the ACE
issue, was the log from before I disabled short-circuit read for all unit tests
do you know? Thanks for taking a look.
TestRSKilledWhenMasterInitializing.testCorrectnessWhenMasterFailOver is flakey
--
Key: HBASE-8900
URL: https://issues.apache.org/jira/browse/HBASE-8900
Project: HBase
Issue Type: Bug
Components: test
Reporter: stack
Assignee: ramkrishna.s.vasudevan
Attachments: 8900.txt
Failed here:
https://builds.apache.org/job/hbase-0.95-on-hadoop2/169/testReport/junit/org.apache.hadoop.hbase.regionserver/TestRSKilledWhenMasterInitializing/testCorrectnessWhenMasterFailOver/
and
http://54.241.6.143/job/HBase-0.95-Hadoop-2/579/org.apache.hbase$hbase-server/testReport/junit/org.apache.hadoop.hbase.regionserver/TestRSKilledWhenMasterInitializing/org_apache_hadoop_hbase_regionserver_TestRSKilledWhenMasterInitializing/
{code}
java.lang.Exception: test timed out after 12 milliseconds
at java.lang.Thread.sleep(Native Method)
at
org.apache.hadoop.hbase.zookeeper.ZKAssign.blockUntilNoRIT(ZKAssign.java:1002)
at
org.apache.hadoop.hbase.regionserver.TestRSKilledWhenMasterInitializing.testCorrectnessWhenMasterFailOver(TestRSKilledWhenMasterInitializing.java:177)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at
org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74)
{code}
and with this:
{code}
java.lang.NullPointerException
at
org.apache.hadoop.hbase.regionserver.TestRSKilledWhenMasterInitializing.tearDownAfterClass(TestRSKilledWhenMasterInitializing.java:83)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33)
at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
at org.junit.runners.Suite.runChild(Suite.java:127)
at org.junit.runners.Suite.runChild(Suite.java:26)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
at java.lang.Thread.run(Thread.java:662)
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8780) a column Family can have VERSIONS less than zero
[
https://issues.apache.org/jira/browse/HBASE-8780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13715367#comment-13715367
]
Demai Ni commented on HBASE-8780:
-
Anoop, Ted and Chunhui, many thanks for the help and review. Is this patch
ready to be committed to trunk? any action I should take at this moment?
thanks... Demai
a column Family can have VERSIONS less than zero
-
Key: HBASE-8780
URL: https://issues.apache.org/jira/browse/HBASE-8780
Project: HBase
Issue Type: Bug
Components: Client
Affects Versions: 0.94.8
Reporter: Demai Ni
Assignee: Demai Ni
Priority: Trivial
Attachments: HBASE-8780-0.94.8-v0.patch, HBASE-8780-test.txt,
HBASE-8780-trunk.patch, HBASE-8780-trunk.patch, HBASE-8780-trunk-v1.patch,
HBASE-8780-trunk-v2.patch
User can create/alter a columnfam and set its VERSION(aka maxVERSIONS) to a
negative or zero value. Although there is a checking in
HColumnDesciptor#construtor, hbase shell command will invoke the
setter(setMaxVersions and setMinVersions) directly, hence by pass the
checking. For example:
{code:title=set VERSIONS = -1}
hbase(main):016:0 create 't5_dn',{NAME='cf1',VERSIONS=-1}
0 row(s) in 1.0420 seconds
hbase(main):017:0 put 't5_dn','row1','cf1:q1','row1cf1_v1'
0 row(s) in 0.0700 seconds
hbase(main):018:0 scan 't5_dn'
ROW COLUMN+CELL
0 row(s) in 0.0090 seconds
hbase(main):019:0 describe 't5_dn'
DESCRIPTION ENABLED
't5_dn', {NAME = 'cf1', REPLICATION_SCOPE = '0', true
KEEP_DELETED_CELLS = 'false', COMPRESSION = 'NONE
', ENCODE_ON_DISK = 'true', BLOCKCACHE = 'true',
MIN_VERSIONS = '0', DATA_BLOCK_ENCODING = 'NONE',
IN_MEMORY = 'false', BLOOMFILTER = 'NONE', TTL =
'2147483647', VERSIONS = '-1', BLOCKSIZE = '655
36'}
1 row(s) in 0.0410 seconds
{code}
above example shows VERSIONS = '-1', and put/scan doesn't keep the data
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8414) Hbck still refers to -ROOT- table to locate .META.
[
https://issues.apache.org/jira/browse/HBASE-8414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13715365#comment-13715365
]
Himanshu Vashishtha commented on HBASE-8414:
I didn't know until today that 8627 was not committed. That one has a patch and
reviews looong time ago.
Hbck still refers to -ROOT- table to locate .META.
--
Key: HBASE-8414
URL: https://issues.apache.org/jira/browse/HBASE-8414
Project: HBase
Issue Type: Bug
Components: hbck
Affects Versions: 0.95.0
Reporter: Himanshu Vashishtha
Assignee: Himanshu Vashishtha
In the current ROOT-less trunk, hbck still tries to fix meta by looking its
location in the .ROOT. table. This happens if there is no .META. assigned
when hbck is ran.
HbaseFsck.java:
{code}
boolean checkMetaRegion() {
...
HRegionLocation rootLocation = connection.locateRegion(
HConstants.ROOT_TABLE_NAME, HConstants.EMPTY_START_ROW);
...
}
{code}
Running hbck while meta is in transition:
{code}
bin/hbase hbck
Version: 0.95.0-SNAPSHOT
ERROR: META region or some of its attributes are null.
ERROR: Fatal error: unable to get root region location. Exiting...
Summary:
2 inconsistencies detected.
Status: INCONSISTENT
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9012) TestBlockReorder.testBlockLocationReorder fails
[ https://issues.apache.org/jira/browse/HBASE-9012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13715371#comment-13715371 ] stack commented on HBASE-9012: -- Test was added by HBASE-6435. [~nkeywal] What you think? The DN had not gone down by the time we call: ServerSocket ss = new ServerSocket(port);// We're taking the port to have a timeout issue later. TestBlockReorder.testBlockLocationReorder fails --- Key: HBASE-9012 URL: https://issues.apache.org/jira/browse/HBASE-9012 Project: HBase Issue Type: Bug Components: test Reporter: stack http://54.241.6.143/job/HBase-0.95/669/org.apache.hbase$hbase-server/testReport/junit/org.apache.hadoop.hbase.fs/TestBlockReorder/testBlockLocationReorder/ java.net.BindException: Address already in use at java.net.PlainSocketImpl.socketBind(Native Method) at java.net.PlainSocketImpl.bind(PlainSocketImpl.java:383) at java.net.ServerSocket.bind(ServerSocket.java:328) at java.net.ServerSocket.init(ServerSocket.java:194) at java.net.ServerSocket.init(ServerSocket.java:106) at org.apache.hadoop.hbase.fs.TestBlockReorder.testBlockLocationReorder(TestBlockReorder.java:182) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-9012) TestBlockReorder.testBlockLocationReorder fails
[ https://issues.apache.org/jira/browse/HBASE-9012?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-9012: - Attachment: 9012.txt Retry bindexceptions. What you think [~nkeywal] If good, I'll commit (come to think of it, @nkeywal is on vacation this week...) TestBlockReorder.testBlockLocationReorder fails --- Key: HBASE-9012 URL: https://issues.apache.org/jira/browse/HBASE-9012 Project: HBase Issue Type: Bug Components: test Reporter: stack Attachments: 9012.txt http://54.241.6.143/job/HBase-0.95/669/org.apache.hbase$hbase-server/testReport/junit/org.apache.hadoop.hbase.fs/TestBlockReorder/testBlockLocationReorder/ java.net.BindException: Address already in use at java.net.PlainSocketImpl.socketBind(Native Method) at java.net.PlainSocketImpl.bind(PlainSocketImpl.java:383) at java.net.ServerSocket.bind(ServerSocket.java:328) at java.net.ServerSocket.init(ServerSocket.java:194) at java.net.ServerSocket.init(ServerSocket.java:106) at org.apache.hadoop.hbase.fs.TestBlockReorder.testBlockLocationReorder(TestBlockReorder.java:182) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8984) Test online snapshots with online merge
[ https://issues.apache.org/jira/browse/HBASE-8984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13715343#comment-13715343 ] Ted Yu commented on HBASE-8984: --- Good. Test online snapshots with online merge --- Key: HBASE-8984 URL: https://issues.apache.org/jira/browse/HBASE-8984 Project: HBase Issue Type: Test Components: snapshots Affects Versions: 0.95.1 Reporter: Matteo Bertozzi Assignee: Matteo Bertozzi Priority: Minor Fix For: 0.95.2 Attachments: HBASE-8984.patch Add unit tests to verify that after an online merge: * taking a snapshot still works * snapshots and cloned tables are still in a good state. Also on the same line, fix the unit test to have more than one region, and test multi RS/multi regions online snapshot. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-9012) TestBlockReorder.testBlockLocationReorder fails
[ https://issues.apache.org/jira/browse/HBASE-9012?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-9012: - Assignee: stack Status: Patch Available (was: Open) TestBlockReorder.testBlockLocationReorder fails --- Key: HBASE-9012 URL: https://issues.apache.org/jira/browse/HBASE-9012 Project: HBase Issue Type: Bug Components: test Reporter: stack Assignee: stack Attachments: 9012.txt http://54.241.6.143/job/HBase-0.95/669/org.apache.hbase$hbase-server/testReport/junit/org.apache.hadoop.hbase.fs/TestBlockReorder/testBlockLocationReorder/ java.net.BindException: Address already in use at java.net.PlainSocketImpl.socketBind(Native Method) at java.net.PlainSocketImpl.bind(PlainSocketImpl.java:383) at java.net.ServerSocket.bind(ServerSocket.java:328) at java.net.ServerSocket.init(ServerSocket.java:194) at java.net.ServerSocket.init(ServerSocket.java:106) at org.apache.hadoop.hbase.fs.TestBlockReorder.testBlockLocationReorder(TestBlockReorder.java:182) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8983) HBaseConnection#deleteAllConnections does not always delete
[ https://issues.apache.org/jira/browse/HBASE-8983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13715414#comment-13715414 ] Ted Yu commented on HBASE-8983: --- v3 is fine with me too. HBaseConnection#deleteAllConnections does not always delete --- Key: HBASE-8983 URL: https://issues.apache.org/jira/browse/HBASE-8983 Project: HBase Issue Type: Bug Components: Client Affects Versions: 0.95.1 Reporter: Nicolas Liochon Assignee: Nicolas Liochon Priority: Trivial Fix For: 0.98.0, 0.95.2 Attachments: 8983.v1.patch, 8983.v2.patch, 8983.v3.patch, 8983-v4.txt Cf; mailing list http://search-hadoop.com/m/wurpu1s8Fhs/liochonsubj=Re+Connection+reference+counting -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8983) HBaseConnection#deleteAllConnections does not always delete
[ https://issues.apache.org/jira/browse/HBASE-8983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13715405#comment-13715405 ] Jean-Daniel Cryans commented on HBASE-8983: --- [~yuzhih...@gmail.com] what was wrong with v3? HBaseConnection#deleteAllConnections does not always delete --- Key: HBASE-8983 URL: https://issues.apache.org/jira/browse/HBASE-8983 Project: HBase Issue Type: Bug Components: Client Affects Versions: 0.95.1 Reporter: Nicolas Liochon Assignee: Nicolas Liochon Priority: Trivial Fix For: 0.98.0, 0.95.2 Attachments: 8983.v1.patch, 8983.v2.patch, 8983.v3.patch, 8983-v4.txt Cf; mailing list http://search-hadoop.com/m/wurpu1s8Fhs/liochonsubj=Re+Connection+reference+counting -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8954) TestSplitLogWorker#testPreemptTask failed
[
https://issues.apache.org/jira/browse/HBASE-8954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jimmy Xiang updated HBASE-8954:
---
Status: Open (was: Patch Available)
Looked the trace again. The log split worker wasn't started at all at that
moment.
TestSplitLogWorker#testPreemptTask failed
-
Key: HBASE-8954
URL: https://issues.apache.org/jira/browse/HBASE-8954
Project: HBase
Issue Type: Test
Reporter: Jimmy Xiang
Assignee: Jimmy Xiang
Priority: Minor
Attachments: trunk-8954-2.patch, trunk-8954-v1.patch,
trunk-8954_v2.patch
http://54.241.6.143/job/HBase-0.95-Hadoop-2/org.apache.hbase$hbase-server/627/testReport/junit/org.apache.hadoop.hbase.regionserver/TestSplitLogWorker/testPreemptTask/
{noformat}
java.lang.AssertionError: expected:1 but was:0
at org.junit.Assert.fail(Assert.java:88)
at org.junit.Assert.failNotEquals(Assert.java:743)
at org.junit.Assert.assertEquals(Assert.java:118)
at org.junit.Assert.assertEquals(Assert.java:555)
at org.junit.Assert.assertEquals(Assert.java:542)
at
org.apache.hadoop.hbase.regionserver.TestSplitLogWorker.testPreemptTask(TestSplitLogWorker.java:211)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
at org.junit.runners.Suite.runChild(Suite.java:127)
at org.junit.runners.Suite.runChild(Suite.java:26)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
at java.lang.Thread.run(Thread.java:662)
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-9014) TestHLog.testAppendClose fails
stack created HBASE-9014:
Summary: TestHLog.testAppendClose fails
Key: HBASE-9014
URL: https://issues.apache.org/jira/browse/HBASE-9014
Project: HBase
Issue Type: Bug
Reporter: stack
http://54.241.6.143/job/HBase-0.95/665/org.apache.hbase$hbase-server/testReport/org.apache.hadoop.hbase.regionserver.wal/TestHLog/testAppendClose/
{code}
Error Message
Problem binding to localhost/127.0.0.1:37036 : Address already in use
Stacktrace
java.net.BindException: Problem binding to localhost/127.0.0.1:37036 : Address
already in use
at org.apache.hadoop.ipc.Server.bind(Server.java:228)
at org.apache.hadoop.ipc.Server$Listener.init(Server.java:302)
at org.apache.hadoop.ipc.Server.init(Server.java:1488)
at org.apache.hadoop.ipc.RPC$Server.init(RPC.java:560)
at org.apache.hadoop.ipc.RPC.getServer(RPC.java:521)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:302)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.init(NameNode.java:536)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1410)
at org.apache.hadoop.hdfs.MiniDFSCluster.init(MiniDFSCluster.java:278)
at
org.apache.hadoop.hbase.HBaseTestingUtility.startMiniDFSClusterForTestHLog(HBaseTestingUtility.java:525)
...
{code}
This testAppendClose stops hdfs and starts it again. It looks problematic.
Has waits of 7 seconds for the hdfs cluster to go down but in this test it
seems like it needs even more time.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-9013) Backport HBASE-8994 (Adding log to chaos monkey actions to show what're performed) to 0.94
Andrew Purtell created HBASE-9013: - Summary: Backport HBASE-8994 (Adding log to chaos monkey actions to show what're performed) to 0.94 Key: HBASE-9013 URL: https://issues.apache.org/jira/browse/HBASE-9013 Project: HBase Issue Type: Bug Affects Versions: 0.94.11 Reporter: Andrew Purtell Assignee: Andrew Purtell Priority: Minor Keep the debugability of the 0.94 chaos monkey in line with 0.95/trunk. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9013) Backport HBASE-8994 (Adding log to chaos monkey actions to show what're performed) to 0.94
[ https://issues.apache.org/jira/browse/HBASE-9013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13715460#comment-13715460 ] stack commented on HBASE-9013: -- +1 Backport HBASE-8994 (Adding log to chaos monkey actions to show what're performed) to 0.94 -- Key: HBASE-9013 URL: https://issues.apache.org/jira/browse/HBASE-9013 Project: HBase Issue Type: Bug Affects Versions: 0.94.11 Reporter: Andrew Purtell Assignee: Andrew Purtell Priority: Minor Keep the debugability of the 0.94 chaos monkey in line with 0.95/trunk. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9014) TestHLog.testAppendClose fails
[
https://issues.apache.org/jira/browse/HBASE-9014?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13715463#comment-13715463
]
stack commented on HBASE-9014:
--
I tried a few things. The big long 7 second wait seems pretty necessary
otherwise I get:
{code}
3 Tests run: 13, Failures: 0, Errors: 5, Skipped: 0, Time elapsed: 52.038 sec
FAILURE!
4 testAppendClose(org.apache.hadoop.hbase.regionserver.wal.TestHLog) Time
elapsed: 40.121 sec ERROR!
5 java.io.IOException: Cannot lock storage
/Users/stack/checkouts/trunk/hbase-server/target/test-data/b7723583-fda7-46c7-a3b5-bde04f2f9b77/dfscluster_945339f9-1cd2-416f-a3e1-0e8a89a4e10a/dfs/name1.
T#
6 ,...at
org.apache.hadoop.hdfs.server.common.Storage$StorageDirectory.lock(Storage.java:599)
7 ,...at
org.apache.hadoop.hdfs.server.common.Storage$StorageDirectory.analyzeStorage(Storage.java:452)
8 ,...at
org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:298)
9 ,...at
org.apache.hadoop.hdfs.server.namenode.FSDirectory.loadFSImage(FSDirectory.java:100)
10 ,...at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.initialize(FSNamesystem.java:411)
11 ,...at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.init(FSNamesystem.java:379)
12 ,...at
org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:284)
13 ,...at
org.apache.hadoop.hdfs.server.namenode.NameNode.init(NameNode.java:536)
14 ,...at
org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1410)
...
{code}
I am tempted to disable this test but it is kinda important.
Leaving open for now to keep an eye on it. Any input appreciated.
TestHLog.testAppendClose fails
--
Key: HBASE-9014
URL: https://issues.apache.org/jira/browse/HBASE-9014
Project: HBase
Issue Type: Bug
Reporter: stack
http://54.241.6.143/job/HBase-0.95/665/org.apache.hbase$hbase-server/testReport/org.apache.hadoop.hbase.regionserver.wal/TestHLog/testAppendClose/
{code}
Error Message
Problem binding to localhost/127.0.0.1:37036 : Address already in use
Stacktrace
java.net.BindException: Problem binding to localhost/127.0.0.1:37036 :
Address already in use
at org.apache.hadoop.ipc.Server.bind(Server.java:228)
at org.apache.hadoop.ipc.Server$Listener.init(Server.java:302)
at org.apache.hadoop.ipc.Server.init(Server.java:1488)
at org.apache.hadoop.ipc.RPC$Server.init(RPC.java:560)
at org.apache.hadoop.ipc.RPC.getServer(RPC.java:521)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:302)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.init(NameNode.java:536)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1410)
at org.apache.hadoop.hdfs.MiniDFSCluster.init(MiniDFSCluster.java:278)
at
org.apache.hadoop.hbase.HBaseTestingUtility.startMiniDFSClusterForTestHLog(HBaseTestingUtility.java:525)
...
{code}
This testAppendClose stops hdfs and starts it again. It looks problematic.
Has waits of 7 seconds for the hdfs cluster to go down but in this test it
seems like it needs even more time.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-9015) Client side coprocessor framework
Andrew Purtell created HBASE-9015: - Summary: Client side coprocessor framework Key: HBASE-9015 URL: https://issues.apache.org/jira/browse/HBASE-9015 Project: HBase Issue Type: New Feature Affects Versions: 0.98.0 Reporter: Andrew Purtell Create a set of pre and post operation hooks within the client where user code can be mixed in via upcalls, similar to how server side coprocessors work. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8954) TestSplitLogWorker#testPreemptTask failed
[
https://issues.apache.org/jira/browse/HBASE-8954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jimmy Xiang updated HBASE-8954:
---
Attachment: trunk-8954-3.patch
Attached patch 3, removed creating hconnection when starting the worker. At
least this is not needed for distributed log splitting. It's introduced for
log replay (HBASE-8680). It means each worker will hold a HConnection even it
is not needed all the time. Can we just create it on demand?
[~jeffreyz], do we really need to create the connection in starting the worker
for log replay?
TestSplitLogWorker#testPreemptTask failed
-
Key: HBASE-8954
URL: https://issues.apache.org/jira/browse/HBASE-8954
Project: HBase
Issue Type: Test
Reporter: Jimmy Xiang
Assignee: Jimmy Xiang
Priority: Minor
Attachments: trunk-8954-2.patch, trunk-8954-3.patch,
trunk-8954-v1.patch, trunk-8954_v2.patch
http://54.241.6.143/job/HBase-0.95-Hadoop-2/org.apache.hbase$hbase-server/627/testReport/junit/org.apache.hadoop.hbase.regionserver/TestSplitLogWorker/testPreemptTask/
{noformat}
java.lang.AssertionError: expected:1 but was:0
at org.junit.Assert.fail(Assert.java:88)
at org.junit.Assert.failNotEquals(Assert.java:743)
at org.junit.Assert.assertEquals(Assert.java:118)
at org.junit.Assert.assertEquals(Assert.java:555)
at org.junit.Assert.assertEquals(Assert.java:542)
at
org.apache.hadoop.hbase.regionserver.TestSplitLogWorker.testPreemptTask(TestSplitLogWorker.java:211)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
at org.junit.runners.Suite.runChild(Suite.java:127)
at org.junit.runners.Suite.runChild(Suite.java:26)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
at java.lang.Thread.run(Thread.java:662)
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9015) Client side coprocessor framework
[ https://issues.apache.org/jira/browse/HBASE-9015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13715485#comment-13715485 ] Andrew Purtell commented on HBASE-9015: --- This was a request I heard multiple times at HBaseCon. Observers for the client should come first and cover HTableInterface and HBaseAdmin, then consider going deeper into HConnection. As follow on work, it might be possible to have transparent remote service invocation if both ends of a coprocessor Service can be implemented by extension code, e.g. a client side Exec coprocessor can be registered via configuration, and a factory could build GeneratedHTable01234 implements HTableInterface with extra methods from those registered extensions. Client side coprocessor framework - Key: HBASE-9015 URL: https://issues.apache.org/jira/browse/HBASE-9015 Project: HBase Issue Type: New Feature Affects Versions: 0.98.0 Reporter: Andrew Purtell Create a set of pre and post operation hooks within the client where user code can be mixed in via upcalls, similar to how server side coprocessors work. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9012) TestBlockReorder.testBlockLocationReorder fails
[
https://issues.apache.org/jira/browse/HBASE-9012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13715480#comment-13715480
]
Hadoop QA commented on HBASE-9012:
--
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12593551/9012.txt
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified tests.
{color:green}+1 hadoop1.0{color}. The patch compiles against the hadoop
1.0 profile.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/6423//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6423//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6423//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6423//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6423//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6423//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6423//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6423//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6423//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/6423//console
This message is automatically generated.
TestBlockReorder.testBlockLocationReorder fails
---
Key: HBASE-9012
URL: https://issues.apache.org/jira/browse/HBASE-9012
Project: HBase
Issue Type: Bug
Components: test
Reporter: stack
Assignee: stack
Attachments: 9012.txt
http://54.241.6.143/job/HBase-0.95/669/org.apache.hbase$hbase-server/testReport/junit/org.apache.hadoop.hbase.fs/TestBlockReorder/testBlockLocationReorder/
java.net.BindException: Address already in use
at java.net.PlainSocketImpl.socketBind(Native Method)
at java.net.PlainSocketImpl.bind(PlainSocketImpl.java:383)
at java.net.ServerSocket.bind(ServerSocket.java:328)
at java.net.ServerSocket.init(ServerSocket.java:194)
at java.net.ServerSocket.init(ServerSocket.java:106)
at
org.apache.hadoop.hbase.fs.TestBlockReorder.testBlockLocationReorder(TestBlockReorder.java:182)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9008) Reenable TestReplicationKillSlaveRS.killOneSlaveRS
[ https://issues.apache.org/jira/browse/HBASE-9008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13715498#comment-13715498 ] Jean-Daniel Cryans commented on HBASE-9008: --- [~stack] that this is failing means either tailing HLogs is broken or that we have data loss in 0.96. Looking at this run http://54.241.6.143/job/HBase-TRUNK/org.apache.hbase$hbase-server/428/testReport/org.apache.hadoop.hbase.replication/TestReplicationKillSlaveRS/killOneSlaveRS/, I can see that at the end there's 3 rows missing. All the test is doing is inserting data in one cluster that replicates to another, and during that time we kill one RS on the latter cluster. Reenable TestReplicationKillSlaveRS.killOneSlaveRS -- Key: HBASE-9008 URL: https://issues.apache.org/jira/browse/HBASE-9008 Project: HBase Issue Type: Bug Components: test Reporter: stack Assignee: Jean-Daniel Cryans hbase-9007 disabled it as flakey. See it for listing of fails and HBASE-8961. Assigning to [~jdcryans] to take a looksee. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8780) a column Family can have VERSIONS less than zero
[
https://issues.apache.org/jira/browse/HBASE-8780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13715503#comment-13715503
]
Ted Yu commented on HBASE-8780:
---
I am having trouble checking in code because of LDAP issue which is being
worked on by infrastructure team.
a column Family can have VERSIONS less than zero
-
Key: HBASE-8780
URL: https://issues.apache.org/jira/browse/HBASE-8780
Project: HBase
Issue Type: Bug
Components: Client
Affects Versions: 0.94.8
Reporter: Demai Ni
Assignee: Demai Ni
Priority: Trivial
Attachments: HBASE-8780-0.94.8-v0.patch, HBASE-8780-test.txt,
HBASE-8780-trunk.patch, HBASE-8780-trunk.patch, HBASE-8780-trunk-v1.patch,
HBASE-8780-trunk-v2.patch
User can create/alter a columnfam and set its VERSION(aka maxVERSIONS) to a
negative or zero value. Although there is a checking in
HColumnDesciptor#construtor, hbase shell command will invoke the
setter(setMaxVersions and setMinVersions) directly, hence by pass the
checking. For example:
{code:title=set VERSIONS = -1}
hbase(main):016:0 create 't5_dn',{NAME='cf1',VERSIONS=-1}
0 row(s) in 1.0420 seconds
hbase(main):017:0 put 't5_dn','row1','cf1:q1','row1cf1_v1'
0 row(s) in 0.0700 seconds
hbase(main):018:0 scan 't5_dn'
ROW COLUMN+CELL
0 row(s) in 0.0090 seconds
hbase(main):019:0 describe 't5_dn'
DESCRIPTION ENABLED
't5_dn', {NAME = 'cf1', REPLICATION_SCOPE = '0', true
KEEP_DELETED_CELLS = 'false', COMPRESSION = 'NONE
', ENCODE_ON_DISK = 'true', BLOCKCACHE = 'true',
MIN_VERSIONS = '0', DATA_BLOCK_ENCODING = 'NONE',
IN_MEMORY = 'false', BLOOMFILTER = 'NONE', TTL =
'2147483647', VERSIONS = '-1', BLOCKSIZE = '655
36'}
1 row(s) in 0.0410 seconds
{code}
above example shows VERSIONS = '-1', and put/scan doesn't keep the data
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8954) TestSplitLogWorker#testPreemptTask failed
[
https://issues.apache.org/jira/browse/HBASE-8954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13715511#comment-13715511
]
Jeffrey Zhong commented on HBASE-8954:
--
[~jxiang] I think you're referring to the following code
{code}
// initialize a new connection for splitlogworker configuration
HConnectionManager.getConnection(conf);
{code}
This reason to do a pre-initialization is that during test I found it take
about 3+ seconds to initialize a connection. Since we want to recovery happen
immediately, so we make the connection ready ASAP. The connection is cached for
a configuration instance so there will be only one connection instance for the
region server.
While you can add a boolean to skip the pre-initialization if
distributedLogReplay is turn off. Thanks.
TestSplitLogWorker#testPreemptTask failed
-
Key: HBASE-8954
URL: https://issues.apache.org/jira/browse/HBASE-8954
Project: HBase
Issue Type: Test
Reporter: Jimmy Xiang
Assignee: Jimmy Xiang
Priority: Minor
Attachments: trunk-8954-2.patch, trunk-8954-3.patch,
trunk-8954-v1.patch, trunk-8954_v2.patch
http://54.241.6.143/job/HBase-0.95-Hadoop-2/org.apache.hbase$hbase-server/627/testReport/junit/org.apache.hadoop.hbase.regionserver/TestSplitLogWorker/testPreemptTask/
{noformat}
java.lang.AssertionError: expected:1 but was:0
at org.junit.Assert.fail(Assert.java:88)
at org.junit.Assert.failNotEquals(Assert.java:743)
at org.junit.Assert.assertEquals(Assert.java:118)
at org.junit.Assert.assertEquals(Assert.java:555)
at org.junit.Assert.assertEquals(Assert.java:542)
at
org.apache.hadoop.hbase.regionserver.TestSplitLogWorker.testPreemptTask(TestSplitLogWorker.java:211)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
at org.junit.runners.Suite.runChild(Suite.java:127)
at org.junit.runners.Suite.runChild(Suite.java:26)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
at java.lang.Thread.run(Thread.java:662)
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8778) Region assigments scan table directory making them slow for huge tables
[
https://issues.apache.org/jira/browse/HBASE-8778?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lars Hofhansl updated HBASE-8778:
-
Attachment: 8778-dirmodtime.txt
Trivial patch that does the table dir modtime check before the table dir is
enumerated.
Region assigments scan table directory making them slow for huge tables
---
Key: HBASE-8778
URL: https://issues.apache.org/jira/browse/HBASE-8778
Project: HBase
Issue Type: Improvement
Reporter: Dave Latham
Assignee: Dave Latham
Fix For: 0.98.0, 0.95.2, 0.94.11
Attachments: 8778-dirmodtime.txt, HBASE-8778-0.94.5.patch,
HBASE-8778-0.94.5-v2.patch
On a table with 130k regions it takes about 3 seconds for a region server to
open a region once it has been assigned.
Watching the threads for a region server running 0.94.5 that is opening many
such regions shows the thread opening the reigon in code like this:
{noformat}
PRI IPC Server handler 4 on 60020 daemon prio=10 tid=0x2aaac07e9000
nid=0x6566 runnable [0x4c46d000]
java.lang.Thread.State: RUNNABLE
at java.lang.String.indexOf(String.java:1521)
at java.net.URI$Parser.scan(URI.java:2912)
at java.net.URI$Parser.parse(URI.java:3004)
at java.net.URI.init(URI.java:736)
at org.apache.hadoop.fs.Path.initialize(Path.java:145)
at org.apache.hadoop.fs.Path.init(Path.java:126)
at org.apache.hadoop.fs.Path.init(Path.java:50)
at
org.apache.hadoop.hdfs.protocol.HdfsFileStatus.getFullPath(HdfsFileStatus.java:215)
at
org.apache.hadoop.hdfs.DistributedFileSystem.makeQualified(DistributedFileSystem.java:252)
at
org.apache.hadoop.hdfs.DistributedFileSystem.listStatus(DistributedFileSystem.java:311)
at
org.apache.hadoop.fs.FilterFileSystem.listStatus(FilterFileSystem.java:159)
at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:842)
at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:867)
at org.apache.hadoop.hbase.util.FSUtils.listStatus(FSUtils.java:1168)
at
org.apache.hadoop.hbase.util.FSTableDescriptors.getTableInfoPath(FSTableDescriptors.java:269)
at
org.apache.hadoop.hbase.util.FSTableDescriptors.getTableInfoPath(FSTableDescriptors.java:255)
at
org.apache.hadoop.hbase.util.FSTableDescriptors.getTableInfoModtime(FSTableDescriptors.java:368)
at
org.apache.hadoop.hbase.util.FSTableDescriptors.get(FSTableDescriptors.java:155)
at
org.apache.hadoop.hbase.util.FSTableDescriptors.get(FSTableDescriptors.java:126)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.openRegion(HRegionServer.java:2834)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.openRegion(HRegionServer.java:2807)
at sun.reflect.GeneratedMethodAccessor64.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:320)
at
org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1426)
{noformat}
To open the region, the region server first loads the latest
HTableDescriptor. Since HBASE-4553 HTableDescriptor's are stored in the file
system at /hbase/tableDir/.tableinfo.sequenceNum. The file with the
largest sequenceNum is the current descriptor. This is done so that the
current descirptor is updated atomically. However, since the filename is not
known in advance FSTableDescriptors it has to do a FileSystem.listStatus
operation which has to list all files in the directory to find it. The
directory also contains all the region directories, so in our case it has to
load 130k FileStatus objects. Even using a globStatus matching function
still transfers all the objects to the client before performing the pattern
matching. Furthermore HDFS uses a default of transferring 1000 directory
entries in each RPC call, so it requires 130 roundtrips to the namenode to
fetch all the directory entries.
Consequently, to reassign all the regions of a table (or a constant fraction
thereof) requires time proportional to the square of the number of regions.
In our case, if a region server fails with 200 such regions, it takes 10+
minutes for them all to be reassigned, after the zk expiration and log
splitting.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8778) Region assigments scan table directory making them slow for huge tables
[
https://issues.apache.org/jira/browse/HBASE-8778?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lars Hofhansl updated HBASE-8778:
-
Status: Patch Available (was: Open)
Let's get a test run with that.
Region assigments scan table directory making them slow for huge tables
---
Key: HBASE-8778
URL: https://issues.apache.org/jira/browse/HBASE-8778
Project: HBase
Issue Type: Improvement
Reporter: Dave Latham
Assignee: Dave Latham
Fix For: 0.98.0, 0.95.2, 0.94.11
Attachments: 8778-dirmodtime.txt, HBASE-8778-0.94.5.patch,
HBASE-8778-0.94.5-v2.patch
On a table with 130k regions it takes about 3 seconds for a region server to
open a region once it has been assigned.
Watching the threads for a region server running 0.94.5 that is opening many
such regions shows the thread opening the reigon in code like this:
{noformat}
PRI IPC Server handler 4 on 60020 daemon prio=10 tid=0x2aaac07e9000
nid=0x6566 runnable [0x4c46d000]
java.lang.Thread.State: RUNNABLE
at java.lang.String.indexOf(String.java:1521)
at java.net.URI$Parser.scan(URI.java:2912)
at java.net.URI$Parser.parse(URI.java:3004)
at java.net.URI.init(URI.java:736)
at org.apache.hadoop.fs.Path.initialize(Path.java:145)
at org.apache.hadoop.fs.Path.init(Path.java:126)
at org.apache.hadoop.fs.Path.init(Path.java:50)
at
org.apache.hadoop.hdfs.protocol.HdfsFileStatus.getFullPath(HdfsFileStatus.java:215)
at
org.apache.hadoop.hdfs.DistributedFileSystem.makeQualified(DistributedFileSystem.java:252)
at
org.apache.hadoop.hdfs.DistributedFileSystem.listStatus(DistributedFileSystem.java:311)
at
org.apache.hadoop.fs.FilterFileSystem.listStatus(FilterFileSystem.java:159)
at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:842)
at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:867)
at org.apache.hadoop.hbase.util.FSUtils.listStatus(FSUtils.java:1168)
at
org.apache.hadoop.hbase.util.FSTableDescriptors.getTableInfoPath(FSTableDescriptors.java:269)
at
org.apache.hadoop.hbase.util.FSTableDescriptors.getTableInfoPath(FSTableDescriptors.java:255)
at
org.apache.hadoop.hbase.util.FSTableDescriptors.getTableInfoModtime(FSTableDescriptors.java:368)
at
org.apache.hadoop.hbase.util.FSTableDescriptors.get(FSTableDescriptors.java:155)
at
org.apache.hadoop.hbase.util.FSTableDescriptors.get(FSTableDescriptors.java:126)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.openRegion(HRegionServer.java:2834)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.openRegion(HRegionServer.java:2807)
at sun.reflect.GeneratedMethodAccessor64.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:320)
at
org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1426)
{noformat}
To open the region, the region server first loads the latest
HTableDescriptor. Since HBASE-4553 HTableDescriptor's are stored in the file
system at /hbase/tableDir/.tableinfo.sequenceNum. The file with the
largest sequenceNum is the current descriptor. This is done so that the
current descirptor is updated atomically. However, since the filename is not
known in advance FSTableDescriptors it has to do a FileSystem.listStatus
operation which has to list all files in the directory to find it. The
directory also contains all the region directories, so in our case it has to
load 130k FileStatus objects. Even using a globStatus matching function
still transfers all the objects to the client before performing the pattern
matching. Furthermore HDFS uses a default of transferring 1000 directory
entries in each RPC call, so it requires 130 roundtrips to the namenode to
fetch all the directory entries.
Consequently, to reassign all the regions of a table (or a constant fraction
thereof) requires time proportional to the square of the number of regions.
In our case, if a region server fails with 200 such regions, it takes 10+
minutes for them all to be reassigned, after the zk expiration and log
splitting.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8954) TestSplitLogWorker#testPreemptTask failed
[
https://issues.apache.org/jira/browse/HBASE-8954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13715535#comment-13715535
]
Jimmy Xiang commented on HBASE-8954:
bq. While you can add a boolean to skip the pre-initialization if
distributedLogReplay is turn off.
Yes, that's my second choice in my mind.
bq. This reason to do a pre-initialization is that during test I found it take
about 3+ seconds to initialize a connection.
Do we have a breakdown on where the time was spent?
TestSplitLogWorker#testPreemptTask failed
-
Key: HBASE-8954
URL: https://issues.apache.org/jira/browse/HBASE-8954
Project: HBase
Issue Type: Test
Reporter: Jimmy Xiang
Assignee: Jimmy Xiang
Priority: Minor
Attachments: trunk-8954-2.patch, trunk-8954-3.patch,
trunk-8954-v1.patch, trunk-8954_v2.patch
http://54.241.6.143/job/HBase-0.95-Hadoop-2/org.apache.hbase$hbase-server/627/testReport/junit/org.apache.hadoop.hbase.regionserver/TestSplitLogWorker/testPreemptTask/
{noformat}
java.lang.AssertionError: expected:1 but was:0
at org.junit.Assert.fail(Assert.java:88)
at org.junit.Assert.failNotEquals(Assert.java:743)
at org.junit.Assert.assertEquals(Assert.java:118)
at org.junit.Assert.assertEquals(Assert.java:555)
at org.junit.Assert.assertEquals(Assert.java:542)
at
org.apache.hadoop.hbase.regionserver.TestSplitLogWorker.testPreemptTask(TestSplitLogWorker.java:211)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
at org.junit.runners.Suite.runChild(Suite.java:127)
at org.junit.runners.Suite.runChild(Suite.java:26)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
at java.lang.Thread.run(Thread.java:662)
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9007) TestReplicationKillSlaveRS.killOneSlaveRS fails
[
https://issues.apache.org/jira/browse/HBASE-9007?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13715536#comment-13715536
]
stack commented on HBASE-9007:
--
[~jdcryans] Thanks for looking. They both fail actually and I'm thinking of
filing a new issue to disable the test mentioned above. Let me fix the title
here though meantime. Thanks for pointing out my messup.
TestReplicationKillSlaveRS.killOneSlaveRS fails
---
Key: HBASE-9007
URL: https://issues.apache.org/jira/browse/HBASE-9007
Project: HBase
Issue Type: Bug
Components: test
Reporter: stack
Assignee: Jean-Daniel Cryans
Fix For: 0.95.2
Attachments: 9007.txt
http://54.241.6.143/job/HBase-TRUNK/org.apache.hbase$hbase-server/428/testReport/org.apache.hadoop.hbase.replication/TestReplicationKillSlaveRS/killOneSlaveRS/
{code}
java.lang.AssertionError: Waited too much time for queueFailover replication.
Waited 13733ms.
at org.junit.Assert.fail(Assert.java:88)
at
org.apache.hadoop.hbase.replication.TestReplicationKillRS.loadTableAndKillRS(TestReplicationKillRS.java:89)
at
org.apache.hadoop.hbase.replication.TestReplicationKillSlaveRS.killOneSlaveRS(TestReplicationKillSlaveRS.java:33)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at
org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74)
{code}
I'm going to disable for now.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9007) TestReplicationKillSlaveRS.killOneSlaveRS fails
[
https://issues.apache.org/jira/browse/HBASE-9007?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13715539#comment-13715539
]
stack commented on HBASE-9007:
--
Rather, let me fix the patch.
TestReplicationKillSlaveRS.killOneSlaveRS fails
---
Key: HBASE-9007
URL: https://issues.apache.org/jira/browse/HBASE-9007
Project: HBase
Issue Type: Bug
Components: test
Reporter: stack
Assignee: Jean-Daniel Cryans
Fix For: 0.95.2
Attachments: 9007.txt
http://54.241.6.143/job/HBase-TRUNK/org.apache.hbase$hbase-server/428/testReport/org.apache.hadoop.hbase.replication/TestReplicationKillSlaveRS/killOneSlaveRS/
{code}
java.lang.AssertionError: Waited too much time for queueFailover replication.
Waited 13733ms.
at org.junit.Assert.fail(Assert.java:88)
at
org.apache.hadoop.hbase.replication.TestReplicationKillRS.loadTableAndKillRS(TestReplicationKillRS.java:89)
at
org.apache.hadoop.hbase.replication.TestReplicationKillSlaveRS.killOneSlaveRS(TestReplicationKillSlaveRS.java:33)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at
org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74)
{code}
I'm going to disable for now.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-9016) Cleanup of HRegion (javadoc, unnecessary inits, unnecessary unboxing)
Alex Newman created HBASE-9016: -- Summary: Cleanup of HRegion (javadoc, unnecessary inits, unnecessary unboxing) Key: HBASE-9016 URL: https://issues.apache.org/jira/browse/HBASE-9016 Project: HBase Issue Type: Bug Reporter: Alex Newman Assignee: Alex Newman Priority: Trivial -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-9016) Cleanup of HRegion (javadoc, unnecessary inits, unnecessary unboxing)
[ https://issues.apache.org/jira/browse/HBASE-9016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Newman updated HBASE-9016: --- Status: Patch Available (was: Open) Cleanup of HRegion (javadoc, unnecessary inits, unnecessary unboxing) - Key: HBASE-9016 URL: https://issues.apache.org/jira/browse/HBASE-9016 Project: HBase Issue Type: Bug Reporter: Alex Newman Assignee: Alex Newman Priority: Trivial Attachments: 9016.txt -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-9016) Cleanup of HRegion (javadoc, unnecessary inits, unnecessary unboxing)
[ https://issues.apache.org/jira/browse/HBASE-9016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Newman updated HBASE-9016: --- Attachment: 9016.txt Cleanup of HRegion (javadoc, unnecessary inits, unnecessary unboxing) - Key: HBASE-9016 URL: https://issues.apache.org/jira/browse/HBASE-9016 Project: HBase Issue Type: Bug Reporter: Alex Newman Assignee: Alex Newman Priority: Trivial Attachments: 9016.txt -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8954) TestSplitLogWorker#testPreemptTask failed
[
https://issues.apache.org/jira/browse/HBASE-8954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jimmy Xiang updated HBASE-8954:
---
Fix Version/s: 0.95.2
0.98.0
Status: Patch Available (was: Open)
TestSplitLogWorker#testPreemptTask failed
-
Key: HBASE-8954
URL: https://issues.apache.org/jira/browse/HBASE-8954
Project: HBase
Issue Type: Test
Reporter: Jimmy Xiang
Assignee: Jimmy Xiang
Priority: Minor
Fix For: 0.98.0, 0.95.2
Attachments: trunk-8954-2.patch, trunk-8954-3.patch,
trunk-8954-4.patch, trunk-8954-v1.patch, trunk-8954_v2.patch
http://54.241.6.143/job/HBase-0.95-Hadoop-2/org.apache.hbase$hbase-server/627/testReport/junit/org.apache.hadoop.hbase.regionserver/TestSplitLogWorker/testPreemptTask/
{noformat}
java.lang.AssertionError: expected:1 but was:0
at org.junit.Assert.fail(Assert.java:88)
at org.junit.Assert.failNotEquals(Assert.java:743)
at org.junit.Assert.assertEquals(Assert.java:118)
at org.junit.Assert.assertEquals(Assert.java:555)
at org.junit.Assert.assertEquals(Assert.java:542)
at
org.apache.hadoop.hbase.regionserver.TestSplitLogWorker.testPreemptTask(TestSplitLogWorker.java:211)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
at org.junit.runners.Suite.runChild(Suite.java:127)
at org.junit.runners.Suite.runChild(Suite.java:26)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
at java.lang.Thread.run(Thread.java:662)
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-9017) Consolidate ChaosMonkey's random objects
Jimmy Xiang created HBASE-9017: -- Summary: Consolidate ChaosMonkey's random objects Key: HBASE-9017 URL: https://issues.apache.org/jira/browse/HBASE-9017 Project: HBase Issue Type: Test Reporter: Jimmy Xiang Assignee: Jimmy Xiang Priority: Minor ChaosMonkey has several random objects. We can just share the same random object. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8954) TestSplitLogWorker#testPreemptTask failed
[
https://issues.apache.org/jira/browse/HBASE-8954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jimmy Xiang updated HBASE-8954:
---
Attachment: trunk-8954-4.patch
Attached v4 which just skips hconnection init in case of distributed log
splitting instead of replay.
TestSplitLogWorker#testPreemptTask failed
-
Key: HBASE-8954
URL: https://issues.apache.org/jira/browse/HBASE-8954
Project: HBase
Issue Type: Test
Reporter: Jimmy Xiang
Assignee: Jimmy Xiang
Priority: Minor
Attachments: trunk-8954-2.patch, trunk-8954-3.patch,
trunk-8954-4.patch, trunk-8954-v1.patch, trunk-8954_v2.patch
http://54.241.6.143/job/HBase-0.95-Hadoop-2/org.apache.hbase$hbase-server/627/testReport/junit/org.apache.hadoop.hbase.regionserver/TestSplitLogWorker/testPreemptTask/
{noformat}
java.lang.AssertionError: expected:1 but was:0
at org.junit.Assert.fail(Assert.java:88)
at org.junit.Assert.failNotEquals(Assert.java:743)
at org.junit.Assert.assertEquals(Assert.java:118)
at org.junit.Assert.assertEquals(Assert.java:555)
at org.junit.Assert.assertEquals(Assert.java:542)
at
org.apache.hadoop.hbase.regionserver.TestSplitLogWorker.testPreemptTask(TestSplitLogWorker.java:211)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
at org.junit.runners.Suite.runChild(Suite.java:127)
at org.junit.runners.Suite.runChild(Suite.java:26)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
at java.lang.Thread.run(Thread.java:662)
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8954) TestSplitLogWorker#testPreemptTask failed
[
https://issues.apache.org/jira/browse/HBASE-8954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13715574#comment-13715574
]
Himanshu Vashishtha commented on HBASE-8954:
+1 looks good as long as log replay is turned off by default. Otherwise, the
test would become flaky again.
TestSplitLogWorker#testPreemptTask failed
-
Key: HBASE-8954
URL: https://issues.apache.org/jira/browse/HBASE-8954
Project: HBase
Issue Type: Test
Reporter: Jimmy Xiang
Assignee: Jimmy Xiang
Priority: Minor
Fix For: 0.98.0, 0.95.2
Attachments: trunk-8954-2.patch, trunk-8954-3.patch,
trunk-8954-4.patch, trunk-8954-v1.patch, trunk-8954_v2.patch
http://54.241.6.143/job/HBase-0.95-Hadoop-2/org.apache.hbase$hbase-server/627/testReport/junit/org.apache.hadoop.hbase.regionserver/TestSplitLogWorker/testPreemptTask/
{noformat}
java.lang.AssertionError: expected:1 but was:0
at org.junit.Assert.fail(Assert.java:88)
at org.junit.Assert.failNotEquals(Assert.java:743)
at org.junit.Assert.assertEquals(Assert.java:118)
at org.junit.Assert.assertEquals(Assert.java:555)
at org.junit.Assert.assertEquals(Assert.java:542)
at
org.apache.hadoop.hbase.regionserver.TestSplitLogWorker.testPreemptTask(TestSplitLogWorker.java:211)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
at org.junit.runners.Suite.runChild(Suite.java:127)
at org.junit.runners.Suite.runChild(Suite.java:26)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
at java.lang.Thread.run(Thread.java:662)
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8778) Region assigments scan table directory making them slow for huge tables
[
https://issues.apache.org/jira/browse/HBASE-8778?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13715575#comment-13715575
]
Hadoop QA commented on HBASE-8778:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12593576/8778-dirmodtime.txt
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 hadoop1.0{color}. The patch compiles against the hadoop
1.0 profile.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/6424//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6424//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6424//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6424//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6424//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6424//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6424//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6424//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6424//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/6424//console
This message is automatically generated.
Region assigments scan table directory making them slow for huge tables
---
Key: HBASE-8778
URL: https://issues.apache.org/jira/browse/HBASE-8778
Project: HBase
Issue Type: Improvement
Reporter: Dave Latham
Assignee: Dave Latham
Fix For: 0.98.0, 0.95.2, 0.94.11
Attachments: 8778-dirmodtime.txt, HBASE-8778-0.94.5.patch,
HBASE-8778-0.94.5-v2.patch
On a table with 130k regions it takes about 3 seconds for a region server to
open a region once it has been assigned.
Watching the threads for a region server running 0.94.5 that is opening many
such regions shows the thread opening the reigon in code like this:
{noformat}
PRI IPC Server handler 4 on 60020 daemon prio=10 tid=0x2aaac07e9000
nid=0x6566 runnable [0x4c46d000]
java.lang.Thread.State: RUNNABLE
at java.lang.String.indexOf(String.java:1521)
at java.net.URI$Parser.scan(URI.java:2912)
at java.net.URI$Parser.parse(URI.java:3004)
at java.net.URI.init(URI.java:736)
at org.apache.hadoop.fs.Path.initialize(Path.java:145)
at org.apache.hadoop.fs.Path.init(Path.java:126)
at org.apache.hadoop.fs.Path.init(Path.java:50)
at
org.apache.hadoop.hdfs.protocol.HdfsFileStatus.getFullPath(HdfsFileStatus.java:215)
at
org.apache.hadoop.hdfs.DistributedFileSystem.makeQualified(DistributedFileSystem.java:252)
at
org.apache.hadoop.hdfs.DistributedFileSystem.listStatus(DistributedFileSystem.java:311)
at
org.apache.hadoop.fs.FilterFileSystem.listStatus(FilterFileSystem.java:159)
at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:842)
at
[jira] [Commented] (HBASE-8954) TestSplitLogWorker#testPreemptTask failed
[
https://issues.apache.org/jira/browse/HBASE-8954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13715584#comment-13715584
]
Jeffrey Zhong commented on HBASE-8954:
--
{code}
Do we have a breakdown on where the time was spent?
{code}
No, I didn't profile the connection creation call.
TestSplitLogWorker#testPreemptTask failed
-
Key: HBASE-8954
URL: https://issues.apache.org/jira/browse/HBASE-8954
Project: HBase
Issue Type: Test
Reporter: Jimmy Xiang
Assignee: Jimmy Xiang
Priority: Minor
Fix For: 0.98.0, 0.95.2
Attachments: trunk-8954-2.patch, trunk-8954-3.patch,
trunk-8954-4.patch, trunk-8954-v1.patch, trunk-8954_v2.patch
http://54.241.6.143/job/HBase-0.95-Hadoop-2/org.apache.hbase$hbase-server/627/testReport/junit/org.apache.hadoop.hbase.regionserver/TestSplitLogWorker/testPreemptTask/
{noformat}
java.lang.AssertionError: expected:1 but was:0
at org.junit.Assert.fail(Assert.java:88)
at org.junit.Assert.failNotEquals(Assert.java:743)
at org.junit.Assert.assertEquals(Assert.java:118)
at org.junit.Assert.assertEquals(Assert.java:555)
at org.junit.Assert.assertEquals(Assert.java:542)
at
org.apache.hadoop.hbase.regionserver.TestSplitLogWorker.testPreemptTask(TestSplitLogWorker.java:211)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
at org.junit.runners.Suite.runChild(Suite.java:127)
at org.junit.runners.Suite.runChild(Suite.java:26)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
at java.lang.Thread.run(Thread.java:662)
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8984) Test online snapshots with online merge
[ https://issues.apache.org/jira/browse/HBASE-8984?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matteo Bertozzi updated HBASE-8984: --- Resolution: Fixed Fix Version/s: 0.98.0 Status: Resolved (was: Patch Available) committed to 0.95 and trunk, thanks for the review! Test online snapshots with online merge --- Key: HBASE-8984 URL: https://issues.apache.org/jira/browse/HBASE-8984 Project: HBase Issue Type: Test Components: snapshots Affects Versions: 0.95.1 Reporter: Matteo Bertozzi Assignee: Matteo Bertozzi Priority: Minor Fix For: 0.98.0, 0.95.2 Attachments: HBASE-8984.patch Add unit tests to verify that after an online merge: * taking a snapshot still works * snapshots and cloned tables are still in a good state. Also on the same line, fix the unit test to have more than one region, and test multi RS/multi regions online snapshot. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8778) Region assigments scan table directory making them slow for huge tables
[
https://issues.apache.org/jira/browse/HBASE-8778?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13715608#comment-13715608
]
Dave Latham commented on HBASE-8778:
I began working on a trunk/0.96 patch that would move the table descriptor
files to a known sub directory as well as take the refactoring, cleanup and
documentation from the 0.94.5 patch above but adding a one time migration
instead of the locking or rolling upgrade support. One issue I ran into is
support for snapshots. The snapshot code calls into FSTableDescriptors to
write a table descriptor file in the snapshot directory. How should this work
when FSTableDescriptors is putting descriptors into a known subdir?
Options I see:
- Snapshots behave just like actual table directories and put their
descriptors into a known subdir. During migration, snapshots are migrated to
move their descriptor into their known subdir.
- New snapshots put table descriptors into known subdir. Reading snapshots
support finding the table descriptor in the subdir or the orig directory so no
migration of snapshots are needed.
- Snapshots continue to store table descriptors directly in the snapshot
directory and FSTableDescriptors is refactored to share code to write sequenced
descriptors in any directory.
Thoughts? I'm leaning toward the last option.
Region assigments scan table directory making them slow for huge tables
---
Key: HBASE-8778
URL: https://issues.apache.org/jira/browse/HBASE-8778
Project: HBase
Issue Type: Improvement
Reporter: Dave Latham
Assignee: Dave Latham
Fix For: 0.98.0, 0.95.2, 0.94.11
Attachments: 8778-dirmodtime.txt, HBASE-8778-0.94.5.patch,
HBASE-8778-0.94.5-v2.patch
On a table with 130k regions it takes about 3 seconds for a region server to
open a region once it has been assigned.
Watching the threads for a region server running 0.94.5 that is opening many
such regions shows the thread opening the reigon in code like this:
{noformat}
PRI IPC Server handler 4 on 60020 daemon prio=10 tid=0x2aaac07e9000
nid=0x6566 runnable [0x4c46d000]
java.lang.Thread.State: RUNNABLE
at java.lang.String.indexOf(String.java:1521)
at java.net.URI$Parser.scan(URI.java:2912)
at java.net.URI$Parser.parse(URI.java:3004)
at java.net.URI.init(URI.java:736)
at org.apache.hadoop.fs.Path.initialize(Path.java:145)
at org.apache.hadoop.fs.Path.init(Path.java:126)
at org.apache.hadoop.fs.Path.init(Path.java:50)
at
org.apache.hadoop.hdfs.protocol.HdfsFileStatus.getFullPath(HdfsFileStatus.java:215)
at
org.apache.hadoop.hdfs.DistributedFileSystem.makeQualified(DistributedFileSystem.java:252)
at
org.apache.hadoop.hdfs.DistributedFileSystem.listStatus(DistributedFileSystem.java:311)
at
org.apache.hadoop.fs.FilterFileSystem.listStatus(FilterFileSystem.java:159)
at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:842)
at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:867)
at org.apache.hadoop.hbase.util.FSUtils.listStatus(FSUtils.java:1168)
at
org.apache.hadoop.hbase.util.FSTableDescriptors.getTableInfoPath(FSTableDescriptors.java:269)
at
org.apache.hadoop.hbase.util.FSTableDescriptors.getTableInfoPath(FSTableDescriptors.java:255)
at
org.apache.hadoop.hbase.util.FSTableDescriptors.getTableInfoModtime(FSTableDescriptors.java:368)
at
org.apache.hadoop.hbase.util.FSTableDescriptors.get(FSTableDescriptors.java:155)
at
org.apache.hadoop.hbase.util.FSTableDescriptors.get(FSTableDescriptors.java:126)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.openRegion(HRegionServer.java:2834)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.openRegion(HRegionServer.java:2807)
at sun.reflect.GeneratedMethodAccessor64.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:320)
at
org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1426)
{noformat}
To open the region, the region server first loads the latest
HTableDescriptor. Since HBASE-4553 HTableDescriptor's are stored in the file
system at /hbase/tableDir/.tableinfo.sequenceNum. The file with the
largest sequenceNum is the current descriptor. This is done so that the
current descirptor is updated atomically. However, since the filename is not
known in advance FSTableDescriptors it has to do a
[jira] [Commented] (HBASE-8995) Add hadoop-1.2 profile
[ https://issues.apache.org/jira/browse/HBASE-8995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13715616#comment-13715616 ] Ted Yu commented on HBASE-8995: --- Integrated to 0.94 Thanks for the review, Lars. Add hadoop-1.2 profile -- Key: HBASE-8995 URL: https://issues.apache.org/jira/browse/HBASE-8995 Project: HBase Issue Type: Improvement Affects Versions: 0.94.9 Reporter: Ted Yu Assignee: Ted Yu Attachments: 8995-v1.txt Hadoop 1.2.0 is Beta release. We should add profile in pom.xml for this release. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8780) A column Family can have VERSIONS less than zero
[
https://issues.apache.org/jira/browse/HBASE-8780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-8780:
--
Summary: A column Family can have VERSIONS less than zero (was: a column
Family can have VERSIONS less than zero )
A column Family can have VERSIONS less than zero
-
Key: HBASE-8780
URL: https://issues.apache.org/jira/browse/HBASE-8780
Project: HBase
Issue Type: Bug
Components: Client
Affects Versions: 0.94.8
Reporter: Demai Ni
Assignee: Demai Ni
Priority: Trivial
Attachments: HBASE-8780-0.94.8-v0.patch, HBASE-8780-test.txt,
HBASE-8780-trunk.patch, HBASE-8780-trunk.patch, HBASE-8780-trunk-v1.patch,
HBASE-8780-trunk-v2.patch
User can create/alter a columnfam and set its VERSION(aka maxVERSIONS) to a
negative or zero value. Although there is a checking in
HColumnDesciptor#construtor, hbase shell command will invoke the
setter(setMaxVersions and setMinVersions) directly, hence by pass the
checking. For example:
{code:title=set VERSIONS = -1}
hbase(main):016:0 create 't5_dn',{NAME='cf1',VERSIONS=-1}
0 row(s) in 1.0420 seconds
hbase(main):017:0 put 't5_dn','row1','cf1:q1','row1cf1_v1'
0 row(s) in 0.0700 seconds
hbase(main):018:0 scan 't5_dn'
ROW COLUMN+CELL
0 row(s) in 0.0090 seconds
hbase(main):019:0 describe 't5_dn'
DESCRIPTION ENABLED
't5_dn', {NAME = 'cf1', REPLICATION_SCOPE = '0', true
KEEP_DELETED_CELLS = 'false', COMPRESSION = 'NONE
', ENCODE_ON_DISK = 'true', BLOCKCACHE = 'true',
MIN_VERSIONS = '0', DATA_BLOCK_ENCODING = 'NONE',
IN_MEMORY = 'false', BLOOMFILTER = 'NONE', TTL =
'2147483647', VERSIONS = '-1', BLOCKSIZE = '655
36'}
1 row(s) in 0.0410 seconds
{code}
above example shows VERSIONS = '-1', and put/scan doesn't keep the data
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8780) A column Family can have VERSIONS less than zero
[
https://issues.apache.org/jira/browse/HBASE-8780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-8780:
--
Fix Version/s: 0.98.0
Hadoop Flags: Reviewed
Integrated to trunk - removed the else in the second check because exception is
thrown in the first check.
Thanks for the patch, Demai.
Thanks for the reviews, Chunhui and Anoop.
A column Family can have VERSIONS less than zero
-
Key: HBASE-8780
URL: https://issues.apache.org/jira/browse/HBASE-8780
Project: HBase
Issue Type: Bug
Components: Client
Affects Versions: 0.94.8
Reporter: Demai Ni
Assignee: Demai Ni
Priority: Trivial
Fix For: 0.98.0
Attachments: HBASE-8780-0.94.8-v0.patch, HBASE-8780-test.txt,
HBASE-8780-trunk.patch, HBASE-8780-trunk.patch, HBASE-8780-trunk-v1.patch,
HBASE-8780-trunk-v2.patch
User can create/alter a columnfam and set its VERSION(aka maxVERSIONS) to a
negative or zero value. Although there is a checking in
HColumnDesciptor#construtor, hbase shell command will invoke the
setter(setMaxVersions and setMinVersions) directly, hence by pass the
checking. For example:
{code:title=set VERSIONS = -1}
hbase(main):016:0 create 't5_dn',{NAME='cf1',VERSIONS=-1}
0 row(s) in 1.0420 seconds
hbase(main):017:0 put 't5_dn','row1','cf1:q1','row1cf1_v1'
0 row(s) in 0.0700 seconds
hbase(main):018:0 scan 't5_dn'
ROW COLUMN+CELL
0 row(s) in 0.0090 seconds
hbase(main):019:0 describe 't5_dn'
DESCRIPTION ENABLED
't5_dn', {NAME = 'cf1', REPLICATION_SCOPE = '0', true
KEEP_DELETED_CELLS = 'false', COMPRESSION = 'NONE
', ENCODE_ON_DISK = 'true', BLOCKCACHE = 'true',
MIN_VERSIONS = '0', DATA_BLOCK_ENCODING = 'NONE',
IN_MEMORY = 'false', BLOOMFILTER = 'NONE', TTL =
'2147483647', VERSIONS = '-1', BLOCKSIZE = '655
36'}
1 row(s) in 0.0410 seconds
{code}
above example shows VERSIONS = '-1', and put/scan doesn't keep the data
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8954) TestSplitLogWorker#testPreemptTask failed
[
https://issues.apache.org/jira/browse/HBASE-8954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13715640#comment-13715640
]
Hadoop QA commented on HBASE-8954:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12593581/trunk-8954-4.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 hadoop1.0{color}. The patch compiles against the hadoop
1.0 profile.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/6425//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6425//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6425//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6425//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6425//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6425//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6425//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6425//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6425//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/6425//console
This message is automatically generated.
TestSplitLogWorker#testPreemptTask failed
-
Key: HBASE-8954
URL: https://issues.apache.org/jira/browse/HBASE-8954
Project: HBase
Issue Type: Test
Reporter: Jimmy Xiang
Assignee: Jimmy Xiang
Priority: Minor
Fix For: 0.98.0, 0.95.2
Attachments: trunk-8954-2.patch, trunk-8954-3.patch,
trunk-8954-4.patch, trunk-8954-v1.patch, trunk-8954_v2.patch
http://54.241.6.143/job/HBase-0.95-Hadoop-2/org.apache.hbase$hbase-server/627/testReport/junit/org.apache.hadoop.hbase.regionserver/TestSplitLogWorker/testPreemptTask/
{noformat}
java.lang.AssertionError: expected:1 but was:0
at org.junit.Assert.fail(Assert.java:88)
at org.junit.Assert.failNotEquals(Assert.java:743)
at org.junit.Assert.assertEquals(Assert.java:118)
at org.junit.Assert.assertEquals(Assert.java:555)
at org.junit.Assert.assertEquals(Assert.java:542)
at
org.apache.hadoop.hbase.regionserver.TestSplitLogWorker.testPreemptTask(TestSplitLogWorker.java:211)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at
[jira] [Updated] (HBASE-9007) TestReplicationKillSlaveRS.killOneSlaveRS fails
[
https://issues.apache.org/jira/browse/HBASE-9007?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-9007:
-
Attachment: 9007.addendum.txt
Made patch agree w/ the subject.
This class of tests all seems to have the same invisible fail problem.
TestReplicationKillSlaveRS.killOneSlaveRS fails
---
Key: HBASE-9007
URL: https://issues.apache.org/jira/browse/HBASE-9007
Project: HBase
Issue Type: Bug
Components: test
Reporter: stack
Assignee: Jean-Daniel Cryans
Fix For: 0.95.2
Attachments: 9007.addendum.txt, 9007.txt
http://54.241.6.143/job/HBase-TRUNK/org.apache.hbase$hbase-server/428/testReport/org.apache.hadoop.hbase.replication/TestReplicationKillSlaveRS/killOneSlaveRS/
{code}
java.lang.AssertionError: Waited too much time for queueFailover replication.
Waited 13733ms.
at org.junit.Assert.fail(Assert.java:88)
at
org.apache.hadoop.hbase.replication.TestReplicationKillRS.loadTableAndKillRS(TestReplicationKillRS.java:89)
at
org.apache.hadoop.hbase.replication.TestReplicationKillSlaveRS.killOneSlaveRS(TestReplicationKillSlaveRS.java:33)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at
org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74)
{code}
I'm going to disable for now.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8954) TestSplitLogWorker#testPreemptTask failed
[
https://issues.apache.org/jira/browse/HBASE-8954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13715644#comment-13715644
]
stack commented on HBASE-8954:
--
+1 for now.
TestSplitLogWorker#testPreemptTask failed
-
Key: HBASE-8954
URL: https://issues.apache.org/jira/browse/HBASE-8954
Project: HBase
Issue Type: Test
Reporter: Jimmy Xiang
Assignee: Jimmy Xiang
Priority: Minor
Fix For: 0.98.0, 0.95.2
Attachments: trunk-8954-2.patch, trunk-8954-3.patch,
trunk-8954-4.patch, trunk-8954-v1.patch, trunk-8954_v2.patch
http://54.241.6.143/job/HBase-0.95-Hadoop-2/org.apache.hbase$hbase-server/627/testReport/junit/org.apache.hadoop.hbase.regionserver/TestSplitLogWorker/testPreemptTask/
{noformat}
java.lang.AssertionError: expected:1 but was:0
at org.junit.Assert.fail(Assert.java:88)
at org.junit.Assert.failNotEquals(Assert.java:743)
at org.junit.Assert.assertEquals(Assert.java:118)
at org.junit.Assert.assertEquals(Assert.java:555)
at org.junit.Assert.assertEquals(Assert.java:542)
at
org.apache.hadoop.hbase.regionserver.TestSplitLogWorker.testPreemptTask(TestSplitLogWorker.java:211)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
at org.junit.runners.Suite.runChild(Suite.java:127)
at org.junit.runners.Suite.runChild(Suite.java:26)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
at java.lang.Thread.run(Thread.java:662)
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-9012) TestBlockReorder.testBlockLocationReorder fails
[ https://issues.apache.org/jira/browse/HBASE-9012?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-9012: - Fix Version/s: 0.95.2 Committed this patch to trunk and 0.95. Lets see how it does going forward. TestBlockReorder.testBlockLocationReorder fails --- Key: HBASE-9012 URL: https://issues.apache.org/jira/browse/HBASE-9012 Project: HBase Issue Type: Bug Components: test Reporter: stack Assignee: stack Fix For: 0.95.2 Attachments: 9012.txt http://54.241.6.143/job/HBase-0.95/669/org.apache.hbase$hbase-server/testReport/junit/org.apache.hadoop.hbase.fs/TestBlockReorder/testBlockLocationReorder/ java.net.BindException: Address already in use at java.net.PlainSocketImpl.socketBind(Native Method) at java.net.PlainSocketImpl.bind(PlainSocketImpl.java:383) at java.net.ServerSocket.bind(ServerSocket.java:328) at java.net.ServerSocket.init(ServerSocket.java:194) at java.net.ServerSocket.init(ServerSocket.java:106) at org.apache.hadoop.hbase.fs.TestBlockReorder.testBlockLocationReorder(TestBlockReorder.java:182) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-9018) Add timeouts on all tests in TestHLogSplit
stack created HBASE-9018: Summary: Add timeouts on all tests in TestHLogSplit Key: HBASE-9018 URL: https://issues.apache.org/jira/browse/HBASE-9018 Project: HBase Issue Type: Bug Components: test Reporter: stack Assignee: stack Fix For: 0.95.2 This test runs twice, once as plane TestHLogSplit and then again as TestHLogSplitCompressed. It shows up from time to time as an 'invisible' test -- a test that is not listed as one of the running tests in a failed build (but is present in builds that pass). Let me try setting timeouts on the 30odd tests in this class to see if that makes it fail fast rather than as an 'invisible'. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8778) Region assigments scan table directory making them slow for huge tables
[
https://issues.apache.org/jira/browse/HBASE-8778?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13715656#comment-13715656
]
Lars Hofhansl commented on HBASE-8778:
--
Either #1 or #3. If the layout is the same between table and snapshot, maybe #1
makes the most sense.
What about the directory modtime check, should we just commit this to 0.94
(provided it solves your problem as you said)?
Region assigments scan table directory making them slow for huge tables
---
Key: HBASE-8778
URL: https://issues.apache.org/jira/browse/HBASE-8778
Project: HBase
Issue Type: Improvement
Reporter: Dave Latham
Assignee: Dave Latham
Fix For: 0.98.0, 0.95.2, 0.94.11
Attachments: 8778-dirmodtime.txt, HBASE-8778-0.94.5.patch,
HBASE-8778-0.94.5-v2.patch
On a table with 130k regions it takes about 3 seconds for a region server to
open a region once it has been assigned.
Watching the threads for a region server running 0.94.5 that is opening many
such regions shows the thread opening the reigon in code like this:
{noformat}
PRI IPC Server handler 4 on 60020 daemon prio=10 tid=0x2aaac07e9000
nid=0x6566 runnable [0x4c46d000]
java.lang.Thread.State: RUNNABLE
at java.lang.String.indexOf(String.java:1521)
at java.net.URI$Parser.scan(URI.java:2912)
at java.net.URI$Parser.parse(URI.java:3004)
at java.net.URI.init(URI.java:736)
at org.apache.hadoop.fs.Path.initialize(Path.java:145)
at org.apache.hadoop.fs.Path.init(Path.java:126)
at org.apache.hadoop.fs.Path.init(Path.java:50)
at
org.apache.hadoop.hdfs.protocol.HdfsFileStatus.getFullPath(HdfsFileStatus.java:215)
at
org.apache.hadoop.hdfs.DistributedFileSystem.makeQualified(DistributedFileSystem.java:252)
at
org.apache.hadoop.hdfs.DistributedFileSystem.listStatus(DistributedFileSystem.java:311)
at
org.apache.hadoop.fs.FilterFileSystem.listStatus(FilterFileSystem.java:159)
at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:842)
at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:867)
at org.apache.hadoop.hbase.util.FSUtils.listStatus(FSUtils.java:1168)
at
org.apache.hadoop.hbase.util.FSTableDescriptors.getTableInfoPath(FSTableDescriptors.java:269)
at
org.apache.hadoop.hbase.util.FSTableDescriptors.getTableInfoPath(FSTableDescriptors.java:255)
at
org.apache.hadoop.hbase.util.FSTableDescriptors.getTableInfoModtime(FSTableDescriptors.java:368)
at
org.apache.hadoop.hbase.util.FSTableDescriptors.get(FSTableDescriptors.java:155)
at
org.apache.hadoop.hbase.util.FSTableDescriptors.get(FSTableDescriptors.java:126)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.openRegion(HRegionServer.java:2834)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.openRegion(HRegionServer.java:2807)
at sun.reflect.GeneratedMethodAccessor64.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:320)
at
org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1426)
{noformat}
To open the region, the region server first loads the latest
HTableDescriptor. Since HBASE-4553 HTableDescriptor's are stored in the file
system at /hbase/tableDir/.tableinfo.sequenceNum. The file with the
largest sequenceNum is the current descriptor. This is done so that the
current descirptor is updated atomically. However, since the filename is not
known in advance FSTableDescriptors it has to do a FileSystem.listStatus
operation which has to list all files in the directory to find it. The
directory also contains all the region directories, so in our case it has to
load 130k FileStatus objects. Even using a globStatus matching function
still transfers all the objects to the client before performing the pattern
matching. Furthermore HDFS uses a default of transferring 1000 directory
entries in each RPC call, so it requires 130 roundtrips to the namenode to
fetch all the directory entries.
Consequently, to reassign all the regions of a table (or a constant fraction
thereof) requires time proportional to the square of the number of regions.
In our case, if a region server fails with 200 such regions, it takes 10+
minutes for them all to be reassigned, after the zk expiration and log
splitting.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA
[jira] [Commented] (HBASE-9016) Cleanup of HRegion (javadoc, unnecessary inits, unnecessary unboxing)
[
https://issues.apache.org/jira/browse/HBASE-9016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13715657#comment-13715657
]
Hadoop QA commented on HBASE-9016:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12593579/9016.txt
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 hadoop1.0{color}. The patch compiles against the hadoop
1.0 profile.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/6426//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6426//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6426//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6426//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6426//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6426//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6426//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6426//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6426//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/6426//console
This message is automatically generated.
Cleanup of HRegion (javadoc, unnecessary inits, unnecessary unboxing)
-
Key: HBASE-9016
URL: https://issues.apache.org/jira/browse/HBASE-9016
Project: HBase
Issue Type: Bug
Reporter: Alex Newman
Assignee: Alex Newman
Priority: Trivial
Attachments: 9016.txt
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HBASE-9018) Add timeouts on all tests in TestHLogSplit
[ https://issues.apache.org/jira/browse/HBASE-9018?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack resolved HBASE-9018. -- Resolution: Fixed Committed to trunk and to 0.95. Add timeouts on all tests in TestHLogSplit -- Key: HBASE-9018 URL: https://issues.apache.org/jira/browse/HBASE-9018 Project: HBase Issue Type: Bug Components: test Reporter: stack Assignee: stack Fix For: 0.95.2 Attachments: 9018.txt This test runs twice, once as plane TestHLogSplit and then again as TestHLogSplitCompressed. It shows up from time to time as an 'invisible' test -- a test that is not listed as one of the running tests in a failed build (but is present in builds that pass). Let me try setting timeouts on the 30odd tests in this class to see if that makes it fail fast rather than as an 'invisible'. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-9018) Add timeouts on all tests in TestHLogSplit
[ https://issues.apache.org/jira/browse/HBASE-9018?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-9018: - Attachment: 9018.txt Adds timer on all tests. Add timeouts on all tests in TestHLogSplit -- Key: HBASE-9018 URL: https://issues.apache.org/jira/browse/HBASE-9018 Project: HBase Issue Type: Bug Components: test Reporter: stack Assignee: stack Fix For: 0.95.2 Attachments: 9018.txt This test runs twice, once as plane TestHLogSplit and then again as TestHLogSplitCompressed. It shows up from time to time as an 'invisible' test -- a test that is not listed as one of the running tests in a failed build (but is present in builds that pass). Let me try setting timeouts on the 30odd tests in this class to see if that makes it fail fast rather than as an 'invisible'. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8935) IntegrationTestBigLinkedList fails under load on 0.94 due to some scan issues
[
https://issues.apache.org/jira/browse/HBASE-8935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-8935:
--
Resolution: Fixed
Status: Resolved (was: Patch Available)
IntegrationTestBigLinkedList fails under load on 0.94 due to some scan issues
-
Key: HBASE-8935
URL: https://issues.apache.org/jira/browse/HBASE-8935
Project: HBase
Issue Type: Bug
Reporter: Sergey Shelukhin
Assignee: Sergey Shelukhin
Attachments: HBASE-8935-v0-94-logging.patch, HBASE-8935-v0.patch
We observe occasional failures (4-5k undefined and unreferenced nodes in the
list, running) when the cluster is stressed while running
IntegrationTestBigLinkedList on 0.94. Unfortunately debug logging is disabled.
If Verify is rerun after the fact, the data is there, so it's not data loss.
All the missing keys come from very narrow range from the very beginning of
one region; most of this region is not affected.
In the case I'm looking at, region range is
{code}\xD0\x0C`=%\xA2\xD3\xA8\x0A\xF8\x917\x05\x94\x1D ..
\xD4\x0Bx\xAF\x88\x0C\xF47\x9A\x9F{\xCE\x0E\x8A{code}
and problematic renage
{code}\xD0\x0C`QLD\xED2\xD5c\x8D\xDB5\x01\xD2H] ...
\xD0\x0D\x89)\x11\x0E8\xC5\x08o\xD7\xE3$\xB3\xAAu{code}
There are no splits and compactions on the region during MR job; there are no
compactions after that could have affected the data, although there is one
flush that happened long after, and region was moved and reopened (before I
ran verify manually that showed that data is in HBase).
One thing that happened was that the scanners used by all the map tasks had
lease expirations during the MR job that had missing data, some of them twice.
First scanner expiration for each task I looked at was exactly 1 minute after
Processing split log message, when the scanner is opened.
Only the tasks where the scanners have expired twice (2 of them) logged any
errors; presumably one expiration in each task happened after the reading was
finished, because everything was slow and scanner wasn't closed in time - no
errors or exceptions are logged in the tasks where scanner only expired once.
The job I ran afterwards had no scanner expirations so it does close them
correctly in normal case...
The task that was reading the problematic range had one scanner expiration
and didn't log any errors.
It's a little bit special (or it may be a total red herring) in that in other
tasks, scanner expired while the task was splitting partial outputs (which
may mean end of input reading?), whereas in the task that lost data spilling
started long (~40s) after the scanner expired.
However there's one another such task (spilling started 25s after scanner
expired) and it didn't log any errors and didn't lose data.
At this point I am not sure about the root cause but I suspect it might be
related to scanner expiration handling, given the patterns.
One thing for sure is that there isn't enough logging... so I will start with
adding that.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8874) PutCombiner is skipping KeyValues while combining puts of same row during bulkload
[
https://issues.apache.org/jira/browse/HBASE-8874?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13715667#comment-13715667
]
Ted Yu commented on HBASE-8874:
---
Shall we set the default value to 1GB ?
PutCombiner is skipping KeyValues while combining puts of same row during
bulkload
--
Key: HBASE-8874
URL: https://issues.apache.org/jira/browse/HBASE-8874
Project: HBase
Issue Type: Bug
Components: mapreduce
Affects Versions: 0.95.0, 0.95.1
Reporter: rajeshbabu
Assignee: rajeshbabu
Priority: Critical
Fix For: 0.98.0, 0.95.2
Attachments: HBASE-8874_trunk.patch
While combining puts of same row in map phase we are using below logic in
PutCombiner#reduce. In for loop first time we will add one Put object to puts
map. Next time onwards we are just overriding key values of a family with key
values of the same family in other put. So we are mostly writing one Put
object to map output and remaining will be skipped(data loss).
{code}
Mapbyte[], Put puts = new TreeMapbyte[], Put(Bytes.BYTES_COMPARATOR);
for (Put p : vals) {
cnt++;
if (!puts.containsKey(p.getRow())) {
puts.put(p.getRow(), p);
} else {
puts.get(p.getRow()).getFamilyMap().putAll(p.getFamilyMap());
}
}
{code}
We need to change logic similar as below because we are sure the rowkey of
all the puts will be same.
{code}
Put finalPut = null;
Mapbyte[], List? extends Cell familyMap = null;
for (Put p : vals) {
cnt++;
if (finalPut==null) {
finalPut = p;
familyMap = finalPut.getFamilyMap();
} else {
for (Entrybyte[], List? extends Cell entry :
p.getFamilyMap().entrySet()) {
List? extends Cell list = familyMap.get(entry.getKey());
if (list == null) {
familyMap.put(entry.getKey(), entry.getValue());
} else {
(((ListKeyValue)list)).addAll((ListKeyValue)entry.getValue());
}
}
}
}
context.write(row, finalPut);
{code}
Also need to implement TODOs mentioned by Nick
{code}
// TODO: would be better if we knew codeK row/code and Put rowkey were
// identical. Then this whole Put buffering business goes away.
// TODO: Could use HeapSize to create an upper bound on the memory size of
// the puts map and flush some portion of the content while looping. This
// flush could result in multiple Puts for a single rowkey. That is
// acceptable because Combiner is run as an optimization and it's not
// critical that all Puts are grouped perfectly.
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HBASE-8059) Enhance test-patch.sh so that patch can specify hadoop-2.0 as the default profile
[
https://issues.apache.org/jira/browse/HBASE-8059?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu resolved HBASE-8059.
---
Resolution: Fixed
Enhance test-patch.sh so that patch can specify hadoop-2.0 as the default
profile
-
Key: HBASE-8059
URL: https://issues.apache.org/jira/browse/HBASE-8059
Project: HBase
Issue Type: Improvement
Reporter: Ted Yu
Fix For: 0.98.0
Attachments: 7904-v5-hadoop-2.0.txt, 8059-v1.txt, 8059-v2.txt,
hadoop-2.0-template-pom.xml
Over in HBASE-7904, I produced a patch which uses hadoop-2.0 as the default
profile.
However, when QA tries to validate compilation against hadoop-2.0:
{code}
[ERROR] The build could not read 1 project - [Help 1]
[ERROR]
[ERROR] The project org.apache.hbase:hbase:0.97-SNAPSHOT
(/Users/tyu/trunk/pom.xml) has 2 errors
[ERROR] 'dependencyManagement.dependencies.dependency.artifactId' for
org.apache.hbase:${compat.module}:jar with value '${compat.module}' does not
match a valid id pattern. @ line 979, column 21
[ERROR] 'dependencyManagement.dependencies.dependency.artifactId' for
org.apache.hbase:${compat.module}:test-jar with value '${compat.module}' does
not match a valid id pattern. @ line 984, column 21
{code}
We should enhance test-patch.sh so that patch with hadoop-2.0 as default
profile doesn't go through validation step against hadoop-2.0
Ideally, the changes in various pom.xml files should be saved as template.
User can specify the hadoop profile to test against in the header of patch
file.
e.g.
{code}
This patch uses hadoop-2.0 as default profile
Index:
hbase-server/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8778) Region assigments scan table directory making them slow for huge tables
[
https://issues.apache.org/jira/browse/HBASE-8778?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13715673#comment-13715673
]
Dave Latham commented on HBASE-8778:
The directory modtime check looks good to me. For us we will probably stick
with the patch I have above for awhile, but I think the modtime would help
others.
For snapshots I can't find a good reference for their actual hdfs layout, but
from a brief look at the code I think it tries to mirror an actual table
directory except for using hfile refs. For #1 I wonder if there are guarantees
that would make sure we can find all snapshots during a migration.
Region assigments scan table directory making them slow for huge tables
---
Key: HBASE-8778
URL: https://issues.apache.org/jira/browse/HBASE-8778
Project: HBase
Issue Type: Improvement
Reporter: Dave Latham
Assignee: Dave Latham
Fix For: 0.98.0, 0.95.2, 0.94.11
Attachments: 8778-dirmodtime.txt, HBASE-8778-0.94.5.patch,
HBASE-8778-0.94.5-v2.patch
On a table with 130k regions it takes about 3 seconds for a region server to
open a region once it has been assigned.
Watching the threads for a region server running 0.94.5 that is opening many
such regions shows the thread opening the reigon in code like this:
{noformat}
PRI IPC Server handler 4 on 60020 daemon prio=10 tid=0x2aaac07e9000
nid=0x6566 runnable [0x4c46d000]
java.lang.Thread.State: RUNNABLE
at java.lang.String.indexOf(String.java:1521)
at java.net.URI$Parser.scan(URI.java:2912)
at java.net.URI$Parser.parse(URI.java:3004)
at java.net.URI.init(URI.java:736)
at org.apache.hadoop.fs.Path.initialize(Path.java:145)
at org.apache.hadoop.fs.Path.init(Path.java:126)
at org.apache.hadoop.fs.Path.init(Path.java:50)
at
org.apache.hadoop.hdfs.protocol.HdfsFileStatus.getFullPath(HdfsFileStatus.java:215)
at
org.apache.hadoop.hdfs.DistributedFileSystem.makeQualified(DistributedFileSystem.java:252)
at
org.apache.hadoop.hdfs.DistributedFileSystem.listStatus(DistributedFileSystem.java:311)
at
org.apache.hadoop.fs.FilterFileSystem.listStatus(FilterFileSystem.java:159)
at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:842)
at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:867)
at org.apache.hadoop.hbase.util.FSUtils.listStatus(FSUtils.java:1168)
at
org.apache.hadoop.hbase.util.FSTableDescriptors.getTableInfoPath(FSTableDescriptors.java:269)
at
org.apache.hadoop.hbase.util.FSTableDescriptors.getTableInfoPath(FSTableDescriptors.java:255)
at
org.apache.hadoop.hbase.util.FSTableDescriptors.getTableInfoModtime(FSTableDescriptors.java:368)
at
org.apache.hadoop.hbase.util.FSTableDescriptors.get(FSTableDescriptors.java:155)
at
org.apache.hadoop.hbase.util.FSTableDescriptors.get(FSTableDescriptors.java:126)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.openRegion(HRegionServer.java:2834)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.openRegion(HRegionServer.java:2807)
at sun.reflect.GeneratedMethodAccessor64.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:320)
at
org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1426)
{noformat}
To open the region, the region server first loads the latest
HTableDescriptor. Since HBASE-4553 HTableDescriptor's are stored in the file
system at /hbase/tableDir/.tableinfo.sequenceNum. The file with the
largest sequenceNum is the current descriptor. This is done so that the
current descirptor is updated atomically. However, since the filename is not
known in advance FSTableDescriptors it has to do a FileSystem.listStatus
operation which has to list all files in the directory to find it. The
directory also contains all the region directories, so in our case it has to
load 130k FileStatus objects. Even using a globStatus matching function
still transfers all the objects to the client before performing the pattern
matching. Furthermore HDFS uses a default of transferring 1000 directory
entries in each RPC call, so it requires 130 roundtrips to the namenode to
fetch all the directory entries.
Consequently, to reassign all the regions of a table (or a constant fraction
thereof) requires time proportional to the square of the number of regions.
In our case, if a region server fails with 200
[jira] [Resolved] (HBASE-7841) Parallelize offline snapshot in DisabledTableSnapshotHandler
[
https://issues.apache.org/jira/browse/HBASE-7841?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu resolved HBASE-7841.
---
Resolution: Won't Fix
Parallelize offline snapshot in DisabledTableSnapshotHandler
Key: HBASE-7841
URL: https://issues.apache.org/jira/browse/HBASE-7841
Project: HBase
Issue Type: Sub-task
Reporter: Ted Yu
Assignee: Ted Yu
Fix For: 0.98.0
Attachments: 7841.txt, 7841-v2.txt, 7841-v3.txt, 7841-v4.txt
In DisabledTableSnapshotHandler, there is TODO:
{code}
// TODO consider parallelizing these operations since they are independent.
Right now its just
// easier to keep them serial though
@Override
public void snapshotRegions(ListPairHRegionInfo, ServerName
regionsAndLocations) throws IOException,
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8935) IntegrationTestBigLinkedList fails under load on 0.94 due to some scan issues - add logging
[
https://issues.apache.org/jira/browse/HBASE-8935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sergey Shelukhin updated HBASE-8935:
Summary: IntegrationTestBigLinkedList fails under load on 0.94 due to some
scan issues - add logging (was: IntegrationTestBigLinkedList fails under load
on 0.94 due to some scan issues)
IntegrationTestBigLinkedList fails under load on 0.94 due to some scan issues
- add logging
---
Key: HBASE-8935
URL: https://issues.apache.org/jira/browse/HBASE-8935
Project: HBase
Issue Type: Bug
Reporter: Sergey Shelukhin
Assignee: Sergey Shelukhin
Attachments: HBASE-8935-v0-94-logging.patch, HBASE-8935-v0.patch
We observe occasional failures (4-5k undefined and unreferenced nodes in the
list, running) when the cluster is stressed while running
IntegrationTestBigLinkedList on 0.94. Unfortunately debug logging is disabled.
If Verify is rerun after the fact, the data is there, so it's not data loss.
All the missing keys come from very narrow range from the very beginning of
one region; most of this region is not affected.
In the case I'm looking at, region range is
{code}\xD0\x0C`=%\xA2\xD3\xA8\x0A\xF8\x917\x05\x94\x1D ..
\xD4\x0Bx\xAF\x88\x0C\xF47\x9A\x9F{\xCE\x0E\x8A{code}
and problematic renage
{code}\xD0\x0C`QLD\xED2\xD5c\x8D\xDB5\x01\xD2H] ...
\xD0\x0D\x89)\x11\x0E8\xC5\x08o\xD7\xE3$\xB3\xAAu{code}
There are no splits and compactions on the region during MR job; there are no
compactions after that could have affected the data, although there is one
flush that happened long after, and region was moved and reopened (before I
ran verify manually that showed that data is in HBase).
One thing that happened was that the scanners used by all the map tasks had
lease expirations during the MR job that had missing data, some of them twice.
First scanner expiration for each task I looked at was exactly 1 minute after
Processing split log message, when the scanner is opened.
Only the tasks where the scanners have expired twice (2 of them) logged any
errors; presumably one expiration in each task happened after the reading was
finished, because everything was slow and scanner wasn't closed in time - no
errors or exceptions are logged in the tasks where scanner only expired once.
The job I ran afterwards had no scanner expirations so it does close them
correctly in normal case...
The task that was reading the problematic range had one scanner expiration
and didn't log any errors.
It's a little bit special (or it may be a total red herring) in that in other
tasks, scanner expired while the task was splitting partial outputs (which
may mean end of input reading?), whereas in the task that lost data spilling
started long (~40s) after the scanner expired.
However there's one another such task (spilling started 25s after scanner
expired) and it didn't log any errors and didn't lose data.
At this point I am not sure about the root cause but I suspect it might be
related to scanner expiration handling, given the patterns.
One thing for sure is that there isn't enough logging... so I will start with
adding that.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8690) Reduce unnecessary getFileStatus hdfs calls in TTL hfile and hlog cleanners
[ https://issues.apache.org/jira/browse/HBASE-8690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-8690: -- Resolution: Fixed Status: Resolved (was: Patch Available) Reduce unnecessary getFileStatus hdfs calls in TTL hfile and hlog cleanners --- Key: HBASE-8690 URL: https://issues.apache.org/jira/browse/HBASE-8690 Project: HBase Issue Type: Improvement Components: master Reporter: Liu Shaohui Assignee: Liu Shaohui Priority: Minor Attachments: 8690-trunk-v3.patch, 8690-trunk-v4.patch, HBASE-8690-0.94-v1.patch, HBASE-8690-trunk-v1.patch, HBASE-8690-trunk-v2.patch For each in file in archive dir, the TimeToLiveHFileCleaner need call getFileStatus to get the modify time of file. Actually the CleanerChore have had the file status when listing its parent dir. When we set the TTL to 7 days in our cluster for data security, the number of files left in archive dir is up to 65 thousands. In each clean period, TimeToLiveHFileCleaner will generate ten thousand getFileStatus call in a short time, which is very heavy for hdfs namenode. Fix: Change the path param to FileStatus in isFileDeletable method and reduce unnecessary getFileStatus hdfs calls in TTL cleaners. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8935) IntegrationTestBigLinkedList fails under load on 0.94 due to some scan issues - add logging
[
https://issues.apache.org/jira/browse/HBASE-8935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sergey Shelukhin updated HBASE-8935:
Fix Version/s: 0.94.10
0.95.2
IntegrationTestBigLinkedList fails under load on 0.94 due to some scan issues
- add logging
---
Key: HBASE-8935
URL: https://issues.apache.org/jira/browse/HBASE-8935
Project: HBase
Issue Type: Bug
Reporter: Sergey Shelukhin
Assignee: Sergey Shelukhin
Fix For: 0.95.2, 0.94.10
Attachments: HBASE-8935-v0-94-logging.patch, HBASE-8935-v0.patch
We observe occasional failures (4-5k undefined and unreferenced nodes in the
list, running) when the cluster is stressed while running
IntegrationTestBigLinkedList on 0.94. Unfortunately debug logging is disabled.
If Verify is rerun after the fact, the data is there, so it's not data loss.
All the missing keys come from very narrow range from the very beginning of
one region; most of this region is not affected.
In the case I'm looking at, region range is
{code}\xD0\x0C`=%\xA2\xD3\xA8\x0A\xF8\x917\x05\x94\x1D ..
\xD4\x0Bx\xAF\x88\x0C\xF47\x9A\x9F{\xCE\x0E\x8A{code}
and problematic renage
{code}\xD0\x0C`QLD\xED2\xD5c\x8D\xDB5\x01\xD2H] ...
\xD0\x0D\x89)\x11\x0E8\xC5\x08o\xD7\xE3$\xB3\xAAu{code}
There are no splits and compactions on the region during MR job; there are no
compactions after that could have affected the data, although there is one
flush that happened long after, and region was moved and reopened (before I
ran verify manually that showed that data is in HBase).
One thing that happened was that the scanners used by all the map tasks had
lease expirations during the MR job that had missing data, some of them twice.
First scanner expiration for each task I looked at was exactly 1 minute after
Processing split log message, when the scanner is opened.
Only the tasks where the scanners have expired twice (2 of them) logged any
errors; presumably one expiration in each task happened after the reading was
finished, because everything was slow and scanner wasn't closed in time - no
errors or exceptions are logged in the tasks where scanner only expired once.
The job I ran afterwards had no scanner expirations so it does close them
correctly in normal case...
The task that was reading the problematic range had one scanner expiration
and didn't log any errors.
It's a little bit special (or it may be a total red herring) in that in other
tasks, scanner expired while the task was splitting partial outputs (which
may mean end of input reading?), whereas in the task that lost data spilling
started long (~40s) after the scanner expired.
However there's one another such task (spilling started 25s after scanner
expired) and it didn't log any errors and didn't lose data.
At this point I am not sure about the root cause but I suspect it might be
related to scanner expiration handling, given the patterns.
One thing for sure is that there isn't enough logging... so I will start with
adding that.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-9008) Reenable TestReplicationKillSlaveRS.killOneSlaveRS
[ https://issues.apache.org/jira/browse/HBASE-9008?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jean-Daniel Cryans updated HBASE-9008: -- Attachment: HBASE-9008.patch This patch re-enables the test and changes the way we kill region servers. I think there's a bug in the way we read from HLogs with half-written edits, so instead of forcefully expiring the session I'm doing a clean shutdown. I have to write more low-level unit tests to verify this, but in the mean time I'm hoping this will make the tests less flakey. I'm also adding more stats to when we are done recovering a queue so that we can see up to where it replicated. Reenable TestReplicationKillSlaveRS.killOneSlaveRS -- Key: HBASE-9008 URL: https://issues.apache.org/jira/browse/HBASE-9008 Project: HBase Issue Type: Bug Components: test Reporter: stack Assignee: Jean-Daniel Cryans Attachments: HBASE-9008.patch hbase-9007 disabled it as flakey. See it for listing of fails and HBASE-8961. Assigning to [~jdcryans] to take a looksee. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9008) Reenable TestReplicationKillSlaveRS.killOneSlaveRS
[ https://issues.apache.org/jira/browse/HBASE-9008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13715692#comment-13715692 ] stack commented on HBASE-9008: -- +1 on patch. Thanks [~jdcryans] Reenable TestReplicationKillSlaveRS.killOneSlaveRS -- Key: HBASE-9008 URL: https://issues.apache.org/jira/browse/HBASE-9008 Project: HBase Issue Type: Bug Components: test Reporter: stack Assignee: Jean-Daniel Cryans Attachments: HBASE-9008.patch hbase-9007 disabled it as flakey. See it for listing of fails and HBASE-8961. Assigning to [~jdcryans] to take a looksee. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8642) [Snapshot] List and delete snapshot by table
[ https://issues.apache.org/jira/browse/HBASE-8642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13715694#comment-13715694 ] Matteo Bertozzi commented on HBASE-8642: patch looks good to me, now doesn't apply cleanly but the fix is trivial. my main concern with this feature is related to the rename semantic, where you rename a table and listing/deleting with the old rename results in including the snapshots made on before the rename table. Which may be fine, since may be considered not totally wrong and we don't have rename support in the core. so, I'm +1 on the code and +0 on the merge, I'll let you decide. [Snapshot] List and delete snapshot by table Key: HBASE-8642 URL: https://issues.apache.org/jira/browse/HBASE-8642 Project: HBase Issue Type: Improvement Components: snapshots Affects Versions: 0.98.0, 0.95.0, 0.95.1, 0.95.2 Reporter: Julian Zhou Assignee: Julian Zhou Priority: Minor Fix For: 0.98.0, 0.95.0, 0.95.2 Attachments: 8642-trunk-0.95-v0.patch, 8642-trunk-0.95-v1.patch, 8642-trunk-0.95-v2.patch Support list and delete snapshot by table name. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8065) bulkload can load the hfile into hbase table,but this mechanism can't remove prior data
[ https://issues.apache.org/jira/browse/HBASE-8065?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-8065: -- Status: Open (was: Patch Available) bulkload can load the hfile into hbase table,but this mechanism can't remove prior data --- Key: HBASE-8065 URL: https://issues.apache.org/jira/browse/HBASE-8065 Project: HBase Issue Type: Improvement Components: IPC/RPC, mapreduce, regionserver Affects Versions: 0.94.0 Environment: hadoop-1.0.2、hbase-0.94.0 Reporter: Yuan Kang Assignee: Yuan Kang Priority: Critical Attachments: LoadIncrementalHFiles-bulkload-can-clean-olddata.patch this patch can do bulkload for one more parameter ‘need to refresh’,when this parameter is true.bulkload can clean the old date in the hbase table ,then do the new date load -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8690) Reduce unnecessary getFileStatus hdfs calls in TTL hfile and hlog cleanners
[ https://issues.apache.org/jira/browse/HBASE-8690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl updated HBASE-8690: - Fix Version/s: 0.98.0 Reduce unnecessary getFileStatus hdfs calls in TTL hfile and hlog cleanners --- Key: HBASE-8690 URL: https://issues.apache.org/jira/browse/HBASE-8690 Project: HBase Issue Type: Improvement Components: master Reporter: Liu Shaohui Assignee: Liu Shaohui Priority: Minor Fix For: 0.98.0 Attachments: 8690-trunk-v3.patch, 8690-trunk-v4.patch, HBASE-8690-0.94-v1.patch, HBASE-8690-trunk-v1.patch, HBASE-8690-trunk-v2.patch For each in file in archive dir, the TimeToLiveHFileCleaner need call getFileStatus to get the modify time of file. Actually the CleanerChore have had the file status when listing its parent dir. When we set the TTL to 7 days in our cluster for data security, the number of files left in archive dir is up to 65 thousands. In each clean period, TimeToLiveHFileCleaner will generate ten thousand getFileStatus call in a short time, which is very heavy for hdfs namenode. Fix: Change the path param to FileStatus in isFileDeletable method and reduce unnecessary getFileStatus hdfs calls in TTL cleaners. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8690) Reduce unnecessary getFileStatus hdfs calls in TTL hfile and hlog cleanners
[ https://issues.apache.org/jira/browse/HBASE-8690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13715735#comment-13715735 ] Lars Hofhansl commented on HBASE-8690: -- Seems both 0.94 and 0.95 would benefit from this. Reduce unnecessary getFileStatus hdfs calls in TTL hfile and hlog cleanners --- Key: HBASE-8690 URL: https://issues.apache.org/jira/browse/HBASE-8690 Project: HBase Issue Type: Improvement Components: master Reporter: Liu Shaohui Assignee: Liu Shaohui Priority: Minor Fix For: 0.98.0 Attachments: 8690-trunk-v3.patch, 8690-trunk-v4.patch, HBASE-8690-0.94-v1.patch, HBASE-8690-trunk-v1.patch, HBASE-8690-trunk-v2.patch For each in file in archive dir, the TimeToLiveHFileCleaner need call getFileStatus to get the modify time of file. Actually the CleanerChore have had the file status when listing its parent dir. When we set the TTL to 7 days in our cluster for data security, the number of files left in archive dir is up to 65 thousands. In each clean period, TimeToLiveHFileCleaner will generate ten thousand getFileStatus call in a short time, which is very heavy for hdfs namenode. Fix: Change the path param to FileStatus in isFileDeletable method and reduce unnecessary getFileStatus hdfs calls in TTL cleaners. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-9017) Consolidate ChaosMonkey's random objects
[ https://issues.apache.org/jira/browse/HBASE-9017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jimmy Xiang updated HBASE-9017: --- Attachment: trunk-9017.patch Consolidate ChaosMonkey's random objects Key: HBASE-9017 URL: https://issues.apache.org/jira/browse/HBASE-9017 Project: HBase Issue Type: Test Reporter: Jimmy Xiang Assignee: Jimmy Xiang Priority: Minor Attachments: trunk-9017.patch ChaosMonkey has several random objects. We can just share the same random object. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-9017) Consolidate ChaosMonkey's random objects
[ https://issues.apache.org/jira/browse/HBASE-9017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jimmy Xiang updated HBASE-9017: --- Fix Version/s: 0.95.2 0.98.0 Status: Patch Available (was: Open) Consolidate ChaosMonkey's random objects Key: HBASE-9017 URL: https://issues.apache.org/jira/browse/HBASE-9017 Project: HBase Issue Type: Test Reporter: Jimmy Xiang Assignee: Jimmy Xiang Priority: Minor Fix For: 0.98.0, 0.95.2 Attachments: trunk-9017.patch ChaosMonkey has several random objects. We can just share the same random object. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HBASE-9019) Port HBASE-8690: Reduce unnecessary getFileStatus hdfs calls in TTL hfile and hlog cleanners to 0.94
Ted Yu created HBASE-9019: - Summary: Port HBASE-8690: Reduce unnecessary getFileStatus hdfs calls in TTL hfile and hlog cleanners to 0.94 Key: HBASE-9019 URL: https://issues.apache.org/jira/browse/HBASE-9019 Project: HBase Issue Type: Improvement Reporter: Ted Yu Priority: Minor See comment here: https://issues.apache.org/jira/browse/HBASE-8690?focusedCommentId=13715735page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13715735 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9008) Reenable TestReplicationKillSlaveRS.killOneSlaveRS
[ https://issues.apache.org/jira/browse/HBASE-9008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13715739#comment-13715739 ] stack commented on HBASE-9008: -- bq. stack that this is failing means either tailing HLogs is broken or that we have data loss in 0.96. I am trying to get us to a state where clean builds are the default. Toward that end, I am disabling tests that no one is looking at or that folks are looking at but meantime the build is failing because of the test so I can move on to find the next obstacle to a clean build. Most tests I try to triage. Tests I can't make sense of w/o spending a good bit of time on, I disable and try and get an expert to look into it. It'd be cool if you were able to find we are losing edits via this unit test. Thanks for looking. Reenable TestReplicationKillSlaveRS.killOneSlaveRS -- Key: HBASE-9008 URL: https://issues.apache.org/jira/browse/HBASE-9008 Project: HBase Issue Type: Bug Components: test Reporter: stack Assignee: Jean-Daniel Cryans Attachments: HBASE-9008.patch hbase-9007 disabled it as flakey. See it for listing of fails and HBASE-8961. Assigning to [~jdcryans] to take a looksee. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HBASE-9019) Port HBASE-8690: Reduce unnecessary getFileStatus hdfs calls in TTL hfile and hlog cleanners to 0.94
[ https://issues.apache.org/jira/browse/HBASE-9019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack resolved HBASE-9019. -- Resolution: Invalid Resolving issue ill-specified. When I click on the link it takes me to the end of the issue; I should not have to read a whole issue to figure out what a new issue is about. If you don't know how to make an issue, don't do it [~ted_yu] Port HBASE-8690: Reduce unnecessary getFileStatus hdfs calls in TTL hfile and hlog cleanners to 0.94 Key: HBASE-9019 URL: https://issues.apache.org/jira/browse/HBASE-9019 Project: HBase Issue Type: Improvement Reporter: Ted Yu Priority: Minor See comment here: https://issues.apache.org/jira/browse/HBASE-8690?focusedCommentId=13715735page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13715735 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9017) Consolidate ChaosMonkey's random objects
[ https://issues.apache.org/jira/browse/HBASE-9017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13715747#comment-13715747 ] stack commented on HBASE-9017: -- lgtm Consolidate ChaosMonkey's random objects Key: HBASE-9017 URL: https://issues.apache.org/jira/browse/HBASE-9017 Project: HBase Issue Type: Test Reporter: Jimmy Xiang Assignee: Jimmy Xiang Priority: Minor Fix For: 0.98.0, 0.95.2 Attachments: trunk-9017.patch ChaosMonkey has several random objects. We can just share the same random object. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-9008) Reenable TestReplicationKillSlaveRS.killOneSlaveRS
[ https://issues.apache.org/jira/browse/HBASE-9008?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jean-Daniel Cryans updated HBASE-9008: -- Status: Patch Available (was: Open) Reenable TestReplicationKillSlaveRS.killOneSlaveRS -- Key: HBASE-9008 URL: https://issues.apache.org/jira/browse/HBASE-9008 Project: HBase Issue Type: Bug Components: test Reporter: stack Assignee: Jean-Daniel Cryans Attachments: HBASE-9008.patch hbase-9007 disabled it as flakey. See it for listing of fails and HBASE-8961. Assigning to [~jdcryans] to take a looksee. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8690) Reduce unnecessary getFileStatus hdfs calls in TTL hfile and hlog cleanners
[ https://issues.apache.org/jira/browse/HBASE-8690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-8690: - Attachment: 8690v4.095.txt What I applied to 0.95. (Trunk patch didn't apply. Interfaces have had the 'public' removed by LarsF from method names etc., so would not apply as is to 0.94) Reduce unnecessary getFileStatus hdfs calls in TTL hfile and hlog cleanners --- Key: HBASE-8690 URL: https://issues.apache.org/jira/browse/HBASE-8690 Project: HBase Issue Type: Improvement Components: master Reporter: Liu Shaohui Assignee: Liu Shaohui Priority: Minor Fix For: 0.98.0 Attachments: 8690-trunk-v3.patch, 8690-trunk-v4.patch, 8690v4.095.txt, HBASE-8690-0.94-v1.patch, HBASE-8690-trunk-v1.patch, HBASE-8690-trunk-v2.patch For each in file in archive dir, the TimeToLiveHFileCleaner need call getFileStatus to get the modify time of file. Actually the CleanerChore have had the file status when listing its parent dir. When we set the TTL to 7 days in our cluster for data security, the number of files left in archive dir is up to 65 thousands. In each clean period, TimeToLiveHFileCleaner will generate ten thousand getFileStatus call in a short time, which is very heavy for hdfs namenode. Fix: Change the path param to FileStatus in isFileDeletable method and reduce unnecessary getFileStatus hdfs calls in TTL cleaners. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
