[jira] [Commented] (HBASE-10400) [hbck] Continue if region dir missing on region merge attempt
[ https://issues.apache.org/jira/browse/HBASE-10400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13881750#comment-13881750 ] Hudson commented on HBASE-10400: FAILURE: Integrated in HBase-0.94-security #392 (See [https://builds.apache.org/job/HBase-0.94-security/392/]) HBASE-10400 [hbck] Continue if region dir missing on region merge attempt (jmhsieh: rev 1560579) * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java [hbck] Continue if region dir missing on region merge attempt - Key: HBASE-10400 URL: https://issues.apache.org/jira/browse/HBASE-10400 Project: HBase Issue Type: Bug Components: hbck Affects Versions: 0.92.2, 0.94.16, 0.96.1.1 Reporter: Jonathan Hsieh Assignee: Jonathan Hsieh Fix For: 0.98.0, 0.96.2, 0.99.0, 0.94.17 Attachments: hbase-10400.patch In a recent support case, the hbck repair tool would eventually hang because we didn't handle the case where merge info is old using hadoop2 fs.listStatus semantics (throw exn instead of return null). this is a trivial patch that handles the newer hadoop2 semantics. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HBASE-10383) Secure Bulk Load for 'completebulkload' fails for version 0.94.15
[ https://issues.apache.org/jira/browse/HBASE-10383?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13881751#comment-13881751 ] Hudson commented on HBASE-10383: FAILURE: Integrated in HBase-0.94-security #392 (See [https://builds.apache.org/job/HBase-0.94-security/392/]) HBASE-10383 Secure Bulk Load for 'completebulkload' fails for version 0.94.15 (jyates: rev 1561243) * /hbase/branches/0.94/security/src/main/java/org/apache/hadoop/hbase/security/access/SecureBulkLoadEndpoint.java * /hbase/branches/0.94/security/src/main/java/org/apache/hadoop/hbase/security/access/SecureBulkLoadProtocol.java * /hbase/branches/0.94/security/src/test/java/org/apache/hadoop/hbase/mapreduce/TestSecureLoadIncrementalHFiles.java * /hbase/branches/0.94/security/src/test/java/org/apache/hadoop/hbase/mapreduce/TestSecureLoadIncrementalHFilesSplitRecovery.java * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/coprocessor/SecureBulkLoadClient.java * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/mapreduce/LoadIncrementalHFiles.java * /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/HadoopSecurityEnabledUserProviderForTesting.java * /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestLoadIncrementalHFiles.java * /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestLoadIncrementalHFilesSplitRecovery.java Secure Bulk Load for 'completebulkload' fails for version 0.94.15 - Key: HBASE-10383 URL: https://issues.apache.org/jira/browse/HBASE-10383 Project: HBase Issue Type: Bug Components: Coprocessors Affects Versions: 0.94.15 Reporter: Kashif J S Assignee: Jesse Yates Priority: Critical Fix For: 0.94.17 Attachments: 10383.txt, HBASE-10383-v2.patch, hbase-10383-jyates-0.94-v0.patch Secure Bulk Load with kerberos enabled fails for Complete Bulk LoadLoadIncrementalHfile with following exception ERROR org.apache.hadoop.hbase.regionserver.HRegionServer: org.apache.hadoop.hbase.ipc.HBaseRPC$UnknownProtocolException: No matching handler for protocol org.apache.hadoop.hbase.security.access.SecureBulkLoadProtocol in region t1,,1389699438035.28bb0284d971d0676cf562efea80199b. at org.apache.hadoop.hbase.regionserver.HRegion.exec(HRegion.java) at org.apache.hadoop.hbase.regionserver.HRegionServer.execCoprocessor(HRegionServer.java) at sun.reflect.GeneratedMethodAccessor18.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java) at java.lang.reflect.Method.invoke(Method.java) at org.apache.hadoop.hbase.ipc.SecureRpcEngine$Server.call(SecureRpcEngine.java) at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java) -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HBASE-10401) [hbck] perform overlap group merges in parallel
[ https://issues.apache.org/jira/browse/HBASE-10401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13881749#comment-13881749 ] Hudson commented on HBASE-10401: FAILURE: Integrated in HBase-0.94-security #392 (See [https://builds.apache.org/job/HBase-0.94-security/392/]) HBASE-10401 [hbck] perform overlap group merges in parallel (jmhsieh: rev 1560776) * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java [hbck] perform overlap group merges in parallel --- Key: HBASE-10401 URL: https://issues.apache.org/jira/browse/HBASE-10401 Project: HBase Issue Type: Bug Components: hbck Affects Versions: 0.92.2, 0.98.0, 0.94.16, 0.99.0, 0.96.1.1 Reporter: Jonathan Hsieh Assignee: Jonathan Hsieh Fix For: 0.98.0, 0.96.2, 0.99.0, 0.94.17 Attachments: hbase-10401.10400.patch, hbase-10401.patch, hbase-10401.v2.94.patch, hbase-10401.v2.patch In a recent support case, we encountered a corrupt hbase that had thousands of overlap groups (regions that had overlapping key ranges). The current implementation repairs these by serially taking a group, perorming a merge and then moving on to the next group. Because assignments and hdfs nn operations are involved each merge could take on the order of seconds. With thousands of overlap groups, this could take hours to complete. This patch makes it so that these independent merge groups are merged in parallel. It uses the same thread pool for other fs info-gathering operations. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HBASE-10156) Fix up the HBASE-8755 slowdown when low contention
[
https://issues.apache.org/jira/browse/HBASE-10156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13881752#comment-13881752
]
Hadoop QA commented on HBASE-10156:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12625129/10156v20.txt
against trunk revision .
ATTACHMENT ID: 12625129
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 30 new
or modified tests.
{color:green}+1 hadoop1.0{color}. The patch compiles against the hadoop
1.0 profile.
{color:green}+1 hadoop1.1{color}. The patch compiles against the hadoop
1.1 profile.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:red}-1 findbugs{color}. The patch appears to introduce 1 new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:red}-1 site{color}. The patch appears to cause mvn site goal to
fail.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/8503//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8503//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8503//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8503//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8503//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8503//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8503//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8503//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8503//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8503//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/8503//console
This message is automatically generated.
Fix up the HBASE-8755 slowdown when low contention
--
Key: HBASE-10156
URL: https://issues.apache.org/jira/browse/HBASE-10156
Project: HBase
Issue Type: Sub-task
Components: wal
Reporter: stack
Assignee: stack
Attachments: 10156.txt, 10156v10.txt, 10156v11.txt, 10156v12.txt,
10156v12.txt, 10156v13.txt, 10156v16.txt, 10156v17.txt, 10156v18.txt,
10156v19.txt, 10156v2.txt, 10156v20.txt, 10156v3.txt, 10156v4.txt,
10156v5.txt, 10156v6.txt, 10156v7.txt, 10156v9.txt, Disrupting.java
HBASE-8755 slows our writes when only a few clients. Fix.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (HBASE-10406) Column family option is not effective in IntegrationTestSendTraceRequests
[
https://issues.apache.org/jira/browse/HBASE-10406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13881757#comment-13881757
]
Hudson commented on HBASE-10406:
SUCCESS: Integrated in HBase-0.98-on-Hadoop-1.1 #100 (See
[https://builds.apache.org/job/HBase-0.98-on-Hadoop-1.1/100/])
HBASE-10406 Column family option is not effective in
IntegrationTestSendTraceRequests (tedyu: rev 1560879)
*
/hbase/branches/0.98/hbase-it/src/test/java/org/apache/hadoop/hbase/trace/IntegrationTestSendTraceRequests.java
Column family option is not effective in IntegrationTestSendTraceRequests
-
Key: HBASE-10406
URL: https://issues.apache.org/jira/browse/HBASE-10406
Project: HBase
Issue Type: Test
Reporter: Ted Yu
Assignee: Ted Yu
Fix For: 0.98.0, 0.99.0
Attachments: 10406-v1.txt
If column family option (-f) is specified, the following error would occur:
{code}
2014-01-23 23:21:20,347 [Thread-2] ERROR util.AbstractHBaseTool
(AbstractHBaseTool.java:run(97)) - Error when parsing command-line arguemnts
org.apache.commons.cli.UnrecognizedOptionException: Unrecognized option: -f
at org.apache.commons.cli.Parser.processOption(Parser.java:363)
at org.apache.commons.cli.Parser.parse(Parser.java:199)
at org.apache.commons.cli.Parser.parse(Parser.java:85)
at
org.apache.hadoop.hbase.util.AbstractHBaseTool.parseArgs(AbstractHBaseTool.java:135)
at
org.apache.hadoop.hbase.util.AbstractHBaseTool.run(AbstractHBaseTool.java:94)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
{code}
This is due to table option is used twice in the following code:
{code}
protected void addOptions() {
addOptWithArg(TABLE_ARG, The table name to target. Will be created if
not there already.);
addOptWithArg(TABLE_ARG, The family to target);
{code}
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (HBASE-10412) Distributed log replay : Cell tags getting missed
[ https://issues.apache.org/jira/browse/HBASE-10412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13881756#comment-13881756 ] Hudson commented on HBASE-10412: SUCCESS: Integrated in HBase-0.98-on-Hadoop-1.1 #100 (See [https://builds.apache.org/job/HBase-0.98-on-Hadoop-1.1/100/]) HBASE-10412 Distributed log replay: Cell tags getting missed (Anoop Sam John) (apurtell: rev 1561231) * /hbase/branches/0.98/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLogSplitter.java * /hbase/branches/0.98/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/WALCellCodec.java * /hbase/branches/0.98/hbase-server/src/test/java/org/apache/hadoop/hbase/security/visibility/TestVisibilityLabels.java * /hbase/branches/0.98/hbase-server/src/test/java/org/apache/hadoop/hbase/security/visibility/TestVisibilityLabelsWithDistributedLogReplay.java Distributed log replay : Cell tags getting missed - Key: HBASE-10412 URL: https://issues.apache.org/jira/browse/HBASE-10412 Project: HBase Issue Type: Bug Affects Versions: 0.98.0 Reporter: Anoop Sam John Assignee: Anoop Sam John Priority: Blocker Fix For: 0.98.0, 0.99.0 Attachments: HBASE-10412.patch, HBASE-10412_V2.patch This is caused by HBASE-10322. The default RPC codec KVCodec strips tags. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HBASE-10408) Intermittent TestDistributedLogSplitting#testLogReplayForDisablingTable failure
[
https://issues.apache.org/jira/browse/HBASE-10408?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13881759#comment-13881759
]
Hudson commented on HBASE-10408:
SUCCESS: Integrated in HBase-0.98-on-Hadoop-1.1 #100 (See
[https://builds.apache.org/job/HBase-0.98-on-Hadoop-1.1/100/])
HBASE-10408 Intermittent
TestDistributedLogSplitting#testLogReplayForDisablingTable failure (tedyu: rev
1561118)
*
/hbase/branches/0.98/hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestDistributedLogSplitting.java
Intermittent TestDistributedLogSplitting#testLogReplayForDisablingTable
failure
---
Key: HBASE-10408
URL: https://issues.apache.org/jira/browse/HBASE-10408
Project: HBase
Issue Type: Test
Reporter: Ted Yu
Assignee: Ted Yu
Fix For: 0.98.0, 0.99.0
Attachments: 10408-v1.txt, 10408-v2.txt, 10408-v3.txt,
TestDistributedLogSplitting.out,
testDistributedLogSplitting-disablingTable.html
Here was the assertion failure:
{code} java.lang.AssertionError: expected:lt;1000 but was:lt;0 at
org.junit.Assert.fail(Assert.java:88) at
org.junit.Assert.failNotEquals(Assert.java:743) at
org.junit.Assert.assertEquals(Assert.java:118) at
org.junit.Assert.assertEquals(Assert.java:555) at
org.junit.Assert.assertEquals(Assert.java:542) at
org.apache.hadoop.hbase.master.TestDistributedLogSplitting.testLogReplayForDisablingTable(TestDistributedLogSplitting.java:838)
{code}
This was due to the loop starting around line 823 not picking up any edits
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (HBASE-10365) HBaseFsck should clean up connection properly when repair is completed
[ https://issues.apache.org/jira/browse/HBASE-10365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13881760#comment-13881760 ] Hudson commented on HBASE-10365: SUCCESS: Integrated in HBase-0.98-on-Hadoop-1.1 #100 (See [https://builds.apache.org/job/HBase-0.98-on-Hadoop-1.1/100/]) HBASE-10365 HBaseFsck should clean up connection properly when repair is completed (Ted Yu) (tedyu: rev 1560581) * /hbase/branches/0.98/hbase-server/src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java HBaseFsck should clean up connection properly when repair is completed -- Key: HBASE-10365 URL: https://issues.apache.org/jira/browse/HBASE-10365 Project: HBase Issue Type: Bug Reporter: Ted Yu Assignee: Ted Yu Fix For: 0.98.0, 0.99.0 Attachments: 10365-v1.txt At the end of exec() method, connections to the cluster are not properly released. Connections should be released upon completion of repair. This was mentioned by Jean-Marc in the thread '[VOTE] The 1st hbase 0.94.16 release candidate is available for download' -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HBASE-10401) [hbck] perform overlap group merges in parallel
[ https://issues.apache.org/jira/browse/HBASE-10401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13881754#comment-13881754 ] Hudson commented on HBASE-10401: SUCCESS: Integrated in HBase-0.98-on-Hadoop-1.1 #100 (See [https://builds.apache.org/job/HBase-0.98-on-Hadoop-1.1/100/]) HBASE-10401 [hbck] perform overlap group merges in parallel (jmhsieh: rev 1560779) * /hbase/branches/0.98/hbase-server/src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java [hbck] perform overlap group merges in parallel --- Key: HBASE-10401 URL: https://issues.apache.org/jira/browse/HBASE-10401 Project: HBase Issue Type: Bug Components: hbck Affects Versions: 0.92.2, 0.98.0, 0.94.16, 0.99.0, 0.96.1.1 Reporter: Jonathan Hsieh Assignee: Jonathan Hsieh Fix For: 0.98.0, 0.96.2, 0.99.0, 0.94.17 Attachments: hbase-10401.10400.patch, hbase-10401.patch, hbase-10401.v2.94.patch, hbase-10401.v2.patch In a recent support case, we encountered a corrupt hbase that had thousands of overlap groups (regions that had overlapping key ranges). The current implementation repairs these by serially taking a group, perorming a merge and then moving on to the next group. Because assignments and hdfs nn operations are involved each merge could take on the order of seconds. With thousands of overlap groups, this could take hours to complete. This patch makes it so that these independent merge groups are merged in parallel. It uses the same thread pool for other fs info-gathering operations. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HBASE-10400) [hbck] Continue if region dir missing on region merge attempt
[ https://issues.apache.org/jira/browse/HBASE-10400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13881755#comment-13881755 ] Hudson commented on HBASE-10400: SUCCESS: Integrated in HBase-0.98-on-Hadoop-1.1 #100 (See [https://builds.apache.org/job/HBase-0.98-on-Hadoop-1.1/100/]) HBASE-10400 [hbck] Continue if region dir missing on region merge attempt (jmhsieh: rev 1560578) * /hbase/branches/0.98/hbase-server/src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java [hbck] Continue if region dir missing on region merge attempt - Key: HBASE-10400 URL: https://issues.apache.org/jira/browse/HBASE-10400 Project: HBase Issue Type: Bug Components: hbck Affects Versions: 0.92.2, 0.94.16, 0.96.1.1 Reporter: Jonathan Hsieh Assignee: Jonathan Hsieh Fix For: 0.98.0, 0.96.2, 0.99.0, 0.94.17 Attachments: hbase-10400.patch In a recent support case, the hbck repair tool would eventually hang because we didn't handle the case where merge info is old using hadoop2 fs.listStatus semantics (throw exn instead of return null). this is a trivial patch that handles the newer hadoop2 semantics. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HBASE-10409) Allow config options related to zookeeper to be passed to IntegrationTestsDriver
[
https://issues.apache.org/jira/browse/HBASE-10409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13881758#comment-13881758
]
Hudson commented on HBASE-10409:
SUCCESS: Integrated in HBase-0.98-on-Hadoop-1.1 #100 (See
[https://builds.apache.org/job/HBase-0.98-on-Hadoop-1.1/100/])
HBASE-10409 Allow config options related to zookeeper to be passed to
IntegrationTestsDriver - revert (tedyu: rev 1560901)
*
/hbase/branches/0.98/hbase-it/src/test/java/org/apache/hadoop/hbase/IntegrationTestsDriver.java
HBASE-10409 Allow config options related to zookeeper to be passed to
IntegrationTestsDriver (Ted Yu) (tedyu: rev 1560803)
*
/hbase/branches/0.98/hbase-it/src/test/java/org/apache/hadoop/hbase/IntegrationTestsDriver.java
Allow config options related to zookeeper to be passed to
IntegrationTestsDriver
Key: HBASE-10409
URL: https://issues.apache.org/jira/browse/HBASE-10409
Project: HBase
Issue Type: Test
Reporter: Ted Yu
Assignee: Ted Yu
Attachments: 10409-v1.txt
Currently IntegrationTestsDriver only accepts one parameter: regex for
integration test.
{code}
public static void main(String[] args) throws Exception {
int ret = ToolRunner.run(new IntegrationTestsDriver(), args);
{code}
To allow IntegrationTestsDriver to be activated through Java API, support for
passing config options related to zookeeper should be added - similarly to
what LoadTestTool supports.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (HBASE-10401) [hbck] perform overlap group merges in parallel
[ https://issues.apache.org/jira/browse/HBASE-10401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13881763#comment-13881763 ] Hudson commented on HBASE-10401: SUCCESS: Integrated in HBase-0.94-JDK7 #32 (See [https://builds.apache.org/job/HBase-0.94-JDK7/32/]) HBASE-10401 [hbck] perform overlap group merges in parallel (jmhsieh: rev 1560776) * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java [hbck] perform overlap group merges in parallel --- Key: HBASE-10401 URL: https://issues.apache.org/jira/browse/HBASE-10401 Project: HBase Issue Type: Bug Components: hbck Affects Versions: 0.92.2, 0.98.0, 0.94.16, 0.99.0, 0.96.1.1 Reporter: Jonathan Hsieh Assignee: Jonathan Hsieh Fix For: 0.98.0, 0.96.2, 0.99.0, 0.94.17 Attachments: hbase-10401.10400.patch, hbase-10401.patch, hbase-10401.v2.94.patch, hbase-10401.v2.patch In a recent support case, we encountered a corrupt hbase that had thousands of overlap groups (regions that had overlapping key ranges). The current implementation repairs these by serially taking a group, perorming a merge and then moving on to the next group. Because assignments and hdfs nn operations are involved each merge could take on the order of seconds. With thousands of overlap groups, this could take hours to complete. This patch makes it so that these independent merge groups are merged in parallel. It uses the same thread pool for other fs info-gathering operations. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HBASE-10400) [hbck] Continue if region dir missing on region merge attempt
[ https://issues.apache.org/jira/browse/HBASE-10400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13881764#comment-13881764 ] Hudson commented on HBASE-10400: SUCCESS: Integrated in HBase-0.94-JDK7 #32 (See [https://builds.apache.org/job/HBase-0.94-JDK7/32/]) HBASE-10400 [hbck] Continue if region dir missing on region merge attempt (jmhsieh: rev 1560579) * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java [hbck] Continue if region dir missing on region merge attempt - Key: HBASE-10400 URL: https://issues.apache.org/jira/browse/HBASE-10400 Project: HBase Issue Type: Bug Components: hbck Affects Versions: 0.92.2, 0.94.16, 0.96.1.1 Reporter: Jonathan Hsieh Assignee: Jonathan Hsieh Fix For: 0.98.0, 0.96.2, 0.99.0, 0.94.17 Attachments: hbase-10400.patch In a recent support case, the hbck repair tool would eventually hang because we didn't handle the case where merge info is old using hadoop2 fs.listStatus semantics (throw exn instead of return null). this is a trivial patch that handles the newer hadoop2 semantics. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HBASE-10383) Secure Bulk Load for 'completebulkload' fails for version 0.94.15
[ https://issues.apache.org/jira/browse/HBASE-10383?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13881765#comment-13881765 ] Hudson commented on HBASE-10383: SUCCESS: Integrated in HBase-0.94-JDK7 #32 (See [https://builds.apache.org/job/HBase-0.94-JDK7/32/]) HBASE-10383 Secure Bulk Load for 'completebulkload' fails for version 0.94.15 (jyates: rev 1561243) * /hbase/branches/0.94/security/src/main/java/org/apache/hadoop/hbase/security/access/SecureBulkLoadEndpoint.java * /hbase/branches/0.94/security/src/main/java/org/apache/hadoop/hbase/security/access/SecureBulkLoadProtocol.java * /hbase/branches/0.94/security/src/test/java/org/apache/hadoop/hbase/mapreduce/TestSecureLoadIncrementalHFiles.java * /hbase/branches/0.94/security/src/test/java/org/apache/hadoop/hbase/mapreduce/TestSecureLoadIncrementalHFilesSplitRecovery.java * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/coprocessor/SecureBulkLoadClient.java * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/mapreduce/LoadIncrementalHFiles.java * /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/HadoopSecurityEnabledUserProviderForTesting.java * /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestLoadIncrementalHFiles.java * /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestLoadIncrementalHFilesSplitRecovery.java Secure Bulk Load for 'completebulkload' fails for version 0.94.15 - Key: HBASE-10383 URL: https://issues.apache.org/jira/browse/HBASE-10383 Project: HBase Issue Type: Bug Components: Coprocessors Affects Versions: 0.94.15 Reporter: Kashif J S Assignee: Jesse Yates Priority: Critical Fix For: 0.94.17 Attachments: 10383.txt, HBASE-10383-v2.patch, hbase-10383-jyates-0.94-v0.patch Secure Bulk Load with kerberos enabled fails for Complete Bulk LoadLoadIncrementalHfile with following exception ERROR org.apache.hadoop.hbase.regionserver.HRegionServer: org.apache.hadoop.hbase.ipc.HBaseRPC$UnknownProtocolException: No matching handler for protocol org.apache.hadoop.hbase.security.access.SecureBulkLoadProtocol in region t1,,1389699438035.28bb0284d971d0676cf562efea80199b. at org.apache.hadoop.hbase.regionserver.HRegion.exec(HRegion.java) at org.apache.hadoop.hbase.regionserver.HRegionServer.execCoprocessor(HRegionServer.java) at sun.reflect.GeneratedMethodAccessor18.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java) at java.lang.reflect.Method.invoke(Method.java) at org.apache.hadoop.hbase.ipc.SecureRpcEngine$Server.call(SecureRpcEngine.java) at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java) -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HBASE-10401) [hbck] perform overlap group merges in parallel
[ https://issues.apache.org/jira/browse/HBASE-10401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13881766#comment-13881766 ] Hudson commented on HBASE-10401: FAILURE: Integrated in hbase-0.96 #269 (See [https://builds.apache.org/job/hbase-0.96/269/]) HBASE-10401 [hbck] perform overlap group merges in parallel (jmhsieh: rev 1560775) * /hbase/branches/0.96/hbase-server/src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java [hbck] perform overlap group merges in parallel --- Key: HBASE-10401 URL: https://issues.apache.org/jira/browse/HBASE-10401 Project: HBase Issue Type: Bug Components: hbck Affects Versions: 0.92.2, 0.98.0, 0.94.16, 0.99.0, 0.96.1.1 Reporter: Jonathan Hsieh Assignee: Jonathan Hsieh Fix For: 0.98.0, 0.96.2, 0.99.0, 0.94.17 Attachments: hbase-10401.10400.patch, hbase-10401.patch, hbase-10401.v2.94.patch, hbase-10401.v2.patch In a recent support case, we encountered a corrupt hbase that had thousands of overlap groups (regions that had overlapping key ranges). The current implementation repairs these by serially taking a group, perorming a merge and then moving on to the next group. Because assignments and hdfs nn operations are involved each merge could take on the order of seconds. With thousands of overlap groups, this could take hours to complete. This patch makes it so that these independent merge groups are merged in parallel. It uses the same thread pool for other fs info-gathering operations. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HBASE-10400) [hbck] Continue if region dir missing on region merge attempt
[ https://issues.apache.org/jira/browse/HBASE-10400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13881767#comment-13881767 ] Hudson commented on HBASE-10400: FAILURE: Integrated in hbase-0.96 #269 (See [https://builds.apache.org/job/hbase-0.96/269/]) HBASE-10400 [hbck] Continue if region dir missing on region merge attempt (jmhsieh: rev 1560580) * /hbase/branches/0.96/hbase-server/src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java [hbck] Continue if region dir missing on region merge attempt - Key: HBASE-10400 URL: https://issues.apache.org/jira/browse/HBASE-10400 Project: HBase Issue Type: Bug Components: hbck Affects Versions: 0.92.2, 0.94.16, 0.96.1.1 Reporter: Jonathan Hsieh Assignee: Jonathan Hsieh Fix For: 0.98.0, 0.96.2, 0.99.0, 0.94.17 Attachments: hbase-10400.patch In a recent support case, the hbck repair tool would eventually hang because we didn't handle the case where merge info is old using hadoop2 fs.listStatus semantics (throw exn instead of return null). this is a trivial patch that handles the newer hadoop2 semantics. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HBASE-10406) Column family option is not effective in IntegrationTestSendTraceRequests
[
https://issues.apache.org/jira/browse/HBASE-10406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13881780#comment-13881780
]
Hudson commented on HBASE-10406:
SUCCESS: Integrated in HBase-TRUNK-on-Hadoop-1.1 #63 (See
[https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-1.1/63/])
HBASE-10406 Column family option is not effective in
IntegrationTestSendTraceRequests (tedyu: rev 1560880)
*
/hbase/trunk/hbase-it/src/test/java/org/apache/hadoop/hbase/trace/IntegrationTestSendTraceRequests.java
Column family option is not effective in IntegrationTestSendTraceRequests
-
Key: HBASE-10406
URL: https://issues.apache.org/jira/browse/HBASE-10406
Project: HBase
Issue Type: Test
Reporter: Ted Yu
Assignee: Ted Yu
Fix For: 0.98.0, 0.99.0
Attachments: 10406-v1.txt
If column family option (-f) is specified, the following error would occur:
{code}
2014-01-23 23:21:20,347 [Thread-2] ERROR util.AbstractHBaseTool
(AbstractHBaseTool.java:run(97)) - Error when parsing command-line arguemnts
org.apache.commons.cli.UnrecognizedOptionException: Unrecognized option: -f
at org.apache.commons.cli.Parser.processOption(Parser.java:363)
at org.apache.commons.cli.Parser.parse(Parser.java:199)
at org.apache.commons.cli.Parser.parse(Parser.java:85)
at
org.apache.hadoop.hbase.util.AbstractHBaseTool.parseArgs(AbstractHBaseTool.java:135)
at
org.apache.hadoop.hbase.util.AbstractHBaseTool.run(AbstractHBaseTool.java:94)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
{code}
This is due to table option is used twice in the following code:
{code}
protected void addOptions() {
addOptWithArg(TABLE_ARG, The table name to target. Will be created if
not there already.);
addOptWithArg(TABLE_ARG, The family to target);
{code}
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (HBASE-10401) [hbck] perform overlap group merges in parallel
[ https://issues.apache.org/jira/browse/HBASE-10401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13881777#comment-13881777 ] Hudson commented on HBASE-10401: SUCCESS: Integrated in HBase-TRUNK-on-Hadoop-1.1 #63 (See [https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-1.1/63/]) HBASE-10401 [hbck] perform overlap group merges in parallel (jmhsieh: rev 1560774) * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java [hbck] perform overlap group merges in parallel --- Key: HBASE-10401 URL: https://issues.apache.org/jira/browse/HBASE-10401 Project: HBase Issue Type: Bug Components: hbck Affects Versions: 0.92.2, 0.98.0, 0.94.16, 0.99.0, 0.96.1.1 Reporter: Jonathan Hsieh Assignee: Jonathan Hsieh Fix For: 0.98.0, 0.96.2, 0.99.0, 0.94.17 Attachments: hbase-10401.10400.patch, hbase-10401.patch, hbase-10401.v2.94.patch, hbase-10401.v2.patch In a recent support case, we encountered a corrupt hbase that had thousands of overlap groups (regions that had overlapping key ranges). The current implementation repairs these by serially taking a group, perorming a merge and then moving on to the next group. Because assignments and hdfs nn operations are involved each merge could take on the order of seconds. With thousands of overlap groups, this could take hours to complete. This patch makes it so that these independent merge groups are merged in parallel. It uses the same thread pool for other fs info-gathering operations. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HBASE-10400) [hbck] Continue if region dir missing on region merge attempt
[ https://issues.apache.org/jira/browse/HBASE-10400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13881778#comment-13881778 ] Hudson commented on HBASE-10400: SUCCESS: Integrated in HBase-TRUNK-on-Hadoop-1.1 #63 (See [https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-1.1/63/]) HBASE-10400 [hbck] Continue if region dir missing on region merge attempt (jmhsieh: rev 1560577) * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java [hbck] Continue if region dir missing on region merge attempt - Key: HBASE-10400 URL: https://issues.apache.org/jira/browse/HBASE-10400 Project: HBase Issue Type: Bug Components: hbck Affects Versions: 0.92.2, 0.94.16, 0.96.1.1 Reporter: Jonathan Hsieh Assignee: Jonathan Hsieh Fix For: 0.98.0, 0.96.2, 0.99.0, 0.94.17 Attachments: hbase-10400.patch In a recent support case, the hbck repair tool would eventually hang because we didn't handle the case where merge info is old using hadoop2 fs.listStatus semantics (throw exn instead of return null). this is a trivial patch that handles the newer hadoop2 semantics. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HBASE-10412) Distributed log replay : Cell tags getting missed
[ https://issues.apache.org/jira/browse/HBASE-10412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13881779#comment-13881779 ] Hudson commented on HBASE-10412: SUCCESS: Integrated in HBase-TRUNK-on-Hadoop-1.1 #63 (See [https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-1.1/63/]) HBASE-10412 Distributed log replay: Cell tags getting missed (Anoop Sam John) (apurtell: rev 1561230) * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLogSplitter.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/WALCellCodec.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/security/visibility/TestVisibilityLabels.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/security/visibility/TestVisibilityLabelsWithDistributedLogReplay.java Distributed log replay : Cell tags getting missed - Key: HBASE-10412 URL: https://issues.apache.org/jira/browse/HBASE-10412 Project: HBase Issue Type: Bug Affects Versions: 0.98.0 Reporter: Anoop Sam John Assignee: Anoop Sam John Priority: Blocker Fix For: 0.98.0, 0.99.0 Attachments: HBASE-10412.patch, HBASE-10412_V2.patch This is caused by HBASE-10322. The default RPC codec KVCodec strips tags. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HBASE-10408) Intermittent TestDistributedLogSplitting#testLogReplayForDisablingTable failure
[
https://issues.apache.org/jira/browse/HBASE-10408?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13881782#comment-13881782
]
Hudson commented on HBASE-10408:
SUCCESS: Integrated in HBase-TRUNK-on-Hadoop-1.1 #63 (See
[https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-1.1/63/])
HBASE-10408 Intermittent
TestDistributedLogSplitting#testLogReplayForDisablingTable failure (tedyu: rev
1561119)
*
/hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestDistributedLogSplitting.java
Intermittent TestDistributedLogSplitting#testLogReplayForDisablingTable
failure
---
Key: HBASE-10408
URL: https://issues.apache.org/jira/browse/HBASE-10408
Project: HBase
Issue Type: Test
Reporter: Ted Yu
Assignee: Ted Yu
Fix For: 0.98.0, 0.99.0
Attachments: 10408-v1.txt, 10408-v2.txt, 10408-v3.txt,
TestDistributedLogSplitting.out,
testDistributedLogSplitting-disablingTable.html
Here was the assertion failure:
{code} java.lang.AssertionError: expected:lt;1000 but was:lt;0 at
org.junit.Assert.fail(Assert.java:88) at
org.junit.Assert.failNotEquals(Assert.java:743) at
org.junit.Assert.assertEquals(Assert.java:118) at
org.junit.Assert.assertEquals(Assert.java:555) at
org.junit.Assert.assertEquals(Assert.java:542) at
org.apache.hadoop.hbase.master.TestDistributedLogSplitting.testLogReplayForDisablingTable(TestDistributedLogSplitting.java:838)
{code}
This was due to the loop starting around line 823 not picking up any edits
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (HBASE-10409) Allow config options related to zookeeper to be passed to IntegrationTestsDriver
[
https://issues.apache.org/jira/browse/HBASE-10409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13881781#comment-13881781
]
Hudson commented on HBASE-10409:
SUCCESS: Integrated in HBase-TRUNK-on-Hadoop-1.1 #63 (See
[https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-1.1/63/])
HBASE-10409 Allow config options related to zookeeper to be passed to
IntegrationTestsDriver - revert (tedyu: rev 1560902)
*
/hbase/trunk/hbase-it/src/test/java/org/apache/hadoop/hbase/IntegrationTestsDriver.java
HBASE-10409 Allow config options related to zookeeper to be passed to
IntegrationTestsDriver (Ted Yu) (tedyu: rev 1560804)
*
/hbase/trunk/hbase-it/src/test/java/org/apache/hadoop/hbase/IntegrationTestsDriver.java
Allow config options related to zookeeper to be passed to
IntegrationTestsDriver
Key: HBASE-10409
URL: https://issues.apache.org/jira/browse/HBASE-10409
Project: HBase
Issue Type: Test
Reporter: Ted Yu
Assignee: Ted Yu
Attachments: 10409-v1.txt
Currently IntegrationTestsDriver only accepts one parameter: regex for
integration test.
{code}
public static void main(String[] args) throws Exception {
int ret = ToolRunner.run(new IntegrationTestsDriver(), args);
{code}
To allow IntegrationTestsDriver to be activated through Java API, support for
passing config options related to zookeeper should be added - similarly to
what LoadTestTool supports.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (HBASE-10365) HBaseFsck should clean up connection properly when repair is completed
[ https://issues.apache.org/jira/browse/HBASE-10365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13881783#comment-13881783 ] Hudson commented on HBASE-10365: SUCCESS: Integrated in HBase-TRUNK-on-Hadoop-1.1 #63 (See [https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-1.1/63/]) HBASE-10365 HBaseFsck should clean up connection properly when repair is completed (tedyu: rev 1560564) * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java HBaseFsck should clean up connection properly when repair is completed -- Key: HBASE-10365 URL: https://issues.apache.org/jira/browse/HBASE-10365 Project: HBase Issue Type: Bug Reporter: Ted Yu Assignee: Ted Yu Fix For: 0.98.0, 0.99.0 Attachments: 10365-v1.txt At the end of exec() method, connections to the cluster are not properly released. Connections should be released upon completion of repair. This was mentioned by Jean-Marc in the thread '[VOTE] The 1st hbase 0.94.16 release candidate is available for download' -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HBASE-10412) Distributed log replay : Cell tags getting missed
[ https://issues.apache.org/jira/browse/HBASE-10412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13881788#comment-13881788 ] Hudson commented on HBASE-10412: FAILURE: Integrated in HBase-TRUNK #4853 (See [https://builds.apache.org/job/HBase-TRUNK/4853/]) HBASE-10412 Distributed log replay: Cell tags getting missed (Anoop Sam John) (apurtell: rev 1561230) * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLogSplitter.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/WALCellCodec.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/security/visibility/TestVisibilityLabels.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/security/visibility/TestVisibilityLabelsWithDistributedLogReplay.java Distributed log replay : Cell tags getting missed - Key: HBASE-10412 URL: https://issues.apache.org/jira/browse/HBASE-10412 Project: HBase Issue Type: Bug Affects Versions: 0.98.0 Reporter: Anoop Sam John Assignee: Anoop Sam John Priority: Blocker Fix For: 0.98.0, 0.99.0 Attachments: HBASE-10412.patch, HBASE-10412_V2.patch This is caused by HBASE-10322. The default RPC codec KVCodec strips tags. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HBASE-10400) [hbck] Continue if region dir missing on region merge attempt
[ https://issues.apache.org/jira/browse/HBASE-10400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13881787#comment-13881787 ] Hudson commented on HBASE-10400: FAILURE: Integrated in HBase-TRUNK #4853 (See [https://builds.apache.org/job/HBase-TRUNK/4853/]) HBASE-10400 [hbck] Continue if region dir missing on region merge attempt (jmhsieh: rev 1560577) * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java [hbck] Continue if region dir missing on region merge attempt - Key: HBASE-10400 URL: https://issues.apache.org/jira/browse/HBASE-10400 Project: HBase Issue Type: Bug Components: hbck Affects Versions: 0.92.2, 0.94.16, 0.96.1.1 Reporter: Jonathan Hsieh Assignee: Jonathan Hsieh Fix For: 0.98.0, 0.96.2, 0.99.0, 0.94.17 Attachments: hbase-10400.patch In a recent support case, the hbck repair tool would eventually hang because we didn't handle the case where merge info is old using hadoop2 fs.listStatus semantics (throw exn instead of return null). this is a trivial patch that handles the newer hadoop2 semantics. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HBASE-10401) [hbck] perform overlap group merges in parallel
[ https://issues.apache.org/jira/browse/HBASE-10401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13881786#comment-13881786 ] Hudson commented on HBASE-10401: FAILURE: Integrated in HBase-TRUNK #4853 (See [https://builds.apache.org/job/HBase-TRUNK/4853/]) HBASE-10401 [hbck] perform overlap group merges in parallel (jmhsieh: rev 1560774) * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java [hbck] perform overlap group merges in parallel --- Key: HBASE-10401 URL: https://issues.apache.org/jira/browse/HBASE-10401 Project: HBase Issue Type: Bug Components: hbck Affects Versions: 0.92.2, 0.98.0, 0.94.16, 0.99.0, 0.96.1.1 Reporter: Jonathan Hsieh Assignee: Jonathan Hsieh Fix For: 0.98.0, 0.96.2, 0.99.0, 0.94.17 Attachments: hbase-10401.10400.patch, hbase-10401.patch, hbase-10401.v2.94.patch, hbase-10401.v2.patch In a recent support case, we encountered a corrupt hbase that had thousands of overlap groups (regions that had overlapping key ranges). The current implementation repairs these by serially taking a group, perorming a merge and then moving on to the next group. Because assignments and hdfs nn operations are involved each merge could take on the order of seconds. With thousands of overlap groups, this could take hours to complete. This patch makes it so that these independent merge groups are merged in parallel. It uses the same thread pool for other fs info-gathering operations. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HBASE-10409) Allow config options related to zookeeper to be passed to IntegrationTestsDriver
[
https://issues.apache.org/jira/browse/HBASE-10409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13881790#comment-13881790
]
Hudson commented on HBASE-10409:
FAILURE: Integrated in HBase-TRUNK #4853 (See
[https://builds.apache.org/job/HBase-TRUNK/4853/])
HBASE-10409 Allow config options related to zookeeper to be passed to
IntegrationTestsDriver - revert (tedyu: rev 1560902)
*
/hbase/trunk/hbase-it/src/test/java/org/apache/hadoop/hbase/IntegrationTestsDriver.java
HBASE-10409 Allow config options related to zookeeper to be passed to
IntegrationTestsDriver (Ted Yu) (tedyu: rev 1560804)
*
/hbase/trunk/hbase-it/src/test/java/org/apache/hadoop/hbase/IntegrationTestsDriver.java
Allow config options related to zookeeper to be passed to
IntegrationTestsDriver
Key: HBASE-10409
URL: https://issues.apache.org/jira/browse/HBASE-10409
Project: HBase
Issue Type: Test
Reporter: Ted Yu
Assignee: Ted Yu
Attachments: 10409-v1.txt
Currently IntegrationTestsDriver only accepts one parameter: regex for
integration test.
{code}
public static void main(String[] args) throws Exception {
int ret = ToolRunner.run(new IntegrationTestsDriver(), args);
{code}
To allow IntegrationTestsDriver to be activated through Java API, support for
passing config options related to zookeeper should be added - similarly to
what LoadTestTool supports.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (HBASE-10406) Column family option is not effective in IntegrationTestSendTraceRequests
[
https://issues.apache.org/jira/browse/HBASE-10406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13881789#comment-13881789
]
Hudson commented on HBASE-10406:
FAILURE: Integrated in HBase-TRUNK #4853 (See
[https://builds.apache.org/job/HBase-TRUNK/4853/])
HBASE-10406 Column family option is not effective in
IntegrationTestSendTraceRequests (tedyu: rev 1560880)
*
/hbase/trunk/hbase-it/src/test/java/org/apache/hadoop/hbase/trace/IntegrationTestSendTraceRequests.java
Column family option is not effective in IntegrationTestSendTraceRequests
-
Key: HBASE-10406
URL: https://issues.apache.org/jira/browse/HBASE-10406
Project: HBase
Issue Type: Test
Reporter: Ted Yu
Assignee: Ted Yu
Fix For: 0.98.0, 0.99.0
Attachments: 10406-v1.txt
If column family option (-f) is specified, the following error would occur:
{code}
2014-01-23 23:21:20,347 [Thread-2] ERROR util.AbstractHBaseTool
(AbstractHBaseTool.java:run(97)) - Error when parsing command-line arguemnts
org.apache.commons.cli.UnrecognizedOptionException: Unrecognized option: -f
at org.apache.commons.cli.Parser.processOption(Parser.java:363)
at org.apache.commons.cli.Parser.parse(Parser.java:199)
at org.apache.commons.cli.Parser.parse(Parser.java:85)
at
org.apache.hadoop.hbase.util.AbstractHBaseTool.parseArgs(AbstractHBaseTool.java:135)
at
org.apache.hadoop.hbase.util.AbstractHBaseTool.run(AbstractHBaseTool.java:94)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
{code}
This is due to table option is used twice in the following code:
{code}
protected void addOptions() {
addOptWithArg(TABLE_ARG, The table name to target. Will be created if
not there already.);
addOptWithArg(TABLE_ARG, The family to target);
{code}
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (HBASE-10408) Intermittent TestDistributedLogSplitting#testLogReplayForDisablingTable failure
[
https://issues.apache.org/jira/browse/HBASE-10408?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13881791#comment-13881791
]
Hudson commented on HBASE-10408:
FAILURE: Integrated in HBase-TRUNK #4853 (See
[https://builds.apache.org/job/HBase-TRUNK/4853/])
HBASE-10408 Intermittent
TestDistributedLogSplitting#testLogReplayForDisablingTable failure (tedyu: rev
1561119)
*
/hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestDistributedLogSplitting.java
Intermittent TestDistributedLogSplitting#testLogReplayForDisablingTable
failure
---
Key: HBASE-10408
URL: https://issues.apache.org/jira/browse/HBASE-10408
Project: HBase
Issue Type: Test
Reporter: Ted Yu
Assignee: Ted Yu
Fix For: 0.98.0, 0.99.0
Attachments: 10408-v1.txt, 10408-v2.txt, 10408-v3.txt,
TestDistributedLogSplitting.out,
testDistributedLogSplitting-disablingTable.html
Here was the assertion failure:
{code} java.lang.AssertionError: expected:lt;1000 but was:lt;0 at
org.junit.Assert.fail(Assert.java:88) at
org.junit.Assert.failNotEquals(Assert.java:743) at
org.junit.Assert.assertEquals(Assert.java:118) at
org.junit.Assert.assertEquals(Assert.java:555) at
org.junit.Assert.assertEquals(Assert.java:542) at
org.apache.hadoop.hbase.master.TestDistributedLogSplitting.testLogReplayForDisablingTable(TestDistributedLogSplitting.java:838)
{code}
This was due to the loop starting around line 823 not picking up any edits
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (HBASE-10365) HBaseFsck should clean up connection properly when repair is completed
[ https://issues.apache.org/jira/browse/HBASE-10365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13881792#comment-13881792 ] Hudson commented on HBASE-10365: FAILURE: Integrated in HBase-TRUNK #4853 (See [https://builds.apache.org/job/HBase-TRUNK/4853/]) HBASE-10365 HBaseFsck should clean up connection properly when repair is completed (tedyu: rev 1560564) * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java HBaseFsck should clean up connection properly when repair is completed -- Key: HBASE-10365 URL: https://issues.apache.org/jira/browse/HBASE-10365 Project: HBase Issue Type: Bug Reporter: Ted Yu Assignee: Ted Yu Fix For: 0.98.0, 0.99.0 Attachments: 10365-v1.txt At the end of exec() method, connections to the cluster are not properly released. Connections should be released upon completion of repair. This was mentioned by Jean-Marc in the thread '[VOTE] The 1st hbase 0.94.16 release candidate is available for download' -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HBASE-10383) Secure Bulk Load for 'completebulkload' fails for version 0.94.15
[ https://issues.apache.org/jira/browse/HBASE-10383?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13881797#comment-13881797 ] Hudson commented on HBASE-10383: FAILURE: Integrated in HBase-0.94 #1265 (See [https://builds.apache.org/job/HBase-0.94/1265/]) HBASE-10383 Secure Bulk Load for 'completebulkload' fails for version 0.94.15 (jyates: rev 1561243) * /hbase/branches/0.94/security/src/main/java/org/apache/hadoop/hbase/security/access/SecureBulkLoadEndpoint.java * /hbase/branches/0.94/security/src/main/java/org/apache/hadoop/hbase/security/access/SecureBulkLoadProtocol.java * /hbase/branches/0.94/security/src/test/java/org/apache/hadoop/hbase/mapreduce/TestSecureLoadIncrementalHFiles.java * /hbase/branches/0.94/security/src/test/java/org/apache/hadoop/hbase/mapreduce/TestSecureLoadIncrementalHFilesSplitRecovery.java * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/coprocessor/SecureBulkLoadClient.java * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/mapreduce/LoadIncrementalHFiles.java * /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/HadoopSecurityEnabledUserProviderForTesting.java * /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestLoadIncrementalHFiles.java * /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/mapreduce/TestLoadIncrementalHFilesSplitRecovery.java Secure Bulk Load for 'completebulkload' fails for version 0.94.15 - Key: HBASE-10383 URL: https://issues.apache.org/jira/browse/HBASE-10383 Project: HBase Issue Type: Bug Components: Coprocessors Affects Versions: 0.94.15 Reporter: Kashif J S Assignee: Jesse Yates Priority: Critical Fix For: 0.94.17 Attachments: 10383.txt, HBASE-10383-v2.patch, hbase-10383-jyates-0.94-v0.patch Secure Bulk Load with kerberos enabled fails for Complete Bulk LoadLoadIncrementalHfile with following exception ERROR org.apache.hadoop.hbase.regionserver.HRegionServer: org.apache.hadoop.hbase.ipc.HBaseRPC$UnknownProtocolException: No matching handler for protocol org.apache.hadoop.hbase.security.access.SecureBulkLoadProtocol in region t1,,1389699438035.28bb0284d971d0676cf562efea80199b. at org.apache.hadoop.hbase.regionserver.HRegion.exec(HRegion.java) at org.apache.hadoop.hbase.regionserver.HRegionServer.execCoprocessor(HRegionServer.java) at sun.reflect.GeneratedMethodAccessor18.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java) at java.lang.reflect.Method.invoke(Method.java) at org.apache.hadoop.hbase.ipc.SecureRpcEngine$Server.call(SecureRpcEngine.java) at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java) -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HBASE-10400) [hbck] Continue if region dir missing on region merge attempt
[ https://issues.apache.org/jira/browse/HBASE-10400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13881796#comment-13881796 ] Hudson commented on HBASE-10400: FAILURE: Integrated in HBase-0.94 #1265 (See [https://builds.apache.org/job/HBase-0.94/1265/]) HBASE-10400 [hbck] Continue if region dir missing on region merge attempt (jmhsieh: rev 1560579) * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java [hbck] Continue if region dir missing on region merge attempt - Key: HBASE-10400 URL: https://issues.apache.org/jira/browse/HBASE-10400 Project: HBase Issue Type: Bug Components: hbck Affects Versions: 0.92.2, 0.94.16, 0.96.1.1 Reporter: Jonathan Hsieh Assignee: Jonathan Hsieh Fix For: 0.98.0, 0.96.2, 0.99.0, 0.94.17 Attachments: hbase-10400.patch In a recent support case, the hbck repair tool would eventually hang because we didn't handle the case where merge info is old using hadoop2 fs.listStatus semantics (throw exn instead of return null). this is a trivial patch that handles the newer hadoop2 semantics. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HBASE-10401) [hbck] perform overlap group merges in parallel
[ https://issues.apache.org/jira/browse/HBASE-10401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13881795#comment-13881795 ] Hudson commented on HBASE-10401: FAILURE: Integrated in HBase-0.94 #1265 (See [https://builds.apache.org/job/HBase-0.94/1265/]) HBASE-10401 [hbck] perform overlap group merges in parallel (jmhsieh: rev 1560776) * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java [hbck] perform overlap group merges in parallel --- Key: HBASE-10401 URL: https://issues.apache.org/jira/browse/HBASE-10401 Project: HBase Issue Type: Bug Components: hbck Affects Versions: 0.92.2, 0.98.0, 0.94.16, 0.99.0, 0.96.1.1 Reporter: Jonathan Hsieh Assignee: Jonathan Hsieh Fix For: 0.98.0, 0.96.2, 0.99.0, 0.94.17 Attachments: hbase-10401.10400.patch, hbase-10401.patch, hbase-10401.v2.94.patch, hbase-10401.v2.patch In a recent support case, we encountered a corrupt hbase that had thousands of overlap groups (regions that had overlapping key ranges). The current implementation repairs these by serially taking a group, perorming a merge and then moving on to the next group. Because assignments and hdfs nn operations are involved each merge could take on the order of seconds. With thousands of overlap groups, this could take hours to complete. This patch makes it so that these independent merge groups are merged in parallel. It uses the same thread pool for other fs info-gathering operations. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HBASE-10416) Improvements to the import flow
[
https://issues.apache.org/jira/browse/HBASE-10416?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13881799#comment-13881799
]
Hadoop QA commented on HBASE-10416:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12625121/HBASE-10416.patch
against trunk revision .
ATTACHMENT ID: 12625121
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified tests.
{color:green}+1 hadoop1.0{color}. The patch compiles against the hadoop
1.0 profile.
{color:green}+1 hadoop1.1{color}. The patch compiles against the hadoop
1.1 profile.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 3
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:red}-1 site{color}. The patch appears to cause mvn site goal to
fail.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/8504//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8504//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8504//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8504//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8504//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8504//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8504//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8504//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8504//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8504//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/8504//console
This message is automatically generated.
Improvements to the import flow
---
Key: HBASE-10416
URL: https://issues.apache.org/jira/browse/HBASE-10416
Project: HBase
Issue Type: New Feature
Components: mapreduce
Reporter: Vasu Mariyala
Attachments: HBASE-10416.patch
Following improvements can be made to the Import logic
a) Make the import extensible (i.e., remove the filter from being a static
member of Import and make it an instance variable of the mapper, make the
mappers or variables of interest protected. )
b) Make sure that the Import calls filterRowKey method of the filter (Useful
if we want to filter the data of an organization based on the row key or
using filters like PrefixFilter which filter the data in filterRowKey method
rather than the filterKeyValue method). The existing test case in
TestImportExport#testWithFilter works with this assumption but is so far
successful because there is only one row inserted into the table.
c) Provide an option to specify the durability during the import (Specifying
the Durability as SKIP_WAL would improve the performance of restore
considerably.) [~lhofhansl] suggested that this should be a parameter to the
import.
d) Some minor refactoring to avoid building a comma separated string for the
filter args.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (HBASE-10081) Since HBASE-7091, HBASE_OPTS cannot be set on the command line
[
https://issues.apache.org/jira/browse/HBASE-10081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13881801#comment-13881801
]
Hadoop QA commented on HBASE-10081:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12624760/HBASE-10081-v1.diff
against trunk revision .
ATTACHMENT ID: 12624760
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 hadoop1.0{color}. The patch compiles against the hadoop
1.0 profile.
{color:green}+1 hadoop1.1{color}. The patch compiles against the hadoop
1.1 profile.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:red}-1 site{color}. The patch appears to cause mvn site goal to
fail.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/8505//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8505//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8505//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8505//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8505//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8505//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8505//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8505//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8505//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8505//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/8505//console
This message is automatically generated.
Since HBASE-7091, HBASE_OPTS cannot be set on the command line
--
Key: HBASE-10081
URL: https://issues.apache.org/jira/browse/HBASE-10081
Project: HBase
Issue Type: Bug
Components: scripts
Affects Versions: 0.98.0, 0.96.0
Reporter: Nicolas Liochon
Assignee: Liu Shaohui
Priority: Minor
Attachments: HBASE-10081-v1.diff
Discussed in HBASE-7091.
It's not critical, but a little bit surprising, as the comments in bin/hbase
doesn't say anything about this. If you create your own hbase-env then it's
not an issue...
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (HBASE-10419) Add multiget support to PerformanceEvaluation
[
https://issues.apache.org/jira/browse/HBASE-10419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13881820#comment-13881820
]
Hadoop QA commented on HBASE-10419:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12625170/HBASE-10419.0.patch
against trunk revision .
ATTACHMENT ID: 12625170
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified tests.
{color:green}+1 hadoop1.0{color}. The patch compiles against the hadoop
1.0 profile.
{color:green}+1 hadoop1.1{color}. The patch compiles against the hadoop
1.1 profile.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:red}-1 site{color}. The patch appears to cause mvn site goal to
fail.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/8506//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8506//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8506//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8506//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8506//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8506//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8506//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8506//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8506//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8506//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/8506//console
This message is automatically generated.
Add multiget support to PerformanceEvaluation
-
Key: HBASE-10419
URL: https://issues.apache.org/jira/browse/HBASE-10419
Project: HBase
Issue Type: Improvement
Components: test
Reporter: Nick Dimiduk
Assignee: Nick Dimiduk
Priority: Minor
Attachments: HBASE-10419.0.patch
Folks planning to use multiget may find this useful.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (HBASE-10400) [hbck] Continue if region dir missing on region merge attempt
[ https://issues.apache.org/jira/browse/HBASE-10400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13881852#comment-13881852 ] Hudson commented on HBASE-10400: FAILURE: Integrated in hbase-0.96-hadoop2 #185 (See [https://builds.apache.org/job/hbase-0.96-hadoop2/185/]) HBASE-10400 [hbck] Continue if region dir missing on region merge attempt (jmhsieh: rev 1560580) * /hbase/branches/0.96/hbase-server/src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java [hbck] Continue if region dir missing on region merge attempt - Key: HBASE-10400 URL: https://issues.apache.org/jira/browse/HBASE-10400 Project: HBase Issue Type: Bug Components: hbck Affects Versions: 0.92.2, 0.94.16, 0.96.1.1 Reporter: Jonathan Hsieh Assignee: Jonathan Hsieh Fix For: 0.98.0, 0.96.2, 0.99.0, 0.94.17 Attachments: hbase-10400.patch In a recent support case, the hbck repair tool would eventually hang because we didn't handle the case where merge info is old using hadoop2 fs.listStatus semantics (throw exn instead of return null). this is a trivial patch that handles the newer hadoop2 semantics. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HBASE-10401) [hbck] perform overlap group merges in parallel
[ https://issues.apache.org/jira/browse/HBASE-10401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13881851#comment-13881851 ] Hudson commented on HBASE-10401: FAILURE: Integrated in hbase-0.96-hadoop2 #185 (See [https://builds.apache.org/job/hbase-0.96-hadoop2/185/]) HBASE-10401 [hbck] perform overlap group merges in parallel (jmhsieh: rev 1560775) * /hbase/branches/0.96/hbase-server/src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java [hbck] perform overlap group merges in parallel --- Key: HBASE-10401 URL: https://issues.apache.org/jira/browse/HBASE-10401 Project: HBase Issue Type: Bug Components: hbck Affects Versions: 0.92.2, 0.98.0, 0.94.16, 0.99.0, 0.96.1.1 Reporter: Jonathan Hsieh Assignee: Jonathan Hsieh Fix For: 0.98.0, 0.96.2, 0.99.0, 0.94.17 Attachments: hbase-10401.10400.patch, hbase-10401.patch, hbase-10401.v2.94.patch, hbase-10401.v2.patch In a recent support case, we encountered a corrupt hbase that had thousands of overlap groups (regions that had overlapping key ranges). The current implementation repairs these by serially taking a group, perorming a merge and then moving on to the next group. Because assignments and hdfs nn operations are involved each merge could take on the order of seconds. With thousands of overlap groups, this could take hours to complete. This patch makes it so that these independent merge groups are merged in parallel. It uses the same thread pool for other fs info-gathering operations. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HBASE-10400) [hbck] Continue if region dir missing on region merge attempt
[ https://issues.apache.org/jira/browse/HBASE-10400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13881893#comment-13881893 ] Hudson commented on HBASE-10400: SUCCESS: Integrated in HBase-0.98 #107 (See [https://builds.apache.org/job/HBase-0.98/107/]) HBASE-10400 [hbck] Continue if region dir missing on region merge attempt (jmhsieh: rev 1560578) * /hbase/branches/0.98/hbase-server/src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java [hbck] Continue if region dir missing on region merge attempt - Key: HBASE-10400 URL: https://issues.apache.org/jira/browse/HBASE-10400 Project: HBase Issue Type: Bug Components: hbck Affects Versions: 0.92.2, 0.94.16, 0.96.1.1 Reporter: Jonathan Hsieh Assignee: Jonathan Hsieh Fix For: 0.98.0, 0.96.2, 0.99.0, 0.94.17 Attachments: hbase-10400.patch In a recent support case, the hbck repair tool would eventually hang because we didn't handle the case where merge info is old using hadoop2 fs.listStatus semantics (throw exn instead of return null). this is a trivial patch that handles the newer hadoop2 semantics. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HBASE-10406) Column family option is not effective in IntegrationTestSendTraceRequests
[
https://issues.apache.org/jira/browse/HBASE-10406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13881895#comment-13881895
]
Hudson commented on HBASE-10406:
SUCCESS: Integrated in HBase-0.98 #107 (See
[https://builds.apache.org/job/HBase-0.98/107/])
HBASE-10406 Column family option is not effective in
IntegrationTestSendTraceRequests (tedyu: rev 1560879)
*
/hbase/branches/0.98/hbase-it/src/test/java/org/apache/hadoop/hbase/trace/IntegrationTestSendTraceRequests.java
Column family option is not effective in IntegrationTestSendTraceRequests
-
Key: HBASE-10406
URL: https://issues.apache.org/jira/browse/HBASE-10406
Project: HBase
Issue Type: Test
Reporter: Ted Yu
Assignee: Ted Yu
Fix For: 0.98.0, 0.99.0
Attachments: 10406-v1.txt
If column family option (-f) is specified, the following error would occur:
{code}
2014-01-23 23:21:20,347 [Thread-2] ERROR util.AbstractHBaseTool
(AbstractHBaseTool.java:run(97)) - Error when parsing command-line arguemnts
org.apache.commons.cli.UnrecognizedOptionException: Unrecognized option: -f
at org.apache.commons.cli.Parser.processOption(Parser.java:363)
at org.apache.commons.cli.Parser.parse(Parser.java:199)
at org.apache.commons.cli.Parser.parse(Parser.java:85)
at
org.apache.hadoop.hbase.util.AbstractHBaseTool.parseArgs(AbstractHBaseTool.java:135)
at
org.apache.hadoop.hbase.util.AbstractHBaseTool.run(AbstractHBaseTool.java:94)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
{code}
This is due to table option is used twice in the following code:
{code}
protected void addOptions() {
addOptWithArg(TABLE_ARG, The table name to target. Will be created if
not there already.);
addOptWithArg(TABLE_ARG, The family to target);
{code}
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (HBASE-10365) HBaseFsck should clean up connection properly when repair is completed
[ https://issues.apache.org/jira/browse/HBASE-10365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13881898#comment-13881898 ] Hudson commented on HBASE-10365: SUCCESS: Integrated in HBase-0.98 #107 (See [https://builds.apache.org/job/HBase-0.98/107/]) HBASE-10365 HBaseFsck should clean up connection properly when repair is completed (Ted Yu) (tedyu: rev 1560581) * /hbase/branches/0.98/hbase-server/src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java HBaseFsck should clean up connection properly when repair is completed -- Key: HBASE-10365 URL: https://issues.apache.org/jira/browse/HBASE-10365 Project: HBase Issue Type: Bug Reporter: Ted Yu Assignee: Ted Yu Fix For: 0.98.0, 0.99.0 Attachments: 10365-v1.txt At the end of exec() method, connections to the cluster are not properly released. Connections should be released upon completion of repair. This was mentioned by Jean-Marc in the thread '[VOTE] The 1st hbase 0.94.16 release candidate is available for download' -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HBASE-10409) Allow config options related to zookeeper to be passed to IntegrationTestsDriver
[
https://issues.apache.org/jira/browse/HBASE-10409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13881896#comment-13881896
]
Hudson commented on HBASE-10409:
SUCCESS: Integrated in HBase-0.98 #107 (See
[https://builds.apache.org/job/HBase-0.98/107/])
HBASE-10409 Allow config options related to zookeeper to be passed to
IntegrationTestsDriver - revert (tedyu: rev 1560901)
*
/hbase/branches/0.98/hbase-it/src/test/java/org/apache/hadoop/hbase/IntegrationTestsDriver.java
HBASE-10409 Allow config options related to zookeeper to be passed to
IntegrationTestsDriver (Ted Yu) (tedyu: rev 1560803)
*
/hbase/branches/0.98/hbase-it/src/test/java/org/apache/hadoop/hbase/IntegrationTestsDriver.java
Allow config options related to zookeeper to be passed to
IntegrationTestsDriver
Key: HBASE-10409
URL: https://issues.apache.org/jira/browse/HBASE-10409
Project: HBase
Issue Type: Test
Reporter: Ted Yu
Assignee: Ted Yu
Attachments: 10409-v1.txt
Currently IntegrationTestsDriver only accepts one parameter: regex for
integration test.
{code}
public static void main(String[] args) throws Exception {
int ret = ToolRunner.run(new IntegrationTestsDriver(), args);
{code}
To allow IntegrationTestsDriver to be activated through Java API, support for
passing config options related to zookeeper should be added - similarly to
what LoadTestTool supports.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (HBASE-10401) [hbck] perform overlap group merges in parallel
[ https://issues.apache.org/jira/browse/HBASE-10401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13881892#comment-13881892 ] Hudson commented on HBASE-10401: SUCCESS: Integrated in HBase-0.98 #107 (See [https://builds.apache.org/job/HBase-0.98/107/]) HBASE-10401 [hbck] perform overlap group merges in parallel (jmhsieh: rev 1560779) * /hbase/branches/0.98/hbase-server/src/main/java/org/apache/hadoop/hbase/util/HBaseFsck.java [hbck] perform overlap group merges in parallel --- Key: HBASE-10401 URL: https://issues.apache.org/jira/browse/HBASE-10401 Project: HBase Issue Type: Bug Components: hbck Affects Versions: 0.92.2, 0.98.0, 0.94.16, 0.99.0, 0.96.1.1 Reporter: Jonathan Hsieh Assignee: Jonathan Hsieh Fix For: 0.98.0, 0.96.2, 0.99.0, 0.94.17 Attachments: hbase-10401.10400.patch, hbase-10401.patch, hbase-10401.v2.94.patch, hbase-10401.v2.patch In a recent support case, we encountered a corrupt hbase that had thousands of overlap groups (regions that had overlapping key ranges). The current implementation repairs these by serially taking a group, perorming a merge and then moving on to the next group. Because assignments and hdfs nn operations are involved each merge could take on the order of seconds. With thousands of overlap groups, this could take hours to complete. This patch makes it so that these independent merge groups are merged in parallel. It uses the same thread pool for other fs info-gathering operations. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HBASE-10408) Intermittent TestDistributedLogSplitting#testLogReplayForDisablingTable failure
[
https://issues.apache.org/jira/browse/HBASE-10408?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13881897#comment-13881897
]
Hudson commented on HBASE-10408:
SUCCESS: Integrated in HBase-0.98 #107 (See
[https://builds.apache.org/job/HBase-0.98/107/])
HBASE-10408 Intermittent
TestDistributedLogSplitting#testLogReplayForDisablingTable failure (tedyu: rev
1561118)
*
/hbase/branches/0.98/hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestDistributedLogSplitting.java
Intermittent TestDistributedLogSplitting#testLogReplayForDisablingTable
failure
---
Key: HBASE-10408
URL: https://issues.apache.org/jira/browse/HBASE-10408
Project: HBase
Issue Type: Test
Reporter: Ted Yu
Assignee: Ted Yu
Fix For: 0.98.0, 0.99.0
Attachments: 10408-v1.txt, 10408-v2.txt, 10408-v3.txt,
TestDistributedLogSplitting.out,
testDistributedLogSplitting-disablingTable.html
Here was the assertion failure:
{code} java.lang.AssertionError: expected:lt;1000 but was:lt;0 at
org.junit.Assert.fail(Assert.java:88) at
org.junit.Assert.failNotEquals(Assert.java:743) at
org.junit.Assert.assertEquals(Assert.java:118) at
org.junit.Assert.assertEquals(Assert.java:555) at
org.junit.Assert.assertEquals(Assert.java:542) at
org.apache.hadoop.hbase.master.TestDistributedLogSplitting.testLogReplayForDisablingTable(TestDistributedLogSplitting.java:838)
{code}
This was due to the loop starting around line 823 not picking up any edits
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (HBASE-10412) Distributed log replay : Cell tags getting missed
[ https://issues.apache.org/jira/browse/HBASE-10412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13881894#comment-13881894 ] Hudson commented on HBASE-10412: SUCCESS: Integrated in HBase-0.98 #107 (See [https://builds.apache.org/job/HBase-0.98/107/]) HBASE-10412 Distributed log replay: Cell tags getting missed (Anoop Sam John) (apurtell: rev 1561231) * /hbase/branches/0.98/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLogSplitter.java * /hbase/branches/0.98/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/WALCellCodec.java * /hbase/branches/0.98/hbase-server/src/test/java/org/apache/hadoop/hbase/security/visibility/TestVisibilityLabels.java * /hbase/branches/0.98/hbase-server/src/test/java/org/apache/hadoop/hbase/security/visibility/TestVisibilityLabelsWithDistributedLogReplay.java Distributed log replay : Cell tags getting missed - Key: HBASE-10412 URL: https://issues.apache.org/jira/browse/HBASE-10412 Project: HBase Issue Type: Bug Affects Versions: 0.98.0 Reporter: Anoop Sam John Assignee: Anoop Sam John Priority: Blocker Fix For: 0.98.0, 0.99.0 Attachments: HBASE-10412.patch, HBASE-10412_V2.patch This is caused by HBASE-10322. The default RPC codec KVCodec strips tags. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HBASE-10378) Divide HLog interface into User and Implementor specific interfaces
[
https://issues.apache.org/jira/browse/HBASE-10378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13881949#comment-13881949
]
Hadoop QA commented on HBASE-10378:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12624933/10378-2.patch
against trunk revision .
ATTACHMENT ID: 12624933
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 128
new or modified tests.
{color:green}+1 hadoop1.0{color}. The patch compiles against the hadoop
1.0 profile.
{color:green}+1 hadoop1.1{color}. The patch compiles against the hadoop
1.1 profile.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:red}-1 site{color}. The patch appears to cause mvn site goal to
fail.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/8507//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8507//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8507//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8507//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8507//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8507//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8507//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8507//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8507//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8507//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/8507//console
This message is automatically generated.
Divide HLog interface into User and Implementor specific interfaces
---
Key: HBASE-10378
URL: https://issues.apache.org/jira/browse/HBASE-10378
Project: HBase
Issue Type: Sub-task
Components: wal
Reporter: Himanshu Vashishtha
Attachments: 10378-1.patch, 10378-2.patch
HBASE-5937 introduces the HLog interface as a first step to support multiple
WAL implementations. This interface is a good start, but has some
limitations/drawbacks in its current state, such as:
1) There is no clear distinction b/w User and Implementor APIs, and it
provides APIs both for WAL users (append, sync, etc) and also WAL
implementors (Reader/Writer interfaces, etc). There are APIs which are very
much implementation specific (getFileNum, etc) and a user such as a
RegionServer shouldn't know about it.
2) There are about 14 methods in FSHLog which are not present in HLog
interface but are used at several places in the unit test code. These tests
typecast HLog to FSHLog, which makes it very difficult to test multiple WAL
implementations without doing some ugly checks.
I'd like to propose some changes in HLog interface that would ease the multi
WAL story:
1) Have two interfaces WAL and WALService. WAL provides APIs for
implementors. WALService provides APIs for users (such as RegionServer).
2) A skeleton implementation of the above two interface as the base class for
other WAL implementations (AbstractWAL). It provides required fields for all
subclasses (fs, conf, log dir, etc). Make a minimal set of test only methods
and add this set in AbstractWAL.
3) HLogFactory returns a WALService reference when creating a WAL instance;
if a user need to access impl specific APIs (there are unit tests which get
WAL
[jira] [Commented] (HBASE-10336) Remove deprecated usage of Hadoop HttpServer in InfoServer
[
https://issues.apache.org/jira/browse/HBASE-10336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13881952#comment-13881952
]
Hadoop QA commented on HBASE-10336:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12624767/HBASE-10336-7.patch
against trunk revision .
ATTACHMENT ID: 12624767
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 48 new
or modified tests.
{color:red}-1 hadoop1.0{color}. The patch failed to compile against the
hadoop 1.0 profile.
Here is snippet of errors:
{code}[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-compiler-plugin:2.5.1:compile (default-compile)
on project hbase-server: Compilation failure: Compilation failure:
[ERROR]
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/http/HttpServer.java:[50,24]
cannot find symbol
[ERROR] symbol : class HadoopIllegalArgumentException
[ERROR] location: package org.apache.hadoop
[ERROR]
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/http/lib/StaticUserWebFilter.java:[41,0]
cannot find symbol
[ERROR] symbol : static DEFAULT_HADOOP_HTTP_STATIC_USER
--
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal
org.apache.maven.plugins:maven-compiler-plugin:2.5.1:compile (default-compile)
on project hbase-server: Compilation failure
at
org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:213)
at
org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)
at
org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)
at
org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:84)
at
org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:59)
--
Caused by: org.apache.maven.plugin.CompilationFailureException: Compilation
failure
at
org.apache.maven.plugin.AbstractCompilerMojo.execute(AbstractCompilerMojo.java:729)
at org.apache.maven.plugin.CompilerMojo.execute(CompilerMojo.java:128)
at
org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:101)
at
org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:209)
... 19 more{code}
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/8509//console
This message is automatically generated.
Remove deprecated usage of Hadoop HttpServer in InfoServer
--
Key: HBASE-10336
URL: https://issues.apache.org/jira/browse/HBASE-10336
Project: HBase
Issue Type: Bug
Affects Versions: 0.99.0
Reporter: Eric Charles
Assignee: Eric Charles
Attachments: HBASE-10336-1.patch, HBASE-10336-2.patch,
HBASE-10336-3.patch, HBASE-10336-4.patch, HBASE-10336-5.patch,
HBASE-10336-6.patch, HBASE-10336-7.patch
Recent changes in Hadoop HttpServer give NPE when running on hadoop
3.0.0-SNAPSHOT. This way we use HttpServer is deprecated and will probably be
not fixed (see HDFS-5760). We'd better move to the new proposed builder
pattern, which means we can no more use inheritance to build our nice
InfoServer.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (HBASE-6581) Build with hadoop.profile=3.0
[
https://issues.apache.org/jira/browse/HBASE-6581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13881958#comment-13881958
]
Hadoop QA commented on HBASE-6581:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12624766/HBASE-6581-8-pom.patch
against trunk revision .
ATTACHMENT ID: 12624766
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 hadoop1.0{color}. The patch compiles against the hadoop
1.0 profile.
{color:green}+1 hadoop1.1{color}. The patch compiles against the hadoop
1.1 profile.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 lineLengths{color}. The patch introduces the following lines
longer than 100:
+assembly
xmlns=http://maven.apache.org/plugins/maven-assembly-plugin/assembly/1.1.1;
xmlns:xsi=http://www.w3.org/2001/XMLSchema-instance;
xsi:schemaLocation=http://maven.apache.org/plugins/maven-assembly-plugin/assembly/1.1.1
http://maven.apache.org/xsd/assembly-1.1.1.xsd;
+ !--This 'all' id is not appended to the produced bundle because we do this:
http://maven.apache.org/plugins/maven-assembly-plugin/faq.html#required-classifiers
--
+!--This plugin's configuration is used to store Eclipse m2e settings
only. It has no influence on the Maven build itself.--
+project xmlns=http://maven.apache.org/POM/4.0.0;
xmlns:xsi=http://www.w3.org/2001/XMLSchema-instance;
xsi:schemaLocation=http://maven.apache.org/POM/4.0.0
http://maven.apache.org/xsd/maven-4.0.0.xsd;
+!--This plugin's configuration is used to store Eclipse m2e settings
only. It has no influence on the Maven build itself.--
+!--This plugin's configuration is used to store Eclipse m2e settings
only. It has no influence on the Maven build itself.--
+!--This plugin's configuration is used to store Eclipse m2e settings
only. It has no influence on the Maven build itself.--
{color:red}-1 site{color}. The patch appears to cause mvn site goal to
fail.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/8508//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8508//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8508//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8508//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8508//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8508//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8508//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8508//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8508//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8508//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/8508//console
This message is automatically generated.
Build with hadoop.profile=3.0
-
Key: HBASE-6581
URL: https://issues.apache.org/jira/browse/HBASE-6581
Project: HBase
Issue Type: Bug
Reporter: Eric Charles
Assignee: Eric Charles
Fix For: 0.98.1
Attachments: HBASE-6581-1.patch, HBASE-6581-2.patch,
HBASE-6581-20130821.patch, HBASE-6581-3.patch, HBASE-6581-4.patch,
HBASE-6581-5.patch, HBASE-6581-6.patch,
[jira] [Commented] (HBASE-6581) Build with hadoop.profile=3.0
[ https://issues.apache.org/jira/browse/HBASE-6581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13881966#comment-13881966 ] Eric Charles commented on HBASE-6581: - didn't 'mvn site' - will check and submit new patch if needed - otherwise, any other feedback on the pom changes? Build with hadoop.profile=3.0 - Key: HBASE-6581 URL: https://issues.apache.org/jira/browse/HBASE-6581 Project: HBase Issue Type: Bug Reporter: Eric Charles Assignee: Eric Charles Fix For: 0.98.1 Attachments: HBASE-6581-1.patch, HBASE-6581-2.patch, HBASE-6581-20130821.patch, HBASE-6581-3.patch, HBASE-6581-4.patch, HBASE-6581-5.patch, HBASE-6581-6.patch, HBASE-6581-7.patch, HBASE-6581-8-pom.patch, HBASE-6581.diff, HBASE-6581.diff Building trunk with hadoop.profile=3.0 gives exceptions (see [1]) due to change in the hadoop maven modules naming (and also usage of 3.0-SNAPSHOT instead of 3.0.0-SNAPSHOT in hbase-common). I can provide a patch that would move most of hadoop dependencies in their respective profiles and will define the correct hadoop deps in the 3.0 profile. Please tell me if that's ok to go this way. Thx, Eric [1] $ mvn clean install -Dhadoop.profile=3.0 [INFO] Scanning for projects... [ERROR] The build could not read 3 projects - [Help 1] [ERROR] [ERROR] The project org.apache.hbase:hbase-server:0.95-SNAPSHOT (/d/hbase.svn/hbase-server/pom.xml) has 3 errors [ERROR] 'dependencies.dependency.version' for org.apache.hadoop:hadoop-common:jar is missing. @ line 655, column 21 [ERROR] 'dependencies.dependency.version' for org.apache.hadoop:hadoop-annotations:jar is missing. @ line 659, column 21 [ERROR] 'dependencies.dependency.version' for org.apache.hadoop:hadoop-minicluster:jar is missing. @ line 663, column 21 [ERROR] [ERROR] The project org.apache.hbase:hbase-common:0.95-SNAPSHOT (/d/hbase.svn/hbase-common/pom.xml) has 3 errors [ERROR] 'dependencies.dependency.version' for org.apache.hadoop:hadoop-common:jar is missing. @ line 170, column 21 [ERROR] 'dependencies.dependency.version' for org.apache.hadoop:hadoop-annotations:jar is missing. @ line 174, column 21 [ERROR] 'dependencies.dependency.version' for org.apache.hadoop:hadoop-minicluster:jar is missing. @ line 178, column 21 [ERROR] [ERROR] The project org.apache.hbase:hbase-it:0.95-SNAPSHOT (/d/hbase.svn/hbase-it/pom.xml) has 3 errors [ERROR] 'dependencies.dependency.version' for org.apache.hadoop:hadoop-common:jar is missing. @ line 220, column 18 [ERROR] 'dependencies.dependency.version' for org.apache.hadoop:hadoop-annotations:jar is missing. @ line 224, column 21 [ERROR] 'dependencies.dependency.version' for org.apache.hadoop:hadoop-minicluster:jar is missing. @ line 228, column 21 [ERROR] -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (HBASE-10156) Fix up the HBASE-8755 slowdown when low contention
[ https://issues.apache.org/jira/browse/HBASE-10156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-10156: -- Attachment: 10156v20.txt Himanshu feedback Fix up the HBASE-8755 slowdown when low contention -- Key: HBASE-10156 URL: https://issues.apache.org/jira/browse/HBASE-10156 Project: HBase Issue Type: Sub-task Components: wal Reporter: stack Assignee: stack Attachments: 10156.txt, 10156v10.txt, 10156v11.txt, 10156v12.txt, 10156v12.txt, 10156v13.txt, 10156v16.txt, 10156v17.txt, 10156v18.txt, 10156v19.txt, 10156v2.txt, 10156v20.txt, 10156v20.txt, 10156v3.txt, 10156v4.txt, 10156v5.txt, 10156v6.txt, 10156v7.txt, 10156v9.txt, Disrupting.java HBASE-8755 slows our writes when only a few clients. Fix. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HBASE-10336) Remove deprecated usage of Hadoop HttpServer in InfoServer
[
https://issues.apache.org/jira/browse/HBASE-10336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13881971#comment-13881971
]
Eric Charles commented on HBASE-10336:
--
on my env (tested with jdk6 and jdk7):
{code}
git stash
git pull upstream trunk
patch -p0 HBASE-10336-7.patch
mvn clean install -DskipTests
...
BUILD SUCCESS
{code}
I wonder jenkins is not happy - anyone could quickly run the above snippet and
report back?
Thx, Eric
Remove deprecated usage of Hadoop HttpServer in InfoServer
--
Key: HBASE-10336
URL: https://issues.apache.org/jira/browse/HBASE-10336
Project: HBase
Issue Type: Bug
Affects Versions: 0.99.0
Reporter: Eric Charles
Assignee: Eric Charles
Attachments: HBASE-10336-1.patch, HBASE-10336-2.patch,
HBASE-10336-3.patch, HBASE-10336-4.patch, HBASE-10336-5.patch,
HBASE-10336-6.patch, HBASE-10336-7.patch
Recent changes in Hadoop HttpServer give NPE when running on hadoop
3.0.0-SNAPSHOT. This way we use HttpServer is deprecated and will probably be
not fixed (see HDFS-5760). We'd better move to the new proposed builder
pattern, which means we can no more use inheritance to build our nice
InfoServer.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (HBASE-10336) Remove deprecated usage of Hadoop HttpServer in InfoServer
[
https://issues.apache.org/jira/browse/HBASE-10336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13881980#comment-13881980
]
Ted Yu commented on HBASE-10336:
As QA reported, compilation against hadoop-1 failed:
{code}
+throw new HadoopIllegalArgumentException(name is not set);
{code}
HadoopIllegalArgumentException is not in hadoop-1
Consider using IllegalArgumentException instead.
Remove deprecated usage of Hadoop HttpServer in InfoServer
--
Key: HBASE-10336
URL: https://issues.apache.org/jira/browse/HBASE-10336
Project: HBase
Issue Type: Bug
Affects Versions: 0.99.0
Reporter: Eric Charles
Assignee: Eric Charles
Attachments: HBASE-10336-1.patch, HBASE-10336-2.patch,
HBASE-10336-3.patch, HBASE-10336-4.patch, HBASE-10336-5.patch,
HBASE-10336-6.patch, HBASE-10336-7.patch
Recent changes in Hadoop HttpServer give NPE when running on hadoop
3.0.0-SNAPSHOT. This way we use HttpServer is deprecated and will probably be
not fixed (see HDFS-5760). We'd better move to the new proposed builder
pattern, which means we can no more use inheritance to build our nice
InfoServer.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (HBASE-6581) Build with hadoop.profile=3.0
[
https://issues.apache.org/jira/browse/HBASE-6581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13881983#comment-13881983
]
Eric Charles commented on HBASE-6581:
-
my 'mvn site' simply hangs (as well on with or without this patch)
{code}
[INFO]
[INFO] Forking HBase - Shell 0.99.0-SNAPSHOT
[INFO]
[INFO]
[INFO] --- build-helper-maven-plugin:1.5:add-source
(jspcSource-packageInfo-source) @ hbase-shell ---
[INFO] Source directory:
/home/eric/wrk/com/hbase.git/hbase-shell/target/generated-jamon added.
[INFO] Source directory:
/home/eric/wrk/com/hbase.git/hbase-shell/target/generated-sources/java added.
[INFO]
[INFO] maven-javadoc-plugin:2.9.1:aggregate (report:aggregate) @ hbase
[INFO] Parent project loaded from repository: org.apache:apache:pom:12
[INFO] Relativizing decoration links with respect to project URL:
http://hbase.apache.org
[INFO] Rendering site with org.apache.maven.skins:maven-stylus-skin:jar:1.5
skin.
[INFO] Skipped About report, file index.html already exists for the English
version.
[INFO] Skipped Source Xref report, file xref/index.html already exists for
the English version.
[INFO] Skipped Test Source Xref report, file xref-test/index.html already
exists for the English version.
[WARNING] title was already defined in properties, ignored title in
head.
[WARNING] title was already defined in properties, ignored title in
head.
[INFO] Generating Project Team report---
maven-project-info-reports-plugin:2.7
[INFO] Generating Mailing Lists report---
maven-project-info-reports-plugin:2.7
[INFO] Generating Continuous Integration report---
maven-project-info-reports-plugin:2.7
[INFO] Generating Issue Tracking report---
maven-project-info-reports-plugin:2.7
[INFO] Generating Project License report---
maven-project-info-reports-plugin:2.7
[INFO] Generating Source Repository report---
maven-project-info-reports-plugin:2.7
[INFO] Generating Project Summary report---
maven-project-info-reports-plugin:2.7
[INFO] Generating Dependency Information report---
maven-project-info-reports-plugin:2.7
[INFO] Generating Project Modules report---
maven-project-info-reports-plugin:2.7
[INFO] Generating Dependency Management report---
maven-project-info-reports-plugin:2.7
[INFO] Generating Dependencies report---
maven-project-info-reports-plugin:2.7
[INFO] Generating Dependency Convergence report---
maven-project-info-reports-plugin:2.7
[INFO] Generating Plugin Management report---
maven-project-info-reports-plugin:2.7
[INFO] Generating Project Plugins report---
maven-project-info-reports-plugin:2.7
[INFO] Generating Distribution Management report---
maven-project-info-reports-plugin:2.7
[INFO] Generating Source Xref report--- maven-jxr-plugin:2.3
[INFO] Generating Test Source Xref report--- maven-jxr-plugin:2.3
[INFO] Generating JavaDocs report--- maven-javadoc-plugin:2.9.1
Hangs...
{code}
Build with hadoop.profile=3.0
-
Key: HBASE-6581
URL: https://issues.apache.org/jira/browse/HBASE-6581
Project: HBase
Issue Type: Bug
Reporter: Eric Charles
Assignee: Eric Charles
Fix For: 0.98.1
Attachments: HBASE-6581-1.patch, HBASE-6581-2.patch,
HBASE-6581-20130821.patch, HBASE-6581-3.patch, HBASE-6581-4.patch,
HBASE-6581-5.patch, HBASE-6581-6.patch, HBASE-6581-7.patch,
HBASE-6581-8-pom.patch, HBASE-6581.diff, HBASE-6581.diff
Building trunk with hadoop.profile=3.0 gives exceptions (see [1]) due to
change in the hadoop maven modules naming (and also usage of 3.0-SNAPSHOT
instead of 3.0.0-SNAPSHOT in hbase-common).
I can provide a patch that would move most of hadoop dependencies in their
respective profiles and will define the correct hadoop deps in the 3.0
profile.
Please tell me if that's ok to go this way.
Thx, Eric
[1]
$ mvn clean install -Dhadoop.profile=3.0
[INFO] Scanning for projects...
[ERROR] The build could not read 3 projects - [Help 1]
[ERROR]
[ERROR] The project org.apache.hbase:hbase-server:0.95-SNAPSHOT
(/d/hbase.svn/hbase-server/pom.xml) has 3 errors
[ERROR] 'dependencies.dependency.version' for
org.apache.hadoop:hadoop-common:jar is missing. @ line 655, column 21
[ERROR] 'dependencies.dependency.version' for
org.apache.hadoop:hadoop-annotations:jar is missing. @ line 659, column 21
[ERROR] 'dependencies.dependency.version' for
org.apache.hadoop:hadoop-minicluster:jar is missing. @ line 663, column 21
[ERROR]
[ERROR] The project org.apache.hbase:hbase-common:0.95-SNAPSHOT
(/d/hbase.svn/hbase-common/pom.xml) has 3 errors
[ERROR] 'dependencies.dependency.version' for
org.apache.hadoop:hadoop-common:jar is missing. @ line 170, column 21
[ERROR] 'dependencies.dependency.version' for
[jira] [Commented] (HBASE-10336) Remove deprecated usage of Hadoop HttpServer in InfoServer
[
https://issues.apache.org/jira/browse/HBASE-10336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13881992#comment-13881992
]
Eric Charles commented on HBASE-10336:
--
[~te...@apache.org] thx for the pointer. Actually, I looked at jenkins output
and didn't see the command invoked not any details on the outputs. just the
exception. I guess the build script mutes this on purpose which makes it more
tricky for newbies like me, if the -Dhadoop.profile=1.0 would have appeared in
the log, I would have seen where the error was coming from, obviously from the
fact I didn't try to compile with hadoop1.
I have decoupled quite more keys (there was the reported
DEFAULT_HADOOP_HTTP_STATIC_USER and a few more). Now I have a more tricky one
{code}
[ERROR]
/home/eric/wrk/com/hbase.git/hbase-server/src/main/java/org/apache/hadoop/hbase/http/conf/ConfServlet.java:[93,10]
writeXml(java.io.OutputStream) in org.apache.hadoop.conf.Configuration cannot
be applied to (java.io.Writer)
{code}
It's the classical ball of wool I'm sure you see what I eman You guys
please tell me: a quick hack for this before falling on the next one or a more
complete solution via 'HBASE-9213 create a unified shim for hadoop 1 and 2 so
that there's one build of HBase' - Depends how quick you want this HBASE-10336
to land.
Remove deprecated usage of Hadoop HttpServer in InfoServer
--
Key: HBASE-10336
URL: https://issues.apache.org/jira/browse/HBASE-10336
Project: HBase
Issue Type: Bug
Affects Versions: 0.99.0
Reporter: Eric Charles
Assignee: Eric Charles
Attachments: HBASE-10336-1.patch, HBASE-10336-2.patch,
HBASE-10336-3.patch, HBASE-10336-4.patch, HBASE-10336-5.patch,
HBASE-10336-6.patch, HBASE-10336-7.patch
Recent changes in Hadoop HttpServer give NPE when running on hadoop
3.0.0-SNAPSHOT. This way we use HttpServer is deprecated and will probably be
not fixed (see HDFS-5760). We'd better move to the new proposed builder
pattern, which means we can no more use inheritance to build our nice
InfoServer.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Updated] (HBASE-10395) endTime won't be set in VerifyReplication if startTime is not set
[
https://issues.apache.org/jira/browse/HBASE-10395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell updated HBASE-10395:
---
Fix Version/s: (was: 0.98.0)
0.98.1
Moved to 0.98.1
endTime won't be set in VerifyReplication if startTime is not set
-
Key: HBASE-10395
URL: https://issues.apache.org/jira/browse/HBASE-10395
Project: HBase
Issue Type: Improvement
Components: mapreduce, Replication
Affects Versions: 0.94.16
Reporter: cuijianwei
Assignee: cuijianwei
Priority: Minor
Fix For: 0.98.1, 0.99.0
Attachments: HBASE-10395-0.94-v1.patch, HBASE-10395-0.94-v2.patch,
HBASE-10395-0.94-v3.patch, HBASE-10395-trunk-v1.patch,
HBASE-10395-trunk-v2.patch, HBASE-10395-trunk-v3.patch
In VerifyReplication, we may set startTime and endTime to restrict the data
to verfiy. However, the endTime won't be set in the program if we only pass
endTime without startTime in command line argument. The reason is the
following code:
{code}
if (startTime != 0) {
scan.setTimeRange(startTime,
endTime == 0 ? HConstants.LATEST_TIMESTAMP : endTime);
}
{code}
The code will ignore endTime setting when not passing startTime in command
line argument. Another place needs to improvement is the help message as
follows:
{code}
System.err.println( stoprow end of the row);
{code}
However, the program actually use endrow to parse the arguments:
{code}
final String endTimeArgKey = --endtime=;
if (cmd.startsWith(endTimeArgKey)) {
endTime = Long.parseLong(cmd.substring(endTimeArgKey.length()));
continue;
}
{code}
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Updated] (HBASE-10336) Remove deprecated usage of Hadoop HttpServer in InfoServer
[ https://issues.apache.org/jira/browse/HBASE-10336?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Charles updated HBASE-10336: - Attachment: HBASE-10336-8.patch [~te...@apache.org] attached the patch which gives writeXml(java.io.OutputStream) in org.apache.hadoop.conf.Configuration cannot be applied to (java.io.Writer) with -Dhadoop.profile=1.0 with -Dhadoop.profile=1.1, it is worse: package org.apache.hadoop.security.ssl does not exist... and so one. Will further investigate my tomorrow. Remove deprecated usage of Hadoop HttpServer in InfoServer -- Key: HBASE-10336 URL: https://issues.apache.org/jira/browse/HBASE-10336 Project: HBase Issue Type: Bug Affects Versions: 0.99.0 Reporter: Eric Charles Assignee: Eric Charles Attachments: HBASE-10336-1.patch, HBASE-10336-2.patch, HBASE-10336-3.patch, HBASE-10336-4.patch, HBASE-10336-5.patch, HBASE-10336-6.patch, HBASE-10336-7.patch, HBASE-10336-8.patch Recent changes in Hadoop HttpServer give NPE when running on hadoop 3.0.0-SNAPSHOT. This way we use HttpServer is deprecated and will probably be not fixed (see HDFS-5760). We'd better move to the new proposed builder pattern, which means we can no more use inheritance to build our nice InfoServer. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HBASE-10336) Remove deprecated usage of Hadoop HttpServer in InfoServer
[
https://issues.apache.org/jira/browse/HBASE-10336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13882013#comment-13882013
]
Hadoop QA commented on HBASE-10336:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12625203/HBASE-10336-8.patch
against trunk revision .
ATTACHMENT ID: 12625203
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 48 new
or modified tests.
{color:red}-1 hadoop1.0{color}. The patch failed to compile against the
hadoop 1.0 profile.
Here is snippet of errors:
{code}[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-compiler-plugin:2.5.1:compile (default-compile)
on project hbase-server: Compilation failure
[ERROR]
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/http/conf/ConfServlet.java:[93,10]
writeXml(java.io.OutputStream) in org.apache.hadoop.conf.Configuration cannot
be applied to (java.io.Writer)
[ERROR] - [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal
org.apache.maven.plugins:maven-compiler-plugin:2.5.1:compile (default-compile)
on project hbase-server: Compilation failure
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/http/conf/ConfServlet.java:[93,10]
writeXml(java.io.OutputStream) in org.apache.hadoop.conf.Configuration cannot
be applied to (java.io.Writer)
at
org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:213)
at
org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)
at
org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)
--
Caused by: org.apache.maven.plugin.CompilationFailureException: Compilation
failure
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/http/conf/ConfServlet.java:[93,10]
writeXml(java.io.OutputStream) in org.apache.hadoop.conf.Configuration cannot
be applied to (java.io.Writer)
at
org.apache.maven.plugin.AbstractCompilerMojo.execute(AbstractCompilerMojo.java:729)
at org.apache.maven.plugin.CompilerMojo.execute(CompilerMojo.java:128)
at
org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:101){code}
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/8514//console
This message is automatically generated.
Remove deprecated usage of Hadoop HttpServer in InfoServer
--
Key: HBASE-10336
URL: https://issues.apache.org/jira/browse/HBASE-10336
Project: HBase
Issue Type: Bug
Affects Versions: 0.99.0
Reporter: Eric Charles
Assignee: Eric Charles
Attachments: HBASE-10336-1.patch, HBASE-10336-2.patch,
HBASE-10336-3.patch, HBASE-10336-4.patch, HBASE-10336-5.patch,
HBASE-10336-6.patch, HBASE-10336-7.patch, HBASE-10336-8.patch
Recent changes in Hadoop HttpServer give NPE when running on hadoop
3.0.0-SNAPSHOT. This way we use HttpServer is deprecated and will probably be
not fixed (see HDFS-5760). We'd better move to the new proposed builder
pattern, which means we can no more use inheritance to build our nice
InfoServer.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (HBASE-10156) Fix up the HBASE-8755 slowdown when low contention
[
https://issues.apache.org/jira/browse/HBASE-10156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13882018#comment-13882018
]
Hadoop QA commented on HBASE-10156:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12625197/10156v20.txt
against trunk revision .
ATTACHMENT ID: 12625197
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 30 new
or modified tests.
{color:green}+1 hadoop1.0{color}. The patch compiles against the hadoop
1.0 profile.
{color:green}+1 hadoop1.1{color}. The patch compiles against the hadoop
1.1 profile.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:red}-1 findbugs{color}. The patch appears to introduce 1 new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:red}-1 site{color}. The patch appears to cause mvn site goal to
fail.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/8513//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8513//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8513//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8513//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8513//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8513//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8513//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8513//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8513//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8513//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/8513//console
This message is automatically generated.
Fix up the HBASE-8755 slowdown when low contention
--
Key: HBASE-10156
URL: https://issues.apache.org/jira/browse/HBASE-10156
Project: HBase
Issue Type: Sub-task
Components: wal
Reporter: stack
Assignee: stack
Attachments: 10156.txt, 10156v10.txt, 10156v11.txt, 10156v12.txt,
10156v12.txt, 10156v13.txt, 10156v16.txt, 10156v17.txt, 10156v18.txt,
10156v19.txt, 10156v2.txt, 10156v20.txt, 10156v20.txt, 10156v3.txt,
10156v4.txt, 10156v5.txt, 10156v6.txt, 10156v7.txt, 10156v9.txt,
Disrupting.java
HBASE-8755 slows our writes when only a few clients. Fix.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Updated] (HBASE-10156) Fix up the HBASE-8755 slowdown when low contention
[ https://issues.apache.org/jira/browse/HBASE-10156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-10156: -- Attachment: 10156v21.txt Findbugs fix. Any +1s out there? Thanks. Fix up the HBASE-8755 slowdown when low contention -- Key: HBASE-10156 URL: https://issues.apache.org/jira/browse/HBASE-10156 Project: HBase Issue Type: Sub-task Components: wal Reporter: stack Assignee: stack Attachments: 10156.txt, 10156v10.txt, 10156v11.txt, 10156v12.txt, 10156v12.txt, 10156v13.txt, 10156v16.txt, 10156v17.txt, 10156v18.txt, 10156v19.txt, 10156v2.txt, 10156v20.txt, 10156v20.txt, 10156v21.txt, 10156v3.txt, 10156v4.txt, 10156v5.txt, 10156v6.txt, 10156v7.txt, 10156v9.txt, Disrupting.java HBASE-8755 slows our writes when only a few clients. Fix. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (HBASE-10156) FSHLog Refactor (WAS - Fix up the HBASE-8755 slowdown when low contention)
[ https://issues.apache.org/jira/browse/HBASE-10156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-10156: -- Summary: FSHLog Refactor (WAS - Fix up the HBASE-8755 slowdown when low contention) (was: Fix up the HBASE-8755 slowdown when low contention) FSHLog Refactor (WAS - Fix up the HBASE-8755 slowdown when low contention) --- Key: HBASE-10156 URL: https://issues.apache.org/jira/browse/HBASE-10156 Project: HBase Issue Type: Sub-task Components: wal Reporter: stack Assignee: stack Attachments: 10156.txt, 10156v10.txt, 10156v11.txt, 10156v12.txt, 10156v12.txt, 10156v13.txt, 10156v16.txt, 10156v17.txt, 10156v18.txt, 10156v19.txt, 10156v2.txt, 10156v20.txt, 10156v20.txt, 10156v21.txt, 10156v3.txt, 10156v4.txt, 10156v5.txt, 10156v6.txt, 10156v7.txt, 10156v9.txt, Disrupting.java HBASE-8755 slows our writes when only a few clients. Fix. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HBASE-10156) FSHLog Refactor (WAS - Fix up the HBASE-8755 slowdown when low contention)
[
https://issues.apache.org/jira/browse/HBASE-10156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13882043#comment-13882043
]
Hadoop QA commented on HBASE-10156:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12625215/10156v21.txt
against trunk revision .
ATTACHMENT ID: 12625215
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 30 new
or modified tests.
{color:red}-1 hadoop1.0{color}. The patch failed to compile against the
hadoop 1.0 profile.
Here is snippet of errors:
{code}[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-compiler-plugin:2.5.1:compile (default-compile)
on project hbase-server: Compilation failure: Compilation failure:
[ERROR]
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/FSHLog.java:[600,8]
cannot find symbol
[ERROR] symbol : variable currentFilenum
[ERROR] location: class org.apache.hadoop.hbase.regionserver.wal.FSHLog
[ERROR]
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/FSHLog.java:[602,32]
cannot find symbol
[ERROR] symbol : variable currentFilenum
--
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal
org.apache.maven.plugins:maven-compiler-plugin:2.5.1:compile (default-compile)
on project hbase-server: Compilation failure
at
org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:213)
at
org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)
at
org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)
at
org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:84)
at
org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:59)
--
Caused by: org.apache.maven.plugin.CompilationFailureException: Compilation
failure
at
org.apache.maven.plugin.AbstractCompilerMojo.execute(AbstractCompilerMojo.java:729)
at org.apache.maven.plugin.CompilerMojo.execute(CompilerMojo.java:128)
at
org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:101)
at
org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:209)
... 19 more{code}
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/8515//console
This message is automatically generated.
FSHLog Refactor (WAS - Fix up the HBASE-8755 slowdown when low contention)
---
Key: HBASE-10156
URL: https://issues.apache.org/jira/browse/HBASE-10156
Project: HBase
Issue Type: Sub-task
Components: wal
Reporter: stack
Assignee: stack
Attachments: 10156.txt, 10156v10.txt, 10156v11.txt, 10156v12.txt,
10156v12.txt, 10156v13.txt, 10156v16.txt, 10156v17.txt, 10156v18.txt,
10156v19.txt, 10156v2.txt, 10156v20.txt, 10156v20.txt, 10156v21.txt,
10156v3.txt, 10156v4.txt, 10156v5.txt, 10156v6.txt, 10156v7.txt, 10156v9.txt,
Disrupting.java
HBASE-8755 slows our writes when only a few clients. Fix.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Updated] (HBASE-10156) FSHLog Refactor (WAS - Fix up the HBASE-8755 slowdown when low contention)
[ https://issues.apache.org/jira/browse/HBASE-10156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-10156: -- Attachment: 10156v21.txt Fixing findbugs. FSHLog Refactor (WAS - Fix up the HBASE-8755 slowdown when low contention) --- Key: HBASE-10156 URL: https://issues.apache.org/jira/browse/HBASE-10156 Project: HBase Issue Type: Sub-task Components: wal Reporter: stack Assignee: stack Attachments: 10156.txt, 10156v10.txt, 10156v11.txt, 10156v12.txt, 10156v12.txt, 10156v13.txt, 10156v16.txt, 10156v17.txt, 10156v18.txt, 10156v19.txt, 10156v2.txt, 10156v20.txt, 10156v20.txt, 10156v21.txt, 10156v21.txt, 10156v3.txt, 10156v4.txt, 10156v5.txt, 10156v6.txt, 10156v7.txt, 10156v9.txt, Disrupting.java HBASE-8755 slows our writes when only a few clients. Fix. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HBASE-6581) Build with hadoop.profile=3.0
[ https://issues.apache.org/jira/browse/HBASE-6581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13882056#comment-13882056 ] stack commented on HBASE-6581: -- Does HBASE-10336 have to go in first [~echarles] before this one? Build with hadoop.profile=3.0 - Key: HBASE-6581 URL: https://issues.apache.org/jira/browse/HBASE-6581 Project: HBase Issue Type: Bug Reporter: Eric Charles Assignee: Eric Charles Fix For: 0.98.1 Attachments: HBASE-6581-1.patch, HBASE-6581-2.patch, HBASE-6581-20130821.patch, HBASE-6581-3.patch, HBASE-6581-4.patch, HBASE-6581-5.patch, HBASE-6581-6.patch, HBASE-6581-7.patch, HBASE-6581-8-pom.patch, HBASE-6581.diff, HBASE-6581.diff Building trunk with hadoop.profile=3.0 gives exceptions (see [1]) due to change in the hadoop maven modules naming (and also usage of 3.0-SNAPSHOT instead of 3.0.0-SNAPSHOT in hbase-common). I can provide a patch that would move most of hadoop dependencies in their respective profiles and will define the correct hadoop deps in the 3.0 profile. Please tell me if that's ok to go this way. Thx, Eric [1] $ mvn clean install -Dhadoop.profile=3.0 [INFO] Scanning for projects... [ERROR] The build could not read 3 projects - [Help 1] [ERROR] [ERROR] The project org.apache.hbase:hbase-server:0.95-SNAPSHOT (/d/hbase.svn/hbase-server/pom.xml) has 3 errors [ERROR] 'dependencies.dependency.version' for org.apache.hadoop:hadoop-common:jar is missing. @ line 655, column 21 [ERROR] 'dependencies.dependency.version' for org.apache.hadoop:hadoop-annotations:jar is missing. @ line 659, column 21 [ERROR] 'dependencies.dependency.version' for org.apache.hadoop:hadoop-minicluster:jar is missing. @ line 663, column 21 [ERROR] [ERROR] The project org.apache.hbase:hbase-common:0.95-SNAPSHOT (/d/hbase.svn/hbase-common/pom.xml) has 3 errors [ERROR] 'dependencies.dependency.version' for org.apache.hadoop:hadoop-common:jar is missing. @ line 170, column 21 [ERROR] 'dependencies.dependency.version' for org.apache.hadoop:hadoop-annotations:jar is missing. @ line 174, column 21 [ERROR] 'dependencies.dependency.version' for org.apache.hadoop:hadoop-minicluster:jar is missing. @ line 178, column 21 [ERROR] [ERROR] The project org.apache.hbase:hbase-it:0.95-SNAPSHOT (/d/hbase.svn/hbase-it/pom.xml) has 3 errors [ERROR] 'dependencies.dependency.version' for org.apache.hadoop:hadoop-common:jar is missing. @ line 220, column 18 [ERROR] 'dependencies.dependency.version' for org.apache.hadoop:hadoop-annotations:jar is missing. @ line 224, column 21 [ERROR] 'dependencies.dependency.version' for org.apache.hadoop:hadoop-minicluster:jar is missing. @ line 228, column 21 [ERROR] -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HBASE-6581) Build with hadoop.profile=3.0
[ https://issues.apache.org/jira/browse/HBASE-6581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13882055#comment-13882055 ] stack commented on HBASE-6581: -- Don't mind the failing mvn site. It is not you. The hanging you see probably completes eventually or it will OOME. Build with hadoop.profile=3.0 - Key: HBASE-6581 URL: https://issues.apache.org/jira/browse/HBASE-6581 Project: HBase Issue Type: Bug Reporter: Eric Charles Assignee: Eric Charles Fix For: 0.98.1 Attachments: HBASE-6581-1.patch, HBASE-6581-2.patch, HBASE-6581-20130821.patch, HBASE-6581-3.patch, HBASE-6581-4.patch, HBASE-6581-5.patch, HBASE-6581-6.patch, HBASE-6581-7.patch, HBASE-6581-8-pom.patch, HBASE-6581.diff, HBASE-6581.diff Building trunk with hadoop.profile=3.0 gives exceptions (see [1]) due to change in the hadoop maven modules naming (and also usage of 3.0-SNAPSHOT instead of 3.0.0-SNAPSHOT in hbase-common). I can provide a patch that would move most of hadoop dependencies in their respective profiles and will define the correct hadoop deps in the 3.0 profile. Please tell me if that's ok to go this way. Thx, Eric [1] $ mvn clean install -Dhadoop.profile=3.0 [INFO] Scanning for projects... [ERROR] The build could not read 3 projects - [Help 1] [ERROR] [ERROR] The project org.apache.hbase:hbase-server:0.95-SNAPSHOT (/d/hbase.svn/hbase-server/pom.xml) has 3 errors [ERROR] 'dependencies.dependency.version' for org.apache.hadoop:hadoop-common:jar is missing. @ line 655, column 21 [ERROR] 'dependencies.dependency.version' for org.apache.hadoop:hadoop-annotations:jar is missing. @ line 659, column 21 [ERROR] 'dependencies.dependency.version' for org.apache.hadoop:hadoop-minicluster:jar is missing. @ line 663, column 21 [ERROR] [ERROR] The project org.apache.hbase:hbase-common:0.95-SNAPSHOT (/d/hbase.svn/hbase-common/pom.xml) has 3 errors [ERROR] 'dependencies.dependency.version' for org.apache.hadoop:hadoop-common:jar is missing. @ line 170, column 21 [ERROR] 'dependencies.dependency.version' for org.apache.hadoop:hadoop-annotations:jar is missing. @ line 174, column 21 [ERROR] 'dependencies.dependency.version' for org.apache.hadoop:hadoop-minicluster:jar is missing. @ line 178, column 21 [ERROR] [ERROR] The project org.apache.hbase:hbase-it:0.95-SNAPSHOT (/d/hbase.svn/hbase-it/pom.xml) has 3 errors [ERROR] 'dependencies.dependency.version' for org.apache.hadoop:hadoop-common:jar is missing. @ line 220, column 18 [ERROR] 'dependencies.dependency.version' for org.apache.hadoop:hadoop-annotations:jar is missing. @ line 224, column 21 [ERROR] 'dependencies.dependency.version' for org.apache.hadoop:hadoop-minicluster:jar is missing. @ line 228, column 21 [ERROR] -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HBASE-6048) Table Scan is failing if offheap cache enabled
[
https://issues.apache.org/jira/browse/HBASE-6048?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13882058#comment-13882058
]
stack commented on HBASE-6048:
--
I have just enabled this and ditto do not see the issue in head of hbase and
hadoop2. Am looking more to add some evidence that it should be removed in
favor of bucket cache direction.
Table Scan is failing if offheap cache enabled
--
Key: HBASE-6048
URL: https://issues.apache.org/jira/browse/HBASE-6048
Project: HBase
Issue Type: Bug
Affects Versions: 0.94.0
Reporter: Gopinathan A
Assignee: ramkrishna.s.vasudevan
Attachments: hbase-6048-94.patch
Table Scan is failing if offheap cache enabled.
{noformat}
2012-05-18 20:03:38,446 DEBUG org.apache.hadoop.hbase.io.hfile.HFileWriterV2:
Initialized with CacheConfig:enabled [cacheDataOnRead=true]
[cacheDataOnWrite=false] [cacheIndexesOnWrite=false]
[cacheBloomsOnWrite=false] [cacheEvictOnClose=false] [cacheCompressed=false]
2012-05-18 20:03:38,446 INFO org.apache.hadoop.hbase.regionserver.StoreFile:
Delete Family Bloom filter type for
hdfs://10.18.40.217:9000/hbase/ufdr/1d4656fd417a07c9171a38b8f4d08510/.tmp/03742024b28f443bb63cfc338d4ca422:
CompoundBloomFilterWriter
2012-05-18 20:04:25,576 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache:
Block cache LRU eviction started; Attempting to free 120.57 MB of
total=1020.57 MB
2012-05-18 20:04:25,655 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache:
Block cache LRU eviction completed; freed=120.82 MB, total=907.89 MB,
single=1012.11 MB, multi=6.12 MB, memory=0 KB
2012-05-18 20:04:25,733 ERROR
org.apache.hadoop.hbase.regionserver.HRegionServer: Failed openScanner
java.lang.IllegalStateException: Schema metrics requested before table/CF
name initialization: {tableName:null,cfName:null}
at
org.apache.hadoop.hbase.regionserver.metrics.SchemaConfigured.getSchemaMetrics(SchemaConfigured.java:182)
at
org.apache.hadoop.hbase.io.hfile.LruBlockCache.updateSizeMetrics(LruBlockCache.java:310)
at
org.apache.hadoop.hbase.io.hfile.LruBlockCache.cacheBlock(LruBlockCache.java:274)
at
org.apache.hadoop.hbase.io.hfile.LruBlockCache.cacheBlock(LruBlockCache.java:293)
at
org.apache.hadoop.hbase.io.hfile.DoubleBlockCache.getBlock(DoubleBlockCache.java:102)
at
org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:296)
at
org.apache.hadoop.hbase.io.hfile.HFileBlockIndex$BlockIndexReader.seekToDataBlock(HFileBlockIndex.java:213)
at
org.apache.hadoop.hbase.io.hfile.HFileReaderV2$AbstractScannerV2.seekTo(HFileReaderV2.java:455)
at
org.apache.hadoop.hbase.io.hfile.HFileReaderV2$AbstractScannerV2.seekTo(HFileReaderV2.java:475)
at
org.apache.hadoop.hbase.regionserver.StoreFileScanner.seekAtOrAfter(StoreFileScanner.java:226)
at
org.apache.hadoop.hbase.regionserver.StoreFileScanner.seek(StoreFileScanner.java:145)
at
org.apache.hadoop.hbase.regionserver.StoreScanner.init(StoreScanner.java:130)
at
org.apache.hadoop.hbase.regionserver.Store.getScanner(Store.java:2001)
at
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.init(HRegion.java:3274)
at
org.apache.hadoop.hbase.regionserver.HRegion.instantiateRegionScanner(HRegion.java:1604)
at
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:1596)
at
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:1572)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.openScanner(HRegionServer.java:2310)
at sun.reflect.GeneratedMethodAccessor45.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364)
at
org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1376)
2012-05-18 20:04:25,828 ERROR
org.apache.hadoop.hbase.regionserver.HRegionServer: Failed openScanner
{noformat}
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (HBASE-10409) Allow config options related to zookeeper to be passed to IntegrationTestsDriver
[
https://issues.apache.org/jira/browse/HBASE-10409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13882060#comment-13882060
]
stack commented on HBASE-10409:
---
So, this is resolved 'later' though a bunch of input from a reviewer to help
enable it get implemented correctly?
Allow config options related to zookeeper to be passed to
IntegrationTestsDriver
Key: HBASE-10409
URL: https://issues.apache.org/jira/browse/HBASE-10409
Project: HBase
Issue Type: Test
Reporter: Ted Yu
Assignee: Ted Yu
Attachments: 10409-v1.txt
Currently IntegrationTestsDriver only accepts one parameter: regex for
integration test.
{code}
public static void main(String[] args) throws Exception {
int ret = ToolRunner.run(new IntegrationTestsDriver(), args);
{code}
To allow IntegrationTestsDriver to be activated through Java API, support for
passing config options related to zookeeper should be added - similarly to
what LoadTestTool supports.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (HBASE-10399) Add documentation for VerifyReplication to refguide
[ https://issues.apache.org/jira/browse/HBASE-10399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13882063#comment-13882063 ] stack commented on HBASE-10399: --- [~gustavoanatoly] That is gracious of you. Wouldn't mind an answer to my question [~ted_yu] Add documentation for VerifyReplication to refguide --- Key: HBASE-10399 URL: https://issues.apache.org/jira/browse/HBASE-10399 Project: HBase Issue Type: Improvement Reporter: Ted Yu Assignee: Gustavo Anatoly Priority: Minor HBase refguide currently doesn't document how VerifyReplication is used for comparing local table with remote table. Document for VerifyReplication should be added so that users know how to use it. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HBASE-7320) Replace calls to KeyValue.getBuffer with appropropriate calls to getRowArray, getFamilyArray(), getQualifierArray, and getValueArray
[ https://issues.apache.org/jira/browse/HBASE-7320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13882067#comment-13882067 ] stack commented on HBASE-7320: -- I already gave +1 on commit. Replace calls to KeyValue.getBuffer with appropropriate calls to getRowArray, getFamilyArray(), getQualifierArray, and getValueArray Key: HBASE-7320 URL: https://issues.apache.org/jira/browse/HBASE-7320 Project: HBase Issue Type: Bug Reporter: Lars Hofhansl Assignee: stack Fix For: 0.99.0 Attachments: 7320-simple.txt In many places this is simple task of just replacing the method name. There, however, quite a few places where we assume that: # the entire KV is backed by a single byte array # the KVs key portion is backed by a single byte array Some of those can easily be fixed, others will need their own jiras. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HBASE-7320) Replace calls to KeyValue.getBuffer with appropropriate calls to getRowArray, getFamilyArray(), getQualifierArray, and getValueArray
[ https://issues.apache.org/jira/browse/HBASE-7320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13882066#comment-13882066 ] stack commented on HBASE-7320: -- bq. All calls to getBuffer() left are of two categories: access a KVs complete key, access the KV as a whole, Now we need to fix those. How you think we fix these Lars? Would the best approach now be to try and implement a new Cell type altogether? That would shake out any reliance on KV? bq. The next part is to get rid of the timestamp array API completely. Can we change the Cell Interface ex post facto? Cell was published in 0.96.0? Or could we deprecate the array accesses on timestamp and behind the scenes return an array if asked but not use it in core codebase? bq. After that, I think, is to writeTo/readFrom files. The writing side might be simple as long we keep the format, in that case the reading side might not have to change at all (it is still OK to back a Cell by a single byte[]) What is to be done here? Adding Cell Interfaces and moving over to use those? Encoders? bq, Big parts are: ScanQueryMatcher, which dissects the KV itself, and all the comparators in KeyValue itself. We would move to use Cell-base comparators rather than KV#comparators? Replace calls to KeyValue.getBuffer with appropropriate calls to getRowArray, getFamilyArray(), getQualifierArray, and getValueArray Key: HBASE-7320 URL: https://issues.apache.org/jira/browse/HBASE-7320 Project: HBase Issue Type: Bug Reporter: Lars Hofhansl Assignee: stack Fix For: 0.99.0 Attachments: 7320-simple.txt In many places this is simple task of just replacing the method name. There, however, quite a few places where we assume that: # the entire KV is backed by a single byte array # the KVs key portion is backed by a single byte array Some of those can easily be fixed, others will need their own jiras. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HBASE-7320) Replace calls to KeyValue.getBuffer with appropropriate calls to getRowArray, getFamilyArray(), getQualifierArray, and getValueArray
[ https://issues.apache.org/jira/browse/HBASE-7320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13882068#comment-13882068 ] stack commented on HBASE-7320: -- bq. As for this patch, should I put it into a subtask? Yes. Replace calls to KeyValue.getBuffer with appropropriate calls to getRowArray, getFamilyArray(), getQualifierArray, and getValueArray Key: HBASE-7320 URL: https://issues.apache.org/jira/browse/HBASE-7320 Project: HBase Issue Type: Bug Reporter: Lars Hofhansl Assignee: stack Fix For: 0.99.0 Attachments: 7320-simple.txt In many places this is simple task of just replacing the method name. There, however, quite a few places where we assume that: # the entire KV is backed by a single byte array # the KVs key portion is backed by a single byte array Some of those can easily be fixed, others will need their own jiras. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HBASE-10419) Add multiget support to PerformanceEvaluation
[ https://issues.apache.org/jira/browse/HBASE-10419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13882069#comment-13882069 ] stack commented on HBASE-10419: --- +1 Add multiget support to PerformanceEvaluation - Key: HBASE-10419 URL: https://issues.apache.org/jira/browse/HBASE-10419 Project: HBase Issue Type: Improvement Components: test Reporter: Nick Dimiduk Assignee: Nick Dimiduk Priority: Minor Attachments: HBASE-10419.0.patch Folks planning to use multiget may find this useful. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HBASE-7320) Replace calls to KeyValue.getBuffer with appropropriate calls to getRowArray, getFamilyArray(), getQualifierArray, and getValueArray
[
https://issues.apache.org/jira/browse/HBASE-7320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13882073#comment-13882073
]
Lars Hofhansl commented on HBASE-7320:
--
bq. Can we change the Cell Interface ex post facto?
Cell has only {{long getTimestamp()}}, so that's clean. I meant removing calls
to getTimestampOffset, etc.
bq. Would the best approach now be to try and implement a new Cell type
altogether?
Maybe, but then we'd have to replace KeyValue everywhere, and the KeyValue type
leaks to the client, so not sure we can do that.
In the end it'd come to the same, either we make a new type, or we fix KeyValue
to be the new type. Either way seems fine.
bq. How you think we fix these Lars?
Need to look at each case individually to see how to fix them. Some are only
printing out the key for example and in that we just print the
row/family/qualifier/timestamp separately.
Replace calls to KeyValue.getBuffer with appropropriate calls to getRowArray,
getFamilyArray(), getQualifierArray, and getValueArray
Key: HBASE-7320
URL: https://issues.apache.org/jira/browse/HBASE-7320
Project: HBase
Issue Type: Bug
Reporter: Lars Hofhansl
Assignee: stack
Fix For: 0.99.0
Attachments: 7320-simple.txt
In many places this is simple task of just replacing the method name.
There, however, quite a few places where we assume that:
# the entire KV is backed by a single byte array
# the KVs key portion is backed by a single byte array
Some of those can easily be fixed, others will need their own jiras.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Created] (HBASE-10420) Replace KV.getBuffer with KV.get{Row|Family|Qualifier|Value|Tags}Array
Lars Hofhansl created HBASE-10420:
-
Summary: Replace KV.getBuffer with
KV.get{Row|Family|Qualifier|Value|Tags}Array
Key: HBASE-10420
URL: https://issues.apache.org/jira/browse/HBASE-10420
Project: HBase
Issue Type: Sub-task
Reporter: Lars Hofhansl
Assignee: Lars Hofhansl
Fix For: 0.99.0
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (HBASE-7320) Replace calls to KeyValue.getBuffer with appropropriate calls to getRowArray, getFamilyArray(), getQualifierArray, and getValueArray
[ https://issues.apache.org/jira/browse/HBASE-7320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13882075#comment-13882075 ] Lars Hofhansl commented on HBASE-7320: -- Filed HBASE-10420. Let's leave this one unassigned as umbrella issue. Replace calls to KeyValue.getBuffer with appropropriate calls to getRowArray, getFamilyArray(), getQualifierArray, and getValueArray Key: HBASE-7320 URL: https://issues.apache.org/jira/browse/HBASE-7320 Project: HBase Issue Type: Bug Reporter: Lars Hofhansl Assignee: stack Fix For: 0.99.0 Attachments: 7320-simple.txt In many places this is simple task of just replacing the method name. There, however, quite a few places where we assume that: # the entire KV is backed by a single byte array # the KVs key portion is backed by a single byte array Some of those can easily be fixed, others will need their own jiras. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (HBASE-10420) Replace KV.getBuffer with KV.get{Row|Family|Qualifier|Value|Tags}Array
[
https://issues.apache.org/jira/browse/HBASE-10420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lars Hofhansl updated HBASE-10420:
--
Attachment: 7320-simple.txt
Patch from parent. Better to do it in a subtask
Replace KV.getBuffer with KV.get{Row|Family|Qualifier|Value|Tags}Array
--
Key: HBASE-10420
URL: https://issues.apache.org/jira/browse/HBASE-10420
Project: HBase
Issue Type: Sub-task
Reporter: Lars Hofhansl
Assignee: Lars Hofhansl
Fix For: 0.99.0
Attachments: 7320-simple.txt
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Updated] (HBASE-10420) Replace KV.getBuffer with KV.get{Row|Family|Qualifier|Value|Tags}Array
[
https://issues.apache.org/jira/browse/HBASE-10420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lars Hofhansl updated HBASE-10420:
--
Description:
We we start by removing KeyValue.getBuffer() in all obvious places.
Obvious places are where we access the buffer by getRowOffset() and
getRowLength(), etc, since we then now we're cross a
row/family/qualifier/value/tags boundary.
The attached patch does that.
Replace KV.getBuffer with KV.get{Row|Family|Qualifier|Value|Tags}Array
--
Key: HBASE-10420
URL: https://issues.apache.org/jira/browse/HBASE-10420
Project: HBase
Issue Type: Sub-task
Reporter: Lars Hofhansl
Assignee: Lars Hofhansl
Fix For: 0.99.0
Attachments: 7320-simple.txt
We we start by removing KeyValue.getBuffer() in all obvious places.
Obvious places are where we access the buffer by getRowOffset() and
getRowLength(), etc, since we then now we're cross a
row/family/qualifier/value/tags boundary.
The attached patch does that.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Resolved] (HBASE-10420) Replace KV.getBuffer with KV.get{Row|Family|Qualifier|Value|Tags}Array
[
https://issues.apache.org/jira/browse/HBASE-10420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lars Hofhansl resolved HBASE-10420.
---
Resolution: Fixed
Hadoop Flags: Reviewed
Committed to trunk. Patch was tested/reviewed/+1'd in parent.
Replace KV.getBuffer with KV.get{Row|Family|Qualifier|Value|Tags}Array
--
Key: HBASE-10420
URL: https://issues.apache.org/jira/browse/HBASE-10420
Project: HBase
Issue Type: Sub-task
Reporter: Lars Hofhansl
Assignee: Lars Hofhansl
Fix For: 0.99.0
Attachments: 7320-simple.txt
We we start by removing KeyValue.getBuffer() in all obvious places.
Obvious places are where we access the buffer by getRowOffset() and
getRowLength(), etc, since we then now we're cross a
row/family/qualifier/value/tags boundary.
The attached patch does that.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Updated] (HBASE-7320) Remove KeyValue.getBuffer()
[ https://issues.apache.org/jira/browse/HBASE-7320?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl updated HBASE-7320: - Summary: Remove KeyValue.getBuffer() (was: Replace calls to KeyValue.getBuffer with appropropriate calls to getRowArray, getFamilyArray(), getQualifierArray, and getValueArray) Remove KeyValue.getBuffer() --- Key: HBASE-7320 URL: https://issues.apache.org/jira/browse/HBASE-7320 Project: HBase Issue Type: Bug Reporter: Lars Hofhansl Assignee: stack Fix For: 0.99.0 Attachments: 7320-simple.txt In many places this is simple task of just replacing the method name. There, however, quite a few places where we assume that: # the entire KV is backed by a single byte array # the KVs key portion is backed by a single byte array Some of those can easily be fixed, others will need their own jiras. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HBASE-10156) FSHLog Refactor (WAS - Fix up the HBASE-8755 slowdown when low contention)
[ https://issues.apache.org/jira/browse/HBASE-10156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13882087#comment-13882087 ] Himanshu Vashishtha commented on HBASE-10156: - +1 (non binding, though) :) Great stuff. Thanks Stack. FSHLog Refactor (WAS - Fix up the HBASE-8755 slowdown when low contention) --- Key: HBASE-10156 URL: https://issues.apache.org/jira/browse/HBASE-10156 Project: HBase Issue Type: Sub-task Components: wal Reporter: stack Assignee: stack Attachments: 10156.txt, 10156v10.txt, 10156v11.txt, 10156v12.txt, 10156v12.txt, 10156v13.txt, 10156v16.txt, 10156v17.txt, 10156v18.txt, 10156v19.txt, 10156v2.txt, 10156v20.txt, 10156v20.txt, 10156v21.txt, 10156v21.txt, 10156v3.txt, 10156v4.txt, 10156v5.txt, 10156v6.txt, 10156v7.txt, 10156v9.txt, Disrupting.java HBASE-8755 slows our writes when only a few clients. Fix. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HBASE-3909) Add dynamic config
[ https://issues.apache.org/jira/browse/HBASE-3909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13882088#comment-13882088 ] stack commented on HBASE-3909: -- Patch is great. Lets get it in. There is a hole in that the admin client does not know how to talk to backup master which we should fix in general but it should not get in the way of this patch. What you think [~aoxiang]? Apply this as is and then add configs as we go? Needs a bit of doc. so devs know this mechanism exists. Would suggest fixing the class comment so it uses javadoc formatting especially given the documentation on the class is so good. Rereading the old comments on this issue talking of hoisting configs up into zk so we can exploit its callback mechanism, this mechanism does none of that and just has us reload the configs from under the running processors. One concern I'd have is that we are reloading the configs but the Server constructors do a bunch of checking and setting on configs. Will these settings be lost when we reload from raw configs on disk. For example, HRS constructor calls FSUtils.setupShortCircuitRead(this.conf). Do these settings persist pas the reload call? Good stuff. Add dynamic config -- Key: HBASE-3909 URL: https://issues.apache.org/jira/browse/HBASE-3909 Project: HBase Issue Type: New Feature Reporter: stack Assignee: Subbu M Iyer Attachments: 3909-102812.patch, 3909-102912.patch, 3909-v1.patch, 3909.v1, 3909_090712-2.patch, HBASE-3909-backport-from-fb-for-trunk-2.patch, HBASE-3909-backport-from-fb-for-trunk.patch, HBase Cluster Config Details.xlsx, patch-v2.patch, testMasterNoCluster.stack I'm sure this issue exists already, at least as part of the discussion around making online schema edits possible, but no hard this having its own issue. Ted started a conversation on this topic up on dev and Todd suggested we lookd at how Hadoop did it over in HADOOP-7001 -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (HBASE-10407) String Format Exception
[
https://issues.apache.org/jira/browse/HBASE-10407?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-10407:
--
Attachment: hbase.patch
Retry for hadoopqa
String Format Exception
---
Key: HBASE-10407
URL: https://issues.apache.org/jira/browse/HBASE-10407
Project: HBase
Issue Type: Bug
Components: hadoop2
Affects Versions: 0.99.0
Reporter: Siwakorn Srisakaokul
Priority: Trivial
Attachments: hbase.patch, hbase.patch
There is a bug in
{{hbase/hbase-hadoop2-compat/src/main/java/org/apache/hadoop/metrics2/lib/MetricMutableQuantiles.java:91/92}}.
When the parameter {{name}} contains a %, it will cause a format exception
by String.Format(...).
*Here is a simple patch to fix it*
{code:title=hbase.patch|borderStyle=solid}
From 43a4b247f5ac69f57264b2c1b20dcca8205514c4 Mon Sep 17 00:00:00 2001
From: Siwakorn Srisakaokul sping...@gmail.com
Date: Wed, 22 Jan 2014 10:09:26 -0800
Subject: [PATCH] Format Patch
---
.../org/apache/hadoop/metrics2/lib/MetricMutableQuantiles.java | 10
--
1 file changed, 4 insertions(+), 6 deletions(-)
diff --git
a/hbase-hadoop2-compat/src/main/java/org/apache/hadoop/metrics2/lib/MetricMutableQuantiles.java
b/hbase-hadoop2-compat/src/main/java/org/apache/hadoop/metrics2/lib/MetricMutableQuantiles.java
index b01f88b..c399323 100644
---
a/hbase-hadoop2-compat/src/main/java/org/apache/hadoop/metrics2/lib/MetricMutableQuantiles.java
+++
b/hbase-hadoop2-compat/src/main/java/org/apache/hadoop/metrics2/lib/MetricMutableQuantiles.java
@@ -82,14 +82,12 @@ public class MetricMutableQuantiles extends MutableMetric
implements MetricHisto
Number of %s for %s with %ds interval, lsName, desc, interval));
// Construct the MetricsInfos for the quantiles, converting to
percentiles
quantileInfos = new MetricsInfo[quantiles.length];
-String nameTemplate = ucName + %dthPercentile + interval + sInterval
-+ uvName;
-String descTemplate = %d percentile + lvName + with + interval
-+ second interval for + desc;
+String nameTemplate = %s%dthPercentile%dsInterval%s;
+String descTemplate = %d percentile %s with %d second interval for %s;
for (int i = 0; i quantiles.length; i++) {
int percentile = (int) (100 * quantiles[i].quantile);
- quantileInfos[i] = info(String.format(nameTemplate, percentile),
- String.format(descTemplate, percentile));
+ quantileInfos[i] = info(String.format(nameTemplate, ucName,
percentile, interval, uvName),
+ String.format(descTemplate, percentile, lvName, interval, desc));
}
estimator = new MetricSampleQuantiles(quantiles);
--
1.8.5.2
{code}
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Updated] (HBASE-10169) Batch coprocessor
[ https://issues.apache.org/jira/browse/HBASE-10169?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-10169: -- Attachment: HBASE-10169-V3.patch Reattach so hadoopqa will notice Batch coprocessor - Key: HBASE-10169 URL: https://issues.apache.org/jira/browse/HBASE-10169 Project: HBase Issue Type: Sub-task Components: Coprocessors Affects Versions: 0.99.0 Reporter: Jingcheng Du Assignee: Jingcheng Du Attachments: Batch Coprocessor Design Document.docx, HBASE-10169-V2.patch, HBASE-10169-V3.patch, HBASE-10169-V3.patch, HBASE-10169.patch This is designed to improve the coprocessor invocation in the client side. Currently the coprocessor invocation is to send a call to each region. If there’s one region server, and 100 regions are located in this server, each coprocessor invocation will send 100 calls, each call uses a single thread in the client side. The threads will run out soon when the coprocessor invocations are heavy. In this design, all the calls to the same region server will be grouped into one in a single coprocessor invocation. This call will be spread into each region in the server side. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HBASE-10399) Add documentation for VerifyReplication to refguide
[ https://issues.apache.org/jira/browse/HBASE-10399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13882091#comment-13882091 ] Ted Yu commented on HBASE-10399: bq. You going to do it? I can do this if Gustavo don't have time. Add documentation for VerifyReplication to refguide --- Key: HBASE-10399 URL: https://issues.apache.org/jira/browse/HBASE-10399 Project: HBase Issue Type: Improvement Reporter: Ted Yu Assignee: Gustavo Anatoly Priority: Minor HBase refguide currently doesn't document how VerifyReplication is used for comparing local table with remote table. Document for VerifyReplication should be added so that users know how to use it. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HBASE-10402) [docs] Add docs on how to create and tag a point release.
[ https://issues.apache.org/jira/browse/HBASE-10402?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13882092#comment-13882092 ] stack commented on HBASE-10402: --- +1 Nice. [docs] Add docs on how to create and tag a point release. - Key: HBASE-10402 URL: https://issues.apache.org/jira/browse/HBASE-10402 Project: HBase Issue Type: Bug Components: documentation Reporter: Jonathan Hsieh Assignee: Jonathan Hsieh Attachments: book.html, hbase-10402.patch I've added a few steps to the How to release section of the ref guide based on what I learned about doing a quickie release. This is what I think I should have done -- if you disagree please suggest how it should be done. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HBASE-10396) The constructor of HBaseAdmin may close the shared HConnection
[
https://issues.apache.org/jira/browse/HBASE-10396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13882093#comment-13882093
]
stack commented on HBASE-10396:
---
[~cuijianwei] Go for it boss.
The constructor of HBaseAdmin may close the shared HConnection
---
Key: HBASE-10396
URL: https://issues.apache.org/jira/browse/HBASE-10396
Project: HBase
Issue Type: Bug
Components: Admin, Client
Affects Versions: 0.94.16
Reporter: cuijianwei
HBaseAdmin has the constructor:
{code}
public HBaseAdmin(Configuration c)
throws MasterNotRunningException, ZooKeeperConnectionException {
this.conf = HBaseConfiguration.create(c);
this.connection = HConnectionManager.getConnection(this.conf);
...
{code}
As shown in above code, HBaseAdmin will get a cached HConnection or create a
new HConnection and use this HConnection to connect to Master. Then,
HBaseAdmin will delete the HConnection when connecting to master fail as
follows:
{code}
while ( true ){
try {
this.connection.getMaster();
return;
} catch (MasterNotRunningException mnre) {
HConnectionManager.deleteStaleConnection(this.connection);
this.connection = HConnectionManager.getConnection(this.conf);
}
{code}
The above code will invoke HConnectionManager#deleteStaleConnection to delete
the HConnection from global HConnection cache. The risk is that the deleted
HConnection might be sharing by other threads, such as HTable or HTablePool.
Then, these threads which sharing the deleted HConnection will get closed
HConnection exception:
{code}
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@61bc59aa
closed
{code}
If users use HTablePool, the situation will become worse because closing
HTable will only return HTable to HTablePool which won't reduce the reference
count of the closed HConnection. Then, the closed HConnection will always be
used before clearing HTablePool. In 0.94, some modules such as Rest server
are using HTablePool, therefore may suffer from this problem.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (HBASE-10156) FSHLog Refactor (WAS - Fix up the HBASE-8755 slowdown when low contention)
[ https://issues.apache.org/jira/browse/HBASE-10156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13882097#comment-13882097 ] Andrew Purtell commented on HBASE-10156: I have looked at a couple of revisions of this patch - it is a fat change that I'm not going to be able to deeply understand ahead of commit. However I've looked at reviews from those who've spent the necessary time (Himanshu, Ram, etc.) and the response to those reviews and this looks ready to try. I'm +1 if one of those guys are also of that opinion, and I see Himanshu is. FSHLog Refactor (WAS - Fix up the HBASE-8755 slowdown when low contention) --- Key: HBASE-10156 URL: https://issues.apache.org/jira/browse/HBASE-10156 Project: HBase Issue Type: Sub-task Components: wal Reporter: stack Assignee: stack Attachments: 10156.txt, 10156v10.txt, 10156v11.txt, 10156v12.txt, 10156v12.txt, 10156v13.txt, 10156v16.txt, 10156v17.txt, 10156v18.txt, 10156v19.txt, 10156v2.txt, 10156v20.txt, 10156v20.txt, 10156v21.txt, 10156v21.txt, 10156v3.txt, 10156v4.txt, 10156v5.txt, 10156v6.txt, 10156v7.txt, 10156v9.txt, Disrupting.java HBASE-8755 slows our writes when only a few clients. Fix. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HBASE-10156) FSHLog Refactor (WAS - Fix up the HBASE-8755 slowdown when low contention)
[
https://issues.apache.org/jira/browse/HBASE-10156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13882102#comment-13882102
]
Hadoop QA commented on HBASE-10156:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12625221/10156v21.txt
against trunk revision .
ATTACHMENT ID: 12625221
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 30 new
or modified tests.
{color:green}+1 hadoop1.0{color}. The patch compiles against the hadoop
1.0 profile.
{color:green}+1 hadoop1.1{color}. The patch compiles against the hadoop
1.1 profile.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:red}-1 site{color}. The patch appears to cause mvn site goal to
fail.
{color:red}-1 core tests{color}. The patch failed these unit tests:
{color:red}-1 core zombie tests{color}. There are 1 zombie test(s):
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/8516//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8516//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8516//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8516//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8516//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8516//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8516//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8516//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8516//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8516//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/8516//console
This message is automatically generated.
FSHLog Refactor (WAS - Fix up the HBASE-8755 slowdown when low contention)
---
Key: HBASE-10156
URL: https://issues.apache.org/jira/browse/HBASE-10156
Project: HBase
Issue Type: Sub-task
Components: wal
Reporter: stack
Assignee: stack
Attachments: 10156.txt, 10156v10.txt, 10156v11.txt, 10156v12.txt,
10156v12.txt, 10156v13.txt, 10156v16.txt, 10156v17.txt, 10156v18.txt,
10156v19.txt, 10156v2.txt, 10156v20.txt, 10156v20.txt, 10156v21.txt,
10156v21.txt, 10156v3.txt, 10156v4.txt, 10156v5.txt, 10156v6.txt,
10156v7.txt, 10156v9.txt, Disrupting.java
HBASE-8755 slows our writes when only a few clients. Fix.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (HBASE-3909) Add dynamic config
[
https://issues.apache.org/jira/browse/HBASE-3909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13882118#comment-13882118
]
Hadoop QA commented on HBASE-3909:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12624975/HBASE-3909-backport-from-fb-for-trunk-2.patch
against trunk revision .
ATTACHMENT ID: 12624975
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified tests.
{color:green}+1 hadoop1.0{color}. The patch compiles against the hadoop
1.0 profile.
{color:green}+1 hadoop1.1{color}. The patch compiles against the hadoop
1.1 profile.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 1
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 lineLengths{color}. The patch introduces the following lines
longer than 100:
+private UpdateConfigurationRequest(boolean noInit) {
this.unknownFields = com.google.protobuf.UnknownFieldSet.getDefaultInstance(); }
+private UpdateConfigurationResponse(boolean noInit) { this.unknownFields =
com.google.protobuf.UnknownFieldSet.getDefaultInstance(); }
+ * coderpc UpdateConfiguration(.UpdateConfigurationRequest) returns
(.UpdateConfigurationResponse);/code
+ * coderpc UpdateConfiguration(.UpdateConfigurationRequest) returns
(.UpdateConfigurationResponse);/code
+private UpdateConfigurationRequest(boolean noInit) { this.unknownFields =
com.google.protobuf.UnknownFieldSet.getDefaultInstance(); }
+private UpdateConfigurationResponse(boolean noInit) { this.unknownFields =
com.google.protobuf.UnknownFieldSet.getDefaultInstance(); }
+ * coderpc UpdateConfiguration(.UpdateConfigurationRequest) returns
(.UpdateConfigurationResponse);/code
+ * coderpc UpdateConfiguration(.UpdateConfigurationRequest) returns
(.UpdateConfigurationResponse);/code
{color:red}-1 site{color}. The patch appears to cause mvn site goal to
fail.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/8517//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8517//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8517//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8517//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8517//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8517//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8517//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8517//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8517//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8517//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/8517//console
This message is automatically generated.
Add dynamic config
--
Key: HBASE-3909
URL: https://issues.apache.org/jira/browse/HBASE-3909
Project: HBase
Issue Type: New Feature
Reporter: stack
Assignee: Subbu M Iyer
Attachments: 3909-102812.patch, 3909-102912.patch, 3909-v1.patch,
3909.v1, 3909_090712-2.patch, HBASE-3909-backport-from-fb-for-trunk-2.patch,
HBASE-3909-backport-from-fb-for-trunk.patch, HBase Cluster Config
Details.xlsx, patch-v2.patch, testMasterNoCluster.stack
I'm sure this issue exists already, at least as part of the discussion around
making online schema edits possible, but no hard this having its own issue.
Ted started a conversation on this topic up on dev and Todd suggested we
lookd at how Hadoop did it over in HADOOP-7001
--
This
[jira] [Commented] (HBASE-10420) Replace KV.getBuffer with KV.get{Row|Family|Qualifier|Value|Tags}Array
[
https://issues.apache.org/jira/browse/HBASE-10420?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13882124#comment-13882124
]
Hudson commented on HBASE-10420:
SUCCESS: Integrated in HBase-TRUNK-on-Hadoop-1.1 #64 (See
[https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-1.1/64/])
HBASE-10420 Replace KV.getBuffer with
KV.get{Row|Family|Qualifier|Value|Tags}Array. (larsh: rev 1561410)
*
/hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/client/Increment.java
*
/hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/filter/SingleColumnValueFilter.java
*
/hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/security/access/TablePermission.java
* /hbase/trunk/hbase-common/src/main/java/org/apache/hadoop/hbase/KeyValue.java
*
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileWriterV2.java
*
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileWriterV3.java
*
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/GetClosestRowBeforeTracker.java
*
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
*
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStore.java
*
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/MemStore.java
*
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/ReversedKeyValueHeap.java
*
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/ScanQueryMatcher.java
*
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFile.java
*
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFileScanner.java
*
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreScanner.java
*
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java
*
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/security/visibility/VisibilityController.java
*
/hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/coprocessor/SampleRegionWALObserver.java
*
/hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/io/TestHalfStoreFileReader.java
*
/hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/io/encoding/TestPrefixTreeEncoding.java
*
/hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFileBlock.java
*
/hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFileWriterV3.java
*
/hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestMultiColumnScanner.java
*
/hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestQueryMatcher.java
*
/hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestStoreFile.java
Replace KV.getBuffer with KV.get{Row|Family|Qualifier|Value|Tags}Array
--
Key: HBASE-10420
URL: https://issues.apache.org/jira/browse/HBASE-10420
Project: HBase
Issue Type: Sub-task
Reporter: Lars Hofhansl
Assignee: Lars Hofhansl
Fix For: 0.99.0
Attachments: 7320-simple.txt
We we start by removing KeyValue.getBuffer() in all obvious places.
Obvious places are where we access the buffer by getRowOffset() and
getRowLength(), etc, since we then now we're cross a
row/family/qualifier/value/tags boundary.
The attached patch does that.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (HBASE-10407) String Format Exception
[
https://issues.apache.org/jira/browse/HBASE-10407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13882126#comment-13882126
]
Hadoop QA commented on HBASE-10407:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12625224/hbase.patch
against trunk revision .
ATTACHMENT ID: 12625224
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 hadoop1.0{color}. The patch compiles against the hadoop
1.0 profile.
{color:green}+1 hadoop1.1{color}. The patch compiles against the hadoop
1.1 profile.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:red}-1 site{color}. The patch appears to cause mvn site goal to
fail.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/8518//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8518//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8518//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8518//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8518//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8518//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8518//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8518//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8518//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8518//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/8518//console
This message is automatically generated.
String Format Exception
---
Key: HBASE-10407
URL: https://issues.apache.org/jira/browse/HBASE-10407
Project: HBase
Issue Type: Bug
Components: hadoop2
Affects Versions: 0.99.0
Reporter: Siwakorn Srisakaokul
Priority: Trivial
Attachments: hbase.patch, hbase.patch
There is a bug in
{{hbase/hbase-hadoop2-compat/src/main/java/org/apache/hadoop/metrics2/lib/MetricMutableQuantiles.java:91/92}}.
When the parameter {{name}} contains a %, it will cause a format exception
by String.Format(...).
*Here is a simple patch to fix it*
{code:title=hbase.patch|borderStyle=solid}
From 43a4b247f5ac69f57264b2c1b20dcca8205514c4 Mon Sep 17 00:00:00 2001
From: Siwakorn Srisakaokul sping...@gmail.com
Date: Wed, 22 Jan 2014 10:09:26 -0800
Subject: [PATCH] Format Patch
---
.../org/apache/hadoop/metrics2/lib/MetricMutableQuantiles.java | 10
--
1 file changed, 4 insertions(+), 6 deletions(-)
diff --git
a/hbase-hadoop2-compat/src/main/java/org/apache/hadoop/metrics2/lib/MetricMutableQuantiles.java
b/hbase-hadoop2-compat/src/main/java/org/apache/hadoop/metrics2/lib/MetricMutableQuantiles.java
index b01f88b..c399323 100644
---
a/hbase-hadoop2-compat/src/main/java/org/apache/hadoop/metrics2/lib/MetricMutableQuantiles.java
+++
b/hbase-hadoop2-compat/src/main/java/org/apache/hadoop/metrics2/lib/MetricMutableQuantiles.java
@@ -82,14 +82,12 @@ public class MetricMutableQuantiles extends MutableMetric
implements MetricHisto
Number of %s for %s with %ds interval, lsName, desc, interval));
// Construct the MetricsInfos for the
[jira] [Updated] (HBASE-10156) FSHLog Refactor (WAS - Fix up the HBASE-8755 slowdown when low contention)
[ https://issues.apache.org/jira/browse/HBASE-10156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-10156: -- Attachment: 10156v21.txt See if the zombie a fluke FSHLog Refactor (WAS - Fix up the HBASE-8755 slowdown when low contention) --- Key: HBASE-10156 URL: https://issues.apache.org/jira/browse/HBASE-10156 Project: HBase Issue Type: Sub-task Components: wal Reporter: stack Assignee: stack Attachments: 10156.txt, 10156v10.txt, 10156v11.txt, 10156v12.txt, 10156v12.txt, 10156v13.txt, 10156v16.txt, 10156v17.txt, 10156v18.txt, 10156v19.txt, 10156v2.txt, 10156v20.txt, 10156v20.txt, 10156v21.txt, 10156v21.txt, 10156v21.txt, 10156v3.txt, 10156v4.txt, 10156v5.txt, 10156v6.txt, 10156v7.txt, 10156v9.txt, Disrupting.java HBASE-8755 slows our writes when only a few clients. Fix. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HBASE-10169) Batch coprocessor
[
https://issues.apache.org/jira/browse/HBASE-10169?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13882137#comment-13882137
]
Hadoop QA commented on HBASE-10169:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12625225/HBASE-10169-V3.patch
against trunk revision .
ATTACHMENT ID: 12625225
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 20 new
or modified tests.
{color:green}+1 hadoop1.0{color}. The patch compiles against the hadoop
1.0 profile.
{color:green}+1 hadoop1.1{color}. The patch compiles against the hadoop
1.1 profile.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 7
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:red}-1 findbugs{color}. The patch appears to introduce 1 new
Findbugs (version 1.3.9) warnings.
{color:red}-1 release audit{color}. The applied patch generated 2 release
audit warnings (more than the trunk's current 0 warnings).
{color:red}-1 lineLengths{color}. The patch introduces the following lines
longer than 100:
+MapServerName, Mapbyte[], byte[] serverRegions = new
HashMapServerName, Mapbyte[], byte[]();
+final RegionServerCoprocessorRpcInvoker.BatchErrors errors = new
RegionServerCoprocessorRpcInvoker.BatchErrors();
+ * the invocations to the same region server will be batched into one call.
The coprocessor service is
+ * the invocations to the same region server will be batched into one call.
The coprocessor service is
+ * {@link
org.apache.hadoop.hbase.client.coprocessor.Batch.Callback#update(byte[],
byte[], Object)}
+final ListClientProtos.CoprocessorServiceRequest multiRequests = new
ArrayListClientProtos.CoprocessorServiceRequest();
+MultiRegionServerCallableMultiCoprocessorServiceResponse callable = new
MultiRegionServerCallableMultiCoprocessorServiceResponse(
+for (CoprocessorServiceResponseOrException responseOrException :
coprocessorResponseOrExceptions) {
+private
CoprocessorServiceResponseOrException(com.google.protobuf.GeneratedMessage.Builder?
builder) {
+private CoprocessorServiceResponseOrException(boolean noInit) {
this.unknownFields = com.google.protobuf.UnknownFieldSet.getDefaultInstance(); }
{color:red}-1 site{color}. The patch appears to cause mvn site goal to
fail.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/8519//testReport/
Release audit warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8519//artifact/trunk/patchprocess/patchReleaseAuditProblems.txt
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8519//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8519//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8519//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8519//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8519//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8519//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8519//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8519//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8519//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/8519//console
This message is automatically generated.
Batch coprocessor
-
Key: HBASE-10169
URL: https://issues.apache.org/jira/browse/HBASE-10169
Project: HBase
Issue Type: Sub-task
Components: Coprocessors
Affects Versions: 0.99.0
Reporter: Jingcheng Du
Assignee: Jingcheng Du
Attachments: Batch Coprocessor Design Document.docx,
HBASE-10169-V2.patch, HBASE-10169-V3.patch, HBASE-10169-V3.patch,
HBASE-10169.patch
This is designed to improve the coprocessor invocation in the
[jira] [Commented] (HBASE-8751) Enable peer cluster to choose/change the ColumnFamilies/Tables it really want to replicate from a source cluster
[
https://issues.apache.org/jira/browse/HBASE-8751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13882138#comment-13882138
]
Jean-Daniel Cryans commented on HBASE-8751:
---
Quick review to get things started.
bq. MiniHBaseCluster master = utility1.startMiniCluster();
This should be part of setUpBeforeClass and the shutdown should be part of
tearDownAfterClass.
bq. htab1A.setWriteBufferSize(1024);
I don't see where you are setting setAutoFlush(false) so you can remove all
those lines.
bq. putAndWaitWithFamily(row1, f1Name, val, htab1A, htab3A);
val is always being passed here, just use it directly in putAndWaitWithFamily?
bq. checkRowWithFamily(row1, f1Name, 0, htab2A);
Same kind of thing, the count is always 0. Rename it to
ensureRowNotReplicated or something like that and don't pass a count.
bq. # Copyright 2010 The Apache Software Foundation
We don't put the year anymore.
bq. hbase set_peer_tableCFs '2', table1; table2:cf1,cf2; table3:cfA,cfB
The tableCFs syntax will need to be documented better. It's not obvious that
all of table1's families are replicated to '2'.
bq. if (!this.getAllPeerIds().contains(id)) {
I think peerExists(id) would be better here. Same for the other places where
getAllPeerIds().contains is called.
{quote}
public MapString, ListString getTableCFs(String id) {
if (!this.peerClusters.containsKey(id)) {
throw new IllegalArgumentException(peer with id = + id + is not
connected);
{quote}
I don't think throwing an IllegalArgumentException is good here, it's called
from removeNonReplicableEdits() which is not something a user would call so
we'll have an unchecked exception bubbling up in the ReplicationSource.
bq. private void readTableCFsZnode() {
This method needs its own unit test.
Enable peer cluster to choose/change the ColumnFamilies/Tables it really want
to replicate from a source cluster
Key: HBASE-8751
URL: https://issues.apache.org/jira/browse/HBASE-8751
Project: HBase
Issue Type: New Feature
Components: Replication
Reporter: Feng Honghua
Assignee: Feng Honghua
Attachments: HBASE-8751-0.94-V0.patch, HBASE-8751-0.94-v1.patch,
HBASE-8751-trunk_v0.patch
Consider scenarios (all cf are with replication-scope=1):
1) cluster S has 3 tables, table A has cfA,cfB, table B has cfX,cfY, table C
has cf1,cf2.
2) cluster X wants to replicate table A : cfA, table B : cfX and table C from
cluster S.
3) cluster Y wants to replicate table B : cfY, table C : cf2 from cluster S.
Current replication implementation can't achieve this since it'll push the
data of all the replicatable column-families from cluster S to all its peers,
X/Y in this scenario.
This improvement provides a fine-grained replication theme which enable peer
cluster to choose the column-families/tables they really want from the source
cluster:
A). Set the table:cf-list for a peer when addPeer:
hbase-shell add_peer '3', zk:1100:/hbase, table1; table2:cf1,cf2;
table3:cf2
B). View the table:cf-list config for a peer using show_peer_tableCFs:
hbase-shell show_peer_tableCFs 1
C). Change/set the table:cf-list for a peer using set_peer_tableCFs:
hbase-shell set_peer_tableCFs '2', table1:cfX; table2:cf1; table3:cf1,cf2
In this theme, replication-scope=1 only means a column-family CAN be
replicated to other clusters, but only the 'table:cf-list list' determines
WHICH cf/table will actually be replicated to a specific peer.
To provide back-compatibility, empty 'table:cf-list list' will replicate all
replicatable cf/table. (this means we don't allow a peer which replicates
nothing from a source cluster, we think it's reasonable: if replicating
nothing why bother adding a peer?)
This improvement addresses the exact problem raised by the first FAQ in
http://hbase.apache.org/replication.html:
GLOBAL means replicate? Any provision to replicate only to cluster X and
not to cluster Y? or is that for later?
Yes, this is for much later.
I also noticed somebody mentioned replication-scope as integer rather than
a boolean is for such fine-grained replication purpose, but I think extending
replication-scope can't achieve the same replication granularity
flexibility as providing above per-peer replication configurations.
This improvement has been running smoothly in our production clusters
(Xiaomi) for several months.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (HBASE-6048) Table Scan is failing if offheap cache enabled
[
https://issues.apache.org/jira/browse/HBASE-6048?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13882139#comment-13882139
]
chunhui shen commented on HBASE-6048:
-
0.96 won't have this issue since SchemaMetrics is not existed.
Table Scan is failing if offheap cache enabled
--
Key: HBASE-6048
URL: https://issues.apache.org/jira/browse/HBASE-6048
Project: HBase
Issue Type: Bug
Affects Versions: 0.94.0
Reporter: Gopinathan A
Assignee: ramkrishna.s.vasudevan
Attachments: hbase-6048-94.patch
Table Scan is failing if offheap cache enabled.
{noformat}
2012-05-18 20:03:38,446 DEBUG org.apache.hadoop.hbase.io.hfile.HFileWriterV2:
Initialized with CacheConfig:enabled [cacheDataOnRead=true]
[cacheDataOnWrite=false] [cacheIndexesOnWrite=false]
[cacheBloomsOnWrite=false] [cacheEvictOnClose=false] [cacheCompressed=false]
2012-05-18 20:03:38,446 INFO org.apache.hadoop.hbase.regionserver.StoreFile:
Delete Family Bloom filter type for
hdfs://10.18.40.217:9000/hbase/ufdr/1d4656fd417a07c9171a38b8f4d08510/.tmp/03742024b28f443bb63cfc338d4ca422:
CompoundBloomFilterWriter
2012-05-18 20:04:25,576 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache:
Block cache LRU eviction started; Attempting to free 120.57 MB of
total=1020.57 MB
2012-05-18 20:04:25,655 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache:
Block cache LRU eviction completed; freed=120.82 MB, total=907.89 MB,
single=1012.11 MB, multi=6.12 MB, memory=0 KB
2012-05-18 20:04:25,733 ERROR
org.apache.hadoop.hbase.regionserver.HRegionServer: Failed openScanner
java.lang.IllegalStateException: Schema metrics requested before table/CF
name initialization: {tableName:null,cfName:null}
at
org.apache.hadoop.hbase.regionserver.metrics.SchemaConfigured.getSchemaMetrics(SchemaConfigured.java:182)
at
org.apache.hadoop.hbase.io.hfile.LruBlockCache.updateSizeMetrics(LruBlockCache.java:310)
at
org.apache.hadoop.hbase.io.hfile.LruBlockCache.cacheBlock(LruBlockCache.java:274)
at
org.apache.hadoop.hbase.io.hfile.LruBlockCache.cacheBlock(LruBlockCache.java:293)
at
org.apache.hadoop.hbase.io.hfile.DoubleBlockCache.getBlock(DoubleBlockCache.java:102)
at
org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:296)
at
org.apache.hadoop.hbase.io.hfile.HFileBlockIndex$BlockIndexReader.seekToDataBlock(HFileBlockIndex.java:213)
at
org.apache.hadoop.hbase.io.hfile.HFileReaderV2$AbstractScannerV2.seekTo(HFileReaderV2.java:455)
at
org.apache.hadoop.hbase.io.hfile.HFileReaderV2$AbstractScannerV2.seekTo(HFileReaderV2.java:475)
at
org.apache.hadoop.hbase.regionserver.StoreFileScanner.seekAtOrAfter(StoreFileScanner.java:226)
at
org.apache.hadoop.hbase.regionserver.StoreFileScanner.seek(StoreFileScanner.java:145)
at
org.apache.hadoop.hbase.regionserver.StoreScanner.init(StoreScanner.java:130)
at
org.apache.hadoop.hbase.regionserver.Store.getScanner(Store.java:2001)
at
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.init(HRegion.java:3274)
at
org.apache.hadoop.hbase.regionserver.HRegion.instantiateRegionScanner(HRegion.java:1604)
at
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:1596)
at
org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:1572)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.openScanner(HRegionServer.java:2310)
at sun.reflect.GeneratedMethodAccessor45.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364)
at
org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1376)
2012-05-18 20:04:25,828 ERROR
org.apache.hadoop.hbase.regionserver.HRegionServer: Failed openScanner
{noformat}
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Resolved] (HBASE-7710) hbase.apache.org should host javadocs for previous releases
[ https://issues.apache.org/jira/browse/HBASE-7710?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk resolved HBASE-7710. - Resolution: Implemented Release Note: 0.94 docs are there. hbase.apache.org should host javadocs for previous releases --- Key: HBASE-7710 URL: https://issues.apache.org/jira/browse/HBASE-7710 Project: HBase Issue Type: Wish Components: build Reporter: Nick Dimiduk Priority: Minor We should be hosting javadocs for all released versions of HBase that we as a community intend to support in the wild. The current and previous major releases seem entirely reasonable to me; I propose 0.92.0 and onward. This is in addition to whatever (nightly?) process produces docs for trunk. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HBASE-8751) Enable peer cluster to choose/change the ColumnFamilies/Tables it really want to replicate from a source cluster
[ https://issues.apache.org/jira/browse/HBASE-8751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13882143#comment-13882143 ] Feng Honghua commented on HBASE-8751: - [~jdcryans], thanks very much for the review, I'll make a new patch accordingly, :-) Enable peer cluster to choose/change the ColumnFamilies/Tables it really want to replicate from a source cluster Key: HBASE-8751 URL: https://issues.apache.org/jira/browse/HBASE-8751 Project: HBase Issue Type: New Feature Components: Replication Reporter: Feng Honghua Assignee: Feng Honghua Attachments: HBASE-8751-0.94-V0.patch, HBASE-8751-0.94-v1.patch, HBASE-8751-trunk_v0.patch Consider scenarios (all cf are with replication-scope=1): 1) cluster S has 3 tables, table A has cfA,cfB, table B has cfX,cfY, table C has cf1,cf2. 2) cluster X wants to replicate table A : cfA, table B : cfX and table C from cluster S. 3) cluster Y wants to replicate table B : cfY, table C : cf2 from cluster S. Current replication implementation can't achieve this since it'll push the data of all the replicatable column-families from cluster S to all its peers, X/Y in this scenario. This improvement provides a fine-grained replication theme which enable peer cluster to choose the column-families/tables they really want from the source cluster: A). Set the table:cf-list for a peer when addPeer: hbase-shell add_peer '3', zk:1100:/hbase, table1; table2:cf1,cf2; table3:cf2 B). View the table:cf-list config for a peer using show_peer_tableCFs: hbase-shell show_peer_tableCFs 1 C). Change/set the table:cf-list for a peer using set_peer_tableCFs: hbase-shell set_peer_tableCFs '2', table1:cfX; table2:cf1; table3:cf1,cf2 In this theme, replication-scope=1 only means a column-family CAN be replicated to other clusters, but only the 'table:cf-list list' determines WHICH cf/table will actually be replicated to a specific peer. To provide back-compatibility, empty 'table:cf-list list' will replicate all replicatable cf/table. (this means we don't allow a peer which replicates nothing from a source cluster, we think it's reasonable: if replicating nothing why bother adding a peer?) This improvement addresses the exact problem raised by the first FAQ in http://hbase.apache.org/replication.html: GLOBAL means replicate? Any provision to replicate only to cluster X and not to cluster Y? or is that for later? Yes, this is for much later. I also noticed somebody mentioned replication-scope as integer rather than a boolean is for such fine-grained replication purpose, but I think extending replication-scope can't achieve the same replication granularity flexibility as providing above per-peer replication configurations. This improvement has been running smoothly in our production clusters (Xiaomi) for several months. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HBASE-3909) Add dynamic config
[ https://issues.apache.org/jira/browse/HBASE-3909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13882144#comment-13882144 ] binlijin commented on HBASE-3909: - Add to rb for review, https://reviews.apache.org/r/17369/ . [~stack] bq. There is a hole in that the admin client does not know how to talk to backup master which we should fix in general but it should not get in the way of this patch. Yes, this patch misses backup master, we can fix it with other patch. bq. Apply this as is and then add configs as we go? Needs a bit of doc. More work will be done, for example doc and add ruby command. bq. One concern I'd have is that we are reloading the configs but the Server constructors do a bunch of checking and setting on configs. Will these settings be lost when we reload from raw configs on disk. For example, HRS constructor calls FSUtils.setupShortCircuitRead(this.conf). Do these settings persist pas the reload call? Yes, at first these are some settings will be lost, we will not have all settings take effect dynamic. Add dynamic config -- Key: HBASE-3909 URL: https://issues.apache.org/jira/browse/HBASE-3909 Project: HBase Issue Type: New Feature Reporter: stack Assignee: Subbu M Iyer Attachments: 3909-102812.patch, 3909-102912.patch, 3909-v1.patch, 3909.v1, 3909_090712-2.patch, HBASE-3909-backport-from-fb-for-trunk-2.patch, HBASE-3909-backport-from-fb-for-trunk.patch, HBase Cluster Config Details.xlsx, patch-v2.patch, testMasterNoCluster.stack I'm sure this issue exists already, at least as part of the discussion around making online schema edits possible, but no hard this having its own issue. Ted started a conversation on this topic up on dev and Todd suggested we lookd at how Hadoop did it over in HADOOP-7001 -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HBASE-10403) Simplify offheap cache configuration
[
https://issues.apache.org/jira/browse/HBASE-10403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13882146#comment-13882146
]
chunhui shen commented on HBASE-10403:
--
As my personal thought, I'd like to fix the usage of BlockCache, Two much
combination would confuse the general users.
In current configuration implementation with patch, user should config the
class name of hbase.block.cache.impl. For me, class name is hard to remember.
Also setting it as 'CombinedBlockCache' or 'DoubleBlockCache' seems unfamiliar.
So, should we use the 'type' instead of 'class name', just like Compression
algorithm and DataBlockEncoding algorithm.
For example,
hbase.block.cache.type='LruCache', default value, use LruBlockCache.
hbase.block.cache.type='BucketCache', according to the value of
'hbase.bucketcache.ioengine', construct the BlockCache.('heap' or 'offheap'
use CombinedBlockCache, 'file' use BucketCache directly as the L2 cache)
hbase.block.cache.type='SlabCache', use DoubleBlockCache.
Finally, I think the patch is quite well now. If we could set an familiar type
value for BlockCache instead of detail class name, it will be better if I'm a
general user.
Simplify offheap cache configuration
Key: HBASE-10403
URL: https://issues.apache.org/jira/browse/HBASE-10403
Project: HBase
Issue Type: Bug
Components: io
Reporter: Nick Dimiduk
Assignee: Nick Dimiduk
Priority: Minor
Attachments: HBASE-10403.0.patch
The BucketCache (HBASE-7404) is a very nice piece of functionality which is
hidden behind complex configuration. Enabling it currently requires manual
calculation of L1 cache. It'd be nice to make this easier to use and conform
better with the existing heap management tools we already have.
Turning it on currently requires explicitly setting LruBlockCache (L1)
instance size and IOEngine (L2) size, making sure that L1 size isn't too big
vs global memstore and total heap. This is further confused by
hbase.bucketcache.size accepting a percentage of total heap OR explicit size
in MB. Enabling SlabCache is slightly easier in that it just accepts whatever
LruBlockCache is provided.
Turning on BucketCache using off-heap mode could look like:
hbase-env.sh:
Set HBASE_REGIONSERVER_OPTS:
-Xmx5000m
-XX:MaxDirectMemorySize=15000m
hbase-site.xml:
- hbase.regionserver.global.memstore.size = 0.7
- hbase.regionserver.onheap.blockcache.size = 0.1
- hbase.regionserver.blockcache.impl = BucketCache
- hbase.bucketcache.ioengine = offheap
The result being a CombinedCache instance with 500m LruBlockCache + 15000m
ByteBufferIOEngine running in direct mode.
This example does a couple things (mostly for the admin):
- knows NOT to enable SlabCache
- s/hfile.block.cache.size/hbase.regionserver.onheap.blockcache.size/
- maintains the validity of HBaseConfiguration's existing check that global
MemStore + LruBlockCache == 0.8
- maps BucketCache into meaning a CombinedCache instance with these
implementations for L1 and L2.
- Figures out appropriate values for hbase.bucketcache.size and
hbase.bucketcache.percentage.in.combinedcache
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Updated] (HBASE-10392) Correct references to hbase.regionserver.global.memstore.upperLimit
[
https://issues.apache.org/jira/browse/HBASE-10392?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nick Dimiduk updated HBASE-10392:
-
Attachment: HBASE-10392.3.patch
You're correct; that was depending on the Hadoop2 API. Now the logic in the two
places is identical. Thanks.
Correct references to hbase.regionserver.global.memstore.upperLimit
---
Key: HBASE-10392
URL: https://issues.apache.org/jira/browse/HBASE-10392
Project: HBase
Issue Type: Bug
Reporter: Nick Dimiduk
Assignee: Nick Dimiduk
Fix For: 0.99.0
Attachments: HBASE-10392.0.patch, HBASE-10392.1.patch,
HBASE-10392.2.patch, HBASE-10392.3.patch
As part of the awesome new HBASE-5349, a couple references to
{{hbase.regionserver.global.memstore.upperLimit}} was missed. Clean those up
to use the new {{hbase.regionserver.global.memstore.size}} instead.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Resolved] (HBASE-8047) Delete or document GetJavaProperty
[ https://issues.apache.org/jira/browse/HBASE-8047?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk resolved HBASE-8047. - Resolution: Invalid Delete or document GetJavaProperty -- Key: HBASE-8047 URL: https://issues.apache.org/jira/browse/HBASE-8047 Project: HBase Issue Type: Improvement Components: util Reporter: Nick Dimiduk Priority: Trivial Does anyone use this? Delete it or document it in book. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
