[jira] [Commented] (HBASE-12404) Task 5 from parent: Replace internal HTable constructor use with HConnection#getTable (0.98, 0.99)
[
https://issues.apache.org/jira/browse/HBASE-12404?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14224172#comment-14224172
]
Hadoop QA commented on HBASE-12404:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12683489/12404v20.txt
against master branch at commit e83082a88816684714d8a563967046e582f9b8c7.
ATTACHMENT ID: 12683489
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 135
new or modified tests.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 checkstyle{color}. The applied patch does not increase the
total number of checkstyle errors
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 lineLengths{color}. The patch introduces the following lines
longer than 100:
+ * a
href=https://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/InterfaceClassification.html;Hadoop
+{@link org.apache.hadoop.hbase.client.Table#coprocessorService(Class, byte[],
byte[], org.apache.hadoop.hbase.client.coprocessor.Batch.Call)}, and
+{@link org.apache.hadoop.hbase.client.Table#coprocessorService(Class, byte[],
byte[], org.apache.hadoop.hbase.client.coprocessor.Batch.Call,
org.apache.hadoop.hbase.client.coprocessor.Batch.Callback)}
+ {@link org.apache.hadoop.hbase.client.Table#coprocessorService(Class,
byte[], byte[], org.apache.hadoop.hbase.client.coprocessor.Batch.Call)}
+ or {@link org.apache.hadoop.hbase.client.Table#coprocessorService(Class,
byte[], byte[], org.apache.hadoop.hbase.client.coprocessor.Batch.Call,
org.apache.hadoop.hbase.client.coprocessor.Batch.Callback)}
+method's argument. Calling {@link
org.apache.hadoop.hbase.client.Table#coprocessorService(Class, byte[], byte[],
org.apache.hadoop.hbase.client.coprocessor.Batch.Call)}
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/11824//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11824//artifact/patchprocess/newPatchFindbugsWarningshbase-rest.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11824//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11824//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11824//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11824//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11824//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11824//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11824//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11824//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11824//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11824//artifact/patchprocess/newPatchFindbugsWarningshbase-annotations.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/11824//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/11824//console
This message is automatically generated.
Task 5 from parent: Replace internal HTable constructor use with
HConnection#getTable (0.98, 0.99)
--
Key: HBASE-12404
URL: https://issues.apache.org/jira/browse/HBASE-12404
Project: HBase
Issue Type: Sub-task
Reporter: stack
Assignee:
[jira] [Updated] (HBASE-12569) Control MaxDirectMemorySize in the same manner as heap size
[ https://issues.apache.org/jira/browse/HBASE-12569?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-12569: -- Resolution: Fixed Release Note: Adds new HEAP_SETTINGS environment variable to ./bin/hbase. Set java on and offheap with this one variable. It equates as follows HEAP_SETTINGS=$JAVA_HEAP_MAX $JAVA_OFFHEAP_MAX Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Pushed to branch-1+ Thanks for the patch [~patrickwhite] Control MaxDirectMemorySize in the same manner as heap size --- Key: HBASE-12569 URL: https://issues.apache.org/jira/browse/HBASE-12569 Project: HBase Issue Type: Improvement Components: scripts Affects Versions: 0.98.7 Environment: CentOS 6 Reporter: Patrick White Assignee: Patrick White Priority: Minor Fix For: 2.0.0, 0.98.9, 0.99.2 Attachments: 0001-HBASE-12569-Update-scripts-to-control-MaxDirectMemor.patch, 0001-HBASE-12569-Update-scripts-to-control-MaxDirectMemor.patch It would make it much easier on us if we could manage MaxDirectMemorySize in the same way as heap. This patch allows that to happen. Note: I have not tested the *.cmd modifications as I don't have a Windows box (at the moment, can test at home) but look like they should work (famous last words). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-12404) Task 5 from parent: Replace internal HTable constructor use with HConnection#getTable (0.98, 0.99)
[ https://issues.apache.org/jira/browse/HBASE-12404?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-12404: -- Attachment: 12404v20.txt Ok. v20 is good. Let me retry to be sure. Will apply this patch if second run passes. Task 5 from parent: Replace internal HTable constructor use with HConnection#getTable (0.98, 0.99) -- Key: HBASE-12404 URL: https://issues.apache.org/jira/browse/HBASE-12404 Project: HBase Issue Type: Sub-task Reporter: stack Assignee: stack Fix For: 0.99.2 Attachments: 0001-HBASE-12404-Task-5-from-parent-Replace-internal-HTab.patch, 12404.txt, 12404.v16.txt, 12404.v16.txt, 12404.v17.txt, 12404v10.txt, 12404v11.txt, 12404v12.txt, 12404v12.txt, 12404v13.txt, 12404v13.txt, 12404v14.txt, 12404v15.txt, 12404v18.txt, 12404v19.txt, 12404v19.txt, 12404v2.txt, 12404v20.txt, 12404v20.txt, 12404v3.txt, 12404v5.txt, 12404v6.txt, 12404v7.txt, 12404v9.txt Do the step 5. from the [~ndimiduk] list in parent issue. Go through src code and change all new HTable to instead be connection.getTable. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12534) Wrong region location cache in client after regions are moved
[
https://issues.apache.org/jira/browse/HBASE-12534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14224180#comment-14224180
]
stack commented on HBASE-12534:
---
[~nkeywal]
bq. So may be this min timeout saves in in the .94 .96 versions (not sure
about .98).
Are you saying keep MIN_RPC_TIMEOUT in 0.94? (and 0.96?). Yeah, we could keep
it and keep it configurable.
Wrong region location cache in client after regions are moved
-
Key: HBASE-12534
URL: https://issues.apache.org/jira/browse/HBASE-12534
Project: HBase
Issue Type: Bug
Affects Versions: 2.0.0
Reporter: Liu Shaohui
Assignee: Liu Shaohui
Priority: Critical
Labels: client
Attachments: HBASE-12534-0.94-v1.diff, HBASE-12534-v1.diff
In our 0.94 hbase cluster, we found that client got wrong region location
cache and did not update it after a region is moved to another regionserver.
The reason is wrong client config and bug in RpcRetryingCaller of hbase
client.
The rpc configs are following:
{code}
hbase.rpc.timeout=1000
hbase.client.pause=200
hbase.client.operation.timeout=1200
{code}
But the client retry number is 3
{code}
hbase.client.retries.number=3
{code}
Assumed that a region is at regionserver A before, and then it is moved to
regionserver B. The client try to make a call to regionserver A and get an
NotServingRegionException. For the rety number is not 1, the region server
location cache is not cleaned. See: RpcRetryingCaller.java#141 and
RegionServerCallable.java#127
{code}
@Override
public void throwable(Throwable t, boolean retrying) {
if (t instanceof SocketTimeoutException ||
} else if (t instanceof NotServingRegionException !retrying) {
// Purge cache entries for this specific region from hbase:meta cache
// since we don't call connect(true) when number of retries is 1.
getConnection().deleteCachedRegionLocation(location);
}
}
{code}
But the call did not retry and throw an SocketTimeoutException for the time
the call will take is larger than the operation timeout.See
RpcRetryingCaller.java#152
{code}
expectedSleep = callable.sleep(pause, tries + 1);
// If, after the planned sleep, there won't be enough time left, we
stop now.
long duration = singleCallDuration(expectedSleep);
if (duration callTimeout) {
String msg = callTimeout= + callTimeout + , callDuration= +
duration +
: + callable.getExceptionMessageAdditionalDetail();
throw (SocketTimeoutException)(new
SocketTimeoutException(msg).initCause(t));
}
{code}
At last, the wrong region location will never be not cleaned up .
[~lhofhansl]
In hbase 0.94, the MIN_RPC_TIMEOUT in singleCallDuration is 2000 in default,
which trigger this bug.
{code}
private long singleCallDuration(final long expectedSleep) {
return (EnvironmentEdgeManager.currentTimeMillis() - this.globalStartTime)
+ MIN_RPC_TIMEOUT + expectedSleep;
}
{code}
But there is risk in master code too.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-10536) ImportTsv should fail fast if any of the column family passed to the job is not present in the table
[
https://issues.apache.org/jira/browse/HBASE-10536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-10536:
--
Attachment: 10536v2.txt
It looks like you need to rebase? Lets see how this does.
ImportTsv should fail fast if any of the column family passed to the job is
not present in the table
Key: HBASE-10536
URL: https://issues.apache.org/jira/browse/HBASE-10536
Project: HBase
Issue Type: Bug
Components: mapreduce
Affects Versions: 0.98.0
Reporter: rajeshbabu
Assignee: denny joseph
Attachments: 10536v2.txt, HBASE-10536.patch, HBASE-10536.patch,
HBASE-10536.patch
While checking 0.98 rc, running bulkload tools. By mistake passed wrong
column family to importtsv. LoadIncrementalHfiles failed with following
exception
{code}
Exception in thread main java.io.IOException: Unmatched family names found:
unmatched family names in HFiles to be bulkloaded: [f1]; valid family names
of table test are: [f]
at
org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.doBulkLoad(LoadIncrementalHFiles.java:241)
at
org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.run(LoadIncrementalHFiles.java:823)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
at
org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.main(LoadIncrementalHFiles.java:828)
{code}
Its better to fail fast if any of the passed column family is not present in
table.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12400) Fix refguide so it does connection#getTable rather than new HTable everywhere.
[ https://issues.apache.org/jira/browse/HBASE-12400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14224218#comment-14224218 ] stack commented on HBASE-12400: --- Thank you for taking this on [~syuanjiang]. In this patch https://issues.apache.org/jira/secure/attachment/12683500/12404v20.txt, find client/package-info.java See how it changes our example code to do the 'new' way. I was thinking of changing places where we have bits of code so there is a 1.0 version, the more prominent, and then the old way of doing it as it is now. I can do this one, np... might take you a good while more time than I since I've had my head in this a while now. Otherwise, ask more questions if not clear. When HBASE-12404 goes in, there will be more examples to pull from. Thanks. Fix refguide so it does connection#getTable rather than new HTable everywhere. -- Key: HBASE-12400 URL: https://issues.apache.org/jira/browse/HBASE-12400 Project: HBase Issue Type: Sub-task Components: documentation Reporter: stack Assignee: Stephen Yuan Jiang Fix For: 0.99.2 The refguide has bits of code in it. The code does 'new HTable' to get a table instance. Rather, it should be promoting the new style where we get a Connection and then do a getTable on it. Ditto for references to 'new HBaseAdmin'. See ConnectionFactory for new style. See also package-info.java in Client for updated example. Misty, if you are game for this one, I can help w/ how it should look. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12546) Validate schema options that require server side class availability
[ https://issues.apache.org/jira/browse/HBASE-12546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14224264#comment-14224264 ] Jiajia Li commented on HBASE-12546: --- hi, [~apurtell], I found that in hbase trunk: (https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java#L1169) there is hcd check, such as check the regionsplitpolicy: (https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java#L1225), but in 0.98 branch, the check is not found: (https://github.com/apache/hbase/blob/0.98/hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java#L1747) , so does the sanityCheckTableDescriptor() function do the schema option validate? please give me some advise, thanks~ Validate schema options that require server side class availability --- Key: HBASE-12546 URL: https://issues.apache.org/jira/browse/HBASE-12546 Project: HBase Issue Type: Bug Reporter: Andrew Purtell Assignee: Jiajia Li Fix For: 2.0.0, 0.98.9, 0.99.2 When processing table create and modification requests we should check the supplied schema options for settings that require mentioned classes to be available from the regionserver classpath (split policies, etc.). If we can't find the class on the classpath when processing the admin request RPC, fail the operation immediately and return an exception rather than allow problems later, such as aborts. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12569) Control MaxDirectMemorySize in the same manner as heap size
[ https://issues.apache.org/jira/browse/HBASE-12569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14224281#comment-14224281 ] Hudson commented on HBASE-12569: SUCCESS: Integrated in HBase-TRUNK #5816 (See [https://builds.apache.org/job/HBase-TRUNK/5816/]) HBASE-12569 Update scripts to control MaxDirectMemorySize via env vars (stack: rev f2be914f73d39d288d40147ff1e582ad5a7989f2) * src/main/docbkx/book.xml * bin/hbase.cmd * conf/hbase-env.cmd * conf/hbase-env.sh * bin/hbase Control MaxDirectMemorySize in the same manner as heap size --- Key: HBASE-12569 URL: https://issues.apache.org/jira/browse/HBASE-12569 Project: HBase Issue Type: Improvement Components: scripts Affects Versions: 0.98.7 Environment: CentOS 6 Reporter: Patrick White Assignee: Patrick White Priority: Minor Fix For: 2.0.0, 0.98.9, 0.99.2 Attachments: 0001-HBASE-12569-Update-scripts-to-control-MaxDirectMemor.patch, 0001-HBASE-12569-Update-scripts-to-control-MaxDirectMemor.patch It would make it much easier on us if we could manage MaxDirectMemorySize in the same way as heap. This patch allows that to happen. Note: I have not tested the *.cmd modifications as I don't have a Windows box (at the moment, can test at home) but look like they should work (famous last words). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12569) Control MaxDirectMemorySize in the same manner as heap size
[ https://issues.apache.org/jira/browse/HBASE-12569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14224295#comment-14224295 ] Hudson commented on HBASE-12569: FAILURE: Integrated in HBase-1.0 #501 (See [https://builds.apache.org/job/HBase-1.0/501/]) HBASE-12569 Update scripts to control MaxDirectMemorySize via env vars (stack: rev 39c67f60310b20ca5ad9c3a1402e1a8f934619b2) * src/main/docbkx/book.xml * conf/hbase-env.cmd * bin/hbase * bin/hbase.cmd * conf/hbase-env.sh Control MaxDirectMemorySize in the same manner as heap size --- Key: HBASE-12569 URL: https://issues.apache.org/jira/browse/HBASE-12569 Project: HBase Issue Type: Improvement Components: scripts Affects Versions: 0.98.7 Environment: CentOS 6 Reporter: Patrick White Assignee: Patrick White Priority: Minor Fix For: 2.0.0, 0.98.9, 0.99.2 Attachments: 0001-HBASE-12569-Update-scripts-to-control-MaxDirectMemor.patch, 0001-HBASE-12569-Update-scripts-to-control-MaxDirectMemor.patch It would make it much easier on us if we could manage MaxDirectMemorySize in the same way as heap. This patch allows that to happen. Note: I have not tested the *.cmd modifications as I don't have a Windows box (at the moment, can test at home) but look like they should work (famous last words). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12558) TestHCM.testClusterStatus Unexpected exception, expectedorg.apache.hadoop.hbase.regionserver.RegionServerStoppedException but wasjunit.framework.AssertionFailedErro
[
https://issues.apache.org/jira/browse/HBASE-12558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14224301#comment-14224301
]
Qiang Tian commented on HBASE-12558:
still looking..(and the UT failure in 11902)
thanks.
TestHCM.testClusterStatus Unexpected exception,
expectedorg.apache.hadoop.hbase.regionserver.RegionServerStoppedException
but wasjunit.framework.AssertionFailedError
-
Key: HBASE-12558
URL: https://issues.apache.org/jira/browse/HBASE-12558
Project: HBase
Issue Type: Bug
Components: test
Reporter: stack
Assignee: stack
Priority: Critical
Fix For: 2.0.0, 0.99.2
Happens for me reliably on mac os x. I looked at fixing it. The listener is
not noticing the publish for whatever reason. Thats where I stopped.
{code}
java.lang.Exception: Unexpected exception,
expectedorg.apache.hadoop.hbase.regionserver.RegionServerStoppedException
but wasjunit.framework.AssertionFailedError
at junit.framework.Assert.fail(Assert.java:57)
at org.apache.hadoop.hbase.Waiter.waitFor(Waiter.java:193)
at
org.apache.hadoop.hbase.HBaseTestingUtility.waitFor(HBaseTestingUtility.java:3537)
at
org.apache.hadoop.hbase.client.TestHCM.testClusterStatus(TestHCM.java:273)
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12404) Task 5 from parent: Replace internal HTable constructor use with HConnection#getTable (0.98, 0.99)
[
https://issues.apache.org/jira/browse/HBASE-12404?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14224308#comment-14224308
]
Hadoop QA commented on HBASE-12404:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12683500/12404v20.txt
against master branch at commit f2be914f73d39d288d40147ff1e582ad5a7989f2.
ATTACHMENT ID: 12683500
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 135
new or modified tests.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 checkstyle{color}. The applied patch does not increase the
total number of checkstyle errors
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 lineLengths{color}. The patch introduces the following lines
longer than 100:
+ * a
href=https://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/InterfaceClassification.html;Hadoop
+{@link org.apache.hadoop.hbase.client.Table#coprocessorService(Class, byte[],
byte[], org.apache.hadoop.hbase.client.coprocessor.Batch.Call)}, and
+{@link org.apache.hadoop.hbase.client.Table#coprocessorService(Class, byte[],
byte[], org.apache.hadoop.hbase.client.coprocessor.Batch.Call,
org.apache.hadoop.hbase.client.coprocessor.Batch.Callback)}
+ {@link org.apache.hadoop.hbase.client.Table#coprocessorService(Class,
byte[], byte[], org.apache.hadoop.hbase.client.coprocessor.Batch.Call)}
+ or {@link org.apache.hadoop.hbase.client.Table#coprocessorService(Class,
byte[], byte[], org.apache.hadoop.hbase.client.coprocessor.Batch.Call,
org.apache.hadoop.hbase.client.coprocessor.Batch.Callback)}
+method's argument. Calling {@link
org.apache.hadoop.hbase.client.Table#coprocessorService(Class, byte[], byte[],
org.apache.hadoop.hbase.client.coprocessor.Batch.Call)}
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/11825//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11825//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11825//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11825//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11825//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11825//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11825//artifact/patchprocess/newPatchFindbugsWarningshbase-rest.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11825//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11825//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11825//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11825//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11825//artifact/patchprocess/newPatchFindbugsWarningshbase-annotations.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/11825//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/11825//console
This message is automatically generated.
Task 5 from parent: Replace internal HTable constructor use with
HConnection#getTable (0.98, 0.99)
--
Key: HBASE-12404
URL: https://issues.apache.org/jira/browse/HBASE-12404
Project: HBase
Issue Type: Sub-task
Reporter: stack
Assignee:
[jira] [Commented] (HBASE-10536) ImportTsv should fail fast if any of the column family passed to the job is not present in the table
[
https://issues.apache.org/jira/browse/HBASE-10536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14224315#comment-14224315
]
Hadoop QA commented on HBASE-10536:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12683502/10536v2.txt
against master branch at commit f2be914f73d39d288d40147ff1e582ad5a7989f2.
ATTACHMENT ID: 12683502
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified tests.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:red}-1 checkstyle{color}. The applied patch generated
3806 checkstyle errors (more than the master's current 3805 errors).
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/11826//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11826//artifact/patchprocess/newPatchFindbugsWarningshbase-rest.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11826//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11826//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11826//artifact/patchprocess/newPatchFindbugsWarningshbase-annotations.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11826//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11826//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11826//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11826//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11826//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11826//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11826//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/11826//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/11826//console
This message is automatically generated.
ImportTsv should fail fast if any of the column family passed to the job is
not present in the table
Key: HBASE-10536
URL: https://issues.apache.org/jira/browse/HBASE-10536
Project: HBase
Issue Type: Bug
Components: mapreduce
Affects Versions: 0.98.0
Reporter: rajeshbabu
Assignee: denny joseph
Attachments: 10536v2.txt, HBASE-10536.patch, HBASE-10536.patch,
HBASE-10536.patch
While checking 0.98 rc, running bulkload tools. By mistake passed wrong
column family to importtsv. LoadIncrementalHfiles failed with following
exception
{code}
Exception in thread main java.io.IOException: Unmatched family names found:
unmatched family names in HFiles to be bulkloaded: [f1]; valid family names
of table test are: [f]
at
org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.doBulkLoad(LoadIncrementalHFiles.java:241)
at
org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.run(LoadIncrementalHFiles.java:823)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
at
[jira] [Commented] (HBASE-10536) ImportTsv should fail fast if any of the column family passed to the job is not present in the table
[
https://issues.apache.org/jira/browse/HBASE-10536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14224421#comment-14224421
]
Ashish Singhi commented on HBASE-10536:
---
minor nits
{code}
+for (String cf : cfSet) {
+ if(table.getTableDescriptor().getFamily(Bytes.toBytes(cf)) == null) {
{code}
Can we extract table.getTableDescriptor() this to a variable and use it instead
of calling this method inside the loop ?
{code}
+ boolean noStrict = Boolean.valueOf(conf.get(NO_STRICT_COL_FAMILY));
{code}
Can we use conf.getBoolean(NO_STRICT_COL_FAMILY, false) ?
{code}
+ for (HColumnDescriptor family :
table.getTableDescriptor().getFamilies())
+familyNames.add(family.getNameAsString());
{code}
Can we enclose the for statement within parenthesis ?
ImportTsv should fail fast if any of the column family passed to the job is
not present in the table
Key: HBASE-10536
URL: https://issues.apache.org/jira/browse/HBASE-10536
Project: HBase
Issue Type: Bug
Components: mapreduce
Affects Versions: 0.98.0
Reporter: rajeshbabu
Assignee: denny joseph
Attachments: 10536v2.txt, HBASE-10536.patch, HBASE-10536.patch,
HBASE-10536.patch
While checking 0.98 rc, running bulkload tools. By mistake passed wrong
column family to importtsv. LoadIncrementalHfiles failed with following
exception
{code}
Exception in thread main java.io.IOException: Unmatched family names found:
unmatched family names in HFiles to be bulkloaded: [f1]; valid family names
of table test are: [f]
at
org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.doBulkLoad(LoadIncrementalHFiles.java:241)
at
org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.run(LoadIncrementalHFiles.java:823)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
at
org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.main(LoadIncrementalHFiles.java:828)
{code}
Its better to fail fast if any of the passed column family is not present in
table.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-10536) ImportTsv should fail fast if any of the column family passed to the job is not present in the table
[
https://issues.apache.org/jira/browse/HBASE-10536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14224440#comment-14224440
]
Ashish Singhi commented on HBASE-10536:
---
Also
{code}
+ String msg =
+ Unmatched family names found: unmatched family names in HFiles
to be bulkloaded:
+ + unmatchedFamilies + ; valid family names of table
+ + Bytes.toString(table.getTableName()) + are:
+ + familyNames + .\n
+ + To disable column family check, use -D +
NO_STRICT_COL_FAMILY + =true.\n ;
{code}
Can we change the error message to something like below or more better ?
Because still the HFiles are not created here.
{code}
+ String msg =
+ Column Families + unmatchedFamilies + specified in +
COLUMNS_CONF_KEY
+ + does not match with any of the table + tableName +
column families
+ + familyNames;
{code}
ImportTsv should fail fast if any of the column family passed to the job is
not present in the table
Key: HBASE-10536
URL: https://issues.apache.org/jira/browse/HBASE-10536
Project: HBase
Issue Type: Bug
Components: mapreduce
Affects Versions: 0.98.0
Reporter: rajeshbabu
Assignee: denny joseph
Attachments: 10536v2.txt, HBASE-10536.patch, HBASE-10536.patch,
HBASE-10536.patch
While checking 0.98 rc, running bulkload tools. By mistake passed wrong
column family to importtsv. LoadIncrementalHfiles failed with following
exception
{code}
Exception in thread main java.io.IOException: Unmatched family names found:
unmatched family names in HFiles to be bulkloaded: [f1]; valid family names
of table test are: [f]
at
org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.doBulkLoad(LoadIncrementalHFiles.java:241)
at
org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.run(LoadIncrementalHFiles.java:823)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
at
org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.main(LoadIncrementalHFiles.java:828)
{code}
Its better to fail fast if any of the passed column family is not present in
table.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-12563) After Starting up mini hbase cluster, Real Configuration of Cluster never set to HBaseTestingUtility
[ https://issues.apache.org/jira/browse/HBASE-12563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Talat UYARER updated HBASE-12563: - Attachment: HBASE-12563-v3.patch I added a test that check master info port. If I close improvements, All test will be failed. BTW MASTER_INFO_PORT does not set anywhere. I overlooked it. Because of that I add again that line. I just removed REGIONSERVER_INFO_PORT After Starting up mini hbase cluster, Real Configuration of Cluster never set to HBaseTestingUtility - Key: HBASE-12563 URL: https://issues.apache.org/jira/browse/HBASE-12563 Project: HBase Issue Type: Bug Affects Versions: 2.0.0, 0.99.1 Reporter: Talat UYARER Assignee: Talat UYARER Attachments: HBASE-12563-v2.patch, HBASE-12563-v3.patch, HBASE-12563.patch When I use startMiniHBaseCluster method. It starts up a Mini Cluster for the tests. While starting It changes some configuration. For example Master's port or Region Server's. After Cluster starting We should set its new configuration to HbaseTestingUtils conf. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12491) TableMapReduceUtil.findContainingJar() NPE
[ https://issues.apache.org/jira/browse/HBASE-12491?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14224723#comment-14224723 ] Solomon Duskis commented on HBASE-12491: [~ndimiduk]: who was that comment addressed to? TableMapReduceUtil.findContainingJar() NPE -- Key: HBASE-12491 URL: https://issues.apache.org/jira/browse/HBASE-12491 Project: HBase Issue Type: Bug Components: mapreduce Affects Versions: 2.0.0, 0.99.2 Reporter: Solomon Duskis Assignee: Solomon Duskis Fix For: 2.0.0, 0.94.25, 0.98.9, 0.99.2 Attachments: HBASE-12491.patch, HBASE-12491.patch Adding a bootclasspath library causes an NPE when running hbase map reduce jobs in TableMapReduceUtil.findContainingJar(). Classes in the library added to the bootclasspath get a null classpathLoader. Check for a null loader in TableMapReduceUtil.findContainingJar(). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12558) TestHCM.testClusterStatus Unexpected exception, expectedorg.apache.hadoop.hbase.regionserver.RegionServerStoppedException but wasjunit.framework.AssertionFailedErro
[
https://issues.apache.org/jira/browse/HBASE-12558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14224725#comment-14224725
]
stack commented on HBASE-12558:
---
Thanks [~tianq] When I was looking at it, the server was publishing the message
but listener never got it.
TestHCM.testClusterStatus Unexpected exception,
expectedorg.apache.hadoop.hbase.regionserver.RegionServerStoppedException
but wasjunit.framework.AssertionFailedError
-
Key: HBASE-12558
URL: https://issues.apache.org/jira/browse/HBASE-12558
Project: HBase
Issue Type: Bug
Components: test
Reporter: stack
Assignee: stack
Priority: Critical
Fix For: 2.0.0, 0.99.2
Happens for me reliably on mac os x. I looked at fixing it. The listener is
not noticing the publish for whatever reason. Thats where I stopped.
{code}
java.lang.Exception: Unexpected exception,
expectedorg.apache.hadoop.hbase.regionserver.RegionServerStoppedException
but wasjunit.framework.AssertionFailedError
at junit.framework.Assert.fail(Assert.java:57)
at org.apache.hadoop.hbase.Waiter.waitFor(Waiter.java:193)
at
org.apache.hadoop.hbase.HBaseTestingUtility.waitFor(HBaseTestingUtility.java:3537)
at
org.apache.hadoop.hbase.client.TestHCM.testClusterStatus(TestHCM.java:273)
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-12490) Replace uses of setAutoFlush(boolean, boolean)
[ https://issues.apache.org/jira/browse/HBASE-12490?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Solomon Duskis updated HBASE-12490: --- Attachment: HBASE-12490B.patch One more time... Replace uses of setAutoFlush(boolean, boolean) -- Key: HBASE-12490 URL: https://issues.apache.org/jira/browse/HBASE-12490 Project: HBase Issue Type: Bug Affects Versions: 2.0.0, 0.99.2 Reporter: Solomon Duskis Assignee: Solomon Duskis Attachments: HBASE-12490.patch, HBASE-12490B.patch, HBASE-12490B.patch, HBASE-12490B.patch The various uses of setAutoFlush() seem to need some tlc. There's a note in HTableInterface: @deprecated in 0.99 since setting clearBufferOnFail is deprecated. Use setAutoFlushTo(boolean) instead. It would be ideal to change all internal uses of setAutoFlush(boolean, boolean) to use setAutoFlushTo, if possible. HTable.setAutoFlush(boolean, boolean) is used in a handful of places. setAutoFlush(false, false) has the same results as HTable.setAutoFlush(false). Calling HTable.setAutoFlush(false, true) has the same affect as Table.setAutoFlushTo(false), assuming HTable.setAutoFlush(false) was not called previously (by default, the second parameter, clearBufferOnFail, is true and should remain true according to the comments). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12490) Replace uses of setAutoFlush(boolean, boolean)
[
https://issues.apache.org/jira/browse/HBASE-12490?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14224752#comment-14224752
]
Hadoop QA commented on HBASE-12490:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12683584/HBASE-12490B.patch
against master branch at commit e6b4300756b7f09a31ba35cb3baf41d294ed6e14.
ATTACHMENT ID: 12683584
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 45 new
or modified tests.
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/11827//console
This message is automatically generated.
Replace uses of setAutoFlush(boolean, boolean)
--
Key: HBASE-12490
URL: https://issues.apache.org/jira/browse/HBASE-12490
Project: HBase
Issue Type: Bug
Affects Versions: 2.0.0, 0.99.2
Reporter: Solomon Duskis
Assignee: Solomon Duskis
Attachments: HBASE-12490.patch, HBASE-12490B.patch,
HBASE-12490B.patch, HBASE-12490B.patch
The various uses of setAutoFlush() seem to need some tlc. There's a note in
HTableInterface: @deprecated in 0.99 since setting clearBufferOnFail is
deprecated. Use setAutoFlushTo(boolean) instead. It would be ideal to
change all internal uses of setAutoFlush(boolean, boolean) to use
setAutoFlushTo, if possible.
HTable.setAutoFlush(boolean, boolean) is used in a handful of places.
setAutoFlush(false, false) has the same results as
HTable.setAutoFlush(false). Calling HTable.setAutoFlush(false, true) has the
same affect as Table.setAutoFlushTo(false), assuming
HTable.setAutoFlush(false) was not called previously (by default, the second
parameter, clearBufferOnFail, is true and should remain true according to the
comments).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-11639) [Visibility controller] Replicate the visibility of Cells as strings
[ https://issues.apache.org/jira/browse/HBASE-11639?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14224848#comment-14224848 ] ramkrishna.s.vasudevan commented on HBASE-11639: Any more reviews appreciated here. [Visibility controller] Replicate the visibility of Cells as strings Key: HBASE-11639 URL: https://issues.apache.org/jira/browse/HBASE-11639 Project: HBase Issue Type: Improvement Components: Replication, security Affects Versions: 0.98.4 Reporter: ramkrishna.s.vasudevan Assignee: ramkrishna.s.vasudevan Labels: VisibilityLabels Fix For: 2.0.0, 0.98.9, 0.99.2 Attachments: HBASE-11639_v11.patch, HBASE-11639_v13.patch, HBASE-11639_v13.patch, HBASE-11639_v14.patch, HBASE-11639_v15.patch, HBASE-11639_v2.patch, HBASE-11639_v2.patch, HBASE-11639_v3.patch, HBASE-11639_v3.patch, HBASE-11639_v5.patch, HBASE-11639_v6.patch, HBASE-11639_v9.patch This issue is aimed at persisting the visibility labels as strings in the WAL rather than Label ordinals. This would help in replicating the label ordinals to the replication cluster as strings directly and also that after HBASE-11553 would help because the replication cluster could have an implementation as string based visibility labels. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-12572) Meta flush hangs
Jimmy Xiang created HBASE-12572: --- Summary: Meta flush hangs Key: HBASE-12572 URL: https://issues.apache.org/jira/browse/HBASE-12572 Project: HBase Issue Type: Bug Affects Versions: 1.0.0 Reporter: Jimmy Xiang Not sure if this is still an issue with the latest branch 1 code. I ran into this with branch 1 commit: 0.99.2-SNAPSHOT, revision=290749fc56d07461441bd532f62d70f562eee588. Jstack shows lots of scanners blocked at close. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-12572) Meta flush hangs
[ https://issues.apache.org/jira/browse/HBASE-12572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jimmy Xiang updated HBASE-12572: Attachment: master.jstack Meta flush hangs Key: HBASE-12572 URL: https://issues.apache.org/jira/browse/HBASE-12572 Project: HBase Issue Type: Bug Affects Versions: 1.0.0 Reporter: Jimmy Xiang Attachments: master.jstack Not sure if this is still an issue with the latest branch 1 code. I ran into this with branch 1 commit: 0.99.2-SNAPSHOT, revision=290749fc56d07461441bd532f62d70f562eee588. Jstack shows lots of scanners blocked at close. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-12490) Replace uses of setAutoFlush(boolean, boolean)
[ https://issues.apache.org/jira/browse/HBASE-12490?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Solomon Duskis updated HBASE-12490: --- Attachment: HBASE-12490C.patch Yet another attempt. Replace uses of setAutoFlush(boolean, boolean) -- Key: HBASE-12490 URL: https://issues.apache.org/jira/browse/HBASE-12490 Project: HBase Issue Type: Bug Affects Versions: 2.0.0, 0.99.2 Reporter: Solomon Duskis Assignee: Solomon Duskis Attachments: HBASE-12490.patch, HBASE-12490B.patch, HBASE-12490B.patch, HBASE-12490B.patch, HBASE-12490C.patch The various uses of setAutoFlush() seem to need some tlc. There's a note in HTableInterface: @deprecated in 0.99 since setting clearBufferOnFail is deprecated. Use setAutoFlushTo(boolean) instead. It would be ideal to change all internal uses of setAutoFlush(boolean, boolean) to use setAutoFlushTo, if possible. HTable.setAutoFlush(boolean, boolean) is used in a handful of places. setAutoFlush(false, false) has the same results as HTable.setAutoFlush(false). Calling HTable.setAutoFlush(false, true) has the same affect as Table.setAutoFlushTo(false), assuming HTable.setAutoFlush(false) was not called previously (by default, the second parameter, clearBufferOnFail, is true and should remain true according to the comments). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-12572) Meta flush hangs
[ https://issues.apache.org/jira/browse/HBASE-12572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jimmy Xiang updated HBASE-12572: Attachment: meta-flushing.png Meta flush hangs Key: HBASE-12572 URL: https://issues.apache.org/jira/browse/HBASE-12572 Project: HBase Issue Type: Bug Affects Versions: 1.0.0 Reporter: Jimmy Xiang Attachments: master.jstack, meta-flushing.png Not sure if this is still an issue with the latest branch 1 code. I ran into this with branch 1 commit: 0.99.2-SNAPSHOT, revision=290749fc56d07461441bd532f62d70f562eee588. Jstack shows lots of scanners blocked at close. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12565) Race condition in HRegion.batchMutate() causes partial data to be written when region closes
[ https://issues.apache.org/jira/browse/HBASE-12565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14224862#comment-14224862 ] Keith David Winkler commented on HBASE-12565: - I am currently working on a patch/fix. Race condition in HRegion.batchMutate() causes partial data to be written when region closes - Key: HBASE-12565 URL: https://issues.apache.org/jira/browse/HBASE-12565 Project: HBase Issue Type: Bug Components: Performance, regionserver Affects Versions: 0.98.6 Reporter: Scott Fines The following sequence of events is possible to occur in HRegion's batchMutate() call: 1. caller attempts to call HRegion.batchMutate() with a batch of N1 records 2. batchMutate acquires region lock in startRegionOperation, then calls doMiniBatchMutation() 3. doMiniBatchMutation acquires one row lock 4. Region closes 5. doMiniBatchMutation attempts to acquire second row lock. When this happens, the lock acquisition will also attempt to acquire the region lock, which fails (because the region is closing). At this stage, doMiniBatchMutation will stop writing further, BUT it WILL write data for the rows whose locks have already been acquired, and advance the index in MiniBatchOperationInProgress. Then, after it terminates successfully, batchMutate() will loop around a second time, and attempt AGAIN to acquire the region closing lock. When that happens, a NotServingRegionException is thrown back to the caller. Thus, we have a race condition where partial data can be written when a region server is closing. The main problem stems from the location of startRegionOperation() calls in batchMutate and doMiniBatchMutation(): 1. batchMutate() reacquires the region lock with each iteration of the loop, which can cause some successful writes to occur, but then fail on others 2. getRowLock() attempts to acquire the region lock once for each row, which allows doMiniBatchMutation to terminate early; this forces batchMutate() to use multiple iterations and results in condition 1 being hit. There appears to be two parts to the solution as well: 1. open an internal path so that doMiniBatchMutation() can acquire row locks without checking for region closure. This will have the added benefit of a significant performance improvement during large batch mutations. 2. move the startRegionOperation() out of the loop in batchMutate() so that multiple iterations of doMiniBatchMutation will not cause the operation to fail. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12490) Replace uses of setAutoFlush(boolean, boolean)
[
https://issues.apache.org/jira/browse/HBASE-12490?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14224866#comment-14224866
]
Nicolas Liochon commented on HBASE-12490:
-
For stuff like:
{code}
-ht.setAutoFlush(false, false);
+ht.setAutoFlush(false);
{code}
It's not a big deal, but I don't really like the 'setAutoFlush(boolean)',
because it looks like a setter while actually it's not. I do prefer
'setAutoFlush(boolean, boolean)' because there is no confusion with a setter,
so it's easier for the reader. The implicit setting of the clearBufferOnFail on
something named like a setter is really confusing imho. I'm not -1, but I'm
-0, if I'm the only one confused here... :-)
Replace uses of setAutoFlush(boolean, boolean)
--
Key: HBASE-12490
URL: https://issues.apache.org/jira/browse/HBASE-12490
Project: HBase
Issue Type: Bug
Affects Versions: 2.0.0, 0.99.2
Reporter: Solomon Duskis
Assignee: Solomon Duskis
Attachments: HBASE-12490.patch, HBASE-12490B.patch,
HBASE-12490B.patch, HBASE-12490B.patch, HBASE-12490C.patch
The various uses of setAutoFlush() seem to need some tlc. There's a note in
HTableInterface: @deprecated in 0.99 since setting clearBufferOnFail is
deprecated. Use setAutoFlushTo(boolean) instead. It would be ideal to
change all internal uses of setAutoFlush(boolean, boolean) to use
setAutoFlushTo, if possible.
HTable.setAutoFlush(boolean, boolean) is used in a handful of places.
setAutoFlush(false, false) has the same results as
HTable.setAutoFlush(false). Calling HTable.setAutoFlush(false, true) has the
same affect as Table.setAutoFlushTo(false), assuming
HTable.setAutoFlush(false) was not called previously (by default, the second
parameter, clearBufferOnFail, is true and should remain true according to the
comments).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-12569) Control MaxDirectMemorySize in the same manner as heap size
[ https://issues.apache.org/jira/browse/HBASE-12569?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Elliott Clark updated HBASE-12569: -- Release Note: Adds new JAVA_OFFHEAP_MAX environment variable to ./bin/hbase. Set the max offheap memory java will request with this one variable. It combines with JAVA_HEAP_MAX as follows HEAP_SETTINGS=$JAVA_HEAP_MAX $JAVA_OFFHEAP_MAX (was: Adds new HEAP_SETTINGS environment variable to ./bin/hbase. Set java on and offheap with this one variable. It equates as follows HEAP_SETTINGS=$JAVA_HEAP_MAX $JAVA_OFFHEAP_MAX) Control MaxDirectMemorySize in the same manner as heap size --- Key: HBASE-12569 URL: https://issues.apache.org/jira/browse/HBASE-12569 Project: HBase Issue Type: Improvement Components: scripts Affects Versions: 0.98.7 Environment: CentOS 6 Reporter: Patrick White Assignee: Patrick White Priority: Minor Fix For: 2.0.0, 0.98.9, 0.99.2 Attachments: 0001-HBASE-12569-Update-scripts-to-control-MaxDirectMemor.patch, 0001-HBASE-12569-Update-scripts-to-control-MaxDirectMemor.patch It would make it much easier on us if we could manage MaxDirectMemorySize in the same way as heap. This patch allows that to happen. Note: I have not tested the *.cmd modifications as I don't have a Windows box (at the moment, can test at home) but look like they should work (famous last words). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12572) Meta flush hangs
[ https://issues.apache.org/jira/browse/HBASE-12572?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14224872#comment-14224872 ] Jimmy Xiang commented on HBASE-12572: - Probably you won't be able to find this commit. It's my local commit to revert surefire to 2.17 (just simple one line pom.xml change). The parent shra is b1f7d7cd32d4c1ea1b9207472dfab6ca257aa800 (HBASE-12448). Meta flush hangs Key: HBASE-12572 URL: https://issues.apache.org/jira/browse/HBASE-12572 Project: HBase Issue Type: Bug Affects Versions: 1.0.0 Reporter: Jimmy Xiang Attachments: master.jstack, meta-flushing.png Not sure if this is still an issue with the latest branch 1 code. I ran into this with branch 1 commit: 0.99.2-SNAPSHOT, revision=290749fc56d07461441bd532f62d70f562eee588. Jstack shows lots of scanners blocked at close. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12490) Replace uses of setAutoFlush(boolean, boolean)
[
https://issues.apache.org/jira/browse/HBASE-12490?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14224873#comment-14224873
]
Hadoop QA commented on HBASE-12490:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12683600/HBASE-12490C.patch
against master branch at commit e6b4300756b7f09a31ba35cb3baf41d294ed6e14.
ATTACHMENT ID: 12683600
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 45 new
or modified tests.
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/11828//console
This message is automatically generated.
Replace uses of setAutoFlush(boolean, boolean)
--
Key: HBASE-12490
URL: https://issues.apache.org/jira/browse/HBASE-12490
Project: HBase
Issue Type: Bug
Affects Versions: 2.0.0, 0.99.2
Reporter: Solomon Duskis
Assignee: Solomon Duskis
Attachments: HBASE-12490.patch, HBASE-12490B.patch,
HBASE-12490B.patch, HBASE-12490B.patch, HBASE-12490C.patch
The various uses of setAutoFlush() seem to need some tlc. There's a note in
HTableInterface: @deprecated in 0.99 since setting clearBufferOnFail is
deprecated. Use setAutoFlushTo(boolean) instead. It would be ideal to
change all internal uses of setAutoFlush(boolean, boolean) to use
setAutoFlushTo, if possible.
HTable.setAutoFlush(boolean, boolean) is used in a handful of places.
setAutoFlush(false, false) has the same results as
HTable.setAutoFlush(false). Calling HTable.setAutoFlush(false, true) has the
same affect as Table.setAutoFlushTo(false), assuming
HTable.setAutoFlush(false) was not called previously (by default, the second
parameter, clearBufferOnFail, is true and should remain true according to the
comments).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-12569) Control MaxDirectMemorySize in the same manner as heap size
[ https://issues.apache.org/jira/browse/HBASE-12569?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Elliott Clark updated HBASE-12569: -- Release Note: Adds new HBASE_OFFHEAPSIZE environment variable to ./bin/hbase. Set the max offheap memory java will request with this one variable. It combines with HBASE_SIZE to determine the max amount of ram that the JVM can request. (was: Adds new JAVA_OFFHEAP_MAX environment variable to ./bin/hbase. Set the max offheap memory java will request with this one variable. It combines with JAVA_HEAP_MAX as follows HEAP_SETTINGS=$JAVA_HEAP_MAX $JAVA_OFFHEAP_MAX) Control MaxDirectMemorySize in the same manner as heap size --- Key: HBASE-12569 URL: https://issues.apache.org/jira/browse/HBASE-12569 Project: HBase Issue Type: Improvement Components: scripts Affects Versions: 0.98.7 Environment: CentOS 6 Reporter: Patrick White Assignee: Patrick White Priority: Minor Fix For: 2.0.0, 0.98.9, 0.99.2 Attachments: 0001-HBASE-12569-Update-scripts-to-control-MaxDirectMemor.patch, 0001-HBASE-12569-Update-scripts-to-control-MaxDirectMemor.patch It would make it much easier on us if we could manage MaxDirectMemorySize in the same way as heap. This patch allows that to happen. Note: I have not tested the *.cmd modifications as I don't have a Windows box (at the moment, can test at home) but look like they should work (famous last words). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12479) Backport HBASE-11689 (Track meta in transition) to 0.98 and branch-1
[ https://issues.apache.org/jira/browse/HBASE-12479?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14224877#comment-14224877 ] Andrew Purtell commented on HBASE-12479: Want to try again [~virag]? Backport HBASE-11689 (Track meta in transition) to 0.98 and branch-1 Key: HBASE-12479 URL: https://issues.apache.org/jira/browse/HBASE-12479 Project: HBase Issue Type: Bug Components: Region Assignment Reporter: Virag Kothari Assignee: Virag Kothari Fix For: 0.98.9, 0.99.2 Attachments: HBASE-12479-0.98.patch, HBASE-12479-0.98_v2.patch, HBASE-12479-branch-1.patch Required for zk-less assignment -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-12569) Control MaxDirectMemorySize in the same manner as heap size
[ https://issues.apache.org/jira/browse/HBASE-12569?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Elliott Clark updated HBASE-12569: -- Release Note: Adds new HBASE_OFFHEAPSIZE environment variable to ./bin/hbase. Set the max offheap memory java will request with this one variable. It combines with HBASE_HEAPSIZE to determine the max amount of ram that the JVM can request. (was: Adds new HBASE_OFFHEAPSIZE environment variable to ./bin/hbase. Set the max offheap memory java will request with this one variable. It combines with HBASE_SIZE to determine the max amount of ram that the JVM can request.) Control MaxDirectMemorySize in the same manner as heap size --- Key: HBASE-12569 URL: https://issues.apache.org/jira/browse/HBASE-12569 Project: HBase Issue Type: Improvement Components: scripts Affects Versions: 0.98.7 Environment: CentOS 6 Reporter: Patrick White Assignee: Patrick White Priority: Minor Fix For: 2.0.0, 0.98.9, 0.99.2 Attachments: 0001-HBASE-12569-Update-scripts-to-control-MaxDirectMemor.patch, 0001-HBASE-12569-Update-scripts-to-control-MaxDirectMemor.patch It would make it much easier on us if we could manage MaxDirectMemorySize in the same way as heap. This patch allows that to happen. Note: I have not tested the *.cmd modifications as I don't have a Windows box (at the moment, can test at home) but look like they should work (famous last words). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12490) Replace uses of setAutoFlush(boolean, boolean)
[ https://issues.apache.org/jira/browse/HBASE-12490?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14224924#comment-14224924 ] Solomon Duskis commented on HBASE-12490: [~nkeywal]: That's a fair point. FWIW, my main objective is to be able to use a method from Table as opposed to a method from HTable. setAutoFlush() in its current state with three different methods is confusing. The current state of clearBufferOnFail being semi-deprecated is confusing. [~ndimiduk] sent an email t the dev group about making a decision on this, and it looks like I was the only one who responded. How can we get consensus on a clean implementation of setAutoFlush()? Replace uses of setAutoFlush(boolean, boolean) -- Key: HBASE-12490 URL: https://issues.apache.org/jira/browse/HBASE-12490 Project: HBase Issue Type: Bug Affects Versions: 2.0.0, 0.99.2 Reporter: Solomon Duskis Assignee: Solomon Duskis Attachments: HBASE-12490.patch, HBASE-12490B.patch, HBASE-12490B.patch, HBASE-12490B.patch, HBASE-12490C.patch The various uses of setAutoFlush() seem to need some tlc. There's a note in HTableInterface: @deprecated in 0.99 since setting clearBufferOnFail is deprecated. Use setAutoFlushTo(boolean) instead. It would be ideal to change all internal uses of setAutoFlush(boolean, boolean) to use setAutoFlushTo, if possible. HTable.setAutoFlush(boolean, boolean) is used in a handful of places. setAutoFlush(false, false) has the same results as HTable.setAutoFlush(false). Calling HTable.setAutoFlush(false, true) has the same affect as Table.setAutoFlushTo(false), assuming HTable.setAutoFlush(false) was not called previously (by default, the second parameter, clearBufferOnFail, is true and should remain true according to the comments). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12404) Task 5 from parent: Replace internal HTable constructor use with HConnection#getTable (0.98, 0.99)
[ https://issues.apache.org/jira/browse/HBASE-12404?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14224927#comment-14224927 ] Hudson commented on HBASE-12404: FAILURE: Integrated in HBase-TRUNK #5817 (See [https://builds.apache.org/job/HBase-TRUNK/5817/]) HBASE-12404 Task 5 from parent: Replace internal HTable constructor use with (stack: rev e6b4300756b7f09a31ba35cb3baf41d294ed6e14) * hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestMasterFailover.java * hbase-server/src/main/java/org/apache/hadoop/hbase/master/handler/CreateTableHandler.java * hbase-client/src/main/java/org/apache/hadoop/hbase/security/visibility/VisibilityClient.java * hbase-client/src/main/java/org/apache/hadoop/hbase/client/coprocessor/AggregationClient.java * hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java * hbase-client/src/main/java/org/apache/hadoop/hbase/MetaTableAccessor.java * hbase-server/src/main/java/org/apache/hadoop/hbase/master/CatalogJanitor.java * hbase-server/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java * hbase-server/src/main/java/org/apache/hadoop/hbase/tool/Canary.java * hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestMaster.java * hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/TableOutputFormat.java * hbase-server/src/test/java/org/apache/hadoop/hbase/backup/example/TestZooKeeperTableArchiveClient.java * hbase-server/src/test/java/org/apache/hadoop/hbase/master/cleaner/TestHFileCleaner.java * hbase-server/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java * hbase-server/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java * hbase-server/src/main/java/org/apache/hadoop/hbase/master/snapshot/TakeSnapshotHandler.java * hbase-server/src/test/java/org/apache/hadoop/hbase/util/hbck/OfflineMetaRebuildTestCore.java * hbase-server/src/main/java/org/apache/hadoop/hbase/mapred/TableMapReduceUtil.java * hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestSplitTransactionOnCluster.java * hbase-client/src/main/java/org/apache/hadoop/hbase/client/ConnectionManager.java * hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestMasterOperationsForRegionReplicas.java * hbase-server/src/main/java/org/apache/hadoop/hbase/master/handler/TableEventHandler.java * hbase-server/src/main/java/org/apache/hadoop/hbase/quotas/QuotaCache.java * hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestSplitLogManager.java * hbase-server/src/main/java/org/apache/hadoop/hbase/master/handler/DeleteTableHandler.java * hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/FavoredNodeLoadBalancer.java * hbase-client/src/main/java/org/apache/hadoop/hbase/client/ConnectionAdapter.java * hbase-server/src/test/java/org/apache/hadoop/hbase/util/MockServer.java * hbase-server/src/test/java/org/apache/hadoop/hbase/util/hbck/TestOfflineMetaRebuildHole.java * hbase-server/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessController.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RegionMergeTransaction.java * hbase-client/src/main/java/org/apache/hadoop/hbase/client/ConnectionUtils.java * hbase-server/src/test/java/org/apache/hadoop/hbase/client/HConnectionTestingUtility.java * hbase-server/src/test/java/org/apache/hadoop/hbase/io/encoding/TestChangingEncoding.java * hbase-server/src/test/java/org/apache/hadoop/hbase/master/MockRegionServer.java * hbase-server/src/test/java/org/apache/hadoop/hbase/util/hbck/TestOfflineMetaRebuildBase.java * hbase-client/src/main/java/org/apache/hadoop/hbase/client/package-info.java * hbase-server/src/test/java/org/apache/hadoop/hbase/replication/TestReplicationStateZKImpl.java * hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionMergeTransactionOnCluster.java * hbase-server/src/test/java/org/apache/hadoop/hbase/coprocessor/TestRegionServerObserver.java * hbase-client/src/main/java/org/apache/hadoop/hbase/client/MetaScanner.java * hbase-server/src/test/java/org/apache/hadoop/hbase/replication/regionserver/TestReplicationSourceManager.java * hbase-server/src/main/java/org/apache/hadoop/hbase/quotas/QuotaUtil.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java * hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/TableInputFormat.java * hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestRestartCluster.java * hbase-server/src/main/java/org/apache/hadoop/hbase/master/handler/MetaServerShutdownHandler.java * hbase-client/src/main/java/org/apache/hadoop/hbase/client/ConnectionFactory.java * hbase-client/src/main/java/org/apache/hadoop/hbase/client/RegistryFactory.java * hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/FavoredNodeAssignmentHelper.java *
[jira] [Commented] (HBASE-12490) Replace uses of setAutoFlush(boolean, boolean)
[ https://issues.apache.org/jira/browse/HBASE-12490?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14224939#comment-14224939 ] Nicolas Liochon commented on HBASE-12490: - Yeah, I saw it but I was ok with you answer so I didn't comment :-) Let's try to decide in this jira (Nick should see it). My point of view is: - we should not change the meaning of setAutoFlush(boolean), as it would be confusing during the upgrade (i.e. someone upgrading from .098 to 1.0 would have its code compiling but with a hidden behavior change) - we should not use setAutoFlush(boolean), may be we should remove it in 1.0. This because of the confusion around it's a setter-like that is not a setter. - I don't think that we need to keep clearBufferOnFail (i.e. we could remove it in 1.0), but may be I'm wrong here. If we do that then we can keep setAutoFlush(boolean), it will become a real setter (and then the points above are not an issue anymore. Replace uses of setAutoFlush(boolean, boolean) -- Key: HBASE-12490 URL: https://issues.apache.org/jira/browse/HBASE-12490 Project: HBase Issue Type: Bug Affects Versions: 2.0.0, 0.99.2 Reporter: Solomon Duskis Assignee: Solomon Duskis Attachments: HBASE-12490.patch, HBASE-12490B.patch, HBASE-12490B.patch, HBASE-12490B.patch, HBASE-12490C.patch The various uses of setAutoFlush() seem to need some tlc. There's a note in HTableInterface: @deprecated in 0.99 since setting clearBufferOnFail is deprecated. Use setAutoFlushTo(boolean) instead. It would be ideal to change all internal uses of setAutoFlush(boolean, boolean) to use setAutoFlushTo, if possible. HTable.setAutoFlush(boolean, boolean) is used in a handful of places. setAutoFlush(false, false) has the same results as HTable.setAutoFlush(false). Calling HTable.setAutoFlush(false, true) has the same affect as Table.setAutoFlushTo(false), assuming HTable.setAutoFlush(false) was not called previously (by default, the second parameter, clearBufferOnFail, is true and should remain true according to the comments). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12534) Wrong region location cache in client after regions are moved
[
https://issues.apache.org/jira/browse/HBASE-12534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14224941#comment-14224941
]
Lars Hofhansl commented on HBASE-12534:
---
With that explanation it seems we can simply get rid of MIN_RPC_TIMEOUT. If
somebody wants to set the rpc timeout low, (s)he should be free to do so. If
that timeout is set too low for the environment in question that's their
problem to fix.
Wrong region location cache in client after regions are moved
-
Key: HBASE-12534
URL: https://issues.apache.org/jira/browse/HBASE-12534
Project: HBase
Issue Type: Bug
Affects Versions: 2.0.0
Reporter: Liu Shaohui
Assignee: Liu Shaohui
Priority: Critical
Labels: client
Attachments: HBASE-12534-0.94-v1.diff, HBASE-12534-v1.diff
In our 0.94 hbase cluster, we found that client got wrong region location
cache and did not update it after a region is moved to another regionserver.
The reason is wrong client config and bug in RpcRetryingCaller of hbase
client.
The rpc configs are following:
{code}
hbase.rpc.timeout=1000
hbase.client.pause=200
hbase.client.operation.timeout=1200
{code}
But the client retry number is 3
{code}
hbase.client.retries.number=3
{code}
Assumed that a region is at regionserver A before, and then it is moved to
regionserver B. The client try to make a call to regionserver A and get an
NotServingRegionException. For the rety number is not 1, the region server
location cache is not cleaned. See: RpcRetryingCaller.java#141 and
RegionServerCallable.java#127
{code}
@Override
public void throwable(Throwable t, boolean retrying) {
if (t instanceof SocketTimeoutException ||
} else if (t instanceof NotServingRegionException !retrying) {
// Purge cache entries for this specific region from hbase:meta cache
// since we don't call connect(true) when number of retries is 1.
getConnection().deleteCachedRegionLocation(location);
}
}
{code}
But the call did not retry and throw an SocketTimeoutException for the time
the call will take is larger than the operation timeout.See
RpcRetryingCaller.java#152
{code}
expectedSleep = callable.sleep(pause, tries + 1);
// If, after the planned sleep, there won't be enough time left, we
stop now.
long duration = singleCallDuration(expectedSleep);
if (duration callTimeout) {
String msg = callTimeout= + callTimeout + , callDuration= +
duration +
: + callable.getExceptionMessageAdditionalDetail();
throw (SocketTimeoutException)(new
SocketTimeoutException(msg).initCause(t));
}
{code}
At last, the wrong region location will never be not cleaned up .
[~lhofhansl]
In hbase 0.94, the MIN_RPC_TIMEOUT in singleCallDuration is 2000 in default,
which trigger this bug.
{code}
private long singleCallDuration(final long expectedSleep) {
return (EnvironmentEdgeManager.currentTimeMillis() - this.globalStartTime)
+ MIN_RPC_TIMEOUT + expectedSleep;
}
{code}
But there is risk in master code too.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12570) Missing/invalid split policy class name brings down your HBase cluster
[ https://issues.apache.org/jira/browse/HBASE-12570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14224942#comment-14224942 ] Enis Soztutar commented on HBASE-12570: --- I forgot HBASE-10591 was not backported into 0.98. I'll create a subtask for that, and create another subtask for other classes we load. Missing/invalid split policy class name brings down your HBase cluster -- Key: HBASE-12570 URL: https://issues.apache.org/jira/browse/HBASE-12570 Project: HBase Issue Type: Bug Reporter: James Taylor See PHOENIX-1473. If a split policy class cannot be resolved, then your HBase cluster will be brought down as each region server that successively attempts to open the region will not find the class and will bring itself down. One idea to prevent this would be to fail the CREATE TABLE or ALTER TABLE admin call if the split policy class cannot be found. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-12559) Provide LoadBalancer with online configuration capability
[ https://issues.apache.org/jira/browse/HBASE-12559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-12559: --- Attachment: 12559-v3.txt Provide LoadBalancer with online configuration capability - Key: HBASE-12559 URL: https://issues.apache.org/jira/browse/HBASE-12559 Project: HBase Issue Type: Improvement Reporter: Ted Yu Assignee: Ted Yu Fix For: 2.0.0 Attachments: 12559-v1.txt, 12559-v2.txt, 12559-v3.txt StochasticLoadBalancer has many knobs which user can adjust. It would increase productivity by allowing StochasticLoadBalancer to accept online configuration changes. LoadBalancer already implements setConf(Configuration) method which reloads relevant configuration parameters. We need to add updateMasterConfiguration() method to Admin which invokes the setConf() method. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-12573) Backport HBASE-10591 Sanity check table configuration in createTable
Enis Soztutar created HBASE-12573: - Summary: Backport HBASE-10591 Sanity check table configuration in createTable Key: HBASE-12573 URL: https://issues.apache.org/jira/browse/HBASE-12573 Project: HBase Issue Type: Sub-task Reporter: Enis Soztutar Assignee: Enis Soztutar Fix For: 0.98.9 In parent jira, it seems that we will be better off backporting HBASE-12570. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12570) Missing/invalid split policy class name brings down your HBase cluster
[ https://issues.apache.org/jira/browse/HBASE-12570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14224945#comment-14224945 ] Lars Hofhansl commented on HBASE-12570: --- I'd be in favor of backporting HBASE-10591 to 0.98. Might lead to surprises, though. [~apurtell], what do you think? Missing/invalid split policy class name brings down your HBase cluster -- Key: HBASE-12570 URL: https://issues.apache.org/jira/browse/HBASE-12570 Project: HBase Issue Type: Bug Reporter: James Taylor See PHOENIX-1473. If a split policy class cannot be resolved, then your HBase cluster will be brought down as each region server that successively attempts to open the region will not find the class and will bring itself down. One idea to prevent this would be to fail the CREATE TABLE or ALTER TABLE admin call if the split policy class cannot be found. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-12574) Update replication metrics to not do so many map look ups.
Elliott Clark created HBASE-12574: - Summary: Update replication metrics to not do so many map look ups. Key: HBASE-12574 URL: https://issues.apache.org/jira/browse/HBASE-12574 Project: HBase Issue Type: Bug Components: metrics Affects Versions: 0.98.6.1 Reporter: Elliott Clark Assignee: Elliott Clark Replication is the last metrics area that still does a significant amount of hash map lookups. Lets re-factor that. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-12575) sanity check coprocessor classes and other classes from configuration are loadable
Enis Soztutar created HBASE-12575: - Summary: sanity check coprocessor classes and other classes from configuration are loadable Key: HBASE-12575 URL: https://issues.apache.org/jira/browse/HBASE-12575 Project: HBase Issue Type: Sub-task Reporter: Enis Soztutar Fix For: 2.0.0, 0.98.9, 0.99.2 We load coprocessors and other classes from configuration. In case of a typo in the class name (or deployment problem) a create table / alter table with wrong class name brings down the whole cluster. Master already does sanity check for compression and region split policy classes introduced in HBASE-10591. We should extend that to check some other common cases as well. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12534) Wrong region location cache in client after regions are moved
[
https://issues.apache.org/jira/browse/HBASE-12534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14224951#comment-14224951
]
Nicolas Liochon commented on HBASE-12534:
-
bq. it seems we can simply get rid of MIN_RPC_TIMEOUT
I'm not against removing it (may be it's too much of a corner case) but it
solves more than a configuration issue.
with the setting above
{code}
hbase.rpc.timeout=1000
hbase.client.operation.timeout=1200
{code}
if the first try fails after 1080ms, then the second try will have a
rpc.timeout of 20ms (hbase.client.pause put aside). The MIN_RPC_TIMEOUT will
say 'that's too low, let's set it to something more reasonable.
We can remove it. What we need to detect however is a setting of 0 (if not it
will be an infinite timeout).
Wrong region location cache in client after regions are moved
-
Key: HBASE-12534
URL: https://issues.apache.org/jira/browse/HBASE-12534
Project: HBase
Issue Type: Bug
Affects Versions: 2.0.0
Reporter: Liu Shaohui
Assignee: Liu Shaohui
Priority: Critical
Labels: client
Attachments: HBASE-12534-0.94-v1.diff, HBASE-12534-v1.diff
In our 0.94 hbase cluster, we found that client got wrong region location
cache and did not update it after a region is moved to another regionserver.
The reason is wrong client config and bug in RpcRetryingCaller of hbase
client.
The rpc configs are following:
{code}
hbase.rpc.timeout=1000
hbase.client.pause=200
hbase.client.operation.timeout=1200
{code}
But the client retry number is 3
{code}
hbase.client.retries.number=3
{code}
Assumed that a region is at regionserver A before, and then it is moved to
regionserver B. The client try to make a call to regionserver A and get an
NotServingRegionException. For the rety number is not 1, the region server
location cache is not cleaned. See: RpcRetryingCaller.java#141 and
RegionServerCallable.java#127
{code}
@Override
public void throwable(Throwable t, boolean retrying) {
if (t instanceof SocketTimeoutException ||
} else if (t instanceof NotServingRegionException !retrying) {
// Purge cache entries for this specific region from hbase:meta cache
// since we don't call connect(true) when number of retries is 1.
getConnection().deleteCachedRegionLocation(location);
}
}
{code}
But the call did not retry and throw an SocketTimeoutException for the time
the call will take is larger than the operation timeout.See
RpcRetryingCaller.java#152
{code}
expectedSleep = callable.sleep(pause, tries + 1);
// If, after the planned sleep, there won't be enough time left, we
stop now.
long duration = singleCallDuration(expectedSleep);
if (duration callTimeout) {
String msg = callTimeout= + callTimeout + , callDuration= +
duration +
: + callable.getExceptionMessageAdditionalDetail();
throw (SocketTimeoutException)(new
SocketTimeoutException(msg).initCause(t));
}
{code}
At last, the wrong region location will never be not cleaned up .
[~lhofhansl]
In hbase 0.94, the MIN_RPC_TIMEOUT in singleCallDuration is 2000 in default,
which trigger this bug.
{code}
private long singleCallDuration(final long expectedSleep) {
return (EnvironmentEdgeManager.currentTimeMillis() - this.globalStartTime)
+ MIN_RPC_TIMEOUT + expectedSleep;
}
{code}
But there is risk in master code too.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-12573) Backport HBASE-10591 Sanity check table configuration in createTable
[ https://issues.apache.org/jira/browse/HBASE-12573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Enis Soztutar updated HBASE-12573: -- Description: In parent jira, it seems that we will be better off backporting HBASE-10591. (was: In parent jira, it seems that we will be better off backporting HBASE-12570. ) Backport HBASE-10591 Sanity check table configuration in createTable Key: HBASE-12573 URL: https://issues.apache.org/jira/browse/HBASE-12573 Project: HBase Issue Type: Sub-task Reporter: Enis Soztutar Assignee: Enis Soztutar Fix For: 0.98.9 In parent jira, it seems that we will be better off backporting HBASE-10591. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12534) Wrong region location cache in client after regions are moved
[
https://issues.apache.org/jira/browse/HBASE-12534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14224955#comment-14224955
]
Lars Hofhansl commented on HBASE-12534:
---
Oh. I see, it's lower bound for each individual round trip. Hmm... It does make
sense to have it then.
Wrong region location cache in client after regions are moved
-
Key: HBASE-12534
URL: https://issues.apache.org/jira/browse/HBASE-12534
Project: HBase
Issue Type: Bug
Affects Versions: 2.0.0
Reporter: Liu Shaohui
Assignee: Liu Shaohui
Priority: Critical
Labels: client
Attachments: HBASE-12534-0.94-v1.diff, HBASE-12534-v1.diff
In our 0.94 hbase cluster, we found that client got wrong region location
cache and did not update it after a region is moved to another regionserver.
The reason is wrong client config and bug in RpcRetryingCaller of hbase
client.
The rpc configs are following:
{code}
hbase.rpc.timeout=1000
hbase.client.pause=200
hbase.client.operation.timeout=1200
{code}
But the client retry number is 3
{code}
hbase.client.retries.number=3
{code}
Assumed that a region is at regionserver A before, and then it is moved to
regionserver B. The client try to make a call to regionserver A and get an
NotServingRegionException. For the rety number is not 1, the region server
location cache is not cleaned. See: RpcRetryingCaller.java#141 and
RegionServerCallable.java#127
{code}
@Override
public void throwable(Throwable t, boolean retrying) {
if (t instanceof SocketTimeoutException ||
} else if (t instanceof NotServingRegionException !retrying) {
// Purge cache entries for this specific region from hbase:meta cache
// since we don't call connect(true) when number of retries is 1.
getConnection().deleteCachedRegionLocation(location);
}
}
{code}
But the call did not retry and throw an SocketTimeoutException for the time
the call will take is larger than the operation timeout.See
RpcRetryingCaller.java#152
{code}
expectedSleep = callable.sleep(pause, tries + 1);
// If, after the planned sleep, there won't be enough time left, we
stop now.
long duration = singleCallDuration(expectedSleep);
if (duration callTimeout) {
String msg = callTimeout= + callTimeout + , callDuration= +
duration +
: + callable.getExceptionMessageAdditionalDetail();
throw (SocketTimeoutException)(new
SocketTimeoutException(msg).initCause(t));
}
{code}
At last, the wrong region location will never be not cleaned up .
[~lhofhansl]
In hbase 0.94, the MIN_RPC_TIMEOUT in singleCallDuration is 2000 in default,
which trigger this bug.
{code}
private long singleCallDuration(final long expectedSleep) {
return (EnvironmentEdgeManager.currentTimeMillis() - this.globalStartTime)
+ MIN_RPC_TIMEOUT + expectedSleep;
}
{code}
But there is risk in master code too.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HBASE-12576) Add metrics for rolling the HLog if there are too few DN's in the write pipeline
Elliott Clark created HBASE-12576: - Summary: Add metrics for rolling the HLog if there are too few DN's in the write pipeline Key: HBASE-12576 URL: https://issues.apache.org/jira/browse/HBASE-12576 Project: HBase Issue Type: Bug Components: metrics Reporter: Elliott Clark Assignee: Elliott Clark -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-12333) Add Integration Test Runner which is more friendly
[ https://issues.apache.org/jira/browse/HBASE-12333?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manukranth Kolloju updated HBASE-12333: --- Attachment: HBASE-12333-v2.patch Incorporated the review comments and made the test patterns conform to the IntegrationTestRunner. Add Integration Test Runner which is more friendly -- Key: HBASE-12333 URL: https://issues.apache.org/jira/browse/HBASE-12333 Project: HBase Issue Type: New Feature Components: test Affects Versions: 2.0.0 Reporter: Manukranth Kolloju Assignee: Manukranth Kolloju Fix For: 2.0.0 Attachments: 0001-HBASE-12333-Add-Integration-Test-Runner-which-is-mor.patch, HBASE-12333-v2.patch Original Estimate: 48h Remaining Estimate: 48h This Jira is intended to add a Driver class which would run a list of Integration tests on an actual hbase cluster. And generate a machine readable results file in JSON and put it on HDFS. The idea is to make it easy to run driver class using a long list of appropriate command line params and wait for the JSON file on the HDFS. This will help in plugging into external automation and makes it easier to maintain Continuous Integration Scripts. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12534) Wrong region location cache in client after regions are moved
[
https://issues.apache.org/jira/browse/HBASE-12534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14224992#comment-14224992
]
stack commented on HBASE-12534:
---
bq. It does make sense to have it then.
Yeah. Configurable though as [~nkeywal] suggests
Wrong region location cache in client after regions are moved
-
Key: HBASE-12534
URL: https://issues.apache.org/jira/browse/HBASE-12534
Project: HBase
Issue Type: Bug
Affects Versions: 2.0.0
Reporter: Liu Shaohui
Assignee: Liu Shaohui
Priority: Critical
Labels: client
Attachments: HBASE-12534-0.94-v1.diff, HBASE-12534-v1.diff
In our 0.94 hbase cluster, we found that client got wrong region location
cache and did not update it after a region is moved to another regionserver.
The reason is wrong client config and bug in RpcRetryingCaller of hbase
client.
The rpc configs are following:
{code}
hbase.rpc.timeout=1000
hbase.client.pause=200
hbase.client.operation.timeout=1200
{code}
But the client retry number is 3
{code}
hbase.client.retries.number=3
{code}
Assumed that a region is at regionserver A before, and then it is moved to
regionserver B. The client try to make a call to regionserver A and get an
NotServingRegionException. For the rety number is not 1, the region server
location cache is not cleaned. See: RpcRetryingCaller.java#141 and
RegionServerCallable.java#127
{code}
@Override
public void throwable(Throwable t, boolean retrying) {
if (t instanceof SocketTimeoutException ||
} else if (t instanceof NotServingRegionException !retrying) {
// Purge cache entries for this specific region from hbase:meta cache
// since we don't call connect(true) when number of retries is 1.
getConnection().deleteCachedRegionLocation(location);
}
}
{code}
But the call did not retry and throw an SocketTimeoutException for the time
the call will take is larger than the operation timeout.See
RpcRetryingCaller.java#152
{code}
expectedSleep = callable.sleep(pause, tries + 1);
// If, after the planned sleep, there won't be enough time left, we
stop now.
long duration = singleCallDuration(expectedSleep);
if (duration callTimeout) {
String msg = callTimeout= + callTimeout + , callDuration= +
duration +
: + callable.getExceptionMessageAdditionalDetail();
throw (SocketTimeoutException)(new
SocketTimeoutException(msg).initCause(t));
}
{code}
At last, the wrong region location will never be not cleaned up .
[~lhofhansl]
In hbase 0.94, the MIN_RPC_TIMEOUT in singleCallDuration is 2000 in default,
which trigger this bug.
{code}
private long singleCallDuration(final long expectedSleep) {
return (EnvironmentEdgeManager.currentTimeMillis() - this.globalStartTime)
+ MIN_RPC_TIMEOUT + expectedSleep;
}
{code}
But there is risk in master code too.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12490) Replace uses of setAutoFlush(boolean, boolean)
[ https://issues.apache.org/jira/browse/HBASE-12490?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14225001#comment-14225001 ] Solomon Duskis commented on HBASE-12490: The javadoc said that clearBufferOnFail=false has been deprecated since 0.96. It seems reasonable to me to remove it, but that's a decision beyond my paygrade. Replace uses of setAutoFlush(boolean, boolean) -- Key: HBASE-12490 URL: https://issues.apache.org/jira/browse/HBASE-12490 Project: HBase Issue Type: Bug Affects Versions: 2.0.0, 0.99.2 Reporter: Solomon Duskis Assignee: Solomon Duskis Attachments: HBASE-12490.patch, HBASE-12490B.patch, HBASE-12490B.patch, HBASE-12490B.patch, HBASE-12490C.patch The various uses of setAutoFlush() seem to need some tlc. There's a note in HTableInterface: @deprecated in 0.99 since setting clearBufferOnFail is deprecated. Use setAutoFlushTo(boolean) instead. It would be ideal to change all internal uses of setAutoFlush(boolean, boolean) to use setAutoFlushTo, if possible. HTable.setAutoFlush(boolean, boolean) is used in a handful of places. setAutoFlush(false, false) has the same results as HTable.setAutoFlush(false). Calling HTable.setAutoFlush(false, true) has the same affect as Table.setAutoFlushTo(false), assuming HTable.setAutoFlush(false) was not called previously (by default, the second parameter, clearBufferOnFail, is true and should remain true according to the comments). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-12576) Add metrics for rolling the HLog if there are too few DN's in the write pipeline
[ https://issues.apache.org/jira/browse/HBASE-12576?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-12576: Component/s: wal Add metrics for rolling the HLog if there are too few DN's in the write pipeline Key: HBASE-12576 URL: https://issues.apache.org/jira/browse/HBASE-12576 Project: HBase Issue Type: Bug Components: metrics, wal Reporter: Elliott Clark Assignee: Elliott Clark -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12491) TableMapReduceUtil.findContainingJar() NPE
[ https://issues.apache.org/jira/browse/HBASE-12491?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14225012#comment-14225012 ] Solomon Duskis commented on HBASE-12491: Can this change be applied to branch-1 as well? TableMapReduceUtil.findContainingJar() NPE -- Key: HBASE-12491 URL: https://issues.apache.org/jira/browse/HBASE-12491 Project: HBase Issue Type: Bug Components: mapreduce Affects Versions: 2.0.0, 0.99.2 Reporter: Solomon Duskis Assignee: Solomon Duskis Fix For: 2.0.0, 0.94.25, 0.98.9, 0.99.2 Attachments: HBASE-12491.patch, HBASE-12491.patch Adding a bootclasspath library causes an NPE when running hbase map reduce jobs in TableMapReduceUtil.findContainingJar(). Classes in the library added to the bootclasspath get a null classpathLoader. Check for a null loader in TableMapReduceUtil.findContainingJar(). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-12577) Disable distributed log replay by default
Enis Soztutar created HBASE-12577: - Summary: Disable distributed log replay by default Key: HBASE-12577 URL: https://issues.apache.org/jira/browse/HBASE-12577 Project: HBase Issue Type: Sub-task Reporter: Enis Soztutar Assignee: Jeffrey Zhong Priority: Critical Fix For: 0.99.2 Distributed log replay is an awesome feature, but due of HBASE-11094, the rolling upgrade story from 0.98 is hard to explain / enforce. The fix for HBASE-11094 only went into 0.98.4, meaning rolling upgrades from 0.98.4- might lose data during the upgrade. I feel no matter how much documentation / warning we do, we cannot prevent users from doing rolling upgrades from 0.98.4- to 1.0. And we do not want to inconvenience the user by requiring a two step rolling upgrade. Thus I think we should disable dist log replay for 1.0, and re-enable it again for 1.1 (if rolling upgrade from 0.98 is not supported). ie. undo: HBASE-10888 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12577) Disable distributed log replay by default
[ https://issues.apache.org/jira/browse/HBASE-12577?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14225023#comment-14225023 ] Sean Busbey commented on HBASE-12577: - +1, though I'd rather see the non-rolling-upgrade barrier move to 2.0. Disable distributed log replay by default - Key: HBASE-12577 URL: https://issues.apache.org/jira/browse/HBASE-12577 Project: HBase Issue Type: Sub-task Reporter: Enis Soztutar Assignee: Jeffrey Zhong Priority: Critical Fix For: 0.99.2 Distributed log replay is an awesome feature, but due of HBASE-11094, the rolling upgrade story from 0.98 is hard to explain / enforce. The fix for HBASE-11094 only went into 0.98.4, meaning rolling upgrades from 0.98.4- might lose data during the upgrade. I feel no matter how much documentation / warning we do, we cannot prevent users from doing rolling upgrades from 0.98.4- to 1.0. And we do not want to inconvenience the user by requiring a two step rolling upgrade. Thus I think we should disable dist log replay for 1.0, and re-enable it again for 1.1 (if rolling upgrade from 0.98 is not supported). ie. undo: HBASE-10888 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-12573) Backport HBASE-10591 Sanity check table configuration in createTable
[
https://issues.apache.org/jira/browse/HBASE-12573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Enis Soztutar updated HBASE-12573:
--
Attachment: hbase-10591_v5-0.98.patch
Here is 0.98 patch that applies cleanly.
I've ran
{code}
HW10676:hbase-0.98$ mvn test -Dtest=TestZooKeeper,TestFromClientSide*,TestAdmin*
{code}
but not the rest of the tests.
Backport HBASE-10591 Sanity check table configuration in createTable
Key: HBASE-12573
URL: https://issues.apache.org/jira/browse/HBASE-12573
Project: HBase
Issue Type: Sub-task
Reporter: Enis Soztutar
Assignee: Enis Soztutar
Fix For: 0.98.9
Attachments: hbase-10591_v5-0.98.patch
In parent jira, it seems that we will be better off backporting HBASE-10591.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12490) Replace uses of setAutoFlush(boolean, boolean)
[ https://issues.apache.org/jira/browse/HBASE-12490?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14225046#comment-14225046 ] Solomon Duskis commented on HBASE-12490: On a different note... Why is the patch process failing? I'm not sure what I did wrong. Replace uses of setAutoFlush(boolean, boolean) -- Key: HBASE-12490 URL: https://issues.apache.org/jira/browse/HBASE-12490 Project: HBase Issue Type: Bug Affects Versions: 2.0.0, 0.99.2 Reporter: Solomon Duskis Assignee: Solomon Duskis Attachments: HBASE-12490.patch, HBASE-12490B.patch, HBASE-12490B.patch, HBASE-12490B.patch, HBASE-12490C.patch The various uses of setAutoFlush() seem to need some tlc. There's a note in HTableInterface: @deprecated in 0.99 since setting clearBufferOnFail is deprecated. Use setAutoFlushTo(boolean) instead. It would be ideal to change all internal uses of setAutoFlush(boolean, boolean) to use setAutoFlushTo, if possible. HTable.setAutoFlush(boolean, boolean) is used in a handful of places. setAutoFlush(false, false) has the same results as HTable.setAutoFlush(false). Calling HTable.setAutoFlush(false, true) has the same affect as Table.setAutoFlushTo(false), assuming HTable.setAutoFlush(false) was not called previously (by default, the second parameter, clearBufferOnFail, is true and should remain true according to the comments). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-12514) Cleanup HFileOutputFormat legacy code
[ https://issues.apache.org/jira/browse/HBASE-12514?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Solomon Duskis updated HBASE-12514: --- Attachment: HBASE-12514.patch Trying again, since I can't replicate the test failures locally. Cleanup HFileOutputFormat legacy code - Key: HBASE-12514 URL: https://issues.apache.org/jira/browse/HBASE-12514 Project: HBase Issue Type: Bug Reporter: Solomon Duskis Assignee: Solomon Duskis Fix For: 2.0.0, 0.99.2 Attachments: HBASE-12514 (1).patch, HBASE-12514.patch, HBASE-12514.patch, HBASE-12514.patch HFileOutputFormat methods route to their HFileOutputFormat2 counterparts. Replace all calls to HFileOutputFormat with HFileOutputFormat2 equivalents. In the spirit of cleanup, add @Override annotations and helper methods that do not require the use of deprecated classes such as HTable. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12514) Cleanup HFileOutputFormat legacy code
[
https://issues.apache.org/jira/browse/HBASE-12514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14225089#comment-14225089
]
Hadoop QA commented on HBASE-12514:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12683632/HBASE-12514.patch
against master branch at commit e6b4300756b7f09a31ba35cb3baf41d294ed6e14.
ATTACHMENT ID: 12683632
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 6 new
or modified tests.
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/11831//console
This message is automatically generated.
Cleanup HFileOutputFormat legacy code
-
Key: HBASE-12514
URL: https://issues.apache.org/jira/browse/HBASE-12514
Project: HBase
Issue Type: Bug
Reporter: Solomon Duskis
Assignee: Solomon Duskis
Fix For: 2.0.0, 0.99.2
Attachments: HBASE-12514 (1).patch, HBASE-12514.patch,
HBASE-12514.patch, HBASE-12514.patch
HFileOutputFormat methods route to their HFileOutputFormat2 counterparts.
Replace all calls to HFileOutputFormat with HFileOutputFormat2 equivalents.
In the spirit of cleanup, add @Override annotations and helper methods that
do not require the use of deprecated classes such as HTable.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-12576) Add metrics for rolling the HLog if there are too few DN's in the write pipeline
[ https://issues.apache.org/jira/browse/HBASE-12576?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Elliott Clark updated HBASE-12576: -- Attachment: HBASE-12576.patch Add metrics for rolling the HLog if there are too few DN's in the write pipeline Key: HBASE-12576 URL: https://issues.apache.org/jira/browse/HBASE-12576 Project: HBase Issue Type: Bug Components: metrics, wal Reporter: Elliott Clark Assignee: Elliott Clark Attachments: HBASE-12576.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-12576) Add metrics for rolling the HLog if there are too few DN's in the write pipeline
[ https://issues.apache.org/jira/browse/HBASE-12576?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Elliott Clark updated HBASE-12576: -- Fix Version/s: 0.99.2 2.0.0 Status: Patch Available (was: Open) Add metrics for rolling the HLog if there are too few DN's in the write pipeline Key: HBASE-12576 URL: https://issues.apache.org/jira/browse/HBASE-12576 Project: HBase Issue Type: Bug Components: metrics, wal Reporter: Elliott Clark Assignee: Elliott Clark Fix For: 2.0.0, 0.99.2 Attachments: HBASE-12576.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12576) Add metrics for rolling the HLog if there are too few DN's in the write pipeline
[
https://issues.apache.org/jira/browse/HBASE-12576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14225104#comment-14225104
]
Hadoop QA commented on HBASE-12576:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12683634/HBASE-12576.patch
against master branch at commit e6b4300756b7f09a31ba35cb3baf41d294ed6e14.
ATTACHMENT ID: 12683634
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:red}-1 javac{color}. The patch appears to cause mvn compile goal to
fail.
Compilation errors resume:
[ERROR] COMPILATION ERROR :
[ERROR]
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/LogRoller.java:[65,9]
method does not override or implement a method from a supertype
[ERROR]
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/hbase-server/src/main/java/org/apache/hadoop/hbase/wal/DisabledWALProvider.java:[113,19]
method logRollRequested in interface
org.apache.hadoop.hbase.regionserver.wal.WALActionsListener cannot be applied
to given types;
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-compiler-plugin:3.2:compile (default-compile) on
project hbase-server: Compilation failure: Compilation failure:
[ERROR]
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/LogRoller.java:[65,9]
method does not override or implement a method from a supertype
[ERROR]
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/hbase-server/src/main/java/org/apache/hadoop/hbase/wal/DisabledWALProvider.java:[113,19]
method logRollRequested in interface
org.apache.hadoop.hbase.regionserver.wal.WALActionsListener cannot be applied
to given types;
[ERROR] required: boolean
[ERROR] found: no arguments
[ERROR] reason: actual and formal argument lists differ in length
[ERROR] - [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e
switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please
read the following articles:
[ERROR] [Help 1]
http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR] mvn goals -rf :hbase-server
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/11832//console
This message is automatically generated.
Add metrics for rolling the HLog if there are too few DN's in the write
pipeline
Key: HBASE-12576
URL: https://issues.apache.org/jira/browse/HBASE-12576
Project: HBase
Issue Type: Bug
Components: metrics, wal
Reporter: Elliott Clark
Assignee: Elliott Clark
Fix For: 2.0.0, 0.99.2
Attachments: HBASE-12576.patch
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12338) Client side scanning prefetching.
[ https://issues.apache.org/jira/browse/HBASE-12338?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14225109#comment-14225109 ] Yi Deng commented on HBASE-12338: - [~stack], I've made the benchmarking. Anything that prevents this patch to be reviewed? Client side scanning prefetching. - Key: HBASE-12338 URL: https://issues.apache.org/jira/browse/HBASE-12338 Project: HBase Issue Type: Sub-task Components: Client Affects Versions: 1.0.0, 2.0.0, 0.98.6.1 Reporter: Yi Deng Assignee: Yi Deng Labels: prefetch, results, scanner Attachments: 0001-Add-ScanPrefetcher-for-client-side-scanning-prefetch.patch, 0001-ScanPrefetcher.patch, 2.0-0001-Add-ScanPrefetcher-for-client-side-scanning-prefetch.patch, 2.0-0002-Add-ScanPrefetcher-for-client-side-scanning-prefetch.patch, 2.0-0003-Add-ScanPrefetcher-for-client-side-scanning-prefetch.patch Since server side prefetching was not proved to be a good way to prefetch, we need to do it on client side. This is a wrapper class that takes any instance of `ResultScanner` as the underneath scanning component. The class will schedule the scanning in a background thread. There is a buffering queue storing prefetched results, whose's length is configurable. The prefetcher will release the thread if the queue is full and wait for results to be consumed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12514) Cleanup HFileOutputFormat legacy code
[ https://issues.apache.org/jira/browse/HBASE-12514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14225110#comment-14225110 ] Solomon Duskis commented on HBASE-12514: This is strange. I got the same problem on another patch. I'd guess this is a problem with the build process rather than my patches. Cleanup HFileOutputFormat legacy code - Key: HBASE-12514 URL: https://issues.apache.org/jira/browse/HBASE-12514 Project: HBase Issue Type: Bug Reporter: Solomon Duskis Assignee: Solomon Duskis Fix For: 2.0.0, 0.99.2 Attachments: HBASE-12514 (1).patch, HBASE-12514.patch, HBASE-12514.patch, HBASE-12514.patch HFileOutputFormat methods route to their HFileOutputFormat2 counterparts. Replace all calls to HFileOutputFormat with HFileOutputFormat2 equivalents. In the spirit of cleanup, add @Override annotations and helper methods that do not require the use of deprecated classes such as HTable. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-12576) Add metrics for rolling the HLog if there are too few DN's in the write pipeline
[ https://issues.apache.org/jira/browse/HBASE-12576?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Elliott Clark updated HBASE-12576: -- Attachment: HBASE-12576-v1.patch Whoops missed adding some files. Add metrics for rolling the HLog if there are too few DN's in the write pipeline Key: HBASE-12576 URL: https://issues.apache.org/jira/browse/HBASE-12576 Project: HBase Issue Type: Bug Components: metrics, wal Reporter: Elliott Clark Assignee: Elliott Clark Fix For: 2.0.0, 0.99.2 Attachments: HBASE-12576-v1.patch, HBASE-12576.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12559) Provide LoadBalancer with online configuration capability
[
https://issues.apache.org/jira/browse/HBASE-12559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14225122#comment-14225122
]
Hadoop QA commented on HBASE-12559:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12683611/12559-v3.txt
against master branch at commit e6b4300756b7f09a31ba35cb3baf41d294ed6e14.
ATTACHMENT ID: 12683611
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 7 new
or modified tests.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:red}-1 checkstyle{color}. The applied patch generated
3781 checkstyle errors (more than the master's current 3779 errors).
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 lineLengths{color}. The patch introduces the following lines
longer than 100:
+ * coderpc
UpdateMasterConfiguration(.UpdateMasterConfigurationRequest) returns
(.UpdateMasterConfigurationResponse);/code
+ * coderpc UpdateMasterConfiguration(.UpdateMasterConfigurationRequest)
returns (.UpdateMasterConfigurationResponse);/code
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.TestNamespace
org.apache.hadoop.hbase.client.TestScannerTimeout
org.apache.hadoop.hbase.client.TestRestoreSnapshotFromClient
org.apache.hadoop.hbase.snapshot.TestRestoreFlushSnapshotFromClient
org.apache.hadoop.hbase.coprocessor.TestRegionObserverScannerOpenHook
org.apache.hadoop.hbase.security.access.TestCellACLWithMultipleVersions
org.apache.hadoop.hbase.coprocessor.TestMasterObserver
org.apache.hadoop.hbase.TestMultiVersions
org.apache.hadoop.hbase.coprocessor.TestRegionServerCoprocessorEndpoint
org.apache.hadoop.hbase.coprocessor.TestDoubleColumnInterpreter
org.apache.hadoop.hbase.coprocessor.TestRegionServerCoprocessorExceptionWithRemove
org.apache.hadoop.hbase.coprocessor.TestWALObserver
org.apache.hadoop.hbase.client.TestTableSnapshotScanner
org.apache.hadoop.hbase.constraint.TestConstraint
org.apache.hadoop.hbase.coprocessor.TestMasterCoprocessorExceptionWithRemove
org.apache.hadoop.hbase.TestZooKeeper
org.apache.hadoop.hbase.coprocessor.TestHTableWrapper
org.apache.hadoop.hbase.coprocessor.TestCoprocessorEndpoint
org.apache.hadoop.hbase.coprocessor.TestRowProcessorEndpoint
org.apache.hadoop.hbase.client.TestHTableMultiplexer
org.apache.hadoop.hbase.snapshot.TestFlushSnapshotFromClient
org.apache.hadoop.hbase.snapshot.TestSecureExportSnapshot
org.apache.hadoop.hbase.coprocessor.TestRegionServerObserver
org.apache.hadoop.hbase.coprocessor.TestClassLoading
org.apache.hadoop.hbase.client.TestReplicaWithCluster
org.apache.hadoop.hbase.coprocessor.TestOpenTableInCoprocessor
org.apache.hadoop.hbase.coprocessor.TestRegionServerCoprocessorExceptionWithAbort
org.apache.hadoop.hbase.client.TestSnapshotMetadata
org.apache.hadoop.hbase.snapshot.TestExportSnapshot
org.apache.hadoop.hbase.client.TestReplicasClient
org.apache.hadoop.hbase.TestJMXListener
org.apache.hadoop.hbase.coprocessor.TestBatchCoprocessorEndpoint
org.apache.hadoop.hbase.TestHColumnDescriptorDefaultVersions
org.apache.hadoop.hbase.client.TestHTableMultiplexerFlushCache
org.apache.hadoop.hbase.coprocessor.TestRegionObserverInterface
org.apache.hadoop.hbase.client.TestFromClientSide3
org.apache.hadoop.hbase.TestIOFencing
org.apache.hadoop.hbase.coprocessor.TestMasterCoprocessorExceptionWithAbort
org.apache.hadoop.hbase.coprocessor.TestCoprocessorStop
org.apache.hadoop.hbase.coprocessor.TestAggregateProtocol
[jira] [Updated] (HBASE-12514) Cleanup HFileOutputFormat legacy code
[ https://issues.apache.org/jira/browse/HBASE-12514?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-12514: -- Attachment: 12514v2.txt Rebase of [~sduskis]'s patch. Cleanup HFileOutputFormat legacy code - Key: HBASE-12514 URL: https://issues.apache.org/jira/browse/HBASE-12514 Project: HBase Issue Type: Bug Reporter: Solomon Duskis Assignee: Solomon Duskis Fix For: 2.0.0, 0.99.2 Attachments: 12514v2.txt, HBASE-12514 (1).patch, HBASE-12514.patch, HBASE-12514.patch, HBASE-12514.patch HFileOutputFormat methods route to their HFileOutputFormat2 counterparts. Replace all calls to HFileOutputFormat with HFileOutputFormat2 equivalents. In the spirit of cleanup, add @Override annotations and helper methods that do not require the use of deprecated classes such as HTable. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-12476) HydraBase Consensus Protocol
[ https://issues.apache.org/jira/browse/HBASE-12476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gaurav Menghani updated HBASE-12476: Attachment: 0002-HydraBase-consensus-protocol.patch Final patch of the squashed commits after addressing Ted's comments. HydraBase Consensus Protocol Key: HBASE-12476 URL: https://issues.apache.org/jira/browse/HBASE-12476 Project: HBase Issue Type: Sub-task Components: Consensus, wal Reporter: Gaurav Menghani Assignee: Gaurav Menghani Attachments: 0001-HydraBase-consensus-protocol.patch, 0002-HydraBase-consensus-protocol.patch This is the first patch for the HydraBase consensus protocol implemented according to the Raft consensus protocol (https://ramcloud.stanford.edu/raft.pdf), as advertised in (HBASE-12259) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12338) Client side scanning prefetching.
[ https://issues.apache.org/jira/browse/HBASE-12338?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14225129#comment-14225129 ] stack commented on HBASE-12338: --- Numbers are nice improvement. Has patch been updated on phabricator? Needs release note on how and why you'd use it. Thanks [~daviddengcn] Client side scanning prefetching. - Key: HBASE-12338 URL: https://issues.apache.org/jira/browse/HBASE-12338 Project: HBase Issue Type: Sub-task Components: Client Affects Versions: 1.0.0, 2.0.0, 0.98.6.1 Reporter: Yi Deng Assignee: Yi Deng Labels: prefetch, results, scanner Attachments: 0001-Add-ScanPrefetcher-for-client-side-scanning-prefetch.patch, 0001-ScanPrefetcher.patch, 2.0-0001-Add-ScanPrefetcher-for-client-side-scanning-prefetch.patch, 2.0-0002-Add-ScanPrefetcher-for-client-side-scanning-prefetch.patch, 2.0-0003-Add-ScanPrefetcher-for-client-side-scanning-prefetch.patch Since server side prefetching was not proved to be a good way to prefetch, we need to do it on client side. This is a wrapper class that takes any instance of `ResultScanner` as the underneath scanning component. The class will schedule the scanning in a background thread. There is a buffering queue storing prefetched results, whose's length is configurable. The prefetcher will release the thread if the queue is full and wait for results to be consumed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12490) Replace uses of setAutoFlush(boolean, boolean)
[ https://issues.apache.org/jira/browse/HBASE-12490?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14225138#comment-14225138 ] stack commented on HBASE-12490: --- [~sduskis] Looks like you need to rebase? Replace uses of setAutoFlush(boolean, boolean) -- Key: HBASE-12490 URL: https://issues.apache.org/jira/browse/HBASE-12490 Project: HBase Issue Type: Bug Affects Versions: 2.0.0, 0.99.2 Reporter: Solomon Duskis Assignee: Solomon Duskis Attachments: HBASE-12490.patch, HBASE-12490B.patch, HBASE-12490B.patch, HBASE-12490B.patch, HBASE-12490C.patch The various uses of setAutoFlush() seem to need some tlc. There's a note in HTableInterface: @deprecated in 0.99 since setting clearBufferOnFail is deprecated. Use setAutoFlushTo(boolean) instead. It would be ideal to change all internal uses of setAutoFlush(boolean, boolean) to use setAutoFlushTo, if possible. HTable.setAutoFlush(boolean, boolean) is used in a handful of places. setAutoFlush(false, false) has the same results as HTable.setAutoFlush(false). Calling HTable.setAutoFlush(false, true) has the same affect as Table.setAutoFlushTo(false), assuming HTable.setAutoFlush(false) was not called previously (by default, the second parameter, clearBufferOnFail, is true and should remain true according to the comments). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12476) HydraBase Consensus Protocol
[ https://issues.apache.org/jira/browse/HBASE-12476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14225141#comment-14225141 ] Elliott Clark commented on HBASE-12476: --- I'm +1 on committing this to a feature branch. I'll take the kick from stack as the same. Pushing this to HBASE-12259 HydraBase Consensus Protocol Key: HBASE-12476 URL: https://issues.apache.org/jira/browse/HBASE-12476 Project: HBase Issue Type: Sub-task Components: Consensus, wal Reporter: Gaurav Menghani Assignee: Gaurav Menghani Attachments: 0001-HydraBase-consensus-protocol.patch, 0002-HydraBase-consensus-protocol.patch This is the first patch for the HydraBase consensus protocol implemented according to the Raft consensus protocol (https://ramcloud.stanford.edu/raft.pdf), as advertised in (HBASE-12259) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12476) HydraBase Consensus Protocol
[ https://issues.apache.org/jira/browse/HBASE-12476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14225144#comment-14225144 ] stack commented on HBASE-12476: --- +1 on creating feature branch HydraBase Consensus Protocol Key: HBASE-12476 URL: https://issues.apache.org/jira/browse/HBASE-12476 Project: HBase Issue Type: Sub-task Components: Consensus, wal Reporter: Gaurav Menghani Assignee: Gaurav Menghani Attachments: 0001-HydraBase-consensus-protocol.patch, 0002-HydraBase-consensus-protocol.patch This is the first patch for the HydraBase consensus protocol implemented according to the Raft consensus protocol (https://ramcloud.stanford.edu/raft.pdf), as advertised in (HBASE-12259) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12576) Add metrics for rolling the HLog if there are too few DN's in the write pipeline
[ https://issues.apache.org/jira/browse/HBASE-12576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14225145#comment-14225145 ] Sean Busbey commented on HBASE-12576: - Shouldn't there be a change to o.a.h.h.regionserver.wal.MetricsWAL? Add metrics for rolling the HLog if there are too few DN's in the write pipeline Key: HBASE-12576 URL: https://issues.apache.org/jira/browse/HBASE-12576 Project: HBase Issue Type: Bug Components: metrics, wal Reporter: Elliott Clark Assignee: Elliott Clark Fix For: 2.0.0, 0.99.2 Attachments: HBASE-12576-v1.patch, HBASE-12576.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12576) Add metrics for rolling the HLog if there are too few DN's in the write pipeline
[ https://issues.apache.org/jira/browse/HBASE-12576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14225153#comment-14225153 ] Sean Busbey commented on HBASE-12576: - Can we expand o.a.h.h.regionserver.wal.TestLogRolling#testLogRollOnDatanodeDeath to include checking for this metric? Right now it relies on breaking the API so it can parse timestamps out of file names (which also then relies on our clocks not being super terrible). Add metrics for rolling the HLog if there are too few DN's in the write pipeline Key: HBASE-12576 URL: https://issues.apache.org/jira/browse/HBASE-12576 Project: HBase Issue Type: Bug Components: metrics, wal Reporter: Elliott Clark Assignee: Elliott Clark Fix For: 2.0.0, 0.99.2 Attachments: HBASE-12576-v1.patch, HBASE-12576.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12333) Add Integration Test Runner which is more friendly
[
https://issues.apache.org/jira/browse/HBASE-12333?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14225156#comment-14225156
]
Hadoop QA commented on HBASE-12333:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12683619/HBASE-12333-v2.patch
against master branch at commit e6b4300756b7f09a31ba35cb3baf41d294ed6e14.
ATTACHMENT ID: 12683619
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 13 new
or modified tests.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 checkstyle{color}. The applied patch does not increase the
total number of checkstyle errors
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:red}-1 release audit{color}. The applied patch generated 1 release
audit warnings (more than the master's current 0 warnings).
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.security.access.TestCellACLWithMultipleVersions
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/11830//testReport/
Release audit warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11830//artifact/patchprocess/patchReleaseAuditWarnings.txt
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11830//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11830//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11830//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11830//artifact/patchprocess/newPatchFindbugsWarningshbase-annotations.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11830//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11830//artifact/patchprocess/newPatchFindbugsWarningshbase-rest.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11830//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11830//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11830//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11830//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11830//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/11830//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/11830//console
This message is automatically generated.
Add Integration Test Runner which is more friendly
--
Key: HBASE-12333
URL: https://issues.apache.org/jira/browse/HBASE-12333
Project: HBase
Issue Type: New Feature
Components: test
Affects Versions: 2.0.0
Reporter: Manukranth Kolloju
Assignee: Manukranth Kolloju
Fix For: 2.0.0
Attachments:
0001-HBASE-12333-Add-Integration-Test-Runner-which-is-mor.patch,
HBASE-12333-v2.patch
Original Estimate: 48h
Remaining Estimate: 48h
This Jira is intended to add a Driver class which would run a list of
Integration tests on an actual hbase cluster. And generate a machine readable
results file in JSON and put it on HDFS. The idea is to make it easy to run
driver class using a long list of appropriate command line params and wait
for the JSON file on the HDFS. This will help in plugging into external
automation and makes it easier to maintain Continuous Integration Scripts.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (HBASE-12476) HydraBase Consensus Protocol
[ https://issues.apache.org/jira/browse/HBASE-12476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Elliott Clark resolved HBASE-12476. --- Resolution: Fixed Hadoop Flags: Reviewed Pushed to feature branch. Small tweak to the pom file to get everything wired up. HydraBase Consensus Protocol Key: HBASE-12476 URL: https://issues.apache.org/jira/browse/HBASE-12476 Project: HBase Issue Type: Sub-task Components: Consensus, wal Reporter: Gaurav Menghani Assignee: Gaurav Menghani Attachments: 0001-HydraBase-consensus-protocol.patch, 0002-HydraBase-consensus-protocol.patch This is the first patch for the HydraBase consensus protocol implemented according to the Raft consensus protocol (https://ramcloud.stanford.edu/raft.pdf), as advertised in (HBASE-12259) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-12404) Task 5 from parent: Replace internal HTable constructor use with HConnection#getTable (0.98, 0.99)
[ https://issues.apache.org/jira/browse/HBASE-12404?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-12404: -- Resolution: Fixed Fix Version/s: 2.0.0 Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Committed to master. Took a bunch of shoe-horning to get it into branch-1 -- not a cherry-pick -- but it took eventually and seems to build fine. Will keep an eye on it. Task 5 from parent: Replace internal HTable constructor use with HConnection#getTable (0.98, 0.99) -- Key: HBASE-12404 URL: https://issues.apache.org/jira/browse/HBASE-12404 Project: HBase Issue Type: Sub-task Reporter: stack Assignee: stack Fix For: 2.0.0, 0.99.2 Attachments: 0001-HBASE-12404-Task-5-from-parent-Replace-internal-HTab.patch, 12404.txt, 12404.v16.txt, 12404.v16.txt, 12404.v17.txt, 12404v10.txt, 12404v11.txt, 12404v12.txt, 12404v12.txt, 12404v13.txt, 12404v13.txt, 12404v14.txt, 12404v15.txt, 12404v18.txt, 12404v19.txt, 12404v19.txt, 12404v2.txt, 12404v20.txt, 12404v20.txt, 12404v3.txt, 12404v5.txt, 12404v6.txt, 12404v7.txt, 12404v9.txt Do the step 5. from the [~ndimiduk] list in parent issue. Go through src code and change all new HTable to instead be connection.getTable. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-12400) Fix refguide so it does connection#getTable rather than new HTable everywhere.
[ https://issues.apache.org/jira/browse/HBASE-12400?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-12400: -- Fix Version/s: 2.0.0 Fix refguide so it does connection#getTable rather than new HTable everywhere. -- Key: HBASE-12400 URL: https://issues.apache.org/jira/browse/HBASE-12400 Project: HBase Issue Type: Sub-task Components: documentation Reporter: stack Assignee: Stephen Yuan Jiang Fix For: 2.0.0, 0.99.2 The refguide has bits of code in it. The code does 'new HTable' to get a table instance. Rather, it should be promoting the new style where we get a Connection and then do a getTable on it. Ditto for references to 'new HBaseAdmin'. See ConnectionFactory for new style. See also package-info.java in Client for updated example. Misty, if you are game for this one, I can help w/ how it should look. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12400) Fix refguide so it does connection#getTable rather than new HTable everywhere.
[ https://issues.apache.org/jira/browse/HBASE-12400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14225168#comment-14225168 ] stack commented on HBASE-12400: --- [~syuanjiang] Are you working on this? Otherwise, I can put up a patch. Or if you want, I can put up a start patch to illustrate and you can take it from there? Fix refguide so it does connection#getTable rather than new HTable everywhere. -- Key: HBASE-12400 URL: https://issues.apache.org/jira/browse/HBASE-12400 Project: HBase Issue Type: Sub-task Components: documentation Reporter: stack Assignee: Stephen Yuan Jiang Fix For: 2.0.0, 0.99.2 The refguide has bits of code in it. The code does 'new HTable' to get a table instance. Rather, it should be promoting the new style where we get a Connection and then do a getTable on it. Ditto for references to 'new HBaseAdmin'. See ConnectionFactory for new style. See also package-info.java in Client for updated example. Misty, if you are game for this one, I can help w/ how it should look. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12576) Add metrics for rolling the HLog if there are too few DN's in the write pipeline
[ https://issues.apache.org/jira/browse/HBASE-12576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14225169#comment-14225169 ] Elliott Clark commented on HBASE-12576: --- Yes there still needs to be a test and to actually wire it up. I was just parking it here until I could get a good way to test this well. Add metrics for rolling the HLog if there are too few DN's in the write pipeline Key: HBASE-12576 URL: https://issues.apache.org/jira/browse/HBASE-12576 Project: HBase Issue Type: Bug Components: metrics, wal Reporter: Elliott Clark Assignee: Elliott Clark Fix For: 2.0.0, 0.99.2 Attachments: HBASE-12576-v1.patch, HBASE-12576.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12576) Add metrics for rolling the HLog if there are too few DN's in the write pipeline
[ https://issues.apache.org/jira/browse/HBASE-12576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14225189#comment-14225189 ] Sean Busbey commented on HBASE-12576: - Ah. that makes sense. Using a mock MetricsWALSource in that test would handle testing everything outside of the compat layers without being too painful AFAICT. Add metrics for rolling the HLog if there are too few DN's in the write pipeline Key: HBASE-12576 URL: https://issues.apache.org/jira/browse/HBASE-12576 Project: HBase Issue Type: Bug Components: metrics, wal Reporter: Elliott Clark Assignee: Elliott Clark Fix For: 2.0.0, 0.99.2 Attachments: HBASE-12576-v1.patch, HBASE-12576.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12570) Missing/invalid split policy class name brings down your HBase cluster
[ https://issues.apache.org/jira/browse/HBASE-12570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14225240#comment-14225240 ] Enis Soztutar commented on HBASE-12570: --- bq. I'd be in favor of backporting HBASE-10591 to 0.98. Might lead to surprises, though. That is why I did not propose that back when I originally did the patch. There is a way to disable this with a config option. Porting the patch with this disabled does not make a lot of sense though. We can either bite the bullet, or not do it in 0.98. I think [~apurtell] can make the call. Missing/invalid split policy class name brings down your HBase cluster -- Key: HBASE-12570 URL: https://issues.apache.org/jira/browse/HBASE-12570 Project: HBase Issue Type: Bug Reporter: James Taylor See PHOENIX-1473. If a split policy class cannot be resolved, then your HBase cluster will be brought down as each region server that successively attempts to open the region will not find the class and will bring itself down. One idea to prevent this would be to fail the CREATE TABLE or ALTER TABLE admin call if the split policy class cannot be found. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12576) Add metrics for rolling the HLog if there are too few DN's in the write pipeline
[
https://issues.apache.org/jira/browse/HBASE-12576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14225257#comment-14225257
]
Hadoop QA commented on HBASE-12576:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12683638/HBASE-12576-v1.patch
against master branch at commit e6b4300756b7f09a31ba35cb3baf41d294ed6e14.
ATTACHMENT ID: 12683638
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 checkstyle{color}. The applied patch does not increase the
total number of checkstyle errors
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.client.TestHCM
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/11833//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11833//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11833//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11833//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11833//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11833//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11833//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11833//artifact/patchprocess/newPatchFindbugsWarningshbase-rest.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11833//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11833//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11833//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11833//artifact/patchprocess/newPatchFindbugsWarningshbase-annotations.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/11833//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/11833//console
This message is automatically generated.
Add metrics for rolling the HLog if there are too few DN's in the write
pipeline
Key: HBASE-12576
URL: https://issues.apache.org/jira/browse/HBASE-12576
Project: HBase
Issue Type: Bug
Components: metrics, wal
Reporter: Elliott Clark
Assignee: Elliott Clark
Fix For: 2.0.0, 0.99.2
Attachments: HBASE-12576-v1.patch, HBASE-12576.patch
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12404) Task 5 from parent: Replace internal HTable constructor use with HConnection#getTable (0.98, 0.99)
[ https://issues.apache.org/jira/browse/HBASE-12404?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14225258#comment-14225258 ] Hudson commented on HBASE-12404: FAILURE: Integrated in HBase-1.0 #502 (See [https://builds.apache.org/job/HBase-1.0/502/]) HBASE-12404 Task 5 from parent: Replace internal HTable constructor use with (stack: rev c0cdaf8400831bada22558febbab9681f606e26c) * hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestClockSkewDetection.java * hbase-server/src/main/java/org/apache/hadoop/hbase/master/MasterRpcServices.java * hbase-server/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java * hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestMasterFailover.java * hbase-server/src/test/java/org/apache/hadoop/hbase/master/MockRegionServer.java * hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/FavoredNodeLoadBalancer.java * hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestMasterNoCluster.java * hbase-client/src/main/java/org/apache/hadoop/hbase/client/HTable.java * hbase-server/src/test/java/org/apache/hadoop/hbase/replication/TestPerTableCFReplication.java * hbase-client/src/main/java/org/apache/hadoop/hbase/client/ConnectionFactory.java * hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestAssignmentManagerOnCluster.java * hbase-server/src/main/java/org/apache/hadoop/hbase/security/token/TokenUtil.java * hbase-server/src/main/java/org/apache/hadoop/hbase/master/snapshot/RestoreSnapshotHandler.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RegionMergeTransaction.java * hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/HRegionPartitioner.java * hbase-client/src/main/java/org/apache/hadoop/hbase/client/coprocessor/package-info.java * hbase-server/src/main/java/org/apache/hadoop/hbase/master/handler/TableEventHandler.java * hbase-server/src/main/java/org/apache/hadoop/hbase/mapred/TableMapReduceUtil.java * hbase-server/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessController.java * hbase-server/src/main/java/org/apache/hadoop/hbase/master/handler/TruncateTableHandler.java * hbase-client/src/main/java/org/apache/hadoop/hbase/client/package-info.java * hbase-server/src/main/java/org/apache/hadoop/hbase/security/visibility/VisibilityController.java * hbase-client/src/main/java/org/apache/hadoop/hbase/client/Scan.java * hbase-server/src/main/java/org/apache/hadoop/hbase/master/handler/MetaServerShutdownHandler.java * hbase-server/src/main/java/org/apache/hadoop/hbase/master/CatalogJanitor.java * hbase-server/src/test/java/org/apache/hadoop/hbase/master/cleaner/TestHFileLinkCleaner.java * hbase-server/src/test/java/org/apache/hadoop/hbase/replication/TestReplicationStateZKImpl.java * hbase-server/src/main/java/org/apache/hadoop/hbase/tool/Canary.java * hbase-client/src/main/java/org/apache/hadoop/hbase/client/ConnectionAdapter.java * hbase-client/src/main/java/org/apache/hadoop/hbase/client/coprocessor/AggregationClient.java * hbase-server/src/test/java/org/apache/hadoop/hbase/util/hbck/TestOfflineMetaRebuildOverlap.java * hbase-server/src/main/java/org/apache/hadoop/hbase/master/handler/ModifyTableHandler.java * hbase-server/src/main/java/org/apache/hadoop/hbase/master/handler/EnableTableHandler.java * hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java * hbase-server/src/test/java/org/apache/hadoop/hbase/TestMetaMigrationConvertingToPB.java * hbase-server/src/test/java/org/apache/hadoop/hbase/master/cleaner/TestLogsCleaner.java * hbase-server/src/test/java/org/apache/hadoop/hbase/client/TestFromClientSide3.java * hbase-server/src/test/java/org/apache/hadoop/hbase/MetaMockingUtil.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/SplitTransaction.java * hbase-server/src/main/java/org/apache/hadoop/hbase/master/handler/DeleteTableHandler.java * hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/FavoredNodeAssignmentHelper.java * hbase-server/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java * hbase-server/src/test/java/org/apache/hadoop/hbase/security/token/TestTokenAuthentication.java * hbase-server/src/main/java/org/apache/hadoop/hbase/procedure/flush/MasterFlushTableProcedureManager.java * hbase-server/src/test/java/org/apache/hadoop/hbase/util/MockServer.java * hbase-server/src/main/java/org/apache/hadoop/hbase/mapred/HRegionPartitioner.java * hbase-server/src/main/java/org/apache/hadoop/hbase/master/handler/DisableTableHandler.java * hbase-server/src/main/java/org/apache/hadoop/hbase/master/snapshot/TakeSnapshotHandler.java * hbase-server/src/test/java/org/apache/hadoop/hbase/MockRegionServerServices.java * hbase-server/src/main/java/org/apache/hadoop/hbase/Server.java * hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestMasterShutdown.java *
[jira] [Updated] (HBASE-12128) Cache configuration and RpcController selection for Table in Connection
[ https://issues.apache.org/jira/browse/HBASE-12128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Enis Soztutar updated HBASE-12128: -- Resolution: Fixed Fix Version/s: (was: 1.0.0) 0.99.2 Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) I've pushed this to branch-1+. Forgot to quote you [~syuanjiang] as the author in the commit msg, sorry about that. Anyway thanks for the patch, and thanks for reviews. Cache configuration and RpcController selection for Table in Connection --- Key: HBASE-12128 URL: https://issues.apache.org/jira/browse/HBASE-12128 Project: HBase Issue Type: Sub-task Affects Versions: 1.0.0, 2.0.0 Reporter: Andrew Purtell Assignee: Stephen Yuan Jiang Fix For: 2.0.0, 0.99.2 Attachments: HBASE-12128.v1-1.0.patch, HBASE-12128.v1-2.0.patch, HBASE-12128.v2-2.0.patch Original Estimate: 120h Time Spent: 72h Remaining Estimate: 48h Creating Table instances should be lightweight. Apps that manage their own Connections are expected to create Tables on demand for each interaction. However we look up values from Hadoop Configuration when constructing Table objects for storing to some of its fields. Configuration is a heavyweight registry that does a lot of string operations and regex matching. Method calls into Configuration account for 48.25% of CPU time when creating the HTable object in 0.98. Another ~48% of CPU is spent constructing the desired RpcController object via reflection in 0.98. Together this can account for ~20% of total on-CPU time of the client. See parent issue for more detail. We are using Connection like a factory for Table. We should cache configuration for Table in Connection. We should also create by reflection once and cache the desired RpcController object, and clone it for new Tables. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12514) Cleanup HFileOutputFormat legacy code
[
https://issues.apache.org/jira/browse/HBASE-12514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14225278#comment-14225278
]
Hadoop QA commented on HBASE-12514:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12683639/12514v2.txt
against master branch at commit e6b4300756b7f09a31ba35cb3baf41d294ed6e14.
ATTACHMENT ID: 12683639
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 7 new
or modified tests.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:red}-1 checkstyle{color}. The applied patch generated
3780 checkstyle errors (more than the master's current 3779 errors).
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/11834//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11834//artifact/patchprocess/newPatchFindbugsWarningshbase-rest.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11834//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11834//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11834//artifact/patchprocess/newPatchFindbugsWarningshbase-annotations.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11834//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11834//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11834//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11834//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11834//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11834//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/11834//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/11834//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/11834//console
This message is automatically generated.
Cleanup HFileOutputFormat legacy code
-
Key: HBASE-12514
URL: https://issues.apache.org/jira/browse/HBASE-12514
Project: HBase
Issue Type: Bug
Reporter: Solomon Duskis
Assignee: Solomon Duskis
Fix For: 2.0.0, 0.99.2
Attachments: 12514v2.txt, HBASE-12514 (1).patch, HBASE-12514.patch,
HBASE-12514.patch, HBASE-12514.patch
HFileOutputFormat methods route to their HFileOutputFormat2 counterparts.
Replace all calls to HFileOutputFormat with HFileOutputFormat2 equivalents.
In the spirit of cleanup, add @Override annotations and helper methods that
do not require the use of deprecated classes such as HTable.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (HBASE-11130) Add support for Master endpoint coprocessors
[ https://issues.apache.org/jira/browse/HBASE-11130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gary Helmling resolved HBASE-11130. --- Resolution: Duplicate This was already done long ago. Add support for Master endpoint coprocessors Key: HBASE-11130 URL: https://issues.apache.org/jira/browse/HBASE-11130 Project: HBase Issue Type: New Feature Components: Coprocessors, master Reporter: Gary Helmling Currently master coprocessors can only function as observers. However, it would be useful in some cases, especially with system tables moving to be served by the master, for master coprocessors to be able to function as endpoints. These coprocessors would then have access to master facilities to be able to perform custom cluster coordination tasks, for example. One example application of this would be for security grant and revoke commands, where a master coprocessor could make use of the procedure mechanism to ensure that all regionservers acknowledge an update before returning success to the client. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12404) Task 5 from parent: Replace internal HTable constructor use with HConnection#getTable (0.98, 0.99)
[ https://issues.apache.org/jira/browse/HBASE-12404?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14225338#comment-14225338 ] Enis Soztutar commented on HBASE-12404: --- Great. Thanks for the monumental effort Stack. Task 5 from parent: Replace internal HTable constructor use with HConnection#getTable (0.98, 0.99) -- Key: HBASE-12404 URL: https://issues.apache.org/jira/browse/HBASE-12404 Project: HBase Issue Type: Sub-task Reporter: stack Assignee: stack Fix For: 2.0.0, 0.99.2 Attachments: 0001-HBASE-12404-Task-5-from-parent-Replace-internal-HTab.patch, 12404.txt, 12404.v16.txt, 12404.v16.txt, 12404.v17.txt, 12404v10.txt, 12404v11.txt, 12404v12.txt, 12404v12.txt, 12404v13.txt, 12404v13.txt, 12404v14.txt, 12404v15.txt, 12404v18.txt, 12404v19.txt, 12404v19.txt, 12404v2.txt, 12404v20.txt, 12404v20.txt, 12404v3.txt, 12404v5.txt, 12404v6.txt, 12404v7.txt, 12404v9.txt Do the step 5. from the [~ndimiduk] list in parent issue. Go through src code and change all new HTable to instead be connection.getTable. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-12578) Change TokenProvider to a SingletonCoprocessorService
Gary Helmling created HBASE-12578:
-
Summary: Change TokenProvider to a SingletonCoprocessorService
Key: HBASE-12578
URL: https://issues.apache.org/jira/browse/HBASE-12578
Project: HBase
Issue Type: Improvement
Components: security
Reporter: Gary Helmling
The {{TokenProvider}} coprocessor service, which is responsible for issuing
HBase delegation tokens, currently runs a region endpoint. In the security
documentation, we recommend configuring this coprocessor for all table regions,
however, we only ever address delegation token requests to the META region.
When {{TokenProvider}} was first added, region coprocessors were the only way
of adding endpoints. But, since then, we've added support for endpoints for
regionserver and master coprocessors. This makes loading {{TokenProvider}} on
all table regions unnecessarily wasteful.
We can reduce the overhead for {{TokenProvider}} and greatly improve it's
scalability by doing the following:
# Convert {{TokenProvider}} to a {{SingletonCoprocessorService}} that is
configured to run on all regionservers. This will ensure a single instance per
regionserver instead of one per region.
# Direct delegation token requests to a random running regionserver so that we
don't hotspot any single instance with requests.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HBASE-12579) Move obtainAuthTokenForJob() methods out of User
Gary Helmling created HBASE-12579:
-
Summary: Move obtainAuthTokenForJob() methods out of User
Key: HBASE-12579
URL: https://issues.apache.org/jira/browse/HBASE-12579
Project: HBase
Issue Type: Improvement
Components: security
Reporter: Gary Helmling
The {{User}} class currently contains some utility methods to obtain HBase
authentication tokens for the given user. However, these methods initiate an
RPC to the {{TokenProvider}} coprocessor endpoint, an action which should not
be part of the User class' responsibilities.
This leads to a couple of problems:
# The way the methods are currently structured, it is impossible to integrate
them with normal connection management for the cluster (the TokenUtil class
constructs its own HTable instance internally).
# The User class is logically part of the hbase-common module, but uses the
TokenUtil class (part of hbase-server, though it should probably be moved to
hbase-client) through reflection, leading to a hidden dependency.
The {{obtainAuthTokenForJob()}} methods should be deprecated and the process of
obtaining authentication tokens should be moved to use the normal connection
lifecycle.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12493) User class should provide a way to re-use existing token
[ https://issues.apache.org/jira/browse/HBASE-12493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14225353#comment-14225353 ] Gary Helmling commented on HBASE-12493: --- Linking to additional refactorings inspired by this issue. User class should provide a way to re-use existing token Key: HBASE-12493 URL: https://issues.apache.org/jira/browse/HBASE-12493 Project: HBase Issue Type: Task Reporter: Brock Noland In HIVE-8874 we had to re-use HBase classes market private to re-use using tokens. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-12580) Zookeeper instantiated even though we might not need it in the shell
Alex Newman created HBASE-12580: --- Summary: Zookeeper instantiated even though we might not need it in the shell Key: HBASE-12580 URL: https://issues.apache.org/jira/browse/HBASE-12580 Project: HBase Issue Type: Bug Reporter: Alex Newman -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-12580) Zookeeper instantiated even though we might not need it in the shell
[ https://issues.apache.org/jira/browse/HBASE-12580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Newman updated HBASE-12580: I wrote an alternative registry and connection manager which does not use zookeeper. However the shell still wants to connect to zookeeper NO MATTER WHAT. I can see it for supporting zk_dump but, for some reason even though the shell commands succeed, I get errors scrolling about why it can't attach to zookeeper Zookeeper instantiated even though we might not need it in the shell Key: HBASE-12580 URL: https://issues.apache.org/jira/browse/HBASE-12580 Project: HBase Issue Type: Bug Reporter: Alex Newman -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12580) Zookeeper instantiated even though we might not need it in the shell
[ https://issues.apache.org/jira/browse/HBASE-12580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14225361#comment-14225361 ] Alex Newman commented on HBASE-12580: - This seems to be the offender. Why do we need our own watcher? def initialize(configuration, formatter) @admin = org.apache.hadoop.hbase.client.HBaseAdmin.new(configuration) connection = @admin.getConnection() @conf = configuration @zk_wrapper = org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.new(configuration, admin, nil) zk = @zk_wrapper.getRecoverableZooKeeper().getZooKeeper() @zk_main = org.apache.zookeeper.ZooKeeperMain.new(zk) @formatter = formatter end Zookeeper instantiated even though we might not need it in the shell Key: HBASE-12580 URL: https://issues.apache.org/jira/browse/HBASE-12580 Project: HBase Issue Type: Bug Reporter: Alex Newman -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (HBASE-12580) Zookeeper instantiated even though we might not need it in the shell
[ https://issues.apache.org/jira/browse/HBASE-12580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Newman reassigned HBASE-12580: --- Assignee: Alex Newman Zookeeper instantiated even though we might not need it in the shell Key: HBASE-12580 URL: https://issues.apache.org/jira/browse/HBASE-12580 Project: HBase Issue Type: Bug Reporter: Alex Newman Assignee: Alex Newman -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-12580) Zookeeper instantiated even though we might not need it in the shell
[ https://issues.apache.org/jira/browse/HBASE-12580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Newman updated HBASE-12580: Description: I wrote an alternative registry and connection manager which does not use zookeeper. However the shell still wants to connect to zookeeper NO MATTER WHAT. I can see it for supporting zk_dump but, for some reason even though the shell commands succeed, I get errors scrolling about why it can't attach to zookeeper Zookeeper instantiated even though we might not need it in the shell Key: HBASE-12580 URL: https://issues.apache.org/jira/browse/HBASE-12580 Project: HBase Issue Type: Bug Reporter: Alex Newman Assignee: Alex Newman I wrote an alternative registry and connection manager which does not use zookeeper. However the shell still wants to connect to zookeeper NO MATTER WHAT. I can see it for supporting zk_dump but, for some reason even though the shell commands succeed, I get errors scrolling about why it can't attach to zookeeper -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-12580) Zookeeper instantiated even though we might not need it in the shell
[ https://issues.apache.org/jira/browse/HBASE-12580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Newman updated HBASE-12580: Affects Version/s: 0.98.8 Zookeeper instantiated even though we might not need it in the shell Key: HBASE-12580 URL: https://issues.apache.org/jira/browse/HBASE-12580 Project: HBase Issue Type: Bug Affects Versions: 0.98.8 Reporter: Alex Newman Assignee: Alex Newman I wrote an alternative registry and connection manager which does not use zookeeper. However the shell still wants to connect to zookeeper NO MATTER WHAT. I can see it for supporting zk_dump but, for some reason even though the shell commands succeed, I get errors scrolling about why it can't attach to zookeeper -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12493) User class should provide a way to re-use existing token
[ https://issues.apache.org/jira/browse/HBASE-12493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14225372#comment-14225372 ] Gary Helmling commented on HBASE-12493: --- Looking at this issue in more detail, there are a few things wrong with our current means of obtaining authentication tokens: * The User class winds up initiating RPCs, which it really shouldn't * There's no way to do normal connection management for the connections/resources used by those RPCs * The TokenProvider coprocessor setup on the server-side is wasteful I've linked to a couple of issues I've created for the related problems. For this issue, though, we can still address the problem without changing the User API: # Make both User and TokenUtil classes public # Move TokenUtil from hbase-server to hbase-client (which seems a more natural place for it) # Add a method to TokenUtil to fetch a token for the given user if it is not already present in the user credentials # Deprecate the User.obtainAuthTokenForJob() methods in favor of the TokenUtil.obtainTokenForJob() equivalents Then, at some point in the future, we can remove the User.obtainAuthTokenForJob() methods entirely. User class should provide a way to re-use existing token Key: HBASE-12493 URL: https://issues.apache.org/jira/browse/HBASE-12493 Project: HBase Issue Type: Task Reporter: Brock Noland In HIVE-8874 we had to re-use HBase classes market private to re-use using tokens. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12580) Zookeeper instantiated even though we might not need it in the shell
[ https://issues.apache.org/jira/browse/HBASE-12580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14225383#comment-14225383 ] stack commented on HBASE-12580: --- Did you use http://hbase.apache.org/xref/org/apache/hadoop/hbase/client/Registry.html... Yeah, have the shell use the Interface? Zookeeper instantiated even though we might not need it in the shell Key: HBASE-12580 URL: https://issues.apache.org/jira/browse/HBASE-12580 Project: HBase Issue Type: Bug Affects Versions: 0.98.8 Reporter: Alex Newman Assignee: Alex Newman I wrote an alternative registry and connection manager which does not use zookeeper. However the shell still wants to connect to zookeeper NO MATTER WHAT. I can see it for supporting zk_dump but, for some reason even though the shell commands succeed, I get errors scrolling about why it can't attach to zookeeper -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12580) Zookeeper instantiated even though we might not need it in the shell
[ https://issues.apache.org/jira/browse/HBASE-12580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14225391#comment-14225391 ] Alex Newman commented on HBASE-12580: - I did create a custom registry. I'd like to point out the code above directly instantiates a zookeeper watcher, even though it may never use it. Zookeeper instantiated even though we might not need it in the shell Key: HBASE-12580 URL: https://issues.apache.org/jira/browse/HBASE-12580 Project: HBase Issue Type: Bug Affects Versions: 0.98.8 Reporter: Alex Newman Assignee: Alex Newman I wrote an alternative registry and connection manager which does not use zookeeper. However the shell still wants to connect to zookeeper NO MATTER WHAT. I can see it for supporting zk_dump but, for some reason even though the shell commands succeed, I get errors scrolling about why it can't attach to zookeeper -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12580) Zookeeper instantiated even though we might not need it in the shell
[ https://issues.apache.org/jira/browse/HBASE-12580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14225392#comment-14225392 ] Alex Newman commented on HBASE-12580: - Much like how the catalog tracker also directly ignores it. Zookeeper instantiated even though we might not need it in the shell Key: HBASE-12580 URL: https://issues.apache.org/jira/browse/HBASE-12580 Project: HBase Issue Type: Bug Affects Versions: 0.98.8 Reporter: Alex Newman Assignee: Alex Newman I wrote an alternative registry and connection manager which does not use zookeeper. However the shell still wants to connect to zookeeper NO MATTER WHAT. I can see it for supporting zk_dump but, for some reason even though the shell commands succeed, I get errors scrolling about why it can't attach to zookeeper -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12580) Zookeeper instantiated even though we might not need it in the shell
[ https://issues.apache.org/jira/browse/HBASE-12580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14225394#comment-14225394 ] stack commented on HBASE-12580: --- Yeah, as you said. It needs to be taught about the Registry Zookeeper instantiated even though we might not need it in the shell Key: HBASE-12580 URL: https://issues.apache.org/jira/browse/HBASE-12580 Project: HBase Issue Type: Bug Affects Versions: 0.98.8 Reporter: Alex Newman Assignee: Alex Newman I wrote an alternative registry and connection manager which does not use zookeeper. However the shell still wants to connect to zookeeper NO MATTER WHAT. I can see it for supporting zk_dump but, for some reason even though the shell commands succeed, I get errors scrolling about why it can't attach to zookeeper -- This message was sent by Atlassian JIRA (v6.3.4#6332)
