[jira] [Commented] (HBASE-12953) RegionServer is not functionally working with AysncRpcClient in secure mode
[
https://issues.apache.org/jira/browse/HBASE-12953?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14328815#comment-14328815
]
Andrey Stepachev commented on HBASE-12953:
--
[~Apache9] it seems so. whenever region is AssignmentManager#onRegionClosed
assignment manager checks table state and found that state is ENABLED and so it
invokes assign once more. Still investigating why that happens, but definitely
that is unrelated problem.
RegionServer is not functionally working with AysncRpcClient in secure mode
---
Key: HBASE-12953
URL: https://issues.apache.org/jira/browse/HBASE-12953
Project: HBase
Issue Type: Bug
Components: security
Affects Versions: 2.0.0, 1.1.0
Reporter: Ashish Singhi
Assignee: zhangduo
Priority: Critical
Fix For: 2.0.0, 1.1.0
Attachments: HBASE-12953.patch, HBASE-12953_1.patch,
HBASE-12953_2.patch, HBASE-12953_2.patch, HBASE-12953_2.patch,
HBASE-12953_2.patch, HBASE-12953_2.patch, HBASE-12953_3 (2).patch,
HBASE-12953_3 (2).patch, HBASE-12953_3.patch, HBASE-12953_3.patch,
HBASE-12953_3.patch, testcase.patch
HBase version 2.0.0
Default value for {{hbase.rpc.client.impl}} is set to AsyncRpcClient.
When trying to install HBase with Kerberos, RegionServer is not working
functionally.
The following log is logged in its log file
{noformat}
2015-02-02 14:59:05,407 WARN [AsyncRpcChannel-pool1-t1]
channel.DefaultChannelPipeline: An exceptionCaught() event was fired, and it
reached at the tail of the pipeline. It usually means the last handler in the
pipeline did not handle the exception.
io.netty.channel.ChannelPipelineException:
org.apache.hadoop.hbase.security.SaslClientHandler.handlerAdded() has thrown
an exception; removed.
at
io.netty.channel.DefaultChannelPipeline.callHandlerAdded0(DefaultChannelPipeline.java:499)
at
io.netty.channel.DefaultChannelPipeline.callHandlerAdded(DefaultChannelPipeline.java:481)
at
io.netty.channel.DefaultChannelPipeline.addFirst0(DefaultChannelPipeline.java:114)
at
io.netty.channel.DefaultChannelPipeline.addFirst(DefaultChannelPipeline.java:97)
at
io.netty.channel.DefaultChannelPipeline.addFirst(DefaultChannelPipeline.java:235)
at
io.netty.channel.DefaultChannelPipeline.addFirst(DefaultChannelPipeline.java:214)
at
org.apache.hadoop.hbase.ipc.AsyncRpcChannel$2.operationComplete(AsyncRpcChannel.java:194)
at
org.apache.hadoop.hbase.ipc.AsyncRpcChannel$2.operationComplete(AsyncRpcChannel.java:157)
at
io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:680)
at
io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:603)
at
io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:563)
at
io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:406)
at
io.netty.channel.DefaultChannelPromise.trySuccess(DefaultChannelPromise.java:82)
at
io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.fulfillConnectPromise(AbstractNioChannel.java:253)
at
io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.finishConnect(AbstractNioChannel.java:288)
at
io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:528)
at
io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:468)
at
io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:382)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:354)
at
io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:116)
at java.lang.Thread.run(Thread.java:745)
Caused by: javax.security.sasl.SaslException: GSS initiate failed [Caused by
GSSException: No valid credentials provided (Mechanism level: Failed to find
any Kerberos tgt)]
at
com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:212)
at
org.apache.hadoop.hbase.security.SaslClientHandler.handlerAdded(SaslClientHandler.java:154)
at
io.netty.channel.DefaultChannelPipeline.callHandlerAdded0(DefaultChannelPipeline.java:486)
... 20 more
Caused by: GSSException: No valid credentials provided (Mechanism level:
Failed to find any Kerberos tgt)
at
sun.security.jgss.krb5.Krb5InitCredential.getInstance(Krb5InitCredential.java:147)
at
sun.security.jgss.krb5.Krb5MechFactory.getCredentialElement(Krb5MechFactory.java:121)
at
sun.security.jgss.krb5.Krb5MechFactory.getMechanismContext(Krb5MechFactory.java:187)
at
sun.security.jgss.GSSManagerImpl.getMechanismContext(GSSManagerImpl.java:223)
at
[jira] [Updated] (HBASE-13054) Provide more tracing information for locking/latching events.
[ https://issues.apache.org/jira/browse/HBASE-13054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajeshbabu Chintaguntla updated HBASE-13054: Attachment: HBASE-13054_v3.patch Thanks for review [~eclark]. Here is the patch addressing the comments. If it's ok I will commit. Provide more tracing information for locking/latching events. - Key: HBASE-13054 URL: https://issues.apache.org/jira/browse/HBASE-13054 Project: HBase Issue Type: Improvement Reporter: Rajeshbabu Chintaguntla Assignee: Rajeshbabu Chintaguntla Fix For: 2.0.0, 1.0.1, 1.1.0, 0.98.11 Attachments: HBASE-13054.patch, HBASE-13054_v2.patch, HBASE-13054_v3.patch Currently not much tracing information available for locking and latching events like row level locking during do mini batch mutations, region level locking during flush, close and so on. It will be better to add the trace information for such events so that it will be useful for finding time spent on locking and waiting time on locks while analyzing performance issues in queries using trace information. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-13076) Region can be openend after it was closed by DisableTableHandler
[ https://issues.apache.org/jira/browse/HBASE-13076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrey Stepachev updated HBASE-13076: - Description: Got situation where region can be opened while table is disabling by DisableTableHandler. Here is relevant log for such situation. There is no clues who issued OPEN to region. Log file attached. A bit more details. It seems that even in case of new state put into meta, it still possible to get previous state. That leads to one more round of assignment invoked in AssignmentManager#onRegionClosed. was: Got situation where region can be opened while table is disabling by DisableTableHandler. Here is relevant log for such situation. There is no clues who issued OPEN to region. Log file attached. Region can be openend after it was closed by DisableTableHandler Key: HBASE-13076 URL: https://issues.apache.org/jira/browse/HBASE-13076 Project: HBase Issue Type: Bug Components: master, Region Assignment Affects Versions: 2.0.0 Reporter: Andrey Stepachev Assignee: Andrey Stepachev Attachments: 23757f039d83f4f17ca18815eae70b28.log Got situation where region can be opened while table is disabling by DisableTableHandler. Here is relevant log for such situation. There is no clues who issued OPEN to region. Log file attached. A bit more details. It seems that even in case of new state put into meta, it still possible to get previous state. That leads to one more round of assignment invoked in AssignmentManager#onRegionClosed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12953) RegionServer is not functionally working with AysncRpcClient in secure mode
[
https://issues.apache.org/jira/browse/HBASE-12953?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14328852#comment-14328852
]
zhangduo commented on HBASE-12953:
--
[~octo47] Thanks for finding this.
[~stack] Seems TestShell failed twice, is this a known problem?
RegionServer is not functionally working with AysncRpcClient in secure mode
---
Key: HBASE-12953
URL: https://issues.apache.org/jira/browse/HBASE-12953
Project: HBase
Issue Type: Bug
Components: security
Affects Versions: 2.0.0, 1.1.0
Reporter: Ashish Singhi
Assignee: zhangduo
Priority: Critical
Fix For: 2.0.0, 1.1.0
Attachments: HBASE-12953.patch, HBASE-12953_1.patch,
HBASE-12953_2.patch, HBASE-12953_2.patch, HBASE-12953_2.patch,
HBASE-12953_2.patch, HBASE-12953_2.patch, HBASE-12953_3 (2).patch,
HBASE-12953_3 (2).patch, HBASE-12953_3.patch, HBASE-12953_3.patch,
HBASE-12953_3.patch, testcase.patch
HBase version 2.0.0
Default value for {{hbase.rpc.client.impl}} is set to AsyncRpcClient.
When trying to install HBase with Kerberos, RegionServer is not working
functionally.
The following log is logged in its log file
{noformat}
2015-02-02 14:59:05,407 WARN [AsyncRpcChannel-pool1-t1]
channel.DefaultChannelPipeline: An exceptionCaught() event was fired, and it
reached at the tail of the pipeline. It usually means the last handler in the
pipeline did not handle the exception.
io.netty.channel.ChannelPipelineException:
org.apache.hadoop.hbase.security.SaslClientHandler.handlerAdded() has thrown
an exception; removed.
at
io.netty.channel.DefaultChannelPipeline.callHandlerAdded0(DefaultChannelPipeline.java:499)
at
io.netty.channel.DefaultChannelPipeline.callHandlerAdded(DefaultChannelPipeline.java:481)
at
io.netty.channel.DefaultChannelPipeline.addFirst0(DefaultChannelPipeline.java:114)
at
io.netty.channel.DefaultChannelPipeline.addFirst(DefaultChannelPipeline.java:97)
at
io.netty.channel.DefaultChannelPipeline.addFirst(DefaultChannelPipeline.java:235)
at
io.netty.channel.DefaultChannelPipeline.addFirst(DefaultChannelPipeline.java:214)
at
org.apache.hadoop.hbase.ipc.AsyncRpcChannel$2.operationComplete(AsyncRpcChannel.java:194)
at
org.apache.hadoop.hbase.ipc.AsyncRpcChannel$2.operationComplete(AsyncRpcChannel.java:157)
at
io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:680)
at
io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:603)
at
io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:563)
at
io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:406)
at
io.netty.channel.DefaultChannelPromise.trySuccess(DefaultChannelPromise.java:82)
at
io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.fulfillConnectPromise(AbstractNioChannel.java:253)
at
io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.finishConnect(AbstractNioChannel.java:288)
at
io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:528)
at
io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:468)
at
io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:382)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:354)
at
io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:116)
at java.lang.Thread.run(Thread.java:745)
Caused by: javax.security.sasl.SaslException: GSS initiate failed [Caused by
GSSException: No valid credentials provided (Mechanism level: Failed to find
any Kerberos tgt)]
at
com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:212)
at
org.apache.hadoop.hbase.security.SaslClientHandler.handlerAdded(SaslClientHandler.java:154)
at
io.netty.channel.DefaultChannelPipeline.callHandlerAdded0(DefaultChannelPipeline.java:486)
... 20 more
Caused by: GSSException: No valid credentials provided (Mechanism level:
Failed to find any Kerberos tgt)
at
sun.security.jgss.krb5.Krb5InitCredential.getInstance(Krb5InitCredential.java:147)
at
sun.security.jgss.krb5.Krb5MechFactory.getCredentialElement(Krb5MechFactory.java:121)
at
sun.security.jgss.krb5.Krb5MechFactory.getMechanismContext(Krb5MechFactory.java:187)
at
sun.security.jgss.GSSManagerImpl.getMechanismContext(GSSManagerImpl.java:223)
at
sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:212)
at
sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:179)
at
[jira] [Updated] (HBASE-13080) Hbase shell message containing extra quote at the end of error message.
[
https://issues.apache.org/jira/browse/HBASE-13080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Abhishek Kumar updated HBASE-13080:
---
Description:
{noformat}
hbase(main):001:0 drop 't2'
ERROR: Table t2 is enabled. Disable it first.'
---
hbase(main):001:0 disable 'noTable'
ERROR: Table csadfsdfsdf does not exist.'
{noformat}
extra quote at the end odf error message seems to be a typo, can be removed
from below two methods in admin.rb :
{noformat}
# Throw exception if table doesn't exist
def tableExists(table_name)
raise ArgumentError, Table #{table_name} does not exist.' unless
exists?(table_name)
end
# Drops a table
def drop(table_name)
tableExists(table_name)
raise ArgumentError, Table #{table_name} is enabled. Disable it first.'
if enabled?(table_name)
{noformat}
was:
{noformat}
hbase(main):001:0 drop 't2'
ERROR: Table t2 is enabled. Disable it first.*'*
---
hbase(main):001:0 disable 'noTable'
ERROR: Table csadfsdfsdf does not exist.*'*
{noformat}
extra quote at the end odf error message seems to be a typo, can be removed
from below two methods in admin.rb :
{noformat}
# Throw exception if table doesn't exist
def tableExists(table_name)
raise ArgumentError, Table #{table_name} does not exist.' unless
exists?(table_name)
end
# Drops a table
def drop(table_name)
tableExists(table_name)
raise ArgumentError, Table #{table_name} is enabled. Disable it first.'
if enabled?(table_name)
{noformat}
Hbase shell message containing extra quote at the end of error message.
---
Key: HBASE-13080
URL: https://issues.apache.org/jira/browse/HBASE-13080
Project: HBase
Issue Type: Improvement
Reporter: Abhishek Kumar
Priority: Trivial
{noformat}
hbase(main):001:0 drop 't2'
ERROR: Table t2 is enabled. Disable it first.'
---
hbase(main):001:0 disable 'noTable'
ERROR: Table csadfsdfsdf does not exist.'
{noformat}
extra quote at the end odf error message seems to be a typo, can be removed
from below two methods in admin.rb :
{noformat}
# Throw exception if table doesn't exist
def tableExists(table_name)
raise ArgumentError, Table #{table_name} does not exist.' unless
exists?(table_name)
end
# Drops a table
def drop(table_name)
tableExists(table_name)
raise ArgumentError, Table #{table_name} is enabled. Disable it
first.' if enabled?(table_name)
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13070) Fix TestCacheOnWrite
[
https://issues.apache.org/jira/browse/HBASE-13070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14328721#comment-14328721
]
Hadoop QA commented on HBASE-13070:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12699829/HBASE-13070_2.patch
against master branch at commit 03d8918142681d4c8abe40e8c8fb32307756d8a8.
ATTACHMENT ID: 12699829
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 4 new
or modified tests.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.1 2.5.2 2.6.0)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 checkstyle{color}. The applied patch does not increase the
total number of checkstyle errors
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.master.TestDistributedLogSplitting
{color:red}-1 core zombie tests{color}. There are 1 zombie test(s):
at
org.apache.hadoop.hbase.coprocessor.TestCoprocessorEndpoint.testCoprocessorError(TestCoprocessorEndpoint.java:310)
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/12924//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12924//artifact/patchprocess/newPatchFindbugsWarningshbase-rest.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12924//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12924//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12924//artifact/patchprocess/newPatchFindbugsWarningshbase-annotations.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12924//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12924//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12924//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12924//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12924//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12924//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12924//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/12924//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/12924//console
This message is automatically generated.
Fix TestCacheOnWrite
Key: HBASE-13070
URL: https://issues.apache.org/jira/browse/HBASE-13070
Project: HBase
Issue Type: Bug
Components: test
Affects Versions: 2.0.0
Reporter: zhangduo
Assignee: zhangduo
Attachments: HBASE-13070.patch, HBASE-13070_1.patch,
HBASE-13070_2.patch
TestCacheOnWrite uses TestHFileWriterV2.randomOrderedKey to generate a random
byte array, then use first 32 bytes as row and remaining part as family and
qualifier. But TestHFileWriterV2.randomOrderedKey may return a byte array
only contains 32 bytes, so there will be zero length family and qualifier.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HBASE-13080) Hbase shell message containing extra quote at the end of error message.
Abhishek Kumar created HBASE-13080:
--
Summary: Hbase shell message containing extra quote at the end of
error message.
Key: HBASE-13080
URL: https://issues.apache.org/jira/browse/HBASE-13080
Project: HBase
Issue Type: Improvement
Reporter: Abhishek Kumar
Priority: Trivial
{noformat}
hbase(main):001:0 drop 't2'
ERROR: Table t2 is enabled. Disable it first.*'*
---
hbase(main):001:0 disable 'noTable'
ERROR: Table csadfsdfsdf does not exist.*'*
{noformat}
extra quote at the end odf error message seems to be a typo, can be removed
from below two methods in admin.rb :
{noformat}
# Throw exception if table doesn't exist
def tableExists(table_name)
raise ArgumentError, Table #{table_name} does not exist.' unless
exists?(table_name)
end
# Drops a table
def drop(table_name)
tableExists(table_name)
raise ArgumentError, Table #{table_name} is enabled. Disable it first.'
if enabled?(table_name)
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13077) BoundedCompletionService doesn't pass trace info to server
[ https://issues.apache.org/jira/browse/HBASE-13077?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14328701#comment-14328701 ] Enis Soztutar commented on HBASE-13077: --- +1. So this means that tracing is not working in branch-1.0 for scans? Not sure that is a sinker. What do you guys think? BoundedCompletionService doesn't pass trace info to server -- Key: HBASE-13077 URL: https://issues.apache.org/jira/browse/HBASE-13077 Project: HBase Issue Type: Bug Components: hbase Affects Versions: 1.0.0, 2.0.0, 1.1.0 Reporter: Jeffrey Zhong Assignee: Jeffrey Zhong Attachments: HBASE-13077.patch Today [~ndimiduk] I found that BoundedCompletionService doesn't pass htrace info to server. This issue causes scan doesn't pass trace info to server. [~enis] FYI. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13070) Fix TestCacheOnWrite
[
https://issues.apache.org/jira/browse/HBASE-13070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14328707#comment-14328707
]
Hadoop QA commented on HBASE-13070:
---
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12699823/HBASE-13070_1.patch
against master branch at commit 03d8918142681d4c8abe40e8c8fb32307756d8a8.
ATTACHMENT ID: 12699823
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 4 new
or modified tests.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.1 2.5.2 2.6.0)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 checkstyle{color}. The applied patch does not increase the
total number of checkstyle errors
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/12923//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12923//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12923//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12923//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12923//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12923//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12923//artifact/patchprocess/newPatchFindbugsWarningshbase-rest.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12923//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12923//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12923//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12923//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12923//artifact/patchprocess/newPatchFindbugsWarningshbase-annotations.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/12923//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/12923//console
This message is automatically generated.
Fix TestCacheOnWrite
Key: HBASE-13070
URL: https://issues.apache.org/jira/browse/HBASE-13070
Project: HBase
Issue Type: Bug
Components: test
Affects Versions: 2.0.0
Reporter: zhangduo
Assignee: zhangduo
Attachments: HBASE-13070.patch, HBASE-13070_1.patch,
HBASE-13070_2.patch
TestCacheOnWrite uses TestHFileWriterV2.randomOrderedKey to generate a random
byte array, then use first 32 bytes as row and remaining part as family and
qualifier. But TestHFileWriterV2.randomOrderedKey may return a byte array
only contains 32 bytes, so there will be zero length family and qualifier.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-13076) Table can be forcibly enabled in AssignmentManager during table disabling.
[ https://issues.apache.org/jira/browse/HBASE-13076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrey Stepachev updated HBASE-13076: - Description: Got situation where region can be opened while table is disabling by DisableTableHandler. Here is relevant log for such situation. There is no clues who issued OPEN to region. Log file attached. UPD: A bit more details. It seems that even in case of new state put into meta, it still possible to get previous state. That leads to one more round of assignment invoked in AssignmentManager#onRegionClosed. UPD: Table become ENABLED, thats leads to regions instructed to assign immediately on onRegionClosed. BulkDisabler will not know about that and will wait indefinitely, because it will not issue unassign for newly opened regions. was: Got situation where region can be opened while table is disabling by DisableTableHandler. Here is relevant log for such situation. There is no clues who issued OPEN to region. Log file attached. A bit more details. It seems that even in case of new state put into meta, it still possible to get previous state. That leads to one more round of assignment invoked in AssignmentManager#onRegionClosed. Table can be forcibly enabled in AssignmentManager during table disabling. -- Key: HBASE-13076 URL: https://issues.apache.org/jira/browse/HBASE-13076 Project: HBase Issue Type: Bug Components: master, Region Assignment Affects Versions: 2.0.0 Reporter: Andrey Stepachev Assignee: Andrey Stepachev Attachments: 23757f039d83f4f17ca18815eae70b28.log Got situation where region can be opened while table is disabling by DisableTableHandler. Here is relevant log for such situation. There is no clues who issued OPEN to region. Log file attached. UPD: A bit more details. It seems that even in case of new state put into meta, it still possible to get previous state. That leads to one more round of assignment invoked in AssignmentManager#onRegionClosed. UPD: Table become ENABLED, thats leads to regions instructed to assign immediately on onRegionClosed. BulkDisabler will not know about that and will wait indefinitely, because it will not issue unassign for newly opened regions. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12953) RegionServer is not functionally working with AysncRpcClient in secure mode
[
https://issues.apache.org/jira/browse/HBASE-12953?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14328872#comment-14328872
]
zhangduo commented on HBASE-12953:
--
https://builds.apache.org/job/PreCommit-HBASE-Build/12916/artifact/hbase-shell/target/surefire-reports/org.apache.hadoop.hbase.client.TestShell.txt
{noformat}
---
Test set: org.apache.hadoop.hbase.client.TestShell
---
Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 86.698 sec
FAILURE! - in org.apache.hadoop.hbase.client.TestShell
testRunShellTests(org.apache.hadoop.hbase.client.TestShell) Time elapsed: 0
sec ERROR!
org.jruby.embed.EvalFailedException: (RuntimeError)
test_Hbase::Table_constructor_should_not_fail_for_existent_tables is already
defined in Hbase::TableConstructorTest
at
org.jruby.embed.internal.EmbedEvalUnitImpl.run(EmbedEvalUnitImpl.java:136)
at
org.jruby.embed.ScriptingContainer.runUnit(ScriptingContainer.java:1263)
at
org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:1308)
at
org.apache.hadoop.hbase.client.TestShell.testRunShellTests(TestShell.java:81)
Caused by: org.jruby.exceptions.RaiseException: (RuntimeError)
test_Hbase::Table_constructor_should_not_fail_for_existent_tables is already
defined in Hbase::TableConstructorTest
at
Declarative.define_test(/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/hbase-shell/src/test/ruby/test_helper.rb:26)
at
(Anonymous).TableConstructorTest(./src/test/ruby/hbase/table_test.rb:36)
at Module.Hbase(./src/test/ruby/hbase/table_test.rb:26)
at (Anonymous).(root)(./src/test/ruby/hbase/table_test.rb:24)
at org.jruby.RubyKernel.load(org/jruby/RubyKernel.java:1087)
at (Anonymous).(root)(./src/test/ruby/hbase/table_test.rb:50)
at org.jruby.RubyArray.each(org/jruby/RubyArray.java:1620)
at (Anonymous).(root)(src/test/ruby/tests_runner.rb:48)
{noformat}
Does anyone know why this happen? Seems a test running twice?
https://builds.apache.org/job/PreCommit-HBASE-Build/12918/artifact/hbase-shell/target/surefire-reports/org.apache.hadoop.hbase.client.TestShell-output.txt
{noformat}
1) Error:
test_The_get/put_methods_should_work_for_data_written_with_Visibility(Hbase::VisibilityLabelsAdminMethodsTest):
ArgumentError: org.apache.hadoop.hbase.DoNotRetryIOException:
org.apache.hadoop.hbase.security.visibility.InvalidLabelException: Label
'TEST_VISIBILITY' doesn't exists
at
org.apache.hadoop.hbase.security.visibility.VisibilityController.setAuths(VisibilityController.java:808)
at
org.apache.hadoop.hbase.protobuf.generated.VisibilityLabelsProtos$VisibilityLabelsService$1.setAuths(VisibilityLabelsProtos.java:6036)
at
org.apache.hadoop.hbase.protobuf.generated.VisibilityLabelsProtos$VisibilityLabelsService.callMethod(VisibilityLabelsProtos.java:6219)
at
org.apache.hadoop.hbase.regionserver.HRegion.execService(HRegion.java:6867)
at
org.apache.hadoop.hbase.regionserver.RSRpcServices.execServiceOnRegion(RSRpcServices.java:1707)
at
org.apache.hadoop.hbase.regionserver.RSRpcServices.execService(RSRpcServices.java:1689)
at
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31309)
at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2038)
at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
at
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
at java.lang.Thread.run(Thread.java:744)
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/hbase-shell/src/main/ruby/hbase/visibility_labels.rb:84:in
`set_auths'
./src/test/ruby/hbase/visibility_labels_admin_test.rb:77:in
`test_The_get/put_methods_should_work_for_data_written_with_Visibility'
org/jruby/RubyProc.java:270:in `call'
org/jruby/RubyKernel.java:2105:in `send'
org/jruby/RubyArray.java:1620:in `each'
org/jruby/RubyArray.java:1620:in `each'
2) Error:
test_The_set/clear_methods_should_work_with_authorizations(Hbase::VisibilityLabelsAdminMethodsTest):
ArgumentError: No authentication set for the given user jenkins
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/hbase-shell/src/main/ruby/hbase/visibility_labels.rb:97:in
`get_auths'
./src/test/ruby/hbase/visibility_labels_admin_test.rb:57:in
`test_The_set/clear_methods_should_work_with_authorizations'
org/jruby/RubyProc.java:270:in `call'
org/jruby/RubyKernel.java:2105:in `send'
org/jruby/RubyArray.java:1620:in `each'
org/jruby/RubyArray.java:1620:in `each'
{noformat}
This seems
[jira] [Updated] (HBASE-13076) Table can be forcibly enabled in AssignmentManager during table disabling.
[ https://issues.apache.org/jira/browse/HBASE-13076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrey Stepachev updated HBASE-13076: - Status: Patch Available (was: Open) Table can be forcibly enabled in AssignmentManager during table disabling. -- Key: HBASE-13076 URL: https://issues.apache.org/jira/browse/HBASE-13076 Project: HBase Issue Type: Bug Components: master, Region Assignment Affects Versions: 2.0.0 Reporter: Andrey Stepachev Assignee: Andrey Stepachev Attachments: 23757f039d83f4f17ca18815eae70b28.log, HBASE-13076.patch Got situation where region can be opened while table is disabling by DisableTableHandler. Here is relevant log for such situation. There is no clues who issued OPEN to region. Log file attached. UPD: A bit more details. It seems that even in case of new state put into meta, it still possible to get previous state. That leads to one more round of assignment invoked in AssignmentManager#onRegionClosed. UPD: Table become ENABLED, thats leads to regions instructed to assign immediately on onRegionClosed. BulkDisabler will not know about that and will wait indefinitely, because it will not issue unassign for newly opened regions. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-5238) Add a log4j category for all edits to META/ROOT
[ https://issues.apache.org/jira/browse/HBASE-5238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrey Stepachev updated HBASE-5238: Attachment: HBASE-5238.patch here is small patch. mutations are logged in json without actual data. Add a log4j category for all edits to META/ROOT --- Key: HBASE-5238 URL: https://issues.apache.org/jira/browse/HBASE-5238 Project: HBase Issue Type: New Feature Components: regionserver Affects Versions: 2.0.0 Reporter: Todd Lipcon Priority: Minor Labels: beginner Attachments: HBASE-5238.patch Occasionally we run into bugs that have corrected META and written some bad data to META/ROOT but it's difficult to understand the order in which things happened. One option is to dump the HLog contents from the servers that hosted META at that time, but then it's interspersed with all other data. It would be nice to add a Log4j Logger to which we log all edits being applied to META and ROOT in textual form at DEBUG level. Then it would be easier to do a cluster-wide log grep to see what happened when. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-5238) Add a log4j category for all edits to META/ROOT
[ https://issues.apache.org/jira/browse/HBASE-5238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrey Stepachev updated HBASE-5238: Assignee: Andrey Stepachev Affects Version/s: 2.0.0 Status: Patch Available (was: Open) Add a log4j category for all edits to META/ROOT --- Key: HBASE-5238 URL: https://issues.apache.org/jira/browse/HBASE-5238 Project: HBase Issue Type: New Feature Components: regionserver Affects Versions: 2.0.0 Reporter: Todd Lipcon Assignee: Andrey Stepachev Priority: Minor Labels: beginner Attachments: HBASE-5238.patch Occasionally we run into bugs that have corrected META and written some bad data to META/ROOT but it's difficult to understand the order in which things happened. One option is to dump the HLog contents from the servers that hosted META at that time, but then it's interspersed with all other data. It would be nice to add a Log4j Logger to which we log all edits being applied to META and ROOT in textual form at DEBUG level. Then it would be easier to do a cluster-wide log grep to see what happened when. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-13076) Table can be forcibly enabled in AssignmentManager during table disabling.
[ https://issues.apache.org/jira/browse/HBASE-13076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrey Stepachev updated HBASE-13076: - Summary: Table can be forcibly enabled in AssignmentManager during table disabling. (was: Region can be openend after it was closed by DisableTableHandler) Table can be forcibly enabled in AssignmentManager during table disabling. -- Key: HBASE-13076 URL: https://issues.apache.org/jira/browse/HBASE-13076 Project: HBase Issue Type: Bug Components: master, Region Assignment Affects Versions: 2.0.0 Reporter: Andrey Stepachev Assignee: Andrey Stepachev Attachments: 23757f039d83f4f17ca18815eae70b28.log Got situation where region can be opened while table is disabling by DisableTableHandler. Here is relevant log for such situation. There is no clues who issued OPEN to region. Log file attached. A bit more details. It seems that even in case of new state put into meta, it still possible to get previous state. That leads to one more round of assignment invoked in AssignmentManager#onRegionClosed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-13076) Table can be forcibly enabled in AssignmentManager during table disabling.
[ https://issues.apache.org/jira/browse/HBASE-13076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrey Stepachev updated HBASE-13076: - Attachment: HBASE-13076.patch Lets try to remove logic which lasts from ZK based table states. Now it should not be necessary to have such code. Table can be forcibly enabled in AssignmentManager during table disabling. -- Key: HBASE-13076 URL: https://issues.apache.org/jira/browse/HBASE-13076 Project: HBase Issue Type: Bug Components: master, Region Assignment Affects Versions: 2.0.0 Reporter: Andrey Stepachev Assignee: Andrey Stepachev Attachments: 23757f039d83f4f17ca18815eae70b28.log, HBASE-13076.patch Got situation where region can be opened while table is disabling by DisableTableHandler. Here is relevant log for such situation. There is no clues who issued OPEN to region. Log file attached. UPD: A bit more details. It seems that even in case of new state put into meta, it still possible to get previous state. That leads to one more round of assignment invoked in AssignmentManager#onRegionClosed. UPD: Table become ENABLED, thats leads to regions instructed to assign immediately on onRegionClosed. BulkDisabler will not know about that and will wait indefinitely, because it will not issue unassign for newly opened regions. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13054) Provide more tracing information for locking/latching events.
[
https://issues.apache.org/jira/browse/HBASE-13054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14328910#comment-14328910
]
Hadoop QA commented on HBASE-13054:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12699854/HBASE-13054_v3.patch
against master branch at commit 03d8918142681d4c8abe40e8c8fb32307756d8a8.
ATTACHMENT ID: 12699854
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.1 2.5.2 2.6.0)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 checkstyle{color}. The applied patch does not increase the
total number of checkstyle errors
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.client.TestShell
{color:red}-1 core zombie tests{color}. There are 1 zombie test(s):
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/12925//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12925//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12925//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12925//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12925//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12925//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12925//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12925//artifact/patchprocess/newPatchFindbugsWarningshbase-rest.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12925//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12925//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12925//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12925//artifact/patchprocess/newPatchFindbugsWarningshbase-annotations.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/12925//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/12925//console
This message is automatically generated.
Provide more tracing information for locking/latching events.
-
Key: HBASE-13054
URL: https://issues.apache.org/jira/browse/HBASE-13054
Project: HBase
Issue Type: Improvement
Reporter: Rajeshbabu Chintaguntla
Assignee: Rajeshbabu Chintaguntla
Fix For: 2.0.0, 1.0.1, 1.1.0, 0.98.11
Attachments: HBASE-13054.patch, HBASE-13054_v2.patch,
HBASE-13054_v3.patch
Currently not much tracing information available for locking and latching
events like row level locking during do mini batch mutations, region level
locking during flush, close and so on. It will be better to add the trace
information for such events so that it will be useful for finding time spent
on locking and waiting time on locks while analyzing performance issues in
queries using trace information.
--
This message was sent by
[jira] [Updated] (HBASE-13080) Hbase shell message containing extra quote at the end of error message.
[
https://issues.apache.org/jira/browse/HBASE-13080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Abhishek Kumar updated HBASE-13080:
---
Attachment: 0001-HBASE-13080-remove-extra-quote-typo.patch
Hbase shell message containing extra quote at the end of error message.
---
Key: HBASE-13080
URL: https://issues.apache.org/jira/browse/HBASE-13080
Project: HBase
Issue Type: Improvement
Reporter: Abhishek Kumar
Priority: Trivial
Attachments: 0001-HBASE-13080-remove-extra-quote-typo.patch
{noformat}
hbase(main):001:0 drop 't2'
ERROR: Table t2 is enabled. Disable it first.'
---
hbase(main):001:0 disable 'noTable'
ERROR: Table csadfsdfsdf does not exist.'
{noformat}
extra quote at the end odf error message seems to be a typo, can be removed
from below two methods in admin.rb :
{noformat}
# Throw exception if table doesn't exist
def tableExists(table_name)
raise ArgumentError, Table #{table_name} does not exist.' unless
exists?(table_name)
end
# Drops a table
def drop(table_name)
tableExists(table_name)
raise ArgumentError, Table #{table_name} is enabled. Disable it
first.' if enabled?(table_name)
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-13058) Hbase shell command 'scan' for non existent table shows unnecessary info for one unrelated existent table.
[
https://issues.apache.org/jira/browse/HBASE-13058?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Abhishek Kumar updated HBASE-13058:
---
Attachment: 0001-HBASE-13058-shell-unknown-table-message-update.patch
shell modification patch attached, let me know your view on this.
Hbase shell command 'scan' for non existent table shows unnecessary info for
one unrelated existent table.
--
Key: HBASE-13058
URL: https://issues.apache.org/jira/browse/HBASE-13058
Project: HBase
Issue Type: Bug
Components: Client
Reporter: Abhishek Kumar
Priority: Trivial
Attachments: 0001-HBASE-13058-Error-messages-in-scan-table.patch,
0001-HBASE-13058-shell-unknown-table-message-update.patch
When scanning for a non existent table in hbase shell, error message in shell
sometimes(based on META table content) displays completely unrelated table
info , which seems to be unnecessary and inconsistent with other error
messages:
{noformat}
hbase(main):016:0 scan 'noTable'
ROW COLUMN+CELL
ERROR: Unknown table Table 'noTable' was not found, got: hbase:namespace.!
-
hbase(main):017:0 scan '01_noTable'
ROW COLUMN+CELL
ERROR: Unknown table 01_noTable!
--
{noformat}
Its happening when doing a META table scan (to locate input table ) and
scanner stops at row of another table (beyond which table can not exist) in
ConnectionManager.locateRegionInMeta:
{noformat}
private RegionLocations locateRegionInMeta(TableName tableName, byte[] row,
boolean useCache, boolean retry, int replicaId) throws
IOException {
.
// possible we got a region of a different table...
if (!regionInfo.getTable().equals(tableName)) {
throw new TableNotFoundException(
Table ' + tableName + ' was not found, got: +
regionInfo.getTable() + .);
}
...
...
{noformat}
Here, we can simply put a debug message(if required) and just throw the
TableNotFoundException(tableName) with only tableName instead of with
scanner positioned row.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13080) Hbase shell message containing extra quote at the end of error message.
[
https://issues.apache.org/jira/browse/HBASE-13080?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14328951#comment-14328951
]
Abhishek Kumar commented on HBASE-13080:
patch attached, pls take a look at it.
Hbase shell message containing extra quote at the end of error message.
---
Key: HBASE-13080
URL: https://issues.apache.org/jira/browse/HBASE-13080
Project: HBase
Issue Type: Improvement
Reporter: Abhishek Kumar
Priority: Trivial
Attachments: 0001-HBASE-13080-remove-extra-quote-typo.patch
{noformat}
hbase(main):001:0 drop 't2'
ERROR: Table t2 is enabled. Disable it first.'
---
hbase(main):001:0 disable 'noTable'
ERROR: Table csadfsdfsdf does not exist.'
{noformat}
extra quote at the end odf error message seems to be a typo, can be removed
from below two methods in admin.rb :
{noformat}
# Throw exception if table doesn't exist
def tableExists(table_name)
raise ArgumentError, Table #{table_name} does not exist.' unless
exists?(table_name)
end
# Drops a table
def drop(table_name)
tableExists(table_name)
raise ArgumentError, Table #{table_name} is enabled. Disable it
first.' if enabled?(table_name)
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13076) Table can be forcibly enabled in AssignmentManager during table disabling.
[
https://issues.apache.org/jira/browse/HBASE-13076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14328956#comment-14328956
]
Hadoop QA commented on HBASE-13076:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12699856/HBASE-13076.patch
against master branch at commit 03d8918142681d4c8abe40e8c8fb32307756d8a8.
ATTACHMENT ID: 12699856
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.1 2.5.2 2.6.0)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 checkstyle{color}. The applied patch does not increase the
total number of checkstyle errors
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/12926//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12926//artifact/patchprocess/newPatchFindbugsWarningshbase-rest.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12926//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12926//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12926//artifact/patchprocess/newPatchFindbugsWarningshbase-annotations.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12926//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12926//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12926//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12926//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12926//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12926//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12926//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/12926//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/12926//console
This message is automatically generated.
Table can be forcibly enabled in AssignmentManager during table disabling.
--
Key: HBASE-13076
URL: https://issues.apache.org/jira/browse/HBASE-13076
Project: HBase
Issue Type: Bug
Components: master, Region Assignment
Affects Versions: 2.0.0
Reporter: Andrey Stepachev
Assignee: Andrey Stepachev
Attachments: 23757f039d83f4f17ca18815eae70b28.log, HBASE-13076.patch
Got situation where region can be opened while table is disabling by
DisableTableHandler. Here is relevant log for such situation. There is no
clues who issued OPEN to region.
Log file attached.
UPD: A bit more details. It seems that even in case of new state put into
meta, it still possible to get previous state.
That leads to one more round of assignment invoked in
AssignmentManager#onRegionClosed.
UPD: Table become ENABLED, thats leads to regions instructed to assign
immediately on onRegionClosed. BulkDisabler will not know about that and will
[jira] [Commented] (HBASE-13080) Hbase shell message containing extra quote at the end of error message.
[
https://issues.apache.org/jira/browse/HBASE-13080?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14328958#comment-14328958
]
Matteo Bertozzi commented on HBASE-13080:
-
looks good, but looks like there is another one
{noformat}
$ git grep \.'\
hbase-shell/src/main/ruby/hbase/admin.rb: raise ArgumentError, Table
#{table_name} does not exist.' unless exists?(table_name)
hbase-shell/src/main/ruby/hbase/admin.rb: raise ArgumentError, Table
#{table_name} is enabled. Disable it first.' if enabled?(table_name)
hbase-shell/src/main/ruby/hbase/admin.rb: raise ArgumentError, Table
#{table_name} is not enabled. Enable it first.' unless enabled?(table_name)
{noformat}
Hbase shell message containing extra quote at the end of error message.
---
Key: HBASE-13080
URL: https://issues.apache.org/jira/browse/HBASE-13080
Project: HBase
Issue Type: Improvement
Reporter: Abhishek Kumar
Priority: Trivial
Attachments: 0001-HBASE-13080-remove-extra-quote-typo.patch
{noformat}
hbase(main):001:0 drop 't2'
ERROR: Table t2 is enabled. Disable it first.'
---
hbase(main):001:0 disable 'noTable'
ERROR: Table csadfsdfsdf does not exist.'
{noformat}
extra quote at the end odf error message seems to be a typo, can be removed
from below two methods in admin.rb :
{noformat}
# Throw exception if table doesn't exist
def tableExists(table_name)
raise ArgumentError, Table #{table_name} does not exist.' unless
exists?(table_name)
end
# Drops a table
def drop(table_name)
tableExists(table_name)
raise ArgumentError, Table #{table_name} is enabled. Disable it
first.' if enabled?(table_name)
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13064) Fix flakey TestEnableTableHandler
[ https://issues.apache.org/jira/browse/HBASE-13064?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14328962#comment-14328962 ] Andrey Stepachev commented on HBASE-13064: -- seems this related to HBASE-13076 Fix flakey TestEnableTableHandler - Key: HBASE-13064 URL: https://issues.apache.org/jira/browse/HBASE-13064 Project: HBase Issue Type: Bug Reporter: Andrey Stepachev Assignee: Andrey Stepachev -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-11544) [Ergonomics] hbase.client.scanner.caching is dogged and will try to return batch even if it means OOME
[ https://issues.apache.org/jira/browse/HBASE-11544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Lawlor updated HBASE-11544: Attachment: HBASE-11544-v3.patch - Cleanup a bunch of whitepace - Configure the scans in failing TestAcidGuarantees to use previous default configuration of caching=100 and maxResultSize=Long.Max_Value [Ergonomics] hbase.client.scanner.caching is dogged and will try to return batch even if it means OOME -- Key: HBASE-11544 URL: https://issues.apache.org/jira/browse/HBASE-11544 Project: HBase Issue Type: Bug Reporter: stack Assignee: Jonathan Lawlor Priority: Critical Labels: beginner Attachments: HBASE-11544-v1.patch, HBASE-11544-v2.patch, HBASE-11544-v3.patch Running some tests, I set hbase.client.scanner.caching=1000. Dataset has large cells. I kept OOME'ing. Serverside, we should measure how much we've accumulated and return to the client whatever we've gathered once we pass out a certain size threshold rather than keep accumulating till we OOME. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13056) Refactor table.jsp code to remove repeated code and make it easier to add new checks
[
https://issues.apache.org/jira/browse/HBASE-13056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14329602#comment-14329602
]
Andrew Purtell commented on HBASE-13056:
The precommit branch build stuff isn't quite right.
Let me look at the 0.98 patch manually. Will apply/push/resolve if it looks
good locally.
Refactor table.jsp code to remove repeated code and make it easier to add new
checks
Key: HBASE-13056
URL: https://issues.apache.org/jira/browse/HBASE-13056
Project: HBase
Issue Type: Improvement
Affects Versions: 2.0.0
Reporter: Vikas Vishwakarma
Assignee: Vikas Vishwakarma
Fix For: 2.0.0
Attachments: HBASE-13056-0.98.patch, HBASE-13056.patch
While trying to fix HBASE-13001, I realized that there is lot of html code
repetition in table.jsp which is making addition of new checks slightly
difficult in the sense I will have to:
1. Add the check at multiple places in the code
Or
2. Repeat the html code again for the new check
So I am proposing to re-factor table.jsp code such that the common html
header/body is loaded without any condition check and then we generate the
condition specific html code
snapshot.jsp follows the same format as explained below:
{noformat}
Current implementation:
if( x ) {
title_x
common_html_header
common_html_body
x_specific_html_body
} else {
title_y
common_html_header
common_html_body
y_specific_html_body
}
New Implementation:
==
if( x ) {
title_x
} else {
title_y
}
common_html_header
common_html_body
if( x ) {
x_specific_html_body
} else {
y_specific_html_body
}
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13057) Provide client utility to easily enable and disable table replication
[ https://issues.apache.org/jira/browse/HBASE-13057?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14329534#comment-14329534 ] Hudson commented on HBASE-13057: FAILURE: Integrated in HBase-TRUNK #6154 (See [https://builds.apache.org/job/HBase-TRUNK/6154/]) HBASE-13057 Provide client utility to easily enable and disable table replication (Ashish Singhi) (tedyu: rev 9a311303a88d355f8c982d2f3cc77bd6427dfb80) * hbase-shell/src/main/ruby/shell/commands/enable_table_replication.rb * hbase-shell/src/main/ruby/shell.rb * hbase-shell/src/main/ruby/hbase/replication_admin.rb * hbase-client/src/main/java/org/apache/hadoop/hbase/client/replication/ReplicationAdmin.java * hbase-shell/src/main/ruby/shell/commands/disable_table_replication.rb * hbase-server/src/test/java/org/apache/hadoop/hbase/client/replication/TestReplicationAdminWithClusters.java Provide client utility to easily enable and disable table replication - Key: HBASE-13057 URL: https://issues.apache.org/jira/browse/HBASE-13057 Project: HBase Issue Type: New Feature Components: Replication Affects Versions: 0.98.10 Reporter: Ashish Singhi Assignee: Ashish Singhi Fix For: 2.0.0, 1.1.0 Attachments: HBASE-13057-v1.patch, HBASE-13057-v2.patch, HBASE-13057.patch Currently to enable replication for a table user has to 1) Disable the table 2) Set the replication scope 3) enable the table and also create a table with same name and CFs in peer cluster(s). To help user we can provide client API's to enable and disable table replication. So that user has to call only one API i.e., enable table replication which will, 1. Create a table with same name and CFs in the peer cluster(s) 2. Disable the table 3. Set the replication scope 4. Enable the table If user wants to disable table replication then he/she only has to call disable table replication API which will, 1. Disable the table 2. Set the replication scope 3. Enable the table. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12853) distributed write pattern to replace ad hoc 'salting'
[ https://issues.apache.org/jira/browse/HBASE-12853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14329574#comment-14329574 ] Michael Segel commented on HBASE-12853: The design seems straight forward, at least as to a starting point. (YMMV) The client will create a reference to a table and then instantiate a scanner object along with any associated filters. The client then passes this object to the server expecting a result set to be returned. On the server side, it seems that the HBase Master (active) gets the scan request and then starts to do the heavy lifting. By providing more intelligence to this process, its possible to do more than just allow for bucketed tables to abstract the buckets and act as if its a regular table. The key question is how to best redesign this initial entry point to allow for such extensibility. distributed write pattern to replace ad hoc 'salting' - Key: HBASE-12853 URL: https://issues.apache.org/jira/browse/HBASE-12853 Project: HBase Issue Type: New Feature Reporter: Michael Segel Priority: Minor In reviewing HBASE-11682 (Description of Hot Spotting), one of the issues is that while 'salting' alleviated regional hot spotting, it increased the complexity required to utilize the data. Through the use of coprocessors, it should be possible to offer a method which distributes the data on write across the cluster and then manages reading the data returning a sort ordered result set, abstracting the underlying process. On table creation, a flag is set to indicate that this is a parallel table. On insert in to the table, if the flag is set to true then a prefix is added to the key. e.g. region server#- or region server #|| where the region server # is an integer between 1 and the number of region servers defined. On read (scan) for each region server defined, a separate scan is created adding the prefix. Since each scan will be in sort order, its possible to strip the prefix and return the lowest value key from each of the subsets. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-13056) Refactor table.jsp code to remove repeated code and make it easier to add new checks
[
https://issues.apache.org/jira/browse/HBASE-13056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell updated HBASE-13056:
---
Fix Version/s: 1.1.0
Refactor table.jsp code to remove repeated code and make it easier to add new
checks
Key: HBASE-13056
URL: https://issues.apache.org/jira/browse/HBASE-13056
Project: HBase
Issue Type: Improvement
Affects Versions: 2.0.0
Reporter: Vikas Vishwakarma
Assignee: Vikas Vishwakarma
Fix For: 2.0.0, 1.1.0
Attachments: HBASE-13056-0.98.patch, HBASE-13056.patch
While trying to fix HBASE-13001, I realized that there is lot of html code
repetition in table.jsp which is making addition of new checks slightly
difficult in the sense I will have to:
1. Add the check at multiple places in the code
Or
2. Repeat the html code again for the new check
So I am proposing to re-factor table.jsp code such that the common html
header/body is loaded without any condition check and then we generate the
condition specific html code
snapshot.jsp follows the same format as explained below:
{noformat}
Current implementation:
if( x ) {
title_x
common_html_header
common_html_body
x_specific_html_body
} else {
title_y
common_html_header
common_html_body
y_specific_html_body
}
New Implementation:
==
if( x ) {
title_x
} else {
title_y
}
common_html_header
common_html_body
if( x ) {
x_specific_html_body
} else {
y_specific_html_body
}
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (HBASE-13056) Refactor table.jsp code to remove repeated code and make it easier to add new checks
[
https://issues.apache.org/jira/browse/HBASE-13056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14329602#comment-14329602
]
Andrew Purtell edited comment on HBASE-13056 at 2/20/15 9:41 PM:
-
The precommit branch build stuff isn't quite right.
Let me look at the 0.98 patch manually. Will apply/push/resolve if it looks
good locally.
Edit: You have been suffering from HBASE-13081
was (Author: apurtell):
The precommit branch build stuff isn't quite right.
Let me look at the 0.98 patch manually. Will apply/push/resolve if it looks
good locally.
Refactor table.jsp code to remove repeated code and make it easier to add new
checks
Key: HBASE-13056
URL: https://issues.apache.org/jira/browse/HBASE-13056
Project: HBase
Issue Type: Improvement
Affects Versions: 2.0.0
Reporter: Vikas Vishwakarma
Assignee: Vikas Vishwakarma
Fix For: 2.0.0
Attachments: HBASE-13056-0.98.patch, HBASE-13056.patch
While trying to fix HBASE-13001, I realized that there is lot of html code
repetition in table.jsp which is making addition of new checks slightly
difficult in the sense I will have to:
1. Add the check at multiple places in the code
Or
2. Repeat the html code again for the new check
So I am proposing to re-factor table.jsp code such that the common html
header/body is loaded without any condition check and then we generate the
condition specific html code
snapshot.jsp follows the same format as explained below:
{noformat}
Current implementation:
if( x ) {
title_x
common_html_header
common_html_body
x_specific_html_body
} else {
title_y
common_html_header
common_html_body
y_specific_html_body
}
New Implementation:
==
if( x ) {
title_x
} else {
title_y
}
common_html_header
common_html_body
if( x ) {
x_specific_html_body
} else {
y_specific_html_body
}
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13081) Branch precommit builds are not updating to branch head before patch application
[
https://issues.apache.org/jira/browse/HBASE-13081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14329650#comment-14329650
]
Andrew Purtell commented on HBASE-13081:
IMHO we should always be checking the wipe the workspace option on every ASF
job configuration.
Branch precommit builds are not updating to branch head before patch
application
Key: HBASE-13081
URL: https://issues.apache.org/jira/browse/HBASE-13081
Project: HBase
Issue Type: Bug
Reporter: Andrew Purtell
See for example
https://builds.apache.org/job/PreCommit-HBASE-Build/12922//console
{noformat}
git checkout 0.98
Previous HEAD position was 03d8918... HBASE-13069 Thrift Http Server returns
an error code of 500 instead of 401 when authentication fails (Srikanth
Srungarapu)
Switched to branch '0.98'
Your branch is behind 'origin/0.98' by 48 commits, and can be fast-forwarded.
(use git pull to update your local branch)
git status
On branch 0.98
Your branch is behind 'origin/0.98' by 48 commits, and can be fast-forwarded.
(use git pull to update your local branch)
Untracked files:
(use git add file... to include in what will be committed)
patchprocess/
nothing added to commit but untracked files present (use git add to track)
{noformat}
Because the local tree is 48 commits behind the head of the 0.98 branch, the
contributor's patch based on the head of 0.98 branch cannot cleanly apply.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HBASE-13081) Branch precommit builds are not updating to branch head before patch application
Andrew Purtell created HBASE-13081:
--
Summary: Branch precommit builds are not updating to branch head
before patch application
Key: HBASE-13081
URL: https://issues.apache.org/jira/browse/HBASE-13081
Project: HBase
Issue Type: Bug
Reporter: Andrew Purtell
See for example
https://builds.apache.org/job/PreCommit-HBASE-Build/12922//console
{noformat}
git checkout 0.98
Previous HEAD position was 03d8918... HBASE-13069 Thrift Http Server returns an
error code of 500 instead of 401 when authentication fails (Srikanth Srungarapu)
Switched to branch '0.98'
Your branch is behind 'origin/0.98' by 48 commits, and can be fast-forwarded.
(use git pull to update your local branch)
git status
On branch 0.98
Your branch is behind 'origin/0.98' by 48 commits, and can be fast-forwarded.
(use git pull to update your local branch)
Untracked files:
(use git add file... to include in what will be committed)
patchprocess/
nothing added to commit but untracked files present (use git add to track)
{noformat}
Because the local tree is 48 commits behind the head of the 0.98 branch, the
contributor's patch based on the head of 0.98 branch cannot cleanly apply.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13070) Fix TestCacheOnWrite
[ https://issues.apache.org/jira/browse/HBASE-13070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14329538#comment-14329538 ] Hudson commented on HBASE-13070: FAILURE: Integrated in HBase-1.0 #763 (See [https://builds.apache.org/job/HBase-1.0/763/]) HBASE-13070 Fix TestCacheOnWrite (Duo Zhang) (tedyu: rev b7025fa7759a894aef7304b48522dc556ed9988a) * hbase-server/src/test/java/org/apache/hadoop/hbase/io/hfile/TestCacheOnWrite.java Fix TestCacheOnWrite Key: HBASE-13070 URL: https://issues.apache.org/jira/browse/HBASE-13070 Project: HBase Issue Type: Bug Components: test Affects Versions: 2.0.0 Reporter: zhangduo Assignee: zhangduo Fix For: 2.0.0, 1.0.1, 1.1.0, 0.98.11 Attachments: HBASE-13070.patch, HBASE-13070_1.patch, HBASE-13070_2.patch TestCacheOnWrite uses TestHFileWriterV2.randomOrderedKey to generate a random byte array, then use first 32 bytes as row and remaining part as family and qualifier. But TestHFileWriterV2.randomOrderedKey may return a byte array only contains 32 bytes, so there will be zero length family and qualifier. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12853) distributed write pattern to replace ad hoc 'salting'
[ https://issues.apache.org/jira/browse/HBASE-12853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14329576#comment-14329576 ] Michael Segel commented on HBASE-12853: I should also add that this is one area that one must take caution in the design because if not done properly or cleanly, it will kill performance. distributed write pattern to replace ad hoc 'salting' - Key: HBASE-12853 URL: https://issues.apache.org/jira/browse/HBASE-12853 Project: HBase Issue Type: New Feature Reporter: Michael Segel Priority: Minor In reviewing HBASE-11682 (Description of Hot Spotting), one of the issues is that while 'salting' alleviated regional hot spotting, it increased the complexity required to utilize the data. Through the use of coprocessors, it should be possible to offer a method which distributes the data on write across the cluster and then manages reading the data returning a sort ordered result set, abstracting the underlying process. On table creation, a flag is set to indicate that this is a parallel table. On insert in to the table, if the flag is set to true then a prefix is added to the key. e.g. region server#- or region server #|| where the region server # is an integer between 1 and the number of region servers defined. On read (scan) for each region server defined, a separate scan is created adding the prefix. Since each scan will be in sort order, its possible to strip the prefix and return the lowest value key from each of the subsets. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-11544) [Ergonomics] hbase.client.scanner.caching is dogged and will try to return batch even if it means OOME
[ https://issues.apache.org/jira/browse/HBASE-11544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Lawlor updated HBASE-11544: Status: Patch Available (was: Open) [Ergonomics] hbase.client.scanner.caching is dogged and will try to return batch even if it means OOME -- Key: HBASE-11544 URL: https://issues.apache.org/jira/browse/HBASE-11544 Project: HBase Issue Type: Bug Reporter: stack Assignee: Jonathan Lawlor Priority: Critical Labels: beginner Attachments: HBASE-11544-v1.patch, HBASE-11544-v2.patch, HBASE-11544-v3.patch Running some tests, I set hbase.client.scanner.caching=1000. Dataset has large cells. I kept OOME'ing. Serverside, we should measure how much we've accumulated and return to the client whatever we've gathered once we pass out a certain size threshold rather than keep accumulating till we OOME. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-11544) [Ergonomics] hbase.client.scanner.caching is dogged and will try to return batch even if it means OOME
[ https://issues.apache.org/jira/browse/HBASE-11544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Lawlor updated HBASE-11544: Status: Open (was: Patch Available) [Ergonomics] hbase.client.scanner.caching is dogged and will try to return batch even if it means OOME -- Key: HBASE-11544 URL: https://issues.apache.org/jira/browse/HBASE-11544 Project: HBase Issue Type: Bug Reporter: stack Assignee: Jonathan Lawlor Priority: Critical Labels: beginner Attachments: HBASE-11544-v1.patch, HBASE-11544-v2.patch Running some tests, I set hbase.client.scanner.caching=1000. Dataset has large cells. I kept OOME'ing. Serverside, we should measure how much we've accumulated and return to the client whatever we've gathered once we pass out a certain size threshold rather than keep accumulating till we OOME. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13081) Branch precommit builds are not updating to branch head before patch application
[
https://issues.apache.org/jira/browse/HBASE-13081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14329614#comment-14329614
]
Sean Busbey commented on HBASE-13081:
-
are we concerned about pulling the whole repo on each run? the easy way to fix
this is to destroy the workspace and require a full checkout each time.
We could instead try to be smart about rebasing up to latest, but that leaves
open the possibility of things getting wedged.
Branch precommit builds are not updating to branch head before patch
application
Key: HBASE-13081
URL: https://issues.apache.org/jira/browse/HBASE-13081
Project: HBase
Issue Type: Bug
Reporter: Andrew Purtell
See for example
https://builds.apache.org/job/PreCommit-HBASE-Build/12922//console
{noformat}
git checkout 0.98
Previous HEAD position was 03d8918... HBASE-13069 Thrift Http Server returns
an error code of 500 instead of 401 when authentication fails (Srikanth
Srungarapu)
Switched to branch '0.98'
Your branch is behind 'origin/0.98' by 48 commits, and can be fast-forwarded.
(use git pull to update your local branch)
git status
On branch 0.98
Your branch is behind 'origin/0.98' by 48 commits, and can be fast-forwarded.
(use git pull to update your local branch)
Untracked files:
(use git add file... to include in what will be committed)
patchprocess/
nothing added to commit but untracked files present (use git add to track)
{noformat}
Because the local tree is 48 commits behind the head of the 0.98 branch, the
contributor's patch based on the head of 0.98 branch cannot cleanly apply.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13070) Fix TestCacheOnWrite
[ https://issues.apache.org/jira/browse/HBASE-13070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14329830#comment-14329830 ] zhangduo commented on HBASE-13070: -- 'clearCache' does not work properly on 0.98, let me try. Fix TestCacheOnWrite Key: HBASE-13070 URL: https://issues.apache.org/jira/browse/HBASE-13070 Project: HBase Issue Type: Bug Components: test Affects Versions: 2.0.0 Reporter: zhangduo Assignee: zhangduo Fix For: 2.0.0, 1.0.1, 1.1.0, 0.98.11 Attachments: HBASE-13070.patch, HBASE-13070_1.patch, HBASE-13070_2.patch TestCacheOnWrite uses TestHFileWriterV2.randomOrderedKey to generate a random byte array, then use first 32 bytes as row and remaining part as family and qualifier. But TestHFileWriterV2.randomOrderedKey may return a byte array only contains 32 bytes, so there will be zero length family and qualifier. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-13081) Branch precommit builds are not updating to branch head before patch application
[
https://issues.apache.org/jira/browse/HBASE-13081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Enis Soztutar updated HBASE-13081:
--
Status: Patch Available (was: Open)
Branch precommit builds are not updating to branch head before patch
application
Key: HBASE-13081
URL: https://issues.apache.org/jira/browse/HBASE-13081
Project: HBase
Issue Type: Bug
Reporter: Andrew Purtell
Assignee: Enis Soztutar
Attachments: foo_branch-1.patch, hbase-13081.patch
See for example
https://builds.apache.org/job/PreCommit-HBASE-Build/12922//console
{noformat}
git checkout 0.98
Previous HEAD position was 03d8918... HBASE-13069 Thrift Http Server returns
an error code of 500 instead of 401 when authentication fails (Srikanth
Srungarapu)
Switched to branch '0.98'
Your branch is behind 'origin/0.98' by 48 commits, and can be fast-forwarded.
(use git pull to update your local branch)
git status
On branch 0.98
Your branch is behind 'origin/0.98' by 48 commits, and can be fast-forwarded.
(use git pull to update your local branch)
Untracked files:
(use git add file... to include in what will be committed)
patchprocess/
nothing added to commit but untracked files present (use git add to track)
{noformat}
Because the local tree is 48 commits behind the head of the 0.98 branch, the
contributor's patch based on the head of 0.98 branch cannot cleanly apply.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-13081) Branch precommit builds are not updating to branch head before patch application
[
https://issues.apache.org/jira/browse/HBASE-13081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Enis Soztutar updated HBASE-13081:
--
Attachment: foo_branch-1.patch
Let's try the build with this dummy patch.
Branch precommit builds are not updating to branch head before patch
application
Key: HBASE-13081
URL: https://issues.apache.org/jira/browse/HBASE-13081
Project: HBase
Issue Type: Bug
Reporter: Andrew Purtell
Assignee: Enis Soztutar
Attachments: foo_branch-1.patch, hbase-13081.patch
See for example
https://builds.apache.org/job/PreCommit-HBASE-Build/12922//console
{noformat}
git checkout 0.98
Previous HEAD position was 03d8918... HBASE-13069 Thrift Http Server returns
an error code of 500 instead of 401 when authentication fails (Srikanth
Srungarapu)
Switched to branch '0.98'
Your branch is behind 'origin/0.98' by 48 commits, and can be fast-forwarded.
(use git pull to update your local branch)
git status
On branch 0.98
Your branch is behind 'origin/0.98' by 48 commits, and can be fast-forwarded.
(use git pull to update your local branch)
Untracked files:
(use git add file... to include in what will be committed)
patchprocess/
nothing added to commit but untracked files present (use git add to track)
{noformat}
Because the local tree is 48 commits behind the head of the 0.98 branch, the
contributor's patch based on the head of 0.98 branch cannot cleanly apply.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-13081) Branch precommit builds are not updating to branch head before patch application
[
https://issues.apache.org/jira/browse/HBASE-13081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Enis Soztutar updated HBASE-13081:
--
Resolution: Fixed
Fix Version/s: 2.0.0
Status: Resolved (was: Patch Available)
It seems that we are all done here. Thanks for reporting!
Branch precommit builds are not updating to branch head before patch
application
Key: HBASE-13081
URL: https://issues.apache.org/jira/browse/HBASE-13081
Project: HBase
Issue Type: Bug
Reporter: Andrew Purtell
Assignee: Enis Soztutar
Fix For: 2.0.0
Attachments: foo_0.98.patch, foo_branch-1.patch, hbase-13081.patch
See for example
https://builds.apache.org/job/PreCommit-HBASE-Build/12922//console
{noformat}
git checkout 0.98
Previous HEAD position was 03d8918... HBASE-13069 Thrift Http Server returns
an error code of 500 instead of 401 when authentication fails (Srikanth
Srungarapu)
Switched to branch '0.98'
Your branch is behind 'origin/0.98' by 48 commits, and can be fast-forwarded.
(use git pull to update your local branch)
git status
On branch 0.98
Your branch is behind 'origin/0.98' by 48 commits, and can be fast-forwarded.
(use git pull to update your local branch)
Untracked files:
(use git add file... to include in what will be committed)
patchprocess/
nothing added to commit but untracked files present (use git add to track)
{noformat}
Because the local tree is 48 commits behind the head of the 0.98 branch, the
contributor's patch based on the head of 0.98 branch cannot cleanly apply.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12972) Region, a supportable public/evolving subset of HRegion
[
https://issues.apache.org/jira/browse/HBASE-12972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14329961#comment-14329961
]
Andrew Purtell commented on HBASE-12972:
I've started work on this. The first patch will be for 0.98 since there is no
version of Phoenix (yet) ported to 1.0 or master.
I did look at master for a bit. Unfortunately there are some aspects of the
Table interface that don't play nicely with HRegion code: close() and
getScanner() are different. The former isn't a big deal but when RegionScanner
needed to implement ResultScanner I stopped there. Not that we can't do it, but
what I've decided to do instead is lift the minimum up out of HRegion into
Region to support coprocessors in the HBase tree and Phoenix together.
I'm treating introduction of the Region interface as a singularity of sorts for
coprocessors: neither source nor binary compatibility will be maintained. I
don't see the harm in a singularity, HRegion isn't supported, that's the point
of this work... to replace it with something that is. :-) However, after there
is a first workable patch if it's not too onerous to make addtional changes
that keep source or binary compatibility then we can do that.
Region, a supportable public/evolving subset of HRegion
---
Key: HBASE-12972
URL: https://issues.apache.org/jira/browse/HBASE-12972
Project: HBase
Issue Type: New Feature
Reporter: Andrew Purtell
Assignee: Andrew Purtell
Fix For: 2.0.0, 1.0.1, 1.1.0, 0.98.11
On HBASE-12566, [~lhofhansl] proposed:
{quote}
Maybe we can have a {{Region}} interface that is to {{HRegion}} is what
{{Store}} is to {{HStore}}. Store marked with {{@InterfaceAudience.Private}}
but used in some coprocessor hooks.
{quote}
By example, now coprocessors have to reach into HRegion in order to
participate in row and region locking protocols, this is one area where the
functionality is legitimate for coprocessors but not for users, so an
in-between interface make sense.
In addition we should promote {{Store}}'s interface audience to
LimitedPrivate(COPROC).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13070) Fix TestCacheOnWrite
[
https://issues.apache.org/jira/browse/HBASE-13070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14329970#comment-14329970
]
Hadoop QA commented on HBASE-13070:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12699982/HBASE-13070-0.98.patch
against 0.98 branch at commit a8555f65f0551569f07f6a7d207ddc0e56fad03a.
ATTACHMENT ID: 12699982
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 5 new
or modified tests.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.1 2.5.2 2.6.0)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated
23 warning messages.
{color:green}+1 checkstyle{color}. The applied patch does not increase the
total number of checkstyle errors
{color:red}-1 findbugs{color}. The patch appears to introduce 4 new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/12932//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12932//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12932//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12932//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12932//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12932//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12932//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12932//artifact/patchprocess/newPatchFindbugsWarningshbase-rest.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12932//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12932//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12932//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12932//artifact/patchprocess/newPatchFindbugsWarningshbase-annotations.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/12932//artifact/patchprocess/checkstyle-aggregate.html
Javadoc warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12932//artifact/patchprocess/patchJavadocWarnings.txt
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/12932//console
This message is automatically generated.
Fix TestCacheOnWrite
Key: HBASE-13070
URL: https://issues.apache.org/jira/browse/HBASE-13070
Project: HBase
Issue Type: Bug
Components: test
Affects Versions: 2.0.0
Reporter: zhangduo
Assignee: zhangduo
Fix For: 2.0.0, 1.0.1, 1.1.0, 0.98.11
Attachments: HBASE-13070-0.98.patch, HBASE-13070.patch,
HBASE-13070_1.patch, HBASE-13070_2.patch
TestCacheOnWrite uses TestHFileWriterV2.randomOrderedKey to generate a random
byte array, then use first 32 bytes as row and remaining part as family and
qualifier. But TestHFileWriterV2.randomOrderedKey may return a byte array
only contains 32 bytes, so there will be zero length family and qualifier.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12954) Ability impaired using HBase on multihomed hosts
[
https://issues.apache.org/jira/browse/HBASE-12954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14330008#comment-14330008
]
stack commented on HBASE-12954:
---
Any attempt at replicating the network scenarios Clay posts originally up in a
cluster and seeing if this patch actually solves the issues originally raised?
Ability impaired using HBase on multihomed hosts
Key: HBASE-12954
URL: https://issues.apache.org/jira/browse/HBASE-12954
Project: HBase
Issue Type: Bug
Affects Versions: 0.98.4
Reporter: Clay B.
Assignee: Ted Yu
Priority: Minor
Attachments: 12954-v1.txt, 12954-v10.txt, 12954-v11.txt,
12954-v12.txt, 12954-v12.txt, 12954-v12.txt, 12954-v13.txt, 12954-v7.txt,
12954-v8.txt, Hadoop Three Interfaces.png
For HBase clusters running on unusual networks (such as NAT'd cloud
environments or physical machines with multiple IP's per network interface)
it would be ideal to have a way to both specify:
# which IP interface to which HBase master or region-server will bind
# what hostname HBase will advertise in Zookeeper both for a master or
region-server process
While efforts such as HBASE-8640 go a long way to normalize these two sources
of information, it is not possible in the current design of the properties
available to an administrator for these to be unambiguously specified.
One has been able to request {{hbase.master.ipc.address}} or
{{hbase.regionserver.ipc.address}} but one can not specify the desired HBase
{{hbase.master.hostname}}. (It was removed in HBASE-1357, further I am
unaware of a region-server equivalent.)
I use a configuration management system to generate all of my configuration
files on a per-machine basis. As such, an option to generate a file
specifying exactly which hostname to use would be helpful.
Today, specifying the bind address for HBase works and one can use an
HBase-only DNS for faking what to put in Zookeeper but this is far from
ideal. Network interfaces have no intrinsic IP address, nor hostname.
Specifing a DNS server is awkward as the DNS server may differ from the
system's resolver and is a single IP address. Similarly, on hosts which use a
transient VIP (e.g. through keepalived) for other services, it means there's
a seemingly non-deterministic hostname choice made by HBase depending on the
state of the VIP at daemon start-up time.
I will attach two networking examples I use which become very difficult to
manage under the current properties.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13076) Table can be forcibly enabled in AssignmentManager during table disabling.
[ https://issues.apache.org/jira/browse/HBASE-13076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14330011#comment-14330011 ] stack commented on HBASE-13076: --- Mighty [~jxiang], any input on this one (Removes handling of a 'state' which makes me a little nervous and has me run to the expert). Table can be forcibly enabled in AssignmentManager during table disabling. -- Key: HBASE-13076 URL: https://issues.apache.org/jira/browse/HBASE-13076 Project: HBase Issue Type: Bug Components: master, Region Assignment Affects Versions: 2.0.0 Reporter: Andrey Stepachev Assignee: Andrey Stepachev Attachments: 23757f039d83f4f17ca18815eae70b28.log, HBASE-13076.patch Got situation where region can be opened while table is disabling by DisableTableHandler. Here is relevant log for such situation. There is no clues who issued OPEN to region. Log file attached. UPD: A bit more details. It seems that even in case of new state put into meta, it still possible to get previous state. That leads to one more round of assignment invoked in AssignmentManager#onRegionClosed. UPD: Table become ENABLED, thats leads to regions instructed to assign immediately on onRegionClosed. BulkDisabler will not know about that and will wait indefinitely, because it will not issue unassign for newly opened regions. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13065) Increasing -Xmx when running TestDistributedLogSplitting
[
https://issues.apache.org/jira/browse/HBASE-13065?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14330029#comment-14330029
]
stack commented on HBASE-13065:
---
Looks like it was H3:
https://builds.apache.org/view/H-L/view/HBase/job/HBase-TRUNK/6155/consoleFull
Asking will be back.
Increasing -Xmx when running TestDistributedLogSplitting
Key: HBASE-13065
URL: https://issues.apache.org/jira/browse/HBASE-13065
Project: HBase
Issue Type: Bug
Components: test
Reporter: zhangduo
Assignee: zhangduo
Fix For: 2.0.0, 1.0.1, 1.1.0, 0.98.11
Attachments: 13065-fix.txt
Found this in PreCommit Build reports
https://builds.apache.org/job/PreCommit-HBASE-Build/12885/artifact/hbase-server/target/surefire-reports/org.apache.hadoop.hbase.master.TestDistributedLogSplitting-output.txt
{noformat}
2015-02-18 03:45:42,141 WARN [RS:4;asf901:41265] util.Sleeper(97): We slept
59018ms instead of 1000ms, this is likely due to a long garbage collecting
pause and it's usually bad, see
http://hbase.apache.org/book.html#trouble.rs.runtime.zkexpired
2015-02-18 03:45:26,750 WARN [JvmPauseMonitor]
util.JvmPauseMonitor$Monitor(167): Detected pause in JVM or host machine (eg
GC): pause of approximately 39767ms
GC pool 'PS MarkSweep' had collection(s): count=65 time=47720ms
{noformat}
Maybe we should increase the max heap size since this test starts 6
regionservers.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-13081) Branch precommit builds are not updating to branch head before patch application
[
https://issues.apache.org/jira/browse/HBASE-13081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Enis Soztutar updated HBASE-13081:
--
Attachment: foo_0.98.patch
0.98 patch.
Branch precommit builds are not updating to branch head before patch
application
Key: HBASE-13081
URL: https://issues.apache.org/jira/browse/HBASE-13081
Project: HBase
Issue Type: Bug
Reporter: Andrew Purtell
Assignee: Enis Soztutar
Attachments: foo_0.98.patch, foo_branch-1.patch, hbase-13081.patch
See for example
https://builds.apache.org/job/PreCommit-HBASE-Build/12922//console
{noformat}
git checkout 0.98
Previous HEAD position was 03d8918... HBASE-13069 Thrift Http Server returns
an error code of 500 instead of 401 when authentication fails (Srikanth
Srungarapu)
Switched to branch '0.98'
Your branch is behind 'origin/0.98' by 48 commits, and can be fast-forwarded.
(use git pull to update your local branch)
git status
On branch 0.98
Your branch is behind 'origin/0.98' by 48 commits, and can be fast-forwarded.
(use git pull to update your local branch)
Untracked files:
(use git add file... to include in what will be committed)
patchprocess/
nothing added to commit but untracked files present (use git add to track)
{noformat}
Because the local tree is 48 commits behind the head of the 0.98 branch, the
contributor's patch based on the head of 0.98 branch cannot cleanly apply.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-11842) Integration test for async wal replication to secondary regions
[ https://issues.apache.org/jira/browse/HBASE-11842?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14329933#comment-14329933 ] Enis Soztutar commented on HBASE-11842: --- bq. Second line above has 'hbase.' prefix. Should it be consistent with the style of the other options ? The usage of hbase prefix is coming from the base class. We should have standardized it, but unfortunately we did not. We can fix it in another patch. bq. Why factor of 0.9 is used above ? 0.9 is used since the run time is just a soft bound on how long the test should run. It is iterative, so expectation is that it will not iterate more if it is around %90 done. Integration test for async wal replication to secondary regions --- Key: HBASE-11842 URL: https://issues.apache.org/jira/browse/HBASE-11842 Project: HBase Issue Type: Sub-task Reporter: Enis Soztutar Assignee: Enis Soztutar Fix For: 2.0.0, 1.1.0 Attachments: hbase-11842_v1.patch, hbase-11842_v2.patch We should add an integration test to cover async wal replication to secondary regions. Async wal replication to region replicas has been committed (HBASE-11568) and flush /compaction handling (HBASE-11569) and failover handling for secondary region replicas (HBASE-11580) is in the works. The test can use LoadTestTool over a table with region replicas. We can write to the primary regions, and read from the secondary region replicas in reader threads to do the verification. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13081) Branch precommit builds are not updating to branch head before patch application
[
https://issues.apache.org/jira/browse/HBASE-13081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14329939#comment-14329939
]
Hadoop QA commented on HBASE-13081:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12699977/foo_branch-1.patch
against branch-1 branch at commit a8555f65f0551569f07f6a7d207ddc0e56fad03a.
ATTACHMENT ID: 12699977
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.1 2.5.2 2.6.0)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 checkstyle{color}. The applied patch does not increase the
total number of checkstyle errors
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/12930//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12930//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12930//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12930//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12930//artifact/patchprocess/newPatchFindbugsWarningshbase-annotations.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12930//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12930//artifact/patchprocess/newPatchFindbugsWarningshbase-rest.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12930//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12930//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12930//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12930//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12930//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/12930//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/12930//console
This message is automatically generated.
Branch precommit builds are not updating to branch head before patch
application
Key: HBASE-13081
URL: https://issues.apache.org/jira/browse/HBASE-13081
Project: HBase
Issue Type: Bug
Reporter: Andrew Purtell
Assignee: Enis Soztutar
Attachments: foo_0.98.patch, foo_branch-1.patch, hbase-13081.patch
See for example
https://builds.apache.org/job/PreCommit-HBASE-Build/12922//console
{noformat}
git checkout 0.98
Previous HEAD position was 03d8918... HBASE-13069 Thrift Http Server returns
an error code of 500 instead of 401 when authentication fails (Srikanth
Srungarapu)
Switched to branch '0.98'
Your branch is behind 'origin/0.98' by 48 commits, and can be fast-forwarded.
(use git pull to update your local branch)
git status
On branch 0.98
Your branch is behind 'origin/0.98' by 48 commits, and can be fast-forwarded.
(use git pull to update your local branch)
Untracked files:
(use git add file... to
[jira] [Commented] (HBASE-13056) Refactor table.jsp code to remove repeated code and make it easier to add new checks
[
https://issues.apache.org/jira/browse/HBASE-13056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14329947#comment-14329947
]
Hudson commented on HBASE-13056:
FAILURE: Integrated in HBase-1.0 #764 (See
[https://builds.apache.org/job/HBase-1.0/764/])
HBASE-13056 Refactor table.jsp code to remove repeated code and make it easier
to add new checks (Vikas Vishwakarma) (apurtell: rev
12ff78a4859ac27b1f9ee8b66dbd52a885843098)
* hbase-server/src/main/resources/hbase-webapps/master/table.jsp
Refactor table.jsp code to remove repeated code and make it easier to add new
checks
Key: HBASE-13056
URL: https://issues.apache.org/jira/browse/HBASE-13056
Project: HBase
Issue Type: Improvement
Affects Versions: 2.0.0
Reporter: Vikas Vishwakarma
Assignee: Vikas Vishwakarma
Fix For: 2.0.0, 1.0.1, 1.1.0, 0.98.11
Attachments: HBASE-13056-0.98.patch, HBASE-13056.patch
While trying to fix HBASE-13001, I realized that there is lot of html code
repetition in table.jsp which is making addition of new checks slightly
difficult in the sense I will have to:
1. Add the check at multiple places in the code
Or
2. Repeat the html code again for the new check
So I am proposing to re-factor table.jsp code such that the common html
header/body is loaded without any condition check and then we generate the
condition specific html code
snapshot.jsp follows the same format as explained below:
{noformat}
Current implementation:
if( x ) {
title_x
common_html_header
common_html_body
x_specific_html_body
} else {
title_y
common_html_header
common_html_body
y_specific_html_body
}
New Implementation:
==
if( x ) {
title_x
} else {
title_y
}
common_html_header
common_html_body
if( x ) {
x_specific_html_body
} else {
y_specific_html_body
}
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13081) Branch precommit builds are not updating to branch head before patch application
[
https://issues.apache.org/jira/browse/HBASE-13081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14329953#comment-14329953
]
Hudson commented on HBASE-13081:
FAILURE: Integrated in HBase-TRUNK #6155 (See
[https://builds.apache.org/job/HBase-TRUNK/6155/])
HBASE-13081 Branch precommit builds are not updating to branch head before
patch application (enis: rev a8555f65f0551569f07f6a7d207ddc0e56fad03a)
* dev-support/test-patch.sh
Branch precommit builds are not updating to branch head before patch
application
Key: HBASE-13081
URL: https://issues.apache.org/jira/browse/HBASE-13081
Project: HBase
Issue Type: Bug
Reporter: Andrew Purtell
Assignee: Enis Soztutar
Fix For: 2.0.0
Attachments: foo_0.98.patch, foo_branch-1.patch, hbase-13081.patch
See for example
https://builds.apache.org/job/PreCommit-HBASE-Build/12922//console
{noformat}
git checkout 0.98
Previous HEAD position was 03d8918... HBASE-13069 Thrift Http Server returns
an error code of 500 instead of 401 when authentication fails (Srikanth
Srungarapu)
Switched to branch '0.98'
Your branch is behind 'origin/0.98' by 48 commits, and can be fast-forwarded.
(use git pull to update your local branch)
git status
On branch 0.98
Your branch is behind 'origin/0.98' by 48 commits, and can be fast-forwarded.
(use git pull to update your local branch)
Untracked files:
(use git add file... to include in what will be committed)
patchprocess/
nothing added to commit but untracked files present (use git add to track)
{noformat}
Because the local tree is 48 commits behind the head of the 0.98 branch, the
contributor's patch based on the head of 0.98 branch cannot cleanly apply.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13070) Fix TestCacheOnWrite
[ https://issues.apache.org/jira/browse/HBASE-13070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14329975#comment-14329975 ] zhangduo commented on HBASE-13070: -- Seems the javadoc and findbugs warnings are not related? Fix TestCacheOnWrite Key: HBASE-13070 URL: https://issues.apache.org/jira/browse/HBASE-13070 Project: HBase Issue Type: Bug Components: test Affects Versions: 2.0.0 Reporter: zhangduo Assignee: zhangduo Fix For: 2.0.0, 1.0.1, 1.1.0, 0.98.11 Attachments: HBASE-13070-0.98.patch, HBASE-13070.patch, HBASE-13070_1.patch, HBASE-13070_2.patch TestCacheOnWrite uses TestHFileWriterV2.randomOrderedKey to generate a random byte array, then use first 32 bytes as row and remaining part as family and qualifier. But TestHFileWriterV2.randomOrderedKey may return a byte array only contains 32 bytes, so there will be zero length family and qualifier. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-11842) Integration test for async wal replication to secondary regions
[
https://issues.apache.org/jira/browse/HBASE-11842?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14329989#comment-14329989
]
Hadoop QA commented on HBASE-11842:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/1262/hbase-11842_v2.patch
against master branch at commit a8555f65f0551569f07f6a7d207ddc0e56fad03a.
ATTACHMENT ID: 1262
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 25 new
or modified tests.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.1 2.5.2 2.6.0)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 checkstyle{color}. The applied patch does not increase the
total number of checkstyle errors
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
{color:red}-1 core zombie tests{color}. There are 1 zombie test(s):
at
org.apache.hadoop.hbase.TestAcidGuarantees.testScanAtomicity(TestAcidGuarantees.java:354)
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/12933//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12933//artifact/patchprocess/newPatchFindbugsWarningshbase-annotations.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12933//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12933//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12933//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12933//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12933//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12933//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12933//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12933//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12933//artifact/patchprocess/newPatchFindbugsWarningshbase-rest.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12933//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/12933//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/12933//console
This message is automatically generated.
Integration test for async wal replication to secondary regions
---
Key: HBASE-11842
URL: https://issues.apache.org/jira/browse/HBASE-11842
Project: HBase
Issue Type: Sub-task
Reporter: Enis Soztutar
Assignee: Enis Soztutar
Fix For: 2.0.0, 1.1.0
Attachments: hbase-11842_v1.patch, hbase-11842_v2.patch
We should add an integration test to cover async wal replication to secondary
regions.
Async wal replication to region replicas has been committed (HBASE-11568) and
flush /compaction handling (HBASE-11569) and failover handling for secondary
region replicas (HBASE-11580) is in the works.
The test can use LoadTestTool over a table with region replicas. We can write
to the primary regions, and read from the secondary region replicas in reader
threads to do the verification.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-5238) Add a log4j category for all edits to META/ROOT
[ https://issues.apache.org/jira/browse/HBASE-5238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14330001#comment-14330001 ] stack commented on HBASE-5238: -- Thanks for the log. I don't think I need this 'META: ' prefix you have on all the logging since it all has MetaTableAccessor as class and its all JSON? I can fix on commit. The release not can be improved in that you should say how to 'enable' this extra logging (or rather, I suppose, it looks like it is ON always -- that is so [~octo47]? If so, I see no harm starting there in 1.1.0 and 2.0.0). Thanks. Add a log4j category for all edits to META/ROOT --- Key: HBASE-5238 URL: https://issues.apache.org/jira/browse/HBASE-5238 Project: HBase Issue Type: New Feature Components: regionserver Affects Versions: 2.0.0 Reporter: Todd Lipcon Assignee: Andrey Stepachev Priority: Minor Labels: beginner Attachments: HBASE-5238.patch, meta2.log Occasionally we run into bugs that have corrected META and written some bad data to META/ROOT but it's difficult to understand the order in which things happened. One option is to dump the HLog contents from the servers that hosted META at that time, but then it's interspersed with all other data. It would be nice to add a Log4j Logger to which we log all edits being applied to META and ROOT in textual form at DEBUG level. Then it would be easier to do a cluster-wide log grep to see what happened when. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-5238) Add a log4j category for all edits to META/ROOT
[ https://issues.apache.org/jira/browse/HBASE-5238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-5238: - Attachment: HBASE-5238.patch Retry Add a log4j category for all edits to META/ROOT --- Key: HBASE-5238 URL: https://issues.apache.org/jira/browse/HBASE-5238 Project: HBase Issue Type: New Feature Components: regionserver Affects Versions: 2.0.0 Reporter: Todd Lipcon Assignee: Andrey Stepachev Priority: Minor Labels: beginner Attachments: HBASE-5238.patch, HBASE-5238.patch, meta2.log Occasionally we run into bugs that have corrected META and written some bad data to META/ROOT but it's difficult to understand the order in which things happened. One option is to dump the HLog contents from the servers that hosted META at that time, but then it's interspersed with all other data. It would be nice to add a Log4j Logger to which we log all edits being applied to META and ROOT in textual form at DEBUG level. Then it would be easier to do a cluster-wide log grep to see what happened when. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-13054) Provide more tracing information for locking/latching events.
[ https://issues.apache.org/jira/browse/HBASE-13054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajeshbabu Chintaguntla updated HBASE-13054: Attachment: HBASE-13054_0.98.patch patch for 0.98. I will commit shortly. Provide more tracing information for locking/latching events. - Key: HBASE-13054 URL: https://issues.apache.org/jira/browse/HBASE-13054 Project: HBase Issue Type: Improvement Reporter: Rajeshbabu Chintaguntla Assignee: Rajeshbabu Chintaguntla Fix For: 2.0.0, 1.0.1, 1.1.0, 0.98.11 Attachments: HBASE-13054.patch, HBASE-13054_0.98.patch, HBASE-13054_v2.patch, HBASE-13054_v3.patch, HBASE-13056_branch-1.patch Currently not much tracing information available for locking and latching events like row level locking during do mini batch mutations, region level locking during flush, close and so on. It will be better to add the trace information for such events so that it will be useful for finding time spent on locking and waiting time on locks while analyzing performance issues in queries using trace information. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13054) Provide more tracing information for locking/latching events.
[
https://issues.apache.org/jira/browse/HBASE-13054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14329994#comment-14329994
]
Hadoop QA commented on HBASE-13054:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/1269/HBASE-13056_branch-1.patch
against branch-1 branch at commit a8555f65f0551569f07f6a7d207ddc0e56fad03a.
ATTACHMENT ID: 1269
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.1 2.5.2 2.6.0)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 checkstyle{color}. The applied patch does not increase the
total number of checkstyle errors
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
{color:red}-1 core zombie tests{color}. There are 1 zombie test(s):
at
org.apache.hadoop.hbase.http.TestHttpServerLifecycle.testStartedServerIsAlive(TestHttpServerLifecycle.java:69)
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/12934//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12934//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12934//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12934//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12934//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12934//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12934//artifact/patchprocess/newPatchFindbugsWarningshbase-rest.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12934//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12934//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12934//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12934//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12934//artifact/patchprocess/newPatchFindbugsWarningshbase-annotations.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/12934//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/12934//console
This message is automatically generated.
Provide more tracing information for locking/latching events.
-
Key: HBASE-13054
URL: https://issues.apache.org/jira/browse/HBASE-13054
Project: HBase
Issue Type: Improvement
Reporter: Rajeshbabu Chintaguntla
Assignee: Rajeshbabu Chintaguntla
Fix For: 2.0.0, 1.0.1, 1.1.0, 0.98.11
Attachments: HBASE-13054.patch, HBASE-13054_0.98.patch,
HBASE-13054_v2.patch, HBASE-13054_v3.patch, HBASE-13056_branch-1.patch
Currently not much tracing information available for locking and latching
events like row level locking during do mini batch mutations, region level
locking during flush, close and so on. It will be better to add the trace
information for such events so that it will be useful for finding time spent
on locking and
[jira] [Commented] (HBASE-13081) Branch precommit builds are not updating to branch head before patch application
[
https://issues.apache.org/jira/browse/HBASE-13081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14329826#comment-14329826
]
Enis Soztutar commented on HBASE-13081:
---
bq. IMHO we should always be checking the wipe the workspace option on every
ASF job configuration
I thought we were doing that already. That is why the test-patch.sh code does
not do {{git pull}}, but just checks out a branch. Jenkins first finds the git
revision, then checks out that revision without bothering with branches at all.
Let me see whether we can do it that way.
Branch precommit builds are not updating to branch head before patch
application
Key: HBASE-13081
URL: https://issues.apache.org/jira/browse/HBASE-13081
Project: HBase
Issue Type: Bug
Reporter: Andrew Purtell
Assignee: Enis Soztutar
See for example
https://builds.apache.org/job/PreCommit-HBASE-Build/12922//console
{noformat}
git checkout 0.98
Previous HEAD position was 03d8918... HBASE-13069 Thrift Http Server returns
an error code of 500 instead of 401 when authentication fails (Srikanth
Srungarapu)
Switched to branch '0.98'
Your branch is behind 'origin/0.98' by 48 commits, and can be fast-forwarded.
(use git pull to update your local branch)
git status
On branch 0.98
Your branch is behind 'origin/0.98' by 48 commits, and can be fast-forwarded.
(use git pull to update your local branch)
Untracked files:
(use git add file... to include in what will be committed)
patchprocess/
nothing added to commit but untracked files present (use git add to track)
{noformat}
Because the local tree is 48 commits behind the head of the 0.98 branch, the
contributor's patch based on the head of 0.98 branch cannot cleanly apply.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-13081) Branch precommit builds are not updating to branch head before patch application
[
https://issues.apache.org/jira/browse/HBASE-13081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell updated HBASE-13081:
---
Assignee: Enis Soztutar
Thanks Enis!
Branch precommit builds are not updating to branch head before patch
application
Key: HBASE-13081
URL: https://issues.apache.org/jira/browse/HBASE-13081
Project: HBase
Issue Type: Bug
Reporter: Andrew Purtell
Assignee: Enis Soztutar
See for example
https://builds.apache.org/job/PreCommit-HBASE-Build/12922//console
{noformat}
git checkout 0.98
Previous HEAD position was 03d8918... HBASE-13069 Thrift Http Server returns
an error code of 500 instead of 401 when authentication fails (Srikanth
Srungarapu)
Switched to branch '0.98'
Your branch is behind 'origin/0.98' by 48 commits, and can be fast-forwarded.
(use git pull to update your local branch)
git status
On branch 0.98
Your branch is behind 'origin/0.98' by 48 commits, and can be fast-forwarded.
(use git pull to update your local branch)
Untracked files:
(use git add file... to include in what will be committed)
patchprocess/
nothing added to commit but untracked files present (use git add to track)
{noformat}
Because the local tree is 48 commits behind the head of the 0.98 branch, the
contributor's patch based on the head of 0.98 branch cannot cleanly apply.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-13070) Fix TestCacheOnWrite
[ https://issues.apache.org/jira/browse/HBASE-13070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhangduo updated HBASE-13070: - Attachment: HBASE-13070-0.98.patch Patch for 0.98. BlockCacheKey uses DataBlockEncoding as part of key on 0.98, but it is not returned within CachedBlock, so just use evictBlocksByHfileName to clearBlockCache. And LruBlockCache.clearCache does not reset elements count on 0.98. Add one line. I suggest to wait for the PreCommit-Build to finish before commiting the patch since it is not just a 'resolve conflicts' patch. Thanks. [~apurtell] [~tedyu] Fix TestCacheOnWrite Key: HBASE-13070 URL: https://issues.apache.org/jira/browse/HBASE-13070 Project: HBase Issue Type: Bug Components: test Affects Versions: 2.0.0 Reporter: zhangduo Assignee: zhangduo Fix For: 2.0.0, 1.0.1, 1.1.0, 0.98.11 Attachments: HBASE-13070-0.98.patch, HBASE-13070.patch, HBASE-13070_1.patch, HBASE-13070_2.patch TestCacheOnWrite uses TestHFileWriterV2.randomOrderedKey to generate a random byte array, then use first 32 bytes as row and remaining part as family and qualifier. But TestHFileWriterV2.randomOrderedKey may return a byte array only contains 32 bytes, so there will be zero length family and qualifier. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13067) Fix caching of stubs to allow IP address changes of restarted remote servers
[ https://issues.apache.org/jira/browse/HBASE-13067?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14329954#comment-14329954 ] Enis Soztutar commented on HBASE-13067: --- +1. This forces a resolution of hostname, but we are doing that anyway down the RPC layer with the same exact mechanism to open up a socket connection. This just moves the resolution earlier so that the stub's cache is ip+hostname based. Fix caching of stubs to allow IP address changes of restarted remote servers Key: HBASE-13067 URL: https://issues.apache.org/jira/browse/HBASE-13067 Project: HBase Issue Type: Sub-task Reporter: Devaraj Das Assignee: Devaraj Das Fix For: 1.1.0 Attachments: 13067-1.txt -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-13054) Provide more tracing information for locking/latching events.
[ https://issues.apache.org/jira/browse/HBASE-13054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajeshbabu Chintaguntla updated HBASE-13054: Attachment: HBASE-13056_branch-1.patch Patch for branch-1 Provide more tracing information for locking/latching events. - Key: HBASE-13054 URL: https://issues.apache.org/jira/browse/HBASE-13054 Project: HBase Issue Type: Improvement Reporter: Rajeshbabu Chintaguntla Assignee: Rajeshbabu Chintaguntla Fix For: 2.0.0, 1.0.1, 1.1.0, 0.98.11 Attachments: HBASE-13054.patch, HBASE-13054_v2.patch, HBASE-13054_v3.patch, HBASE-13056_branch-1.patch Currently not much tracing information available for locking and latching events like row level locking during do mini batch mutations, region level locking during flush, close and so on. It will be better to add the trace information for such events so that it will be useful for finding time spent on locking and waiting time on locks while analyzing performance issues in queries using trace information. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-13070) Fix TestCacheOnWrite
[ https://issues.apache.org/jira/browse/HBASE-13070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-13070: --- Resolution: Fixed Status: Resolved (was: Patch Available) Integrated to 0.98 branch. Thanks Duo. Fix TestCacheOnWrite Key: HBASE-13070 URL: https://issues.apache.org/jira/browse/HBASE-13070 Project: HBase Issue Type: Bug Components: test Affects Versions: 2.0.0 Reporter: zhangduo Assignee: zhangduo Fix For: 2.0.0, 1.0.1, 1.1.0, 0.98.11 Attachments: HBASE-13070-0.98.patch, HBASE-13070.patch, HBASE-13070_1.patch, HBASE-13070_2.patch TestCacheOnWrite uses TestHFileWriterV2.randomOrderedKey to generate a random byte array, then use first 32 bytes as row and remaining part as family and qualifier. But TestHFileWriterV2.randomOrderedKey may return a byte array only contains 32 bytes, so there will be zero length family and qualifier. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12990) MetaScanner should be replaced by MetaTableAccessor
[ https://issues.apache.org/jira/browse/HBASE-12990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14330007#comment-14330007 ] stack commented on HBASE-12990: --- [~octo47] How this going? Let me know if you want to try something or if can help. MetaScanner should be replaced by MetaTableAccessor --- Key: HBASE-12990 URL: https://issues.apache.org/jira/browse/HBASE-12990 Project: HBase Issue Type: Improvement Components: Client Affects Versions: 2.0.0 Reporter: Andrey Stepachev Assignee: Andrey Stepachev Attachments: HBASE-12990.patch, HBASE-12990.v2.patch, HBASE-12990.v3.patch, HBASE-12990.v4.patch MetaScanner and MetaTableAccessor do similar things, but seems they tend to diverge. Let's have only one thing to enquiry META. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13071) Hbase Streaming Scan Feature
[ https://issues.apache.org/jira/browse/HBASE-13071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14330005#comment-14330005 ] stack commented on HBASE-13071: --- [~eshcar] Design doc looks straightforward... no continental shifts going on here. One question for you is if you are 'up' on all that has gone on in this space previous? If not, let me dig it up for you (In particular, an experiment that added faux streaming via a hacked up new port on the server that allowed client via a new channel get a KV/Cell at a time with nice improvements in throughput) On 'hbase.client.scanner.async.prefetch=true', let me suggest that default is that this feature is on, not off, by default. Why would you want current behavior if this is available. In fact, IMO, don't bother offering this config presuming the throughput is better after this change as I expect it wil be. Hbase Streaming Scan Feature Key: HBASE-13071 URL: https://issues.apache.org/jira/browse/HBASE-13071 Project: HBase Issue Type: New Feature Reporter: Eshcar Hillel Attachments: HBaseStreamingScanDesign.pdf A scan operation iterates over all rows of a table or a subrange of the table. The synchronous nature in which the data is served at the client side hinders the speed the application traverses the data: it increases the overall processing time, and may cause a great variance in the times the application waits for the next piece of data. The scanner next() method at the client side invokes an RPC to the regionserver and then stores the results in a cache. The application can specify how many rows will be transmitted per RPC; by default this is set to 100 rows. The cache can be considered as a producer-consumer queue, where the hbase client pushes the data to the queue and the application consumes it. Currently this queue is synchronous, i.e., blocking. More specifically, when the application consumed all the data from the cache --- so the cache is empty --- the hbase client retrieves additional data from the server and re-fills the cache with new data. During this time the application is blocked. Under the assumption that the application processing time can be balanced by the time it takes to retrieve the data, an asynchronous approach can reduce the time the application is waiting for data. We attach a design document. We also have a patch that is based on a private branch, and some evaluation results of this code. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13065) Increasing -Xmx when running TestDistributedLogSplitting
[
https://issues.apache.org/jira/browse/HBASE-13065?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14330016#comment-14330016
]
zhangduo commented on HBASE-13065:
--
[~stack] Seems HBase-TRUNK jenkins is running on a 32bit machine? So there are
only 4GB address space. We increase heap from 1900m to 2800m, then there is
much less address space for native memory.
In TestAcidGuarantees, we create connection per read write thread but do not
close it, so we create lots of AsyncRpcClient instance. Each instance has its
own netty thread pool, so there are too many threads and will easily exceed the
4GB address space(VIRT, not RES) limit and then make the test fail.
{noformat}
java.lang.OutOfMemoryError: unable to create new native thread
at java.lang.Thread.start0(Native Method)
at java.lang.Thread.start(Thread.java:693)
at
io.netty.util.concurrent.SingleThreadEventExecutor.shutdownGracefully(SingleThreadEventExecutor.java:557)
at
io.netty.util.concurrent.MultithreadEventExecutorGroup.shutdownGracefully(MultithreadEventExecutorGroup.java:146)
at
io.netty.util.concurrent.AbstractEventExecutorGroup.shutdownGracefully(AbstractEventExecutorGroup.java:69)
at
org.apache.hadoop.hbase.ipc.AsyncRpcClient.close(AsyncRpcClient.java:253)
at
org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.internalClose(ConnectionManager.java:2373)
at
org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.close(ConnectionManager.java:2386)
at
org.apache.hadoop.hbase.HBaseTestingUtility.shutdownMiniCluster(HBaseTestingUtility.java:1055)
at
org.apache.hadoop.hbase.TestAcidGuarantees.testGetAtomicity(TestAcidGuarantees.java:346)
{noformat}
I wonder why we run jenkins job on 32bit machines? Seems PreCommit-Build is run
on 64bit machines.
Increasing -Xmx when running TestDistributedLogSplitting
Key: HBASE-13065
URL: https://issues.apache.org/jira/browse/HBASE-13065
Project: HBase
Issue Type: Bug
Components: test
Reporter: zhangduo
Assignee: zhangduo
Fix For: 2.0.0, 1.0.1, 1.1.0, 0.98.11
Attachments: 13065-fix.txt
Found this in PreCommit Build reports
https://builds.apache.org/job/PreCommit-HBASE-Build/12885/artifact/hbase-server/target/surefire-reports/org.apache.hadoop.hbase.master.TestDistributedLogSplitting-output.txt
{noformat}
2015-02-18 03:45:42,141 WARN [RS:4;asf901:41265] util.Sleeper(97): We slept
59018ms instead of 1000ms, this is likely due to a long garbage collecting
pause and it's usually bad, see
http://hbase.apache.org/book.html#trouble.rs.runtime.zkexpired
2015-02-18 03:45:26,750 WARN [JvmPauseMonitor]
util.JvmPauseMonitor$Monitor(167): Detected pause in JVM or host machine (eg
GC): pause of approximately 39767ms
GC pool 'PS MarkSweep' had collection(s): count=65 time=47720ms
{noformat}
Maybe we should increase the max heap size since this test starts 6
regionservers.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-11544) [Ergonomics] hbase.client.scanner.caching is dogged and will try to return batch even if it means OOME
[ https://issues.apache.org/jira/browse/HBASE-11544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14329780#comment-14329780 ] Jonathan Lawlor commented on HBASE-11544: - Going to investigate this issue further [Ergonomics] hbase.client.scanner.caching is dogged and will try to return batch even if it means OOME -- Key: HBASE-11544 URL: https://issues.apache.org/jira/browse/HBASE-11544 Project: HBase Issue Type: Bug Reporter: stack Assignee: Jonathan Lawlor Priority: Critical Labels: beginner Attachments: HBASE-11544-v1.patch, HBASE-11544-v2.patch, HBASE-11544-v3.patch Running some tests, I set hbase.client.scanner.caching=1000. Dataset has large cells. I kept OOME'ing. Serverside, we should measure how much we've accumulated and return to the client whatever we've gathered once we pass out a certain size threshold rather than keep accumulating till we OOME. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13081) Branch precommit builds are not updating to branch head before patch application
[
https://issues.apache.org/jira/browse/HBASE-13081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14329816#comment-14329816
]
Enis Soztutar commented on HBASE-13081:
---
Yes, this is my bad. I had a fix I did not push yet. Let me do that now.
Branch precommit builds are not updating to branch head before patch
application
Key: HBASE-13081
URL: https://issues.apache.org/jira/browse/HBASE-13081
Project: HBase
Issue Type: Bug
Reporter: Andrew Purtell
See for example
https://builds.apache.org/job/PreCommit-HBASE-Build/12922//console
{noformat}
git checkout 0.98
Previous HEAD position was 03d8918... HBASE-13069 Thrift Http Server returns
an error code of 500 instead of 401 when authentication fails (Srikanth
Srungarapu)
Switched to branch '0.98'
Your branch is behind 'origin/0.98' by 48 commits, and can be fast-forwarded.
(use git pull to update your local branch)
git status
On branch 0.98
Your branch is behind 'origin/0.98' by 48 commits, and can be fast-forwarded.
(use git pull to update your local branch)
Untracked files:
(use git add file... to include in what will be committed)
patchprocess/
nothing added to commit but untracked files present (use git add to track)
{noformat}
Because the local tree is 48 commits behind the head of the 0.98 branch, the
contributor's patch based on the head of 0.98 branch cannot cleanly apply.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13081) Branch precommit builds are not updating to branch head before patch application
[
https://issues.apache.org/jira/browse/HBASE-13081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14329853#comment-14329853
]
Hadoop QA commented on HBASE-13081:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12699979/foo_0.98.patch
against 0.98 branch at commit a8555f65f0551569f07f6a7d207ddc0e56fad03a.
ATTACHMENT ID: 12699979
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/12931//console
This message is automatically generated.
Branch precommit builds are not updating to branch head before patch
application
Key: HBASE-13081
URL: https://issues.apache.org/jira/browse/HBASE-13081
Project: HBase
Issue Type: Bug
Reporter: Andrew Purtell
Assignee: Enis Soztutar
Attachments: foo_0.98.patch, foo_branch-1.patch, hbase-13081.patch
See for example
https://builds.apache.org/job/PreCommit-HBASE-Build/12922//console
{noformat}
git checkout 0.98
Previous HEAD position was 03d8918... HBASE-13069 Thrift Http Server returns
an error code of 500 instead of 401 when authentication fails (Srikanth
Srungarapu)
Switched to branch '0.98'
Your branch is behind 'origin/0.98' by 48 commits, and can be fast-forwarded.
(use git pull to update your local branch)
git status
On branch 0.98
Your branch is behind 'origin/0.98' by 48 commits, and can be fast-forwarded.
(use git pull to update your local branch)
Untracked files:
(use git add file... to include in what will be committed)
patchprocess/
nothing added to commit but untracked files present (use git add to track)
{noformat}
Because the local tree is 48 commits behind the head of the 0.98 branch, the
contributor's patch based on the head of 0.98 branch cannot cleanly apply.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-13056) Refactor table.jsp code to remove repeated code and make it easier to add new checks
[
https://issues.apache.org/jira/browse/HBASE-13056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell updated HBASE-13056:
---
Resolution: Fixed
Fix Version/s: 0.98.11
1.0.1
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
Pushed to remaining branches. Not a functional change. Poked around the UI with
changes applied, looks fine.
Refactor table.jsp code to remove repeated code and make it easier to add new
checks
Key: HBASE-13056
URL: https://issues.apache.org/jira/browse/HBASE-13056
Project: HBase
Issue Type: Improvement
Affects Versions: 2.0.0
Reporter: Vikas Vishwakarma
Assignee: Vikas Vishwakarma
Fix For: 2.0.0, 1.0.1, 1.1.0, 0.98.11
Attachments: HBASE-13056-0.98.patch, HBASE-13056.patch
While trying to fix HBASE-13001, I realized that there is lot of html code
repetition in table.jsp which is making addition of new checks slightly
difficult in the sense I will have to:
1. Add the check at multiple places in the code
Or
2. Repeat the html code again for the new check
So I am proposing to re-factor table.jsp code such that the common html
header/body is loaded without any condition check and then we generate the
condition specific html code
snapshot.jsp follows the same format as explained below:
{noformat}
Current implementation:
if( x ) {
title_x
common_html_header
common_html_body
x_specific_html_body
} else {
title_y
common_html_header
common_html_body
y_specific_html_body
}
New Implementation:
==
if( x ) {
title_x
} else {
title_y
}
common_html_header
common_html_body
if( x ) {
x_specific_html_body
} else {
y_specific_html_body
}
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (HBASE-9604) Add metric on short-circuit reads
[ https://issues.apache.org/jira/browse/HBASE-9604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack resolved HBASE-9604. -- Resolution: Duplicate Dup of HBASE-8868 Add metric on short-circuit reads - Key: HBASE-9604 URL: https://issues.apache.org/jira/browse/HBASE-9604 Project: HBase Issue Type: Task Components: metrics Reporter: stack Assignee: stack Fix For: 2.0.0, 1.1.0 Got this from a Colin message this afternoon: There are HDFS statistics that HBase could be checking by calling DFSInputStream#getReadStatistics. This tells you how many of your reads have been remote, local, short-circuit, etc. You could file an HBase JIRA for them to roll those up into the HBase stats. Seems like a good idea to me. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-8868) add metric to report client shortcircuit reads
[ https://issues.apache.org/jira/browse/HBASE-8868?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14330014#comment-14330014 ] stack commented on HBASE-8868: -- See related HBASE-9604 add metric to report client shortcircuit reads -- Key: HBASE-8868 URL: https://issues.apache.org/jira/browse/HBASE-8868 Project: HBase Issue Type: Improvement Components: metrics, regionserver Affects Versions: 0.94.8, 0.95.1 Reporter: Viral Bajaria Priority: Minor With the availability of shortcircuit reads, when the feature is enabled there is no metric which exposes how many times the regionserver was able to shortcircuit the read and not make a IPC to the datanode. It will be great to add the metric and expose it via Ganglia. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13058) Hbase shell command 'scan' for non existent table shows unnecessary info for one unrelated existent table.
[
https://issues.apache.org/jira/browse/HBASE-13058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14330012#comment-14330012
]
stack commented on HBASE-13058:
---
Patch looks good. How did you discover we need the second hunk, the extra check
for table not found? Otherwise +1
Hbase shell command 'scan' for non existent table shows unnecessary info for
one unrelated existent table.
--
Key: HBASE-13058
URL: https://issues.apache.org/jira/browse/HBASE-13058
Project: HBase
Issue Type: Bug
Components: Client
Reporter: Abhishek Kumar
Priority: Trivial
Attachments: 0001-HBASE-13058-Error-messages-in-scan-table.patch,
0001-HBASE-13058-shell-unknown-table-message-update.patch
When scanning for a non existent table in hbase shell, error message in shell
sometimes(based on META table content) displays completely unrelated table
info , which seems to be unnecessary and inconsistent with other error
messages:
{noformat}
hbase(main):016:0 scan 'noTable'
ROW COLUMN+CELL
ERROR: Unknown table Table 'noTable' was not found, got: hbase:namespace.!
-
hbase(main):017:0 scan '01_noTable'
ROW COLUMN+CELL
ERROR: Unknown table 01_noTable!
--
{noformat}
Its happening when doing a META table scan (to locate input table ) and
scanner stops at row of another table (beyond which table can not exist) in
ConnectionManager.locateRegionInMeta:
{noformat}
private RegionLocations locateRegionInMeta(TableName tableName, byte[] row,
boolean useCache, boolean retry, int replicaId) throws
IOException {
.
// possible we got a region of a different table...
if (!regionInfo.getTable().equals(tableName)) {
throw new TableNotFoundException(
Table ' + tableName + ' was not found, got: +
regionInfo.getTable() + .);
}
...
...
{noformat}
Here, we can simply put a debug message(if required) and just throw the
TableNotFoundException(tableName) with only tableName instead of with
scanner positioned row.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13056) Refactor table.jsp code to remove repeated code and make it easier to add new checks
[
https://issues.apache.org/jira/browse/HBASE-13056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14330018#comment-14330018
]
Hudson commented on HBASE-13056:
SUCCESS: Integrated in HBase-0.98 #864 (See
[https://builds.apache.org/job/HBase-0.98/864/])
HBASE-13056 Refactor table.jsp code to remove repeated code and make it easier
to add new checks (Vikas Vishwakarma) (apurtell: rev
b8e273d880af757d72655b2e9dce16817839d96b)
* hbase-server/src/main/resources/hbase-webapps/master/table.jsp
Refactor table.jsp code to remove repeated code and make it easier to add new
checks
Key: HBASE-13056
URL: https://issues.apache.org/jira/browse/HBASE-13056
Project: HBase
Issue Type: Improvement
Affects Versions: 2.0.0
Reporter: Vikas Vishwakarma
Assignee: Vikas Vishwakarma
Fix For: 2.0.0, 1.0.1, 1.1.0, 0.98.11
Attachments: HBASE-13056-0.98.patch, HBASE-13056.patch
While trying to fix HBASE-13001, I realized that there is lot of html code
repetition in table.jsp which is making addition of new checks slightly
difficult in the sense I will have to:
1. Add the check at multiple places in the code
Or
2. Repeat the html code again for the new check
So I am proposing to re-factor table.jsp code such that the common html
header/body is loaded without any condition check and then we generate the
condition specific html code
snapshot.jsp follows the same format as explained below:
{noformat}
Current implementation:
if( x ) {
title_x
common_html_header
common_html_body
x_specific_html_body
} else {
title_y
common_html_header
common_html_body
y_specific_html_body
}
New Implementation:
==
if( x ) {
title_x
} else {
title_y
}
common_html_header
common_html_body
if( x ) {
x_specific_html_body
} else {
y_specific_html_body
}
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-13032) Migration of states should be performed once META is assigned and onlined.
[ https://issues.apache.org/jira/browse/HBASE-13032?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-13032: -- Resolution: Fixed Fix Version/s: 1.1.0 2.0.0 Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Pushed to master branch. Only thing I pushed to branch-1.1 was change in comment in HMaster since HMaster already had the change (did not add the test to branch-1 since failed dependencies). Thanks for the patch [~octo47] Migration of states should be performed once META is assigned and onlined. -- Key: HBASE-13032 URL: https://issues.apache.org/jira/browse/HBASE-13032 Project: HBase Issue Type: Bug Affects Versions: 2.0.0 Reporter: Andrey Stepachev Assignee: Andrey Stepachev Fix For: 2.0.0, 1.1.0 Attachments: HBASE-13032.patch, HBASE-13032.patch, HBASE-13032.patch, HBASE-13032.patch Thread 82 (localhost:58722.activeMasterManager): State: TIMED_WAITING Blocked count: 43 Waited count: 783 Stack: java.lang.Object.wait(Native Method) org.apache.hadoop.hbase.client.AsyncProcess.waitForMaximumCurrentTasks(AsyncProcess.java:1656) org.apache.hadoop.hbase.client.AsyncProcess.waitForAllPreviousOpsAndReset(AsyncProcess.java:1685) org.apache.hadoop.hbase.client.BufferedMutatorImpl.backgroundFlushCommits(BufferedMutatorImpl.java:208) org.apache.hadoop.hbase.client.BufferedMutatorImpl.flush(BufferedMutatorImpl.java:183) org.apache.hadoop.hbase.client.HTable.flushCommits(HTable.java:1395) org.apache.hadoop.hbase.client.HTable.put(HTable.java:1007) org.apache.hadoop.hbase.MetaTableAccessor.put(MetaTableAccessor.java:1116) org.apache.hadoop.hbase.MetaTableAccessor.putToMetaTable(MetaTableAccessor.java:1106) org.apache.hadoop.hbase.MetaTableAccessor.updateTableState(MetaTableAccessor.java:1429) org.apache.hadoop.hbase.MetaTableAccessor.updateTableState(MetaTableAccessor.java:941) org.apache.hadoop.hbase.master.TableStateManager.udpateMetaState(TableStateManager.java:181) org.apache.hadoop.hbase.master.TableStateManager.setTableState(TableStateManager.java:68) org.apache.hadoop.hbase.master.HMaster.initializeZKBasedSystemTrackers(HMaster.java:573) org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:640) org.apache.hadoop.hbase.master.HMaster.access$500(HMaster.java:170) org.apache.hadoop.hbase.master.HMaster$1.run(HMaster.java:1462) java.lang.Thread.run(Thread.java:745) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13065) Increasing -Xmx when running TestDistributedLogSplitting
[
https://issues.apache.org/jira/browse/HBASE-13065?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14330025#comment-14330025
]
stack commented on HBASE-13065:
---
Configuration for machines on trunk is (ubuntu||Hadoop) !jenkins-cloud-4GB
!H11 Some one of those must be the 32bitter. I do not have direct access.
Let me ask a fellow who I know has access. Thanks for digging in [~Apache9]
Increasing -Xmx when running TestDistributedLogSplitting
Key: HBASE-13065
URL: https://issues.apache.org/jira/browse/HBASE-13065
Project: HBase
Issue Type: Bug
Components: test
Reporter: zhangduo
Assignee: zhangduo
Fix For: 2.0.0, 1.0.1, 1.1.0, 0.98.11
Attachments: 13065-fix.txt
Found this in PreCommit Build reports
https://builds.apache.org/job/PreCommit-HBASE-Build/12885/artifact/hbase-server/target/surefire-reports/org.apache.hadoop.hbase.master.TestDistributedLogSplitting-output.txt
{noformat}
2015-02-18 03:45:42,141 WARN [RS:4;asf901:41265] util.Sleeper(97): We slept
59018ms instead of 1000ms, this is likely due to a long garbage collecting
pause and it's usually bad, see
http://hbase.apache.org/book.html#trouble.rs.runtime.zkexpired
2015-02-18 03:45:26,750 WARN [JvmPauseMonitor]
util.JvmPauseMonitor$Monitor(167): Detected pause in JVM or host machine (eg
GC): pause of approximately 39767ms
GC pool 'PS MarkSweep' had collection(s): count=65 time=47720ms
{noformat}
Maybe we should increase the max heap size since this test starts 6
regionservers.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13045) Ensure MapReduce javadoc example works
[ https://issues.apache.org/jira/browse/HBASE-13045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14330030#comment-14330030 ] stack commented on HBASE-13045: --- Skimmed. +1. Ensure MapReduce javadoc example works -- Key: HBASE-13045 URL: https://issues.apache.org/jira/browse/HBASE-13045 Project: HBase Issue Type: Improvement Components: mapreduce Reporter: Sean Busbey Assignee: Sean Busbey Priority: Minor Fix For: 0.98.11 Attachments: HBASE-13045-0.98.1.patch.txt Backport the docs / test parts of HBASE-13027 that are applicable to 0.98. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-5238) Add a log4j category for all edits to META/ROOT
[ https://issues.apache.org/jira/browse/HBASE-5238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14329123#comment-14329123 ] stack commented on HBASE-5238: -- This looks great [~octo47] +1 Mind pasting a sample of what log looks like in here. Needs a release note too so folks get to know of this cool trick. Add a log4j category for all edits to META/ROOT --- Key: HBASE-5238 URL: https://issues.apache.org/jira/browse/HBASE-5238 Project: HBase Issue Type: New Feature Components: regionserver Affects Versions: 2.0.0 Reporter: Todd Lipcon Assignee: Andrey Stepachev Priority: Minor Labels: beginner Attachments: HBASE-5238.patch Occasionally we run into bugs that have corrected META and written some bad data to META/ROOT but it's difficult to understand the order in which things happened. One option is to dump the HLog contents from the servers that hosted META at that time, but then it's interspersed with all other data. It would be nice to add a Log4j Logger to which we log all edits being applied to META and ROOT in textual form at DEBUG level. Then it would be easier to do a cluster-wide log grep to see what happened when. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12953) RegionServer is not functionally working with AysncRpcClient in secure mode
[
https://issues.apache.org/jira/browse/HBASE-12953?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14329035#comment-14329035
]
zhangduo commented on HBASE-12953:
--
This is found in a failed run,
https://builds.apache.org/job/PreCommit-HBASE-Build/12918/artifact/hbase-shell/target/surefire-reports/org.apache.hadoop.hbase.client.TestShell-output.txt
{noformat}
2015-02-20 01:44:17,961 DEBUG [B.defaultRpcServer.handler=3,queue=0,port=47118]
visibility.DefaultVisibilityLabelServiceImpl(224): Adding the label
TEST_VISIBILITY
2015-02-20 01:44:17,966 INFO [main]
client.ConnectionManager$HConnectionImplementation(1661): Closing zookeeper
sessionid=0x14ba4a6e11b015f
2015-02-20 01:44:17,967 DEBUG [main] ipc.AsyncRpcClient(244): Stopping async
HBase RPC client
{noformat}
And this is in a success run,
https://builds.apache.org/job/PreCommit-HBASE-Build/12926/artifact/hbase-shell/target/surefire-reports/org.apache.hadoop.hbase.client.TestShell-output.txt
{noformat}
2015-02-20 14:12:48,681 DEBUG [B.defaultRpcServer.handler=3,queue=0,port=43394]
visibility.DefaultVisibilityLabelServiceImpl(224): Adding the label
TEST_VISIBILITY
2015-02-20 14:12:48,684 DEBUG [main-EventThread]
zookeeper.ZooKeeperWatcher(381): master:57132-0x14ba75420fb,
quorum=localhost:63493, baseZNode=/hbase Received ZooKeeper Event,
type=NodeDataChanged, state=SyncConnected, path=/hbase/visibility/labels
2015-02-20 14:12:48,684 DEBUG [main-EventThread]
zookeeper.ZooKeeperWatcher(381): regionserver:43394-0x14ba75420fb0001,
quorum=localhost:63493, baseZNode=/hbase Received ZooKeeper Event,
type=NodeDataChanged, state=SyncConnected, path=/hbase/visibility/labels
2015-02-20 14:12:48,686 INFO [main]
client.ConnectionManager$HConnectionImplementation(1661): Closing zookeeper
sessionid=0x14ba75420fb015f
2015-02-20 14:12:48,688 DEBUG [main] ipc.AsyncRpcClient(244): Stopping async
HBase RPC client
{noformat}
The failed one missing the zk watcher logs.
I'm not familiar with Visibility Labels. I skimmed the code in
DefaultVisibilityLabelServiceImpl, seems we do not update labelsCache directly
when addLabels and just write it to zk and then let a zk watcher notify us to
update labelsCache? So there could be situation that after addLabels returns
successfully, the labelsCache still does not contain the added labels? And
setAuth will check labelsCache so there could be a cache miss?
[~stack], is there anybody can give some suggestions? Thanks.
RegionServer is not functionally working with AysncRpcClient in secure mode
---
Key: HBASE-12953
URL: https://issues.apache.org/jira/browse/HBASE-12953
Project: HBase
Issue Type: Bug
Components: security
Affects Versions: 2.0.0, 1.1.0
Reporter: Ashish Singhi
Assignee: zhangduo
Priority: Critical
Fix For: 2.0.0, 1.1.0
Attachments: HBASE-12953.patch, HBASE-12953_1.patch,
HBASE-12953_2.patch, HBASE-12953_2.patch, HBASE-12953_2.patch,
HBASE-12953_2.patch, HBASE-12953_2.patch, HBASE-12953_3 (2).patch,
HBASE-12953_3 (2).patch, HBASE-12953_3.patch, HBASE-12953_3.patch,
HBASE-12953_3.patch, testcase.patch
HBase version 2.0.0
Default value for {{hbase.rpc.client.impl}} is set to AsyncRpcClient.
When trying to install HBase with Kerberos, RegionServer is not working
functionally.
The following log is logged in its log file
{noformat}
2015-02-02 14:59:05,407 WARN [AsyncRpcChannel-pool1-t1]
channel.DefaultChannelPipeline: An exceptionCaught() event was fired, and it
reached at the tail of the pipeline. It usually means the last handler in the
pipeline did not handle the exception.
io.netty.channel.ChannelPipelineException:
org.apache.hadoop.hbase.security.SaslClientHandler.handlerAdded() has thrown
an exception; removed.
at
io.netty.channel.DefaultChannelPipeline.callHandlerAdded0(DefaultChannelPipeline.java:499)
at
io.netty.channel.DefaultChannelPipeline.callHandlerAdded(DefaultChannelPipeline.java:481)
at
io.netty.channel.DefaultChannelPipeline.addFirst0(DefaultChannelPipeline.java:114)
at
io.netty.channel.DefaultChannelPipeline.addFirst(DefaultChannelPipeline.java:97)
at
io.netty.channel.DefaultChannelPipeline.addFirst(DefaultChannelPipeline.java:235)
at
io.netty.channel.DefaultChannelPipeline.addFirst(DefaultChannelPipeline.java:214)
at
org.apache.hadoop.hbase.ipc.AsyncRpcChannel$2.operationComplete(AsyncRpcChannel.java:194)
at
org.apache.hadoop.hbase.ipc.AsyncRpcChannel$2.operationComplete(AsyncRpcChannel.java:157)
at
io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:680)
at
io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:603)
at
[jira] [Commented] (HBASE-11869) Support snapshot owner
[ https://issues.apache.org/jira/browse/HBASE-11869?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14329081#comment-14329081 ] Devaraj Das commented on HBASE-11869: - +1 for 1.1 Support snapshot owner -- Key: HBASE-11869 URL: https://issues.apache.org/jira/browse/HBASE-11869 Project: HBase Issue Type: Improvement Reporter: Liu Shaohui Assignee: Liu Shaohui Priority: Minor Fix For: 2.0.0 Attachments: HBASE-11869-1.0-v1.diff, HBASE-11869-trunk-v1.diff, HBASE-11869-trunk-v3.diff, HBASE-11869-trunk-v4.diff In current codebase, the table snapshot operations only can be done by the global admin , not by the table admin. There is a multi-tenant hbase cluster, each table has different snapshot policies, eg: do snapshot per week, or snapshot after the new data are imported. We want to release the snapshot permission to each table admin. According to [~mbertozzi]'s suggestion, we implement the snapshot owner feature. * The user with table admin permission can create snapshot and the owner of this snapshot is this user. * The owner of snapshot can delete and restore the snapshot. * Only the user with global admin permission can clone a snapshot, for this operation creates a new table. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-12954) Ability impaired using HBase on multihomed hosts
[
https://issues.apache.org/jira/browse/HBASE-12954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14329057#comment-14329057
]
Devaraj Das commented on HBASE-12954:
-
[~yuzhih...@gmail.com] there might be an issue at the RPC layer if security is
turned on. For the login() calls, the hostname that is used is the custom
hostname (and that seems to be right). When the RpcClient tries to setup a
secure communication to a remote server, it constructs the principal of the
remote server (usually the principals are of the form hbase/_HOST@REALM).
During the principal construction, it does a reverse DNS lookup to get the
server name and the _HOST is replaced with that server name to get the
principal that the RpcClient then uses. The question is if a reverse DNS is
done, whether the custom hostname would be returned or not. If not, we'd need
to address the issue.
Ability impaired using HBase on multihomed hosts
Key: HBASE-12954
URL: https://issues.apache.org/jira/browse/HBASE-12954
Project: HBase
Issue Type: Bug
Affects Versions: 0.98.4
Reporter: Clay B.
Assignee: Ted Yu
Priority: Minor
Attachments: 12954-v1.txt, 12954-v10.txt, 12954-v11.txt,
12954-v12.txt, 12954-v12.txt, 12954-v12.txt, 12954-v13.txt, 12954-v7.txt,
12954-v8.txt, Hadoop Three Interfaces.png
For HBase clusters running on unusual networks (such as NAT'd cloud
environments or physical machines with multiple IP's per network interface)
it would be ideal to have a way to both specify:
# which IP interface to which HBase master or region-server will bind
# what hostname HBase will advertise in Zookeeper both for a master or
region-server process
While efforts such as HBASE-8640 go a long way to normalize these two sources
of information, it is not possible in the current design of the properties
available to an administrator for these to be unambiguously specified.
One has been able to request {{hbase.master.ipc.address}} or
{{hbase.regionserver.ipc.address}} but one can not specify the desired HBase
{{hbase.master.hostname}}. (It was removed in HBASE-1357, further I am
unaware of a region-server equivalent.)
I use a configuration management system to generate all of my configuration
files on a per-machine basis. As such, an option to generate a file
specifying exactly which hostname to use would be helpful.
Today, specifying the bind address for HBase works and one can use an
HBase-only DNS for faking what to put in Zookeeper but this is far from
ideal. Network interfaces have no intrinsic IP address, nor hostname.
Specifing a DNS server is awkward as the DNS server may differ from the
system's resolver and is a single IP address. Similarly, on hosts which use a
transient VIP (e.g. through keepalived) for other services, it means there's
a seemingly non-deterministic hostname choice made by HBase depending on the
state of the VIP at daemon start-up time.
I will attach two networking examples I use which become very difficult to
manage under the current properties.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13077) BoundedCompletionService doesn't pass trace info to server
[ https://issues.apache.org/jira/browse/HBASE-13077?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14329118#comment-14329118 ] stack commented on HBASE-13077: --- [~enis] It is not a sinker. Tracing does not work end-to-end currently (Our Colin is actively getting incubator htrace to work end-to-end across hbase and hdfs) so you cannot sink the RC because trace is broke. BoundedCompletionService doesn't pass trace info to server -- Key: HBASE-13077 URL: https://issues.apache.org/jira/browse/HBASE-13077 Project: HBase Issue Type: Bug Components: hbase Affects Versions: 1.0.0, 2.0.0, 1.1.0 Reporter: Jeffrey Zhong Assignee: Jeffrey Zhong Attachments: HBASE-13077.patch Today [~ndimiduk] I found that BoundedCompletionService doesn't pass htrace info to server. This issue causes scan doesn't pass trace info to server. [~enis] FYI. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-13054) Provide more tracing information for locking/latching events.
[ https://issues.apache.org/jira/browse/HBASE-13054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajeshbabu Chintaguntla updated HBASE-13054: Resolution: Fixed Status: Resolved (was: Patch Available) Pushed to 0.98+ branches. Thanks for review [~apurtell],[~eclark]. Provide more tracing information for locking/latching events. - Key: HBASE-13054 URL: https://issues.apache.org/jira/browse/HBASE-13054 Project: HBase Issue Type: Improvement Reporter: Rajeshbabu Chintaguntla Assignee: Rajeshbabu Chintaguntla Fix For: 2.0.0, 1.0.1, 1.1.0, 0.98.11 Attachments: HBASE-13054.patch, HBASE-13054_0.98.patch, HBASE-13054_v2.patch, HBASE-13054_v3.patch, HBASE-13056_branch-1.patch Currently not much tracing information available for locking and latching events like row level locking during do mini batch mutations, region level locking during flush, close and so on. It will be better to add the trace information for such events so that it will be useful for finding time spent on locking and waiting time on locks while analyzing performance issues in queries using trace information. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13056) Refactor table.jsp code to remove repeated code and make it easier to add new checks
[
https://issues.apache.org/jira/browse/HBASE-13056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14330042#comment-14330042
]
Hudson commented on HBASE-13056:
FAILURE: Integrated in HBase-0.98-on-Hadoop-1.1 #822 (See
[https://builds.apache.org/job/HBase-0.98-on-Hadoop-1.1/822/])
HBASE-13056 Refactor table.jsp code to remove repeated code and make it easier
to add new checks (Vikas Vishwakarma) (apurtell: rev
b8e273d880af757d72655b2e9dce16817839d96b)
* hbase-server/src/main/resources/hbase-webapps/master/table.jsp
Refactor table.jsp code to remove repeated code and make it easier to add new
checks
Key: HBASE-13056
URL: https://issues.apache.org/jira/browse/HBASE-13056
Project: HBase
Issue Type: Improvement
Affects Versions: 2.0.0
Reporter: Vikas Vishwakarma
Assignee: Vikas Vishwakarma
Fix For: 2.0.0, 1.0.1, 1.1.0, 0.98.11
Attachments: HBASE-13056-0.98.patch, HBASE-13056.patch
While trying to fix HBASE-13001, I realized that there is lot of html code
repetition in table.jsp which is making addition of new checks slightly
difficult in the sense I will have to:
1. Add the check at multiple places in the code
Or
2. Repeat the html code again for the new check
So I am proposing to re-factor table.jsp code such that the common html
header/body is loaded without any condition check and then we generate the
condition specific html code
snapshot.jsp follows the same format as explained below:
{noformat}
Current implementation:
if( x ) {
title_x
common_html_header
common_html_body
x_specific_html_body
} else {
title_y
common_html_header
common_html_body
y_specific_html_body
}
New Implementation:
==
if( x ) {
title_x
} else {
title_y
}
common_html_header
common_html_body
if( x ) {
x_specific_html_body
} else {
y_specific_html_body
}
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13054) Provide more tracing information for locking/latching events.
[
https://issues.apache.org/jira/browse/HBASE-13054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14330043#comment-14330043
]
Hadoop QA commented on HBASE-13054:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/1271/HBASE-13054_0.98.patch
against 0.98 branch at commit a8555f65f0551569f07f6a7d207ddc0e56fad03a.
ATTACHMENT ID: 1271
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.1 2.5.2 2.6.0)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated
22 warning messages.
{color:green}+1 checkstyle{color}. The applied patch does not increase the
total number of checkstyle errors
{color:red}-1 findbugs{color}. The patch appears to introduce 4 new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
{color:red}-1 core zombie tests{color}. There are 1 zombie test(s):
at
org.apache.hadoop.hbase.regionserver.TestCompoundBloomFilter.testCompoundBloomFilter(TestCompoundBloomFilter.java:159)
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/12935//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12935//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12935//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12935//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12935//artifact/patchprocess/newPatchFindbugsWarningshbase-rest.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12935//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12935//artifact/patchprocess/newPatchFindbugsWarningshbase-annotations.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12935//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12935//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12935//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12935//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12935//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/12935//artifact/patchprocess/checkstyle-aggregate.html
Javadoc warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12935//artifact/patchprocess/patchJavadocWarnings.txt
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/12935//console
This message is automatically generated.
Provide more tracing information for locking/latching events.
-
Key: HBASE-13054
URL: https://issues.apache.org/jira/browse/HBASE-13054
Project: HBase
Issue Type: Improvement
Reporter: Rajeshbabu Chintaguntla
Assignee: Rajeshbabu Chintaguntla
Fix For: 2.0.0, 1.0.1, 1.1.0, 0.98.11
Attachments: HBASE-13054.patch, HBASE-13054_0.98.patch,
HBASE-13054_v2.patch, HBASE-13054_v3.patch, HBASE-13056_branch-1.patch
Currently not much tracing information available for locking and latching
events like row level locking during do mini batch mutations, region level
locking during flush, close and so on. It will be
[jira] [Updated] (HBASE-13001) NullPointer in master logs for table.jsp
[
https://issues.apache.org/jira/browse/HBASE-13001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-13001:
--
Resolution: Fixed
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
Pushed to master branch. Thanks for the patch [~vik.karma]
NullPointer in master logs for table.jsp
Key: HBASE-13001
URL: https://issues.apache.org/jira/browse/HBASE-13001
Project: HBase
Issue Type: Bug
Affects Versions: 2.0.0, 0.98.10
Reporter: Vikas Vishwakarma
Assignee: Vikas Vishwakarma
Priority: Trivial
Fix For: 2.0.0
Attachments: HBASE-13001.patch, Table_not_ready.png
Seeing a NullPointer issue in master logs for table.jsp probably similar to
HBASE-6607
{noformat}
2015-02-09 14:04:00,622 ERROR org.mortbay.log: /table.jsp
java.lang.NullPointerException
at
org.apache.hadoop.hbase.generated.master.table_jsp._jspService(table_jsp.java:71)
at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:98)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
at
org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
at
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1221)
at
org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter.doFilter(StaticUserWebFilter.java:109)
at
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
at
org.apache.hadoop.http.HttpServer$QuotingInputFilter.doFilter(HttpServer.java:1087)
at
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
at
org.apache.hadoop.http.NoCacheFilter.doFilter(NoCacheFilter.java:45)
at
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
at
org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399)
at
org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
at
org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
at
org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
at
org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450)
at
org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
at
org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
at org.mortbay.jetty.Server.handle(Server.java:326)
at
org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
at
org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928)
at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549)
at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
at
org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:410)
at
org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13001) NullPointer in master logs for table.jsp
[
https://issues.apache.org/jira/browse/HBASE-13001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14330046#comment-14330046
]
Vikas Vishwakarma commented on HBASE-13001:
---
many thanks [~stack] !
NullPointer in master logs for table.jsp
Key: HBASE-13001
URL: https://issues.apache.org/jira/browse/HBASE-13001
Project: HBase
Issue Type: Bug
Affects Versions: 2.0.0, 0.98.10
Reporter: Vikas Vishwakarma
Assignee: Vikas Vishwakarma
Priority: Trivial
Fix For: 2.0.0
Attachments: HBASE-13001.patch, Table_not_ready.png
Seeing a NullPointer issue in master logs for table.jsp probably similar to
HBASE-6607
{noformat}
2015-02-09 14:04:00,622 ERROR org.mortbay.log: /table.jsp
java.lang.NullPointerException
at
org.apache.hadoop.hbase.generated.master.table_jsp._jspService(table_jsp.java:71)
at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:98)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
at
org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
at
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1221)
at
org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter.doFilter(StaticUserWebFilter.java:109)
at
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
at
org.apache.hadoop.http.HttpServer$QuotingInputFilter.doFilter(HttpServer.java:1087)
at
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
at
org.apache.hadoop.http.NoCacheFilter.doFilter(NoCacheFilter.java:45)
at
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
at
org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399)
at
org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
at
org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
at
org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
at
org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450)
at
org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
at
org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
at org.mortbay.jetty.Server.handle(Server.java:326)
at
org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
at
org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928)
at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549)
at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
at
org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:410)
at
org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-5238) Add a log4j category for all edits to META/ROOT
[
https://issues.apache.org/jira/browse/HBASE-5238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14330063#comment-14330063
]
Hadoop QA commented on HBASE-5238:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/1274/HBASE-5238.patch
against master branch at commit a8555f65f0551569f07f6a7d207ddc0e56fad03a.
ATTACHMENT ID: 1274
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.1 2.5.2 2.6.0)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 checkstyle{color}. The applied patch does not increase the
total number of checkstyle errors
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
{color:red}-1 core zombie tests{color}. There are 1 zombie test(s):
at
org.apache.hadoop.hbase.namespace.TestNamespaceAuditor.testRegionMerge(TestNamespaceAuditor.java:308)
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/12936//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12936//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12936//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12936//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12936//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12936//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12936//artifact/patchprocess/newPatchFindbugsWarningshbase-rest.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12936//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12936//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12936//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12936//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12936//artifact/patchprocess/newPatchFindbugsWarningshbase-annotations.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/12936//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/12936//console
This message is automatically generated.
Add a log4j category for all edits to META/ROOT
---
Key: HBASE-5238
URL: https://issues.apache.org/jira/browse/HBASE-5238
Project: HBase
Issue Type: New Feature
Components: regionserver
Affects Versions: 2.0.0
Reporter: Todd Lipcon
Assignee: Andrey Stepachev
Priority: Minor
Labels: beginner
Attachments: HBASE-5238.patch, HBASE-5238.patch, meta2.log
Occasionally we run into bugs that have corrected META and written some bad
data to META/ROOT but it's difficult to understand the order in which things
happened. One option is to dump the HLog contents from the servers that
hosted META at that time, but then it's interspersed with all other data. It
would be nice to add a Log4j Logger to which we log all edits being applied
to META
[jira] [Commented] (HBASE-13032) Migration of states should be performed once META is assigned and onlined.
[ https://issues.apache.org/jira/browse/HBASE-13032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14330071#comment-14330071 ] Hudson commented on HBASE-13032: FAILURE: Integrated in HBase-1.1 #203 (See [https://builds.apache.org/job/HBase-1.1/203/]) HBASE-13032 Migration of states should be performed once META is assigned and onlined (Andrey Stepachev) (stack: rev 61901c86b1c4b66b0d22553b0a6e375a0ec84b68) * hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java Migration of states should be performed once META is assigned and onlined. -- Key: HBASE-13032 URL: https://issues.apache.org/jira/browse/HBASE-13032 Project: HBase Issue Type: Bug Affects Versions: 2.0.0 Reporter: Andrey Stepachev Assignee: Andrey Stepachev Fix For: 2.0.0, 1.1.0 Attachments: HBASE-13032.patch, HBASE-13032.patch, HBASE-13032.patch, HBASE-13032.patch Thread 82 (localhost:58722.activeMasterManager): State: TIMED_WAITING Blocked count: 43 Waited count: 783 Stack: java.lang.Object.wait(Native Method) org.apache.hadoop.hbase.client.AsyncProcess.waitForMaximumCurrentTasks(AsyncProcess.java:1656) org.apache.hadoop.hbase.client.AsyncProcess.waitForAllPreviousOpsAndReset(AsyncProcess.java:1685) org.apache.hadoop.hbase.client.BufferedMutatorImpl.backgroundFlushCommits(BufferedMutatorImpl.java:208) org.apache.hadoop.hbase.client.BufferedMutatorImpl.flush(BufferedMutatorImpl.java:183) org.apache.hadoop.hbase.client.HTable.flushCommits(HTable.java:1395) org.apache.hadoop.hbase.client.HTable.put(HTable.java:1007) org.apache.hadoop.hbase.MetaTableAccessor.put(MetaTableAccessor.java:1116) org.apache.hadoop.hbase.MetaTableAccessor.putToMetaTable(MetaTableAccessor.java:1106) org.apache.hadoop.hbase.MetaTableAccessor.updateTableState(MetaTableAccessor.java:1429) org.apache.hadoop.hbase.MetaTableAccessor.updateTableState(MetaTableAccessor.java:941) org.apache.hadoop.hbase.master.TableStateManager.udpateMetaState(TableStateManager.java:181) org.apache.hadoop.hbase.master.TableStateManager.setTableState(TableStateManager.java:68) org.apache.hadoop.hbase.master.HMaster.initializeZKBasedSystemTrackers(HMaster.java:573) org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:640) org.apache.hadoop.hbase.master.HMaster.access$500(HMaster.java:170) org.apache.hadoop.hbase.master.HMaster$1.run(HMaster.java:1462) java.lang.Thread.run(Thread.java:745) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13070) Fix TestCacheOnWrite
[ https://issues.apache.org/jira/browse/HBASE-13070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14329126#comment-14329126 ] stack commented on HBASE-13070: --- Excellent debugging. +1 Fix TestCacheOnWrite Key: HBASE-13070 URL: https://issues.apache.org/jira/browse/HBASE-13070 Project: HBase Issue Type: Bug Components: test Affects Versions: 2.0.0 Reporter: zhangduo Assignee: zhangduo Attachments: HBASE-13070.patch, HBASE-13070_1.patch, HBASE-13070_2.patch TestCacheOnWrite uses TestHFileWriterV2.randomOrderedKey to generate a random byte array, then use first 32 bytes as row and remaining part as family and qualifier. But TestHFileWriterV2.randomOrderedKey may return a byte array only contains 32 bytes, so there will be zero length family and qualifier. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13070) Fix TestCacheOnWrite
[ https://issues.apache.org/jira/browse/HBASE-13070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14329144#comment-14329144 ] Ted Yu commented on HBASE-13070: Integrated to branch-1 and master branch. Thanks for the patch, Duo. Thanks for the review, Stack. Fix TestCacheOnWrite Key: HBASE-13070 URL: https://issues.apache.org/jira/browse/HBASE-13070 Project: HBase Issue Type: Bug Components: test Affects Versions: 2.0.0 Reporter: zhangduo Assignee: zhangduo Fix For: 2.0.0, 1.1.0 Attachments: HBASE-13070.patch, HBASE-13070_1.patch, HBASE-13070_2.patch TestCacheOnWrite uses TestHFileWriterV2.randomOrderedKey to generate a random byte array, then use first 32 bytes as row and remaining part as family and qualifier. But TestHFileWriterV2.randomOrderedKey may return a byte array only contains 32 bytes, so there will be zero length family and qualifier. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-5238) Add a log4j category for all edits to META/ROOT
[ https://issues.apache.org/jira/browse/HBASE-5238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrey Stepachev updated HBASE-5238: Attachment: meta2.log attached example log. release note: added log4j category org.apache.hadoop.hbase.META that will receive all meta updates in textual (json) representation. Add a log4j category for all edits to META/ROOT --- Key: HBASE-5238 URL: https://issues.apache.org/jira/browse/HBASE-5238 Project: HBase Issue Type: New Feature Components: regionserver Affects Versions: 2.0.0 Reporter: Todd Lipcon Assignee: Andrey Stepachev Priority: Minor Labels: beginner Attachments: HBASE-5238.patch, meta2.log Occasionally we run into bugs that have corrected META and written some bad data to META/ROOT but it's difficult to understand the order in which things happened. One option is to dump the HLog contents from the servers that hosted META at that time, but then it's interspersed with all other data. It would be nice to add a Log4j Logger to which we log all edits being applied to META and ROOT in textual form at DEBUG level. Then it would be easier to do a cluster-wide log grep to see what happened when. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-13070) Fix TestCacheOnWrite
[ https://issues.apache.org/jira/browse/HBASE-13070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-13070: --- Fix Version/s: 1.1.0 2.0.0 Hadoop Flags: Reviewed Fix TestCacheOnWrite Key: HBASE-13070 URL: https://issues.apache.org/jira/browse/HBASE-13070 Project: HBase Issue Type: Bug Components: test Affects Versions: 2.0.0 Reporter: zhangduo Assignee: zhangduo Fix For: 2.0.0, 1.1.0 Attachments: HBASE-13070.patch, HBASE-13070_1.patch, HBASE-13070_2.patch TestCacheOnWrite uses TestHFileWriterV2.randomOrderedKey to generate a random byte array, then use first 32 bytes as row and remaining part as family and qualifier. But TestHFileWriterV2.randomOrderedKey may return a byte array only contains 32 bytes, so there will be zero length family and qualifier. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13054) Provide more tracing information for locking/latching events.
[ https://issues.apache.org/jira/browse/HBASE-13054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14329191#comment-14329191 ] Elliott Clark commented on HBASE-13054: --- +1 from me. Provide more tracing information for locking/latching events. - Key: HBASE-13054 URL: https://issues.apache.org/jira/browse/HBASE-13054 Project: HBase Issue Type: Improvement Reporter: Rajeshbabu Chintaguntla Assignee: Rajeshbabu Chintaguntla Fix For: 2.0.0, 1.0.1, 1.1.0, 0.98.11 Attachments: HBASE-13054.patch, HBASE-13054_v2.patch, HBASE-13054_v3.patch Currently not much tracing information available for locking and latching events like row level locking during do mini batch mutations, region level locking during flush, close and so on. It will be better to add the trace information for such events so that it will be useful for finding time spent on locking and waiting time on locks while analyzing performance issues in queries using trace information. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-13082) Coarsen StoreScanner locks to RegionScanner
Lars Hofhansl created HBASE-13082: - Summary: Coarsen StoreScanner locks to RegionScanner Key: HBASE-13082 URL: https://issues.apache.org/jira/browse/HBASE-13082 Project: HBase Issue Type: Bug Reporter: Lars Hofhansl Continuing where HBASE-10015 left of. We can avoid locking (and memory fencing) inside StoreScanner by deferring to the lock already held by the RegionScanner. In tests this shows quite a scan improvement and reduced CPU (the fences make the cores wait for memory fetches). There are some drawbacks too: * All calls to RegionScanner need to be remain synchronized * Implementors of coprocessors need to be diligent in following the locking contract. For example Phoenix does not lock RegionScanner.nextRaw() and required in the documentation (not picking on Phoenix, this one is my fault as I told them it's OK) * possible starving of flushes and compaction with heavy read load. RegionScanner operations would keep getting the locks and the flushes/compactions would not be able finalize the set of files. I'll have a patch soon. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13070) Fix TestCacheOnWrite
[ https://issues.apache.org/jira/browse/HBASE-13070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14330069#comment-14330069 ] Hudson commented on HBASE-13070: FAILURE: Integrated in HBase-0.98 #865 (See [https://builds.apache.org/job/HBase-0.98/865/]) HBASE-13070 Fix TestCacheOnWrite (Duo Zhang) (tedyu: rev ed6613a06e666e7d2a4f82daf602a847c81c4f59) * hbase-server/src/test/java/org/apache/hadoop/hbase/io/hfile/TestCacheOnWrite.java * hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/LruBlockCache.java Fix TestCacheOnWrite Key: HBASE-13070 URL: https://issues.apache.org/jira/browse/HBASE-13070 Project: HBase Issue Type: Bug Components: test Affects Versions: 2.0.0 Reporter: zhangduo Assignee: zhangduo Fix For: 2.0.0, 1.0.1, 1.1.0, 0.98.11 Attachments: HBASE-13070-0.98.patch, HBASE-13070.patch, HBASE-13070_1.patch, HBASE-13070_2.patch TestCacheOnWrite uses TestHFileWriterV2.randomOrderedKey to generate a random byte array, then use first 32 bytes as row and remaining part as family and qualifier. But TestHFileWriterV2.randomOrderedKey may return a byte array only contains 32 bytes, so there will be zero length family and qualifier. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13032) Migration of states should be performed once META is assigned and onlined.
[ https://issues.apache.org/jira/browse/HBASE-13032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14330065#comment-14330065 ] Hudson commented on HBASE-13032: FAILURE: Integrated in HBase-TRUNK #6156 (See [https://builds.apache.org/job/HBase-TRUNK/6156/]) HBASE-13032 Migration of states should be performed once META is assigned and onlined (Andrey Stepachev) (stack: rev 7af56998c5d42afc84ddf8c81126331715ef6ccf) * hbase-server/src/test/java/org/apache/hadoop/hbase/master/TestTableStateManager.java * hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java Migration of states should be performed once META is assigned and onlined. -- Key: HBASE-13032 URL: https://issues.apache.org/jira/browse/HBASE-13032 Project: HBase Issue Type: Bug Affects Versions: 2.0.0 Reporter: Andrey Stepachev Assignee: Andrey Stepachev Fix For: 2.0.0, 1.1.0 Attachments: HBASE-13032.patch, HBASE-13032.patch, HBASE-13032.patch, HBASE-13032.patch Thread 82 (localhost:58722.activeMasterManager): State: TIMED_WAITING Blocked count: 43 Waited count: 783 Stack: java.lang.Object.wait(Native Method) org.apache.hadoop.hbase.client.AsyncProcess.waitForMaximumCurrentTasks(AsyncProcess.java:1656) org.apache.hadoop.hbase.client.AsyncProcess.waitForAllPreviousOpsAndReset(AsyncProcess.java:1685) org.apache.hadoop.hbase.client.BufferedMutatorImpl.backgroundFlushCommits(BufferedMutatorImpl.java:208) org.apache.hadoop.hbase.client.BufferedMutatorImpl.flush(BufferedMutatorImpl.java:183) org.apache.hadoop.hbase.client.HTable.flushCommits(HTable.java:1395) org.apache.hadoop.hbase.client.HTable.put(HTable.java:1007) org.apache.hadoop.hbase.MetaTableAccessor.put(MetaTableAccessor.java:1116) org.apache.hadoop.hbase.MetaTableAccessor.putToMetaTable(MetaTableAccessor.java:1106) org.apache.hadoop.hbase.MetaTableAccessor.updateTableState(MetaTableAccessor.java:1429) org.apache.hadoop.hbase.MetaTableAccessor.updateTableState(MetaTableAccessor.java:941) org.apache.hadoop.hbase.master.TableStateManager.udpateMetaState(TableStateManager.java:181) org.apache.hadoop.hbase.master.TableStateManager.setTableState(TableStateManager.java:68) org.apache.hadoop.hbase.master.HMaster.initializeZKBasedSystemTrackers(HMaster.java:573) org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:640) org.apache.hadoop.hbase.master.HMaster.access$500(HMaster.java:170) org.apache.hadoop.hbase.master.HMaster$1.run(HMaster.java:1462) java.lang.Thread.run(Thread.java:745) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-11544) [Ergonomics] hbase.client.scanner.caching is dogged and will try to return batch even if it means OOME
[
https://issues.apache.org/jira/browse/HBASE-11544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14329769#comment-14329769
]
Hadoop QA commented on HBASE-11544:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12699942/HBASE-11544-v3.patch
against master branch at commit 9a311303a88d355f8c982d2f3cc77bd6427dfb80.
ATTACHMENT ID: 12699942
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 24 new
or modified tests.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.1 2.5.2 2.6.0)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 checkstyle{color}. The applied patch does not increase the
total number of checkstyle errors
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 lineLengths{color}. The patch introduces the following lines
longer than 100:
+ if (!((mutable_bitField0_ 0x0002) == 0x0002)
input.getBytesUntilLimit() 0) {
+ private java.util.Listjava.lang.Boolean partialFlagPerResult_ =
java.util.Collections.emptyList();
+ \001(\014\022\020\n\010stop_row\030\004
\001(\014\022\027\n\006filter\030\005 \001(\0132\007 +
+ new java.lang.String[] { Cell, AssociatedCellCount,
Exists, Stale, Partial, });
+ new java.lang.String[] { Region, Scan, ScannerId,
NumberOfRows, CloseScanner, NextCallSeq, ClientHandlesPartials, });
+ new java.lang.String[] { CellsPerResult,
PartialFlagPerResult, ScannerId, MoreResults, Ttl, Results, Stale,
});
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
{color:red}-1 core zombie tests{color}. There are 1 zombie test(s):
at
org.apache.hadoop.hbase.TestAcidGuarantees.testScanAtomicity(TestAcidGuarantees.java:354)
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/12929//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12929//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12929//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12929//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12929//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12929//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12929//artifact/patchprocess/newPatchFindbugsWarningshbase-rest.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12929//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12929//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12929//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12929//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/12929//artifact/patchprocess/newPatchFindbugsWarningshbase-annotations.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/12929//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/12929//console
This message is automatically generated.
[Ergonomics] hbase.client.scanner.caching is dogged and will try to return
batch even if it means OOME
--
Key: HBASE-11544
URL: https://issues.apache.org/jira/browse/HBASE-11544
Project: HBase
Issue Type: Bug
Reporter: stack
Assignee: Jonathan Lawlor
[jira] [Updated] (HBASE-13045) Ensure MapReduce javadoc example works
[ https://issues.apache.org/jira/browse/HBASE-13045?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated HBASE-13045: Resolution: Fixed Status: Resolved (was: Patch Available) pushed to 0.98. thanks for the review. Ensure MapReduce javadoc example works -- Key: HBASE-13045 URL: https://issues.apache.org/jira/browse/HBASE-13045 Project: HBase Issue Type: Improvement Components: mapreduce Reporter: Sean Busbey Assignee: Sean Busbey Priority: Minor Fix For: 0.98.11 Attachments: HBASE-13045-0.98.1.patch.txt Backport the docs / test parts of HBASE-13027 that are applicable to 0.98. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13082) Coarsen StoreScanner locks to RegionScanner
[ https://issues.apache.org/jira/browse/HBASE-13082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14330048#comment-14330048 ] stack commented on HBASE-13082: --- Looking forward to the ride [~larsh] Coarsen StoreScanner locks to RegionScanner --- Key: HBASE-13082 URL: https://issues.apache.org/jira/browse/HBASE-13082 Project: HBase Issue Type: Bug Reporter: Lars Hofhansl Continuing where HBASE-10015 left of. We can avoid locking (and memory fencing) inside StoreScanner by deferring to the lock already held by the RegionScanner. In tests this shows quite a scan improvement and reduced CPU (the fences make the cores wait for memory fetches). There are some drawbacks too: * All calls to RegionScanner need to be remain synchronized * Implementors of coprocessors need to be diligent in following the locking contract. For example Phoenix does not lock RegionScanner.nextRaw() and required in the documentation (not picking on Phoenix, this one is my fault as I told them it's OK) * possible starving of flushes and compaction with heavy read load. RegionScanner operations would keep getting the locks and the flushes/compactions would not be able finalize the set of files. I'll have a patch soon. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-12990) MetaScanner should be replaced by MetaTableAccessor
[ https://issues.apache.org/jira/browse/HBASE-12990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrey Stepachev updated HBASE-12990: - Status: Open (was: Patch Available) seems not clean removal, will redo that step by step. MetaScanner should be replaced by MetaTableAccessor --- Key: HBASE-12990 URL: https://issues.apache.org/jira/browse/HBASE-12990 Project: HBase Issue Type: Improvement Components: Client Affects Versions: 2.0.0 Reporter: Andrey Stepachev Assignee: Andrey Stepachev Attachments: HBASE-12990.patch, HBASE-12990.v2.patch, HBASE-12990.v3.patch, HBASE-12990.v4.patch MetaScanner and MetaTableAccessor do similar things, but seems they tend to diverge. Let's have only one thing to enquiry META. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
