[jira] [Commented] (HBASE-11405) Multiple invocations of hbck in parallel disables balancer permanently
[
https://issues.apache.org/jira/browse/HBASE-11405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14041750#comment-14041750
]
bharath v commented on HBASE-11405:
---
Related question: Should be disable parallel hbcks permanently? A quick look at
the code suggests this might cause inconsistencies as well.
Multiple invocations of hbck in parallel disables balancer permanently
---
Key: HBASE-11405
URL: https://issues.apache.org/jira/browse/HBASE-11405
Project: HBase
Issue Type: Bug
Components: Balancer, hbck
Affects Versions: 0.99.0
Reporter: bharath v
This is because of the following piece of code in hbck
{code:borderStyle=solid}
boolean oldBalancer = admin.setBalancerRunning(false, true);
try {
onlineConsistencyRepair();
}
finally {
admin.setBalancerRunning(oldBalancer, false);
}
{code}
Newer invocations set oldBalancer to false as it was disabled by previous
invocations and this disables balancer permanently unless its manually turned
on by the user. Easy to reproduce, just run hbck 100 times in a loop in 2
different sessions and you can see that balancer is set to false in the
HMaster logs.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11274) More general single-row Condition Mutation
[
https://issues.apache.org/jira/browse/HBASE-11274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14041769#comment-14041769
]
Hadoop QA commented on HBASE-11274:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12652126/HBASE-11274-trunk-v2.diff
against trunk revision .
ATTACHMENT ID: 12652126
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 6 new
or modified tests.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 3
warning messages.
{color:red}-1 findbugs{color}. The patch appears to introduce 5 new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 lineLengths{color}. The patch introduces the following lines
longer than 100:
+ return rpcCallerFactory.Boolean
newCaller().callWithRetries(callable, this.operationTimeout);
+ public SingleColumnValueCondition(final byte[] row, final byte[] family,
final byte[] qualifier, final byte[] value) {
+SingleColumnValueCondition condition = new SingleColumnValueCondition(row,
family, qualifier, value);
+SingleColumnValueCondition condition = new SingleColumnValueCondition(row,
family, qualifier, CompareOp.GREATER, value);
+SingleColumnValueCondition condition = new SingleColumnValueCondition(row,
family, qualifier, CompareOp.LESS, value);
+ dition\022\013\n\003row\030\001 \001(\014\022\016\n\006family\030\002
\001(\014\022\021\n\tq +
+ new java.lang.String[] { Row, Family, Qualifier,
CompareType, Comparator, Name, SerializedCondition, });
+ result = ((RegionObserver)
env.getInstance()).preCheckAndMutateAfterRowLock(ctx, condition,
+AuthResult authResult = permissionGranted(OpType.CHECK_AND_MUTATE, user,
env, condition.getFamilyMap(),
+conditions.addCondition(new SingleColumnValueCondition(ROW, FAMILY,
QUALIFIER, Bytes.toBytes(value1)));
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.coprocessor.TestRegionObserverInterface
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/9834//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9834//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9834//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9834//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9834//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9834//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9834//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9834//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9834//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9834//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/9834//console
This message is automatically generated.
More general single-row Condition Mutation
--
Key: HBASE-11274
URL: https://issues.apache.org/jira/browse/HBASE-11274
Project: HBase
Issue Type: Improvement
Affects Versions: 1.0.0
Reporter: Liu Shaohui
Assignee: Liu Shaohui
Attachments: HBASE-11274-trunk-v1.diff, HBASE-11274-trunk-v2.diff
Currently, the checkAndDelete and checkAndPut interface only support atomic
mutation with single condition. But in actual apps, we need more general
condition-mutation that support multi conditions and logical expression with
those conditions.
For example, to support the following sql
[jira] [Commented] (HBASE-11405) Multiple invocations of hbck in parallel disables balancer permanently
[
https://issues.apache.org/jira/browse/HBASE-11405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14041806#comment-14041806
]
Anoop Sam John commented on HBASE-11405:
bq.Related question: Should be disable parallel hbcks permanently?
+1 for that.
Multiple invocations of hbck in parallel disables balancer permanently
---
Key: HBASE-11405
URL: https://issues.apache.org/jira/browse/HBASE-11405
Project: HBase
Issue Type: Bug
Components: Balancer, hbck
Affects Versions: 0.99.0
Reporter: bharath v
This is because of the following piece of code in hbck
{code:borderStyle=solid}
boolean oldBalancer = admin.setBalancerRunning(false, true);
try {
onlineConsistencyRepair();
}
finally {
admin.setBalancerRunning(oldBalancer, false);
}
{code}
Newer invocations set oldBalancer to false as it was disabled by previous
invocations and this disables balancer permanently unless its manually turned
on by the user. Easy to reproduce, just run hbck 100 times in a loop in 2
different sessions and you can see that balancer is set to false in the
HMaster logs.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11400) Edit, consolidate, and update Compression and data encoding docs
[
https://issues.apache.org/jira/browse/HBASE-11400?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14041813#comment-14041813
]
Hadoop QA commented on HBASE-11400:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12652127/HBASE-11400.patch
against trunk revision .
ATTACHMENT ID: 12652127
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+0 tests included{color}. The patch appears to be a
documentation patch that doesn't require tests.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:red}-1 findbugs{color}. The patch appears to introduce 2 new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 lineLengths{color}. The patch introduces the following lines
longer than 100:
+ Compression/link wiki page for information on configuring
LZO support for HBase. /para
+ $ hbase org.apache.hadoop.hbase.util.CompressionTest
hdfs://replaceablehost/path/to/hbase/replaceable snappy
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/9835//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9835//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9835//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9835//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9835//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9835//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9835//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9835//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9835//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9835//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/9835//console
This message is automatically generated.
Edit, consolidate, and update Compression and data encoding docs
Key: HBASE-11400
URL: https://issues.apache.org/jira/browse/HBASE-11400
Project: HBase
Issue Type: Improvement
Components: documentation
Reporter: Misty Stanley-Jones
Assignee: Misty Stanley-Jones
Priority: Minor
Attachments: HBASE-11400.patch
Current docs are here: http://hbase.apache.org/book.html#compression.test
It could use some editing and expansion.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11380) HRegion lock object is not being released properly, leading to snapshot failure
[
https://issues.apache.org/jira/browse/HBASE-11380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14041828#comment-14041828
]
Hudson commented on HBASE-11380:
SUCCESS: Integrated in HBase-TRUNK #5228 (See
[https://builds.apache.org/job/HBase-TRUNK/5228/])
HBASE-11380 HRegion lock object is not being released properly, leading to
snapshot failure (tedyu: rev af9e519e00764af9d372a22953b03e660c9aa6f0)
* hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/client/TableSnapshotScanner.java
HRegion lock object is not being released properly, leading to snapshot
failure
---
Key: HBASE-11380
URL: https://issues.apache.org/jira/browse/HBASE-11380
Project: HBase
Issue Type: Bug
Components: regionserver
Affects Versions: 0.98.3
Reporter: Craig Condit
Assignee: Ted Yu
Priority: Blocker
Fix For: 0.99.0, 0.98.4
Attachments: 11380-v1.txt, 11380-v2.txt, 11380-v3.txt,
HBASE-11380-v2-0.98.3.txt
Background:
We are attempting to create ~ 750 table snapshots on a nightly basis for use
in MR jobs. The jobs are run in batches, with a maximum of around 20 jobs
running simultaneously.
We have started to see the following in our region server logs (after 1 day
uptime):
{noformat}
java.lang.Error: Maximum lock count exceeded
at
java.util.concurrent.locks.ReentrantReadWriteLock$Sync.fullTryAcquireShared(ReentrantReadWriteLock.java:531)
at
java.util.concurrent.locks.ReentrantReadWriteLock$Sync.tryAcquireShared(ReentrantReadWriteLock.java:491)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1326)
at
java.util.concurrent.locks.ReentrantReadWriteLock$ReadLock.tryLock(ReentrantReadWriteLock.java:873)
at org.apache.hadoop.hbase.regionserver.HRegion.lock(HRegion.java:5904)
at org.apache.hadoop.hbase.regionserver.HRegion.lock(HRegion.java:5891)
at
org.apache.hadoop.hbase.regionserver.HRegion.startRegionOperation(HRegion.java:5798)
at
org.apache.hadoop.hbase.regionserver.HRegion.startRegionOperation(HRegion.java:5761)
at
org.apache.hadoop.hbase.regionserver.HRegion.processRowsWithLocks(HRegion.java:4891)
at
org.apache.hadoop.hbase.regionserver.HRegion.mutateRowsWithLocks(HRegion.java:4856)
at
org.apache.hadoop.hbase.regionserver.HRegion.mutateRowsWithLocks(HRegion.java:4838)
at
org.apache.hadoop.hbase.regionserver.HRegion.mutateRow(HRegion.java:4829)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.mutateRows(HRegionServer.java:4390)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.multi(HRegionServer.java:3362)
at
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:29503)
at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2012)
at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:98)
at
org.apache.hadoop.hbase.ipc.SimpleRpcScheduler.consumerLoop(SimpleRpcScheduler.java:168)
at
org.apache.hadoop.hbase.ipc.SimpleRpcScheduler.access$000(SimpleRpcScheduler.java:39)
at
org.apache.hadoop.hbase.ipc.SimpleRpcScheduler$1.run(SimpleRpcScheduler.java:111)
at java.lang.Thread.run(Thread.java:744)
{noformat}
Not sure of the cause, but the result is that snapshots cannot be created. We
see this in our client logs:
{noformat}
Exception in thread main
org.apache.hadoop.hbase.snapshot.HBaseSnapshotException:
org.apache.hadoop.hbase.snapshot.HBaseSnapshotException: Snapshot {
ss=test-snapshot-20140619143753294 table=test type=FLUSH } had an error.
Procedure test-snapshot-20140619143753294 {
waiting=[p3plpadata038.internal,60020,1403140682587,
p3plpadata056.internal,60020,1403140865123,
p3plpadata072.internal,60020,1403141022569]
done=[p3plpadata023.internal,60020,1403140552227,
p3plpadata009.internal,60020,1403140487826] }
at
org.apache.hadoop.hbase.master.snapshot.SnapshotManager.isSnapshotDone(SnapshotManager.java:342)
at
org.apache.hadoop.hbase.master.HMaster.isSnapshotDone(HMaster.java:2907)
at
org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java:40494)
at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2012)
at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:98)
at
org.apache.hadoop.hbase.ipc.FifoRpcScheduler$1.run(FifoRpcScheduler.java:73)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at
[jira] [Commented] (HBASE-11380) HRegion lock object is not being released properly, leading to snapshot failure
[
https://issues.apache.org/jira/browse/HBASE-11380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14041858#comment-14041858
]
Hudson commented on HBASE-11380:
FAILURE: Integrated in HBase-0.98 #351 (See
[https://builds.apache.org/job/HBase-0.98/351/])
HBASE-11380 HRegion lock object is not being released properly, leading to
snapshot failure (tedyu: rev d073f7beb4fb56c613a364dd14bbfc56ae85ce3d)
* hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
*
hbase-server/src/main/java/org/apache/hadoop/hbase/client/TableSnapshotScanner.java
HRegion lock object is not being released properly, leading to snapshot
failure
---
Key: HBASE-11380
URL: https://issues.apache.org/jira/browse/HBASE-11380
Project: HBase
Issue Type: Bug
Components: regionserver
Affects Versions: 0.98.3
Reporter: Craig Condit
Assignee: Ted Yu
Priority: Blocker
Fix For: 0.99.0, 0.98.4
Attachments: 11380-v1.txt, 11380-v2.txt, 11380-v3.txt,
HBASE-11380-v2-0.98.3.txt
Background:
We are attempting to create ~ 750 table snapshots on a nightly basis for use
in MR jobs. The jobs are run in batches, with a maximum of around 20 jobs
running simultaneously.
We have started to see the following in our region server logs (after 1 day
uptime):
{noformat}
java.lang.Error: Maximum lock count exceeded
at
java.util.concurrent.locks.ReentrantReadWriteLock$Sync.fullTryAcquireShared(ReentrantReadWriteLock.java:531)
at
java.util.concurrent.locks.ReentrantReadWriteLock$Sync.tryAcquireShared(ReentrantReadWriteLock.java:491)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1326)
at
java.util.concurrent.locks.ReentrantReadWriteLock$ReadLock.tryLock(ReentrantReadWriteLock.java:873)
at org.apache.hadoop.hbase.regionserver.HRegion.lock(HRegion.java:5904)
at org.apache.hadoop.hbase.regionserver.HRegion.lock(HRegion.java:5891)
at
org.apache.hadoop.hbase.regionserver.HRegion.startRegionOperation(HRegion.java:5798)
at
org.apache.hadoop.hbase.regionserver.HRegion.startRegionOperation(HRegion.java:5761)
at
org.apache.hadoop.hbase.regionserver.HRegion.processRowsWithLocks(HRegion.java:4891)
at
org.apache.hadoop.hbase.regionserver.HRegion.mutateRowsWithLocks(HRegion.java:4856)
at
org.apache.hadoop.hbase.regionserver.HRegion.mutateRowsWithLocks(HRegion.java:4838)
at
org.apache.hadoop.hbase.regionserver.HRegion.mutateRow(HRegion.java:4829)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.mutateRows(HRegionServer.java:4390)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.multi(HRegionServer.java:3362)
at
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:29503)
at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2012)
at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:98)
at
org.apache.hadoop.hbase.ipc.SimpleRpcScheduler.consumerLoop(SimpleRpcScheduler.java:168)
at
org.apache.hadoop.hbase.ipc.SimpleRpcScheduler.access$000(SimpleRpcScheduler.java:39)
at
org.apache.hadoop.hbase.ipc.SimpleRpcScheduler$1.run(SimpleRpcScheduler.java:111)
at java.lang.Thread.run(Thread.java:744)
{noformat}
Not sure of the cause, but the result is that snapshots cannot be created. We
see this in our client logs:
{noformat}
Exception in thread main
org.apache.hadoop.hbase.snapshot.HBaseSnapshotException:
org.apache.hadoop.hbase.snapshot.HBaseSnapshotException: Snapshot {
ss=test-snapshot-20140619143753294 table=test type=FLUSH } had an error.
Procedure test-snapshot-20140619143753294 {
waiting=[p3plpadata038.internal,60020,1403140682587,
p3plpadata056.internal,60020,1403140865123,
p3plpadata072.internal,60020,1403141022569]
done=[p3plpadata023.internal,60020,1403140552227,
p3plpadata009.internal,60020,1403140487826] }
at
org.apache.hadoop.hbase.master.snapshot.SnapshotManager.isSnapshotDone(SnapshotManager.java:342)
at
org.apache.hadoop.hbase.master.HMaster.isSnapshotDone(HMaster.java:2907)
at
org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java:40494)
at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2012)
at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:98)
at
org.apache.hadoop.hbase.ipc.FifoRpcScheduler$1.run(FifoRpcScheduler.java:73)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at
[jira] [Commented] (HBASE-11380) HRegion lock object is not being released properly, leading to snapshot failure
[
https://issues.apache.org/jira/browse/HBASE-11380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14041888#comment-14041888
]
Hudson commented on HBASE-11380:
FAILURE: Integrated in HBase-0.98-on-Hadoop-1.1 #333 (See
[https://builds.apache.org/job/HBase-0.98-on-Hadoop-1.1/333/])
HBASE-11380 HRegion lock object is not being released properly, leading to
snapshot failure (tedyu: rev d073f7beb4fb56c613a364dd14bbfc56ae85ce3d)
*
hbase-server/src/main/java/org/apache/hadoop/hbase/client/TableSnapshotScanner.java
* hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
HRegion lock object is not being released properly, leading to snapshot
failure
---
Key: HBASE-11380
URL: https://issues.apache.org/jira/browse/HBASE-11380
Project: HBase
Issue Type: Bug
Components: regionserver
Affects Versions: 0.98.3
Reporter: Craig Condit
Assignee: Ted Yu
Priority: Blocker
Fix For: 0.99.0, 0.98.4
Attachments: 11380-v1.txt, 11380-v2.txt, 11380-v3.txt,
HBASE-11380-v2-0.98.3.txt
Background:
We are attempting to create ~ 750 table snapshots on a nightly basis for use
in MR jobs. The jobs are run in batches, with a maximum of around 20 jobs
running simultaneously.
We have started to see the following in our region server logs (after 1 day
uptime):
{noformat}
java.lang.Error: Maximum lock count exceeded
at
java.util.concurrent.locks.ReentrantReadWriteLock$Sync.fullTryAcquireShared(ReentrantReadWriteLock.java:531)
at
java.util.concurrent.locks.ReentrantReadWriteLock$Sync.tryAcquireShared(ReentrantReadWriteLock.java:491)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1326)
at
java.util.concurrent.locks.ReentrantReadWriteLock$ReadLock.tryLock(ReentrantReadWriteLock.java:873)
at org.apache.hadoop.hbase.regionserver.HRegion.lock(HRegion.java:5904)
at org.apache.hadoop.hbase.regionserver.HRegion.lock(HRegion.java:5891)
at
org.apache.hadoop.hbase.regionserver.HRegion.startRegionOperation(HRegion.java:5798)
at
org.apache.hadoop.hbase.regionserver.HRegion.startRegionOperation(HRegion.java:5761)
at
org.apache.hadoop.hbase.regionserver.HRegion.processRowsWithLocks(HRegion.java:4891)
at
org.apache.hadoop.hbase.regionserver.HRegion.mutateRowsWithLocks(HRegion.java:4856)
at
org.apache.hadoop.hbase.regionserver.HRegion.mutateRowsWithLocks(HRegion.java:4838)
at
org.apache.hadoop.hbase.regionserver.HRegion.mutateRow(HRegion.java:4829)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.mutateRows(HRegionServer.java:4390)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.multi(HRegionServer.java:3362)
at
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:29503)
at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2012)
at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:98)
at
org.apache.hadoop.hbase.ipc.SimpleRpcScheduler.consumerLoop(SimpleRpcScheduler.java:168)
at
org.apache.hadoop.hbase.ipc.SimpleRpcScheduler.access$000(SimpleRpcScheduler.java:39)
at
org.apache.hadoop.hbase.ipc.SimpleRpcScheduler$1.run(SimpleRpcScheduler.java:111)
at java.lang.Thread.run(Thread.java:744)
{noformat}
Not sure of the cause, but the result is that snapshots cannot be created. We
see this in our client logs:
{noformat}
Exception in thread main
org.apache.hadoop.hbase.snapshot.HBaseSnapshotException:
org.apache.hadoop.hbase.snapshot.HBaseSnapshotException: Snapshot {
ss=test-snapshot-20140619143753294 table=test type=FLUSH } had an error.
Procedure test-snapshot-20140619143753294 {
waiting=[p3plpadata038.internal,60020,1403140682587,
p3plpadata056.internal,60020,1403140865123,
p3plpadata072.internal,60020,1403141022569]
done=[p3plpadata023.internal,60020,1403140552227,
p3plpadata009.internal,60020,1403140487826] }
at
org.apache.hadoop.hbase.master.snapshot.SnapshotManager.isSnapshotDone(SnapshotManager.java:342)
at
org.apache.hadoop.hbase.master.HMaster.isSnapshotDone(HMaster.java:2907)
at
org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java:40494)
at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2012)
at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:98)
at
org.apache.hadoop.hbase.ipc.FifoRpcScheduler$1.run(FifoRpcScheduler.java:73)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at
[jira] [Updated] (HBASE-11387) metrics: wrong totalRequestCount
[ https://issues.apache.org/jira/browse/HBASE-11387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Liochon updated HBASE-11387: Resolution: Fixed Fix Version/s: 0.98.4 Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Committed to master 0.98, thanks for the reviews! metrics: wrong totalRequestCount Key: HBASE-11387 URL: https://issues.apache.org/jira/browse/HBASE-11387 Project: HBase Issue Type: Bug Components: metrics, regionserver Affects Versions: 0.99.0, 0.98.3 Reporter: Nicolas Liochon Assignee: Nicolas Liochon Fix For: 0.99.0, 0.98.4 Attachments: 11387.98.v1.patch, 11387.v1.patch, 11387.v2.patch We have an unit test here, but it tests for greater than instead of equals. So we didn't see that the number was the double of the actual value. As well we were not testing the multi case. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11390) PerformanceEvaluation: add an option to use a single connection
[ https://issues.apache.org/jira/browse/HBASE-11390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Liochon updated HBASE-11390: Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Committed to master, thanks for the review, Matteo Nick! PerformanceEvaluation: add an option to use a single connection --- Key: HBASE-11390 URL: https://issues.apache.org/jira/browse/HBASE-11390 Project: HBase Issue Type: Improvement Components: test Affects Versions: 0.99.0 Reporter: Nicolas Liochon Assignee: Nicolas Liochon Fix For: 0.99.0 Attachments: 11390.v1.patch, 11390.v2.patch The PE tool uses one connection per client. It does not match some use cases when we have multiple threads sharing the same connection. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11403) Fix race conditions around Object#notify
[ https://issues.apache.org/jira/browse/HBASE-11403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14041922#comment-14041922 ] Nicolas Liochon commented on HBASE-11403: - Committed to master, patch does not apply to 0.98, specific patch it on its way. Fix race conditions around Object#notify Key: HBASE-11403 URL: https://issues.apache.org/jira/browse/HBASE-11403 Project: HBase Issue Type: Bug Components: Client, regionserver Affects Versions: 0.99.0, 0.98.3 Reporter: Nicolas Liochon Assignee: Nicolas Liochon Fix For: 0.99.0, 0.98.4 Attachments: 11403.v1.patch We do have some race conditions there. We don't see them fail in the unit tests, because our #wait are bounded. But from a performance point of view, they do occur. I've reviewed them and fix all the issue I found excepted in the AM (haven't reviewed this one, may be it's fine). On a perf test, this seems to improve the max latency. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (HBASE-11406) ExportSnapshot#run better catch Throwable for ExportSnapshot#runCopyJob
Qiang Tian created HBASE-11406: -- Summary: ExportSnapshot#run better catch Throwable for ExportSnapshot#runCopyJob Key: HBASE-11406 URL: https://issues.apache.org/jira/browse/HBASE-11406 Project: HBase Issue Type: Improvement Components: snapshots Reporter: Qiang Tian Assignee: Qiang Tian Priority: Trivial ExportSnapshot#run better catch Throwable for ExportSnapshot#runCopyJob since runCopyJob involves hadoop and mapreduce. other errors could be thrown, catching them like below at first place is helpful for error handling and problem identification. 14/06/24 18:39:30 ERROR snapshot.ExportSnapshot: Snapshot export failed java.lang.NoSuchMethodError: org/apache/hadoop/mapred/JobTracker.getAddress(Lorg/apache/hadoop/conf/Configuration;)Ljava/net/InetSocketAddress; at org.apache.hadoop.mapred.JobTrackerClientProtocolProvider.create(JobTrackerClientProtocolProvider.java:44) at org.apache.hadoop.mapreduce.Cluster.initialize(Cluster.java:95) at org.apache.hadoop.mapreduce.Cluster.init(Cluster.java:82) at org.apache.hadoop.mapreduce.Cluster.init(Cluster.java:75) at org.apache.hadoop.hbase.mapreduce.JobUtil.getStagingDir(JobUtil.java:54) at org.apache.hadoop.hbase.snapshot.ExportSnapshot.getInputFolderPath(ExportSnapshot.java:541) at org.apache.hadoop.hbase.snapshot.ExportSnapshot.runCopyJob(ExportSnapshot.java:609) at org.apache.hadoop.hbase.snapshot.ExportSnapshot.run(ExportSnapshot.java:776) at org.apache.hadoop.hbase.backup.BackupCopier.copy(BackupCopier.java:248) at org.apache.hadoop.hbase.backup.BackupHandler.snapshotCopy(BackupHandler.java:559) at org.apache.hadoop.hbase.backup.BackupHandler.call(BackupHandler.java:141) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:314) at java.util.concurrent.FutureTask.run(FutureTask.java:149) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:897) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:919) at java.lang.Thread.run(Thread.java:738) -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11406) ExportSnapshot#run better catch Throwable for ExportSnapshot#runCopyJob
[ https://issues.apache.org/jira/browse/HBASE-11406?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Qiang Tian updated HBASE-11406: --- Attachment: hbase11406-master.patch ExportSnapshot#run better catch Throwable for ExportSnapshot#runCopyJob --- Key: HBASE-11406 URL: https://issues.apache.org/jira/browse/HBASE-11406 Project: HBase Issue Type: Improvement Components: snapshots Reporter: Qiang Tian Assignee: Qiang Tian Priority: Trivial Attachments: hbase11406-master.patch ExportSnapshot#run better catch Throwable for ExportSnapshot#runCopyJob since runCopyJob involves hadoop and mapreduce. other errors could be thrown, catching them like below at first place is helpful for error handling and problem identification. 14/06/24 18:39:30 ERROR snapshot.ExportSnapshot: Snapshot export failed java.lang.NoSuchMethodError: org/apache/hadoop/mapred/JobTracker.getAddress(Lorg/apache/hadoop/conf/Configuration;)Ljava/net/InetSocketAddress; at org.apache.hadoop.mapred.JobTrackerClientProtocolProvider.create(JobTrackerClientProtocolProvider.java:44) at org.apache.hadoop.mapreduce.Cluster.initialize(Cluster.java:95) at org.apache.hadoop.mapreduce.Cluster.init(Cluster.java:82) at org.apache.hadoop.mapreduce.Cluster.init(Cluster.java:75) at org.apache.hadoop.hbase.mapreduce.JobUtil.getStagingDir(JobUtil.java:54) at org.apache.hadoop.hbase.snapshot.ExportSnapshot.getInputFolderPath(ExportSnapshot.java:541) at org.apache.hadoop.hbase.snapshot.ExportSnapshot.runCopyJob(ExportSnapshot.java:609) at org.apache.hadoop.hbase.snapshot.ExportSnapshot.run(ExportSnapshot.java:776) at org.apache.hadoop.hbase.backup.BackupCopier.copy(BackupCopier.java:248) at org.apache.hadoop.hbase.backup.BackupHandler.snapshotCopy(BackupHandler.java:559) at org.apache.hadoop.hbase.backup.BackupHandler.call(BackupHandler.java:141) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:314) at java.util.concurrent.FutureTask.run(FutureTask.java:149) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:897) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:919) at java.lang.Thread.run(Thread.java:738) -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (HBASE-11407) hbase-client should not require Jackson for pure HBase queries be executed
Sergey Beryozkin created HBASE-11407: Summary: hbase-client should not require Jackson for pure HBase queries be executed Key: HBASE-11407 URL: https://issues.apache.org/jira/browse/HBASE-11407 Project: HBase Issue Type: Improvement Components: Client Reporter: Sergey Beryozkin Priority: Minor Including the hbase-client module dependency and excluding Jackson dependencies causes the pure HBase query (run with HTableInterface) fail with Jackson ObjectMapper ClassNotFoundException. This is due to org.apache.hadoop.hbase.client.Operation having ObjectMapper statically initialized. Moving ObjectMapper to a dedicated utility will help. The patch will be attached. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11407) hbase-client should not require Jackson for pure HBase queries be executed
[ https://issues.apache.org/jira/browse/HBASE-11407?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Beryozkin updated HBASE-11407: - Attachment: diff.txt The patch introduces a simple JsonMapper utility and updates Operation to delegate to JsonMapper if needed hbase-client should not require Jackson for pure HBase queries be executed --- Key: HBASE-11407 URL: https://issues.apache.org/jira/browse/HBASE-11407 Project: HBase Issue Type: Improvement Components: Client Reporter: Sergey Beryozkin Priority: Minor Attachments: diff.txt Including the hbase-client module dependency and excluding Jackson dependencies causes the pure HBase query (run with HTableInterface) fail with Jackson ObjectMapper ClassNotFoundException. This is due to org.apache.hadoop.hbase.client.Operation having ObjectMapper statically initialized. Moving ObjectMapper to a dedicated utility will help. The patch will be attached. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11380) HRegion lock object is not being released properly, leading to snapshot failure
[
https://issues.apache.org/jira/browse/HBASE-11380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-11380:
---
Resolution: Fixed
Status: Resolved (was: Patch Available)
HRegion lock object is not being released properly, leading to snapshot
failure
---
Key: HBASE-11380
URL: https://issues.apache.org/jira/browse/HBASE-11380
Project: HBase
Issue Type: Bug
Components: regionserver
Affects Versions: 0.98.3
Reporter: Craig Condit
Assignee: Ted Yu
Priority: Blocker
Fix For: 0.99.0, 0.98.4
Attachments: 11380-v1.txt, 11380-v2.txt, 11380-v3.txt,
HBASE-11380-v2-0.98.3.txt
Background:
We are attempting to create ~ 750 table snapshots on a nightly basis for use
in MR jobs. The jobs are run in batches, with a maximum of around 20 jobs
running simultaneously.
We have started to see the following in our region server logs (after 1 day
uptime):
{noformat}
java.lang.Error: Maximum lock count exceeded
at
java.util.concurrent.locks.ReentrantReadWriteLock$Sync.fullTryAcquireShared(ReentrantReadWriteLock.java:531)
at
java.util.concurrent.locks.ReentrantReadWriteLock$Sync.tryAcquireShared(ReentrantReadWriteLock.java:491)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1326)
at
java.util.concurrent.locks.ReentrantReadWriteLock$ReadLock.tryLock(ReentrantReadWriteLock.java:873)
at org.apache.hadoop.hbase.regionserver.HRegion.lock(HRegion.java:5904)
at org.apache.hadoop.hbase.regionserver.HRegion.lock(HRegion.java:5891)
at
org.apache.hadoop.hbase.regionserver.HRegion.startRegionOperation(HRegion.java:5798)
at
org.apache.hadoop.hbase.regionserver.HRegion.startRegionOperation(HRegion.java:5761)
at
org.apache.hadoop.hbase.regionserver.HRegion.processRowsWithLocks(HRegion.java:4891)
at
org.apache.hadoop.hbase.regionserver.HRegion.mutateRowsWithLocks(HRegion.java:4856)
at
org.apache.hadoop.hbase.regionserver.HRegion.mutateRowsWithLocks(HRegion.java:4838)
at
org.apache.hadoop.hbase.regionserver.HRegion.mutateRow(HRegion.java:4829)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.mutateRows(HRegionServer.java:4390)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.multi(HRegionServer.java:3362)
at
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:29503)
at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2012)
at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:98)
at
org.apache.hadoop.hbase.ipc.SimpleRpcScheduler.consumerLoop(SimpleRpcScheduler.java:168)
at
org.apache.hadoop.hbase.ipc.SimpleRpcScheduler.access$000(SimpleRpcScheduler.java:39)
at
org.apache.hadoop.hbase.ipc.SimpleRpcScheduler$1.run(SimpleRpcScheduler.java:111)
at java.lang.Thread.run(Thread.java:744)
{noformat}
Not sure of the cause, but the result is that snapshots cannot be created. We
see this in our client logs:
{noformat}
Exception in thread main
org.apache.hadoop.hbase.snapshot.HBaseSnapshotException:
org.apache.hadoop.hbase.snapshot.HBaseSnapshotException: Snapshot {
ss=test-snapshot-20140619143753294 table=test type=FLUSH } had an error.
Procedure test-snapshot-20140619143753294 {
waiting=[p3plpadata038.internal,60020,1403140682587,
p3plpadata056.internal,60020,1403140865123,
p3plpadata072.internal,60020,1403141022569]
done=[p3plpadata023.internal,60020,1403140552227,
p3plpadata009.internal,60020,1403140487826] }
at
org.apache.hadoop.hbase.master.snapshot.SnapshotManager.isSnapshotDone(SnapshotManager.java:342)
at
org.apache.hadoop.hbase.master.HMaster.isSnapshotDone(HMaster.java:2907)
at
org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java:40494)
at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2012)
at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:98)
at
org.apache.hadoop.hbase.ipc.FifoRpcScheduler$1.run(FifoRpcScheduler.java:73)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)
Caused by:
org.apache.hadoop.hbase.errorhandling.ForeignException$ProxyThrowable via
[jira] [Updated] (HBASE-11407) hbase-client should not require Jackson for pure HBase queries be executed
[ https://issues.apache.org/jira/browse/HBASE-11407?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Beryozkin updated HBASE-11407: - Affects Version/s: 0.98.3 Fix Version/s: 0.98.4 hbase-client should not require Jackson for pure HBase queries be executed --- Key: HBASE-11407 URL: https://issues.apache.org/jira/browse/HBASE-11407 Project: HBase Issue Type: Improvement Components: Client Affects Versions: 0.98.3 Reporter: Sergey Beryozkin Priority: Minor Fix For: 0.98.4 Attachments: diff.txt Including the hbase-client module dependency and excluding Jackson dependencies causes the pure HBase query (run with HTableInterface) fail with Jackson ObjectMapper ClassNotFoundException. This is due to org.apache.hadoop.hbase.client.Operation having ObjectMapper statically initialized. Moving ObjectMapper to a dedicated utility will help. The patch will be attached. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11407) hbase-client should not require Jackson for pure HBase queries be executed
[
https://issues.apache.org/jira/browse/HBASE-11407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14041958#comment-14041958
]
Nicolas Liochon commented on HBASE-11407:
-
Who is calling this method? We can't just remove it from the client to simplify
our dependencies?
If we don't remove it from the pom, or add a specific test, anyone can add the
dependency back in any innocent patch (ok, transitive dependencies make this
easy to break anyway, but still).
process-wise, we will need the patch for master as well (I've just checked, it
contains this toJSON stuff as well...)
The master says
{code}
// TODO make this configurable
// TODO Do we need this anymore now we have protobuffed it all?
{code}
I think we can just remove all this (from the client at least).
[~saint@gmail.com], wdyt?
hbase-client should not require Jackson for pure HBase queries be executed
---
Key: HBASE-11407
URL: https://issues.apache.org/jira/browse/HBASE-11407
Project: HBase
Issue Type: Improvement
Components: Client
Affects Versions: 0.98.3
Reporter: Sergey Beryozkin
Priority: Minor
Fix For: 0.98.4
Attachments: diff.txt
Including the hbase-client module dependency and excluding Jackson
dependencies causes the pure HBase query (run with HTableInterface) fail with
Jackson ObjectMapper ClassNotFoundException.
This is due to org.apache.hadoop.hbase.client.Operation having ObjectMapper
statically initialized.
Moving ObjectMapper to a dedicated utility will help. The patch will be
attached.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11407) hbase-client should not require Jackson for pure HBase queries be executed
[ https://issues.apache.org/jira/browse/HBASE-11407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14041965#comment-14041965 ] Sergey Beryozkin commented on HBASE-11407: -- The patch is not about removing the dependency in hbase-client pom.xml, it simply moves Jackson ObjectMapper static initialization into a dedicated utility: this will let those consumers of HBase Client API who do not expect to do any sort of JSON formatting exclude Jackson dependency in their own poms. In my case it is the Apache CXF server where not all users may want to have Jackson dependencies. Removing it completely as per the comment may be a better solution but I guess it is more sensitive solution and may take longer to be done. hbase-client should not require Jackson for pure HBase queries be executed --- Key: HBASE-11407 URL: https://issues.apache.org/jira/browse/HBASE-11407 Project: HBase Issue Type: Improvement Components: Client Affects Versions: 0.98.3 Reporter: Sergey Beryozkin Priority: Minor Fix For: 0.98.4 Attachments: diff.txt Including the hbase-client module dependency and excluding Jackson dependencies causes the pure HBase query (run with HTableInterface) fail with Jackson ObjectMapper ClassNotFoundException. This is due to org.apache.hadoop.hbase.client.Operation having ObjectMapper statically initialized. Moving ObjectMapper to a dedicated utility will help. The patch will be attached. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11407) hbase-client should not require Jackson for pure HBase queries be executed
[ https://issues.apache.org/jira/browse/HBASE-11407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14041977#comment-14041977 ] Nicolas Liochon commented on HBASE-11407: - bq. , it simply moves Jackson ObjectMapper static initialization into a dedicated utility Yeah, I saw that. That's what I'm saying it may break again at any time. In this case I think that just removing the dependency is better as it lowers the complexity instead of increasing it... It could be an issue for 0.98 (interface change) but should be done for master as a part of this patch imho. hbase-client should not require Jackson for pure HBase queries be executed --- Key: HBASE-11407 URL: https://issues.apache.org/jira/browse/HBASE-11407 Project: HBase Issue Type: Improvement Components: Client Affects Versions: 0.98.3 Reporter: Sergey Beryozkin Priority: Minor Fix For: 0.98.4 Attachments: diff.txt Including the hbase-client module dependency and excluding Jackson dependencies causes the pure HBase query (run with HTableInterface) fail with Jackson ObjectMapper ClassNotFoundException. This is due to org.apache.hadoop.hbase.client.Operation having ObjectMapper statically initialized. Moving ObjectMapper to a dedicated utility will help. The patch will be attached. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11403) Fix race conditions around Object#notify
[ https://issues.apache.org/jira/browse/HBASE-11403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Liochon updated HBASE-11403: Attachment: 11403.98.v1.patch Fix race conditions around Object#notify Key: HBASE-11403 URL: https://issues.apache.org/jira/browse/HBASE-11403 Project: HBase Issue Type: Bug Components: Client, regionserver Affects Versions: 0.99.0, 0.98.3 Reporter: Nicolas Liochon Assignee: Nicolas Liochon Fix For: 0.99.0, 0.98.4 Attachments: 11403.98.v1.patch, 11403.v1.patch We do have some race conditions there. We don't see them fail in the unit tests, because our #wait are bounded. But from a performance point of view, they do occur. I've reviewed them and fix all the issue I found excepted in the AM (haven't reviewed this one, may be it's fine). On a perf test, this seems to improve the max latency. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11403) Fix race conditions around Object#notify
[ https://issues.apache.org/jira/browse/HBASE-11403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14041995#comment-14041995 ] Nicolas Liochon commented on HBASE-11403: - 11409.98.v1 is what I committed to 0.98, after having executed successfully the small medium tests. Fix race conditions around Object#notify Key: HBASE-11403 URL: https://issues.apache.org/jira/browse/HBASE-11403 Project: HBase Issue Type: Bug Components: Client, regionserver Affects Versions: 0.99.0, 0.98.3 Reporter: Nicolas Liochon Assignee: Nicolas Liochon Fix For: 0.99.0, 0.98.4 Attachments: 11403.98.v1.patch, 11403.v1.patch We do have some race conditions there. We don't see them fail in the unit tests, because our #wait are bounded. But from a performance point of view, they do occur. I've reviewed them and fix all the issue I found excepted in the AM (haven't reviewed this one, may be it's fine). On a perf test, this seems to improve the max latency. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11403) Fix race conditions around Object#notify
[ https://issues.apache.org/jira/browse/HBASE-11403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Liochon updated HBASE-11403: Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) committed to master and 0.98, thanks for the review Stack! Fix race conditions around Object#notify Key: HBASE-11403 URL: https://issues.apache.org/jira/browse/HBASE-11403 Project: HBase Issue Type: Bug Components: Client, regionserver Affects Versions: 0.99.0, 0.98.3 Reporter: Nicolas Liochon Assignee: Nicolas Liochon Fix For: 0.99.0, 0.98.4 Attachments: 11403.98.v1.patch, 11403.v1.patch We do have some race conditions there. We don't see them fail in the unit tests, because our #wait are bounded. But from a performance point of view, they do occur. I've reviewed them and fix all the issue I found excepted in the AM (haven't reviewed this one, may be it's fine). On a perf test, this seems to improve the max latency. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11403) Fix race conditions around Object#notify
[
https://issues.apache.org/jira/browse/HBASE-11403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14041997#comment-14041997
]
Hadoop QA commented on HBASE-11403:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12652180/11403.98.v1.patch
against trunk revision .
ATTACHMENT ID: 12652180
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/9836//console
This message is automatically generated.
Fix race conditions around Object#notify
Key: HBASE-11403
URL: https://issues.apache.org/jira/browse/HBASE-11403
Project: HBase
Issue Type: Bug
Components: Client, regionserver
Affects Versions: 0.99.0, 0.98.3
Reporter: Nicolas Liochon
Assignee: Nicolas Liochon
Fix For: 0.99.0, 0.98.4
Attachments: 11403.98.v1.patch, 11403.v1.patch
We do have some race conditions there. We don't see them fail in the unit
tests, because our #wait are bounded. But from a performance point of view,
they do occur. I've reviewed them and fix all the issue I found excepted in
the AM (haven't reviewed this one, may be it's fine).
On a perf test, this seems to improve the max latency.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11407) hbase-client should not require Jackson for pure HBase queries be executed
[ https://issues.apache.org/jira/browse/HBASE-11407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14042001#comment-14042001 ] Sergey Beryozkin commented on HBASE-11407: -- IMHO you are over complicating it. This is not a bug fix, when you say it will 'break again' it is just confusing. Your claim it is about increasing the complexity is also not accurate. The patch moves the ObjectMapper only on the path where it might be needed, i.e, on to toJson(...) code path. Right now it is needed no matter what, whether toJson will be called or not. hbase-client should not require Jackson for pure HBase queries be executed --- Key: HBASE-11407 URL: https://issues.apache.org/jira/browse/HBASE-11407 Project: HBase Issue Type: Improvement Components: Client Affects Versions: 0.98.3 Reporter: Sergey Beryozkin Priority: Minor Fix For: 0.98.4 Attachments: diff.txt Including the hbase-client module dependency and excluding Jackson dependencies causes the pure HBase query (run with HTableInterface) fail with Jackson ObjectMapper ClassNotFoundException. This is due to org.apache.hadoop.hbase.client.Operation having ObjectMapper statically initialized. Moving ObjectMapper to a dedicated utility will help. The patch will be attached. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11407) hbase-client should not require Jackson for pure HBase queries be executed
[ https://issues.apache.org/jira/browse/HBASE-11407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14042002#comment-14042002 ] Sergey Beryozkin commented on HBASE-11407: -- +1 though to dropping the dependency if you and other experts prefer it. AFAIK HBase client requiring a dependency is not a problem for me as long as it can be optionally excluded which is what this patch supports hbase-client should not require Jackson for pure HBase queries be executed --- Key: HBASE-11407 URL: https://issues.apache.org/jira/browse/HBASE-11407 Project: HBase Issue Type: Improvement Components: Client Affects Versions: 0.98.3 Reporter: Sergey Beryozkin Priority: Minor Fix For: 0.98.4 Attachments: diff.txt Including the hbase-client module dependency and excluding Jackson dependencies causes the pure HBase query (run with HTableInterface) fail with Jackson ObjectMapper ClassNotFoundException. This is due to org.apache.hadoop.hbase.client.Operation having ObjectMapper statically initialized. Moving ObjectMapper to a dedicated utility will help. The patch will be attached. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11407) hbase-client should not require Jackson for pure HBase queries be executed
[ https://issues.apache.org/jira/browse/HBASE-11407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14042013#comment-14042013 ] Nicolas Liochon commented on HBASE-11407: - Yeah that's what I prefer. Except if it's actually used somewhere then yes we have to pay the cost, but the extra dependency in the hbase client does not justify it. We should not workaround our dependencies (I do understand that you don't care much as you have an immediate issue to solve, but on the long term it's better for all of us). I'm +1 for your patch if we remove the dep in the master. hbase-client should not require Jackson for pure HBase queries be executed --- Key: HBASE-11407 URL: https://issues.apache.org/jira/browse/HBASE-11407 Project: HBase Issue Type: Improvement Components: Client Affects Versions: 0.98.3 Reporter: Sergey Beryozkin Priority: Minor Fix For: 0.98.4 Attachments: diff.txt Including the hbase-client module dependency and excluding Jackson dependencies causes the pure HBase query (run with HTableInterface) fail with Jackson ObjectMapper ClassNotFoundException. This is due to org.apache.hadoop.hbase.client.Operation having ObjectMapper statically initialized. Moving ObjectMapper to a dedicated utility will help. The patch will be attached. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11407) hbase-client should not require Jackson for pure HBase queries be executed
[ https://issues.apache.org/jira/browse/HBASE-11407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14042016#comment-14042016 ] Sergey Beryozkin commented on HBASE-11407: -- Hi, you are right. Sorry for being too defensive about my patch. I'm happy the issue is visible now to people who understand HBase better than I do :-) so one way or another it will optimized. Thanks, Sergey hbase-client should not require Jackson for pure HBase queries be executed --- Key: HBASE-11407 URL: https://issues.apache.org/jira/browse/HBASE-11407 Project: HBase Issue Type: Improvement Components: Client Affects Versions: 0.98.3 Reporter: Sergey Beryozkin Priority: Minor Fix For: 0.98.4 Attachments: diff.txt Including the hbase-client module dependency and excluding Jackson dependencies causes the pure HBase query (run with HTableInterface) fail with Jackson ObjectMapper ClassNotFoundException. This is due to org.apache.hadoop.hbase.client.Operation having ObjectMapper statically initialized. Moving ObjectMapper to a dedicated utility will help. The patch will be attached. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-10885) Support visibility expressions on Deletes
[ https://issues.apache.org/jira/browse/HBASE-10885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14042029#comment-14042029 ] Anoop Sam John commented on HBASE-10885: Done. Added some comments in RB. Support visibility expressions on Deletes - Key: HBASE-10885 URL: https://issues.apache.org/jira/browse/HBASE-10885 Project: HBase Issue Type: Improvement Affects Versions: 0.98.1 Reporter: Andrew Purtell Assignee: ramkrishna.s.vasudevan Priority: Blocker Fix For: 0.99.0, 0.98.4 Attachments: 10885-org.apache.hadoop.hbase.security.visibility.TestVisibilityLabelsWithDeletes-output.txt, HBASE-10885_1.patch, HBASE-10885_2.patch, HBASE-10885_new_tag_type_1.patch, HBASE-10885_new_tag_type_2.patch, HBASE-10885_v1.patch, HBASE-10885_v2.patch, HBASE-10885_v2.patch, HBASE-10885_v2.patch, HBASE-10885_v3.patch, HBASE-10885_v4.patch, HBASE-10885_v5.patch, HBASE-10885_v7.patch Accumulo can specify visibility expressions for delete markers. During compaction the cells covered by the tombstone are determined in part by matching the visibility expression. This is useful for the use case of data set coalescing, where entries from multiple data sets carrying different labels are combined into one common large table. Later, a subset of entries can be conveniently removed using visibility expressions. Currently doing the same in HBase would only be possible with a custom coprocessor. Otherwise, a Delete will affect all cells covered by the tombstone regardless of any visibility expression scoping. This is correct behavior in that no data spill is possible, but certainly could be surprising, and is only meant to be transitional. We decided not to support visibility expressions on Deletes to control the complexity of the initial implementation. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11387) metrics: wrong totalRequestCount
[ https://issues.apache.org/jira/browse/HBASE-11387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14042033#comment-14042033 ] Hudson commented on HBASE-11387: SUCCESS: Integrated in HBase-0.98 #352 (See [https://builds.apache.org/job/HBase-0.98/352/]) HBASE-11387 metrics: wrong totalRequestCount (nkeywal: rev cee00a7207a6cdfc73455ce81bb8007e35420e08) * hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionServerMetrics.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java metrics: wrong totalRequestCount Key: HBASE-11387 URL: https://issues.apache.org/jira/browse/HBASE-11387 Project: HBase Issue Type: Bug Components: metrics, regionserver Affects Versions: 0.99.0, 0.98.3 Reporter: Nicolas Liochon Assignee: Nicolas Liochon Fix For: 0.99.0, 0.98.4 Attachments: 11387.98.v1.patch, 11387.v1.patch, 11387.v2.patch We have an unit test here, but it tests for greater than instead of equals. So we didn't see that the number was the double of the actual value. As well we were not testing the multi case. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11403) Fix race conditions around Object#notify
[ https://issues.apache.org/jira/browse/HBASE-11403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14042045#comment-14042045 ] Hudson commented on HBASE-11403: SUCCESS: Integrated in HBase-TRUNK #5229 (See [https://builds.apache.org/job/HBase-TRUNK/5229/]) HBASE-11403 Fix race conditions around Object#notify (nkeywal: rev ab72babd97838317fa0a380fc4d49bf2703ad17c) * hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/LogRoller.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/handler/OpenRegionHandler.java * hbase-server/src/main/java/org/apache/hadoop/hbase/master/ServerManager.java * hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/bucket/BucketCache.java * hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncProcess.java * hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/RpcClient.java Fix race conditions around Object#notify Key: HBASE-11403 URL: https://issues.apache.org/jira/browse/HBASE-11403 Project: HBase Issue Type: Bug Components: Client, regionserver Affects Versions: 0.99.0, 0.98.3 Reporter: Nicolas Liochon Assignee: Nicolas Liochon Fix For: 0.99.0, 0.98.4 Attachments: 11403.98.v1.patch, 11403.v1.patch We do have some race conditions there. We don't see them fail in the unit tests, because our #wait are bounded. But from a performance point of view, they do occur. I've reviewed them and fix all the issue I found excepted in the AM (haven't reviewed this one, may be it's fine). On a perf test, this seems to improve the max latency. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11387) metrics: wrong totalRequestCount
[ https://issues.apache.org/jira/browse/HBASE-11387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14042044#comment-14042044 ] Hudson commented on HBASE-11387: SUCCESS: Integrated in HBase-TRUNK #5229 (See [https://builds.apache.org/job/HBase-TRUNK/5229/]) HBASE-11387 metrics: wrong totalRequestCount (nkeywal: rev b2914be9d32876a5a3337bcfed7dc68c5b7e7a98) * hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionServerMetrics.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RSRpcServices.java metrics: wrong totalRequestCount Key: HBASE-11387 URL: https://issues.apache.org/jira/browse/HBASE-11387 Project: HBase Issue Type: Bug Components: metrics, regionserver Affects Versions: 0.99.0, 0.98.3 Reporter: Nicolas Liochon Assignee: Nicolas Liochon Fix For: 0.99.0, 0.98.4 Attachments: 11387.98.v1.patch, 11387.v1.patch, 11387.v2.patch We have an unit test here, but it tests for greater than instead of equals. So we didn't see that the number was the double of the actual value. As well we were not testing the multi case. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11390) PerformanceEvaluation: add an option to use a single connection
[ https://issues.apache.org/jira/browse/HBASE-11390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14042046#comment-14042046 ] Hudson commented on HBASE-11390: SUCCESS: Integrated in HBase-TRUNK #5229 (See [https://builds.apache.org/job/HBase-TRUNK/5229/]) HBASE-11390 PerformanceEvaluation: add an option to use a single connection (nkeywal: rev e991fd8055112e2ddc0bbeade2ef74504e2407dd) * hbase-server/src/test/java/org/apache/hadoop/hbase/PerformanceEvaluation.java PerformanceEvaluation: add an option to use a single connection --- Key: HBASE-11390 URL: https://issues.apache.org/jira/browse/HBASE-11390 Project: HBase Issue Type: Improvement Components: test Affects Versions: 0.99.0 Reporter: Nicolas Liochon Assignee: Nicolas Liochon Fix For: 0.99.0 Attachments: 11390.v1.patch, 11390.v2.patch The PE tool uses one connection per client. It does not match some use cases when we have multiple threads sharing the same connection. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11387) metrics: wrong totalRequestCount
[ https://issues.apache.org/jira/browse/HBASE-11387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14042086#comment-14042086 ] Hudson commented on HBASE-11387: SUCCESS: Integrated in HBase-0.98-on-Hadoop-1.1 #334 (See [https://builds.apache.org/job/HBase-0.98-on-Hadoop-1.1/334/]) HBASE-11387 metrics: wrong totalRequestCount (nkeywal: rev cee00a7207a6cdfc73455ce81bb8007e35420e08) * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java * hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionServerMetrics.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java metrics: wrong totalRequestCount Key: HBASE-11387 URL: https://issues.apache.org/jira/browse/HBASE-11387 Project: HBase Issue Type: Bug Components: metrics, regionserver Affects Versions: 0.99.0, 0.98.3 Reporter: Nicolas Liochon Assignee: Nicolas Liochon Fix For: 0.99.0, 0.98.4 Attachments: 11387.98.v1.patch, 11387.v1.patch, 11387.v2.patch We have an unit test here, but it tests for greater than instead of equals. So we didn't see that the number was the double of the actual value. As well we were not testing the multi case. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11388) The order parameter is wrong when invoking the constructor of the ReplicationPeer In the method getPeer of the class ReplicationPeersZKImpl
[
https://issues.apache.org/jira/browse/HBASE-11388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14042089#comment-14042089
]
Qianxi Zhang commented on HBASE-11388:
--
[~jdcryans] I think about your idea, I think we should not delete clusterKey
which is an attribute of the ReplicationPeer. The others could invoke the
getClusterKey method to get it though we can infer it using
ZKUtil#getZooKeeperClusterKey(), but I think this is more reasonable. Pls
correct me if am wrong.
The order parameter is wrong when invoking the constructor of the
ReplicationPeer In the method getPeer of the class ReplicationPeersZKImpl
-
Key: HBASE-11388
URL: https://issues.apache.org/jira/browse/HBASE-11388
Project: HBase
Issue Type: Bug
Components: Replication
Affects Versions: 0.99.0, 0.98.3
Reporter: Qianxi Zhang
Assignee: Qianxi Zhang
Priority: Minor
Fix For: 0.99.0
Attachments: HBASE_11388.patch
The parameters is Configurationi, ClusterKey and id in the constructor
of the class ReplicationPeer. But he order parameter is Configurationi,
id and ClusterKey when invoking the constructor of the ReplicationPeer In
the method getPeer of the class ReplicationPeersZKImpl
ReplicationPeer#76
{code}
public ReplicationPeer(Configuration conf, String key, String id) throws
ReplicationException {
this.conf = conf;
this.clusterKey = key;
this.id = id;
try {
this.reloadZkWatcher();
} catch (IOException e) {
throw new ReplicationException(Error connecting to peer cluster with
peerId= + id, e);
}
}
{code}
ReplicationPeersZKImpl#498
{code}
ReplicationPeer peer =
new ReplicationPeer(peerConf, peerId,
ZKUtil.getZooKeeperClusterKey(peerConf));
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11407) hbase-client should not require Jackson for pure HBase queries be executed
[ https://issues.apache.org/jira/browse/HBASE-11407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14042102#comment-14042102 ] Nicolas Liochon commented on HBASE-11407: - No worries :-) . Let's wait for the bay area to wake up, we will see what the others say. Would you be ok to remove the dependency in the master if we go this way? If so I can add you as a contributor and assign you this jira. hbase-client should not require Jackson for pure HBase queries be executed --- Key: HBASE-11407 URL: https://issues.apache.org/jira/browse/HBASE-11407 Project: HBase Issue Type: Improvement Components: Client Affects Versions: 0.98.3 Reporter: Sergey Beryozkin Priority: Minor Fix For: 0.98.4 Attachments: diff.txt Including the hbase-client module dependency and excluding Jackson dependencies causes the pure HBase query (run with HTableInterface) fail with Jackson ObjectMapper ClassNotFoundException. This is due to org.apache.hadoop.hbase.client.Operation having ObjectMapper statically initialized. Moving ObjectMapper to a dedicated utility will help. The patch will be attached. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11407) hbase-client should not require Jackson for pure HBase queries be executed
[ https://issues.apache.org/jira/browse/HBASE-11407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14042178#comment-14042178 ] Sergey Beryozkin commented on HBASE-11407: -- Sure, I'd be up for it. But I can see Operation.toString() documentation says that a JSON representation is created by default so I'm not sure how sensitive would it be to remove the dependency. hbase-client should not require Jackson for pure HBase queries be executed --- Key: HBASE-11407 URL: https://issues.apache.org/jira/browse/HBASE-11407 Project: HBase Issue Type: Improvement Components: Client Affects Versions: 0.98.3 Reporter: Sergey Beryozkin Priority: Minor Fix For: 0.98.4 Attachments: diff.txt Including the hbase-client module dependency and excluding Jackson dependencies causes the pure HBase query (run with HTableInterface) fail with Jackson ObjectMapper ClassNotFoundException. This is due to org.apache.hadoop.hbase.client.Operation having ObjectMapper statically initialized. Moving ObjectMapper to a dedicated utility will help. The patch will be attached. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11407) hbase-client should not require Jackson for pure HBase queries be executed
[ https://issues.apache.org/jira/browse/HBASE-11407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=1404#comment-1404 ] Nicolas Liochon commented on HBASE-11407: - Then we will just remove the comment about 'Do we need this anymore' :-) hbase-client should not require Jackson for pure HBase queries be executed --- Key: HBASE-11407 URL: https://issues.apache.org/jira/browse/HBASE-11407 Project: HBase Issue Type: Improvement Components: Client Affects Versions: 0.98.3 Reporter: Sergey Beryozkin Priority: Minor Fix For: 0.98.4 Attachments: diff.txt Including the hbase-client module dependency and excluding Jackson dependencies causes the pure HBase query (run with HTableInterface) fail with Jackson ObjectMapper ClassNotFoundException. This is due to org.apache.hadoop.hbase.client.Operation having ObjectMapper statically initialized. Moving ObjectMapper to a dedicated utility will help. The patch will be attached. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11407) hbase-client should not require Jackson for pure HBase queries be executed
[ https://issues.apache.org/jira/browse/HBASE-11407?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Liochon updated HBASE-11407: Assignee: Sergey Beryozkin hbase-client should not require Jackson for pure HBase queries be executed --- Key: HBASE-11407 URL: https://issues.apache.org/jira/browse/HBASE-11407 Project: HBase Issue Type: Improvement Components: Client Affects Versions: 0.98.3 Reporter: Sergey Beryozkin Assignee: Sergey Beryozkin Priority: Minor Fix For: 0.98.4 Attachments: diff.txt Including the hbase-client module dependency and excluding Jackson dependencies causes the pure HBase query (run with HTableInterface) fail with Jackson ObjectMapper ClassNotFoundException. This is due to org.apache.hadoop.hbase.client.Operation having ObjectMapper statically initialized. Moving ObjectMapper to a dedicated utility will help. The patch will be attached. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11354) HConnectionImplementation#DelayedClosing does not start
[
https://issues.apache.org/jira/browse/HBASE-11354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14042237#comment-14042237
]
Nicolas Liochon commented on HBASE-11354:
-
bq. Did you add DelayedClosing
I haven't double checked, but I guess it's me a long time ago in HBASE-5399.
It's sad there is no test for this, I guess it could have been done with the
environmentEdgeManager...
bq. Was it around keeping up Connections a short while in case another
connection needed zk else we'd let it go?
Yes. I do remember I tried at the beginning to stop the connection immediately.
HConnectionImplementation#DelayedClosing does not start
---
Key: HBASE-11354
URL: https://issues.apache.org/jira/browse/HBASE-11354
Project: HBase
Issue Type: Bug
Components: Client
Affects Versions: 0.99.0, 0.98.3
Reporter: Qianxi Zhang
Assignee: Qianxi Zhang
Priority: Minor
Attachments: HBASE_11354.patch
The method createAndStart in class DelayedClosing only creates a instance,
but forgets to start it. So thread delayedClosing is not running all the time.
ConnectionManager#1623
{code}
static DelayedClosing createAndStart(HConnectionImplementation hci){
Stoppable stoppable = new Stoppable() {
private volatile boolean isStopped = false;
@Override public void stop(String why) { isStopped = true;}
@Override public boolean isStopped() {return isStopped;}
};
return new DelayedClosing(hci, stoppable);
}
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11297) Remove some synchros in the rpcServer responder
[ https://issues.apache.org/jira/browse/HBASE-11297?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14042250#comment-14042250 ] Nicolas Liochon commented on HBASE-11297: - Thanks. In theory, this patch impacts A, B D, as YCSB use a single connection per client: so it's only the multiput that does have multiple threads on the same connections. My results (on a different test, on master, and with the v3) are a little bit better. I think the code with this patch is slightly better, so I would like to commit it. Are you guys ok? [~saint@gmail.com] any opinion? Remove some synchros in the rpcServer responder --- Key: HBASE-11297 URL: https://issues.apache.org/jira/browse/HBASE-11297 Project: HBase Issue Type: Bug Components: regionserver Affects Versions: 0.99.0 Reporter: Nicolas Liochon Assignee: Nicolas Liochon Fix For: 0.99.0 Attachments: 11297.v1.patch, 11297.v2.patch, 11297.v2.v98.patch, 11297.v3.patch This is on top of another patch that I'm going to put into another jira. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11403) Fix race conditions around Object#notify
[ https://issues.apache.org/jira/browse/HBASE-11403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14042274#comment-14042274 ] Hudson commented on HBASE-11403: SUCCESS: Integrated in HBase-0.98 #353 (See [https://builds.apache.org/job/HBase-0.98/353/]) HBASE-11403 Fix race conditions around Object#notify (nkeywal: rev 01d5e7626c04315995333cef424cc5ba9cf4be72) * hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncProcess.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/handler/OpenRegionHandler.java * hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/RpcClient.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/LogRoller.java * hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/bucket/BucketCache.java * hbase-server/src/main/java/org/apache/hadoop/hbase/master/ServerManager.java Fix race conditions around Object#notify Key: HBASE-11403 URL: https://issues.apache.org/jira/browse/HBASE-11403 Project: HBase Issue Type: Bug Components: Client, regionserver Affects Versions: 0.99.0, 0.98.3 Reporter: Nicolas Liochon Assignee: Nicolas Liochon Fix For: 0.99.0, 0.98.4 Attachments: 11403.98.v1.patch, 11403.v1.patch We do have some race conditions there. We don't see them fail in the unit tests, because our #wait are bounded. But from a performance point of view, they do occur. I've reviewed them and fix all the issue I found excepted in the AM (haven't reviewed this one, may be it's fine). On a perf test, this seems to improve the max latency. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11403) Fix race conditions around Object#notify
[ https://issues.apache.org/jira/browse/HBASE-11403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14042276#comment-14042276 ] Hudson commented on HBASE-11403: FAILURE: Integrated in HBase-0.98-on-Hadoop-1.1 #335 (See [https://builds.apache.org/job/HBase-0.98-on-Hadoop-1.1/335/]) HBASE-11403 Fix race conditions around Object#notify (nkeywal: rev 01d5e7626c04315995333cef424cc5ba9cf4be72) * hbase-client/src/main/java/org/apache/hadoop/hbase/client/AsyncProcess.java * hbase-client/src/main/java/org/apache/hadoop/hbase/ipc/RpcClient.java * hbase-server/src/main/java/org/apache/hadoop/hbase/master/ServerManager.java * hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/bucket/BucketCache.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/handler/OpenRegionHandler.java * hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/LogRoller.java Fix race conditions around Object#notify Key: HBASE-11403 URL: https://issues.apache.org/jira/browse/HBASE-11403 Project: HBase Issue Type: Bug Components: Client, regionserver Affects Versions: 0.99.0, 0.98.3 Reporter: Nicolas Liochon Assignee: Nicolas Liochon Fix For: 0.99.0, 0.98.4 Attachments: 11403.98.v1.patch, 11403.v1.patch We do have some race conditions there. We don't see them fail in the unit tests, because our #wait are bounded. But from a performance point of view, they do occur. I've reviewed them and fix all the issue I found excepted in the AM (haven't reviewed this one, may be it's fine). On a perf test, this seems to improve the max latency. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-10933) hbck -fixHdfsOrphans is not working properly it throws null pointer exception
[
https://issues.apache.org/jira/browse/HBASE-10933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14042283#comment-14042283
]
Jonathan Hsieh commented on HBASE-10933:
What do you mean by exclusive? -- in the [a, b), [b, c) sense where b is in
the second but not the first? If we did next we'd still be able to sneak
'abc\x0' betwen 'abc' and 'abd'
Taking a step back, my main concern is that this nextKey is a public method in
Bytes -- a common util class used internally and by users. If a call lives
there, the api and its semantics must be clear and correct in all cases.
If you made this a private method in HbaseFsck, (thus only scoped to that
tool), I wouldn't be concerned.
bq. // I just thought this can be a generic code. You never know it could be
required later in the development. If still not required, I will remove it now ?
Please remove and just keep the specific version. Though you never know, it
tends to be you never need and makes the code harder to read (or requires the
reader to read more code). Because this is in the public api, I'm more
concerned.
hbck -fixHdfsOrphans is not working properly it throws null pointer exception
-
Key: HBASE-10933
URL: https://issues.apache.org/jira/browse/HBASE-10933
Project: HBase
Issue Type: Bug
Components: hbck
Affects Versions: 0.94.16, 0.98.2
Reporter: Deepak Sharma
Assignee: Kashif J S
Priority: Critical
Fix For: 0.99.0, 0.94.22
Attachments: HBASE-10933-0.94-v1.patch, HBASE-10933-0.94-v2.patch,
HBASE-10933-trunk-v1.patch, HBASE-10933-trunk-v2.patch, TestResults-0.94.txt,
TestResults-trunk.txt
if we regioninfo file is not existing in hbase region then if we run hbck
repair or hbck -fixHdfsOrphans
then it is not able to resolve this problem it throws null pointer exception
{code}
2014-04-08 20:11:49,750 INFO [main] util.HBaseFsck
(HBaseFsck.java:adoptHdfsOrphans(470)) - Attempting to handle orphan hdfs
dir:
hdfs://10.18.40.28:54310/hbase/TestHdfsOrphans1/5a3de9ca65e587cb05c9384a3981c950
java.lang.NullPointerException
at
org.apache.hadoop.hbase.util.HBaseFsck$TableInfo.access$000(HBaseFsck.java:1939)
at
org.apache.hadoop.hbase.util.HBaseFsck.adoptHdfsOrphan(HBaseFsck.java:497)
at
org.apache.hadoop.hbase.util.HBaseFsck.adoptHdfsOrphans(HBaseFsck.java:471)
at
org.apache.hadoop.hbase.util.HBaseFsck.restoreHdfsIntegrity(HBaseFsck.java:591)
at
org.apache.hadoop.hbase.util.HBaseFsck.offlineHdfsIntegrityRepair(HBaseFsck.java:369)
at org.apache.hadoop.hbase.util.HBaseFsck.onlineHbck(HBaseFsck.java:447)
at org.apache.hadoop.hbase.util.HBaseFsck.exec(HBaseFsck.java:3769)
at org.apache.hadoop.hbase.util.HBaseFsck.run(HBaseFsck.java:3587)
at
com.huawei.isap.test.smartump.hadoop.hbase.HbaseHbckRepair.repairToFixHdfsOrphans(HbaseHbckRepair.java:244)
at
com.huawei.isap.test.smartump.hadoop.hbase.HbaseHbckRepair.setUp(HbaseHbckRepair.java:84)
at junit.framework.TestCase.runBare(TestCase.java:132)
at junit.framework.TestResult$1.protect(TestResult.java:110)
at junit.framework.TestResult.runProtected(TestResult.java:128)
at junit.framework.TestResult.run(TestResult.java:113)
at junit.framework.TestCase.run(TestCase.java:124)
at junit.framework.TestSuite.runTest(TestSuite.java:243)
at junit.framework.TestSuite.run(TestSuite.java:238)
at
org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:83)
at
org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:50)
at
org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
at
org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:467)
at
org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:683)
at
org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:390)
at
org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:197)
{code}
problem i got it is because since in HbaseFsck class
{code}
private void adoptHdfsOrphan(HbckInfo hi)
{code}
we are intializing tableinfo using SortedMapString, TableInfo tablesInfo
object
{code}
TableInfo tableInfo = tablesInfo.get(tableName);
{code}
but in private SortedMapString, TableInfo loadHdfsRegionInfos()
{code}
for (HbckInfo hbi: hbckInfos) {
if (hbi.getHdfsHRI() == null) {
// was an orphan
continue;
}
{code}
we have check if a region is orphan then that table will can not be added in
SortedMapString, TableInfo tablesInfo
so later
[jira] [Commented] (HBASE-11388) The order parameter is wrong when invoking the constructor of the ReplicationPeer In the method getPeer of the class ReplicationPeersZKImpl
[
https://issues.apache.org/jira/browse/HBASE-11388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14042287#comment-14042287
]
Jean-Daniel Cryans commented on HBASE-11388:
Ah, sorry, I wasn't clear. I was thinking that we should only remove it from
the constructor, but then populate that field within the constructor using
getZooKeeperClusterKey.
The order parameter is wrong when invoking the constructor of the
ReplicationPeer In the method getPeer of the class ReplicationPeersZKImpl
-
Key: HBASE-11388
URL: https://issues.apache.org/jira/browse/HBASE-11388
Project: HBase
Issue Type: Bug
Components: Replication
Affects Versions: 0.99.0, 0.98.3
Reporter: Qianxi Zhang
Assignee: Qianxi Zhang
Priority: Minor
Fix For: 0.99.0
Attachments: HBASE_11388.patch
The parameters is Configurationi, ClusterKey and id in the constructor
of the class ReplicationPeer. But he order parameter is Configurationi,
id and ClusterKey when invoking the constructor of the ReplicationPeer In
the method getPeer of the class ReplicationPeersZKImpl
ReplicationPeer#76
{code}
public ReplicationPeer(Configuration conf, String key, String id) throws
ReplicationException {
this.conf = conf;
this.clusterKey = key;
this.id = id;
try {
this.reloadZkWatcher();
} catch (IOException e) {
throw new ReplicationException(Error connecting to peer cluster with
peerId= + id, e);
}
}
{code}
ReplicationPeersZKImpl#498
{code}
ReplicationPeer peer =
new ReplicationPeer(peerConf, peerId,
ZKUtil.getZooKeeperClusterKey(peerConf));
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11407) hbase-client should not require Jackson for pure HBase queries be executed
[ https://issues.apache.org/jira/browse/HBASE-11407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14042345#comment-14042345 ] Andrew Purtell commented on HBASE-11407: IIRC this is used for the slow operation logging in IPC tracing on the server. Since we retain the Jackson dep for the server package this change should be fine. Please include a note in Javadoc for Operation#toJSON that it won't work any more unless the application has Jackson on the classpath. Curious if others think this is an issue with backwards compat. hbase-client should not require Jackson for pure HBase queries be executed --- Key: HBASE-11407 URL: https://issues.apache.org/jira/browse/HBASE-11407 Project: HBase Issue Type: Improvement Components: Client Affects Versions: 0.98.3 Reporter: Sergey Beryozkin Assignee: Sergey Beryozkin Priority: Minor Fix For: 0.98.4 Attachments: diff.txt Including the hbase-client module dependency and excluding Jackson dependencies causes the pure HBase query (run with HTableInterface) fail with Jackson ObjectMapper ClassNotFoundException. This is due to org.apache.hadoop.hbase.client.Operation having ObjectMapper statically initialized. Moving ObjectMapper to a dedicated utility will help. The patch will be attached. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11407) hbase-client should not require Jackson for pure HBase queries be executed
[ https://issues.apache.org/jira/browse/HBASE-11407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14042346#comment-14042346 ] Andrew Purtell commented on HBASE-11407: Also the diff on this issue is missing any POM changes that remove Jackson deps. Assume those are warranted. Let's get another patch up with all of the necessary changes and run it through HadoopQA? hbase-client should not require Jackson for pure HBase queries be executed --- Key: HBASE-11407 URL: https://issues.apache.org/jira/browse/HBASE-11407 Project: HBase Issue Type: Improvement Components: Client Affects Versions: 0.98.3 Reporter: Sergey Beryozkin Assignee: Sergey Beryozkin Priority: Minor Fix For: 0.98.4 Attachments: diff.txt Including the hbase-client module dependency and excluding Jackson dependencies causes the pure HBase query (run with HTableInterface) fail with Jackson ObjectMapper ClassNotFoundException. This is due to org.apache.hadoop.hbase.client.Operation having ObjectMapper statically initialized. Moving ObjectMapper to a dedicated utility will help. The patch will be attached. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11407) hbase-client should not require Jackson for pure HBase queries be executed
[ https://issues.apache.org/jira/browse/HBASE-11407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14042367#comment-14042367 ] Nicolas Liochon commented on HBASE-11407: - bq. IIRC this is used for the slow operation logging in IPC tracing on the server. Yeah, I've checked, you're right, toString() is used in a few logs. Not anymore for slowOperation: the rpcServer uses its own json mapper. But we should not log anyway as it's a security issue (iirc it's now correct in the slowOperation and does not expose any potentially private field). hbase-client should not require Jackson for pure HBase queries be executed --- Key: HBASE-11407 URL: https://issues.apache.org/jira/browse/HBASE-11407 Project: HBase Issue Type: Improvement Components: Client Affects Versions: 0.98.3 Reporter: Sergey Beryozkin Assignee: Sergey Beryozkin Priority: Minor Fix For: 0.98.4 Attachments: diff.txt Including the hbase-client module dependency and excluding Jackson dependencies causes the pure HBase query (run with HTableInterface) fail with Jackson ObjectMapper ClassNotFoundException. This is due to org.apache.hadoop.hbase.client.Operation having ObjectMapper statically initialized. Moving ObjectMapper to a dedicated utility will help. The patch will be attached. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11370) SSH doesn't need to scan meta if not using ZK for assignment
[
https://issues.apache.org/jira/browse/HBASE-11370?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14042450#comment-14042450
]
stack commented on HBASE-11370:
---
nit: Is the javadoc right still?
- * @return map of servers not online to their assigned regions, as stored
- * in META
+ * @return set of servers not online, as stored in META
We are not looking at META anymore?
Why remove the methods in MasterFileSystem? They are not used anymore?
What is lastHost?
- public synchronized RegionState createRegionState(
- final HRegionInfo hri, State newState, ServerName serverName) {
+ public synchronized RegionState createRegionState(final HRegionInfo hri,
+ State newState, ServerName serverName, ServerName lastHost) {
You add to a public method but don't update javadoc maybe there is none on
this method?
Oh, looks like it is the server the region was last reported to be up on. That
comes from in-memory state?
And the new splitrRegion and mergeRegion methods are related to this patch?
They were moved?
Why we need to do this in TestHBCK now?
+ // Remove daughters from regionStates
Is it a test problem or new state or a state we always had but you figured it
exists?
lgtm Jimmy. Yeah, be great if mighty [~jeffreyz] had a looksee
SSH doesn't need to scan meta if not using ZK for assignment
Key: HBASE-11370
URL: https://issues.apache.org/jira/browse/HBASE-11370
Project: HBase
Issue Type: Improvement
Reporter: Jimmy Xiang
Assignee: Jimmy Xiang
Priority: Minor
Fix For: 0.99.0
Attachments: hbase-11370.patch
If we don't use ZK for assignment, the meta content should be the same as
that in memory. So we should be able to avoid a meta scan.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HBASE-11376) Presplit table in IntegrationTestBigLinkedList's Generator tool
[ https://issues.apache.org/jira/browse/HBASE-11376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Enis Soztutar updated HBASE-11376: -- Summary: Presplit table in IntegrationTestBigLinkedList's Generator tool (was: IntegrationTestBigLinkedList's Generator tool does not generate keys belonging to all regions in a large table.) Presplit table in IntegrationTestBigLinkedList's Generator tool --- Key: HBASE-11376 URL: https://issues.apache.org/jira/browse/HBASE-11376 Project: HBase Issue Type: Improvement Reporter: Vandana Ayyalasomayajula Assignee: Vandana Ayyalasomayajula Priority: Minor Attachments: HBASE-11376_1.patch, HBASE-11376_2.patch, HBASE-11376_3.patch When IntegrationTestBigLinkedList's generator tool is used to generate keys to a large table ( 2200 regions), only some regions have keys and others are empty. It would be good to generate keys to all the regions of the table. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11102) Document JDK versions supported by each release
[ https://issues.apache.org/jira/browse/HBASE-11102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14042479#comment-14042479 ] Enis Soztutar commented on HBASE-11102: --- bq. Do we want to have a general goal to purge the Ref Guide of pre-0.94 info in general, for consistency's sake? I think that would be ok. Document JDK versions supported by each release --- Key: HBASE-11102 URL: https://issues.apache.org/jira/browse/HBASE-11102 Project: HBase Issue Type: Sub-task Components: documentation Reporter: Enis Soztutar Assignee: Misty Stanley-Jones Fix For: 0.99.0 Attachments: HBASE-11102-1.patch, HBASE-11102.patch We can make use of a JDK version x HBase version matrix to explain which JDK version is supported and required. 0.94, 0.96, and 0.98 releases all support JDK6 and JDK7. For 1.0, there is a discussion thread to decide whether to drop JDK6 support. There has been some work to support JDK8. We can also document that. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11376) Presplit table in IntegrationTestBigLinkedList's Generator tool
[ https://issues.apache.org/jira/browse/HBASE-11376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Enis Soztutar updated HBASE-11376: -- Fix Version/s: 0.99.0 Presplit table in IntegrationTestBigLinkedList's Generator tool --- Key: HBASE-11376 URL: https://issues.apache.org/jira/browse/HBASE-11376 Project: HBase Issue Type: Improvement Reporter: Vandana Ayyalasomayajula Assignee: Vandana Ayyalasomayajula Priority: Minor Fix For: 0.99.0 Attachments: HBASE-11376_1.patch, HBASE-11376_2.patch, HBASE-11376_3.patch When IntegrationTestBigLinkedList's generator tool is used to generate keys to a large table ( 2200 regions), only some regions have keys and others are empty. It would be good to generate keys to all the regions of the table. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11376) Presplit table in IntegrationTestBigLinkedList's Generator tool
[ https://issues.apache.org/jira/browse/HBASE-11376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Enis Soztutar updated HBASE-11376: -- Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) I've committed this. Thanks Vandana. Presplit table in IntegrationTestBigLinkedList's Generator tool --- Key: HBASE-11376 URL: https://issues.apache.org/jira/browse/HBASE-11376 Project: HBase Issue Type: Improvement Reporter: Vandana Ayyalasomayajula Assignee: Vandana Ayyalasomayajula Priority: Minor Fix For: 0.99.0 Attachments: HBASE-11376_1.patch, HBASE-11376_2.patch, HBASE-11376_3.patch When IntegrationTestBigLinkedList's generator tool is used to generate keys to a large table ( 2200 regions), only some regions have keys and others are empty. It would be good to generate keys to all the regions of the table. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11405) Multiple invocations of hbck in parallel disables balancer permanently
[
https://issues.apache.org/jira/browse/HBASE-11405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14042503#comment-14042503
]
Jean-Marc Spaggiari commented on HBASE-11405:
-
I agree. hbck in parallel might create some mess... We might want to block that.
Multiple invocations of hbck in parallel disables balancer permanently
---
Key: HBASE-11405
URL: https://issues.apache.org/jira/browse/HBASE-11405
Project: HBase
Issue Type: Bug
Components: Balancer, hbck
Affects Versions: 0.99.0
Reporter: bharath v
This is because of the following piece of code in hbck
{code:borderStyle=solid}
boolean oldBalancer = admin.setBalancerRunning(false, true);
try {
onlineConsistencyRepair();
}
finally {
admin.setBalancerRunning(oldBalancer, false);
}
{code}
Newer invocations set oldBalancer to false as it was disabled by previous
invocations and this disables balancer permanently unless its manually turned
on by the user. Easy to reproduce, just run hbck 100 times in a loop in 2
different sessions and you can see that balancer is set to false in the
HMaster logs.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11401) Issue with seqNo binding for KV mvcc
[
https://issues.apache.org/jira/browse/HBASE-11401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14042520#comment-14042520
]
Jeffrey Zhong commented on HBASE-11401:
---
The order below may work:
{code}
row lock acquire + appendNoSync() + waitForSeqAssigned + memstore.add + row
lock release + sync()
{code}
Issue with seqNo binding for KV mvcc
Key: HBASE-11401
URL: https://issues.apache.org/jira/browse/HBASE-11401
Project: HBase
Issue Type: Bug
Affects Versions: 0.99.0
Reporter: Anoop Sam John
Priority: Critical
Fix For: 0.99.0
After HBASE-8763, we have combined KV mvcc and HLog seqNo. This is
implemented in a tricky way now.
In HRegion on write path, we first write to memstore and then write to HLog
finally sync log. So at the time of write to memstore we dont know the WAL
seqNo. To overcome this, we hold the ref to the KV objects just added to
memstore and pass those also to write to wal call. Once the seqNo is
obtained, we will reset the mvcc is those KVs with this seqNo. (While write
to memstore we wrote kvs with a very high temp value for mvcc so that
concurrent readers wont see them)
This model works well with the DefaultMemstore. During the write there wont
be any concurrent call to snapshot().
But now we have memstore as a pluggable interface. The above model of late
binding assumes that the memstore internal datastructure continue to refer to
same java objects. This might not be true always. Like in HBASE-10713, in
btw the kvs can be converted into a CellBlock. If we discontinue to refer to
same KV java objects, we will fail in getting the seqNo assigned as kv mvcc.
If we were doing write and sync to wal and then write to memstore, this would
have get solved. But this model we changed (in 94 I believe) for better perf.
Under HRegion level lock, we write to memstore and then to wal. Finally out
of lock we do the the log sync. So we can not change it now
I tried changing the order of ops within the lock (ie. write to log and then
to memstore) so that we can get the seqNo when write to memstore. But because
of the new HLog write model, we are not guarenteed to get the write to done
immediately.
One possible way can be add a new API in Log level, to get a next seqNo
alone. Call this first and then using which write to memstore and then to wal
(using this seqNo). Just a random thought. Not tried.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Created] (HBASE-11408) multiple SLF4J bindings warning messages when running HBase shell
Duo Xu created HBASE-11408: -- Summary: multiple SLF4J bindings warning messages when running HBase shell Key: HBASE-11408 URL: https://issues.apache.org/jira/browse/HBASE-11408 Project: HBase Issue Type: Bug Affects Versions: 0.98.3, 0.98.2 Reporter: Duo Xu When running hbase shell, we saw warnings like this: SLF4J: Class path contains multiple SLF4J bindings. SLF4J: Found binding in [jar:file:/C:/apps/dist/hbase-0.98.0.2.1.3.0-1928-hadoop2/lib/slf4j-log4j12-1.6.4.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: Found binding in [jar:file:/C:/apps/dist/hadoop-2.4.0.2.1.3.0-1928/share/hadoop/common/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.cla ss] SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11408) multiple SLF4J bindings warning messages when running HBase shell
[ https://issues.apache.org/jira/browse/HBASE-11408?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Xu updated HBASE-11408: --- Description: When running hbase shell, we saw warnings like this: SLF4J: Class path contains multiple SLF4J bindings. SLF4J: Found binding in [jar:file:/C:/apps/dist/hbase-0.98.0.2.1.3.0-1928-hadoop2/lib/slf4j-log4j12-1.6.4.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: Found binding in [jar:file:/C:/apps/dist/hadoop-2.4.0.2.1.3.0-1928/share/hadoop/common/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation. was: When running hbase shell, we saw warnings like this: SLF4J: Class path contains multiple SLF4J bindings. SLF4J: Found binding in [jar:file:/C:/apps/dist/hbase-0.98.0.2.1.3.0-1928-hadoop2/lib/slf4j-log4j12-1.6.4.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: Found binding in [jar:file:/C:/apps/dist/hadoop-2.4.0.2.1.3.0-1928/share/hadoop/common/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.cla ss] SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation. multiple SLF4J bindings warning messages when running HBase shell --- Key: HBASE-11408 URL: https://issues.apache.org/jira/browse/HBASE-11408 Project: HBase Issue Type: Bug Affects Versions: 0.98.2, 0.98.3 Reporter: Duo Xu When running hbase shell, we saw warnings like this: SLF4J: Class path contains multiple SLF4J bindings. SLF4J: Found binding in [jar:file:/C:/apps/dist/hbase-0.98.0.2.1.3.0-1928-hadoop2/lib/slf4j-log4j12-1.6.4.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: Found binding in [jar:file:/C:/apps/dist/hadoop-2.4.0.2.1.3.0-1928/share/hadoop/common/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11370) SSH doesn't need to scan meta if not using ZK for assignment
[ https://issues.apache.org/jira/browse/HBASE-11370?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14042631#comment-14042631 ] Jimmy Xiang commented on HBASE-11370: - bq. nit: Is the javadoc right still? I will fix it. It should be something like return a set of servers not online that host some region per META bq. We are not looking at META anymore? If not using ZK for assignment, SSH doesn't look at META anymore. bq. Why remove the methods in MasterFileSystem? They are not used anymore? It has some methods that used to be used by SSH. I forgot to purge it in HBASE-11059. bq. What is lastHost? It is the host where the region is previously assigned to. I will add some comment here. serverName is the host where the region is transitioning now. They could be different. bq. That comes from in-memory state? That's from the META during rebuilding the user regions. bq. And the new splitrRegion and mergeRegion methods are related to this patch? They were moved? Yes, they are related. They are changed a little to update the in-memory state too. bq. Why we need to do this in TestHBCK now? This is because of the change to splitRegion above. SSH doesn't need to scan meta if not using ZK for assignment Key: HBASE-11370 URL: https://issues.apache.org/jira/browse/HBASE-11370 Project: HBase Issue Type: Improvement Reporter: Jimmy Xiang Assignee: Jimmy Xiang Priority: Minor Fix For: 0.99.0 Attachments: hbase-11370.patch If we don't use ZK for assignment, the meta content should be the same as that in memory. So we should be able to avoid a meta scan. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11408) multiple SLF4J bindings warning messages when running HBase shell
[ https://issues.apache.org/jira/browse/HBASE-11408?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14042635#comment-14042635 ] stack commented on HBASE-11408: --- You have hadoop on your hbase CLASSPATH? How are you starting your HBase? multiple SLF4J bindings warning messages when running HBase shell --- Key: HBASE-11408 URL: https://issues.apache.org/jira/browse/HBASE-11408 Project: HBase Issue Type: Bug Affects Versions: 0.98.2, 0.98.3 Reporter: Duo Xu When running hbase shell, we saw warnings like this: SLF4J: Class path contains multiple SLF4J bindings. SLF4J: Found binding in [jar:file:/C:/apps/dist/hbase-0.98.0.2.1.3.0-1928-hadoop2/lib/slf4j-log4j12-1.6.4.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: Found binding in [jar:file:/C:/apps/dist/hadoop-2.4.0.2.1.3.0-1928/share/hadoop/common/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11370) SSH doesn't need to scan meta if not using ZK for assignment
[ https://issues.apache.org/jira/browse/HBASE-11370?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14042638#comment-14042638 ] stack commented on HBASE-11370: --- Go for it +1 SSH doesn't need to scan meta if not using ZK for assignment Key: HBASE-11370 URL: https://issues.apache.org/jira/browse/HBASE-11370 Project: HBase Issue Type: Improvement Reporter: Jimmy Xiang Assignee: Jimmy Xiang Priority: Minor Fix For: 0.99.0 Attachments: hbase-11370.patch If we don't use ZK for assignment, the meta content should be the same as that in memory. So we should be able to avoid a meta scan. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11339) HBase MOB
[ https://issues.apache.org/jira/browse/HBASE-11339?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14042655#comment-14042655 ] Jonathan Hsieh commented on HBASE-11339: In the pdf design, is there one MobManager per RS or one MobManager per table or one MobManager per region? Is the mob hfiles kind of like a shared cf that all regions with mobs eventually throw their data into? Can you explain what happens if I have a RS with regions, some belonging to tableA and and some belonging to tableB. Let's say all writes to tableA and tableB have Mobs in them. # a region gets full and decides to flush. we generate one mob file. 10 separate flushes, 10 separate mob files. # an admin user issues a flush tableA command and there are multiple tableA regions on the rs. How many mob files are generated? one mob file per region in tableA on the rs? exactly one because only one table was flushed? exactly one because only one table was flushed? # the node goes down cleanly, causing all regions to be flushed. how many mobfiles are generated. one mob file per region on the rs, one mob file per table on the rs, or exactly one because there is only one rs? Where are the mob files written to? are they in the region dir, the family dir, the table dir or something else? In 98, the dir structure is /hbase/namespace/table/region/cf/hfile. Where do the mob files for region1 of tableA go and where does the mob files for region2 of tableB go to? bq. Yes, this could improve the compaction. But this doesn't reduce the twice writing for the mob file. Ok, so this is essentially equal -- both the pdf and the cf approach require a minimum of 2x.writes of mob data bq. Saving the mob files by stores than by region server is more efficient to use the TTL to clean the expired mobs. With this It sounds like new mob file per region, and that mobs would still generate the same number of files as the separate cf's approach. Can't we (or do we already) have the ttl optimization in our existing cf's since our hfiles have start and end ts in them? ... (i think I need to understand the answers to the first section before some of this makes sense to me.) HBase MOB - Key: HBASE-11339 URL: https://issues.apache.org/jira/browse/HBASE-11339 Project: HBase Issue Type: New Feature Components: regionserver, Scanners Reporter: Jingcheng Du Assignee: Jingcheng Du Attachments: HBase LOB Design.pdf It's quite useful to save the medium binary data like images, documents into Apache HBase. Unfortunately directly saving the binary MOB(medium object) to HBase leads to a worse performance since the frequent split and compaction. In this design, the MOB data are stored in an more efficient way, which keeps a high write/read performance and guarantees the data consistency in Apache HBase. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11297) Remove some synchros in the rpcServer responder
[ https://issues.apache.org/jira/browse/HBASE-11297?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14042665#comment-14042665 ] stack commented on HBASE-11297: --- Very nice numbers there [~apurtell] I am good w/ the tradeoff. I like the patch. Remove some synchros in the rpcServer responder --- Key: HBASE-11297 URL: https://issues.apache.org/jira/browse/HBASE-11297 Project: HBase Issue Type: Bug Components: regionserver Affects Versions: 0.99.0 Reporter: Nicolas Liochon Assignee: Nicolas Liochon Fix For: 0.99.0 Attachments: 11297.v1.patch, 11297.v2.patch, 11297.v2.v98.patch, 11297.v3.patch This is on top of another patch that I'm going to put into another jira. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11354) HConnectionImplementation#DelayedClosing does not start
[
https://issues.apache.org/jira/browse/HBASE-11354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14042670#comment-14042670
]
stack commented on HBASE-11354:
---
[~nkeywal] Should we try and keep it then? Apply this patch so it is in action
again?
HConnectionImplementation#DelayedClosing does not start
---
Key: HBASE-11354
URL: https://issues.apache.org/jira/browse/HBASE-11354
Project: HBase
Issue Type: Bug
Components: Client
Affects Versions: 0.99.0, 0.98.3
Reporter: Qianxi Zhang
Assignee: Qianxi Zhang
Priority: Minor
Attachments: HBASE_11354.patch
The method createAndStart in class DelayedClosing only creates a instance,
but forgets to start it. So thread delayedClosing is not running all the time.
ConnectionManager#1623
{code}
static DelayedClosing createAndStart(HConnectionImplementation hci){
Stoppable stoppable = new Stoppable() {
private volatile boolean isStopped = false;
@Override public void stop(String why) { isStopped = true;}
@Override public boolean isStopped() {return isStopped;}
};
return new DelayedClosing(hci, stoppable);
}
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11407) hbase-client should not require Jackson for pure HBase queries be executed
[ https://issues.apache.org/jira/browse/HBASE-11407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14042688#comment-14042688 ] Sergey Beryozkin commented on HBASE-11407: -- The reason pom.xml is not updated is that the goal of the patch was to let consumers of hbase-client who do pure HBase queries and wishing to optimize their dependencies exclude Jackson in their own poms. As far as I understand it may still be needed in certain cases. So I guess the dependency needs to stay for now, with a provided patch most users will be able to exclude it if they want to. I'm not sure making the dependency optional is the right solution given that it is still used on some code paths (logging): one may say that a number of users who currently depend on Jackson is at least no less than a number of users like me who want to exclude it :-). Thanks, Sergey hbase-client should not require Jackson for pure HBase queries be executed --- Key: HBASE-11407 URL: https://issues.apache.org/jira/browse/HBASE-11407 Project: HBase Issue Type: Improvement Components: Client Affects Versions: 0.98.3 Reporter: Sergey Beryozkin Assignee: Sergey Beryozkin Priority: Minor Fix For: 0.98.4 Attachments: diff.txt Including the hbase-client module dependency and excluding Jackson dependencies causes the pure HBase query (run with HTableInterface) fail with Jackson ObjectMapper ClassNotFoundException. This is due to org.apache.hadoop.hbase.client.Operation having ObjectMapper statically initialized. Moving ObjectMapper to a dedicated utility will help. The patch will be attached. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11027) Remove kv.isDeleteXX() and related methods and use CellUtil apis.
[
https://issues.apache.org/jira/browse/HBASE-11027?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14042691#comment-14042691
]
stack commented on HBASE-11027:
---
This is great.
Have isDelete(final Cell cell) { call public static boolean isDelete(final byte
type) { if you make a new patch? Can do it on commit.
I suppose removing the methods from KV is ok since it is marked Private
audience.
+1
Remove kv.isDeleteXX() and related methods and use CellUtil apis.
-
Key: HBASE-11027
URL: https://issues.apache.org/jira/browse/HBASE-11027
Project: HBase
Issue Type: Sub-task
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan
Fix For: 0.99.0
Attachments: HBASE-11027.patch
WE have code like
{code}
kv.isLatestTimestamp() kv.isDeleteType()
{code}
We could remove them and use CellUtil.isDeleteType() so that Cells can be
directly used instead of Converting a cell to kv.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11396) Invalid meta entries can lead to unstartable master
[
https://issues.apache.org/jira/browse/HBASE-11396?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14042712#comment-14042712
]
stack commented on HBASE-11396:
---
Yeah, was going to say add a test since easy to do but this is good as is I'd
say. Funny we've not had this more often... (Others having 'finest moments' --
smile). Good on you Craig.
Invalid meta entries can lead to unstartable master
---
Key: HBASE-11396
URL: https://issues.apache.org/jira/browse/HBASE-11396
Project: HBase
Issue Type: Bug
Components: master
Affects Versions: 0.98.3
Reporter: Craig Condit
Attachments: HBASE-11396-v1.patch
Recently I accidentally kill -9'd all regionservers in my cluster (don't
ask.. not my finest moment).
This let to some corruption of the meta table, causing the following
exception to be output on the HBase Master during startup, and culminating
with the Master aborting:
{noformat}
java.lang.IllegalArgumentException: Wrong length: 13, expected 8
at
org.apache.hadoop.hbase.util.Bytes.explainWrongLengthOrOffset(Bytes.java:600)
at org.apache.hadoop.hbase.util.Bytes.toLong(Bytes.java:578)
at
org.apache.hadoop.hbase.HRegionInfo.getServerName(HRegionInfo.java:1059)
at
org.apache.hadoop.hbase.HRegionInfo.getHRegionInfoAndServerName(HRegionInfo.java:987)
at
org.apache.hadoop.hbase.master.AssignmentManager.rebuildUserRegions(AssignmentManager.java:2678)
at
org.apache.hadoop.hbase.master.AssignmentManager.joinCluster(AssignmentManager.java:465)
at
org.apache.hadoop.hbase.master.HMaster.finishInitialization(HMaster.java:910)
at org.apache.hadoop.hbase.master.HMaster.run(HMaster.java:615)
at java.lang.Thread.run(Thread.java:744)
{noformat}
No tools that I am aware of were able to clean this up. I added a short patch
to catch and log this exception and return null from
HRegionInfo.getServerName(). This allowed for startup of the cluster, and
hbase hbck to repair the damage.
Creating ticket to submit patch in case anyone else finds this useful.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11102) Document JDK versions supported by each release
[ https://issues.apache.org/jira/browse/HBASE-11102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14042716#comment-14042716 ] Misty Stanley-Jones commented on HBASE-11102: - Fine with me. I'll make it a point of policy to refactor such things out as I go. Document JDK versions supported by each release --- Key: HBASE-11102 URL: https://issues.apache.org/jira/browse/HBASE-11102 Project: HBase Issue Type: Sub-task Components: documentation Reporter: Enis Soztutar Assignee: Misty Stanley-Jones Fix For: 0.99.0 Attachments: HBASE-11102-1.patch, HBASE-11102.patch We can make use of a JDK version x HBase version matrix to explain which JDK version is supported and required. 0.94, 0.96, and 0.98 releases all support JDK6 and JDK7. For 1.0, there is a discussion thread to decide whether to drop JDK6 support. There has been some work to support JDK8. We can also document that. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11352) When HMaster starts up it deletes the tmp snapshot directory, if you are exporting a snapshot at that time the job will fail
[ https://issues.apache.org/jira/browse/HBASE-11352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] churro morales updated HBASE-11352: --- Attachment: HBASE-11352-v2.0.94.patch Fixed the long log lines and updated the expiration time to hours. When HMaster starts up it deletes the tmp snapshot directory, if you are exporting a snapshot at that time the job will fail Key: HBASE-11352 URL: https://issues.apache.org/jira/browse/HBASE-11352 Project: HBase Issue Type: Bug Affects Versions: 0.94.19 Reporter: churro morales Attachments: HBASE-11352-0.94.patch, HBASE-11352-v2.0.94.patch We are exporting a very large table. The export snapshot job takes 7+ days to complete. During that time we had to bounce HMaster. When HMaster initializes, it initializes the SnapshotManager which subsequently deletes the .tmp directory. If this happens while the ExportSnapshot job is running the reference files get removed and the job fails. Maybe we could put some sort of token such that when this job is running HMaster wont reset the tmp directory. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-4495) CatalogTracker has an identity crisis; needs to be cut-back in scope
[
https://issues.apache.org/jira/browse/HBASE-4495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mikhail Antonov updated HBASE-4495:
---
Status: Open (was: Patch Available)
CatalogTracker has an identity crisis; needs to be cut-back in scope
Key: HBASE-4495
URL: https://issues.apache.org/jira/browse/HBASE-4495
Project: HBase
Issue Type: Improvement
Affects Versions: 0.94.0
Reporter: stack
Assignee: Mikhail Antonov
Attachments: HBASE-4495.patch, HBASE-4495.patch, HBASE-4495.patch
CT needs a good reworking. I'd suggest its scope be cut way down to only
deal in zk transactions rather than zk and reading meta location in hbase
(over an HConnection) and being a purveyor of HRegionInterfaces on meta and
root servers and being an Abortable and a verifier of catalog locations.
Once this is done, I would suggest it then better belongs over under the zk
package and that the Meta* classes then move to client package.
Here's some messy notes I added to head of CT class in hbase-3446 where I
spent some time trying to make out what it was CT did.
{code}
// TODO: This class needs a rethink. The original intent was that it would
be
// the one-stop-shop for root and meta locations and that it would get this
// info from reading and watching zk state. The class was to be used by
// servers when they needed to know of root and meta movement but also by
// client-side (inside in HTable) so rather than figure root and meta
// locations on fault, the client would instead get notifications out of zk.
//
// But this original intent is frustrated by the fact that this class has to
// read an hbase table, the -ROOT- table, to figure out the .META. region
// location which means we depend on an HConnection. HConnection will do
// retrying but also, it has its own mechanism for finding root and meta
// locations (and for 'verifying'; it tries the location and if it fails,
does
// new lookup, etc.). So, at least for now, HConnection (or HTable) can't
// have a CT since CT needs a HConnection (Even then, do want HT to have a
CT?
// For HT keep up a session with ZK? Rather, shouldn't we do like
asynchbase
// where we'd open a connection to zk, read what we need then let the
// connection go?). The 'fix' is make it so both root and meta addresses
// are wholey up in zk -- not in zk (root) -- and in an hbase table (meta).
//
// But even then, this class does 'verification' of the location and it does
// this by making a call over an HConnection (which will do its own root
// and meta lookups). Isn't this verification 'useless' since when we
// return, whatever is dependent on the result of this call then needs to
// use HConnection; what we have verified may change in meantime
(HConnection
// uses the CT primitives, the root and meta trackers finding root
locations).
//
// When meta is moved to zk, this class may make more sense. In the
// meantime, it does not cohere. It should just watch meta and root and
// NOT do verification -- let that be out in HConnection since its going to
// be done there ultimately anyways.
//
// This class has spread throughout the codebase. It needs to be reigned
in.
// This class should be used server-side only, even if we move meta location
// up into zk. Currently its used over in the client package. Its used in
// MetaReader and MetaEditor classes usually just to get the Configuration
// its using (It does this indirectly by asking its HConnection for its
// Configuration and even then this is just used to get an HConnection out
on
// the other end). St.Ack 10/23/2011.
//
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HBASE-4495) CatalogTracker has an identity crisis; needs to be cut-back in scope
[
https://issues.apache.org/jira/browse/HBASE-4495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mikhail Antonov updated HBASE-4495:
---
Status: Patch Available (was: Open)
CatalogTracker has an identity crisis; needs to be cut-back in scope
Key: HBASE-4495
URL: https://issues.apache.org/jira/browse/HBASE-4495
Project: HBase
Issue Type: Improvement
Affects Versions: 0.94.0
Reporter: stack
Assignee: Mikhail Antonov
Attachments: HBASE-4495.patch, HBASE-4495.patch, HBASE-4495.patch
CT needs a good reworking. I'd suggest its scope be cut way down to only
deal in zk transactions rather than zk and reading meta location in hbase
(over an HConnection) and being a purveyor of HRegionInterfaces on meta and
root servers and being an Abortable and a verifier of catalog locations.
Once this is done, I would suggest it then better belongs over under the zk
package and that the Meta* classes then move to client package.
Here's some messy notes I added to head of CT class in hbase-3446 where I
spent some time trying to make out what it was CT did.
{code}
// TODO: This class needs a rethink. The original intent was that it would
be
// the one-stop-shop for root and meta locations and that it would get this
// info from reading and watching zk state. The class was to be used by
// servers when they needed to know of root and meta movement but also by
// client-side (inside in HTable) so rather than figure root and meta
// locations on fault, the client would instead get notifications out of zk.
//
// But this original intent is frustrated by the fact that this class has to
// read an hbase table, the -ROOT- table, to figure out the .META. region
// location which means we depend on an HConnection. HConnection will do
// retrying but also, it has its own mechanism for finding root and meta
// locations (and for 'verifying'; it tries the location and if it fails,
does
// new lookup, etc.). So, at least for now, HConnection (or HTable) can't
// have a CT since CT needs a HConnection (Even then, do want HT to have a
CT?
// For HT keep up a session with ZK? Rather, shouldn't we do like
asynchbase
// where we'd open a connection to zk, read what we need then let the
// connection go?). The 'fix' is make it so both root and meta addresses
// are wholey up in zk -- not in zk (root) -- and in an hbase table (meta).
//
// But even then, this class does 'verification' of the location and it does
// this by making a call over an HConnection (which will do its own root
// and meta lookups). Isn't this verification 'useless' since when we
// return, whatever is dependent on the result of this call then needs to
// use HConnection; what we have verified may change in meantime
(HConnection
// uses the CT primitives, the root and meta trackers finding root
locations).
//
// When meta is moved to zk, this class may make more sense. In the
// meantime, it does not cohere. It should just watch meta and root and
// NOT do verification -- let that be out in HConnection since its going to
// be done there ultimately anyways.
//
// This class has spread throughout the codebase. It needs to be reigned
in.
// This class should be used server-side only, even if we move meta location
// up into zk. Currently its used over in the client package. Its used in
// MetaReader and MetaEditor classes usually just to get the Configuration
// its using (It does this indirectly by asking its HConnection for its
// Configuration and even then this is just used to get an HConnection out
on
// the other end). St.Ack 10/23/2011.
//
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11352) When HMaster starts up it deletes the tmp snapshot directory, if you are exporting a snapshot at that time the job will fail
[ https://issues.apache.org/jira/browse/HBASE-11352?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14042736#comment-14042736 ] Ted Yu commented on HBASE-11352: Can you attach patch for trunk ? Thanks When HMaster starts up it deletes the tmp snapshot directory, if you are exporting a snapshot at that time the job will fail Key: HBASE-11352 URL: https://issues.apache.org/jira/browse/HBASE-11352 Project: HBase Issue Type: Bug Affects Versions: 0.94.19 Reporter: churro morales Attachments: HBASE-11352-0.94.patch, HBASE-11352-v2.0.94.patch We are exporting a very large table. The export snapshot job takes 7+ days to complete. During that time we had to bounce HMaster. When HMaster initializes, it initializes the SnapshotManager which subsequently deletes the .tmp directory. If this happens while the ExportSnapshot job is running the reference files get removed and the job fails. Maybe we could put some sort of token such that when this job is running HMaster wont reset the tmp directory. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-4495) CatalogTracker has an identity crisis; needs to be cut-back in scope
[
https://issues.apache.org/jira/browse/HBASE-4495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mikhail Antonov updated HBASE-4495:
---
Attachment: HBASE-4495.patch
patch v2 for testing
CatalogTracker has an identity crisis; needs to be cut-back in scope
Key: HBASE-4495
URL: https://issues.apache.org/jira/browse/HBASE-4495
Project: HBase
Issue Type: Improvement
Affects Versions: 0.94.0
Reporter: stack
Assignee: Mikhail Antonov
Attachments: HBASE-4495.patch, HBASE-4495.patch, HBASE-4495.patch
CT needs a good reworking. I'd suggest its scope be cut way down to only
deal in zk transactions rather than zk and reading meta location in hbase
(over an HConnection) and being a purveyor of HRegionInterfaces on meta and
root servers and being an Abortable and a verifier of catalog locations.
Once this is done, I would suggest it then better belongs over under the zk
package and that the Meta* classes then move to client package.
Here's some messy notes I added to head of CT class in hbase-3446 where I
spent some time trying to make out what it was CT did.
{code}
// TODO: This class needs a rethink. The original intent was that it would
be
// the one-stop-shop for root and meta locations and that it would get this
// info from reading and watching zk state. The class was to be used by
// servers when they needed to know of root and meta movement but also by
// client-side (inside in HTable) so rather than figure root and meta
// locations on fault, the client would instead get notifications out of zk.
//
// But this original intent is frustrated by the fact that this class has to
// read an hbase table, the -ROOT- table, to figure out the .META. region
// location which means we depend on an HConnection. HConnection will do
// retrying but also, it has its own mechanism for finding root and meta
// locations (and for 'verifying'; it tries the location and if it fails,
does
// new lookup, etc.). So, at least for now, HConnection (or HTable) can't
// have a CT since CT needs a HConnection (Even then, do want HT to have a
CT?
// For HT keep up a session with ZK? Rather, shouldn't we do like
asynchbase
// where we'd open a connection to zk, read what we need then let the
// connection go?). The 'fix' is make it so both root and meta addresses
// are wholey up in zk -- not in zk (root) -- and in an hbase table (meta).
//
// But even then, this class does 'verification' of the location and it does
// this by making a call over an HConnection (which will do its own root
// and meta lookups). Isn't this verification 'useless' since when we
// return, whatever is dependent on the result of this call then needs to
// use HConnection; what we have verified may change in meantime
(HConnection
// uses the CT primitives, the root and meta trackers finding root
locations).
//
// When meta is moved to zk, this class may make more sense. In the
// meantime, it does not cohere. It should just watch meta and root and
// NOT do verification -- let that be out in HConnection since its going to
// be done there ultimately anyways.
//
// This class has spread throughout the codebase. It needs to be reigned
in.
// This class should be used server-side only, even if we move meta location
// up into zk. Currently its used over in the client package. Its used in
// MetaReader and MetaEditor classes usually just to get the Configuration
// its using (It does this indirectly by asking its HConnection for its
// Configuration and even then this is just used to get an HConnection out
on
// the other end). St.Ack 10/23/2011.
//
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11407) hbase-client should not require Jackson for pure HBase queries be executed
[ https://issues.apache.org/jira/browse/HBASE-11407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14042761#comment-14042761 ] Nicolas Liochon commented on HBASE-11407: - :-) If we remove the dependency from the pom w/o removing ot from the code, the tools around maven dependency will complain. I'm not a fan. Andrew, what do you think of just removing the feature in .99? hbase-client should not require Jackson for pure HBase queries be executed --- Key: HBASE-11407 URL: https://issues.apache.org/jira/browse/HBASE-11407 Project: HBase Issue Type: Improvement Components: Client Affects Versions: 0.98.3 Reporter: Sergey Beryozkin Assignee: Sergey Beryozkin Priority: Minor Fix For: 0.98.4 Attachments: diff.txt Including the hbase-client module dependency and excluding Jackson dependencies causes the pure HBase query (run with HTableInterface) fail with Jackson ObjectMapper ClassNotFoundException. This is due to org.apache.hadoop.hbase.client.Operation having ObjectMapper statically initialized. Moving ObjectMapper to a dedicated utility will help. The patch will be attached. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11407) hbase-client should not require Jackson for pure HBase queries be executed
[ https://issues.apache.org/jira/browse/HBASE-11407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14042777#comment-14042777 ] stack commented on HBASE-11407: --- @nkeywal toJSON on all Objects? Its not used any more as you note. Kinda nice to have but if needed, if we just make the Operations implement beans, we can provide JSON with an external util class. I'd be good stripping it from this base, important class that is shared client and server-side. hbase-client should not require Jackson for pure HBase queries be executed --- Key: HBASE-11407 URL: https://issues.apache.org/jira/browse/HBASE-11407 Project: HBase Issue Type: Improvement Components: Client Affects Versions: 0.98.3 Reporter: Sergey Beryozkin Assignee: Sergey Beryozkin Priority: Minor Fix For: 0.98.4 Attachments: diff.txt Including the hbase-client module dependency and excluding Jackson dependencies causes the pure HBase query (run with HTableInterface) fail with Jackson ObjectMapper ClassNotFoundException. This is due to org.apache.hadoop.hbase.client.Operation having ObjectMapper statically initialized. Moving ObjectMapper to a dedicated utility will help. The patch will be attached. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11396) Invalid meta entries can lead to unstartable master
[
https://issues.apache.org/jira/browse/HBASE-11396?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-11396:
---
Resolution: Fixed
Fix Version/s: 0.98.4
0.99.0
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
Integrated to 0.98 and trunk.
Thanks for the patch, Craig.
Invalid meta entries can lead to unstartable master
---
Key: HBASE-11396
URL: https://issues.apache.org/jira/browse/HBASE-11396
Project: HBase
Issue Type: Bug
Components: master
Affects Versions: 0.98.3
Reporter: Craig Condit
Fix For: 0.99.0, 0.98.4
Attachments: HBASE-11396-v1.patch
Recently I accidentally kill -9'd all regionservers in my cluster (don't
ask.. not my finest moment).
This let to some corruption of the meta table, causing the following
exception to be output on the HBase Master during startup, and culminating
with the Master aborting:
{noformat}
java.lang.IllegalArgumentException: Wrong length: 13, expected 8
at
org.apache.hadoop.hbase.util.Bytes.explainWrongLengthOrOffset(Bytes.java:600)
at org.apache.hadoop.hbase.util.Bytes.toLong(Bytes.java:578)
at
org.apache.hadoop.hbase.HRegionInfo.getServerName(HRegionInfo.java:1059)
at
org.apache.hadoop.hbase.HRegionInfo.getHRegionInfoAndServerName(HRegionInfo.java:987)
at
org.apache.hadoop.hbase.master.AssignmentManager.rebuildUserRegions(AssignmentManager.java:2678)
at
org.apache.hadoop.hbase.master.AssignmentManager.joinCluster(AssignmentManager.java:465)
at
org.apache.hadoop.hbase.master.HMaster.finishInitialization(HMaster.java:910)
at org.apache.hadoop.hbase.master.HMaster.run(HMaster.java:615)
at java.lang.Thread.run(Thread.java:744)
{noformat}
No tools that I am aware of were able to clean this up. I added a short patch
to catch and log this exception and return null from
HRegionInfo.getServerName(). This allowed for startup of the cluster, and
hbase hbck to repair the damage.
Creating ticket to submit patch in case anyone else finds this useful.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-7608) Considering JDK8
[ https://issues.apache.org/jira/browse/HBASE-7608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14042825#comment-14042825 ] Misty Stanley-Jones commented on HBASE-7608: Wrong JIRA. Sorry. Considering JDK8 Key: HBASE-7608 URL: https://issues.apache.org/jira/browse/HBASE-7608 Project: HBase Issue Type: Umbrella Reporter: Andrew Purtell Priority: Trivial Musings (as subtasks) on experimental ideas for when JRE8 is a viable runtime. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-7608) Considering JDK8
[ https://issues.apache.org/jira/browse/HBASE-7608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misty Stanley-Jones updated HBASE-7608: --- Attachment: HBASE-11102-2.patch Removed mention of older versions and updated info for JDK 6 and 8 for 1.0. Considering JDK8 Key: HBASE-7608 URL: https://issues.apache.org/jira/browse/HBASE-7608 Project: HBase Issue Type: Umbrella Reporter: Andrew Purtell Priority: Trivial Musings (as subtasks) on experimental ideas for when JRE8 is a viable runtime. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-7608) Considering JDK8
[ https://issues.apache.org/jira/browse/HBASE-7608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misty Stanley-Jones updated HBASE-7608: --- Attachment: (was: HBASE-11102-2.patch) Considering JDK8 Key: HBASE-7608 URL: https://issues.apache.org/jira/browse/HBASE-7608 Project: HBase Issue Type: Umbrella Reporter: Andrew Purtell Priority: Trivial Musings (as subtasks) on experimental ideas for when JRE8 is a viable runtime. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11102) Document JDK versions supported by each release
[ https://issues.apache.org/jira/browse/HBASE-11102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misty Stanley-Jones updated HBASE-11102: Attachment: HBASE-11102-2.patch Removed mention of older releases than 0.94.x and updated info about JDK 6 and 8 for 1.0. Document JDK versions supported by each release --- Key: HBASE-11102 URL: https://issues.apache.org/jira/browse/HBASE-11102 Project: HBase Issue Type: Sub-task Components: documentation Reporter: Enis Soztutar Assignee: Misty Stanley-Jones Fix For: 0.99.0 Attachments: HBASE-11102-1.patch, HBASE-11102-2.patch, HBASE-11102.patch We can make use of a JDK version x HBase version matrix to explain which JDK version is supported and required. 0.94, 0.96, and 0.98 releases all support JDK6 and JDK7. For 1.0, there is a discussion thread to decide whether to drop JDK6 support. There has been some work to support JDK8. We can also document that. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-4495) CatalogTracker has an identity crisis; needs to be cut-back in scope
[
https://issues.apache.org/jira/browse/HBASE-4495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14042836#comment-14042836
]
Hadoop QA commented on HBASE-4495:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12652295/HBASE-4495.patch
against trunk revision .
ATTACHMENT ID: 12652295
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 126
new or modified tests.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 3
warning messages.
{color:red}-1 findbugs{color}. The patch appears to introduce 2 new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 lineLengths{color}. The patch introduces the following lines
longer than 100:
+#org.apache.hadoop.hbase.mapreduce.TestTableInputFormatScan,org.apache.hadoop.hbase.catalog.TestMetaTableAccessorNoCluster,org.apache.hadoop.hbase.catalog.TestMetaTableAccessor,org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat,org.apache.hadoop.hbase.mapred.TestTableMapReduce,org.apache.hadoop.hbase.coprocessor.TestMasterCoprocessorExceptionWithAbort,org.apache.hadoop.hbase.coprocessor.TestMasterCoprocessorExceptionWithRemove,org.apache.hadoop.hbase.client.TestAdmin,org.apache.hadoop.hbase.master.TestMasterFailover,org.apache.hadoop.hbase.regionserver.wal.TestLogRolling,org.apache.hadoop.hbase.master.TestDistributedLogSplitting,org.apache.hadoop.hbase.master.TestMasterRestartAfterDisablingTable,org.apache.hadoop.hbase.TestGlobalMemStoreSize,
+ public static int getRegionCount(final Configuration c, final String
tableName) throws IOException {
+ * Adds a (single) hbase:meta row for the specified new region and its
daughters. Note that this does
+ * Adds a (single) hbase:meta row for the specified new region and its
daughters. Note that this does
+ final HRegionInfo regionInfo, final
ServerName sn, final long openSeqNum)
+ HRegionInfo mergedRegion, HRegionInfo
regionA, HRegionInfo regionB,
+ private static void multiMutate(HTable table, byte[] row, Mutation...
mutations) throws IOException {
+MultiRowMutationProtos.MutateRowsRequest.Builder mmrBuilder =
MultiRowMutationProtos.MutateRowsRequest.newBuilder();
+
mmrBuilder.addMutationRequest(ProtobufUtil.toMutation(ClientProtos.MutationProto.MutationType.PUT,
mutation));
+
mmrBuilder.addMutationRequest(ProtobufUtil.toMutation(ClientProtos.MutationProto.MutationType.DELETE,
mutation));
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.regionserver.TestHRegion
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/9838//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9838//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9838//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9838//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9838//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9838//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9838//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9838//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9838//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9838//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/9838//console
This message is automatically generated.
CatalogTracker has an identity crisis; needs to be cut-back in scope
[jira] [Commented] (HBASE-10885) Support visibility expressions on Deletes
[ https://issues.apache.org/jira/browse/HBASE-10885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14042845#comment-14042845 ] Andrew Purtell commented on HBASE-10885: Sorry Ram. Anoop added good comments. I added a couple complimentary ones. Support visibility expressions on Deletes - Key: HBASE-10885 URL: https://issues.apache.org/jira/browse/HBASE-10885 Project: HBase Issue Type: Improvement Affects Versions: 0.98.1 Reporter: Andrew Purtell Assignee: ramkrishna.s.vasudevan Priority: Blocker Fix For: 0.99.0, 0.98.4 Attachments: 10885-org.apache.hadoop.hbase.security.visibility.TestVisibilityLabelsWithDeletes-output.txt, HBASE-10885_1.patch, HBASE-10885_2.patch, HBASE-10885_new_tag_type_1.patch, HBASE-10885_new_tag_type_2.patch, HBASE-10885_v1.patch, HBASE-10885_v2.patch, HBASE-10885_v2.patch, HBASE-10885_v2.patch, HBASE-10885_v3.patch, HBASE-10885_v4.patch, HBASE-10885_v5.patch, HBASE-10885_v7.patch Accumulo can specify visibility expressions for delete markers. During compaction the cells covered by the tombstone are determined in part by matching the visibility expression. This is useful for the use case of data set coalescing, where entries from multiple data sets carrying different labels are combined into one common large table. Later, a subset of entries can be conveniently removed using visibility expressions. Currently doing the same in HBase would only be possible with a custom coprocessor. Otherwise, a Delete will affect all cells covered by the tombstone regardless of any visibility expression scoping. This is correct behavior in that no data spill is possible, but certainly could be surprising, and is only meant to be transitional. We decided not to support visibility expressions on Deletes to control the complexity of the initial implementation. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11403) Fix race conditions around Object#notify
[ https://issues.apache.org/jira/browse/HBASE-11403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14042857#comment-14042857 ] Andrew Purtell commented on HBASE-11403: Belated +1, lgtm Fix race conditions around Object#notify Key: HBASE-11403 URL: https://issues.apache.org/jira/browse/HBASE-11403 Project: HBase Issue Type: Bug Components: Client, regionserver Affects Versions: 0.99.0, 0.98.3 Reporter: Nicolas Liochon Assignee: Nicolas Liochon Fix For: 0.99.0, 0.98.4 Attachments: 11403.98.v1.patch, 11403.v1.patch We do have some race conditions there. We don't see them fail in the unit tests, because our #wait are bounded. But from a performance point of view, they do occur. I've reviewed them and fix all the issue I found excepted in the AM (haven't reviewed this one, may be it's fine). On a perf test, this seems to improve the max latency. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11407) hbase-client should not require Jackson for pure HBase queries be executed
[ https://issues.apache.org/jira/browse/HBASE-11407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14042862#comment-14042862 ] Andrew Purtell commented on HBASE-11407: I agree with Stack hbase-client should not require Jackson for pure HBase queries be executed --- Key: HBASE-11407 URL: https://issues.apache.org/jira/browse/HBASE-11407 Project: HBase Issue Type: Improvement Components: Client Affects Versions: 0.98.3 Reporter: Sergey Beryozkin Assignee: Sergey Beryozkin Priority: Minor Fix For: 0.98.4 Attachments: diff.txt Including the hbase-client module dependency and excluding Jackson dependencies causes the pure HBase query (run with HTableInterface) fail with Jackson ObjectMapper ClassNotFoundException. This is due to org.apache.hadoop.hbase.client.Operation having ObjectMapper statically initialized. Moving ObjectMapper to a dedicated utility will help. The patch will be attached. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11405) Multiple invocations of hbck in parallel disables balancer permanently
[
https://issues.apache.org/jira/browse/HBASE-11405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14042864#comment-14042864
]
Andrew Purtell commented on HBASE-11405:
+1, would be a better fix to test in HBCK startup if another is running and
error out if so
Multiple invocations of hbck in parallel disables balancer permanently
---
Key: HBASE-11405
URL: https://issues.apache.org/jira/browse/HBASE-11405
Project: HBase
Issue Type: Bug
Components: Balancer, hbck
Affects Versions: 0.99.0
Reporter: bharath v
This is because of the following piece of code in hbck
{code:borderStyle=solid}
boolean oldBalancer = admin.setBalancerRunning(false, true);
try {
onlineConsistencyRepair();
}
finally {
admin.setBalancerRunning(oldBalancer, false);
}
{code}
Newer invocations set oldBalancer to false as it was disabled by previous
invocations and this disables balancer permanently unless its manually turned
on by the user. Easy to reproduce, just run hbck 100 times in a loop in 2
different sessions and you can see that balancer is set to false in the
HMaster logs.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11406) ExportSnapshot#run better catch Throwable for ExportSnapshot#runCopyJob
[ https://issues.apache.org/jira/browse/HBASE-11406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14042863#comment-14042863 ] Andrew Purtell commented on HBASE-11406: Not much we can do if you are using HBase built for one version of Hadoop on a Hadoop of a different version though. ExportSnapshot#run better catch Throwable for ExportSnapshot#runCopyJob --- Key: HBASE-11406 URL: https://issues.apache.org/jira/browse/HBASE-11406 Project: HBase Issue Type: Improvement Components: snapshots Reporter: Qiang Tian Assignee: Qiang Tian Priority: Trivial Attachments: hbase11406-master.patch ExportSnapshot#run better catch Throwable for ExportSnapshot#runCopyJob since runCopyJob involves hadoop and mapreduce. other errors could be thrown, catching them like below at first place is helpful for error handling and problem identification. 14/06/24 18:39:30 ERROR snapshot.ExportSnapshot: Snapshot export failed java.lang.NoSuchMethodError: org/apache/hadoop/mapred/JobTracker.getAddress(Lorg/apache/hadoop/conf/Configuration;)Ljava/net/InetSocketAddress; at org.apache.hadoop.mapred.JobTrackerClientProtocolProvider.create(JobTrackerClientProtocolProvider.java:44) at org.apache.hadoop.mapreduce.Cluster.initialize(Cluster.java:95) at org.apache.hadoop.mapreduce.Cluster.init(Cluster.java:82) at org.apache.hadoop.mapreduce.Cluster.init(Cluster.java:75) at org.apache.hadoop.hbase.mapreduce.JobUtil.getStagingDir(JobUtil.java:54) at org.apache.hadoop.hbase.snapshot.ExportSnapshot.getInputFolderPath(ExportSnapshot.java:541) at org.apache.hadoop.hbase.snapshot.ExportSnapshot.runCopyJob(ExportSnapshot.java:609) at org.apache.hadoop.hbase.snapshot.ExportSnapshot.run(ExportSnapshot.java:776) at org.apache.hadoop.hbase.backup.BackupCopier.copy(BackupCopier.java:248) at org.apache.hadoop.hbase.backup.BackupHandler.snapshotCopy(BackupHandler.java:559) at org.apache.hadoop.hbase.backup.BackupHandler.call(BackupHandler.java:141) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:314) at java.util.concurrent.FutureTask.run(FutureTask.java:149) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:897) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:919) at java.lang.Thread.run(Thread.java:738) -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11407) hbase-client should not require Jackson for pure HBase queries be executed
[ https://issues.apache.org/jira/browse/HBASE-11407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14042869#comment-14042869 ] stack commented on HBASE-11407: --- To finish, I'd be good w/ committing [~sergey_beryozkin] patch for now and doing the JSON stripping in another patch. hbase-client should not require Jackson for pure HBase queries be executed --- Key: HBASE-11407 URL: https://issues.apache.org/jira/browse/HBASE-11407 Project: HBase Issue Type: Improvement Components: Client Affects Versions: 0.98.3 Reporter: Sergey Beryozkin Assignee: Sergey Beryozkin Priority: Minor Fix For: 0.98.4 Attachments: diff.txt Including the hbase-client module dependency and excluding Jackson dependencies causes the pure HBase query (run with HTableInterface) fail with Jackson ObjectMapper ClassNotFoundException. This is due to org.apache.hadoop.hbase.client.Operation having ObjectMapper statically initialized. Moving ObjectMapper to a dedicated utility will help. The patch will be attached. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-4495) CatalogTracker has an identity crisis; needs to be cut-back in scope
[
https://issues.apache.org/jira/browse/HBASE-4495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-4495:
-
Attachment: HBASE-4495.patch
Retry to see if the TestHRegion failure 'real'.
CatalogTracker has an identity crisis; needs to be cut-back in scope
Key: HBASE-4495
URL: https://issues.apache.org/jira/browse/HBASE-4495
Project: HBase
Issue Type: Improvement
Affects Versions: 0.94.0
Reporter: stack
Assignee: Mikhail Antonov
Attachments: HBASE-4495.patch, HBASE-4495.patch, HBASE-4495.patch,
HBASE-4495.patch
CT needs a good reworking. I'd suggest its scope be cut way down to only
deal in zk transactions rather than zk and reading meta location in hbase
(over an HConnection) and being a purveyor of HRegionInterfaces on meta and
root servers and being an Abortable and a verifier of catalog locations.
Once this is done, I would suggest it then better belongs over under the zk
package and that the Meta* classes then move to client package.
Here's some messy notes I added to head of CT class in hbase-3446 where I
spent some time trying to make out what it was CT did.
{code}
// TODO: This class needs a rethink. The original intent was that it would
be
// the one-stop-shop for root and meta locations and that it would get this
// info from reading and watching zk state. The class was to be used by
// servers when they needed to know of root and meta movement but also by
// client-side (inside in HTable) so rather than figure root and meta
// locations on fault, the client would instead get notifications out of zk.
//
// But this original intent is frustrated by the fact that this class has to
// read an hbase table, the -ROOT- table, to figure out the .META. region
// location which means we depend on an HConnection. HConnection will do
// retrying but also, it has its own mechanism for finding root and meta
// locations (and for 'verifying'; it tries the location and if it fails,
does
// new lookup, etc.). So, at least for now, HConnection (or HTable) can't
// have a CT since CT needs a HConnection (Even then, do want HT to have a
CT?
// For HT keep up a session with ZK? Rather, shouldn't we do like
asynchbase
// where we'd open a connection to zk, read what we need then let the
// connection go?). The 'fix' is make it so both root and meta addresses
// are wholey up in zk -- not in zk (root) -- and in an hbase table (meta).
//
// But even then, this class does 'verification' of the location and it does
// this by making a call over an HConnection (which will do its own root
// and meta lookups). Isn't this verification 'useless' since when we
// return, whatever is dependent on the result of this call then needs to
// use HConnection; what we have verified may change in meantime
(HConnection
// uses the CT primitives, the root and meta trackers finding root
locations).
//
// When meta is moved to zk, this class may make more sense. In the
// meantime, it does not cohere. It should just watch meta and root and
// NOT do verification -- let that be out in HConnection since its going to
// be done there ultimately anyways.
//
// This class has spread throughout the codebase. It needs to be reigned
in.
// This class should be used server-side only, even if we move meta location
// up into zk. Currently its used over in the client package. Its used in
// MetaReader and MetaEditor classes usually just to get the Configuration
// its using (It does this indirectly by asking its HConnection for its
// Configuration and even then this is just used to get an HConnection out
on
// the other end). St.Ack 10/23/2011.
//
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11370) SSH doesn't need to scan meta if not using ZK for assignment
[ https://issues.apache.org/jira/browse/HBASE-11370?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14042872#comment-14042872 ] Enis Soztutar commented on HBASE-11370: --- Sorry, I was reading the source for RegionStateStore, and totally confused for why we need both sn and server name columns. It will be very confusing to have both of these column names as it is. Not in every case they are equal? if we cannot merge those, should we at least change the name from sn to server_for_rit or smt like that? SSH doesn't need to scan meta if not using ZK for assignment Key: HBASE-11370 URL: https://issues.apache.org/jira/browse/HBASE-11370 Project: HBase Issue Type: Improvement Reporter: Jimmy Xiang Assignee: Jimmy Xiang Priority: Minor Fix For: 0.99.0 Attachments: hbase-11370.patch If we don't use ZK for assignment, the meta content should be the same as that in memory. So we should be able to avoid a meta scan. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11102) Document JDK versions supported by each release
[ https://issues.apache.org/jira/browse/HBASE-11102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-11102: -- Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Committed to master. Thanks Misty. Document JDK versions supported by each release --- Key: HBASE-11102 URL: https://issues.apache.org/jira/browse/HBASE-11102 Project: HBase Issue Type: Sub-task Components: documentation Reporter: Enis Soztutar Assignee: Misty Stanley-Jones Fix For: 0.99.0 Attachments: HBASE-11102-1.patch, HBASE-11102-2.patch, HBASE-11102.patch We can make use of a JDK version x HBase version matrix to explain which JDK version is supported and required. 0.94, 0.96, and 0.98 releases all support JDK6 and JDK7. For 1.0, there is a discussion thread to decide whether to drop JDK6 support. There has been some work to support JDK8. We can also document that. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11370) SSH doesn't need to scan meta if not using ZK for assignment
[ https://issues.apache.org/jira/browse/HBASE-11370?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14042878#comment-14042878 ] stack commented on HBASE-11370: --- I like what Enis is saying. A name that makes the relationship between the two columns more clear is a good idea. SSH doesn't need to scan meta if not using ZK for assignment Key: HBASE-11370 URL: https://issues.apache.org/jira/browse/HBASE-11370 Project: HBase Issue Type: Improvement Reporter: Jimmy Xiang Assignee: Jimmy Xiang Priority: Minor Fix For: 0.99.0 Attachments: hbase-11370.patch If we don't use ZK for assignment, the meta content should be the same as that in memory. So we should be able to avoid a meta scan. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11363) Access checks in preCompact and preCompactSelection are out of sync
[ https://issues.apache.org/jira/browse/HBASE-11363?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-11363: --- Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Access checks in preCompact and preCompactSelection are out of sync --- Key: HBASE-11363 URL: https://issues.apache.org/jira/browse/HBASE-11363 Project: HBase Issue Type: Bug Affects Versions: 0.98.3 Reporter: Andrew Purtell Assignee: Andrew Purtell Fix For: 0.99.0, 0.98.4 Attachments: HBASE-11363.patch As discussed on HBASE-6192, it looks like someone cut and pasted the access check from preCompact into preCompactSelection at one time and, later, another change was made that relaxed permissions for compaction requests from ADMIN to ADMIN|CREATE. We do not need an access check in preCompactSelection since a request to compact is already mediated by preCompact. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (HBASE-11409) Add more flexibility for input directory structure to LoadIncrementalHFiles
churro morales created HBASE-11409: -- Summary: Add more flexibility for input directory structure to LoadIncrementalHFiles Key: HBASE-11409 URL: https://issues.apache.org/jira/browse/HBASE-11409 Project: HBase Issue Type: Bug Affects Versions: 0.94.20 Reporter: churro morales Use case: We were trying to combine two very large tables into a single table. Thus we ran jobs in one datacenter that populated certain column families and another datacenter which populated other column families. Took a snapshot and exported them to their respective datacenters. Wanted to simply take the hdfs restored snapshot and use LoadIncremental to merge the data. It would be nice to add support where we could run LoadIncremental on a directory where the depth of store files is something other than two (current behavior). With snapshots it would be nice if you could pass a restored hdfs snapshot's directory and have the tool run. I am attaching a patch where I parameterize the bulkLoad timeout as well as the default store file depth. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11409) Add more flexibility for input directory structure to LoadIncrementalHFiles
[ https://issues.apache.org/jira/browse/HBASE-11409?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] churro morales updated HBASE-11409: --- Attachment: HBASE-11409-0.94.patch Add more flexibility for input directory structure to LoadIncrementalHFiles --- Key: HBASE-11409 URL: https://issues.apache.org/jira/browse/HBASE-11409 Project: HBase Issue Type: Bug Affects Versions: 0.94.20 Reporter: churro morales Attachments: HBASE-11409-0.94.patch Use case: We were trying to combine two very large tables into a single table. Thus we ran jobs in one datacenter that populated certain column families and another datacenter which populated other column families. Took a snapshot and exported them to their respective datacenters. Wanted to simply take the hdfs restored snapshot and use LoadIncremental to merge the data. It would be nice to add support where we could run LoadIncremental on a directory where the depth of store files is something other than two (current behavior). With snapshots it would be nice if you could pass a restored hdfs snapshot's directory and have the tool run. I am attaching a patch where I parameterize the bulkLoad timeout as well as the default store file depth. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11324) Update 2.5.2.8. Managed Compactions
[ https://issues.apache.org/jira/browse/HBASE-11324?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-11324: -- Resolution: Fixed Fix Version/s: 0.99.0 Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Committed for LarsG. Thanks Misty. Update 2.5.2.8. Managed Compactions --- Key: HBASE-11324 URL: https://issues.apache.org/jira/browse/HBASE-11324 Project: HBase Issue Type: Bug Components: Compaction, documentation Reporter: Misty Stanley-Jones Assignee: Misty Stanley-Jones Fix For: 0.99.0 Attachments: HBASE-11324.patch 2.5.2.8. Managed Compactions This says that managed compactions happen every day but that is no longer accurate. Yank this section and point them to the compaction section instead I think. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11370) SSH doesn't need to scan meta if not using ZK for assignment
[ https://issues.apache.org/jira/browse/HBASE-11370?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14042890#comment-14042890 ] Jimmy Xiang commented on HBASE-11370: - How about rit_sn to be shorter? SSH doesn't need to scan meta if not using ZK for assignment Key: HBASE-11370 URL: https://issues.apache.org/jira/browse/HBASE-11370 Project: HBase Issue Type: Improvement Reporter: Jimmy Xiang Assignee: Jimmy Xiang Priority: Minor Fix For: 0.99.0 Attachments: hbase-11370.patch If we don't use ZK for assignment, the meta content should be the same as that in memory. So we should be able to avoid a meta scan. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11088) Support Visibility Expression Deletes in Shell
[ https://issues.apache.org/jira/browse/HBASE-11088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14042892#comment-14042892 ] Andrew Purtell commented on HBASE-11088: Let's do this without adding new commands. Support Visibility Expression Deletes in Shell -- Key: HBASE-11088 URL: https://issues.apache.org/jira/browse/HBASE-11088 Project: HBase Issue Type: Sub-task Reporter: ramkrishna.s.vasudevan Assignee: ramkrishna.s.vasudevan Priority: Blocker Fix For: 0.99.0, 0.98.4 Attachments: HBASE-11058.patch, HBASE-11058_2.patch, HBASE-11088_3.patch -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11384) [Visibility Controller]Check for users covering authorizations for every mutation
[ https://issues.apache.org/jira/browse/HBASE-11384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-11384: --- Fix Version/s: (was: 0.98.4) 0.98.5 [Visibility Controller]Check for users covering authorizations for every mutation - Key: HBASE-11384 URL: https://issues.apache.org/jira/browse/HBASE-11384 Project: HBase Issue Type: Sub-task Affects Versions: 0.98.3 Reporter: ramkrishna.s.vasudevan Assignee: ramkrishna.s.vasudevan Fix For: 0.99.0, 0.98.5 As part of discussions, it is better that every mutation either Put/Delete with Visibility expressions should validate if the expression has labels for which the user has authorization. If not fail the mutation. Suppose User A is assoicated with A,B and C. The put has a visibility expression AD. Then fail the mutation as D is not associated with User A. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11370) SSH doesn't need to scan meta if not using ZK for assignment
[
https://issues.apache.org/jira/browse/HBASE-11370?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14042891#comment-14042891
]
Jeffrey Zhong commented on HBASE-11370:
---
The following seems not right to me or the name are confusing.
{code}
+ if (lastHost != null newState != State.SPLIT) {
+lastAssignments.put(encodedName, lastHost);
+regionAssignments.put(hri, lastHost);
{code}
I think regionAssignments should only be updated for State.Open not for all
other cases.
SSH doesn't need to scan meta if not using ZK for assignment
Key: HBASE-11370
URL: https://issues.apache.org/jira/browse/HBASE-11370
Project: HBase
Issue Type: Improvement
Reporter: Jimmy Xiang
Assignee: Jimmy Xiang
Priority: Minor
Fix For: 0.99.0
Attachments: hbase-11370.patch
If we don't use ZK for assignment, the meta content should be the same as
that in memory. So we should be able to avoid a meta scan.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11370) SSH doesn't need to scan meta if not using ZK for assignment
[ https://issues.apache.org/jira/browse/HBASE-11370?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14042893#comment-14042893 ] Enis Soztutar commented on HBASE-11370: --- bq. How about rit_sn to be shorter? rit_sn sounds good. Can you also comment on whether we actually need two different columns for RIT and client interactions. If we had one, the client would more quickly discover the newly assigned region. SSH doesn't need to scan meta if not using ZK for assignment Key: HBASE-11370 URL: https://issues.apache.org/jira/browse/HBASE-11370 Project: HBase Issue Type: Improvement Reporter: Jimmy Xiang Assignee: Jimmy Xiang Priority: Minor Fix For: 0.99.0 Attachments: hbase-11370.patch If we don't use ZK for assignment, the meta content should be the same as that in memory. So we should be able to avoid a meta scan. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Resolved] (HBASE-11385) [Visibility Controller]Check covering permission of the user issuing Delete for deletes without Cell Visibility
[ https://issues.apache.org/jira/browse/HBASE-11385?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell resolved HBASE-11385. Resolution: Duplicate Fix Version/s: (was: 0.98.4) (was: 0.99.0) Assignee: (was: ramkrishna.s.vasudevan) Deletes without visibility expressions should match only cells without visibility labels. The work here ends up being equivalent to HBASE-11384, so closing as dup. [Visibility Controller]Check covering permission of the user issuing Delete for deletes without Cell Visibility --- Key: HBASE-11385 URL: https://issues.apache.org/jira/browse/HBASE-11385 Project: HBase Issue Type: Sub-task Affects Versions: 0.98.3 Reporter: ramkrishna.s.vasudevan In cases where delete does have the Cell visibility passed for exact label matching, then check the user issuing delete and use the user's authorizations to find out the covering cells. Will prepare a patch once the base issue gets checked in. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11391) Thrift table creation will fail with default TTL with sanity checks
[ https://issues.apache.org/jira/browse/HBASE-11391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14042900#comment-14042900 ] Andrew Purtell commented on HBASE-11391: +1 Thrift table creation will fail with default TTL with sanity checks --- Key: HBASE-11391 URL: https://issues.apache.org/jira/browse/HBASE-11391 Project: HBase Issue Type: Bug Reporter: Enis Soztutar Assignee: Enis Soztutar Fix For: 0.99.0, 0.98.4 Attachments: hbae-11391_v1.patch Creating a table from thrift without explicitly supplying TTL fails because of the new checks introduced in HBASE-10591. -- This message was sent by Atlassian JIRA (v6.2#6252)
