[jira] [Assigned] (HBASE-9113) Expose region statistics on table.jsp
[ https://issues.apache.org/jira/browse/HBASE-9113?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] samar reassigned HBASE-9113: Assignee: samar > Expose region statistics on table.jsp > - > > Key: HBASE-9113 > URL: https://issues.apache.org/jira/browse/HBASE-9113 > Project: HBase > Issue Type: New Feature > Components: Admin, UI >Reporter: Bryan Beaudreault >Assignee: samar >Priority: Minor > > While Hannibal (https://github.com/sentric/hannibal) is great, the goal > should be to eventually make it obsolete by providing the same features in > the main HBase web UI (and HBaseAdmin API). > The first step for that is region statistics on the table.jsp. Please > provide the same statistics per-region on table.jsp as in rs-status.jsp. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8949) hbase.mapreduce.hfileoutputformat.blocksize should configure with blocksize of a table
[ https://issues.apache.org/jira/browse/HBASE-8949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13727407#comment-13727407 ] Lars Hofhansl commented on HBASE-8949: -- If you edit this jira you'll find the "Release Note" field. I'm happy to write the release note, too. > hbase.mapreduce.hfileoutputformat.blocksize should configure with blocksize > of a table > -- > > Key: HBASE-8949 > URL: https://issues.apache.org/jira/browse/HBASE-8949 > Project: HBase > Issue Type: Bug > Components: mapreduce >Reporter: rajeshbabu >Assignee: rajeshbabu > Fix For: 0.98.0, 0.95.2, 0.94.11 > > Attachments: HBASE-8949_94_2.patch, HBASE-8949_94.patch, > HBASE-8949_trunk_2.patch, HBASE-8949_trunk.patch > > > While initializing mapreduce job we are not configuring > hbase.mapreduce.hfileoutputformat.blocksize, so hfiles are always creating > with 64kb (default)block size even though tables has different block size. > We need to configure it with block size from table descriptor. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9023) TestIOFencing.testFencingAroundCompactionAfterWALSync
[
https://issues.apache.org/jira/browse/HBASE-9023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13727399#comment-13727399
]
stack commented on HBASE-9023:
--
Failed this evening:
https://builds.apache.org/job/HBase-TRUNK/4331/testReport/org.apache.hadoop.hbase/TestIOFencing/testFencingAroundCompactionAfterWALSync/
> TestIOFencing.testFencingAroundCompactionAfterWALSync
> -
>
> Key: HBASE-9023
> URL: https://issues.apache.org/jira/browse/HBASE-9023
> Project: HBase
> Issue Type: Bug
> Components: test
>Reporter: stack
> Fix For: 0.95.2
>
>
> Any one want to take a look at this one?
> https://builds.apache.org/job/HBase-TRUNK/4283/testReport/org.apache.hadoop.hbase/TestIOFencing/testFencingAroundCompactionAfterWALSync/
> {code}
> java.lang.AssertionError
> at org.junit.Assert.fail(Assert.java:86)
> at org.junit.Assert.assertTrue(Assert.java:41)
> at org.junit.Assert.assertTrue(Assert.java:52)
> at org.apache.hadoop.hbase.TestIOFencing.doTest(TestIOFencing.java:263)
> at
> org.apache.hadoop.hbase.TestIOFencing.testFencingAroundCompactionAfterWALSync(TestIOFencing.java:217)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
> at org.junit.runners.Suite.runChild(Suite.java:127)
> at org.junit.runners.Suite.runChild(Suite.java:26)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
> at java.lang.Thread.run(Thread.java:662)
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8408) Implement namespace
[
https://issues.apache.org/jira/browse/HBASE-8408?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13727398#comment-13727398
]
stack commented on HBASE-8408:
--
I was having trouble up in rb loading page #3 so moved to raw patch. Here is
some feedback up to TableSplit:
{code}
+ hbase-common/src/main/java/org/apache/hadoop/hbase/HConstants.java
Good you upped the fs version number.
I wonder if the tablename defines don't better belong in the new TableName
class.
On the name of the datadir being .data, is that right? Did we talk about
changing the names of dirs after ns goes in to remove the '.' ? Would that be
after
this patch?
+ hbase-common/src/main/java/org/apache/hadoop/hbase/KeyValue.java
So the system ns is called 'hbase' and not 'system'?
HConstants.META_TABLE_NAME.getName().length; // 'hbase.meta' length
+ hbase-common/src/main/java/org/apache/hadoop/hbase/NamespaceDescriptor.java
Looks good. When we list them, they will look like an HTableDescriptor listing
in
the ruby map format Thats consistent (if ugly but ugly ain't your fault)
+ hbase-common/src/main/java/org/apache/hadoop/hbase/TableName.java
I suppose it is ok haveing NS and TN in hbase-common. They have no dependency
on anything else.
What is a qualifier name in TN? A ns? Or it looks like it is the old
tablename? Should say
in javadoc since can confuse
TN#valueOf(byte []) and TN#valueOf(String) duplicate code.
Yeah, add some javadoc on what qualifier is. An example? Yeah, javadoc needs
examples.
Fix javadoc (missing @param)
+ hbase-server/src/main/java/org/apache/hadoop/hbase/NamespaceUpgrade.java
Suggest putting this class into a migration subpackage (it is where we used to
put
shortlived migration classes in the past -- see 0.92... )
The class javadoc is off we use ':' now?
Looking at this script, can it resume if it fails midway through a migration
and it then
gets restarted again? It does not look like it.
Good that it implements Tool.
Does it change the fs file version? I don't see it.
+ hbase-server/src/main/java/org/apache/hadoop/hbase/io/HFileLink.java
You have unit test for your new hfilelink regex?
You have a note about ':' being illegal but it is floating in the middle of the
file
unattached.
Yeah, do these changes have test coverage?
{code}
More to come.
> Implement namespace
> ---
>
> Key: HBASE-8408
> URL: https://issues.apache.org/jira/browse/HBASE-8408
> Project: HBase
> Issue Type: Sub-task
>Reporter: Francis Liu
>Assignee: Francis Liu
> Attachments: HBASE-8015_11.patch, HBASE-8015_1.patch,
> HBASE-8015_2.patch, HBASE-8015_3.patch, HBASE-8015_4.patch,
> HBASE-8015_5.patch, HBASE-8015_6.patch, HBASE-8015_7.patch,
> HBASE-8015_8.patch, HBASE-8015_9.patch, HBASE-8015.patch,
> TestNamespaceMigration.tgz, TestNamespaceUpgrade.tgz
>
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8693) DataType: provide extensible type API
[
https://issues.apache.org/jira/browse/HBASE-8693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13727396#comment-13727396
]
Hadoop QA commented on HBASE-8693:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12595534/0001-HBASE-8693-Extensible-data-types-API.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 32 new
or modified tests.
{color:red}-1 hadoop1.0{color}. The patch failed to compile against the
hadoop 1.0 profile.
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/6566//console
This message is automatically generated.
> DataType: provide extensible type API
> -
>
> Key: HBASE-8693
> URL: https://issues.apache.org/jira/browse/HBASE-8693
> Project: HBase
> Issue Type: Sub-task
> Components: Client
>Reporter: Nick Dimiduk
>Assignee: Nick Dimiduk
> Fix For: 0.95.2
>
> Attachments: 0001-HBASE-8693-Extensible-data-types-API.patch,
> 0001-HBASE-8693-Extensible-data-types-API.patch,
> 0001-HBASE-8693-Extensible-data-types-API.patch,
> 0001-HBASE-8693-Extensible-data-types-API.patch,
> 0001-HBASE-8693-Extensible-data-types-API.patch,
> 0001-HBASE-8693-Extensible-data-types-API.patch,
> 0001-HBASE-8693-Extensible-data-types-API.patch,
> 0002-HBASE-8693-example-Use-DataType-API-to-build-regionN.patch,
> KijiFormattedEntityId.java
>
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8949) hbase.mapreduce.hfileoutputformat.blocksize should configure with blocksize of a table
[ https://issues.apache.org/jira/browse/HBASE-8949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13727395#comment-13727395 ] rajeshbabu commented on HBASE-8949: --- [~lhofhansl] I was not able to add release notes. I didn't find link to add it. > hbase.mapreduce.hfileoutputformat.blocksize should configure with blocksize > of a table > -- > > Key: HBASE-8949 > URL: https://issues.apache.org/jira/browse/HBASE-8949 > Project: HBase > Issue Type: Bug > Components: mapreduce >Reporter: rajeshbabu >Assignee: rajeshbabu > Fix For: 0.98.0, 0.95.2, 0.94.11 > > Attachments: HBASE-8949_94_2.patch, HBASE-8949_94.patch, > HBASE-8949_trunk_2.patch, HBASE-8949_trunk.patch > > > While initializing mapreduce job we are not configuring > hbase.mapreduce.hfileoutputformat.blocksize, so hfiles are always creating > with 64kb (default)block size even though tables has different block size. > We need to configure it with block size from table descriptor. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8201) OrderedBytes: an ordered encoding strategy
[
https://issues.apache.org/jira/browse/HBASE-8201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13727394#comment-13727394
]
Hadoop QA commented on HBASE-8201:
--
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12595533/0001-HBASE-8201-OrderedBytes-order-preserving-encoding.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 4 new
or modified tests.
{color:green}+1 hadoop1.0{color}. The patch compiles against the hadoop
1.0 profile.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/6565//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6565//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6565//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6565//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6565//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6565//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6565//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6565//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6565//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/6565//console
This message is automatically generated.
> OrderedBytes: an ordered encoding strategy
> --
>
> Key: HBASE-8201
> URL: https://issues.apache.org/jira/browse/HBASE-8201
> Project: HBase
> Issue Type: Sub-task
> Components: Client
>Reporter: Nick Dimiduk
>Assignee: Nick Dimiduk
> Fix For: 0.95.2
>
> Attachments:
> 0001-HBASE-8201-OrderedBytes-order-preserving-encoding.patch,
> 0001-HBASE-8201-OrderedBytes-order-preserving-encoding.patch,
> 0001-HBASE-8201-OrderedBytes-provides-order-preserving-se.patch,
> 0001-HBASE-8201-OrderedBytes-provides-order-preserving-se.patch,
> 0001-HBASE-8201-OrderedBytes-provides-order-preserving-se.patch,
> 0001-HBASE-8201-OrderedBytes-provides-order-preserving-se.patch,
> 0001-HBASE-8201-OrderedBytes-provides-order-preserving-se.patch,
> 0001-HBASE-8201-OrderedBytes-provides-order-preserving-se.patch,
> 0001-HBASE-8201-OrderedBytes-provides-order-preserving-se.patch,
> 0001-HBASE-8201-OrderedBytes-provides-order-preserving-se.patch,
> 0001-HBASE-8201-OrderedBytes-provides-order-preserving-se.patch,
> 0001-HBASE-8201-OrderedBytes-provides-order-preserving-se.patch,
> 0001-HBASE-8201-OrderedBytes-provides-order-preserving-se.patch
>
>
> Once the spec is agreed upon, it must be implemented.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8912) [0.94] AssignmentManager throws IllegalStateException from PENDING_OPEN to OFFLINE
[
https://issues.apache.org/jira/browse/HBASE-8912?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lars Hofhansl updated HBASE-8912:
-
Fix Version/s: (was: 0.94.11)
0.94.12
No patch. Need to move on.
> [0.94] AssignmentManager throws IllegalStateException from PENDING_OPEN to
> OFFLINE
> --
>
> Key: HBASE-8912
> URL: https://issues.apache.org/jira/browse/HBASE-8912
> Project: HBase
> Issue Type: Bug
>Reporter: Enis Soztutar
> Fix For: 0.94.12
>
> Attachments: HBase-0.94 #1036 test - testRetrying [Jenkins].html
>
>
> AM throws this exception which subsequently causes the master to abort:
> {code}
> java.lang.IllegalStateException: Unexpected state :
> testRetrying,jjj,1372891751115.9b828792311001062a5ff4b1038fe33b.
> state=PENDING_OPEN, ts=1372891751912,
> server=hemera.apache.org,39064,1372891746132 .. Cannot transit it to OFFLINE.
> at
> org.apache.hadoop.hbase.master.AssignmentManager.setOfflineInZooKeeper(AssignmentManager.java:1879)
> at
> org.apache.hadoop.hbase.master.AssignmentManager.assign(AssignmentManager.java:1688)
> at
> org.apache.hadoop.hbase.master.AssignmentManager.assign(AssignmentManager.java:1424)
> at
> org.apache.hadoop.hbase.master.AssignmentManager.assign(AssignmentManager.java:1399)
> at
> org.apache.hadoop.hbase.master.AssignmentManager.assign(AssignmentManager.java:1394)
> at
> org.apache.hadoop.hbase.master.handler.ClosedRegionHandler.process(ClosedRegionHandler.java:105)
> at
> org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:175)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
> at java.lang.Thread.run(Thread.java:662)
> {code}
> This exception trace is from the failing test TestMetaReaderEditor which is
> failing pretty frequently, but looking at the test code, I think this is not
> a test-only issue, but affects the main code path.
> https://builds.apache.org/job/HBase-0.94/1036/testReport/junit/org.apache.hadoop.hbase.catalog/TestMetaReaderEditor/testRetrying/
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8353) -ROOT-/.META. regions are hanging if master restarted while closing -ROOT-/.META. regions on dead RS
[
https://issues.apache.org/jira/browse/HBASE-8353?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lars Hofhansl updated HBASE-8353:
-
Fix Version/s: (was: 0.94.11)
0.94.12
So this has been open since 0.94.7. Should finish this. Moving on to 0.94.12
for now.
[~jxiang] What do you think. Is this safe, good to go? (Can't say I'm an expert
on the this part of the code).
> -ROOT-/.META. regions are hanging if master restarted while closing
> -ROOT-/.META. regions on dead RS
>
>
> Key: HBASE-8353
> URL: https://issues.apache.org/jira/browse/HBASE-8353
> Project: HBase
> Issue Type: Bug
> Components: Region Assignment
>Affects Versions: 0.94.6
>Reporter: rajeshbabu
>Assignee: rajeshbabu
> Fix For: 0.94.12
>
> Attachments: HBASE-8353_94_2.patch, HBASE-8353_94_3.patch,
> HBASE-8353_94_4.patch, HBASE-8353_94.patch
>
>
> ROOT/META are not getting assigned if master restarted while closing
> ROOT/META.
> Lets suppose catalog table regions in M_ZK_REGION_CLOSING state during master
> initialization and then just we are adding the them to RIT and waiting for
> TM. {code}
> if (isOnDeadServer(regionInfo, deadServers) &&
> (data.getOrigin() == null ||
> !serverManager.isServerOnline(data.getOrigin( {
> // If was on dead server, its closed now. Force to OFFLINE and this
> // will get it reassigned if appropriate
> forceOffline(regionInfo, data);
> } else {
> // Just insert region into RIT.
> // If this never updates the timeout will trigger new assignment
> regionsInTransition.put(encodedRegionName, new RegionState(
> regionInfo, RegionState.State.CLOSING,
> data.getStamp(), data.getOrigin()));
> }
> {code}
> isOnDeadServer always return false to ROOT/META because deadServers is null.
> Even TM cannot close them properly because its not available in online
> regions since its not yet assigned.
> {code}
> synchronized (this.regions) {
> // Check if this region is currently assigned
> if (!regions.containsKey(region)) {
> LOG.debug("Attempted to unassign region " +
> region.getRegionNameAsString() + " but it is not " +
> "currently assigned anywhere");
> return;
> }
> }
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8949) hbase.mapreduce.hfileoutputformat.blocksize should configure with blocksize of a table
[ https://issues.apache.org/jira/browse/HBASE-8949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13727389#comment-13727389 ] Lars Hofhansl commented on HBASE-8949: -- Looks good to me. It makes little sense to force a block size, when the next time the data in this HFile is compacted it'll be rewritten with the new block size (same way it does not make to set a bloom filter to any but the configuration set in the table). +1 from me. Need to add release notes if this goes into 0.94. > hbase.mapreduce.hfileoutputformat.blocksize should configure with blocksize > of a table > -- > > Key: HBASE-8949 > URL: https://issues.apache.org/jira/browse/HBASE-8949 > Project: HBase > Issue Type: Bug > Components: mapreduce >Reporter: rajeshbabu >Assignee: rajeshbabu > Fix For: 0.98.0, 0.95.2, 0.94.11 > > Attachments: HBASE-8949_94_2.patch, HBASE-8949_94.patch, > HBASE-8949_trunk_2.patch, HBASE-8949_trunk.patch > > > While initializing mapreduce job we are not configuring > hbase.mapreduce.hfileoutputformat.blocksize, so hfiles are always creating > with 64kb (default)block size even though tables has different block size. > We need to configure it with block size from table descriptor. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8204) Don't use hdfs append during lease recovery
[ https://issues.apache.org/jira/browse/HBASE-8204?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13727383#comment-13727383 ] Hudson commented on HBASE-8204: --- SUCCESS: Integrated in HBase-0.94-security #240 (See [https://builds.apache.org/job/HBase-0.94-security/240/]) HBASE-8670 [0.94] Backport HBASE-8449,HBASE-8204 and HBASE-8699 to 0.94 (Refactor recoverLease retries and pauses) (enis: rev 1509532) * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/util/FSHDFSUtils.java * /hbase/branches/0.94/src/main/resources/hbase-default.xml * /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestHLog.java * /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/util/TestFSHDFSUtils.java > Don't use hdfs append during lease recovery > --- > > Key: HBASE-8204 > URL: https://issues.apache.org/jira/browse/HBASE-8204 > Project: HBase > Issue Type: Bug > Components: MTTR, wal >Affects Versions: 0.95.2 >Reporter: Nicolas Liochon >Assignee: Nicolas Liochon > Fix For: 0.95.1, 0.95.2 > > Attachments: 8204.v2.patch, 8204.v2.patch, 8204.v3.patch, > 8204.v4.patch, HBASE-8204.v1.patch > > > See patch :-). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9107) [0.94] Backport HBASE-6950 TestAcidGuarantees system test now flushes too aggressively to 0.94
[ https://issues.apache.org/jira/browse/HBASE-9107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13727385#comment-13727385 ] Hudson commented on HBASE-9107: --- SUCCESS: Integrated in HBase-0.94-security #240 (See [https://builds.apache.org/job/HBase-0.94-security/240/]) HBASE-9107 [0.94] Backport HBASE-6950 TestAcidGuarantees system test now flushes too aggressively to 0.94 (enis: rev 1509533) * /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/TestAcidGuarantees.java > [0.94] Backport HBASE-6950 TestAcidGuarantees system test now flushes too > aggressively to 0.94 > -- > > Key: HBASE-9107 > URL: https://issues.apache.org/jira/browse/HBASE-9107 > Project: HBase > Issue Type: Test >Reporter: Enis Soztutar >Assignee: Enis Soztutar > Fix For: 0.94.11 > > Attachments: hbase-9107_v1.patch > > > As the description says. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9085) Integration Tests fails because of bug in teardown phase where the cluster state is not being restored properly.
[
https://issues.apache.org/jira/browse/HBASE-9085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13727388#comment-13727388
]
Hudson commented on HBASE-9085:
---
SUCCESS: Integrated in HBase-0.94-security #240 (See
[https://builds.apache.org/job/HBase-0.94-security/240/])
HBASE-9085 Integration Tests fails because of bug in teardown phase where the
cluster state is not being restored properly. (gautam) (enis: rev 1509540)
*
/hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/DistributedHBaseCluster.java
> Integration Tests fails because of bug in teardown phase where the cluster
> state is not being restored properly.
>
>
> Key: HBASE-9085
> URL: https://issues.apache.org/jira/browse/HBASE-9085
> Project: HBase
> Issue Type: Bug
> Components: test
>Affects Versions: 0.95.0, 0.94.9, 0.94.10
>Reporter: gautam
>Assignee: gautam
> Fix For: 0.98.0, 0.95.2, 0.94.11
>
> Attachments: HBASE-9085.patch._0.94, HBASE-9085.patch._0.95_or_trunk
>
>
> I was running the following test over a Distributed Cluster:
> bin/hbase org.apache.hadoop.hbase.IntegrationTestsDriver
> IntegrationTestDataIngestSlowDeterministic
> The IntegrationTestingUtility.restoreCluster() is called in the teardown
> phase of the test.
> For a distributed cluster, it ends up calling
> DistributedHBaseCluster.restoreClusterStatus, which does the task
> of restoring the cluster back to original state.
> The restore steps done here, does not solve one specific case:
> When the initial HBase Master is currently down, and the current HBase Master
> is different from the initial one.
> You get into this flow:
> //check whether current master has changed
> if (!ServerName.isSameHostnameAndPort(initial.getMaster(),
> current.getMaster())) {
> .
> }
> In the above code path, the current backup masters are stopped, and the
> current active master is also stopped.
> At this point, for the aforementioned usecase, none of the Hbase Masters
> would be available, hence the subsequent
> attempts to do any operation over the cluster would fail, resulting in Test
> Failure.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8449) Refactor recoverLease retries and pauses informed by findings over in hbase-8389
[ https://issues.apache.org/jira/browse/HBASE-8449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13727382#comment-13727382 ] Hudson commented on HBASE-8449: --- SUCCESS: Integrated in HBase-0.94-security #240 (See [https://builds.apache.org/job/HBase-0.94-security/240/]) HBASE-8670 [0.94] Backport HBASE-8449,HBASE-8204 and HBASE-8699 to 0.94 (Refactor recoverLease retries and pauses) (enis: rev 1509532) * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/util/FSHDFSUtils.java * /hbase/branches/0.94/src/main/resources/hbase-default.xml * /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestHLog.java * /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/util/TestFSHDFSUtils.java > Refactor recoverLease retries and pauses informed by findings over in > hbase-8389 > > > Key: HBASE-8449 > URL: https://issues.apache.org/jira/browse/HBASE-8449 > Project: HBase > Issue Type: Bug > Components: Filesystem Integration >Affects Versions: 0.94.7, 0.95.0 >Reporter: stack >Assignee: stack >Priority: Critical > Fix For: 0.98.0, 0.95.1 > > Attachments: 8449.txt, 8449v2.txt, 8449v3.txt, 8449v4.txt, > 8449v6.txt, 8449v7.txt > > > HBASE-8359 is an interesting issue that roams near and far. This issue is > about making use of the findings handily summarized on the end of hbase-8359 > which have it that trunk needs refactor around how it does its recoverLease > handling (and that the patch committed against HBASE-8359 is not what we want > going forward). > This issue is about making a patch that adds a lag between recoverLease > invocations where the lag is related to dfs timeouts -- the hdfs-side dfs > timeout -- and optionally makes use of the isFileClosed API if it is > available (a facility that is not yet committed to a branch near you and > unlikely to be within your locality with a good while to come). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8670) [0.94] Backport HBASE-8449,HBASE-8204 and HBASE-8699 to 0.94 (Refactor recoverLease retries and pauses)
[ https://issues.apache.org/jira/browse/HBASE-8670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13727387#comment-13727387 ] Hudson commented on HBASE-8670: --- SUCCESS: Integrated in HBase-0.94-security #240 (See [https://builds.apache.org/job/HBase-0.94-security/240/]) HBASE-8670 [0.94] Backport HBASE-8449,HBASE-8204 and HBASE-8699 to 0.94 (Refactor recoverLease retries and pauses) (enis: rev 1509532) * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/util/FSHDFSUtils.java * /hbase/branches/0.94/src/main/resources/hbase-default.xml * /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestHLog.java * /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/util/TestFSHDFSUtils.java > [0.94] Backport HBASE-8449,HBASE-8204 and HBASE-8699 to 0.94 (Refactor > recoverLease retries and pauses) > --- > > Key: HBASE-8670 > URL: https://issues.apache.org/jira/browse/HBASE-8670 > Project: HBase > Issue Type: Bug > Components: Filesystem Integration, master, wal >Reporter: Enis Soztutar >Assignee: Enis Soztutar > Fix For: 0.94.11 > > Attachments: hbase-8670_v1.patch, hbase-8670_v2.patch > > > Some history: > Up until 0.94.8, Hbase did not check the result of recoverLease() call, but > things kind of worked since we are checking for 0-length files in distributed > log split tasks from region servers. If lease recovery is not finished, the > log file will report 0 length, and the task will fail, and master will then > re-call recoverLease() and reassign the task. This scheme might fail for log > files that are larger than 1 hdfs block though. > In 0.94.8, we committed (HBASE-8354, which is backport of HBASE-7878) and > later increased the sleep time to 4 secs in HBASE-8389. > However, the proper solution arrived in trunk in HBASE-8449 which uses a > backoff sleep policy + isFileClosed() api. We should backport this patch to > 0.94 as well. > isFileClosed() is released in Hadoop 1.2.0 (HDFS-4774) and 2.0.5(HDFS-4525). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8699) Parameter to DistributedFileSystem#isFileClosed should be of type Path
[
https://issues.apache.org/jira/browse/HBASE-8699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13727381#comment-13727381
]
Hudson commented on HBASE-8699:
---
SUCCESS: Integrated in HBase-0.94-security #240 (See
[https://builds.apache.org/job/HBase-0.94-security/240/])
HBASE-8670 [0.94] Backport HBASE-8449,HBASE-8204 and HBASE-8699 to 0.94
(Refactor recoverLease retries and pauses) (enis: rev 1509532)
*
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/util/FSHDFSUtils.java
* /hbase/branches/0.94/src/main/resources/hbase-default.xml
*
/hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestHLog.java
*
/hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/util/TestFSHDFSUtils.java
> Parameter to DistributedFileSystem#isFileClosed should be of type Path
> --
>
> Key: HBASE-8699
> URL: https://issues.apache.org/jira/browse/HBASE-8699
> Project: HBase
> Issue Type: Bug
> Components: wal
>Reporter: Ted Yu
>Assignee: Ted Yu
> Fix For: 0.98.0, 0.95.2
>
> Attachments: 8699-v1.txt, 8699-v3.txt, 8699-v4.txt, 8699-v5.txt,
> 8699-v5.txt
>
>
> Here is current code of FSHDFSUtils#isFileClosed():
> {code}
> boolean isFileClosed(final DistributedFileSystem dfs, final Path p) {
> try {
> Method m = dfs.getClass().getMethod("isFileClosed", new Class[]
> {String.class});
> return (Boolean) m.invoke(dfs, p.toString());
> {code}
> We look for isFileClosed method with parameter type of String.
> However, from
> hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DistributedFileSystem.java
> (branch-2):
> {code}
> public boolean isFileClosed(Path src) throws IOException {
> {code}
> The parameter type is of Path.
> This means we would get NoSuchMethodException.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9106) Do not fail TestAcidGuarantees for exceptions on table flush
[
https://issues.apache.org/jira/browse/HBASE-9106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13727384#comment-13727384
]
Hudson commented on HBASE-9106:
---
SUCCESS: Integrated in HBase-0.94-security #240 (See
[https://builds.apache.org/job/HBase-0.94-security/240/])
HBASE-9106 Do not fail TestAcidGuarantees for exceptions on table flush (enis:
rev 1509534)
*
/hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/TestAcidGuarantees.java
> Do not fail TestAcidGuarantees for exceptions on table flush
>
>
> Key: HBASE-9106
> URL: https://issues.apache.org/jira/browse/HBASE-9106
> Project: HBase
> Issue Type: Test
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
> Fix For: 0.98.0, 0.95.2, 0.94.11
>
> Attachments: hbase-9106-0.94_v1.patch, hbase-9106_v1.patch
>
>
> TestAcidGuarantees failed in one run due to a flush taking longer than
> >60sec, with:
> {code}
> HBaseClient$CallTimeoutException: Call id=152, waitTime=60007,
> rpcTimetout=6
> {code}
> We should ignore the exceptions coming from table flushes, since they are not
> essential to the test.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6950) TestAcidGuarantees system test now flushes too aggressively
[
https://issues.apache.org/jira/browse/HBASE-6950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13727386#comment-13727386
]
Hudson commented on HBASE-6950:
---
SUCCESS: Integrated in HBase-0.94-security #240 (See
[https://builds.apache.org/job/HBase-0.94-security/240/])
HBASE-9107 [0.94] Backport HBASE-6950 TestAcidGuarantees system test now
flushes too aggressively to 0.94 (enis: rev 1509533)
*
/hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/TestAcidGuarantees.java
> TestAcidGuarantees system test now flushes too aggressively
> ---
>
> Key: HBASE-6950
> URL: https://issues.apache.org/jira/browse/HBASE-6950
> Project: HBase
> Issue Type: Bug
> Components: test
>Affects Versions: 0.92.2, 0.94.2, 0.95.2
>Reporter: Gregory Chanan
>Assignee: Gregory Chanan
>Priority: Minor
> Fix For: 0.95.0
>
> Attachments: HBASE-6950.patch
>
>
> HBASE-6552 caused the TestAcidGuarantees system test to flush more
> aggressively, because flushes are where ACID problems have occurred in the
> past.
> After some more cluster testing, it seems like this too aggressive; my
> clusters eventually can't keep up with the number of flushes/compactions and
> start getting SocketTimeoutExceptions. We could try to optimize the
> flushes/compactions, but since this workload would never occur in practice, I
> don't think it is worth the effort. Instead, let's just only flush once a
> minute. This is arbitrary, but seems to work.
> Here is my comment in the (upcoming) patch:
> {code}
> // Flushing has been a source of ACID violations previously (see HBASE-2856),
> so ideally,
> // we would flush as often as possible. On a running cluster, this isn't
> practical:
> // (1) we will cause a lot of load due to all the flushing and compacting
> // (2) we cannot change the flushing/compacting related Configuration options
> to try to
> // alleviate this
> // (3) it is an unrealistic workload, since no one would actually flush that
> often.
> // Therefore, let's flush every minute to have more flushes than usual, but
> not overload
> // the running cluster.
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6580) Deprecate HTablePool in favor of HConnection.getTable(...)
[ https://issues.apache.org/jira/browse/HBASE-6580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13727370#comment-13727370 ] stack commented on HBASE-6580: -- Could do it in 0.94. Doing the deprecation this late in the game probably doesn't really count toward 0.96. > Deprecate HTablePool in favor of HConnection.getTable(...) > -- > > Key: HBASE-6580 > URL: https://issues.apache.org/jira/browse/HBASE-6580 > Project: HBase > Issue Type: Improvement >Affects Versions: 0.94.6, 0.95.0 >Reporter: Lars Hofhansl >Assignee: Lars Hofhansl >Priority: Minor > Fix For: 0.98.0, 0.95.2, 0.94.11 > > Attachments: 6580-trunk.txt, HBASE-6580_v1.patch, HBASE-6580_v2.patch > > > Update: > I now propose deprecating HTablePool and instead introduce a getTable method > on HConnection and allow HConnection to manage the ThreadPool. > Initial proposal: > Here I propose a very simple TablePool. > It could be called LightHTablePool (or something - if you have a better name). > Internally it would maintain an HConnection and an Executor service and each > invocation of getTable(...) would create a new HTable and close() would just > close it. > In testing I find this more light weight than HTablePool and easier to > monitor in terms of resources used. > It would hardly be more than a few dozen lines of code. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6580) Deprecate HTablePool in favor of HConnection.getTable(...)
[ https://issues.apache.org/jira/browse/HBASE-6580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13727368#comment-13727368 ] Lars Hofhansl commented on HBASE-6580: -- Also, do this in 0.94? So we get the deprecation in? I'm fine with punting on this in 0.94 as it is really just convenience. The existing HTable constructors provide all this functionality (albeit in a more clumsy way). > Deprecate HTablePool in favor of HConnection.getTable(...) > -- > > Key: HBASE-6580 > URL: https://issues.apache.org/jira/browse/HBASE-6580 > Project: HBase > Issue Type: Improvement >Affects Versions: 0.94.6, 0.95.0 >Reporter: Lars Hofhansl >Assignee: Lars Hofhansl >Priority: Minor > Fix For: 0.98.0, 0.95.2, 0.94.11 > > Attachments: 6580-trunk.txt, HBASE-6580_v1.patch, HBASE-6580_v2.patch > > > Update: > I now propose deprecating HTablePool and instead introduce a getTable method > on HConnection and allow HConnection to manage the ThreadPool. > Initial proposal: > Here I propose a very simple TablePool. > It could be called LightHTablePool (or something - if you have a better name). > Internally it would maintain an HConnection and an Executor service and each > invocation of getTable(...) would create a new HTable and close() would just > close it. > In testing I find this more light weight than HTablePool and easier to > monitor in terms of resources used. > It would hardly be more than a few dozen lines of code. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8760) possible loss of data in snapshot taken after region split
[ https://issues.apache.org/jira/browse/HBASE-8760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl updated HBASE-8760: - Fix Version/s: (was: 0.94.11) 0.94.12 Anything happening on this? > possible loss of data in snapshot taken after region split > -- > > Key: HBASE-8760 > URL: https://issues.apache.org/jira/browse/HBASE-8760 > Project: HBase > Issue Type: Bug > Components: snapshots >Affects Versions: 0.94.8, 0.95.1 >Reporter: Jerry He > Fix For: 0.98.0, 0.95.2, 0.94.12 > > Attachments: HBase-8760-0.94.8.patch, HBase-8760-0.94.8-v1.patch, > HBASE-8760-thz-v0.patch > > > Right after a region split but before the daughter regions are compacted, we > have two daughter regions containing Reference files to the parent hfiles. > If we take snapshot right at the moment, the snapshot will succeed, but it > will only contain the daughter Reference files. Since there is no hold on the > parent hfiles, they will be deleted by the HFile Cleaner after they are no > longer needed by the daughter regions soon after. > A minimum we need to do is the keep these parent hfiles from being deleted. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8434) Allow enabling hbase 8354 to support real lease recovery
[ https://issues.apache.org/jira/browse/HBASE-8434?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl updated HBASE-8434: - Fix Version/s: (was: 0.94.11) 0.94.12 I am tempted to just close this. Moving to 0.94.12, for now. > Allow enabling hbase 8354 to support real lease recovery > > > Key: HBASE-8434 > URL: https://issues.apache.org/jira/browse/HBASE-8434 > Project: HBase > Issue Type: Improvement > Components: MTTR >Reporter: Varun Sharma >Assignee: Varun Sharma > Fix For: 0.94.12 > > Attachments: 8434.patch > > > Please see discussion in HBase 8389. > For environments where lease recovery time can be bounded on the HDFS side > through tight timeouts, provide a toggle for users who want the WAL splitting > to continue only after the lease is truly recovered and the file is closed. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-9054) HBaseAdmin#isTableDisabled() should check table existence before checking zk state.
[ https://issues.apache.org/jira/browse/HBASE-9054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl updated HBASE-9054: - Fix Version/s: (was: 0.94.11) 0.94.12 Make a patch [~beneguo]? > HBaseAdmin#isTableDisabled() should check table existence before checking zk > state. > > > Key: HBASE-9054 > URL: https://issues.apache.org/jira/browse/HBASE-9054 > Project: HBase > Issue Type: Bug > Components: Admin >Affects Versions: 0.94.10 >Reporter: Bene Guo > Fix For: 0.94.12 > > > To avoid compatibility issues with older versions HBaseAdmin#isTableDisabled > and HBaseAdmin#isTableEnabled()(The HBASE-8538 fix isTableEnabled.) returning > true even if the table state is null. Its also returning true even a table is > not present. We should confirm table existence from .META. before checking in > zk. If table not present or deleted, then It will throw > TableNotFoundException. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8836) Separate reader and writer thread pool in RegionServer, so that write throughput will not be impacted when the read load is very high
[ https://issues.apache.org/jira/browse/HBASE-8836?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl updated HBASE-8836: - Fix Version/s: (was: 0.94.11) 0.94.12 Moving on to 0.94.12. We need to maintain the batch nature *especially* for update (so that they go in batch through doMiniBatchMutation). > Separate reader and writer thread pool in RegionServer, so that write > throughput will not be impacted when the read load is very high > - > > Key: HBASE-8836 > URL: https://issues.apache.org/jira/browse/HBASE-8836 > Project: HBase > Issue Type: New Feature > Components: Performance, regionserver >Affects Versions: 0.94.8 >Reporter: Tianying Chang >Assignee: Tianying Chang > Fix For: 0.98.0, 0.95.2, 0.94.12 > > Attachments: hbase-8836.patch, Hbase-8836-perfNumber.pdf > > > We found that when the read load on a specific RS is high, the write > throughput also get impacted dramatically, and even cause write data loss > sometimes. We want to prioritize the write by putting them in a separate > queue from the read request, so that slower read will not make fast write > wait nu-necessarily long. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8408) Implement namespace
[ https://issues.apache.org/jira/browse/HBASE-8408?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13727361#comment-13727361 ] stack commented on HBASE-8408: -- Don't worry about line lengths I'd say. Ok On TestNSUpgrade failing for now. Make sure it passes locally for you. No worries on findbugs. Can do after patch goes in. We can help there. Ditto on javadoc fixes. > Implement namespace > --- > > Key: HBASE-8408 > URL: https://issues.apache.org/jira/browse/HBASE-8408 > Project: HBase > Issue Type: Sub-task >Reporter: Francis Liu >Assignee: Francis Liu > Attachments: HBASE-8015_11.patch, HBASE-8015_1.patch, > HBASE-8015_2.patch, HBASE-8015_3.patch, HBASE-8015_4.patch, > HBASE-8015_5.patch, HBASE-8015_6.patch, HBASE-8015_7.patch, > HBASE-8015_8.patch, HBASE-8015_9.patch, HBASE-8015.patch, > TestNamespaceMigration.tgz, TestNamespaceUpgrade.tgz > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9075) [0.94] Backport HBASE-5760 Unit tests should write only under /target to 0.94
[ https://issues.apache.org/jira/browse/HBASE-9075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13727358#comment-13727358 ] Lars Hofhansl commented on HBASE-9075: -- Cool. +1 > [0.94] Backport HBASE-5760 Unit tests should write only under /target to 0.94 > - > > Key: HBASE-9075 > URL: https://issues.apache.org/jira/browse/HBASE-9075 > Project: HBase > Issue Type: Test > Components: test >Reporter: Enis Soztutar >Assignee: Enis Soztutar > Fix For: 0.94.11 > > Attachments: hbase-9075_v1.patch > > > Backporting HBASE-5760 is a good idea. 0.94 tests mess up the root level > directory a lot. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6580) Deprecate HTablePool in favor of HConnection.getTable(...)
[ https://issues.apache.org/jira/browse/HBASE-6580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13727357#comment-13727357 ] Lars Hofhansl commented on HBASE-6580: -- bq. the difference between HCM#getConnection and HCM#createConnection is that the latter will ALWAYS create a "cluster connection"? Yep. It also indicates that the caller needs to manage the lifecycle (i.e. eventually close()'ing the connection). Should probably deprecate HCM#getConnection as well. bq. (How hard to change HTable into an Interface?) HTable is the implementation of HTableInterface. The patch only exposes HTableInterface. Should we rename HTableInterface to HTable and HTable to HTableImplementation? Oh, I see. You saying HTable should not have any public constructors? Can deprecate those as well. > Deprecate HTablePool in favor of HConnection.getTable(...) > -- > > Key: HBASE-6580 > URL: https://issues.apache.org/jira/browse/HBASE-6580 > Project: HBase > Issue Type: Improvement >Affects Versions: 0.94.6, 0.95.0 >Reporter: Lars Hofhansl >Assignee: Lars Hofhansl >Priority: Minor > Fix For: 0.98.0, 0.95.2, 0.94.11 > > Attachments: 6580-trunk.txt, HBASE-6580_v1.patch, HBASE-6580_v2.patch > > > Update: > I now propose deprecating HTablePool and instead introduce a getTable method > on HConnection and allow HConnection to manage the ThreadPool. > Initial proposal: > Here I propose a very simple TablePool. > It could be called LightHTablePool (or something - if you have a better name). > Internally it would maintain an HConnection and an Executor service and each > invocation of getTable(...) would create a new HTable and close() would just > close it. > In testing I find this more light weight than HTablePool and easier to > monitor in terms of resources used. > It would hardly be more than a few dozen lines of code. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8408) Implement namespace
[ https://issues.apache.org/jira/browse/HBASE-8408?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13727356#comment-13727356 ] Francis Liu commented on HBASE-8408: Thanks Stack! Should I worry about lineLengths? A lot of the files go beyond 100 (proto, jamon, etc)? Also TestNamespaceUpgrade will always fail with hadoopqa since it does not include the TGZ. Working on the other zombie. BTW I just went through the findbugs list to figure which ones I added but can't seem to find the last one. Is this there a tool to identify which ones were added by the patch. > Implement namespace > --- > > Key: HBASE-8408 > URL: https://issues.apache.org/jira/browse/HBASE-8408 > Project: HBase > Issue Type: Sub-task >Reporter: Francis Liu >Assignee: Francis Liu > Attachments: HBASE-8015_11.patch, HBASE-8015_1.patch, > HBASE-8015_2.patch, HBASE-8015_3.patch, HBASE-8015_4.patch, > HBASE-8015_5.patch, HBASE-8015_6.patch, HBASE-8015_7.patch, > HBASE-8015_8.patch, HBASE-8015_9.patch, HBASE-8015.patch, > TestNamespaceMigration.tgz, TestNamespaceUpgrade.tgz > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8067) TestHFileArchiving.testArchiveOnTableDelete sometimes fails
[
https://issues.apache.org/jira/browse/HBASE-8067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lars Hofhansl updated HBASE-8067:
-
Fix Version/s: (was: 0.94.12)
0.94.11
> TestHFileArchiving.testArchiveOnTableDelete sometimes fails
> ---
>
> Key: HBASE-8067
> URL: https://issues.apache.org/jira/browse/HBASE-8067
> Project: HBase
> Issue Type: Bug
> Components: Admin, master, test
>Affects Versions: 0.94.6, 0.95.2
>Reporter: Matteo Bertozzi
>Assignee: Matteo Bertozzi
> Fix For: 0.98.0, 0.95.2, 0.94.11
>
> Attachments: 8067-0.94.txt, 8067-trunk.txt, HBASE-8067-debug.patch,
> HBASE-8067-v0.patch
>
>
> it seems that testArchiveOnTableDelete() fails because the archiving in
> DeleteTableHandler is still in progress when admin.deleteTable() returns.
> {code}
> Error Message
> Archived files are missing some of the store files!
> Stacktrace
> java.lang.AssertionError: Archived files are missing some of the store files!
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.assertTrue(Assert.java:41)
> at
> org.apache.hadoop.hbase.backup.TestHFileArchiving.testArchiveOnTableDelete(TestHFileArchiving.java:262)
> {code}
> (Looking at the problem in a more generic way, we don't have any way to
> inform the client when an async operation is completed)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6580) Deprecate HTablePool in favor of HConnection.getTable(...)
[ https://issues.apache.org/jira/browse/HBASE-6580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13727350#comment-13727350 ] stack commented on HBASE-6580: -- The difference between HCM#getConnection and HCM#createConnection is that the latter will ALWAYS create a "cluster connection"? That sounds good. (How hard to change HTable into an Interface?) API looks good to me. > Deprecate HTablePool in favor of HConnection.getTable(...) > -- > > Key: HBASE-6580 > URL: https://issues.apache.org/jira/browse/HBASE-6580 > Project: HBase > Issue Type: Improvement >Affects Versions: 0.94.6, 0.95.0 >Reporter: Lars Hofhansl >Assignee: Lars Hofhansl >Priority: Minor > Fix For: 0.98.0, 0.95.2, 0.94.11 > > Attachments: 6580-trunk.txt, HBASE-6580_v1.patch, HBASE-6580_v2.patch > > > Update: > I now propose deprecating HTablePool and instead introduce a getTable method > on HConnection and allow HConnection to manage the ThreadPool. > Initial proposal: > Here I propose a very simple TablePool. > It could be called LightHTablePool (or something - if you have a better name). > Internally it would maintain an HConnection and an Executor service and each > invocation of getTable(...) would create a new HTable and close() would just > close it. > In testing I find this more light weight than HTablePool and easier to > monitor in terms of resources used. > It would hardly be more than a few dozen lines of code. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6580) Deprecate HTablePool in favor of HConnection.getTable(...)
[ https://issues.apache.org/jira/browse/HBASE-6580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13727349#comment-13727349 ] Lars Hofhansl commented on HBASE-6580: -- I'll like to get consensus here first. Then I'll post the new API to the dev list to get more feedback. Could use some help with the documentation in the Javadoc in HConnection and HConnectionManager. > Deprecate HTablePool in favor of HConnection.getTable(...) > -- > > Key: HBASE-6580 > URL: https://issues.apache.org/jira/browse/HBASE-6580 > Project: HBase > Issue Type: Improvement >Affects Versions: 0.94.6, 0.95.0 >Reporter: Lars Hofhansl >Assignee: Lars Hofhansl >Priority: Minor > Fix For: 0.98.0, 0.95.2, 0.94.11 > > Attachments: 6580-trunk.txt, HBASE-6580_v1.patch, HBASE-6580_v2.patch > > > Update: > I now propose deprecating HTablePool and instead introduce a getTable method > on HConnection and allow HConnection to manage the ThreadPool. > Initial proposal: > Here I propose a very simple TablePool. > It could be called LightHTablePool (or something - if you have a better name). > Internally it would maintain an HConnection and an Executor service and each > invocation of getTable(...) would create a new HTable and close() would just > close it. > In testing I find this more light weight than HTablePool and easier to > monitor in terms of resources used. > It would hardly be more than a few dozen lines of code. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8670) [0.94] Backport HBASE-8449,HBASE-8204 and HBASE-8699 to 0.94 (Refactor recoverLease retries and pauses)
[ https://issues.apache.org/jira/browse/HBASE-8670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl updated HBASE-8670: - Resolution: Fixed Status: Resolved (was: Patch Available) > [0.94] Backport HBASE-8449,HBASE-8204 and HBASE-8699 to 0.94 (Refactor > recoverLease retries and pauses) > --- > > Key: HBASE-8670 > URL: https://issues.apache.org/jira/browse/HBASE-8670 > Project: HBase > Issue Type: Bug > Components: Filesystem Integration, master, wal >Reporter: Enis Soztutar >Assignee: Enis Soztutar > Fix For: 0.94.11 > > Attachments: hbase-8670_v1.patch, hbase-8670_v2.patch > > > Some history: > Up until 0.94.8, Hbase did not check the result of recoverLease() call, but > things kind of worked since we are checking for 0-length files in distributed > log split tasks from region servers. If lease recovery is not finished, the > log file will report 0 length, and the task will fail, and master will then > re-call recoverLease() and reassign the task. This scheme might fail for log > files that are larger than 1 hdfs block though. > In 0.94.8, we committed (HBASE-8354, which is backport of HBASE-7878) and > later increased the sleep time to 4 secs in HBASE-8389. > However, the proper solution arrived in trunk in HBASE-8449 which uses a > backoff sleep policy + isFileClosed() api. We should backport this patch to > 0.94 as well. > isFileClosed() is released in Hadoop 1.2.0 (HDFS-4774) and 2.0.5(HDFS-4525). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HBASE-8067) TestHFileArchiving.testArchiveOnTableDelete sometimes fails
[
https://issues.apache.org/jira/browse/HBASE-8067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack resolved HBASE-8067.
--
Resolution: Fixed
Resolving. Hasn't failed in ages.
> TestHFileArchiving.testArchiveOnTableDelete sometimes fails
> ---
>
> Key: HBASE-8067
> URL: https://issues.apache.org/jira/browse/HBASE-8067
> Project: HBase
> Issue Type: Bug
> Components: Admin, master, test
>Affects Versions: 0.94.6, 0.95.2
>Reporter: Matteo Bertozzi
>Assignee: Matteo Bertozzi
> Fix For: 0.98.0, 0.95.2, 0.94.12
>
> Attachments: 8067-0.94.txt, 8067-trunk.txt, HBASE-8067-debug.patch,
> HBASE-8067-v0.patch
>
>
> it seems that testArchiveOnTableDelete() fails because the archiving in
> DeleteTableHandler is still in progress when admin.deleteTable() returns.
> {code}
> Error Message
> Archived files are missing some of the store files!
> Stacktrace
> java.lang.AssertionError: Archived files are missing some of the store files!
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.assertTrue(Assert.java:41)
> at
> org.apache.hadoop.hbase.backup.TestHFileArchiving.testArchiveOnTableDelete(TestHFileArchiving.java:262)
> {code}
> (Looking at the problem in a more generic way, we don't have any way to
> inform the client when an async operation is completed)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6580) Deprecate HTablePool in favor of HConnection.getTable(...)
[
https://issues.apache.org/jira/browse/HBASE-6580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lars Hofhansl updated HBASE-6580:
-
Attachment: 6580-trunk.txt
Here's the proposed new API:
* HConnectionManager:
{code}
public static HConnection createConnection(Configuration conf)
public static HConnection createConnection(Configuration conf,
ExecutorService pool)
{code}
* HConnection:
{code}
public HTableInterface getTable(byte[] tableName) throws IOException {
public HTableInterface getTable(byte[] tableName, ExecutorService pool)
throws IOException {
public HTableInterface getTable(String tableName) throws IOException {
{code}
By default HConnectionImplementation will create an ExecutorService when
needed. The ExecutorService can optionally passed be passed in.
HTableInterfaces are retrieved from the HConnection. By default the
HConnection's ExecutorService is used, but optionally that can be overridden
for each HTable.
> Deprecate HTablePool in favor of HConnection.getTable(...)
> --
>
> Key: HBASE-6580
> URL: https://issues.apache.org/jira/browse/HBASE-6580
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 0.94.6, 0.95.0
>Reporter: Lars Hofhansl
>Assignee: Lars Hofhansl
>Priority: Minor
> Fix For: 0.98.0, 0.95.2, 0.94.11
>
> Attachments: 6580-trunk.txt, HBASE-6580_v1.patch, HBASE-6580_v2.patch
>
>
> Update:
> I now propose deprecating HTablePool and instead introduce a getTable method
> on HConnection and allow HConnection to manage the ThreadPool.
> Initial proposal:
> Here I propose a very simple TablePool.
> It could be called LightHTablePool (or something - if you have a better name).
> Internally it would maintain an HConnection and an Executor service and each
> invocation of getTable(...) would create a new HTable and close() would just
> close it.
> In testing I find this more light weight than HTablePool and easier to
> monitor in terms of resources used.
> It would hardly be more than a few dozen lines of code.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8663) a HBase Shell command to list the tables replicated from current cluster
[
https://issues.apache.org/jira/browse/HBASE-8663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13727341#comment-13727341
]
stack commented on HBASE-8663:
--
Is that test failure because of your patch [~nidmhbase]?
> a HBase Shell command to list the tables replicated from current cluster
>
>
> Key: HBASE-8663
> URL: https://issues.apache.org/jira/browse/HBASE-8663
> Project: HBase
> Issue Type: New Feature
> Components: Replication, shell
> Environment: clusters setup as Master and Slave for replication of
> tables
>Reporter: Demai Ni
>Assignee: Demai Ni
>Priority: Critical
> Attachments: HBASE-8663.PATCH, HBASE-8663-trunk-v0.patch,
> HBASE-8663-trunk-v1.patch, HBASE-8663-v2.PATCH
>
>
> Thanks for the discussion and very good suggestions,I'd reduce the scope of
> this jira to only display the tables replicated from current cluster. Since
> currently no good(accurate and consistent) way to flag a table on slave
> cluster, this jira will not cover such scenario. Instead, the patch will be
> flexible enough to adapt such scenario and a follow up JIRA will be opened to
> address such situation.
> The shell command and output will be like. Since all replication is 'global',
> so no need to display the cluster name here. In the future, the command will
> be extended for other scenarios, such as 1) replicated only to selected peers
> or 2) indicate table:colfam on slave side
> {code: title=hbase shell command:list_replicated_tables |borderStyle=solid}
> hbase(main):001:0> list_replicated_tables
> TABLE:COLUMNFAMILY ReplicationType
>
> t1_dn:cf1 GLOBAL
>
> t2_dn:cf2 GLOBAL
>
> usertable:familyGLOBAL
>
> 3 row(s) in 0.4110 seconds
> hbase(main):003:0> list_replicated_tables "dn"
> TABLE:COLUMNFAMILY ReplicationType
>
> t1_dn:cf1 GLOBAL
>
> t2_dn:cf2 GLOBAL
>
> 2 row(s) in 0.0280 seconds
> {code}
> -- The original JIRA description, keep as the history of
> discussion ---
> This jira is to provide a hbase shell command which can give user can
> overview of the tables/columnfamilies currently being replicated. The
> information will help system administrator for design and planning, and also
> help application programmer to know which tables/columns should be
> watchout(for example, not to modify a replicated columnfamily on the slave
> cluster)
> Currently there is no easy way to tell which table(s)/columnfamily(ies)
> replicated from or to a particular cluster.
>
> On Master Cluster, an indirect method can be used by combining two steps: 1)
> $describe 'usertable' and 2) $list_peers to map the REPLICATION_SCOPE to
> target(aka slave) cluster
>
> On slave cluster, this is no existing API/methods to list all the tables
> replicated to this cluster.
> Here is an example, and prototype for Master cluster
> {code: title=hbase shell command:list_replicated_tables |borderStyle=solid}
> hbase(main):001:0> list_replicated_tables
> TABLE COLUMNFAMILY TARGET_CLUSTER
> scores coursehdtest017.svl.ibm.com:2181:/hbase
> t3_dn cf1 hdtest017.svl.ibm.com:2181:/hbase
> usertable familyhdtest017.svl.ibm.com:2181:/hbase
> 3 row(s) in 0.3380 seconds
> {code}
> -- end of original description
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8663) a HBase Shell command to list the tables replicated from current cluster
[
https://issues.apache.org/jira/browse/HBASE-8663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13727339#comment-13727339
]
Hadoop QA commented on HBASE-8663:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12595531/HBASE-8663-trunk-v1.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 hadoop1.0{color}. The patch compiles against the hadoop
1.0 profile.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 1
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:red}-1 site{color}. The patch appears to cause mvn site goal to
fail.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.client.TestAdmin
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/6564//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6564//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6564//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6564//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6564//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6564//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6564//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6564//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6564//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/6564//console
This message is automatically generated.
> a HBase Shell command to list the tables replicated from current cluster
>
>
> Key: HBASE-8663
> URL: https://issues.apache.org/jira/browse/HBASE-8663
> Project: HBase
> Issue Type: New Feature
> Components: Replication, shell
> Environment: clusters setup as Master and Slave for replication of
> tables
>Reporter: Demai Ni
>Assignee: Demai Ni
>Priority: Critical
> Attachments: HBASE-8663.PATCH, HBASE-8663-trunk-v0.patch,
> HBASE-8663-trunk-v1.patch, HBASE-8663-v2.PATCH
>
>
> Thanks for the discussion and very good suggestions,I'd reduce the scope of
> this jira to only display the tables replicated from current cluster. Since
> currently no good(accurate and consistent) way to flag a table on slave
> cluster, this jira will not cover such scenario. Instead, the patch will be
> flexible enough to adapt such scenario and a follow up JIRA will be opened to
> address such situation.
> The shell command and output will be like. Since all replication is 'global',
> so no need to display the cluster name here. In the future, the command will
> be extended for other scenarios, such as 1) replicated only to selected peers
> or 2) indicate table:colfam on slave side
> {code: title=hbase shell command:list_replicated_tables |borderStyle=solid}
> hbase(main):001:0> list_replicated_tables
> TABLE:COLUMNFAMILY ReplicationType
>
> t1_dn:cf1 GLOBAL
>
> t2_dn:
[jira] [Commented] (HBASE-6580) Deprecate HTablePool in favor of HConnection.getTable(...)
[ https://issues.apache.org/jira/browse/HBASE-6580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13727338#comment-13727338 ] stack commented on HBASE-6580: -- [~lhofhansl] This one going to go in? > Deprecate HTablePool in favor of HConnection.getTable(...) > -- > > Key: HBASE-6580 > URL: https://issues.apache.org/jira/browse/HBASE-6580 > Project: HBase > Issue Type: Improvement >Affects Versions: 0.94.6, 0.95.0 >Reporter: Lars Hofhansl >Assignee: Lars Hofhansl >Priority: Minor > Fix For: 0.98.0, 0.95.2, 0.94.11 > > Attachments: HBASE-6580_v1.patch, HBASE-6580_v2.patch > > > Update: > I now propose deprecating HTablePool and instead introduce a getTable method > on HConnection and allow HConnection to manage the ThreadPool. > Initial proposal: > Here I propose a very simple TablePool. > It could be called LightHTablePool (or something - if you have a better name). > Internally it would maintain an HConnection and an Executor service and each > invocation of getTable(...) would create a new HTable and close() would just > close it. > In testing I find this more light weight than HTablePool and easier to > monitor in terms of resources used. > It would hardly be more than a few dozen lines of code. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7183) print WARN message if hbase.replication.sizeOfLogQueue is too big
[ https://issues.apache.org/jira/browse/HBASE-7183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Webster updated HBASE-7183: --- Status: Patch Available (was: Open) > print WARN message if hbase.replication.sizeOfLogQueue is too big > - > > Key: HBASE-7183 > URL: https://issues.apache.org/jira/browse/HBASE-7183 > Project: HBase > Issue Type: Improvement > Components: Replication >Reporter: Sho Shimauchi > Labels: noob > Attachments: HBASE_7183.patch > > > A metric hbase.replication.sizeOfLogQueue may become big when replication is > delaying. > It would be useful if HBase prints WARN log which tells > hbase.replication.sizeOfLogQueue is too big. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-7183) print WARN message if hbase.replication.sizeOfLogQueue is too big
[ https://issues.apache.org/jira/browse/HBASE-7183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13727329#comment-13727329 ] Michael Webster commented on HBASE-7183: Hi, Here is a patch that checks the queue size against a configurable property, replication.source.log.queue.warn, each time a new log is enqueued. Per the above comments, the default threshold for printing the log message is 2. > print WARN message if hbase.replication.sizeOfLogQueue is too big > - > > Key: HBASE-7183 > URL: https://issues.apache.org/jira/browse/HBASE-7183 > Project: HBase > Issue Type: Improvement > Components: Replication >Reporter: Sho Shimauchi > Labels: noob > Attachments: HBASE_7183.patch > > > A metric hbase.replication.sizeOfLogQueue may become big when replication is > delaying. > It would be useful if HBase prints WARN log which tells > hbase.replication.sizeOfLogQueue is too big. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-7183) print WARN message if hbase.replication.sizeOfLogQueue is too big
[ https://issues.apache.org/jira/browse/HBASE-7183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Webster updated HBASE-7183: --- Attachment: HBASE_7183.patch > print WARN message if hbase.replication.sizeOfLogQueue is too big > - > > Key: HBASE-7183 > URL: https://issues.apache.org/jira/browse/HBASE-7183 > Project: HBase > Issue Type: Improvement > Components: Replication >Reporter: Sho Shimauchi > Labels: noob > Attachments: HBASE_7183.patch > > > A metric hbase.replication.sizeOfLogQueue may become big when replication is > delaying. > It would be useful if HBase prints WARN log which tells > hbase.replication.sizeOfLogQueue is too big. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HBASE-9085) Integration Tests fails because of bug in teardown phase where the cluster state is not being restored properly.
[
https://issues.apache.org/jira/browse/HBASE-9085?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Enis Soztutar resolved HBASE-9085.
--
Resolution: Fixed
Hadoop Flags: Reviewed
I've committed this to 0.94,0.95 and trunk. Thanks for the patch gautam.
> Integration Tests fails because of bug in teardown phase where the cluster
> state is not being restored properly.
>
>
> Key: HBASE-9085
> URL: https://issues.apache.org/jira/browse/HBASE-9085
> Project: HBase
> Issue Type: Bug
> Components: test
>Affects Versions: 0.95.0, 0.94.9, 0.94.10
>Reporter: gautam
>Assignee: gautam
> Fix For: 0.98.0, 0.95.2, 0.94.11
>
> Attachments: HBASE-9085.patch._0.94, HBASE-9085.patch._0.95_or_trunk
>
>
> I was running the following test over a Distributed Cluster:
> bin/hbase org.apache.hadoop.hbase.IntegrationTestsDriver
> IntegrationTestDataIngestSlowDeterministic
> The IntegrationTestingUtility.restoreCluster() is called in the teardown
> phase of the test.
> For a distributed cluster, it ends up calling
> DistributedHBaseCluster.restoreClusterStatus, which does the task
> of restoring the cluster back to original state.
> The restore steps done here, does not solve one specific case:
> When the initial HBase Master is currently down, and the current HBase Master
> is different from the initial one.
> You get into this flow:
> //check whether current master has changed
> if (!ServerName.isSameHostnameAndPort(initial.getMaster(),
> current.getMaster())) {
> .
> }
> In the above code path, the current backup masters are stopped, and the
> current active master is also stopped.
> At this point, for the aforementioned usecase, none of the Hbase Masters
> would be available, hence the subsequent
> attempts to do any operation over the cluster would fail, resulting in Test
> Failure.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-9106) Do not fail TestAcidGuarantees for exceptions on table flush
[
https://issues.apache.org/jira/browse/HBASE-9106?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Enis Soztutar updated HBASE-9106:
-
Resolution: Fixed
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
Committed this. Thanks Stack for review.
> Do not fail TestAcidGuarantees for exceptions on table flush
>
>
> Key: HBASE-9106
> URL: https://issues.apache.org/jira/browse/HBASE-9106
> Project: HBase
> Issue Type: Test
>Reporter: Enis Soztutar
>Assignee: Enis Soztutar
> Fix For: 0.98.0, 0.95.2, 0.94.11
>
> Attachments: hbase-9106-0.94_v1.patch, hbase-9106_v1.patch
>
>
> TestAcidGuarantees failed in one run due to a flush taking longer than
> >60sec, with:
> {code}
> HBaseClient$CallTimeoutException: Call id=152, waitTime=60007,
> rpcTimetout=6
> {code}
> We should ignore the exceptions coming from table flushes, since they are not
> essential to the test.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HBASE-9107) [0.94] Backport HBASE-6950 TestAcidGuarantees system test now flushes too aggressively to 0.94
[ https://issues.apache.org/jira/browse/HBASE-9107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Enis Soztutar resolved HBASE-9107. -- Resolution: Fixed Hadoop Flags: Reviewed Committed this. Thanks Andrew. > [0.94] Backport HBASE-6950 TestAcidGuarantees system test now flushes too > aggressively to 0.94 > -- > > Key: HBASE-9107 > URL: https://issues.apache.org/jira/browse/HBASE-9107 > Project: HBase > Issue Type: Test >Reporter: Enis Soztutar >Assignee: Enis Soztutar > Fix For: 0.94.11 > > Attachments: hbase-9107_v1.patch > > > As the description says. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9075) [0.94] Backport HBASE-5760 Unit tests should write only under /target to 0.94
[ https://issues.apache.org/jira/browse/HBASE-9075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13727308#comment-13727308 ] Enis Soztutar commented on HBASE-9075: -- bq. Why does it remove testGetSetOfHTD? That tests a deprecated method, and the test has been removed in 0.95. That method creates a new HBaseConfiguration, so the test actually has to write to /tmp/hbase-$username. We probably do not want this. > [0.94] Backport HBASE-5760 Unit tests should write only under /target to 0.94 > - > > Key: HBASE-9075 > URL: https://issues.apache.org/jira/browse/HBASE-9075 > Project: HBase > Issue Type: Test > Components: test >Reporter: Enis Soztutar >Assignee: Enis Soztutar > Fix For: 0.94.11 > > Attachments: hbase-9075_v1.patch > > > Backporting HBASE-5760 is a good idea. 0.94 tests mess up the root level > directory a lot. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8670) [0.94] Backport HBASE-8449,HBASE-8204 and HBASE-8699 to 0.94 (Refactor recoverLease retries and pauses)
[ https://issues.apache.org/jira/browse/HBASE-8670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Enis Soztutar updated HBASE-8670: - Hadoop Flags: Reviewed Status: Patch Available (was: Open) I've committed this. Thanks for the reviews. > [0.94] Backport HBASE-8449,HBASE-8204 and HBASE-8699 to 0.94 (Refactor > recoverLease retries and pauses) > --- > > Key: HBASE-8670 > URL: https://issues.apache.org/jira/browse/HBASE-8670 > Project: HBase > Issue Type: Bug > Components: Filesystem Integration, master, wal >Reporter: Enis Soztutar >Assignee: Enis Soztutar > Fix For: 0.94.11 > > Attachments: hbase-8670_v1.patch, hbase-8670_v2.patch > > > Some history: > Up until 0.94.8, Hbase did not check the result of recoverLease() call, but > things kind of worked since we are checking for 0-length files in distributed > log split tasks from region servers. If lease recovery is not finished, the > log file will report 0 length, and the task will fail, and master will then > re-call recoverLease() and reassign the task. This scheme might fail for log > files that are larger than 1 hdfs block though. > In 0.94.8, we committed (HBASE-8354, which is backport of HBASE-7878) and > later increased the sleep time to 4 secs in HBASE-8389. > However, the proper solution arrived in trunk in HBASE-8449 which uses a > backoff sleep policy + isFileClosed() api. We should backport this patch to > 0.94 as well. > isFileClosed() is released in Hadoop 1.2.0 (HDFS-4774) and 2.0.5(HDFS-4525). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6580) Deprecate HTablePool in favor of HConnection.getTable(...)
[ https://issues.apache.org/jira/browse/HBASE-6580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl updated HBASE-6580: - Description: Update: I now propose deprecating HTablePool and instead introduce a getTable method on HConnection and allow HConnection to manage the ThreadPool. Initial proposal: Here I propose a very simple TablePool. It could be called LightHTablePool (or something - if you have a better name). Internally it would maintain an HConnection and an Executor service and each invocation of getTable(...) would create a new HTable and close() would just close it. In testing I find this more light weight than HTablePool and easier to monitor in terms of resources used. It would hardly be more than a few dozen lines of code. was: Update: I now propose deprecating HTablePool and instead introduce a getTable Initial proposal: Here I propose a very simple TablePool. It could be called LightHTablePool (or something - if you have a better name). Internally it would maintain an HConnection and an Executor service and each invocation of getTable(...) would create a new HTable and close() would just close it. In testing I find this more light weight than HTablePool and easier to monitor in terms of resources used. It would hardly be more than a few dozen lines of code. > Deprecate HTablePool in favor of HConnection.getTable(...) > -- > > Key: HBASE-6580 > URL: https://issues.apache.org/jira/browse/HBASE-6580 > Project: HBase > Issue Type: Improvement >Affects Versions: 0.94.6, 0.95.0 >Reporter: Lars Hofhansl >Assignee: Lars Hofhansl >Priority: Minor > Fix For: 0.98.0, 0.95.2, 0.94.11 > > Attachments: HBASE-6580_v1.patch, HBASE-6580_v2.patch > > > Update: > I now propose deprecating HTablePool and instead introduce a getTable method > on HConnection and allow HConnection to manage the ThreadPool. > Initial proposal: > Here I propose a very simple TablePool. > It could be called LightHTablePool (or something - if you have a better name). > Internally it would maintain an HConnection and an Executor service and each > invocation of getTable(...) would create a new HTable and close() would just > close it. > In testing I find this more light weight than HTablePool and easier to > monitor in terms of resources used. > It would hardly be more than a few dozen lines of code. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6580) Deprecate HTablePool in favor of HConnection.getTable(...)
[ https://issues.apache.org/jira/browse/HBASE-6580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl updated HBASE-6580: - Summary: Deprecate HTablePool in favor of HConnection.getTable(...) (was: Deprecate HTablePool in favor of HConnection.getHTable(...)) > Deprecate HTablePool in favor of HConnection.getTable(...) > -- > > Key: HBASE-6580 > URL: https://issues.apache.org/jira/browse/HBASE-6580 > Project: HBase > Issue Type: Improvement >Affects Versions: 0.94.6, 0.95.0 >Reporter: Lars Hofhansl >Assignee: Lars Hofhansl >Priority: Minor > Fix For: 0.98.0, 0.95.2, 0.94.11 > > Attachments: HBASE-6580_v1.patch, HBASE-6580_v2.patch > > > Update: > I now propose deprecating HTablePool and instead introduce a getTable > Initial proposal: > Here I propose a very simple TablePool. > It could be called LightHTablePool (or something - if you have a better name). > Internally it would maintain an HConnection and an Executor service and each > invocation of getTable(...) would create a new HTable and close() would just > close it. > In testing I find this more light weight than HTablePool and easier to > monitor in terms of resources used. > It would hardly be more than a few dozen lines of code. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6580) Deprecate HTablePool in favor of HConnection.getHTable(...)
[ https://issues.apache.org/jira/browse/HBASE-6580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl updated HBASE-6580: - Description: Update: I now propose deprecating HTablePool and instead introduce a getTable Initial proposal: Here I propose a very simple TablePool. It could be called LightHTablePool (or something - if you have a better name). Internally it would maintain an HConnection and an Executor service and each invocation of getTable(...) would create a new HTable and close() would just close it. In testing I find this more light weight than HTablePool and easier to monitor in terms of resources used. It would hardly be more than a few dozen lines of code. was: Here I propose a very simple TablePool. It could be called LightHTablePool (or something - if you have a better name). Internally it would maintain an HConnection and an Executor service and each invocation of getTable(...) would create a new HTable and close() would just close it. In testing I find this more light weight than HTablePool and easier to monitor in terms of resources used. It would hardly be more than a few dozen lines of code. > Deprecate HTablePool in favor of HConnection.getHTable(...) > --- > > Key: HBASE-6580 > URL: https://issues.apache.org/jira/browse/HBASE-6580 > Project: HBase > Issue Type: Improvement >Affects Versions: 0.94.6, 0.95.0 >Reporter: Lars Hofhansl >Assignee: Lars Hofhansl >Priority: Minor > Fix For: 0.98.0, 0.95.2, 0.94.11 > > Attachments: HBASE-6580_v1.patch, HBASE-6580_v2.patch > > > Update: > I now propose deprecating HTablePool and instead introduce a getTable > Initial proposal: > Here I propose a very simple TablePool. > It could be called LightHTablePool (or something - if you have a better name). > Internally it would maintain an HConnection and an Executor service and each > invocation of getTable(...) would create a new HTable and close() would just > close it. > In testing I find this more light weight than HTablePool and easier to > monitor in terms of resources used. > It would hardly be more than a few dozen lines of code. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-6580) Deprecate HTablePool in favor of HConnection.getHTable(...)
[ https://issues.apache.org/jira/browse/HBASE-6580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl updated HBASE-6580: - Summary: Deprecate HTablePool in favor of HConnection.getHTable(...) (was: New HTable pool, based on HBase(byte[], HConnection, ExecutorService) constructor) > Deprecate HTablePool in favor of HConnection.getHTable(...) > --- > > Key: HBASE-6580 > URL: https://issues.apache.org/jira/browse/HBASE-6580 > Project: HBase > Issue Type: Improvement >Affects Versions: 0.94.6, 0.95.0 >Reporter: Lars Hofhansl >Assignee: Lars Hofhansl >Priority: Minor > Fix For: 0.98.0, 0.95.2, 0.94.11 > > Attachments: HBASE-6580_v1.patch, HBASE-6580_v2.patch > > > Here I propose a very simple TablePool. > It could be called LightHTablePool (or something - if you have a better name). > Internally it would maintain an HConnection and an Executor service and each > invocation of getTable(...) would create a new HTable and close() would just > close it. > In testing I find this more light weight than HTablePool and easier to > monitor in terms of resources used. > It would hardly be more than a few dozen lines of code. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8619) [HBase Client]: Improve client to retry if master is down when requesting getHTableDescriptor
[
https://issues.apache.org/jira/browse/HBASE-8619?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lars Hofhansl updated HBASE-8619:
-
Fix Version/s: (was: 0.94.11)
0.94.12
Any update?
Pushing to 0.94.12
> [HBase Client]: Improve client to retry if master is down when requesting
> getHTableDescriptor
> -
>
> Key: HBASE-8619
> URL: https://issues.apache.org/jira/browse/HBASE-8619
> Project: HBase
> Issue Type: Bug
> Components: Client
>Affects Versions: 0.94.8
>Reporter: Aleksandr Shulman
>Assignee: Elliott Clark
>Priority: Minor
> Fix For: 0.94.12
>
>
> In our rolling upgrade testing, running ImportTsv failed in the job
> submission phase when the master was down. This was when the master was
> failing over to the backup master. In this case, a retry would have been
> helpful and made sure that the job would get submitted.
> A good solution would be to refresh the master information before placing the
> call to getHTableDescriptor.
> Command:
> {code} sudo -u hbase hbase org.apache.hadoop.hbase.mapreduce.ImportTsv
> -Dimporttsv.columns=HBASE_ROW_KEY,c1,c2,c3
> -Dimporttsv.bulk.output=/user/hbase/storeFiles2_2/import2_table1369439156
> import2_table1369439156 /user/hbase/tsv2{code}
> Here is the stack trace:
> {code} 13/05/24 16:55:49 INFO compress.CodecPool: Got brand-new compressor
> [.deflate]
> 16:45:44 Exception in thread "main"
> java.lang.reflect.UndeclaredThrowableException
> 16:45:44 at $Proxy7.getHTableDescriptors(Unknown Source)
> 16:45:44 at
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getHTableDescriptor(HConnectionManager.java:1861)
> 16:45:44 at
> org.apache.hadoop.hbase.client.HTable.getTableDescriptor(HTable.java:440)
> 16:45:44 at
> org.apache.hadoop.hbase.mapreduce.HFileOutputFormat.configureCompression(HFileOutputFormat.java:458)
> 16:45:44 at
> org.apache.hadoop.hbase.mapreduce.HFileOutputFormat.configureIncrementalLoad(HFileOutputFormat.java:375)
> 16:45:44 at
> org.apache.hadoop.hbase.mapreduce.ImportTsv.createSubmittableJob(ImportTsv.java:280)
> 16:45:44 at
> org.apache.hadoop.hbase.mapreduce.ImportTsv.main(ImportTsv.java:424)
> 16:45:44 Caused by: java.io.IOException: Call to
> hbase-rolling-6.ent.cloudera.com/10.20.186.99:22001 failed on local
> exception: java.io.EOFException
> 16:45:44 at
> org.apache.hadoop.hbase.ipc.HBaseClient.wrapException(HBaseClient.java:1030)
> 16:45:44 at
> org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:999)
> 16:45:44 at
> org.apache.hadoop.hbase.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:86)
> 16:45:44 ... 7 more
> 16:45:44 Caused by: java.io.EOFException
> 16:45:44 at java.io.DataInputStream.readInt(DataInputStream.java:375)
> 16:45:44 at
> org.apache.hadoop.hbase.ipc.HBaseClient$Connection.receiveResponse(HBaseClient.java:646)
> 16:45:44 at
> org.apache.hadoop.hbase.ipc.HBaseClient$Connection.run(HBaseClient.java:580){code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8773) Can be setup the COMPRESSION base on HTable in meta or user set in Configuration
[ https://issues.apache.org/jira/browse/HBASE-8773?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl updated HBASE-8773: - Fix Version/s: (was: 0.94.11) 0.94.12 No movement. Pushing to 0.94.12. > Can be setup the COMPRESSION base on HTable in meta or user set in > Configuration > > > Key: HBASE-8773 > URL: https://issues.apache.org/jira/browse/HBASE-8773 > Project: HBase > Issue Type: New Feature > Components: HFile >Affects Versions: 0.94.8 >Reporter: jay wong >Priority: Minor > Fix For: 0.94.12 > > Attachments: HBASE-8773.patch > > > when I want create HFile with the ImportTsv. I found that if i set the > compression in the Configuration or not, It's always invalid。 > It because of the method 'configureIncrementalLoad' in HFileOutputFormat will > set the compression with the HTable in meta. So if add a configuration to > switch use set compression with HTable or Not -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8771) ensure replication_scope's value is either local(0) or global(1)
[
https://issues.apache.org/jira/browse/HBASE-8771?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lars Hofhansl updated HBASE-8771:
-
Fix Version/s: (was: 0.94.11)
Unscheduling this from 0.94. We can do this in 0.95+
> ensure replication_scope's value is either local(0) or global(1)
>
>
> Key: HBASE-8771
> URL: https://issues.apache.org/jira/browse/HBASE-8771
> Project: HBase
> Issue Type: Bug
> Components: Replication
>Affects Versions: 0.94.8
>Reporter: Demai Ni
>Assignee: Demai Ni
>Priority: Minor
> Attachments: HBASE-8771-0.94.8-v0.patch
>
>
> For replication_scope, only two values are meaningful:
> {code}
> public static final int REPLICATION_SCOPE_LOCAL = 0;
> public static final int REPLICATION_SCOPE_GLOBAL = 1;
> {code}
> However, there is no checking for that, so currently user can set it to any
> integer value. And all non-zero value will be treated as 1(GLOBAL).
> This jira is to add a checking in HColumnDescriptor#setScope() so that only 0
> and 1 will be accept during create_table or alter_table.
> In the future, we can leverage replication_scope to store for info. For
> example:
> -1: A columnfam is replicated from another cluster in MASTER_SLAVE setup (i.e
> readonly)
> 2 : A columnfam is set MASTER_MASTER
> Probably a major improve JIRA is needed for the future usage. It will be good
> to ensure the scope value at this moment.
> {code:title=Testing|borderStyle=solid}
> hbase(main):002:0> create 't1_dn',{NAME=>'cf1',REPLICATION_SCOPE=>2}
> ERROR: java.lang.IllegalArgumentException: Replication Scope must be either
> 0(local) or 1(global)
> ...
> hbase(main):004:0> alter 't1_dn',{NAME=>'cf1',REPLICATION_SCOPE=>-1}
> ERROR: java.lang.IllegalArgumentException: Replication Scope must be either
> 0(local) or 1(global)
> ...
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8816) Add support of loading multiple tables into LoadTestTool
[ https://issues.apache.org/jira/browse/HBASE-8816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13727299#comment-13727299 ] Lars Hofhansl commented on HBASE-8816: -- Wanna commit? Otherwise I can push to 0.94.12. > Add support of loading multiple tables into LoadTestTool > > > Key: HBASE-8816 > URL: https://issues.apache.org/jira/browse/HBASE-8816 > Project: HBase > Issue Type: Test > Components: test >Affects Versions: 0.94.9 >Reporter: Jeffrey Zhong >Assignee: Jeffrey Zhong >Priority: Minor > Fix For: 0.98.0, 0.95.2, 0.94.11 > > Attachments: hbase-8816.patch, hbase-8816-v1.patch, > hbase-8816-v2.patch > > > Introducing an optional parameter 'concurrent_factor' into LoadTestTool. When > it's specified with positive integer n, LoadTestTool will load n tables > parallely. -tn parameter value becomes table name prefix. Tables are created > with name in format _1..._n. A sample command line "-tn test > -concurrent_factor 2" will create & load tables:"test_1" and "test_2" > The motivation is to add a handy way to load multiple tables concurrently. In > addition, we could use this option to test resource leakage of long running > clients. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8565) stop-hbase.sh clean up: backup master
[
https://issues.apache.org/jira/browse/HBASE-8565?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lars Hofhansl updated HBASE-8565:
-
Fix Version/s: (was: 0.94.11)
0.94.12
Make a patch, Jerry?
Pushing to 0.94.12.
> stop-hbase.sh clean up: backup master
> -
>
> Key: HBASE-8565
> URL: https://issues.apache.org/jira/browse/HBASE-8565
> Project: HBase
> Issue Type: Bug
> Components: master, scripts
>Affects Versions: 0.94.7, 0.95.0
>Reporter: Jerry He
>Priority: Minor
> Fix For: 0.98.0, 0.95.2, 0.94.12
>
>
> In stop-hbase.sh:
> {code}
> # TODO: store backup masters in ZooKeeper and have the primary send them a
> shutdown message
> # stop any backup masters
> "$bin"/hbase-daemons.sh --config "${HBASE_CONF_DIR}" \
> --hosts "${HBASE_BACKUP_MASTERS}" stop master-backup
> {code}
> After HBASE-5213, stop-hbase.sh -> hbase master stop will bring up the backup
> master too via the cluster status znode.
> We should not need the above code anymore.
> Another issue happens when the current master died and the backup master
> became the active master.
> {code}
> nohup nice -n ${HBASE_NICENESS:-0} "$HBASE_HOME"/bin/hbase \
>--config "${HBASE_CONF_DIR}" \
>master stop "$@" > "$logout" 2>&1 < /dev/null &
> waitForProcessEnd `cat $pid` 'stop-master-command'
> {code}
> We can still issue 'hbase-stop.sh' from the old master.
> stop-hbase.sh -> hbase master stop -> look for active master -> request
> shutdown
> This process still works.
> But the waitForProcessEnd statement will not work since the local master pid
> is not relevant anymore.
> What is the best way in the this case?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8224) Publish hbase build against h1 and h2 adding '-hadoop1' or '-hadoop2' to version string
[
https://issues.apache.org/jira/browse/HBASE-8224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13727300#comment-13727300
]
Hadoop QA commented on HBASE-8224:
--
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12595528/8224v5.txt
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified tests.
{color:green}+1 hadoop1.0{color}. The patch compiles against the hadoop
1.0 profile.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/6563//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6563//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6563//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6563//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6563//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6563//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6563//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6563//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6563//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/6563//console
This message is automatically generated.
> Publish hbase build against h1 and h2 adding '-hadoop1' or '-hadoop2' to
> version string
> ---
>
> Key: HBASE-8224
> URL: https://issues.apache.org/jira/browse/HBASE-8224
> Project: HBase
> Issue Type: Task
>Reporter: stack
>Assignee: stack
>Priority: Blocker
> Fix For: 0.98.0, 0.95.2
>
> Attachments: 8224-adding.classifiers.txt, 8224.gen.script.txt,
> 8224.gen.scriptv3.txt, 8224.gen.scriptv3.txt, 8224v5.txt,
> hbase-8224-proto1.patch
>
>
> So we can publish both the hadoop1 and the hadoop2 jars to a maven
> repository, and so we can publish two packages, one for hadoop1 and one for
> hadoop2, given how maven works, our only alternative (to the best of my
> knowledge and after consulting others) is by amending the version string to
> include hadoop1 or hadoop2.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9087) Handlers being blocked during reads
[ https://issues.apache.org/jira/browse/HBASE-9087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13727298#comment-13727298 ] Lars Hofhansl commented on HBASE-9087: -- I know Elliott asked that on the mailing list, just to be sure, are you closing your scanners on the client? > Handlers being blocked during reads > --- > > Key: HBASE-9087 > URL: https://issues.apache.org/jira/browse/HBASE-9087 > Project: HBase > Issue Type: Bug > Components: Performance >Affects Versions: 0.94.7, 0.95.1 >Reporter: Pablo Medina >Assignee: Elliott Clark > Fix For: 0.98.0, 0.95.2, 0.94.12 > > Attachments: HBASE-9087-0.patch, HBASE-9087-1.patch > > > I'm having a lot of handlers (90 - 300 aprox) being blocked when reading > rows. They are blocked during changedReaderObserver registration. > Lars Hofhansl suggests to change the implementation of changedReaderObserver > from CopyOnWriteList to ConcurrentHashMap. > Here is a stack trace: > "IPC Server handler 99 on 60020" daemon prio=10 tid=0x41c84000 > nid=0x2244 waiting on condition [0x7ff51fefd000] >java.lang.Thread.State: WAITING (parking) > at sun.misc.Unsafe.park(Native Method) > - parking to wait for <0xc5c13ae8> (a > java.util.concurrent.locks.ReentrantLock$NonfairSync) > at java.util.concurrent.locks.LockSupport.park(LockSupport.java:156) > at > java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:811) > at > java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:842) > at > java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1178) > at > java.util.concurrent.locks.ReentrantLock$NonfairSync.lock(ReentrantLock.java:186) > at > java.util.concurrent.locks.ReentrantLock.lock(ReentrantLock.java:262) > at > java.util.concurrent.CopyOnWriteArrayList.addIfAbsent(CopyOnWriteArrayList.java:553) > at > java.util.concurrent.CopyOnWriteArraySet.add(CopyOnWriteArraySet.java:221) > at > org.apache.hadoop.hbase.regionserver.Store.addChangedReaderObserver(Store.java:1085) > at > org.apache.hadoop.hbase.regionserver.StoreScanner.(StoreScanner.java:138) > at > org.apache.hadoop.hbase.regionserver.Store.getScanner(Store.java:2077) > at > org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.(HRegion.java:3755) > at > org.apache.hadoop.hbase.regionserver.HRegion.instantiateRegionScanner(HRegion.java:1804) > at > org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:1796) > at > org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:1771) > at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:4776) > at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:4750) > at > org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:2152) > at > org.apache.hadoop.hbase.regionserver.HRegionServer.multi(HRegionServer.java:3700) > at sun.reflect.GeneratedMethodAccessor26.invoke(Unknown Source) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > at java.lang.reflect.Method.invoke(Method.java:597) > at > org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:320) > at > org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1426) > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9086) Add some options to improve count performance
[ https://issues.apache.org/jira/browse/HBASE-9086?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13727293#comment-13727293 ] Lars Hofhansl commented on HBASE-9086: -- Need to test this carefully (correctness and performance). FilterList is a bit dicey with mutating filters (such as KeyOnlyFilter). > Add some options to improve count performance > - > > Key: HBASE-9086 > URL: https://issues.apache.org/jira/browse/HBASE-9086 > Project: HBase > Issue Type: Wish > Components: shell >Affects Versions: 0.94.2 >Reporter: Cheney Sun > Attachments: HBase-9086.patch, HBase-9086_v0.2.patch > > > The current count command in HBase shell is quite slow if the row size is > very big (100+kB each). It would be helpful to provide some option to specify > the column to count, which could give user a chance to reduce the data volume > to scan. > IMHO, only count the row key would be the ideal solution. Not sure how > difficult to implement it. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9087) Handlers being blocked during reads
[ https://issues.apache.org/jira/browse/HBASE-9087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13727291#comment-13727291 ] Lars Hofhansl commented on HBASE-9087: -- You'll have to build it yourself. If that is an issue, I can build one for you. > Handlers being blocked during reads > --- > > Key: HBASE-9087 > URL: https://issues.apache.org/jira/browse/HBASE-9087 > Project: HBase > Issue Type: Bug > Components: Performance >Affects Versions: 0.94.7, 0.95.1 >Reporter: Pablo Medina >Assignee: Elliott Clark > Fix For: 0.98.0, 0.95.2, 0.94.12 > > Attachments: HBASE-9087-0.patch, HBASE-9087-1.patch > > > I'm having a lot of handlers (90 - 300 aprox) being blocked when reading > rows. They are blocked during changedReaderObserver registration. > Lars Hofhansl suggests to change the implementation of changedReaderObserver > from CopyOnWriteList to ConcurrentHashMap. > Here is a stack trace: > "IPC Server handler 99 on 60020" daemon prio=10 tid=0x41c84000 > nid=0x2244 waiting on condition [0x7ff51fefd000] >java.lang.Thread.State: WAITING (parking) > at sun.misc.Unsafe.park(Native Method) > - parking to wait for <0xc5c13ae8> (a > java.util.concurrent.locks.ReentrantLock$NonfairSync) > at java.util.concurrent.locks.LockSupport.park(LockSupport.java:156) > at > java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:811) > at > java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:842) > at > java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1178) > at > java.util.concurrent.locks.ReentrantLock$NonfairSync.lock(ReentrantLock.java:186) > at > java.util.concurrent.locks.ReentrantLock.lock(ReentrantLock.java:262) > at > java.util.concurrent.CopyOnWriteArrayList.addIfAbsent(CopyOnWriteArrayList.java:553) > at > java.util.concurrent.CopyOnWriteArraySet.add(CopyOnWriteArraySet.java:221) > at > org.apache.hadoop.hbase.regionserver.Store.addChangedReaderObserver(Store.java:1085) > at > org.apache.hadoop.hbase.regionserver.StoreScanner.(StoreScanner.java:138) > at > org.apache.hadoop.hbase.regionserver.Store.getScanner(Store.java:2077) > at > org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.(HRegion.java:3755) > at > org.apache.hadoop.hbase.regionserver.HRegion.instantiateRegionScanner(HRegion.java:1804) > at > org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:1796) > at > org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:1771) > at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:4776) > at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:4750) > at > org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:2152) > at > org.apache.hadoop.hbase.regionserver.HRegionServer.multi(HRegionServer.java:3700) > at sun.reflect.GeneratedMethodAccessor26.invoke(Unknown Source) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > at java.lang.reflect.Method.invoke(Method.java:597) > at > org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:320) > at > org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1426) > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8408) Implement namespace
[ https://issues.apache.org/jira/browse/HBASE-8408?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13727285#comment-13727285 ] stack commented on HBASE-8408: -- Good. Keep fixing the javadoc warnings, zombie tests (there should be none in trunk now [~toffer]) and take a look at the findbugs failings. The site failure is likely not your issue. I can see the patch going in w/ findbugs and javadoc issues for solving in a subsequent patch. Tests should pass. Use hadoopqa for checking by submitting new patches. Going over to review on rb now... > Implement namespace > --- > > Key: HBASE-8408 > URL: https://issues.apache.org/jira/browse/HBASE-8408 > Project: HBase > Issue Type: Sub-task >Reporter: Francis Liu >Assignee: Francis Liu > Attachments: HBASE-8015_11.patch, HBASE-8015_1.patch, > HBASE-8015_2.patch, HBASE-8015_3.patch, HBASE-8015_4.patch, > HBASE-8015_5.patch, HBASE-8015_6.patch, HBASE-8015_7.patch, > HBASE-8015_8.patch, HBASE-8015_9.patch, HBASE-8015.patch, > TestNamespaceMigration.tgz, TestNamespaceUpgrade.tgz > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8706) Some improvement in snapshot
[
https://issues.apache.org/jira/browse/HBASE-8706?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13727271#comment-13727271
]
Hudson commented on HBASE-8706:
---
SUCCESS: Integrated in HBase-0.94 #1087 (See
[https://builds.apache.org/job/HBase-0.94/1087/])
HBASE-9029 Backport HBASE-8706 Some improvement in snapshot to 0.94 (Jerry He,
original patch by Matteo) (larsh: rev 1509512)
*
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/snapshot/SnapshotManager.java
*
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/snapshot/TakeSnapshotHandler.java
*
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/procedure/ProcedureCoordinator.java
*
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/procedure/ProcedureMember.java
*
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/procedure/Subprocedure.java
*
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/snapshot/RegionServerSnapshotManager.java
*
/hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/procedure/TestProcedureCoordinator.java
*
/hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/procedure/TestProcedureMember.java
*
/hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/procedure/TestZKProcedure.java
> Some improvement in snapshot
>
>
> Key: HBASE-8706
> URL: https://issues.apache.org/jira/browse/HBASE-8706
> Project: HBase
> Issue Type: Bug
> Components: snapshots
>Affects Versions: 0.94.8, 0.95.0
>Reporter: binlijin
>Assignee: Matteo Bertozzi
> Fix For: 0.98.0, 0.95.2
>
> Attachments: HBASE-8706-2.patch, HBASE-8706-3.patch,
> HBASE-8706.patch, HBASE-8706-v4.patch, HBASE-8706-v4.patch
>
>
> (1)timeout for Procedure can not be configured.
> {code}
> Procedure's timeout
> ProcedureCoordinator
> final static long TIMEOUT_MILLIS_DEFAULT = 6;
>createProcedure(ForeignExceptionDispatcher fed, String procName, byte[]
> procArgs,
> List expectedMembers) {
> // build the procedure
> return new Procedure(this, fed, WAKE_MILLIS_DEFAULT,
> TIMEOUT_MILLIS_DEFAULT,
> procName, procArgs, expectedMembers);
> }
> RegionServerSnapshotManager:
> /** Conf key for max time to keep threads in snapshot request pool waiting
> */
> public static final String SNAPSHOT_TIMEOUT_MILLIS_KEY =
> "hbase.snapshot.region.timeout";
> /** Keep threads alive in request pool for max of 60 seconds */
> public static final long SNAPSHOT_TIMEOUT_MILLIS_DEFAULT = 6;
> public Subprocedure buildSubprocedure(SnapshotDescription snapshot) {
> long timeoutMillis = conf.getLong(SNAPSHOT_TIMEOUT_MILLIS_KEY,
> SNAPSHOT_TIMEOUT_MILLIS_DEFAULT);
> case FLUSH:
> SnapshotSubprocedurePool taskManager =
> new SnapshotSubprocedurePool(rss.getServerName().toString(), conf);
> }
> {code}
> (2)TakeSnapshotHandler
> after snapshotRegions we should call monitor.rethrowException(); to check if
> there is exception and if there is we can skip the verifySnapshot
> (3)too much error message when error happened in some place.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9029) Backport HBASE-8706 Some improvement in snapshot to 0.94
[ https://issues.apache.org/jira/browse/HBASE-9029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13727272#comment-13727272 ] Hudson commented on HBASE-9029: --- SUCCESS: Integrated in HBase-0.94 #1087 (See [https://builds.apache.org/job/HBase-0.94/1087/]) HBASE-9029 Backport HBASE-8706 Some improvement in snapshot to 0.94 (Jerry He, original patch by Matteo) (larsh: rev 1509512) * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/snapshot/SnapshotManager.java * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/snapshot/TakeSnapshotHandler.java * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/procedure/ProcedureCoordinator.java * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/procedure/ProcedureMember.java * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/procedure/Subprocedure.java * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/snapshot/RegionServerSnapshotManager.java * /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/procedure/TestProcedureCoordinator.java * /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/procedure/TestProcedureMember.java * /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/procedure/TestZKProcedure.java > Backport HBASE-8706 Some improvement in snapshot to 0.94 > > > Key: HBASE-9029 > URL: https://issues.apache.org/jira/browse/HBASE-9029 > Project: HBase > Issue Type: Improvement > Components: snapshots >Affects Versions: 0.94.9 >Reporter: Jerry He >Assignee: Jerry He >Priority: Minor > Fix For: 0.94.11 > > Attachments: HBase-9029-0.94.patch > > > 'HBASE-8706 Some improvement in snapshot' has some good parameter tuning and > improvement for snapshot handling, making snapshot more robust. > It will nice to put it in 0.94. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9029) Backport HBASE-8706 Some improvement in snapshot to 0.94
[ https://issues.apache.org/jira/browse/HBASE-9029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13727251#comment-13727251 ] Hudson commented on HBASE-9029: --- SUCCESS: Integrated in HBase-0.94-security #239 (See [https://builds.apache.org/job/HBase-0.94-security/239/]) HBASE-9029 Backport HBASE-8706 Some improvement in snapshot to 0.94 (Jerry He, original patch by Matteo) (larsh: rev 1509512) * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/snapshot/SnapshotManager.java * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/snapshot/TakeSnapshotHandler.java * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/procedure/ProcedureCoordinator.java * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/procedure/ProcedureMember.java * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/procedure/Subprocedure.java * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/snapshot/RegionServerSnapshotManager.java * /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/procedure/TestProcedureCoordinator.java * /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/procedure/TestProcedureMember.java * /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/procedure/TestZKProcedure.java > Backport HBASE-8706 Some improvement in snapshot to 0.94 > > > Key: HBASE-9029 > URL: https://issues.apache.org/jira/browse/HBASE-9029 > Project: HBase > Issue Type: Improvement > Components: snapshots >Affects Versions: 0.94.9 >Reporter: Jerry He >Assignee: Jerry He >Priority: Minor > Fix For: 0.94.11 > > Attachments: HBase-9029-0.94.patch > > > 'HBASE-8706 Some improvement in snapshot' has some good parameter tuning and > improvement for snapshot handling, making snapshot more robust. > It will nice to put it in 0.94. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8706) Some improvement in snapshot
[
https://issues.apache.org/jira/browse/HBASE-8706?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13727250#comment-13727250
]
Hudson commented on HBASE-8706:
---
SUCCESS: Integrated in HBase-0.94-security #239 (See
[https://builds.apache.org/job/HBase-0.94-security/239/])
HBASE-9029 Backport HBASE-8706 Some improvement in snapshot to 0.94 (Jerry He,
original patch by Matteo) (larsh: rev 1509512)
*
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/snapshot/SnapshotManager.java
*
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/snapshot/TakeSnapshotHandler.java
*
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/procedure/ProcedureCoordinator.java
*
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/procedure/ProcedureMember.java
*
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/procedure/Subprocedure.java
*
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/snapshot/RegionServerSnapshotManager.java
*
/hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/procedure/TestProcedureCoordinator.java
*
/hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/procedure/TestProcedureMember.java
*
/hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/procedure/TestZKProcedure.java
> Some improvement in snapshot
>
>
> Key: HBASE-8706
> URL: https://issues.apache.org/jira/browse/HBASE-8706
> Project: HBase
> Issue Type: Bug
> Components: snapshots
>Affects Versions: 0.94.8, 0.95.0
>Reporter: binlijin
>Assignee: Matteo Bertozzi
> Fix For: 0.98.0, 0.95.2
>
> Attachments: HBASE-8706-2.patch, HBASE-8706-3.patch,
> HBASE-8706.patch, HBASE-8706-v4.patch, HBASE-8706-v4.patch
>
>
> (1)timeout for Procedure can not be configured.
> {code}
> Procedure's timeout
> ProcedureCoordinator
> final static long TIMEOUT_MILLIS_DEFAULT = 6;
>createProcedure(ForeignExceptionDispatcher fed, String procName, byte[]
> procArgs,
> List expectedMembers) {
> // build the procedure
> return new Procedure(this, fed, WAKE_MILLIS_DEFAULT,
> TIMEOUT_MILLIS_DEFAULT,
> procName, procArgs, expectedMembers);
> }
> RegionServerSnapshotManager:
> /** Conf key for max time to keep threads in snapshot request pool waiting
> */
> public static final String SNAPSHOT_TIMEOUT_MILLIS_KEY =
> "hbase.snapshot.region.timeout";
> /** Keep threads alive in request pool for max of 60 seconds */
> public static final long SNAPSHOT_TIMEOUT_MILLIS_DEFAULT = 6;
> public Subprocedure buildSubprocedure(SnapshotDescription snapshot) {
> long timeoutMillis = conf.getLong(SNAPSHOT_TIMEOUT_MILLIS_KEY,
> SNAPSHOT_TIMEOUT_MILLIS_DEFAULT);
> case FLUSH:
> SnapshotSubprocedurePool taskManager =
> new SnapshotSubprocedurePool(rss.getServerName().toString(), conf);
> }
> {code}
> (2)TakeSnapshotHandler
> after snapshotRegions we should call monitor.rethrowException(); to check if
> there is exception and if there is we can skip the verifySnapshot
> (3)too much error message when error happened in some place.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9079) FilterList getNextKeyHint skips rows that should be included in the results
[
https://issues.apache.org/jira/browse/HBASE-9079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13727245#comment-13727245
]
Hadoop QA commented on HBASE-9079:
--
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12595522/HBASE-9079-trunk.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified tests.
{color:green}+1 hadoop1.0{color}. The patch compiles against the hadoop
1.0 profile.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/6562//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6562//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6562//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6562//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6562//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6562//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6562//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6562//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6562//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/6562//console
This message is automatically generated.
> FilterList getNextKeyHint skips rows that should be included in the results
> ---
>
> Key: HBASE-9079
> URL: https://issues.apache.org/jira/browse/HBASE-9079
> Project: HBase
> Issue Type: Bug
> Components: Filters
>Affects Versions: 0.94.10
>Reporter: Viral Bajaria
> Attachments: HBASE-9079-0.94.patch, HBASE-9079-trunk.patch
>
>
> I hit a weird issue/bug and am able to reproduce the error consistently. The
> problem arises when FilterList has two filters where each implements the
> getNextKeyHint method.
> The way the current implementation works is, StoreScanner will call
> matcher.getNextKeyHint() whenever it gets a SEEK_NEXT_USING_HINT. This in
> turn will call filter.getNextKeyHint() which at this stage is of type
> FilterList. The implementation in FilterList iterates through all the filters
> and keeps the max KeyValue that it sees. All is fine if you wrap filters in
> FilterList in which only one of them implements getNextKeyHint. but if
> multiple of them implement then that's where things get weird.
> For example:
> - create two filters: one is FuzzyRowFilter and second is ColumnRangeFilter.
> Both of them implement getNextKeyHint
> - wrap them in FilterList with MUST_PASS_ALL
> - FuzzyRowFilter will seek to the correct first row and then pass it to
> ColumnRangeFilter which will return the SEEK_NEXT_USING_HINT code.
> - Now in FilterList when getNextKeyHint is called, it calls the one on
> FuzzyRow first which basically says what the next row should be. While in
> reality we want the ColumnRangeFilter to give the seek hint.
> - The above behavior skips data that should be returned, which I have
> verified by using a RowFilter with RegexStringComparator.
> I updated the FilterList to maintain state on which filter returns the
> SEEK_NEXT_USING_HINT and in getNextKeyHint, I invoke the method on the saved
> filter and reset that state. I te
[jira] [Commented] (HBASE-9102) HFile block pre-loading for large sequential scan
[ https://issues.apache.org/jira/browse/HBASE-9102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13727242#comment-13727242 ] Chao Shi commented on HBASE-9102: - bq. The client is able to enable/disable on each request basis. Is this switch available for now? I guess this is enough to improve under our workload (as most of our scan requests only touch 1 block). For such requests, we can enable this switch to use pread. > HFile block pre-loading for large sequential scan > - > > Key: HBASE-9102 > URL: https://issues.apache.org/jira/browse/HBASE-9102 > Project: HBase > Issue Type: Improvement >Affects Versions: 0.89-fb >Reporter: Liyin Tang >Assignee: Liyin Tang > > The current HBase scan model cannot take full advantage of the aggrediate > disk throughput, especially for the large sequential scan cases. And for the > large sequential scan, it is easy to predict what the next block to read in > advance so that it can pre-load and decompress/decoded these data blocks from > HDFS into block cache right before the current read point. > Therefore, this jira is to optimized the large sequential scan performance by > pre-loading the HFile blocks into the block cache in a stream fashion so that > the scan query can read from the cache directly. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8224) Publish hbase build against h1 and h2 adding '-hadoop1' or '-hadoop2' to version string
[ https://issues.apache.org/jira/browse/HBASE-8224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13727233#comment-13727233 ] stack commented on HBASE-8224: -- [~brocknoland] Do the SNAPSHOTs work for you boss? > Publish hbase build against h1 and h2 adding '-hadoop1' or '-hadoop2' to > version string > --- > > Key: HBASE-8224 > URL: https://issues.apache.org/jira/browse/HBASE-8224 > Project: HBase > Issue Type: Task >Reporter: stack >Assignee: stack >Priority: Blocker > Fix For: 0.98.0, 0.95.2 > > Attachments: 8224-adding.classifiers.txt, 8224.gen.script.txt, > 8224.gen.scriptv3.txt, 8224.gen.scriptv3.txt, 8224v5.txt, > hbase-8224-proto1.patch > > > So we can publish both the hadoop1 and the hadoop2 jars to a maven > repository, and so we can publish two packages, one for hadoop1 and one for > hadoop2, given how maven works, our only alternative (to the best of my > knowledge and after consulting others) is by amending the version string to > include hadoop1 or hadoop2. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9098) During recovery use ZK as the source of truth for region state
[
https://issues.apache.org/jira/browse/HBASE-9098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13727228#comment-13727228
]
Ted Yu commented on HBASE-9098:
---
{code}
- // when a region is in recovering state, it can only accept writes not reads
- private volatile boolean recovering = false;
{code}
Nice simplification above.
{code}
- public void prepareLogReplay(Set serverNames) throws IOException
{
+ public void prepareLogReplay(ServerName serverName, List
regions) throws IOException {
{code}
There is conversion later in the patch:
{code}
+ for(HRegionInfo tmpRegionInfo : regions) {
+regionSet.add(tmpRegionInfo);
{code}
Why not passing Set as parameter ?
{code}
+ // verify current region is indeed in recovering state
+ try {
+if (SplitLogManager.isRegionMarkedRecoveringInZK(watcher,
loc.getRegionInfo()
+.getEncodedName()) == false) {
{code}
The comment seems to be different from the check above.
> During recovery use ZK as the source of truth for region state
> ---
>
> Key: HBASE-9098
> URL: https://issues.apache.org/jira/browse/HBASE-9098
> Project: HBase
> Issue Type: Bug
> Components: regionserver
>Affects Versions: 0.95.0
>Reporter: Devaraj Das
>Assignee: Jeffrey Zhong
>Priority: Blocker
> Fix For: 0.95.2
>
> Attachments: hbase-9098.patch
>
>
> In HLogSplitter:locateRegionAndRefreshLastFlushedSequenceId(HConnection,
> byte[], byte[], String), we talk to the replayee regionserver to figure out
> whether a region is in recovery or not. We should look at ZK only for this
> piece of information (since that is the source of truth for recovery
> otherwise).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9086) Add some options to improve count performance
[ https://issues.apache.org/jira/browse/HBASE-9086?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13727225#comment-13727225 ] Cheney Sun commented on HBASE-9086: --- @Lars, no, that's not what I expect. the new patch was uploaded, and pick up the Jean-Marc's suggestion by adding FirstKeyOnlyFilter and KeyOnlyFilter to a Filter list. > Add some options to improve count performance > - > > Key: HBASE-9086 > URL: https://issues.apache.org/jira/browse/HBASE-9086 > Project: HBase > Issue Type: Wish > Components: shell >Affects Versions: 0.94.2 >Reporter: Cheney Sun > Attachments: HBase-9086.patch, HBase-9086_v0.2.patch > > > The current count command in HBase shell is quite slow if the row size is > very big (100+kB each). It would be helpful to provide some option to specify > the column to count, which could give user a chance to reduce the data volume > to scan. > IMHO, only count the row key would be the ideal solution. Not sure how > difficult to implement it. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-9086) Add some options to improve count performance
[ https://issues.apache.org/jira/browse/HBASE-9086?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheney Sun updated HBASE-9086: -- Attachment: HBase-9086.patch > Add some options to improve count performance > - > > Key: HBASE-9086 > URL: https://issues.apache.org/jira/browse/HBASE-9086 > Project: HBase > Issue Type: Wish > Components: shell >Affects Versions: 0.94.2 >Reporter: Cheney Sun > Attachments: HBase-9086.patch, HBase-9086_v0.2.patch > > > The current count command in HBase shell is quite slow if the row size is > very big (100+kB each). It would be helpful to provide some option to specify > the column to count, which could give user a chance to reduce the data volume > to scan. > IMHO, only count the row key would be the ideal solution. Not sure how > difficult to implement it. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-9086) Add some options to improve count performance
[ https://issues.apache.org/jira/browse/HBASE-9086?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheney Sun updated HBASE-9086: -- Attachment: (was: HBase-9086.patch) > Add some options to improve count performance > - > > Key: HBASE-9086 > URL: https://issues.apache.org/jira/browse/HBASE-9086 > Project: HBase > Issue Type: Wish > Components: shell >Affects Versions: 0.94.2 >Reporter: Cheney Sun > Attachments: HBase-9086.patch, HBase-9086_v0.2.patch > > > The current count command in HBase shell is quite slow if the row size is > very big (100+kB each). It would be helpful to provide some option to specify > the column to count, which could give user a chance to reduce the data volume > to scan. > IMHO, only count the row key would be the ideal solution. Not sure how > difficult to implement it. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-9086) Add some options to improve count performance
[ https://issues.apache.org/jira/browse/HBASE-9086?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheney Sun updated HBASE-9086: -- Attachment: HBase-9086_v0.2.patch > Add some options to improve count performance > - > > Key: HBASE-9086 > URL: https://issues.apache.org/jira/browse/HBASE-9086 > Project: HBase > Issue Type: Wish > Components: shell >Affects Versions: 0.94.2 >Reporter: Cheney Sun > Attachments: HBase-9086.patch, HBase-9086_v0.2.patch > > > The current count command in HBase shell is quite slow if the row size is > very big (100+kB each). It would be helpful to provide some option to specify > the column to count, which could give user a chance to reduce the data volume > to scan. > IMHO, only count the row key would be the ideal solution. Not sure how > difficult to implement it. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-9098) During recovery use ZK as the source of truth for region state
[ https://issues.apache.org/jira/browse/HBASE-9098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeffrey Zhong updated HBASE-9098: - Attachment: hbase-9098.patch > During recovery use ZK as the source of truth for region state > --- > > Key: HBASE-9098 > URL: https://issues.apache.org/jira/browse/HBASE-9098 > Project: HBase > Issue Type: Bug > Components: regionserver >Affects Versions: 0.95.0 >Reporter: Devaraj Das >Assignee: Jeffrey Zhong >Priority: Blocker > Fix For: 0.95.2 > > Attachments: hbase-9098.patch > > > In HLogSplitter:locateRegionAndRefreshLastFlushedSequenceId(HConnection, > byte[], byte[], String), we talk to the replayee regionserver to figure out > whether a region is in recovery or not. We should look at ZK only for this > piece of information (since that is the source of truth for recovery > otherwise). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-9098) During recovery use ZK as the source of truth for region state
[ https://issues.apache.org/jira/browse/HBASE-9098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeffrey Zhong updated HBASE-9098: - Status: Patch Available (was: Open) > During recovery use ZK as the source of truth for region state > --- > > Key: HBASE-9098 > URL: https://issues.apache.org/jira/browse/HBASE-9098 > Project: HBase > Issue Type: Bug > Components: regionserver >Affects Versions: 0.95.0 >Reporter: Devaraj Das >Assignee: Jeffrey Zhong >Priority: Blocker > Fix For: 0.95.2 > > Attachments: hbase-9098.patch > > > In HLogSplitter:locateRegionAndRefreshLastFlushedSequenceId(HConnection, > byte[], byte[], String), we talk to the replayee regionserver to figure out > whether a region is in recovery or not. We should look at ZK only for this > piece of information (since that is the source of truth for recovery > otherwise). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8408) Implement namespace
[ https://issues.apache.org/jira/browse/HBASE-8408?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Francis Liu updated HBASE-8408: --- Attachment: HBASE-8015_11.patch > Implement namespace > --- > > Key: HBASE-8408 > URL: https://issues.apache.org/jira/browse/HBASE-8408 > Project: HBase > Issue Type: Sub-task >Reporter: Francis Liu >Assignee: Francis Liu > Attachments: HBASE-8015_11.patch, HBASE-8015_1.patch, > HBASE-8015_2.patch, HBASE-8015_3.patch, HBASE-8015_4.patch, > HBASE-8015_5.patch, HBASE-8015_6.patch, HBASE-8015_7.patch, > HBASE-8015_8.patch, HBASE-8015_9.patch, HBASE-8015.patch, > TestNamespaceMigration.tgz, TestNamespaceUpgrade.tgz > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9096) Disable split during log replay
[ https://issues.apache.org/jira/browse/HBASE-9096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13727221#comment-13727221 ] Ted Yu commented on HBASE-9096: --- +1 > Disable split during log replay > --- > > Key: HBASE-9096 > URL: https://issues.apache.org/jira/browse/HBASE-9096 > Project: HBase > Issue Type: Bug >Reporter: Devaraj Das >Assignee: Jeffrey Zhong > Attachments: hbase-9096.patch > > > When regions are allowed to take writes during recovery, we could end up in a > situation where a split of a region might be triggered. That would close the > old region leading to failure of the ongoing replay. In discussions with > [~jeffreyz], it seemed to make sense to just disable split during recovery. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-9096) Disable split during log replay
[ https://issues.apache.org/jira/browse/HBASE-9096?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeffrey Zhong updated HBASE-9096: - Status: Patch Available (was: Open) > Disable split during log replay > --- > > Key: HBASE-9096 > URL: https://issues.apache.org/jira/browse/HBASE-9096 > Project: HBase > Issue Type: Bug >Reporter: Devaraj Das >Assignee: Jeffrey Zhong > Attachments: hbase-9096.patch > > > When regions are allowed to take writes during recovery, we could end up in a > situation where a split of a region might be triggered. That would close the > old region leading to failure of the ongoing replay. In discussions with > [~jeffreyz], it seemed to make sense to just disable split during recovery. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-9096) Disable split during log replay
[ https://issues.apache.org/jira/browse/HBASE-9096?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeffrey Zhong updated HBASE-9096: - Attachment: hbase-9096.patch > Disable split during log replay > --- > > Key: HBASE-9096 > URL: https://issues.apache.org/jira/browse/HBASE-9096 > Project: HBase > Issue Type: Bug >Reporter: Devaraj Das >Assignee: Jeffrey Zhong > Attachments: hbase-9096.patch > > > When regions are allowed to take writes during recovery, we could end up in a > situation where a split of a region might be triggered. That would close the > old region leading to failure of the ongoing replay. In discussions with > [~jeffreyz], it seemed to make sense to just disable split during recovery. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9087) Handlers being blocked during reads
[ https://issues.apache.org/jira/browse/HBASE-9087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13727218#comment-13727218 ] Pablo Medina commented on HBASE-9087: - I'll try to run it tomorrow. Where I can find the version with the patch applied? > Handlers being blocked during reads > --- > > Key: HBASE-9087 > URL: https://issues.apache.org/jira/browse/HBASE-9087 > Project: HBase > Issue Type: Bug > Components: Performance >Affects Versions: 0.94.7, 0.95.1 >Reporter: Pablo Medina >Assignee: Elliott Clark > Fix For: 0.98.0, 0.95.2, 0.94.12 > > Attachments: HBASE-9087-0.patch, HBASE-9087-1.patch > > > I'm having a lot of handlers (90 - 300 aprox) being blocked when reading > rows. They are blocked during changedReaderObserver registration. > Lars Hofhansl suggests to change the implementation of changedReaderObserver > from CopyOnWriteList to ConcurrentHashMap. > Here is a stack trace: > "IPC Server handler 99 on 60020" daemon prio=10 tid=0x41c84000 > nid=0x2244 waiting on condition [0x7ff51fefd000] >java.lang.Thread.State: WAITING (parking) > at sun.misc.Unsafe.park(Native Method) > - parking to wait for <0xc5c13ae8> (a > java.util.concurrent.locks.ReentrantLock$NonfairSync) > at java.util.concurrent.locks.LockSupport.park(LockSupport.java:156) > at > java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:811) > at > java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:842) > at > java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1178) > at > java.util.concurrent.locks.ReentrantLock$NonfairSync.lock(ReentrantLock.java:186) > at > java.util.concurrent.locks.ReentrantLock.lock(ReentrantLock.java:262) > at > java.util.concurrent.CopyOnWriteArrayList.addIfAbsent(CopyOnWriteArrayList.java:553) > at > java.util.concurrent.CopyOnWriteArraySet.add(CopyOnWriteArraySet.java:221) > at > org.apache.hadoop.hbase.regionserver.Store.addChangedReaderObserver(Store.java:1085) > at > org.apache.hadoop.hbase.regionserver.StoreScanner.(StoreScanner.java:138) > at > org.apache.hadoop.hbase.regionserver.Store.getScanner(Store.java:2077) > at > org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.(HRegion.java:3755) > at > org.apache.hadoop.hbase.regionserver.HRegion.instantiateRegionScanner(HRegion.java:1804) > at > org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:1796) > at > org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:1771) > at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:4776) > at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:4750) > at > org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:2152) > at > org.apache.hadoop.hbase.regionserver.HRegionServer.multi(HRegionServer.java:3700) > at sun.reflect.GeneratedMethodAccessor26.invoke(Unknown Source) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > at java.lang.reflect.Method.invoke(Method.java:597) > at > org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:320) > at > org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1426) > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8462) Custom timestamps should not be allowed to be negative
[ https://issues.apache.org/jira/browse/HBASE-8462?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl updated HBASE-8462: - Fix Version/s: (was: 0.94.11) Let's not do this in 0.94. > Custom timestamps should not be allowed to be negative > -- > > Key: HBASE-8462 > URL: https://issues.apache.org/jira/browse/HBASE-8462 > Project: HBase > Issue Type: Bug > Components: Client >Reporter: Enis Soztutar >Assignee: Enis Soztutar > Fix For: 0.98.0, 0.95.2 > > Attachments: hbase-8462_v1.patch, hbase-8462_v2.patch > > > Client supplied timestamps should not be allowed to be negative, otherwise > unpredictable results will follow. Especially, since we are encoding the ts > using Bytes.Bytes(long), negative timestamps are sorted after positive ones. > Plus, the new PB messages define ts' as uint64. > Credit goes to Huned Lokhandwala for reporting this. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8067) TestHFileArchiving.testArchiveOnTableDelete sometimes fails
[
https://issues.apache.org/jira/browse/HBASE-8067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lars Hofhansl updated HBASE-8067:
-
Fix Version/s: (was: 0.94.11)
0.94.12
Can we just close this one?
> TestHFileArchiving.testArchiveOnTableDelete sometimes fails
> ---
>
> Key: HBASE-8067
> URL: https://issues.apache.org/jira/browse/HBASE-8067
> Project: HBase
> Issue Type: Bug
> Components: Admin, master, test
>Affects Versions: 0.94.6, 0.95.2
>Reporter: Matteo Bertozzi
>Assignee: Matteo Bertozzi
> Fix For: 0.98.0, 0.95.2, 0.94.12
>
> Attachments: 8067-0.94.txt, 8067-trunk.txt, HBASE-8067-debug.patch,
> HBASE-8067-v0.patch
>
>
> it seems that testArchiveOnTableDelete() fails because the archiving in
> DeleteTableHandler is still in progress when admin.deleteTable() returns.
> {code}
> Error Message
> Archived files are missing some of the store files!
> Stacktrace
> java.lang.AssertionError: Archived files are missing some of the store files!
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.assertTrue(Assert.java:41)
> at
> org.apache.hadoop.hbase.backup.TestHFileArchiving.testArchiveOnTableDelete(TestHFileArchiving.java:262)
> {code}
> (Looking at the problem in a more generic way, we don't have any way to
> inform the client when an async operation is completed)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9075) [0.94] Backport HBASE-5760 Unit tests should write only under /target to 0.94
[ https://issues.apache.org/jira/browse/HBASE-9075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13727209#comment-13727209 ] Lars Hofhansl commented on HBASE-9075: -- Looks good. Why does it remove testGetSetOfHTD? > [0.94] Backport HBASE-5760 Unit tests should write only under /target to 0.94 > - > > Key: HBASE-9075 > URL: https://issues.apache.org/jira/browse/HBASE-9075 > Project: HBase > Issue Type: Test > Components: test >Reporter: Enis Soztutar >Assignee: Enis Soztutar > Fix For: 0.94.11 > > Attachments: hbase-9075_v1.patch > > > Backporting HBASE-5760 is a good idea. 0.94 tests mess up the root level > directory a lot. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8803) region_mover.rb should move multiple regions at a time
[ https://issues.apache.org/jira/browse/HBASE-8803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl updated HBASE-8803: - Fix Version/s: (was: 0.94.11) 0.94.12 Pushing to 0.94.12. If you clean up the patch in time for 0.94.11, JM, we can pull it back in. > region_mover.rb should move multiple regions at a time > -- > > Key: HBASE-8803 > URL: https://issues.apache.org/jira/browse/HBASE-8803 > Project: HBase > Issue Type: Bug > Components: Usability >Affects Versions: 0.98.0, 0.94.8, 0.95.1 >Reporter: Jean-Marc Spaggiari >Assignee: Jean-Marc Spaggiari >Priority: Critical > Fix For: 0.98.0, 0.95.2, 0.94.12 > > Attachments: HBASE-8803-v0-trunk.patch, HBASE-8803-v1-0.94.patch, > HBASE-8803-v1-trunk.patch, HBASE-8803-v2-0.94.patch, > HBASE-8803-v2-0.94.patch, HBASE-8803-v3-0.94.patch, HBASE-8803-v4-0.94.patch > > Original Estimate: 48h > Remaining Estimate: 48h > > When there is many regions in a cluster, rolling_restart can take hours > because region_mover is moving the regions one by one. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8667) Master and Regionserver not able to communicate if both bound to different network interfaces on the same machine.
[
https://issues.apache.org/jira/browse/HBASE-8667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lars Hofhansl updated HBASE-8667:
-
Fix Version/s: (was: 0.94.11)
0.94.12
Pushing again
> Master and Regionserver not able to communicate if both bound to different
> network interfaces on the same machine.
> --
>
> Key: HBASE-8667
> URL: https://issues.apache.org/jira/browse/HBASE-8667
> Project: HBase
> Issue Type: Bug
> Components: IPC/RPC
>Reporter: rajeshbabu
>Assignee: rajeshbabu
>Priority: Critical
> Fix For: 0.98.0, 0.95.2, 0.94.12
>
> Attachments: HBASE-8667_trunk.patch, HBASE-8667_Trunk.patch,
> HBASE-8667_Trunk-V2.patch, HBASE-8667_trunk_v4.patch,
> HBASE-8667_trunk_v5.patch, HBASE-8667_trunk_v6.patch
>
>
> While testing HBASE-8640 fix found that master and regionserver running on
> different interfaces are not communicating properly.
> I have two interfaces 1) lo 2) eth0 in my machine and default hostname
> interface is lo.
> I have configured master ipc address to ip of eth0 interface.
> Started master and regionserver on the same machine.
> 1) master rpc server bound to eth0 and RS rpc server bound to lo
> 2) Since rpc client is not binding to any ip address, when RS is reporting RS
> startup its getting registered with eth0 ip address(but actually it should
> register localhost)
> Here are RS logs:
> {code}
> 2013-05-31 06:05:28,608 WARN [regionserver60020]
> org.apache.hadoop.hbase.regionserver.HRegionServer: reportForDuty failed;
> sleeping and then retrying.
> 2013-05-31 06:05:31,609 INFO [regionserver60020]
> org.apache.hadoop.hbase.regionserver.HRegionServer: Attempting connect to
> Master server at 192.168.0.100,6,1369960497008
> 2013-05-31 06:05:31,609 INFO [regionserver60020]
> org.apache.hadoop.hbase.regionserver.HRegionServer: Telling master at
> 192.168.0.100,6,1369960497008 that we are up with port=60020,
> startcode=1369960502544
> 2013-05-31 06:05:31,618 DEBUG [regionserver60020]
> org.apache.hadoop.hbase.regionserver.HRegionServer: Config from master:
> hbase.rootdir=hdfs://localhost:2851/hbase
> 2013-05-31 06:05:31,618 DEBUG [regionserver60020]
> org.apache.hadoop.hbase.regionserver.HRegionServer: Config from master:
> fs.default.name=hdfs://localhost:2851
> 2013-05-31 06:05:31,618 INFO [regionserver60020]
> org.apache.hadoop.hbase.regionserver.HRegionServer: Master passed us a
> different hostname to use; was=localhost, but now=192.168.0.100
> {code}
> Here are master logs:
> {code}
> 2013-05-31 06:05:31,615 INFO [IPC Server handler 9 on 6]
> org.apache.hadoop.hbase.master.ServerManager: Registering
> server=192.168.0.100,60020,1369960502544
> {code}
> Since master has wrong rpc server address of RS, META is not getting assigned.
> {code}
> 2013-05-31 06:05:34,362 DEBUG [master-192.168.0.100,6,1369960497008]
> org.apache.hadoop.hbase.master.AssignmentManager: No previous transition plan
> was found (or we are ignoring an existing plan) for .META.,,1.1028785192 so
> generated a random one; hri=.META.,,1.1028785192, src=,
> dest=192.168.0.100,60020,1369960502544; 1 (online=1, available=1) available
> servers, forceNewPlan=false
> -
> org.apache.hadoop.hbase.master.AssignmentManager: Failed assignment of
> .META.,,1.1028785192 to 192.168.0.100,60020,1369960502544, trying to assign
> elsewhere instead; try=1 of 10
> java.net.ConnectException: Connection refused
> at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
> at
> sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:592)
> at
> org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206)
> at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:511)
> at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:481)
> at
> org.apache.hadoop.hbase.ipc.RpcClient$Connection.setupConnection(RpcClient.java:549)
> at
> org.apache.hadoop.hbase.ipc.RpcClient$Connection.setupIOstreams(RpcClient.java:813)
> at
> org.apache.hadoop.hbase.ipc.RpcClient.getConnection(RpcClient.java:1422)
> at org.apache.hadoop.hbase.ipc.RpcClient.call(RpcClient.java:1315)
> at
> org.apache.hadoop.hbase.ipc.RpcClient.callBlockingMethod(RpcClient.java:1532)
> at
> org.apache.hadoop.hbase.ipc.RpcClient$BlockingRpcChannelImplementation.callBlockingMethod(RpcClient.java:1587)
> at
> org.apache.hadoop.hbase.protobuf.generated.AdminProtos$AdminService$BlockingStub.openRegion(AdminProtos.java:15039)
> at
> org.apache.hadoop.hbase.master.ServerManager.sendRegionOpen(ServerManager.java:627)
> at
> org.apache.hadoop.hbase.master.AssignmentManager.assign(As
[jira] [Commented] (HBASE-9103) Print lines that are longer than allowed in HadoopQA output.
[ https://issues.apache.org/jira/browse/HBASE-9103?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13727203#comment-13727203 ] Hudson commented on HBASE-9103: --- FAILURE: Integrated in HBase-TRUNK #4331 (See [https://builds.apache.org/job/HBase-TRUNK/4331/]) HBASE-9103 Addendum adds the missing back quote (tedyu: rev 1509435) * /hbase/trunk/dev-support/test-patch.sh > Print lines that are longer than allowed in HadoopQA output. > > > Key: HBASE-9103 > URL: https://issues.apache.org/jira/browse/HBASE-9103 > Project: HBase > Issue Type: Improvement >Reporter: Jesse Yates >Assignee: Jesse Yates > Fix For: 0.98.0 > > Attachments: 9103.addendum, hbase-9103-v0.patch > > > Its a little annoying to not know which lines are longer - helpful to find > the ones that are over. Generally, this will be a small number of lines that > the formatter didn't get quite right, so massive logging statements should be > rare -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HBASE-9029) Backport HBASE-8706 Some improvement in snapshot to 0.94
[ https://issues.apache.org/jira/browse/HBASE-9029?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl resolved HBASE-9029. -- Resolution: Fixed Committed to 0.94. Thanks for the backport Jerry. > Backport HBASE-8706 Some improvement in snapshot to 0.94 > > > Key: HBASE-9029 > URL: https://issues.apache.org/jira/browse/HBASE-9029 > Project: HBase > Issue Type: Improvement > Components: snapshots >Affects Versions: 0.94.9 >Reporter: Jerry He >Assignee: Jerry He >Priority: Minor > Fix For: 0.94.11 > > Attachments: HBase-9029-0.94.patch > > > 'HBASE-8706 Some improvement in snapshot' has some good parameter tuning and > improvement for snapshot handling, making snapshot more robust. > It will nice to put it in 0.94. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8752) Backport HBASE-6466 to 0.94
[ https://issues.apache.org/jira/browse/HBASE-8752?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl updated HBASE-8752: - Fix Version/s: (was: 0.94.11) > Backport HBASE-6466 to 0.94 > --- > > Key: HBASE-8752 > URL: https://issues.apache.org/jira/browse/HBASE-8752 > Project: HBase > Issue Type: Improvement > Components: regionserver >Affects Versions: 0.94.8 >Reporter: Richard Ding >Assignee: Richard Ding >Priority: Minor > Attachments: HBASE-8752.patch > > > 0.94 already supports multi-thread compaction. It will be good it also > supports multi-thread memstore flush, so that users can tune the number of > threads for both compaction and flushing when running a heavy-write load. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8595) Add rename operation in hbase shell
[ https://issues.apache.org/jira/browse/HBASE-8595?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl updated HBASE-8595: - Fix Version/s: (was: 0.94.11) Removing from 0.94 > Add rename operation in hbase shell > --- > > Key: HBASE-8595 > URL: https://issues.apache.org/jira/browse/HBASE-8595 > Project: HBase > Issue Type: New Feature > Components: shell >Affects Versions: 0.94.8, 0.95.1 >Reporter: Aleksandr Shulman >Assignee: Aleksandr Shulman >Priority: Minor > Fix For: 0.95.2 > > Attachments: HBASE-8595-v0.patch > > > We can use a set of snapshot commands to elegantly rename a table. It would > be nice to wrap all those commands in a single call. > http://hbase.apache.org/book.html#table.rename > Also -- the documentation is missing the last step where the original table > needs to be deleted. I can add that to the docbook. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8752) Backport HBASE-6466 to 0.94
[ https://issues.apache.org/jira/browse/HBASE-8752?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl updated HBASE-8752: - Fix Version/s: 0.94.12 Moving to 0.94.12 for lack of interest. > Backport HBASE-6466 to 0.94 > --- > > Key: HBASE-8752 > URL: https://issues.apache.org/jira/browse/HBASE-8752 > Project: HBase > Issue Type: Improvement > Components: regionserver >Affects Versions: 0.94.8 >Reporter: Richard Ding >Assignee: Richard Ding >Priority: Minor > Fix For: 0.94.12 > > Attachments: HBASE-8752.patch > > > 0.94 already supports multi-thread compaction. It will be good it also > supports multi-thread memstore flush, so that users can tune the number of > threads for both compaction and flushing when running a heavy-write load. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-9087) Handlers being blocked during reads
[ https://issues.apache.org/jira/browse/HBASE-9087?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lars Hofhansl updated HBASE-9087: - Priority: Major (was: Critical) Fix Version/s: (was: 0.94.11) 0.94.12 > Handlers being blocked during reads > --- > > Key: HBASE-9087 > URL: https://issues.apache.org/jira/browse/HBASE-9087 > Project: HBase > Issue Type: Bug > Components: Performance >Affects Versions: 0.94.7, 0.95.1 >Reporter: Pablo Medina >Assignee: Elliott Clark > Fix For: 0.98.0, 0.95.2, 0.94.12 > > Attachments: HBASE-9087-0.patch, HBASE-9087-1.patch > > > I'm having a lot of handlers (90 - 300 aprox) being blocked when reading > rows. They are blocked during changedReaderObserver registration. > Lars Hofhansl suggests to change the implementation of changedReaderObserver > from CopyOnWriteList to ConcurrentHashMap. > Here is a stack trace: > "IPC Server handler 99 on 60020" daemon prio=10 tid=0x41c84000 > nid=0x2244 waiting on condition [0x7ff51fefd000] >java.lang.Thread.State: WAITING (parking) > at sun.misc.Unsafe.park(Native Method) > - parking to wait for <0xc5c13ae8> (a > java.util.concurrent.locks.ReentrantLock$NonfairSync) > at java.util.concurrent.locks.LockSupport.park(LockSupport.java:156) > at > java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:811) > at > java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:842) > at > java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1178) > at > java.util.concurrent.locks.ReentrantLock$NonfairSync.lock(ReentrantLock.java:186) > at > java.util.concurrent.locks.ReentrantLock.lock(ReentrantLock.java:262) > at > java.util.concurrent.CopyOnWriteArrayList.addIfAbsent(CopyOnWriteArrayList.java:553) > at > java.util.concurrent.CopyOnWriteArraySet.add(CopyOnWriteArraySet.java:221) > at > org.apache.hadoop.hbase.regionserver.Store.addChangedReaderObserver(Store.java:1085) > at > org.apache.hadoop.hbase.regionserver.StoreScanner.(StoreScanner.java:138) > at > org.apache.hadoop.hbase.regionserver.Store.getScanner(Store.java:2077) > at > org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.(HRegion.java:3755) > at > org.apache.hadoop.hbase.regionserver.HRegion.instantiateRegionScanner(HRegion.java:1804) > at > org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:1796) > at > org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:1771) > at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:4776) > at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:4750) > at > org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:2152) > at > org.apache.hadoop.hbase.regionserver.HRegionServer.multi(HRegionServer.java:3700) > at sun.reflect.GeneratedMethodAccessor26.invoke(Unknown Source) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > at java.lang.reflect.Method.invoke(Method.java:597) > at > org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:320) > at > org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1426) > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8693) DataType: provide extensible type API
[ https://issues.apache.org/jira/browse/HBASE-8693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-8693: Hadoop Flags: Reviewed Status: Patch Available (was: Open) > DataType: provide extensible type API > - > > Key: HBASE-8693 > URL: https://issues.apache.org/jira/browse/HBASE-8693 > Project: HBase > Issue Type: Sub-task > Components: Client >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk > Fix For: 0.95.2 > > Attachments: 0001-HBASE-8693-Extensible-data-types-API.patch, > 0001-HBASE-8693-Extensible-data-types-API.patch, > 0001-HBASE-8693-Extensible-data-types-API.patch, > 0001-HBASE-8693-Extensible-data-types-API.patch, > 0001-HBASE-8693-Extensible-data-types-API.patch, > 0001-HBASE-8693-Extensible-data-types-API.patch, > 0001-HBASE-8693-Extensible-data-types-API.patch, > 0002-HBASE-8693-example-Use-DataType-API-to-build-regionN.patch, > KijiFormattedEntityId.java > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8693) DataType: provide extensible type API
[ https://issues.apache.org/jira/browse/HBASE-8693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13727199#comment-13727199 ] Nick Dimiduk commented on HBASE-8693: - On RB. > DataType: provide extensible type API > - > > Key: HBASE-8693 > URL: https://issues.apache.org/jira/browse/HBASE-8693 > Project: HBase > Issue Type: Sub-task > Components: Client >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk > Fix For: 0.95.2 > > Attachments: 0001-HBASE-8693-Extensible-data-types-API.patch, > 0001-HBASE-8693-Extensible-data-types-API.patch, > 0001-HBASE-8693-Extensible-data-types-API.patch, > 0001-HBASE-8693-Extensible-data-types-API.patch, > 0001-HBASE-8693-Extensible-data-types-API.patch, > 0001-HBASE-8693-Extensible-data-types-API.patch, > 0001-HBASE-8693-Extensible-data-types-API.patch, > 0002-HBASE-8693-example-Use-DataType-API-to-build-regionN.patch, > KijiFormattedEntityId.java > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8089) Add type support
[ https://issues.apache.org/jira/browse/HBASE-8089?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-8089: Fix Version/s: (was: 0.95.2) 0.98.0 > Add type support > > > Key: HBASE-8089 > URL: https://issues.apache.org/jira/browse/HBASE-8089 > Project: HBase > Issue Type: New Feature > Components: Client >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk > Fix For: 0.98.0 > > Attachments: HBASE-8089-types.txt, HBASE-8089-types.txt, > HBASE-8089-types.txt, HBASE-8089-types.txt, hbase data types WIP.pdf > > > This proposal outlines an improvement to HBase that provides for a set of > types, above and beyond the existing "byte-bucket" strategy. This is intended > to reduce user-level duplication of effort, provide better support for > 3rd-party integration, and provide an overall improved experience for > developers using HBase. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-9091) Update ByteRange to maintain consumer's position
[ https://issues.apache.org/jira/browse/HBASE-9091?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-9091: Fix Version/s: (was: 0.95.2) > Update ByteRange to maintain consumer's position > > > Key: HBASE-9091 > URL: https://issues.apache.org/jira/browse/HBASE-9091 > Project: HBase > Issue Type: Improvement > Components: Client >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk > Attachments: 0001-HBASE-9091-Extend-ByteRange.patch, > 0001-HBASE-9091-Extend-ByteRange.patch > > > ByteRange is a useful alternative to Java's ByteBuffer. Notably, it is > mutable and an instance can be assigned over a byte[] after instantiation. > This is valuable as a performance consideration when working with byte[] > slices in a tight loop. Its current design is such that it is not possible to > consume a portion of the range while performing activities like decoding an > object without altering the definition of the range. It should provide a > position that is independent from the range's offset and length to make > partial reads easier. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8693) DataType: provide extensible type API
[ https://issues.apache.org/jira/browse/HBASE-8693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-8693: Attachment: 0001-HBASE-8693-Extensible-data-types-API.patch Based on new version of OrderedBytes. Does not use ByteRange or ByteBuffer, just a byte[] and offset. > DataType: provide extensible type API > - > > Key: HBASE-8693 > URL: https://issues.apache.org/jira/browse/HBASE-8693 > Project: HBase > Issue Type: Sub-task > Components: Client >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk > Fix For: 0.95.2 > > Attachments: 0001-HBASE-8693-Extensible-data-types-API.patch, > 0001-HBASE-8693-Extensible-data-types-API.patch, > 0001-HBASE-8693-Extensible-data-types-API.patch, > 0001-HBASE-8693-Extensible-data-types-API.patch, > 0001-HBASE-8693-Extensible-data-types-API.patch, > 0001-HBASE-8693-Extensible-data-types-API.patch, > 0001-HBASE-8693-Extensible-data-types-API.patch, > 0002-HBASE-8693-example-Use-DataType-API-to-build-regionN.patch, > KijiFormattedEntityId.java > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8201) OrderedBytes: an ordered encoding strategy
[ https://issues.apache.org/jira/browse/HBASE-8201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13727196#comment-13727196 ] Nick Dimiduk commented on HBASE-8201: - I should add, this version provides Order and OrderedBytes, no ordered data types. Now on RB. > OrderedBytes: an ordered encoding strategy > -- > > Key: HBASE-8201 > URL: https://issues.apache.org/jira/browse/HBASE-8201 > Project: HBase > Issue Type: Sub-task > Components: Client >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk > Fix For: 0.95.2 > > Attachments: > 0001-HBASE-8201-OrderedBytes-order-preserving-encoding.patch, > 0001-HBASE-8201-OrderedBytes-order-preserving-encoding.patch, > 0001-HBASE-8201-OrderedBytes-provides-order-preserving-se.patch, > 0001-HBASE-8201-OrderedBytes-provides-order-preserving-se.patch, > 0001-HBASE-8201-OrderedBytes-provides-order-preserving-se.patch, > 0001-HBASE-8201-OrderedBytes-provides-order-preserving-se.patch, > 0001-HBASE-8201-OrderedBytes-provides-order-preserving-se.patch, > 0001-HBASE-8201-OrderedBytes-provides-order-preserving-se.patch, > 0001-HBASE-8201-OrderedBytes-provides-order-preserving-se.patch, > 0001-HBASE-8201-OrderedBytes-provides-order-preserving-se.patch, > 0001-HBASE-8201-OrderedBytes-provides-order-preserving-se.patch, > 0001-HBASE-8201-OrderedBytes-provides-order-preserving-se.patch, > 0001-HBASE-8201-OrderedBytes-provides-order-preserving-se.patch > > > Once the spec is agreed upon, it must be implemented. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8201) OrderedBytes: an ordered encoding strategy
[ https://issues.apache.org/jira/browse/HBASE-8201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-8201: Hadoop Flags: Reviewed Status: Patch Available (was: Open) > OrderedBytes: an ordered encoding strategy > -- > > Key: HBASE-8201 > URL: https://issues.apache.org/jira/browse/HBASE-8201 > Project: HBase > Issue Type: Sub-task > Components: Client >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk > Fix For: 0.95.2 > > Attachments: > 0001-HBASE-8201-OrderedBytes-order-preserving-encoding.patch, > 0001-HBASE-8201-OrderedBytes-order-preserving-encoding.patch, > 0001-HBASE-8201-OrderedBytes-provides-order-preserving-se.patch, > 0001-HBASE-8201-OrderedBytes-provides-order-preserving-se.patch, > 0001-HBASE-8201-OrderedBytes-provides-order-preserving-se.patch, > 0001-HBASE-8201-OrderedBytes-provides-order-preserving-se.patch, > 0001-HBASE-8201-OrderedBytes-provides-order-preserving-se.patch, > 0001-HBASE-8201-OrderedBytes-provides-order-preserving-se.patch, > 0001-HBASE-8201-OrderedBytes-provides-order-preserving-se.patch, > 0001-HBASE-8201-OrderedBytes-provides-order-preserving-se.patch, > 0001-HBASE-8201-OrderedBytes-provides-order-preserving-se.patch, > 0001-HBASE-8201-OrderedBytes-provides-order-preserving-se.patch, > 0001-HBASE-8201-OrderedBytes-provides-order-preserving-se.patch > > > Once the spec is agreed upon, it must be implemented. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-8201) OrderedBytes: an ordered encoding strategy
[
https://issues.apache.org/jira/browse/HBASE-8201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nick Dimiduk updated HBASE-8201:
Attachment: 0001-HBASE-8201-OrderedBytes-order-preserving-encoding.patch
One more time. The API in this version has no dependencies -- all methods take
a {{byte[] buff}} and {{int offset}} as first two arguments. Per my previous
comment, that means the client must pass over an encoded value twice: once to
get the encoded value, and again to advance the {{offset}}.
> OrderedBytes: an ordered encoding strategy
> --
>
> Key: HBASE-8201
> URL: https://issues.apache.org/jira/browse/HBASE-8201
> Project: HBase
> Issue Type: Sub-task
> Components: Client
>Reporter: Nick Dimiduk
>Assignee: Nick Dimiduk
> Fix For: 0.95.2
>
> Attachments:
> 0001-HBASE-8201-OrderedBytes-order-preserving-encoding.patch,
> 0001-HBASE-8201-OrderedBytes-order-preserving-encoding.patch,
> 0001-HBASE-8201-OrderedBytes-provides-order-preserving-se.patch,
> 0001-HBASE-8201-OrderedBytes-provides-order-preserving-se.patch,
> 0001-HBASE-8201-OrderedBytes-provides-order-preserving-se.patch,
> 0001-HBASE-8201-OrderedBytes-provides-order-preserving-se.patch,
> 0001-HBASE-8201-OrderedBytes-provides-order-preserving-se.patch,
> 0001-HBASE-8201-OrderedBytes-provides-order-preserving-se.patch,
> 0001-HBASE-8201-OrderedBytes-provides-order-preserving-se.patch,
> 0001-HBASE-8201-OrderedBytes-provides-order-preserving-se.patch,
> 0001-HBASE-8201-OrderedBytes-provides-order-preserving-se.patch,
> 0001-HBASE-8201-OrderedBytes-provides-order-preserving-se.patch,
> 0001-HBASE-8201-OrderedBytes-provides-order-preserving-se.patch
>
>
> Once the spec is agreed upon, it must be implemented.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9091) Update ByteRange to maintain consumer's position
[ https://issues.apache.org/jira/browse/HBASE-9091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13727190#comment-13727190 ] Nick Dimiduk commented on HBASE-9091: - In the interest of squeezing OrderedBytes and DataType into 0.95.2, I've taken the approach described above. We can go back and make improvements to the API later if we so desire. > Update ByteRange to maintain consumer's position > > > Key: HBASE-9091 > URL: https://issues.apache.org/jira/browse/HBASE-9091 > Project: HBase > Issue Type: Improvement > Components: Client >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk > Fix For: 0.95.2 > > Attachments: 0001-HBASE-9091-Extend-ByteRange.patch, > 0001-HBASE-9091-Extend-ByteRange.patch > > > ByteRange is a useful alternative to Java's ByteBuffer. Notably, it is > mutable and an instance can be assigned over a byte[] after instantiation. > This is valuable as a performance consideration when working with byte[] > slices in a tight loop. Its current design is such that it is not possible to > consume a portion of the range while performing activities like decoding an > object without altering the definition of the range. It should provide a > position that is independent from the range's offset and length to make > partial reads easier. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-9091) Update ByteRange to maintain consumer's position
[ https://issues.apache.org/jira/browse/HBASE-9091?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-9091: Issue Type: Improvement (was: Sub-task) Parent: (was: HBASE-8089) > Update ByteRange to maintain consumer's position > > > Key: HBASE-9091 > URL: https://issues.apache.org/jira/browse/HBASE-9091 > Project: HBase > Issue Type: Improvement > Components: Client >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk > Fix For: 0.95.2 > > Attachments: 0001-HBASE-9091-Extend-ByteRange.patch, > 0001-HBASE-9091-Extend-ByteRange.patch > > > ByteRange is a useful alternative to Java's ByteBuffer. Notably, it is > mutable and an instance can be assigned over a byte[] after instantiation. > This is valuable as a performance consideration when working with byte[] > slices in a tight loop. Its current design is such that it is not possible to > consume a portion of the range while performing activities like decoding an > object without altering the definition of the range. It should provide a > position that is independent from the range's offset and length to make > partial reads easier. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-9087) Handlers being blocked during reads
[ https://issues.apache.org/jira/browse/HBASE-9087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13727189#comment-13727189 ] Lars Hofhansl commented on HBASE-9087: -- I also tried a bunch of scenarios and could not find one where this improves performance. [~pablomedina85], any chance to run your scenario with this patch applied? > Handlers being blocked during reads > --- > > Key: HBASE-9087 > URL: https://issues.apache.org/jira/browse/HBASE-9087 > Project: HBase > Issue Type: Bug > Components: Performance >Affects Versions: 0.94.7, 0.95.1 >Reporter: Pablo Medina >Assignee: Elliott Clark >Priority: Critical > Fix For: 0.98.0, 0.95.2, 0.94.11 > > Attachments: HBASE-9087-0.patch, HBASE-9087-1.patch > > > I'm having a lot of handlers (90 - 300 aprox) being blocked when reading > rows. They are blocked during changedReaderObserver registration. > Lars Hofhansl suggests to change the implementation of changedReaderObserver > from CopyOnWriteList to ConcurrentHashMap. > Here is a stack trace: > "IPC Server handler 99 on 60020" daemon prio=10 tid=0x41c84000 > nid=0x2244 waiting on condition [0x7ff51fefd000] >java.lang.Thread.State: WAITING (parking) > at sun.misc.Unsafe.park(Native Method) > - parking to wait for <0xc5c13ae8> (a > java.util.concurrent.locks.ReentrantLock$NonfairSync) > at java.util.concurrent.locks.LockSupport.park(LockSupport.java:156) > at > java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:811) > at > java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:842) > at > java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1178) > at > java.util.concurrent.locks.ReentrantLock$NonfairSync.lock(ReentrantLock.java:186) > at > java.util.concurrent.locks.ReentrantLock.lock(ReentrantLock.java:262) > at > java.util.concurrent.CopyOnWriteArrayList.addIfAbsent(CopyOnWriteArrayList.java:553) > at > java.util.concurrent.CopyOnWriteArraySet.add(CopyOnWriteArraySet.java:221) > at > org.apache.hadoop.hbase.regionserver.Store.addChangedReaderObserver(Store.java:1085) > at > org.apache.hadoop.hbase.regionserver.StoreScanner.(StoreScanner.java:138) > at > org.apache.hadoop.hbase.regionserver.Store.getScanner(Store.java:2077) > at > org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.(HRegion.java:3755) > at > org.apache.hadoop.hbase.regionserver.HRegion.instantiateRegionScanner(HRegion.java:1804) > at > org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:1796) > at > org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:1771) > at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:4776) > at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:4750) > at > org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:2152) > at > org.apache.hadoop.hbase.regionserver.HRegionServer.multi(HRegionServer.java:3700) > at sun.reflect.GeneratedMethodAccessor26.invoke(Unknown Source) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > at java.lang.reflect.Method.invoke(Method.java:597) > at > org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:320) > at > org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1426) > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
