[jira] [Commented] (HBASE-14463) Severe performance downgrade when parallel reading a single key from BucketCache
[
https://issues.apache.org/jira/browse/HBASE-14463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14973118#comment-14973118
]
Yu Li commented on HBASE-14463:
---
Ping, any more comments here fellows?
A summary of testing done:
# For the multi thread reading single key scenario, it's covered by the
existing UT case TestBucketCache#testCacheMultiThreaded. The testing result
shows obvious perf improvement with current patch
# For the more common random read case, with the same query key distribution
and testing on one node real cluster, result shows no performance downgrade
with current patch
> Severe performance downgrade when parallel reading a single key from
> BucketCache
>
>
> Key: HBASE-14463
> URL: https://issues.apache.org/jira/browse/HBASE-14463
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.98.14, 1.1.2
>Reporter: Yu Li
>Assignee: Yu Li
> Fix For: 2.0.0, 1.2.0, 1.3.0, 0.98.16
>
> Attachments: GC_with_WeakObjectPool.png, HBASE-14463.patch,
> HBASE-14463_v11.patch, HBASE-14463_v12.patch, HBASE-14463_v2.patch,
> HBASE-14463_v3.patch, HBASE-14463_v4.patch, HBASE-14463_v5.patch,
> TestBucketCache-new_with_IdLock.png,
> TestBucketCache-new_with_IdReadWriteLock.png,
> TestBucketCache_with_IdLock-latest.png, TestBucketCache_with_IdLock.png,
> TestBucketCache_with_IdReadWriteLock-latest.png,
> TestBucketCache_with_IdReadWriteLock-resolveLockLeak.png,
> TestBucketCache_with_IdReadWriteLock.png, pe_use_same_keys.patch,
> test-results.tar.gz
>
>
> We store feature data of online items in HBase, do machine learning on these
> features, and supply the outputs to our online search engine. In such
> scenario we will launch hundreds of yarn workers and each worker will read
> all features of one item(i.e. single rowkey in HBase), so there'll be heavy
> parallel reading on a single rowkey.
> We were using LruCache but start to try BucketCache recently to resolve gc
> issue, and just as titled we have observed severe performance downgrade.
> After some analytics we found the root cause is the lock in
> BucketCache#getBlock, as shown below
> {code}
> try {
> lockEntry = offsetLock.getLockEntry(bucketEntry.offset());
> // ...
> if (bucketEntry.equals(backingMap.get(key))) {
> // ...
> int len = bucketEntry.getLength();
> Cacheable cachedBlock = ioEngine.read(bucketEntry.offset(), len,
> bucketEntry.deserializerReference(this.deserialiserMap));
> {code}
> Since ioEnging.read involves array copy, it's much more time-costed than the
> operation in LruCache. And since we're using synchronized in
> IdLock#getLockEntry, parallel read dropping on the same bucket would be
> executed in serial, which causes a really bad performance.
> To resolve the problem, we propose to use ReentranceReadWriteLock in
> BucketCache, and introduce a new class called IdReadWriteLock to implement it.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14648) Reenable TestWALProcedureStoreOnHDFS#testWalRollOnLowReplication
[ https://issues.apache.org/jira/browse/HBASE-14648?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14973142#comment-14973142 ] Hudson commented on HBASE-14648: SUCCESS: Integrated in HBase-1.3-IT #271 (See [https://builds.apache.org/job/HBase-1.3-IT/271/]) HBASE-14648 Reenable (stack: rev d76dbb4f84d8e606ab8d319aab7337ad030c85e8) * hbase-server/src/test/java/org/apache/hadoop/hbase/master/procedure/TestWALProcedureStoreOnHDFS.java > Reenable TestWALProcedureStoreOnHDFS#testWalRollOnLowReplication > > > Key: HBASE-14648 > URL: https://issues.apache.org/jira/browse/HBASE-14648 > Project: HBase > Issue Type: Sub-task > Components: test >Reporter: stack >Assignee: Heng Chen >Priority: Critical > Fix For: 2.0.0, 1.2.0, 1.3.0 > > Attachments: HBASE-14648 (2).patch, HBASE-14648.patch, > HBASE-14648.patch, HBASE-14648.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14684) Try to remove all MiniMapReduceCluster in unit tests
[ https://issues.apache.org/jira/browse/HBASE-14684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Heng Chen updated HBASE-14684: -- Attachment: (was: HBASE-14684_v1.patch) > Try to remove all MiniMapReduceCluster in unit tests > > > Key: HBASE-14684 > URL: https://issues.apache.org/jira/browse/HBASE-14684 > Project: HBase > Issue Type: Improvement > Components: test >Reporter: Heng Chen >Assignee: Heng Chen >Priority: Critical > Fix For: 2.0.0 > > Attachments: HBASE-14684.patch, HBASE-14684_v1.patch > > > As discussion in dev list, we will try to do MR job without > MiniMapReduceCluster. > Testcases will run faster and more reliable. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14648) Reenable TestWALProcedureStoreOnHDFS#testWalRollOnLowReplication
[ https://issues.apache.org/jira/browse/HBASE-14648?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14973104#comment-14973104 ] Hudson commented on HBASE-14648: SUCCESS: Integrated in HBase-1.2 #305 (See [https://builds.apache.org/job/HBase-1.2/305/]) HBASE-14648 Reenable (stack: rev 9ad77becf4fb2d3e768c916900d0792c0287a56f) * hbase-server/src/test/java/org/apache/hadoop/hbase/master/procedure/TestWALProcedureStoreOnHDFS.java > Reenable TestWALProcedureStoreOnHDFS#testWalRollOnLowReplication > > > Key: HBASE-14648 > URL: https://issues.apache.org/jira/browse/HBASE-14648 > Project: HBase > Issue Type: Sub-task > Components: test >Reporter: stack >Assignee: Heng Chen >Priority: Critical > Fix For: 2.0.0, 1.2.0, 1.3.0 > > Attachments: HBASE-14648 (2).patch, HBASE-14648.patch, > HBASE-14648.patch, HBASE-14648.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14624) BucketCache.freeBlock is too expensive
[ https://issues.apache.org/jira/browse/HBASE-14624?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14973143#comment-14973143 ] Hudson commented on HBASE-14624: SUCCESS: Integrated in HBase-1.3-IT #271 (See [https://builds.apache.org/job/HBase-1.3-IT/271/]) HBASE-14624 BucketCache.freeBlock is too expensive (tedyu: rev 40887c94b715f58b016b12d0b2d9c80032d0262b) * hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/bucket/BucketAllocator.java > BucketCache.freeBlock is too expensive > -- > > Key: HBASE-14624 > URL: https://issues.apache.org/jira/browse/HBASE-14624 > Project: HBase > Issue Type: Bug > Components: BlockCache >Affects Versions: 1.0.0 >Reporter: Randy Fox >Assignee: Ted Yu > Fix For: 2.0.0, 1.2.0, 1.3.0, 1.0.3, 1.1.3, 0.98.16 > > Attachments: 14624-v1.txt, 14624-v2.txt, 14624-v3.txt > > > Moving regions is unacceptably slow when using bucket cache, as it takes too > long to free all the blocks. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14624) BucketCache.freeBlock is too expensive
[ https://issues.apache.org/jira/browse/HBASE-14624?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14973116#comment-14973116 ] Hudson commented on HBASE-14624: FAILURE: Integrated in HBase-0.98-on-Hadoop-1.1 #1121 (See [https://builds.apache.org/job/HBase-0.98-on-Hadoop-1.1/1121/]) HBASE-14624 BucketCache.freeBlock is too expensive (tedyu: rev f2150149669fc236e122e1e165584e9561714177) * hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/bucket/BucketAllocator.java > BucketCache.freeBlock is too expensive > -- > > Key: HBASE-14624 > URL: https://issues.apache.org/jira/browse/HBASE-14624 > Project: HBase > Issue Type: Bug > Components: BlockCache >Affects Versions: 1.0.0 >Reporter: Randy Fox >Assignee: Ted Yu > Fix For: 2.0.0, 1.2.0, 1.3.0, 1.0.3, 1.1.3, 0.98.16 > > Attachments: 14624-v1.txt, 14624-v2.txt, 14624-v3.txt > > > Moving regions is unacceptably slow when using bucket cache, as it takes too > long to free all the blocks. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14648) Reenable TestWALProcedureStoreOnHDFS#testWalRollOnLowReplication
[ https://issues.apache.org/jira/browse/HBASE-14648?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14973135#comment-14973135 ] Hudson commented on HBASE-14648: SUCCESS: Integrated in HBase-TRUNK #6958 (See [https://builds.apache.org/job/HBase-TRUNK/6958/]) HBASE-14648 Reenable (stack: rev 46c646d6154d6a5291c308e065802248f4e4e7b9) * hbase-server/src/test/java/org/apache/hadoop/hbase/master/procedure/TestWALProcedureStoreOnHDFS.java > Reenable TestWALProcedureStoreOnHDFS#testWalRollOnLowReplication > > > Key: HBASE-14648 > URL: https://issues.apache.org/jira/browse/HBASE-14648 > Project: HBase > Issue Type: Sub-task > Components: test >Reporter: stack >Assignee: Heng Chen >Priority: Critical > Fix For: 2.0.0, 1.2.0, 1.3.0 > > Attachments: HBASE-14648 (2).patch, HBASE-14648.patch, > HBASE-14648.patch, HBASE-14648.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14684) Try to remove all MiniMapReduceCluster in unit tests
[ https://issues.apache.org/jira/browse/HBASE-14684?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14973074#comment-14973074 ] Heng Chen commented on HBASE-14684: --- Printing hanging tests Hanging test : org.apache.hadoop.hbase.TestFullLogReconstruction Hanging test : org.apache.hadoop.hbase.util.TestHBaseFsckOneRS Hanging test : org.apache.hadoop.hbase.util.TestHBaseFsckTwoRS Printing Failing tests Failing test : org.apache.hadoop.hbase.snapshot.TestMobFlushSnapshotFromClient Failing test : org.apache.hadoop.hbase.snapshot.TestFlushSnapshotFromClient All tests has no relates with MR. I will retry. > Try to remove all MiniMapReduceCluster in unit tests > > > Key: HBASE-14684 > URL: https://issues.apache.org/jira/browse/HBASE-14684 > Project: HBase > Issue Type: Improvement > Components: test >Reporter: Heng Chen >Assignee: Heng Chen >Priority: Critical > Fix For: 2.0.0 > > Attachments: HBASE-14684.patch, HBASE-14684_v1.patch > > > As discussion in dev list, we will try to do MR job without > MiniMapReduceCluster. > Testcases will run faster and more reliable. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14684) Try to remove all MiniMapReduceCluster in unit tests
[ https://issues.apache.org/jira/browse/HBASE-14684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Heng Chen updated HBASE-14684: -- Attachment: HBASE-14684_v1.patch > Try to remove all MiniMapReduceCluster in unit tests > > > Key: HBASE-14684 > URL: https://issues.apache.org/jira/browse/HBASE-14684 > Project: HBase > Issue Type: Improvement > Components: test >Reporter: Heng Chen >Assignee: Heng Chen >Priority: Critical > Fix For: 2.0.0 > > Attachments: HBASE-14684.patch, HBASE-14684_v1.patch, > HBASE-14684_v1.patch > > > As discussion in dev list, we will try to do MR job without > MiniMapReduceCluster. > Testcases will run faster and more reliable. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14684) Try to remove all MiniMapReduceCluster in unit tests
[
https://issues.apache.org/jira/browse/HBASE-14684?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14973145#comment-14973145
]
Hadoop QA commented on HBASE-14684:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12768582/HBASE-14684_v1.patch
against master branch at commit 46c646d6154d6a5291c308e065802248f4e4e7b9.
ATTACHMENT ID: 12768582
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 90 new
or modified tests.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.6.1 2.7.0
2.7.1)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 protoc{color}. The applied patch does not increase the
total number of protoc compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 checkstyle{color}. The applied patch does not increase the
total number of checkstyle errors
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn post-site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/16218//testReport/
Release Findbugs (version 2.0.3)warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/16218//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/16218//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/16218//console
This message is automatically generated.
> Try to remove all MiniMapReduceCluster in unit tests
>
>
> Key: HBASE-14684
> URL: https://issues.apache.org/jira/browse/HBASE-14684
> Project: HBase
> Issue Type: Improvement
> Components: test
>Reporter: Heng Chen
>Assignee: Heng Chen
>Priority: Critical
> Fix For: 2.0.0
>
> Attachments: HBASE-14684.patch, HBASE-14684_v1.patch
>
>
> As discussion in dev list, we will try to do MR job without
> MiniMapReduceCluster.
> Testcases will run faster and more reliable.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14648) Reenable TestWALProcedureStoreOnHDFS#testWalRollOnLowReplication
[ https://issues.apache.org/jira/browse/HBASE-14648?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14973098#comment-14973098 ] Hudson commented on HBASE-14648: FAILURE: Integrated in HBase-1.3 #306 (See [https://builds.apache.org/job/HBase-1.3/306/]) HBASE-14648 Reenable (stack: rev d76dbb4f84d8e606ab8d319aab7337ad030c85e8) * hbase-server/src/test/java/org/apache/hadoop/hbase/master/procedure/TestWALProcedureStoreOnHDFS.java > Reenable TestWALProcedureStoreOnHDFS#testWalRollOnLowReplication > > > Key: HBASE-14648 > URL: https://issues.apache.org/jira/browse/HBASE-14648 > Project: HBase > Issue Type: Sub-task > Components: test >Reporter: stack >Assignee: Heng Chen >Priority: Critical > Fix For: 2.0.0, 1.2.0, 1.3.0 > > Attachments: HBASE-14648 (2).patch, HBASE-14648.patch, > HBASE-14648.patch, HBASE-14648.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-11544) [Ergonomics] hbase.client.scanner.caching is dogged and will try to return batch even if it means OOME
[ https://issues.apache.org/jira/browse/HBASE-11544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14973195#comment-14973195 ] Mindaugas Kairys commented on HBASE-11544: -- I'm unable to get partial results in mapreduce mapper jobs. I set setAllowPartialResults(true) for scan, but job still fails with OOME on large rows. I found that a Scan copy constructor does not bring field "allowPartialResults" to a new Scan objects and it is possibly the reason. I have also tried to run pure Scan client which scans all rows succesfully when using setAllowPartialResults(true). But fails if I use a Scan object constructed from a previous one by a copy constructor (new Scan(originalScan)). > [Ergonomics] hbase.client.scanner.caching is dogged and will try to return > batch even if it means OOME > -- > > Key: HBASE-11544 > URL: https://issues.apache.org/jira/browse/HBASE-11544 > Project: HBase > Issue Type: Bug >Reporter: stack >Assignee: Jonathan Lawlor >Priority: Critical > Fix For: 2.0.0, 1.1.0 > > Attachments: Allocation_Hot_Spots.html, > HBASE-11544-addendum-v1.patch, HBASE-11544-addendum-v2.patch, > HBASE-11544-branch_1_0-v1.patch, HBASE-11544-branch_1_0-v2.patch, > HBASE-11544-v1.patch, HBASE-11544-v2.patch, HBASE-11544-v3.patch, > HBASE-11544-v4.patch, HBASE-11544-v5.patch, HBASE-11544-v6.patch, > HBASE-11544-v6.patch, HBASE-11544-v6.patch, HBASE-11544-v7.patch, > HBASE-11544-v8-branch-1.patch, HBASE-11544-v8.patch, gc.j.png, h.png, > hits.j.png, m.png, mean.png, net.j.png, q (2).png > > > Running some tests, I set hbase.client.scanner.caching=1000. Dataset has > large cells. I kept OOME'ing. > Serverside, we should measure how much we've accumulated and return to the > client whatever we've gathered once we pass out a certain size threshold > rather than keep accumulating till we OOME. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-13867) Add endpoint coprocessor guide to HBase book
[ https://issues.apache.org/jira/browse/HBASE-13867?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gaurav Bhardwaj updated HBASE-13867: Status: Patch Available (was: In Progress) Rebuild the pach. Please use [HBASE-13867.4.patch|https://issues.apache.org/jira/secure/attachment/12768601/HBASE-13867.4.patch] > Add endpoint coprocessor guide to HBase book > > > Key: HBASE-13867 > URL: https://issues.apache.org/jira/browse/HBASE-13867 > Project: HBase > Issue Type: Task > Components: Coprocessors, documentation >Reporter: Vladimir Rodionov >Assignee: Gaurav Bhardwaj > Fix For: 2.0.0 > > Attachments: HBASE-13867.1.patch, HBASE-13867.2.patch, > HBASE-13867.2.patch, HBASE-13867.3.patch, HBASE-13867.4.patch > > > Endpoint coprocessors are very poorly documented. > Coprocessor section of HBase book must be updated either with its own > endpoint coprocessors HOW-TO guide or, at least, with the link(s) to some > other guides. There is good description here: > http://www.3pillarglobal.com/insights/hbase-coprocessors -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-13867) Add endpoint coprocessor guide to HBase book
[ https://issues.apache.org/jira/browse/HBASE-13867?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gaurav Bhardwaj updated HBASE-13867: Attachment: HBASE-13867.4.patch > Add endpoint coprocessor guide to HBase book > > > Key: HBASE-13867 > URL: https://issues.apache.org/jira/browse/HBASE-13867 > Project: HBase > Issue Type: Task > Components: Coprocessors, documentation >Reporter: Vladimir Rodionov >Assignee: Gaurav Bhardwaj > Fix For: 2.0.0 > > Attachments: HBASE-13867.1.patch, HBASE-13867.2.patch, > HBASE-13867.2.patch, HBASE-13867.3.patch, HBASE-13867.4.patch > > > Endpoint coprocessors are very poorly documented. > Coprocessor section of HBase book must be updated either with its own > endpoint coprocessors HOW-TO guide or, at least, with the link(s) to some > other guides. There is good description here: > http://www.3pillarglobal.com/insights/hbase-coprocessors -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13867) Add endpoint coprocessor guide to HBase book
[ https://issues.apache.org/jira/browse/HBASE-13867?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14973208#comment-14973208 ] Gaurav Bhardwaj commented on HBASE-13867: - [~misty], I build the patch as per your steps and I was able to apply it on a clean branch (after hard reset) and no error was observed. Please use the [HBASE-13867.4.patch|https://issues.apache.org/jira/secure/attachment/12768601/HBASE-13867.4.patch]. Now there shouldn't be any trailing white space issue. > Add endpoint coprocessor guide to HBase book > > > Key: HBASE-13867 > URL: https://issues.apache.org/jira/browse/HBASE-13867 > Project: HBase > Issue Type: Task > Components: Coprocessors, documentation >Reporter: Vladimir Rodionov >Assignee: Gaurav Bhardwaj > Fix For: 2.0.0 > > Attachments: HBASE-13867.1.patch, HBASE-13867.2.patch, > HBASE-13867.2.patch, HBASE-13867.3.patch, HBASE-13867.4.patch > > > Endpoint coprocessors are very poorly documented. > Coprocessor section of HBase book must be updated either with its own > endpoint coprocessors HOW-TO guide or, at least, with the link(s) to some > other guides. There is good description here: > http://www.3pillarglobal.com/insights/hbase-coprocessors -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14693) Add client-side metrics for received pushback signals
[
https://issues.apache.org/jira/browse/HBASE-14693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Heng Chen updated HBASE-14693:
--
Attachment: HBASE-14693.patch
I made a simple patch.
I only do metrics in {{AsyncProcess.receiveMultiAction}}.
I don't know is it enough?
> Add client-side metrics for received pushback signals
> -
>
> Key: HBASE-14693
> URL: https://issues.apache.org/jira/browse/HBASE-14693
> Project: HBase
> Issue Type: Improvement
>Reporter: Andrew Purtell
> Attachments: HBASE-14693.patch
>
>
> HBASE-12911 added client side metrics. HBASE-5162 added a mechanism for
> sending advisory backpressure signals to clients when the server is heavily
> loaded, and HBASE-12702 and subtasks backported this to all active branches.
> Add client-side metrics for received pushback signal.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-13867) Add endpoint coprocessor guide to HBase book
[ https://issues.apache.org/jira/browse/HBASE-13867?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gaurav Bhardwaj updated HBASE-13867: Status: In Progress (was: Patch Available) > Add endpoint coprocessor guide to HBase book > > > Key: HBASE-13867 > URL: https://issues.apache.org/jira/browse/HBASE-13867 > Project: HBase > Issue Type: Task > Components: Coprocessors, documentation >Reporter: Vladimir Rodionov >Assignee: Gaurav Bhardwaj > Fix For: 2.0.0 > > Attachments: HBASE-13867.1.patch, HBASE-13867.2.patch, > HBASE-13867.2.patch, HBASE-13867.3.patch > > > Endpoint coprocessors are very poorly documented. > Coprocessor section of HBase book must be updated either with its own > endpoint coprocessors HOW-TO guide or, at least, with the link(s) to some > other guides. There is good description here: > http://www.3pillarglobal.com/insights/hbase-coprocessors -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-13867) Add endpoint coprocessor guide to HBase book
[ https://issues.apache.org/jira/browse/HBASE-13867?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14973568#comment-14973568 ] Gaurav Bhardwaj commented on HBASE-13867: - Thanks [~misty]! > Add endpoint coprocessor guide to HBase book > > > Key: HBASE-13867 > URL: https://issues.apache.org/jira/browse/HBASE-13867 > Project: HBase > Issue Type: Task > Components: Coprocessors, documentation >Reporter: Vladimir Rodionov >Assignee: Gaurav Bhardwaj > Fix For: 2.0.0 > > Attachments: HBASE-13867.1.patch, HBASE-13867.2.patch, > HBASE-13867.2.patch, HBASE-13867.3.patch, HBASE-13867.4.patch > > > Endpoint coprocessors are very poorly documented. > Coprocessor section of HBase book must be updated either with its own > endpoint coprocessors HOW-TO guide or, at least, with the link(s) to some > other guides. There is good description here: > http://www.3pillarglobal.com/insights/hbase-coprocessors -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14675) Exorcise deprecated Put#add(...) and replace with Put#addColumn(...)
[
https://issues.apache.org/jira/browse/HBASE-14675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14973526#comment-14973526
]
Hadoop QA commented on HBASE-14675:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12768627/hbase-14675.v2.patch
against master branch at commit 46c646d6154d6a5291c308e065802248f4e4e7b9.
ATTACHMENT ID: 12768627
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 514
new or modified tests.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.6.1 2.7.0
2.7.1)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 protoc{color}. The applied patch does not increase the
total number of protoc compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:red}-1 checkstyle{color}. The applied patch generated
1733 checkstyle errors (more than the master's current 1732 errors).
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 lineLengths{color}. The patch introduces the following lines
longer than 100:
+putOfMerged.addColumn(HConstants.CATALOG_FAMILY,
HConstants.MERGEA_QUALIFIER, regionA.toByteArray());
+putOfMerged.addColumn(HConstants.CATALOG_FAMILY,
HConstants.MERGEB_QUALIFIER, regionB.toByteArray());
+p.addColumn(Bytes.toBytes("col"), Bytes.toBytes("ql"),
Bytes.toBytes("value" + numberOfPuts));
+ p2.addColumn(TEST_FAMILY, Bytes.add(TEST_MULTI_CQ, Bytes.toBytes(d)),
Bytes.toBytes(d * 0.10));
+ put.addColumn(Bytes.toBytes("f1"), Bytes.toBytes("c" + j),
timestamp, Bytes.toBytes(i + "_c" + j));
+ put.addColumn(FAMILY_NAME_BYTES, Bytes.toBytes(QUALIFIER_PREFIX + i),
(long) i, Bytes.toBytes(VALUE_PREFIX + i));
+ put.addColumn(CF_BYTES, Bytes.toBytes("col" + j), ts,
Bytes.toBytes("value" + i + "_" + j + "_" + ts));
+ put.addColumn(FAMILY_BYTES, Bytes.toBytes("col" + iCol),
Bytes.toBytes("value" + iFile + "_" + iRow + "_" + iCol));
+ put.addColumn(FAMILY_BYTES, Bytes.toBytes("col" + iCol), ts +
version, Bytes.toBytes("value" + iFile + "_" + iRow + "_" + iCol));
+ outCollector.collect(outKey, new
Put(Bytes.toBytes("rowKey2")).addColumn(COLUMN_FAMILY, COLUMN_QUALIFIER,
Bytes.toBytes(name)));
{color:green}+1 site{color}. The mvn post-site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/16222//testReport/
Release Findbugs (version 2.0.3)warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/16222//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/16222//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/16222//console
This message is automatically generated.
> Exorcise deprecated Put#add(...) and replace with Put#addColumn(...)
>
>
> Key: HBASE-14675
> URL: https://issues.apache.org/jira/browse/HBASE-14675
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 2.0.0
>Reporter: Jonathan Hsieh
>Assignee: Jonathan Hsieh
> Fix For: 2.0.0
>
> Attachments: hbase-14675.patch, hbase-14675.patch,
> hbase-14675.v2.patch
>
>
> The Put API changed from #add(...) to #addColumn(...). This updates all
> instances of it and removes it from the Put (which was added for hbase 1.0.0)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-14684) Try to remove all MiniMapReduceCluster in unit tests
[ https://issues.apache.org/jira/browse/HBASE-14684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Heng Chen updated HBASE-14684: -- Attachment: HBASE-14684_v1.patch > Try to remove all MiniMapReduceCluster in unit tests > > > Key: HBASE-14684 > URL: https://issues.apache.org/jira/browse/HBASE-14684 > Project: HBase > Issue Type: Improvement > Components: test >Reporter: Heng Chen >Assignee: Heng Chen >Priority: Critical > Fix For: 2.0.0 > > Attachments: HBASE-14684.patch, HBASE-14684_v1.patch > > > As discussion in dev list, we will try to do MR job without > MiniMapReduceCluster. > Testcases will run faster and more reliable. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14684) Try to remove all MiniMapReduceCluster in unit tests
[ https://issues.apache.org/jira/browse/HBASE-14684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Heng Chen updated HBASE-14684: -- Attachment: (was: HBASE-14684_v1.patch) > Try to remove all MiniMapReduceCluster in unit tests > > > Key: HBASE-14684 > URL: https://issues.apache.org/jira/browse/HBASE-14684 > Project: HBase > Issue Type: Improvement > Components: test >Reporter: Heng Chen >Assignee: Heng Chen >Priority: Critical > Fix For: 2.0.0 > > Attachments: HBASE-14684.patch, HBASE-14684_v1.patch > > > As discussion in dev list, we will try to do MR job without > MiniMapReduceCluster. > Testcases will run faster and more reliable. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14672) Exorcise deprecated Delete#delete* apis
[ https://issues.apache.org/jira/browse/HBASE-14672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh updated HBASE-14672: --- Attachment: hbase-14672-v2.patch updated to include changes to shell and recently introduced conflicts. > Exorcise deprecated Delete#delete* apis > --- > > Key: HBASE-14672 > URL: https://issues.apache.org/jira/browse/HBASE-14672 > Project: HBase > Issue Type: Improvement >Affects Versions: 2.0.0 >Reporter: Jonathan Hsieh >Assignee: Jonathan Hsieh > Fix For: 2.0.0 > > Attachments: hbase-14672-v2.patch, hbase-14672.patch > > > Delete#delete* apis were replaced with Delete#add* apis. This converts all > instances of it and removes Delete#delete* apis. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-12986) Compaction pressure based client pushback
[
https://issues.apache.org/jira/browse/HBASE-12986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Heng Chen updated HBASE-12986:
--
Attachment: HBASE-12986.patch
> Compaction pressure based client pushback
> -
>
> Key: HBASE-12986
> URL: https://issues.apache.org/jira/browse/HBASE-12986
> Project: HBase
> Issue Type: Improvement
>Reporter: Andrew Purtell
>Assignee: Andrew Purtell
> Fix For: 2.0.0, 1.3.0
>
> Attachments: HBASE-12986.patch
>
>
> HBASE-8329 recently introduced on all branches {{double
> RegionServerServices#getCompactionPressure()}}, which returns a value greater
> than or equal to 0.0, and any value greater than 1.0 means we have exceeded
> the store file limit on some stores. It could be reasonable to send this
> value along in server load statistics (clamping max at 1.0), and consider it
> as an additional term in the ExponentialClientBackoffPolicy.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-12986) Compaction pressure based client pushback
[
https://issues.apache.org/jira/browse/HBASE-12986?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14973692#comment-14973692
]
Heng Chen commented on HBASE-12986:
---
Any progress?
I make a simple patch. :)
{quote}
(clamping max at 1.0)
{quote}
Currently, {{compactionPressure}} >= 0, as comments, if
{{compactionPressure}}>=1, it means heavy compaction pressure.
So i just set it to be 1 if it greater than 1.
As for the first comment, it sounds a great change, maybe we can do it in
another issue
{quote}
We might also want to leave the current ExponentialClientBackoffPolicy as is
and introduce a new version of it that decouples the response to load factors
from contributions of load factor terms to the equation.
{quote}
> Compaction pressure based client pushback
> -
>
> Key: HBASE-12986
> URL: https://issues.apache.org/jira/browse/HBASE-12986
> Project: HBase
> Issue Type: Improvement
>Reporter: Andrew Purtell
>Assignee: Andrew Purtell
> Fix For: 2.0.0, 1.3.0
>
>
> HBASE-8329 recently introduced on all branches {{double
> RegionServerServices#getCompactionPressure()}}, which returns a value greater
> than or equal to 0.0, and any value greater than 1.0 means we have exceeded
> the store file limit on some stores. It could be reasonable to send this
> value along in server load statistics (clamping max at 1.0), and consider it
> as an additional term in the ExponentialClientBackoffPolicy.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14654) Reenable TestMultiParallel#testActiveThreadsCount
[ https://issues.apache.org/jira/browse/HBASE-14654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14973712#comment-14973712 ] Heng Chen commented on HBASE-14654: --- Suggestions? [~stack] > Reenable TestMultiParallel#testActiveThreadsCount > - > > Key: HBASE-14654 > URL: https://issues.apache.org/jira/browse/HBASE-14654 > Project: HBase > Issue Type: Bug >Reporter: Heng Chen > Attachments: HBASE-14654.patch, HBASE-14654.patch > > > It was disabled in HBASE-14642, this issue should reenable it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-13867) Add endpoint coprocessor guide to HBase book
[ https://issues.apache.org/jira/browse/HBASE-13867?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Misty Stanley-Jones updated HBASE-13867: Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) It worked great! Thanks for your contribution, [~gbhardwaj]. I changed the commit message on commit (removed "Patch for" from the front). > Add endpoint coprocessor guide to HBase book > > > Key: HBASE-13867 > URL: https://issues.apache.org/jira/browse/HBASE-13867 > Project: HBase > Issue Type: Task > Components: Coprocessors, documentation >Reporter: Vladimir Rodionov >Assignee: Gaurav Bhardwaj > Fix For: 2.0.0 > > Attachments: HBASE-13867.1.patch, HBASE-13867.2.patch, > HBASE-13867.2.patch, HBASE-13867.3.patch, HBASE-13867.4.patch > > > Endpoint coprocessors are very poorly documented. > Coprocessor section of HBase book must be updated either with its own > endpoint coprocessors HOW-TO guide or, at least, with the link(s) to some > other guides. There is good description here: > http://www.3pillarglobal.com/insights/hbase-coprocessors -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-14695) Fix some easy HTML warnings
Misty Stanley-Jones created HBASE-14695:
---
Summary: Fix some easy HTML warnings
Key: HBASE-14695
URL: https://issues.apache.org/jira/browse/HBASE-14695
Project: HBase
Issue Type: Bug
Components: documentation
Reporter: Misty Stanley-Jones
Assignee: Misty Stanley-Jones
Priority: Minor
Fix For: 2.0.0
There are a few links to top-level pages but missing the trailing /, and a few
please where we link to pages like replication.html even though we have a
single-page book now. There are also a few broken links in the APIdocs,
probably due to code we have specifically filtered out of the Javadoc:
{code}
#
# ERROR 4 files had broken links
#
/devapidocs/org/apache/hadoop/hbase/wal/NamespaceGroupingStrategy.html
had 1 broken link
/devapidocs/org/apache/hadoop/hbase/wal/RegionGroupingProvider.RegionGroupingStrategy.html
/devapidocs/org/apache/hadoop/hbase/wal/package-tree.html
had 1 broken link
/devapidocs/org/apache/hadoop/hbase/wal/RegionGroupingProvider.RegionGroupingStrategy.html
/devapidocs/overview-tree.html
had 2 broken links
/devapidocs/org/apache/hadoop/hbase/io/encoding/BufferedDataBlockEncoder.SeekerState.html
/devapidocs/org/apache/hadoop/hbase/wal/RegionGroupingProvider.RegionGroupingStrategy.html
/devapidocs/serialized-form.html
had 26 broken links
/devapidocs/src-html/com/google/protobuf/DescriptorProtos.DescriptorProto.ExtensionRange.html
/devapidocs/src-html/com/google/protobuf/DescriptorProtos.DescriptorProto.html
/devapidocs/src-html/com/google/protobuf/DescriptorProtos.EnumDescriptorProto.html
/devapidocs/src-html/com/google/protobuf/DescriptorProtos.EnumOptions.html
/devapidocs/src-html/com/google/protobuf/DescriptorProtos.EnumValueDescriptorProto.html
/devapidocs/src-html/com/google/protobuf/DescriptorProtos.EnumValueOptions.html
/devapidocs/src-html/com/google/protobuf/DescriptorProtos.FieldDescriptorProto.html
/devapidocs/src-html/com/google/protobuf/DescriptorProtos.FieldOptions.html
/devapidocs/src-html/com/google/protobuf/DescriptorProtos.FileDescriptorProto.html
/devapidocs/src-html/com/google/protobuf/DescriptorProtos.FileDescriptorSet.html
/devapidocs/src-html/com/google/protobuf/DescriptorProtos.FileOptions.html
/devapidocs/src-html/com/google/protobuf/DescriptorProtos.MessageOptions.html
/devapidocs/src-html/com/google/protobuf/DescriptorProtos.MethodDescriptorProto.html
/devapidocs/src-html/com/google/protobuf/DescriptorProtos.MethodOptions.html
/devapidocs/src-html/com/google/protobuf/DescriptorProtos.ServiceDescriptorProto.html
/devapidocs/src-html/com/google/protobuf/DescriptorProtos.ServiceOptions.html
/devapidocs/src-html/com/google/protobuf/DescriptorProtos.SourceCodeInfo.Location.html
/devapidocs/src-html/com/google/protobuf/DescriptorProtos.SourceCodeInfo.html
/devapidocs/src-html/com/google/protobuf/DescriptorProtos.UninterpretedOption.NamePart.html
/devapidocs/src-html/com/google/protobuf/DescriptorProtos.UninterpretedOption.html
/devapidocs/src-html/com/google/protobuf/GeneratedMessage.html
/devapidocs/src-html/com/google/protobuf/GeneratedMessageLite.html
/devapidocs/src-html/org/apache/hadoop/hbase/spark/ColumnFilter$.html
/devapidocs/src-html/org/apache/hadoop/hbase/spark/HBaseContext$.html
/devapidocs/src-html/org/apache/hadoop/hbase/spark/RowKeyFilter$.html
/devapidocs/src-html/org/apache/hadoop/hbase/spark/SchemaQualifierDefinition$.html
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14331) a single callQueue related improvements
[
https://issues.apache.org/jira/browse/HBASE-14331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14973705#comment-14973705
]
ramkrishna.s.vasudevan commented on HBASE-14331:
+1 on creating sub tasks where ever possible.
bq. Avoid calculating the deadline value each time.
+1 on this. I noticed this and hence tried to create a wrapper and pre
calculate the deadline and priority and just use that in the comparison instead
of doing it every time.
> a single callQueue related improvements
> ---
>
> Key: HBASE-14331
> URL: https://issues.apache.org/jira/browse/HBASE-14331
> Project: HBase
> Issue Type: Improvement
> Components: IPC/RPC, Performance
>Reporter: Hiroshi Ikeda
>Assignee: Hiroshi Ikeda
>Priority: Minor
> Attachments: BlockingQueuesPerformanceTestApp-output.pdf,
> BlockingQueuesPerformanceTestApp-output.txt,
> BlockingQueuesPerformanceTestApp.java, CallQueuePerformanceTestApp.java,
> HBASE-14331-V2.patch, HBASE-14331-V3.patch, HBASE-14331-V4.patch,
> HBASE-14331-V5.patch, HBASE-14331-V6.patch, HBASE-14331-V6.patch,
> HBASE-14331.patch, HBASE-14331.patch, SemaphoreBasedBlockingQueue.java,
> SemaphoreBasedLinkedBlockingQueue.java,
> SemaphoreBasedPriorityBlockingQueue.java
>
>
> {{LinkedBlockingQueue}} well separates locks between the {{take}} method and
> the {{put}} method, but not between takers, and not between putters. These
> methods are implemented to take locks at the almost beginning of their logic.
> HBASE-11355 introduces multiple call-queues to reduce such possible
> congestion, but I doubt that it is required to stick to {{BlockingQueue}}.
> There are the other shortcomings of using {{BlockingQueue}}. When using
> multiple queues, since {{BlockingQueue}} blocks threads it is required to
> prepare enough threads for each queue. It is possible that there is a queue
> starving for threads while there is another queue where threads are idle.
> Even if you can tune parameters to avoid such situations, the tuning is not
> so trivial.
> I suggest using a single {{ConcurrentLinkedQueue}} with {{Semaphore}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-12986) Compaction pressure based client pushback
[
https://issues.apache.org/jira/browse/HBASE-12986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Heng Chen updated HBASE-12986:
--
Status: Patch Available (was: Open)
> Compaction pressure based client pushback
> -
>
> Key: HBASE-12986
> URL: https://issues.apache.org/jira/browse/HBASE-12986
> Project: HBase
> Issue Type: Improvement
>Reporter: Andrew Purtell
>Assignee: Andrew Purtell
> Fix For: 2.0.0, 1.3.0
>
> Attachments: HBASE-12986.patch
>
>
> HBASE-8329 recently introduced on all branches {{double
> RegionServerServices#getCompactionPressure()}}, which returns a value greater
> than or equal to 0.0, and any value greater than 1.0 means we have exceeded
> the store file limit on some stores. It could be reasonable to send this
> value along in server load statistics (clamping max at 1.0), and consider it
> as an additional term in the ExponentialClientBackoffPolicy.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-14675) Exorcise deprecated Put#add(...) and replace with Put#addColumn(...)
[ https://issues.apache.org/jira/browse/HBASE-14675?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh updated HBASE-14675: --- Attachment: hbase-14675.v2.patch updated to fix calls in ruby shell code and conflicts with recent patches. > Exorcise deprecated Put#add(...) and replace with Put#addColumn(...) > > > Key: HBASE-14675 > URL: https://issues.apache.org/jira/browse/HBASE-14675 > Project: HBase > Issue Type: Improvement >Affects Versions: 2.0.0 >Reporter: Jonathan Hsieh >Assignee: Jonathan Hsieh > Fix For: 2.0.0 > > Attachments: hbase-14675.patch, hbase-14675.patch, > hbase-14675.v2.patch > > > The Put API changed from #add(...) to #addColumn(...). This updates all > instances of it and removes it from the Put (which was added for hbase 1.0.0) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14693) Add client-side metrics for received pushback signals
[
https://issues.apache.org/jira/browse/HBASE-14693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14973726#comment-14973726
]
Hadoop QA commented on HBASE-14693:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12768606/HBASE-14693.patch
against master branch at commit 8a2cef3315516501627c7a30bdcf989b12a32303.
ATTACHMENT ID: 12768606
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 4 new
or modified tests.
{color:red}-1 javac{color}. The patch appears to cause mvn compile goal to
fail with Hadoop version 2.4.0.
Compilation errors resume:
[ERROR] COMPILATION ERROR :
[ERROR]
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build@2/hbase/hbase-client/src/main/java/org/apache/hadoop/hbase/client/MetricsConnection.java:[174,28]
cannot find symbol
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-compiler-plugin:3.2:compile (default-compile) on
project hbase-client: Compilation failure
[ERROR]
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build@2/hbase/hbase-client/src/main/java/org/apache/hadoop/hbase/client/MetricsConnection.java:[174,28]
cannot find symbol
[ERROR] symbol: method
putIfAbsent(org.apache.hadoop.hbase.ServerName,java.util.concurrent.ConcurrentHashMap)
[ERROR] location: variable serverStats of type
java.util.Map>
[ERROR] -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e
switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please
read the following articles:
[ERROR] [Help 1]
http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR] mvn -rf :hbase-client
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/16225//console
This message is automatically generated.
> Add client-side metrics for received pushback signals
> -
>
> Key: HBASE-14693
> URL: https://issues.apache.org/jira/browse/HBASE-14693
> Project: HBase
> Issue Type: Improvement
>Reporter: Andrew Purtell
> Attachments: HBASE-14693.patch
>
>
> HBASE-12911 added client side metrics. HBASE-5162 added a mechanism for
> sending advisory backpressure signals to clients when the server is heavily
> loaded, and HBASE-12702 and subtasks backported this to all active branches.
> Add client-side metrics for received pushback signal.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14672) Exorcise deprecated Delete#delete* apis
[
https://issues.apache.org/jira/browse/HBASE-14672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14973528#comment-14973528
]
Hadoop QA commented on HBASE-14672:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12768626/hbase-14672-v2.patch
against master branch at commit 46c646d6154d6a5291c308e065802248f4e4e7b9.
ATTACHMENT ID: 12768626
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 105
new or modified tests.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.6.1 2.7.0
2.7.1)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 protoc{color}. The applied patch does not increase the
total number of protoc compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 checkstyle{color}. The applied patch does not increase the
total number of checkstyle errors
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn post-site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/16221//testReport/
Release Findbugs (version 2.0.3)warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/16221//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/16221//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/16221//console
This message is automatically generated.
> Exorcise deprecated Delete#delete* apis
> ---
>
> Key: HBASE-14672
> URL: https://issues.apache.org/jira/browse/HBASE-14672
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 2.0.0
>Reporter: Jonathan Hsieh
>Assignee: Jonathan Hsieh
> Fix For: 2.0.0
>
> Attachments: hbase-14672-v2.patch, hbase-14672.patch
>
>
> Delete#delete* apis were replaced with Delete#add* apis. This converts all
> instances of it and removes Delete#delete* apis.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14694) Scan copy constructor doesn't handle allowPartialResults
[ https://issues.apache.org/jira/browse/HBASE-14694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14973671#comment-14973671 ] Anoop Sam John commented on HBASE-14694: Also mentioned abt this attribute not honored while doing Scan object to String convert and vice versa. Want to fix that also here? > Scan copy constructor doesn't handle allowPartialResults > > > Key: HBASE-14694 > URL: https://issues.apache.org/jira/browse/HBASE-14694 > Project: HBase > Issue Type: Bug >Reporter: Ted Yu >Assignee: Ted Yu > Attachments: 14694-v1.txt > > > As [~kairs] reported in HBASE-11544, copy ctor of Scan doesn't copy > allowPartialResults. > This issue is to fix this defect. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14693) Add client-side metrics for received pushback signals
[ https://issues.apache.org/jira/browse/HBASE-14693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Heng Chen updated HBASE-14693: -- Status: Patch Available (was: Open) > Add client-side metrics for received pushback signals > - > > Key: HBASE-14693 > URL: https://issues.apache.org/jira/browse/HBASE-14693 > Project: HBase > Issue Type: Improvement >Reporter: Andrew Purtell > Attachments: HBASE-14693.patch > > > HBASE-12911 added client side metrics. HBASE-5162 added a mechanism for > sending advisory backpressure signals to clients when the server is heavily > loaded, and HBASE-12702 and subtasks backported this to all active branches. > Add client-side metrics for received pushback signal. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14684) Try to remove all MiniMapReduceCluster in unit tests
[
https://issues.apache.org/jira/browse/HBASE-14684?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14973738#comment-14973738
]
Hadoop QA commented on HBASE-14684:
---
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12768638/HBASE-14684_v1.patch
against master branch at commit 8a2cef3315516501627c7a30bdcf989b12a32303.
ATTACHMENT ID: 12768638
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 90 new
or modified tests.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.6.1 2.7.0
2.7.1)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 protoc{color}. The applied patch does not increase the
total number of protoc compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 checkstyle{color}. The applied patch does not increase the
total number of checkstyle errors
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn post-site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/16223//testReport/
Release Findbugs (version 2.0.3)warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/16223//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/16223//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/16223//console
This message is automatically generated.
> Try to remove all MiniMapReduceCluster in unit tests
>
>
> Key: HBASE-14684
> URL: https://issues.apache.org/jira/browse/HBASE-14684
> Project: HBase
> Issue Type: Improvement
> Components: test
>Reporter: Heng Chen
>Assignee: Heng Chen
>Priority: Critical
> Fix For: 2.0.0
>
> Attachments: HBASE-14684.patch, HBASE-14684_v1.patch
>
>
> As discussion in dev list, we will try to do MR job without
> MiniMapReduceCluster.
> Testcases will run faster and more reliable.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13867) Add endpoint coprocessor guide to HBase book
[ https://issues.apache.org/jira/browse/HBASE-13867?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14973571#comment-14973571 ] Hudson commented on HBASE-13867: FAILURE: Integrated in HBase-TRUNK #6959 (See [https://builds.apache.org/job/HBase-TRUNK/6959/]) HBASE-13867: Add endpoint coprocessor guide to HBase book. (mstanleyjones: rev 8a2cef3315516501627c7a30bdcf989b12a32303) * src/main/asciidoc/_chapters/cp.adoc > Add endpoint coprocessor guide to HBase book > > > Key: HBASE-13867 > URL: https://issues.apache.org/jira/browse/HBASE-13867 > Project: HBase > Issue Type: Task > Components: Coprocessors, documentation >Reporter: Vladimir Rodionov >Assignee: Gaurav Bhardwaj > Fix For: 2.0.0 > > Attachments: HBASE-13867.1.patch, HBASE-13867.2.patch, > HBASE-13867.2.patch, HBASE-13867.3.patch, HBASE-13867.4.patch > > > Endpoint coprocessors are very poorly documented. > Coprocessor section of HBase book must be updated either with its own > endpoint coprocessors HOW-TO guide or, at least, with the link(s) to some > other guides. There is good description here: > http://www.3pillarglobal.com/insights/hbase-coprocessors -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-14694) Scan copy constructor doesn't handle allowPartialResults
Ted Yu created HBASE-14694: -- Summary: Scan copy constructor doesn't handle allowPartialResults Key: HBASE-14694 URL: https://issues.apache.org/jira/browse/HBASE-14694 Project: HBase Issue Type: Bug Reporter: Ted Yu Assignee: Ted Yu As [~kairs] reported in HBASE-11544, copy ctor of Scan doesn't copy allowPartialResults. This issue is to fix this defect. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-11544) [Ergonomics] hbase.client.scanner.caching is dogged and will try to return batch even if it means OOME
[ https://issues.apache.org/jira/browse/HBASE-11544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14973336#comment-14973336 ] Mindaugas Kairys commented on HBASE-11544: -- And i also see that there is an issue with TableMapReduceUtil methods convertStringToScan and convertScanToString which also lose the field allowPartialResults. And perhaps it would be good to have a config property (hbase.mapreduce.scan.partialrows ?) for specifying this feature for a jobs. > [Ergonomics] hbase.client.scanner.caching is dogged and will try to return > batch even if it means OOME > -- > > Key: HBASE-11544 > URL: https://issues.apache.org/jira/browse/HBASE-11544 > Project: HBase > Issue Type: Bug >Reporter: stack >Assignee: Jonathan Lawlor >Priority: Critical > Fix For: 2.0.0, 1.1.0 > > Attachments: Allocation_Hot_Spots.html, > HBASE-11544-addendum-v1.patch, HBASE-11544-addendum-v2.patch, > HBASE-11544-branch_1_0-v1.patch, HBASE-11544-branch_1_0-v2.patch, > HBASE-11544-v1.patch, HBASE-11544-v2.patch, HBASE-11544-v3.patch, > HBASE-11544-v4.patch, HBASE-11544-v5.patch, HBASE-11544-v6.patch, > HBASE-11544-v6.patch, HBASE-11544-v6.patch, HBASE-11544-v7.patch, > HBASE-11544-v8-branch-1.patch, HBASE-11544-v8.patch, gc.j.png, h.png, > hits.j.png, m.png, mean.png, net.j.png, q (2).png > > > Running some tests, I set hbase.client.scanner.caching=1000. Dataset has > large cells. I kept OOME'ing. > Serverside, we should measure how much we've accumulated and return to the > client whatever we've gathered once we pass out a certain size threshold > rather than keep accumulating till we OOME. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14694) Scan copy constructor doesn't handle allowPartialResults
[
https://issues.apache.org/jira/browse/HBASE-14694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14973338#comment-14973338
]
Hadoop QA commented on HBASE-14694:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12768607/14694-v1.txt
against master branch at commit 46c646d6154d6a5291c308e065802248f4e4e7b9.
ATTACHMENT ID: 12768607
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.6.1 2.7.0
2.7.1)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 protoc{color}. The applied patch does not increase the
total number of protoc compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 checkstyle{color}. The applied patch does not increase the
total number of checkstyle errors
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn post-site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/16220//testReport/
Release Findbugs (version 2.0.3)warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/16220//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/16220//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/16220//console
This message is automatically generated.
> Scan copy constructor doesn't handle allowPartialResults
>
>
> Key: HBASE-14694
> URL: https://issues.apache.org/jira/browse/HBASE-14694
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Ted Yu
> Attachments: 14694-v1.txt
>
>
> As [~kairs] reported in HBASE-11544, copy ctor of Scan doesn't copy
> allowPartialResults.
> This issue is to fix this defect.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-14694) Scan copy constructor doesn't handle allowPartialResults
[ https://issues.apache.org/jira/browse/HBASE-14694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-14694: --- Attachment: 14694-v1.txt > Scan copy constructor doesn't handle allowPartialResults > > > Key: HBASE-14694 > URL: https://issues.apache.org/jira/browse/HBASE-14694 > Project: HBase > Issue Type: Bug >Reporter: Ted Yu >Assignee: Ted Yu > Attachments: 14694-v1.txt > > > As [~kairs] reported in HBASE-11544, copy ctor of Scan doesn't copy > allowPartialResults. > This issue is to fix this defect. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (HBASE-14694) Scan copy constructor doesn't handle allowPartialResults
[ https://issues.apache.org/jira/browse/HBASE-14694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HBASE-14694: --- Status: Patch Available (was: Open) > Scan copy constructor doesn't handle allowPartialResults > > > Key: HBASE-14694 > URL: https://issues.apache.org/jira/browse/HBASE-14694 > Project: HBase > Issue Type: Bug >Reporter: Ted Yu >Assignee: Ted Yu > Attachments: 14694-v1.txt > > > As [~kairs] reported in HBASE-11544, copy ctor of Scan doesn't copy > allowPartialResults. > This issue is to fix this defect. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-14680) Two configs for snapshot timeout and better defaults
[
https://issues.apache.org/jira/browse/HBASE-14680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14973310#comment-14973310
]
Heng Chen commented on HBASE-14680:
---
{code}
Fetching https://builds.apache.org/job/PreCommit-HBASE-Build/16212/console
Printing hanging tests
Hanging test : org.apache.hadoop.hbase.util.TestMergeTable
Hanging test : org.apache.hadoop.hbase.util.TestHBaseFsckOneRS
Hanging test : org.apache.hadoop.hbase.util.TestHBaseFsckTwoRS
Hanging test : org.apache.hadoop.hbase.util.TestMiniClusterLoadEncoded
Printing Failing tests
{code}
All hanged tests have no relates with this patch
> Two configs for snapshot timeout and better defaults
>
>
> Key: HBASE-14680
> URL: https://issues.apache.org/jira/browse/HBASE-14680
> Project: HBase
> Issue Type: Bug
>Reporter: Enis Soztutar
> Fix For: 2.0.0, 1.3.0, 1.2.1, 1.0.3, 1.1.3, 0.98.16
>
> Attachments: HBASE-14680.patch, HBASE-14680_v1.patch,
> HBASE-14680_v2.patch
>
>
> One of the clusters timed out taking a snapshot for a disabled table. The
> table is big enough, and the master operation takes more than 1 min to
> complete. However while trying to increase the timeout, we noticed that there
> are two parameters with very similar names configuring different things:
> {{hbase.snapshot.master.timeout.millis}} is defined in
> SnapshotDescriptionUtils and is send to client side and used in disabled
> table snapshot.
> {{hbase.snapshot.master.timeoutMillis}} is defined in SnapshotManager and
> used as the timeout for the procedure execution.
> So, there are a couple of improvements that we can do:
> - 1 min is too low for big tables. We need to set this to 5 min or 10 min by
> default. Even a 6T table which is medium sized fails.
> - Unify the two timeouts into one. Decide on either of them, and deprecate
> the other. Use the biggest one for BC.
> - Add the timeout to hbase-default.xml.
> - Why do we even have a timeout for disabled table snapshots? The master is
> doing the work so we should not timeout in any case.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14463) Severe performance downgrade when parallel reading a single key from BucketCache
[
https://issues.apache.org/jira/browse/HBASE-14463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14973266#comment-14973266
]
Ted Yu commented on HBASE-14463:
I think the latest round of performance benchmarking shows acceptable results.
+1 from me.
> Severe performance downgrade when parallel reading a single key from
> BucketCache
>
>
> Key: HBASE-14463
> URL: https://issues.apache.org/jira/browse/HBASE-14463
> Project: HBase
> Issue Type: Bug
>Affects Versions: 0.98.14, 1.1.2
>Reporter: Yu Li
>Assignee: Yu Li
> Fix For: 2.0.0, 1.2.0, 1.3.0, 0.98.16
>
> Attachments: GC_with_WeakObjectPool.png, HBASE-14463.patch,
> HBASE-14463_v11.patch, HBASE-14463_v12.patch, HBASE-14463_v2.patch,
> HBASE-14463_v3.patch, HBASE-14463_v4.patch, HBASE-14463_v5.patch,
> TestBucketCache-new_with_IdLock.png,
> TestBucketCache-new_with_IdReadWriteLock.png,
> TestBucketCache_with_IdLock-latest.png, TestBucketCache_with_IdLock.png,
> TestBucketCache_with_IdReadWriteLock-latest.png,
> TestBucketCache_with_IdReadWriteLock-resolveLockLeak.png,
> TestBucketCache_with_IdReadWriteLock.png, pe_use_same_keys.patch,
> test-results.tar.gz
>
>
> We store feature data of online items in HBase, do machine learning on these
> features, and supply the outputs to our online search engine. In such
> scenario we will launch hundreds of yarn workers and each worker will read
> all features of one item(i.e. single rowkey in HBase), so there'll be heavy
> parallel reading on a single rowkey.
> We were using LruCache but start to try BucketCache recently to resolve gc
> issue, and just as titled we have observed severe performance downgrade.
> After some analytics we found the root cause is the lock in
> BucketCache#getBlock, as shown below
> {code}
> try {
> lockEntry = offsetLock.getLockEntry(bucketEntry.offset());
> // ...
> if (bucketEntry.equals(backingMap.get(key))) {
> // ...
> int len = bucketEntry.getLength();
> Cacheable cachedBlock = ioEngine.read(bucketEntry.offset(), len,
> bucketEntry.deserializerReference(this.deserialiserMap));
> {code}
> Since ioEnging.read involves array copy, it's much more time-costed than the
> operation in LruCache. And since we're using synchronized in
> IdLock#getLockEntry, parallel read dropping on the same bucket would be
> executed in serial, which causes a really bad performance.
> To resolve the problem, we propose to use ReentranceReadWriteLock in
> BucketCache, and introduce a new class called IdReadWriteLock to implement it.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-13867) Add endpoint coprocessor guide to HBase book
[
https://issues.apache.org/jira/browse/HBASE-13867?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14973307#comment-14973307
]
Hadoop QA commented on HBASE-13867:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12768601/HBASE-13867.4.patch
against master branch at commit 46c646d6154d6a5291c308e065802248f4e4e7b9.
ATTACHMENT ID: 12768601
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified tests.
{color:green}+1 hadoop versions{color}. The patch compiles with all
supported hadoop versions (2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.6.1 2.7.0
2.7.1)
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 protoc{color}. The applied patch does not increase the
total number of protoc compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 checkstyle{color}. The applied patch does not increase the
total number of checkstyle errors
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 lineLengths{color}. The patch introduces the following lines
longer than 100:
+link:https://hbase.apache.org/devapidocs/org/apache/hadoop/hbase/coprocessor/BaseRegionObserver.html[BaseRegionObserver]
+link:https://hbase.apache.org/devapidocs/org/apache/hadoop/hbase/coprocessor/CoprocessorService.html[CoprocessorService]).
+link:https://hbase.apache.org/apidocs/org/apache/hadoop/hbase/coprocessor/package-summary.html[coprocessor]
+link:http://hbase.apache.org/devapidocs/org/apache/hadoop/hbase/coprocessor/RegionObserver.html#prePut%28org.apache.hadoop.hbase.coprocessor.ObserverContext,%20org.apache.hadoop.hbase.client.Put,%20org.apache.hadoop.hbase.regionserver.wal.WALEdit,%20org.apache.hadoop.hbase.client.Durability%29[`prePut`]
+link:http://hbase.apache.org/devapidocs/org/apache/hadoop/hbase/coprocessor/RegionObserver.html[RegionCoprocessor].
+link:http://hbase.apache.org/devapidocs/org/apache/hadoop/hbase/coprocessor/RegionObserver.html#postPut%28org.apache.hadoop.hbase.coprocessor.ObserverContext,%20org.apache.hadoop.hbase.client.Put,%20org.apache.hadoop.hbase.regionserver.wal.WALEdit,%20org.apache.hadoop.hbase.client.Durability%29[`postPut`]
+link:http://hbase.apache.org/devapidocs/org/apache/hadoop/hbase/coprocessor/RegionObserver.html[RegionCoprocessor]:
+link:https://hbase.apache.org/devapidocs/org/apache/hadoop/hbase/coprocessor/RegionObserver.html[RegionObserver].
+link:https://hbase.apache.org/devapidocs/org/apache/hadoop/hbase/coprocessor/RegionServerObserver.html[RegionServerObserver].
+link:https://hbase.apache.org/devapidocs/org/apache/hadoop/hbase/coprocessor/MasterObserver.html[MasterObserver].
{color:green}+1 site{color}. The mvn post-site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/16219//testReport/
Release Findbugs (version 2.0.3)warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/16219//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors:
https://builds.apache.org/job/PreCommit-HBASE-Build/16219//artifact/patchprocess/checkstyle-aggregate.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/16219//console
This message is automatically generated.
> Add endpoint coprocessor guide to HBase book
>
>
> Key: HBASE-13867
> URL: https://issues.apache.org/jira/browse/HBASE-13867
> Project: HBase
> Issue Type: Task
> Components: Coprocessors, documentation
>Reporter: Vladimir Rodionov
>Assignee: Gaurav Bhardwaj
> Fix For: 2.0.0
>
> Attachments: HBASE-13867.1.patch, HBASE-13867.2.patch,
> HBASE-13867.2.patch, HBASE-13867.3.patch, HBASE-13867.4.patch
>
>
> Endpoint coprocessors are very poorly documented.
> Coprocessor section of HBase book must be updated either with its own
> endpoint coprocessors HOW-TO guide or, at least, with the link(s) to some
> other guides. There is good description here:
> http://www.3pillarglobal.com/insights/hbase-coprocessors
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HBASE-14684) Try to remove all MiniMapReduceCluster in unit tests
[
https://issues.apache.org/jira/browse/HBASE-14684?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14973313#comment-14973313
]
Heng Chen commented on HBASE-14684:
---
{code}
org.apache.hadoop.hbase.client.TestReplicaWithCluster.xml : XML document
structures must start and end within the same entity. Nested exception: XML
document structures must start and end within the same entity.
at org.dom4j.io.SAXReader.read(SAXReader.java:482)
at org.dom4j.io.SAXReader.read(SAXReader.java:264)
at hudson.tasks.junit.SuiteResult.parse(SuiteResult.java:123)
at hudson.tasks.junit.TestResult.parse(TestResult.java:282)
at hudson.tasks.junit.TestResult.parsePossiblyEmpty(TestResult.java:228)
at hudson.tasks.junit.TestResult.parse(TestResult.java:163)
at hudson.tasks.junit.TestResult.parse(TestResult.java:146)
at hudson.tasks.junit.TestResult.(TestResult.java:122)
at
hudson.tasks.junit.JUnitParser$ParseResultCallable.invoke(JUnitParser.java:119)
at
hudson.tasks.junit.JUnitParser$ParseResultCallable.invoke(JUnitParser.java:93)
at hudson.FilePath$FileCallableWrapper.call(FilePath.java:2688)
at hudson.remoting.UserRequest.perform(UserRequest.java:121)
at hudson.remoting.UserRequest.perform(UserRequest.java:49)
at hudson.remoting.Request$2.run(Request.java:326)
at
hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:68)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
{code}
I found tests failed frequently when parse xml. Why?
Is this xml file written by other process when i read it?
> Try to remove all MiniMapReduceCluster in unit tests
>
>
> Key: HBASE-14684
> URL: https://issues.apache.org/jira/browse/HBASE-14684
> Project: HBase
> Issue Type: Improvement
> Components: test
>Reporter: Heng Chen
>Assignee: Heng Chen
>Priority: Critical
> Fix For: 2.0.0
>
> Attachments: HBASE-14684.patch, HBASE-14684_v1.patch
>
>
> As discussion in dev list, we will try to do MR job without
> MiniMapReduceCluster.
> Testcases will run faster and more reliable.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
