[jira] [Commented] (HBASE-6870) HTable#coprocessorExec always scan the whole table
[ https://issues.apache.org/jira/browse/HBASE-6870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13758372#comment-13758372 ] Jonathan Hsieh commented on HBASE-6870: --- updated the description to make it easier to understand > HTable#coprocessorExec always scan the whole table > --- > > Key: HBASE-6870 > URL: https://issues.apache.org/jira/browse/HBASE-6870 > Project: HBase > Issue Type: Improvement > Components: Coprocessors >Affects Versions: 0.94.1, 0.95.0, 0.95.2 >Reporter: chunhui shen >Assignee: chunhui shen >Priority: Critical > Fix For: 0.98.0, 0.94.8, 0.95.1 > > Attachments: 6870-v4.txt, hbase-0.94-6870v6.patch, > hbase-0.95-6870v6.patch, HBASE-6870.patch, HBASE-6870-testPerformance.patch, > HBASE-6870v2.patch, HBASE-6870v3.patch, hbase-6870v5.patch, hbase-6870v6.patch > > > In current logic, HTable#coprocessorExec always scans the entire META table, > loading it into memory and then filters the keys to return only those that > fall in specified range. The version after the patch only scans the portions > of meta that are in the specified key range, and returns them. Put simply -- > before we did a load-all-then-filter; afterwards we only-scan-what-is-needed. > The former has low efficiency and greatly impacts the Regionserver carrying > .META. when there are many coprocessorExec requests. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6870) HTable#coprocessorExec always scan the whole table
[ https://issues.apache.org/jira/browse/HBASE-6870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13658900#comment-13658900 ] Hudson commented on HBASE-6870: --- Integrated in HBase-0.94-security #141 (See [https://builds.apache.org/job/HBase-0.94-security/141/]) HBASE-6870 HTable#coprocessorExec always scan the whole table (Revision 1470374) Result = FAILURE zjushch : Files : * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/client/HTable.java > HTable#coprocessorExec always scan the whole table > --- > > Key: HBASE-6870 > URL: https://issues.apache.org/jira/browse/HBASE-6870 > Project: HBase > Issue Type: Improvement > Components: Coprocessors >Affects Versions: 0.94.1, 0.95.0, 0.95.2 >Reporter: chunhui shen >Assignee: chunhui shen >Priority: Critical > Fix For: 0.98.0, 0.94.8, 0.95.1 > > Attachments: 6870-v4.txt, hbase-0.94-6870v6.patch, > hbase-0.95-6870v6.patch, HBASE-6870.patch, HBASE-6870-testPerformance.patch, > HBASE-6870v2.patch, HBASE-6870v3.patch, hbase-6870v5.patch, hbase-6870v6.patch > > > In current logic, HTable#coprocessorExec always scan the whole table, its > efficiency is low and will affect the Regionserver carrying .META. under > large coprocessorExec requests -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6870) HTable#coprocessorExec always scan the whole table
[ https://issues.apache.org/jira/browse/HBASE-6870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13638846#comment-13638846 ] Hudson commented on HBASE-6870: --- Integrated in HBase-0.94 #964 (See [https://builds.apache.org/job/HBase-0.94/964/]) HBASE-6870 HTable#coprocessorExec always scan the whole table (Revision 1470374) Result = SUCCESS zjushch : Files : * /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/client/HTable.java > HTable#coprocessorExec always scan the whole table > --- > > Key: HBASE-6870 > URL: https://issues.apache.org/jira/browse/HBASE-6870 > Project: HBase > Issue Type: Improvement > Components: Coprocessors >Affects Versions: 0.94.1, 0.95.0, 0.95.2 >Reporter: chunhui shen >Assignee: chunhui shen >Priority: Critical > Fix For: 0.98.0, 0.94.8, 0.95.1 > > Attachments: 6870-v4.txt, hbase-0.94-6870v6.patch, > hbase-0.95-6870v6.patch, HBASE-6870.patch, HBASE-6870-testPerformance.patch, > HBASE-6870v2.patch, HBASE-6870v3.patch, hbase-6870v5.patch, hbase-6870v6.patch > > > In current logic, HTable#coprocessorExec always scan the whole table, its > efficiency is low and will affect the Regionserver carrying .META. under > large coprocessorExec requests -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6870) HTable#coprocessorExec always scan the whole table
[ https://issues.apache.org/jira/browse/HBASE-6870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13638425#comment-13638425 ] Hudson commented on HBASE-6870: --- Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #506 (See [https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/506/]) HBASE-6870 HTable#coprocessorExec always scan the whole table (Revision 1470376) Result = FAILURE zjushch : Files : * /hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/client/HTable.java > HTable#coprocessorExec always scan the whole table > --- > > Key: HBASE-6870 > URL: https://issues.apache.org/jira/browse/HBASE-6870 > Project: HBase > Issue Type: Improvement > Components: Coprocessors >Affects Versions: 0.94.1, 0.95.0, 0.95.2 >Reporter: chunhui shen >Assignee: chunhui shen >Priority: Critical > Fix For: 0.98.0, 0.94.8, 0.95.1 > > Attachments: 6870-v4.txt, hbase-0.94-6870v6.patch, > hbase-0.95-6870v6.patch, HBASE-6870.patch, HBASE-6870-testPerformance.patch, > HBASE-6870v2.patch, HBASE-6870v3.patch, hbase-6870v5.patch, hbase-6870v6.patch > > > In current logic, HTable#coprocessorExec always scan the whole table, its > efficiency is low and will affect the Regionserver carrying .META. under > large coprocessorExec requests -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6870) HTable#coprocessorExec always scan the whole table
[ https://issues.apache.org/jira/browse/HBASE-6870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13638307#comment-13638307 ] Hudson commented on HBASE-6870: --- Integrated in hbase-0.95-on-hadoop2 #76 (See [https://builds.apache.org/job/hbase-0.95-on-hadoop2/76/]) HBASE-6870 HTable#coprocessorExec always scan the whole table (Revision 1470375) Result = FAILURE zjushch : Files : * /hbase/branches/0.95/hbase-client/src/main/java/org/apache/hadoop/hbase/client/HTable.java > HTable#coprocessorExec always scan the whole table > --- > > Key: HBASE-6870 > URL: https://issues.apache.org/jira/browse/HBASE-6870 > Project: HBase > Issue Type: Improvement > Components: Coprocessors >Affects Versions: 0.94.1, 0.95.0, 0.95.2 >Reporter: chunhui shen >Assignee: chunhui shen >Priority: Critical > Fix For: 0.98.0, 0.94.8, 0.95.1 > > Attachments: 6870-v4.txt, hbase-0.94-6870v6.patch, > hbase-0.95-6870v6.patch, HBASE-6870.patch, HBASE-6870-testPerformance.patch, > HBASE-6870v2.patch, HBASE-6870v3.patch, hbase-6870v5.patch, hbase-6870v6.patch > > > In current logic, HTable#coprocessorExec always scan the whole table, its > efficiency is low and will affect the Regionserver carrying .META. under > large coprocessorExec requests -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6870) HTable#coprocessorExec always scan the whole table
[ https://issues.apache.org/jira/browse/HBASE-6870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13638105#comment-13638105 ] Hudson commented on HBASE-6870: --- Integrated in hbase-0.95 #156 (See [https://builds.apache.org/job/hbase-0.95/156/]) HBASE-6870 HTable#coprocessorExec always scan the whole table (Revision 1470375) Result = FAILURE zjushch : Files : * /hbase/branches/0.95/hbase-client/src/main/java/org/apache/hadoop/hbase/client/HTable.java > HTable#coprocessorExec always scan the whole table > --- > > Key: HBASE-6870 > URL: https://issues.apache.org/jira/browse/HBASE-6870 > Project: HBase > Issue Type: Improvement > Components: Coprocessors >Affects Versions: 0.94.1, 0.95.0, 0.95.2 >Reporter: chunhui shen >Assignee: chunhui shen >Priority: Critical > Fix For: 0.98.0, 0.94.8, 0.95.1 > > Attachments: 6870-v4.txt, hbase-0.94-6870v6.patch, > hbase-0.95-6870v6.patch, HBASE-6870.patch, HBASE-6870-testPerformance.patch, > HBASE-6870v2.patch, HBASE-6870v3.patch, hbase-6870v5.patch, hbase-6870v6.patch > > > In current logic, HTable#coprocessorExec always scan the whole table, its > efficiency is low and will affect the Regionserver carrying .META. under > large coprocessorExec requests -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6870) HTable#coprocessorExec always scan the whole table
[ https://issues.apache.org/jira/browse/HBASE-6870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13638062#comment-13638062 ] Hudson commented on HBASE-6870: --- Integrated in HBase-TRUNK #4073 (See [https://builds.apache.org/job/HBase-TRUNK/4073/]) HBASE-6870 HTable#coprocessorExec always scan the whole table (Revision 1470376) Result = SUCCESS zjushch : Files : * /hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/client/HTable.java > HTable#coprocessorExec always scan the whole table > --- > > Key: HBASE-6870 > URL: https://issues.apache.org/jira/browse/HBASE-6870 > Project: HBase > Issue Type: Improvement > Components: Coprocessors >Affects Versions: 0.94.1, 0.95.0, 0.95.2 >Reporter: chunhui shen >Assignee: chunhui shen >Priority: Critical > Fix For: 0.98.0, 0.94.8, 0.95.1 > > Attachments: 6870-v4.txt, hbase-0.94-6870v6.patch, > hbase-0.95-6870v6.patch, HBASE-6870.patch, HBASE-6870-testPerformance.patch, > HBASE-6870v2.patch, HBASE-6870v3.patch, hbase-6870v5.patch, hbase-6870v6.patch > > > In current logic, HTable#coprocessorExec always scan the whole table, its > efficiency is low and will affect the Regionserver carrying .META. under > large coprocessorExec requests -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6870) HTable#coprocessorExec always scan the whole table
[ https://issues.apache.org/jira/browse/HBASE-6870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13637425#comment-13637425 ] Ted Yu commented on HBASE-6870: --- +1 from me as well. > HTable#coprocessorExec always scan the whole table > --- > > Key: HBASE-6870 > URL: https://issues.apache.org/jira/browse/HBASE-6870 > Project: HBase > Issue Type: Improvement > Components: Coprocessors >Affects Versions: 0.94.1, 0.95.0, 0.95.2 >Reporter: chunhui shen >Assignee: chunhui shen >Priority: Critical > Fix For: 0.98.0, 0.95.1 > > Attachments: 6870-v4.txt, HBASE-6870.patch, > HBASE-6870-testPerformance.patch, HBASE-6870v2.patch, HBASE-6870v3.patch, > hbase-6870v5.patch, hbase-6870v6.patch > > > In current logic, HTable#coprocessorExec always scan the whole table, its > efficiency is low and will affect the Regionserver carrying .META. under > large coprocessorExec requests -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6870) HTable#coprocessorExec always scan the whole table
[ https://issues.apache.org/jira/browse/HBASE-6870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13637302#comment-13637302 ] Nicolas Liochon commented on HBASE-6870: +1 for v6! > HTable#coprocessorExec always scan the whole table > --- > > Key: HBASE-6870 > URL: https://issues.apache.org/jira/browse/HBASE-6870 > Project: HBase > Issue Type: Improvement > Components: Coprocessors >Affects Versions: 0.94.1, 0.95.0, 0.95.2 >Reporter: chunhui shen >Assignee: chunhui shen >Priority: Critical > Fix For: 0.98.0, 0.95.1 > > Attachments: 6870-v4.txt, HBASE-6870.patch, > HBASE-6870-testPerformance.patch, HBASE-6870v2.patch, HBASE-6870v3.patch, > hbase-6870v5.patch, hbase-6870v6.patch > > > In current logic, HTable#coprocessorExec always scan the whole table, its > efficiency is low and will affect the Regionserver carrying .META. under > large coprocessorExec requests -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6870) HTable#coprocessorExec always scan the whole table
[
https://issues.apache.org/jira/browse/HBASE-6870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13637193#comment-13637193
]
Hadoop QA commented on HBASE-6870:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12579669/hbase-6870v6.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/5371//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5371//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5371//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5371//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5371//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5371//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5371//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5371//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5371//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/5371//console
This message is automatically generated.
> HTable#coprocessorExec always scan the whole table
> ---
>
> Key: HBASE-6870
> URL: https://issues.apache.org/jira/browse/HBASE-6870
> Project: HBase
> Issue Type: Improvement
> Components: Coprocessors
>Affects Versions: 0.94.1, 0.95.0, 0.95.2
>Reporter: chunhui shen
>Assignee: chunhui shen
>Priority: Critical
> Fix For: 0.98.0, 0.95.1
>
> Attachments: 6870-v4.txt, HBASE-6870.patch,
> HBASE-6870-testPerformance.patch, HBASE-6870v2.patch, HBASE-6870v3.patch,
> hbase-6870v5.patch, hbase-6870v6.patch
>
>
> In current logic, HTable#coprocessorExec always scan the whole table, its
> efficiency is low and will affect the Regionserver carrying .META. under
> large coprocessorExec requests
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6870) HTable#coprocessorExec always scan the whole table
[ https://issues.apache.org/jira/browse/HBASE-6870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13637168#comment-13637168 ] chunhui shen commented on HBASE-6870: - [~liochon] Thanks for the suggestion. Addressing these comments in patch v6. Now using a Pair of List instead of Map. > HTable#coprocessorExec always scan the whole table > --- > > Key: HBASE-6870 > URL: https://issues.apache.org/jira/browse/HBASE-6870 > Project: HBase > Issue Type: Improvement > Components: Coprocessors >Affects Versions: 0.94.1, 0.95.0, 0.95.2 >Reporter: chunhui shen >Assignee: chunhui shen >Priority: Critical > Fix For: 0.98.0, 0.95.1 > > Attachments: 6870-v4.txt, HBASE-6870.patch, > HBASE-6870-testPerformance.patch, HBASE-6870v2.patch, HBASE-6870v3.patch, > hbase-6870v5.patch > > > In current logic, HTable#coprocessorExec always scan the whole table, its > efficiency is low and will affect the Regionserver carrying .META. under > large coprocessorExec requests -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6870) HTable#coprocessorExec always scan the whole table
[ https://issues.apache.org/jira/browse/HBASE-6870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13632652#comment-13632652 ] Nicolas Liochon commented on HBASE-6870: Fascinating improvement I would say :-) Some comments: - You've reorganized the imports, that will create conflicts when people will merge their patches ;-] - There is a parameter, but it's never used useCache with a value set to false. Wouldn't be better to remove it? - Just a question: why a LinkedHashMap? - The method should return the interface (Map) but not the real type. - Using the Map with a byte[] is brittle imho. - Does getKeysToRegionsInRange needs to be public? > HTable#coprocessorExec always scan the whole table > --- > > Key: HBASE-6870 > URL: https://issues.apache.org/jira/browse/HBASE-6870 > Project: HBase > Issue Type: Improvement > Components: Coprocessors >Affects Versions: 0.94.1, 0.95.0, 0.95.2 >Reporter: chunhui shen >Assignee: chunhui shen >Priority: Critical > Fix For: 0.98.0, 0.95.1 > > Attachments: 6870-v4.txt, HBASE-6870.patch, > HBASE-6870-testPerformance.patch, HBASE-6870v2.patch, HBASE-6870v3.patch, > hbase-6870v5.patch > > > In current logic, HTable#coprocessorExec always scan the whole table, its > efficiency is low and will affect the Regionserver carrying .META. under > large coprocessorExec requests -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6870) HTable#coprocessorExec always scan the whole table
[ https://issues.apache.org/jira/browse/HBASE-6870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13628586#comment-13628586 ] chunhui shen commented on HBASE-6870: - About the performance: Total 256 regions in the table Concurrent 170 thread Each thread do AggregationClient#rowCount for one region Before the patch: Total 5s+ After the patch: Total 200ms+ > HTable#coprocessorExec always scan the whole table > --- > > Key: HBASE-6870 > URL: https://issues.apache.org/jira/browse/HBASE-6870 > Project: HBase > Issue Type: Improvement > Components: Coprocessors >Affects Versions: 0.94.1, 0.95.0, 0.95.2 >Reporter: chunhui shen >Assignee: chunhui shen >Priority: Critical > Fix For: 0.98.0, 0.95.1 > > Attachments: 6870-v4.txt, HBASE-6870.patch, > HBASE-6870-testPerformance.patch, HBASE-6870v2.patch, HBASE-6870v3.patch, > hbase-6870v5.patch > > > In current logic, HTable#coprocessorExec always scan the whole table, its > efficiency is low and will affect the Regionserver carrying .META. under > large coprocessorExec requests -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6870) HTable#coprocessorExec always scan the whole table
[
https://issues.apache.org/jira/browse/HBASE-6870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13624631#comment-13624631
]
Hadoop QA commented on HBASE-6870:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12577422/hbase-6870v5.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.zookeeper.lock.TestZKInterProcessReadWriteLock
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/5164//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5164//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5164//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5164//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5164//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5164//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5164//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5164//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5164//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/5164//console
This message is automatically generated.
> HTable#coprocessorExec always scan the whole table
> ---
>
> Key: HBASE-6870
> URL: https://issues.apache.org/jira/browse/HBASE-6870
> Project: HBase
> Issue Type: Improvement
> Components: Coprocessors
>Affects Versions: 0.94.1, 0.95.0, 0.95.2
>Reporter: chunhui shen
>Assignee: chunhui shen
>Priority: Critical
> Fix For: 0.95.1, 0.98.0
>
> Attachments: 6870-v4.txt, HBASE-6870.patch,
> HBASE-6870-testPerformance.patch, HBASE-6870v2.patch, HBASE-6870v3.patch,
> hbase-6870v5.patch
>
>
> In current logic, HTable#coprocessorExec always scan the whole table, its
> efficiency is low and will affect the Regionserver carrying .META. under
> large coprocessorExec requests
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6870) HTable#coprocessorExec always scan the whole table
[ https://issues.apache.org/jira/browse/HBASE-6870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13624605#comment-13624605 ] chunhui shen commented on HBASE-6870: - bq.When useCache is true, what caching mechanism would getKeysToRegionsInRange() use ? We will use the cached region locations in HConnectionImplementation > HTable#coprocessorExec always scan the whole table > --- > > Key: HBASE-6870 > URL: https://issues.apache.org/jira/browse/HBASE-6870 > Project: HBase > Issue Type: Improvement > Components: Coprocessors >Affects Versions: 0.94.1, 0.95.0, 0.95.2 >Reporter: chunhui shen >Assignee: chunhui shen >Priority: Critical > Fix For: 0.95.1, 0.98.0 > > Attachments: 6870-v4.txt, HBASE-6870.patch, > HBASE-6870-testPerformance.patch, HBASE-6870v2.patch, HBASE-6870v3.patch, > hbase-6870v5.patch > > > In current logic, HTable#coprocessorExec always scan the whole table, its > efficiency is low and will affect the Regionserver carrying .META. under > large coprocessorExec requests -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6870) HTable#coprocessorExec always scan the whole table
[
https://issues.apache.org/jira/browse/HBASE-6870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13624601#comment-13624601
]
Ted Yu commented on HBASE-6870:
---
{code}
+if (useCache) {
+ return new ArrayList(getKeysToRegionsInRange(start, end, true)
+ .keySet());
{code}
When useCache is true, what caching mechanism would getKeysToRegionsInRange()
use ?
> HTable#coprocessorExec always scan the whole table
> ---
>
> Key: HBASE-6870
> URL: https://issues.apache.org/jira/browse/HBASE-6870
> Project: HBase
> Issue Type: Improvement
> Components: Coprocessors
>Affects Versions: 0.94.1, 0.95.0, 0.95.2
>Reporter: chunhui shen
>Assignee: chunhui shen
>Priority: Critical
> Fix For: 0.95.1, 0.98.0
>
> Attachments: 6870-v4.txt, HBASE-6870.patch,
> HBASE-6870-testPerformance.patch, HBASE-6870v2.patch, HBASE-6870v3.patch,
> hbase-6870v5.patch
>
>
> In current logic, HTable#coprocessorExec always scan the whole table, its
> efficiency is low and will affect the Regionserver carrying .META. under
> large coprocessorExec requests
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6870) HTable#coprocessorExec always scan the whole table
[ https://issues.apache.org/jira/browse/HBASE-6870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13624599#comment-13624599 ] chunhui shen commented on HBASE-6870: - bq.How come useCache being true gets translated into includeEndKey being true ? The useCache flag is not related with includeEndKey, includeEndKey is always true in HTable#coprocessorService. bq.I wonder if we should provide a mechanism for coprocessors to specify whether fresh .META. scan is wanted. Making a useCache flag is used for performance comparision in my initial purpose, I think there's no necessary to provide a choice for fresh .META. scan since it is still able to get a invalid region location from the .META. > HTable#coprocessorExec always scan the whole table > --- > > Key: HBASE-6870 > URL: https://issues.apache.org/jira/browse/HBASE-6870 > Project: HBase > Issue Type: Improvement > Components: Coprocessors >Affects Versions: 0.94.1, 0.95.0, 0.95.2 >Reporter: chunhui shen >Assignee: chunhui shen >Priority: Critical > Fix For: 0.95.1, 0.98.0 > > Attachments: 6870-v4.txt, HBASE-6870.patch, > HBASE-6870-testPerformance.patch, HBASE-6870v2.patch, HBASE-6870v3.patch, > hbase-6870v5.patch > > > In current logic, HTable#coprocessorExec always scan the whole table, its > efficiency is low and will affect the Regionserver carrying .META. under > large coprocessorExec requests -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6870) HTable#coprocessorExec always scan the whole table
[
https://issues.apache.org/jira/browse/HBASE-6870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13622619#comment-13622619
]
Hadoop QA commented on HBASE-6870:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12577014/6870-v4.txt
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 3
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 lineLengths{color}. The patch introduces lines longer than
100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/5136//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5136//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5136//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5136//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5136//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5136//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5136//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5136//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/5136//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/5136//console
This message is automatically generated.
> HTable#coprocessorExec always scan the whole table
> ---
>
> Key: HBASE-6870
> URL: https://issues.apache.org/jira/browse/HBASE-6870
> Project: HBase
> Issue Type: Improvement
> Components: Coprocessors
>Affects Versions: 0.94.1, 0.95.0, 0.96.0
>Reporter: chunhui shen
>Assignee: chunhui shen
>Priority: Critical
> Fix For: 0.95.1, 0.98.0
>
> Attachments: 6870-v4.txt, HBASE-6870.patch,
> HBASE-6870-testPerformance.patch, HBASE-6870v2.patch, HBASE-6870v3.patch
>
>
> In current logic, HTable#coprocessorExec always scan the whole table, its
> efficiency is low and will affect the Regionserver carrying .META. under
> large coprocessorExec requests
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6870) HTable#coprocessorExec always scan the whole table
[
https://issues.apache.org/jira/browse/HBASE-6870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13622582#comment-13622582
]
Ted Yu commented on HBASE-6870:
---
{code}
+ public LinkedHashMap getKeysToRegionsInRange(
+ final byte[] startKey, final byte[] endKey, final boolean includeEndKey)
{code}
Later we have:
{code}
+if (useCache) {
+ return new ArrayList(getKeysToRegionsInRange(start, end, true)
+ .keySet());
{code}
How come useCache being true gets translated into includeEndKey being true ?
I wonder if we should provide a mechanism for coprocessors to specify whether
fresh .META. scan is wanted.
> HTable#coprocessorExec always scan the whole table
> ---
>
> Key: HBASE-6870
> URL: https://issues.apache.org/jira/browse/HBASE-6870
> Project: HBase
> Issue Type: Improvement
> Components: Coprocessors
>Affects Versions: 0.94.1, 0.95.0, 0.96.0
>Reporter: chunhui shen
>Assignee: chunhui shen
>Priority: Critical
> Fix For: 0.95.1, 0.98.0
>
> Attachments: 6870-v4.txt, HBASE-6870.patch,
> HBASE-6870-testPerformance.patch, HBASE-6870v2.patch, HBASE-6870v3.patch
>
>
> In current logic, HTable#coprocessorExec always scan the whole table, its
> efficiency is low and will affect the Regionserver carrying .META. under
> large coprocessorExec requests
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6870) HTable#coprocessorExec always scan the whole table
[ https://issues.apache.org/jira/browse/HBASE-6870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13621706#comment-13621706 ] Lars Hofhansl commented on HBASE-6870: -- Seems like a fix we'd want in 0.94 as well. > HTable#coprocessorExec always scan the whole table > --- > > Key: HBASE-6870 > URL: https://issues.apache.org/jira/browse/HBASE-6870 > Project: HBase > Issue Type: Improvement > Components: Coprocessors >Affects Versions: 0.94.1, 0.95.0, 0.96.0 >Reporter: chunhui shen >Assignee: chunhui shen >Priority: Critical > Fix For: 0.95.1, 0.98.0 > > Attachments: HBASE-6870.patch, HBASE-6870-testPerformance.patch, > HBASE-6870v2.patch, HBASE-6870v3.patch > > > In current logic, HTable#coprocessorExec always scan the whole table, its > efficiency is low and will affect the Regionserver carrying .META. under > large coprocessorExec requests -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6870) HTable#coprocessorExec always scan the whole table
[ https://issues.apache.org/jira/browse/HBASE-6870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13621701#comment-13621701 ] chunhui shen commented on HBASE-6870: - Current trunk still exists this problem: HTable#coprocessorService -> getStartKeysInRange -> getStartEndKeys -> getRegionLocations -> MetaScanner.allTableRegions(getConfiguration(), getTableName(), false) Scanning the whole table regions for each coprocessorService... I will update the patch after the vocation. Thanks for reviving, I have already forgot this one... > HTable#coprocessorExec always scan the whole table > --- > > Key: HBASE-6870 > URL: https://issues.apache.org/jira/browse/HBASE-6870 > Project: HBase > Issue Type: Improvement > Components: Coprocessors >Affects Versions: 0.94.1, 0.95.0, 0.96.0 >Reporter: chunhui shen >Assignee: chunhui shen >Priority: Critical > Fix For: 0.95.1, 0.98.0 > > Attachments: HBASE-6870.patch, HBASE-6870-testPerformance.patch, > HBASE-6870v2.patch, HBASE-6870v3.patch > > > In current logic, HTable#coprocessorExec always scan the whole table, its > efficiency is low and will affect the Regionserver carrying .META. under > large coprocessorExec requests -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6870) HTable#coprocessorExec always scan the whole table
[ https://issues.apache.org/jira/browse/HBASE-6870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13617635#comment-13617635 ] Nicolas Liochon commented on HBASE-6870: I'm bumping the priority a little, because imho it means that anything that happen on .meta. will have a strong impact for anyone using coprocessors. I can help if needed of course. > HTable#coprocessorExec always scan the whole table > --- > > Key: HBASE-6870 > URL: https://issues.apache.org/jira/browse/HBASE-6870 > Project: HBase > Issue Type: Improvement > Components: Coprocessors >Affects Versions: 0.94.1, 0.95.0, 0.96.0 >Reporter: chunhui shen >Assignee: chunhui shen > Attachments: HBASE-6870.patch, HBASE-6870-testPerformance.patch, > HBASE-6870v2.patch, HBASE-6870v3.patch > > > In current logic, HTable#coprocessorExec always scan the whole table, its > efficiency is low and will affect the Regionserver carrying .META. under > large coprocessorExec requests -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6870) HTable#coprocessorExec always scan the whole table
[ https://issues.apache.org/jira/browse/HBASE-6870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13593790#comment-13593790 ] Gary Helmling commented on HBASE-6870: -- [~yuzhih...@gmail.com] Yes, I'm looking through the last patch to get up to speed. > HTable#coprocessorExec always scan the whole table > --- > > Key: HBASE-6870 > URL: https://issues.apache.org/jira/browse/HBASE-6870 > Project: HBase > Issue Type: Improvement > Components: Coprocessors >Affects Versions: 0.94.1 >Reporter: chunhui shen >Assignee: chunhui shen > Attachments: HBASE-6870.patch, HBASE-6870-testPerformance.patch, > HBASE-6870v2.patch, HBASE-6870v3.patch > > > In current logic, HTable#coprocessorExec always scan the whole table, its > efficiency is low and will affect the Regionserver carrying .META. under > large coprocessorExec requests -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6870) HTable#coprocessorExec always scan the whole table
[ https://issues.apache.org/jira/browse/HBASE-6870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13593772#comment-13593772 ] Ted Yu commented on HBASE-6870: --- [~ghelmling]: Do you think this issue should be revived ? > HTable#coprocessorExec always scan the whole table > --- > > Key: HBASE-6870 > URL: https://issues.apache.org/jira/browse/HBASE-6870 > Project: HBase > Issue Type: Improvement > Components: Coprocessors >Affects Versions: 0.94.1 >Reporter: chunhui shen >Assignee: chunhui shen > Attachments: HBASE-6870.patch, HBASE-6870-testPerformance.patch, > HBASE-6870v2.patch, HBASE-6870v3.patch > > > In current logic, HTable#coprocessorExec always scan the whole table, its > efficiency is low and will affect the Regionserver carrying .META. under > large coprocessorExec requests -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6870) HTable#coprocessorExec always scan the whole table
[
https://issues.apache.org/jira/browse/HBASE-6870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13463466#comment-13463466
]
chunhui shen commented on HBASE-6870:
-
[~v.himanshu]
These two if statements is not made by this patch, so I just keep the previous.
{code}
public LinkedHashMap getKeysToRegionsInRange(
{code}
Yes, it could be private.
Thanks for the review.
I will rework patch with other comments later
> HTable#coprocessorExec always scan the whole table
> ---
>
> Key: HBASE-6870
> URL: https://issues.apache.org/jira/browse/HBASE-6870
> Project: HBase
> Issue Type: Improvement
> Components: Coprocessors
>Affects Versions: 0.94.1
>Reporter: chunhui shen
>Assignee: chunhui shen
> Attachments: HBASE-6870.patch, HBASE-6870-testPerformance.patch,
> HBASE-6870v2.patch, HBASE-6870v3.patch
>
>
> In current logic, HTable#coprocessorExec always scan the whole table, its
> efficiency is low and will affect the Regionserver carrying .META. under
> large coprocessorExec requests
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6870) HTable#coprocessorExec always scan the whole table
[
https://issues.apache.org/jira/browse/HBASE-6870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13463031#comment-13463031
]
Himanshu Vashishtha commented on HBASE-6870:
Looked at the patch:
Can you make the these two if statements in-line
{code}
+if (Bytes.compareTo(start, startKeys[i]) >= 0) {
+ if (Bytes.equals(endKeys[i], HConstants.EMPTY_END_ROW)
+ || Bytes.compareTo(start, endKeys[i]) < 0) {
+rangeKeys.add(start);
+ }
{code}
Can it be private?
{code}
+ public LinkedHashMap getKeysToRegionsInRange(
{code}
Re: Andrew's concern regarding cache use: 6877 will take care of region move
too? cache may become stale for reasons other than splits too. Will look at
6877.
> HTable#coprocessorExec always scan the whole table
> ---
>
> Key: HBASE-6870
> URL: https://issues.apache.org/jira/browse/HBASE-6870
> Project: HBase
> Issue Type: Improvement
> Components: Coprocessors
>Affects Versions: 0.94.1
>Reporter: chunhui shen
>Assignee: chunhui shen
> Attachments: HBASE-6870.patch, HBASE-6870-testPerformance.patch,
> HBASE-6870v2.patch, HBASE-6870v3.patch
>
>
> In current logic, HTable#coprocessorExec always scan the whole table, its
> efficiency is low and will affect the Regionserver carrying .META. under
> large coprocessorExec requests
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6870) HTable#coprocessorExec always scan the whole table
[ https://issues.apache.org/jira/browse/HBASE-6870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13462479#comment-13462479 ] Andrew Purtell commented on HBASE-6870: --- Thanks [~zjushch]. > HTable#coprocessorExec always scan the whole table > --- > > Key: HBASE-6870 > URL: https://issues.apache.org/jira/browse/HBASE-6870 > Project: HBase > Issue Type: Improvement > Components: Coprocessors >Affects Versions: 0.94.1 >Reporter: chunhui shen >Assignee: chunhui shen > Attachments: HBASE-6870.patch, HBASE-6870-testPerformance.patch, > HBASE-6870v2.patch, HBASE-6870v3.patch > > > In current logic, HTable#coprocessorExec always scan the whole table, its > efficiency is low and will affect the Regionserver carrying .META. under > large coprocessorExec requests -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6870) HTable#coprocessorExec always scan the whole table
[ https://issues.apache.org/jira/browse/HBASE-6870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13462475#comment-13462475 ] chunhui shen commented on HBASE-6870: - Coprocessor exec result is incorrect if cached region location is wrong HBASE-6877 > HTable#coprocessorExec always scan the whole table > --- > > Key: HBASE-6870 > URL: https://issues.apache.org/jira/browse/HBASE-6870 > Project: HBase > Issue Type: Improvement > Components: Coprocessors >Affects Versions: 0.94.1 >Reporter: chunhui shen >Assignee: chunhui shen > Attachments: HBASE-6870.patch, HBASE-6870-testPerformance.patch, > HBASE-6870v2.patch, HBASE-6870v3.patch > > > In current logic, HTable#coprocessorExec always scan the whole table, its > efficiency is low and will affect the Regionserver carrying .META. under > large coprocessorExec requests -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6870) HTable#coprocessorExec always scan the whole table
[ https://issues.apache.org/jira/browse/HBASE-6870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13462385#comment-13462385 ] chunhui shen commented on HBASE-6870: - @Andrew First, we will also use cached location without this patch, see the details through HConnectionImplementation#processExecs->ExecRPCInvoker#invoke Second, if cached location is wrong, processExecs also will retry , also see the details in the ExecRPCInvoker#invoke Third, if one parent region is split when processExecs, current logic will cause the result incorrect, I will upload the patch to fix this bug after test in our environment At last, this patch is mainly in order to increase the performance for processExecs, I could provide the performance test later. > HTable#coprocessorExec always scan the whole table > --- > > Key: HBASE-6870 > URL: https://issues.apache.org/jira/browse/HBASE-6870 > Project: HBase > Issue Type: Improvement > Components: Coprocessors >Affects Versions: 0.94.1 >Reporter: chunhui shen >Assignee: chunhui shen > Attachments: HBASE-6870.patch, HBASE-6870v2.patch, HBASE-6870v3.patch > > > In current logic, HTable#coprocessorExec always scan the whole table, its > efficiency is low and will affect the Regionserver carrying .META. under > large coprocessorExec requests -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6870) HTable#coprocessorExec always scan the whole table
[ https://issues.apache.org/jira/browse/HBASE-6870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13461965#comment-13461965 ] Andrew Purtell commented on HBASE-6870: --- A coprocessorExec to an incorrect (cached) location will cause an exception to be sent back to the client, processExecs does not do transparent relocation. I don't think we want this. The patch looks ok without the cache. > HTable#coprocessorExec always scan the whole table > --- > > Key: HBASE-6870 > URL: https://issues.apache.org/jira/browse/HBASE-6870 > Project: HBase > Issue Type: Improvement > Components: Coprocessors >Affects Versions: 0.94.1 >Reporter: chunhui shen >Assignee: chunhui shen > Attachments: HBASE-6870.patch, HBASE-6870v2.patch, HBASE-6870v3.patch > > > In current logic, HTable#coprocessorExec always scan the whole table, its > efficiency is low and will affect the Regionserver carrying .META. under > large coprocessorExec requests -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6870) HTable#coprocessorExec always scan the whole table
[ https://issues.apache.org/jira/browse/HBASE-6870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13461614#comment-13461614 ] chunhui shen commented on HBASE-6870: - We have took a simple test: Concurrent 170 thread doing coprocessorExec Each coprocessorExec do a scan for one region Total 256 regions in the table Before the patch: Total 5s+ After the patch: Total 200ms+ > HTable#coprocessorExec always scan the whole table > --- > > Key: HBASE-6870 > URL: https://issues.apache.org/jira/browse/HBASE-6870 > Project: HBase > Issue Type: Improvement > Components: Coprocessors >Affects Versions: 0.94.1 >Reporter: chunhui shen >Assignee: chunhui shen > Attachments: HBASE-6870.patch, HBASE-6870v2.patch > > > In current logic, HTable#coprocessorExec always scan the whole table, its > efficiency is low and will affect the Regionserver carrying .META. under > large coprocessorExec requests -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
