[jira] [Commented] (HBASE-5664) CP hooks in Scan flow for fast forward when filter filters out a row
[
https://issues.apache.org/jira/browse/HBASE-5664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13570989#comment-13570989
]
Hudson commented on HBASE-5664:
---
Integrated in HBase-0.94-security-on-Hadoop-23 #11 (See
[https://builds.apache.org/job/HBase-0.94-security-on-Hadoop-23/11/])
HBASE-5664 CP hooks in Scan flow for fast forward when filter filters out a
row (Anoop Sam John) (Revision 1441701)
Result = FAILURE
larsh :
Files :
*
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/coprocessor/BaseRegionObserver.java
*
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/coprocessor/RegionObserver.java
*
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
*
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/RegionCoprocessorHost.java
CP hooks in Scan flow for fast forward when filter filters out a row
Key: HBASE-5664
URL: https://issues.apache.org/jira/browse/HBASE-5664
Project: HBase
Issue Type: Improvement
Components: Coprocessors, Filters
Affects Versions: 0.92.1
Reporter: Anoop Sam John
Assignee: Anoop Sam John
Fix For: 0.96.0, 0.94.5
Attachments: HBASE-5664_94.patch, HBASE-5664_94_V2.patch,
HBASE-5664_94_V3.patch, HBASE-5664_Trunk.patch, HBASE-5664_Trunk_V2.patch
In HRegion.nextInternal(int limit, String metric)
We have while(true) loop so as to fetch a next result which satisfies
filter condition. When Filter filters out the current fetched row we call
nextRow(byte [] currentRow) before going with the next row.
{code}
if (results.isEmpty() || filterRow()) {
// this seems like a redundant step - we already consumed the row
// there're no left overs.
// the reasons for calling this method are:
// 1. reset the filters.
// 2. provide a hook to fast forward the row (used by subclasses)
nextRow(currentRow);
{code}
// 2. provide a hook to fast forward the row (used by subclasses)
We can provide same feature of fast forward support for the CP also.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5664) CP hooks in Scan flow for fast forward when filter filters out a row
[
https://issues.apache.org/jira/browse/HBASE-5664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13569469#comment-13569469
]
Hudson commented on HBASE-5664:
---
Integrated in HBase-0.94-security #106 (See
[https://builds.apache.org/job/HBase-0.94-security/106/])
HBASE-5664 CP hooks in Scan flow for fast forward when filter filters out a
row (Anoop Sam John) (Revision 1441701)
Result = SUCCESS
larsh :
Files :
*
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/coprocessor/BaseRegionObserver.java
*
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/coprocessor/RegionObserver.java
*
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
*
/hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/RegionCoprocessorHost.java
CP hooks in Scan flow for fast forward when filter filters out a row
Key: HBASE-5664
URL: https://issues.apache.org/jira/browse/HBASE-5664
Project: HBase
Issue Type: Improvement
Components: Coprocessors, Filters
Affects Versions: 0.92.1
Reporter: Anoop Sam John
Assignee: Anoop Sam John
Fix For: 0.96.0, 0.94.5
Attachments: HBASE-5664_94.patch, HBASE-5664_94_V2.patch,
HBASE-5664_94_V3.patch, HBASE-5664_Trunk.patch, HBASE-5664_Trunk_V2.patch
In HRegion.nextInternal(int limit, String metric)
We have while(true) loop so as to fetch a next result which satisfies
filter condition. When Filter filters out the current fetched row we call
nextRow(byte [] currentRow) before going with the next row.
{code}
if (results.isEmpty() || filterRow()) {
// this seems like a redundant step - we already consumed the row
// there're no left overs.
// the reasons for calling this method are:
// 1. reset the filters.
// 2. provide a hook to fast forward the row (used by subclasses)
nextRow(currentRow);
{code}
// 2. provide a hook to fast forward the row (used by subclasses)
We can provide same feature of fast forward support for the CP also.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5664) CP hooks in Scan flow for fast forward when filter filters out a row
[
https://issues.apache.org/jira/browse/HBASE-5664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13569424#comment-13569424
]
Lars Hofhansl commented on HBASE-5664:
--
OK... Redid the test with exact same versions of HBase (+- the patch).
No penalty measurable. Going to commit.
CP hooks in Scan flow for fast forward when filter filters out a row
Key: HBASE-5664
URL: https://issues.apache.org/jira/browse/HBASE-5664
Project: HBase
Issue Type: Improvement
Components: Coprocessors, Filters
Affects Versions: 0.92.1
Reporter: Anoop Sam John
Assignee: Anoop Sam John
Fix For: 0.96.0, 0.94.5
Attachments: HBASE-5664_94.patch, HBASE-5664_94_V2.patch,
HBASE-5664_94_V3.patch, HBASE-5664_Trunk.patch, HBASE-5664_Trunk_V2.patch
In HRegion.nextInternal(int limit, String metric)
We have while(true) loop so as to fetch a next result which satisfies
filter condition. When Filter filters out the current fetched row we call
nextRow(byte [] currentRow) before going with the next row.
{code}
if (results.isEmpty() || filterRow()) {
// this seems like a redundant step - we already consumed the row
// there're no left overs.
// the reasons for calling this method are:
// 1. reset the filters.
// 2. provide a hook to fast forward the row (used by subclasses)
nextRow(currentRow);
{code}
// 2. provide a hook to fast forward the row (used by subclasses)
We can provide same feature of fast forward support for the CP also.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5664) CP hooks in Scan flow for fast forward when filter filters out a row
[
https://issues.apache.org/jira/browse/HBASE-5664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13569437#comment-13569437
]
Anoop Sam John commented on HBASE-5664:
---
Thanks [~yuzhih...@gmail.com], [~lhofhansl], [~apurtell] for your reviews and
comments
CP hooks in Scan flow for fast forward when filter filters out a row
Key: HBASE-5664
URL: https://issues.apache.org/jira/browse/HBASE-5664
Project: HBase
Issue Type: Improvement
Components: Coprocessors, Filters
Affects Versions: 0.92.1
Reporter: Anoop Sam John
Assignee: Anoop Sam John
Fix For: 0.96.0, 0.94.5
Attachments: HBASE-5664_94.patch, HBASE-5664_94_V2.patch,
HBASE-5664_94_V3.patch, HBASE-5664_Trunk.patch, HBASE-5664_Trunk_V2.patch
In HRegion.nextInternal(int limit, String metric)
We have while(true) loop so as to fetch a next result which satisfies
filter condition. When Filter filters out the current fetched row we call
nextRow(byte [] currentRow) before going with the next row.
{code}
if (results.isEmpty() || filterRow()) {
// this seems like a redundant step - we already consumed the row
// there're no left overs.
// the reasons for calling this method are:
// 1. reset the filters.
// 2. provide a hook to fast forward the row (used by subclasses)
nextRow(currentRow);
{code}
// 2. provide a hook to fast forward the row (used by subclasses)
We can provide same feature of fast forward support for the CP also.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5664) CP hooks in Scan flow for fast forward when filter filters out a row
[
https://issues.apache.org/jira/browse/HBASE-5664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13567450#comment-13567450
]
Lars Hofhansl commented on HBASE-5664:
--
Hmm... Yes. Maybe something else slowed it down. Lemme try again.
CP hooks in Scan flow for fast forward when filter filters out a row
Key: HBASE-5664
URL: https://issues.apache.org/jira/browse/HBASE-5664
Project: HBase
Issue Type: Improvement
Components: Coprocessors, Filters
Affects Versions: 0.92.1
Reporter: Anoop Sam John
Assignee: Anoop Sam John
Fix For: 0.96.0, 0.94.5
Attachments: HBASE-5664_94.patch, HBASE-5664_94_V2.patch,
HBASE-5664_94_V3.patch, HBASE-5664_Trunk.patch, HBASE-5664_Trunk_V2.patch
In HRegion.nextInternal(int limit, String metric)
We have while(true) loop so as to fetch a next result which satisfies
filter condition. When Filter filters out the current fetched row we call
nextRow(byte [] currentRow) before going with the next row.
{code}
if (results.isEmpty() || filterRow()) {
// this seems like a redundant step - we already consumed the row
// there're no left overs.
// the reasons for calling this method are:
// 1. reset the filters.
// 2. provide a hook to fast forward the row (used by subclasses)
nextRow(currentRow);
{code}
// 2. provide a hook to fast forward the row (used by subclasses)
We can provide same feature of fast forward support for the CP also.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5664) CP hooks in Scan flow for fast forward when filter filters out a row
[
https://issues.apache.org/jira/browse/HBASE-5664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13567557#comment-13567557
]
Anoop Sam John commented on HBASE-5664:
---
Thanks Lars. Yes the performance degrade in your test looks strange.
This patch not adding up any extra lines in the normal scan path.
CP hooks in Scan flow for fast forward when filter filters out a row
Key: HBASE-5664
URL: https://issues.apache.org/jira/browse/HBASE-5664
Project: HBase
Issue Type: Improvement
Components: Coprocessors, Filters
Affects Versions: 0.92.1
Reporter: Anoop Sam John
Assignee: Anoop Sam John
Fix For: 0.96.0, 0.94.5
Attachments: HBASE-5664_94.patch, HBASE-5664_94_V2.patch,
HBASE-5664_94_V3.patch, HBASE-5664_Trunk.patch, HBASE-5664_Trunk_V2.patch
In HRegion.nextInternal(int limit, String metric)
We have while(true) loop so as to fetch a next result which satisfies
filter condition. When Filter filters out the current fetched row we call
nextRow(byte [] currentRow) before going with the next row.
{code}
if (results.isEmpty() || filterRow()) {
// this seems like a redundant step - we already consumed the row
// there're no left overs.
// the reasons for calling this method are:
// 1. reset the filters.
// 2. provide a hook to fast forward the row (used by subclasses)
nextRow(currentRow);
{code}
// 2. provide a hook to fast forward the row (used by subclasses)
We can provide same feature of fast forward support for the CP also.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5664) CP hooks in Scan flow for fast forward when filter filters out a row
[
https://issues.apache.org/jira/browse/HBASE-5664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13566279#comment-13566279
]
Anoop Sam John commented on HBASE-5664:
---
[~lhofhansl]
The new CP call is in nextRow(byte [] currentRow, int offset, short length)
This method is called only when a row is filtered out.
{code}
if (filterRowKey(currentRow, offset, length)) {
-nextRow(currentRow, offset, length);
+boolean moreRows = nextRow(currentRow, offset, length);
if (isEmptyRow || filterRow()) {
-nextRow(currentRow, offset, length);
+boolean moreRows = nextRow(currentRow, offset, length);
if (results.isEmpty()) {
- nextRow(currentRow, offset, length);
+ boolean moreRows = nextRow(currentRow, offset, length);
{code}
So in a normal scan how this will affect? Sorry if I am missing some thing. I
really not getting..:(
{quote}
Also, looking at the patch again, we're not actually filtering rows, but ending
the scan, right?
So the coprocessor hook is misnamed.
{quote}
The hook is for notifying some row is filtered out. This is for the purpose of
doing a fast forward rather than a normal go to next row. So by this fast
forward some time CP implementation can see no more rows remaining in this
region and that is why it returns a boolean to indicate that. If the CP says no
more rows, the scan ends there
CP hooks in Scan flow for fast forward when filter filters out a row
Key: HBASE-5664
URL: https://issues.apache.org/jira/browse/HBASE-5664
Project: HBase
Issue Type: Improvement
Components: Coprocessors, Filters
Affects Versions: 0.92.1
Reporter: Anoop Sam John
Assignee: Anoop Sam John
Fix For: 0.96.0, 0.94.5
Attachments: HBASE-5664_94.patch, HBASE-5664_94_V2.patch,
HBASE-5664_94_V3.patch, HBASE-5664_Trunk.patch, HBASE-5664_Trunk_V2.patch
In HRegion.nextInternal(int limit, String metric)
We have while(true) loop so as to fetch a next result which satisfies
filter condition. When Filter filters out the current fetched row we call
nextRow(byte [] currentRow) before going with the next row.
{code}
if (results.isEmpty() || filterRow()) {
// this seems like a redundant step - we already consumed the row
// there're no left overs.
// the reasons for calling this method are:
// 1. reset the filters.
// 2. provide a hook to fast forward the row (used by subclasses)
nextRow(currentRow);
{code}
// 2. provide a hook to fast forward the row (used by subclasses)
We can provide same feature of fast forward support for the CP also.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5664) CP hooks in Scan flow for fast forward when filter filters out a row
[
https://issues.apache.org/jira/browse/HBASE-5664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13566214#comment-13566214
]
Lars Hofhansl commented on HBASE-5664:
--
So when scanning 10m rows with 1 col each, with scanner caching set to 1, I
can detect a 5-10% slowdown with this patch.
CP hooks in Scan flow for fast forward when filter filters out a row
Key: HBASE-5664
URL: https://issues.apache.org/jira/browse/HBASE-5664
Project: HBase
Issue Type: Improvement
Components: Coprocessors, Filters
Affects Versions: 0.92.1
Reporter: Anoop Sam John
Assignee: Anoop Sam John
Fix For: 0.96.0, 0.94.5
Attachments: HBASE-5664_94.patch, HBASE-5664_94_V2.patch,
HBASE-5664_94_V3.patch, HBASE-5664_Trunk.patch, HBASE-5664_Trunk_V2.patch
In HRegion.nextInternal(int limit, String metric)
We have while(true) loop so as to fetch a next result which satisfies
filter condition. When Filter filters out the current fetched row we call
nextRow(byte [] currentRow) before going with the next row.
{code}
if (results.isEmpty() || filterRow()) {
// this seems like a redundant step - we already consumed the row
// there're no left overs.
// the reasons for calling this method are:
// 1. reset the filters.
// 2. provide a hook to fast forward the row (used by subclasses)
nextRow(currentRow);
{code}
// 2. provide a hook to fast forward the row (used by subclasses)
We can provide same feature of fast forward support for the CP also.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5664) CP hooks in Scan flow for fast forward when filter filters out a row
[
https://issues.apache.org/jira/browse/HBASE-5664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13566216#comment-13566216
]
Lars Hofhansl commented on HBASE-5664:
--
With all data in the blockcache as well. So together this should be the worst
case scenario for this change.
CP hooks in Scan flow for fast forward when filter filters out a row
Key: HBASE-5664
URL: https://issues.apache.org/jira/browse/HBASE-5664
Project: HBase
Issue Type: Improvement
Components: Coprocessors, Filters
Affects Versions: 0.92.1
Reporter: Anoop Sam John
Assignee: Anoop Sam John
Fix For: 0.96.0, 0.94.5
Attachments: HBASE-5664_94.patch, HBASE-5664_94_V2.patch,
HBASE-5664_94_V3.patch, HBASE-5664_Trunk.patch, HBASE-5664_Trunk_V2.patch
In HRegion.nextInternal(int limit, String metric)
We have while(true) loop so as to fetch a next result which satisfies
filter condition. When Filter filters out the current fetched row we call
nextRow(byte [] currentRow) before going with the next row.
{code}
if (results.isEmpty() || filterRow()) {
// this seems like a redundant step - we already consumed the row
// there're no left overs.
// the reasons for calling this method are:
// 1. reset the filters.
// 2. provide a hook to fast forward the row (used by subclasses)
nextRow(currentRow);
{code}
// 2. provide a hook to fast forward the row (used by subclasses)
We can provide same feature of fast forward support for the CP also.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5664) CP hooks in Scan flow for fast forward when filter filters out a row
[
https://issues.apache.org/jira/browse/HBASE-5664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13566217#comment-13566217
]
Anoop Sam John commented on HBASE-5664:
---
Thanks Lars for the test.
You mean in your test there is filtering happening on rows?
I am not fully getting ur test case.
CP hooks in Scan flow for fast forward when filter filters out a row
Key: HBASE-5664
URL: https://issues.apache.org/jira/browse/HBASE-5664
Project: HBase
Issue Type: Improvement
Components: Coprocessors, Filters
Affects Versions: 0.92.1
Reporter: Anoop Sam John
Assignee: Anoop Sam John
Fix For: 0.96.0, 0.94.5
Attachments: HBASE-5664_94.patch, HBASE-5664_94_V2.patch,
HBASE-5664_94_V3.patch, HBASE-5664_Trunk.patch, HBASE-5664_Trunk_V2.patch
In HRegion.nextInternal(int limit, String metric)
We have while(true) loop so as to fetch a next result which satisfies
filter condition. When Filter filters out the current fetched row we call
nextRow(byte [] currentRow) before going with the next row.
{code}
if (results.isEmpty() || filterRow()) {
// this seems like a redundant step - we already consumed the row
// there're no left overs.
// the reasons for calling this method are:
// 1. reset the filters.
// 2. provide a hook to fast forward the row (used by subclasses)
nextRow(currentRow);
{code}
// 2. provide a hook to fast forward the row (used by subclasses)
We can provide same feature of fast forward support for the CP also.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5664) CP hooks in Scan flow for fast forward when filter filters out a row
[
https://issues.apache.org/jira/browse/HBASE-5664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13566220#comment-13566220
]
Lars Hofhansl commented on HBASE-5664:
--
Just scanning without any filtering. Takes ~20s without this patch and ~22s
with this patch.
A question I had: Why do you need to do this skipping via a coprocessor? Would
a filter not work?
CP hooks in Scan flow for fast forward when filter filters out a row
Key: HBASE-5664
URL: https://issues.apache.org/jira/browse/HBASE-5664
Project: HBase
Issue Type: Improvement
Components: Coprocessors, Filters
Affects Versions: 0.92.1
Reporter: Anoop Sam John
Assignee: Anoop Sam John
Fix For: 0.96.0, 0.94.5
Attachments: HBASE-5664_94.patch, HBASE-5664_94_V2.patch,
HBASE-5664_94_V3.patch, HBASE-5664_Trunk.patch, HBASE-5664_Trunk_V2.patch
In HRegion.nextInternal(int limit, String metric)
We have while(true) loop so as to fetch a next result which satisfies
filter condition. When Filter filters out the current fetched row we call
nextRow(byte [] currentRow) before going with the next row.
{code}
if (results.isEmpty() || filterRow()) {
// this seems like a redundant step - we already consumed the row
// there're no left overs.
// the reasons for calling this method are:
// 1. reset the filters.
// 2. provide a hook to fast forward the row (used by subclasses)
nextRow(currentRow);
{code}
// 2. provide a hook to fast forward the row (used by subclasses)
We can provide same feature of fast forward support for the CP also.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5664) CP hooks in Scan flow for fast forward when filter filters out a row
[
https://issues.apache.org/jira/browse/HBASE-5664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13566224#comment-13566224
]
Anoop Sam John commented on HBASE-5664:
---
bq.Just scanning without any filtering.
That looks strange Lars. The new code will come into picture only when there is
some filtering happens. Otherwise in your test flow of Scan absolutely no
addition of code. Am I missing some thing?
CP hooks in Scan flow for fast forward when filter filters out a row
Key: HBASE-5664
URL: https://issues.apache.org/jira/browse/HBASE-5664
Project: HBase
Issue Type: Improvement
Components: Coprocessors, Filters
Affects Versions: 0.92.1
Reporter: Anoop Sam John
Assignee: Anoop Sam John
Fix For: 0.96.0, 0.94.5
Attachments: HBASE-5664_94.patch, HBASE-5664_94_V2.patch,
HBASE-5664_94_V3.patch, HBASE-5664_Trunk.patch, HBASE-5664_Trunk_V2.patch
In HRegion.nextInternal(int limit, String metric)
We have while(true) loop so as to fetch a next result which satisfies
filter condition. When Filter filters out the current fetched row we call
nextRow(byte [] currentRow) before going with the next row.
{code}
if (results.isEmpty() || filterRow()) {
// this seems like a redundant step - we already consumed the row
// there're no left overs.
// the reasons for calling this method are:
// 1. reset the filters.
// 2. provide a hook to fast forward the row (used by subclasses)
nextRow(currentRow);
{code}
// 2. provide a hook to fast forward the row (used by subclasses)
We can provide same feature of fast forward support for the CP also.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5664) CP hooks in Scan flow for fast forward when filter filters out a row
[
https://issues.apache.org/jira/browse/HBASE-5664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13566227#comment-13566227
]
Lars Hofhansl commented on HBASE-5664:
--
Nope. It always executes the check for the coprocessor through the
coprocessorhost in the next() method, that does make a difference; at least in
this extreme scenario.
Not saying we shouldn't do it, just saying that it comes with a price in the
general scanning code.
Also, looking at the patch again, we're not actually filtering rows, but ending
the scan, right?
So the coprocessor hook is misnamed.
CP hooks in Scan flow for fast forward when filter filters out a row
Key: HBASE-5664
URL: https://issues.apache.org/jira/browse/HBASE-5664
Project: HBase
Issue Type: Improvement
Components: Coprocessors, Filters
Affects Versions: 0.92.1
Reporter: Anoop Sam John
Assignee: Anoop Sam John
Fix For: 0.96.0, 0.94.5
Attachments: HBASE-5664_94.patch, HBASE-5664_94_V2.patch,
HBASE-5664_94_V3.patch, HBASE-5664_Trunk.patch, HBASE-5664_Trunk_V2.patch
In HRegion.nextInternal(int limit, String metric)
We have while(true) loop so as to fetch a next result which satisfies
filter condition. When Filter filters out the current fetched row we call
nextRow(byte [] currentRow) before going with the next row.
{code}
if (results.isEmpty() || filterRow()) {
// this seems like a redundant step - we already consumed the row
// there're no left overs.
// the reasons for calling this method are:
// 1. reset the filters.
// 2. provide a hook to fast forward the row (used by subclasses)
nextRow(currentRow);
{code}
// 2. provide a hook to fast forward the row (used by subclasses)
We can provide same feature of fast forward support for the CP also.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5664) CP hooks in Scan flow for fast forward when filter filters out a row
[
https://issues.apache.org/jira/browse/HBASE-5664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13563763#comment-13563763
]
Lars Hofhansl commented on HBASE-5664:
--
Thanks Anoop. Will do as soon as I get to it (the next two days or so)
CP hooks in Scan flow for fast forward when filter filters out a row
Key: HBASE-5664
URL: https://issues.apache.org/jira/browse/HBASE-5664
Project: HBase
Issue Type: Improvement
Components: Coprocessors, Filters
Affects Versions: 0.92.1
Reporter: Anoop Sam John
Assignee: Anoop Sam John
Fix For: 0.96.0, 0.94.5
Attachments: HBASE-5664_94.patch, HBASE-5664_94_V2.patch,
HBASE-5664_94_V3.patch, HBASE-5664_Trunk.patch, HBASE-5664_Trunk_V2.patch
In HRegion.nextInternal(int limit, String metric)
We have while(true) loop so as to fetch a next result which satisfies
filter condition. When Filter filters out the current fetched row we call
nextRow(byte [] currentRow) before going with the next row.
{code}
if (results.isEmpty() || filterRow()) {
// this seems like a redundant step - we already consumed the row
// there're no left overs.
// the reasons for calling this method are:
// 1. reset the filters.
// 2. provide a hook to fast forward the row (used by subclasses)
nextRow(currentRow);
{code}
// 2. provide a hook to fast forward the row (used by subclasses)
We can provide same feature of fast forward support for the CP also.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5664) CP hooks in Scan flow for fast forward when filter filters out a row
[
https://issues.apache.org/jira/browse/HBASE-5664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13563380#comment-13563380
]
Hadoop QA commented on HBASE-5664:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12566616/HBASE-5664_94_V3.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/4197//console
This message is automatically generated.
CP hooks in Scan flow for fast forward when filter filters out a row
Key: HBASE-5664
URL: https://issues.apache.org/jira/browse/HBASE-5664
Project: HBase
Issue Type: Improvement
Components: Coprocessors, Filters
Affects Versions: 0.92.1
Reporter: Anoop Sam John
Assignee: Anoop Sam John
Fix For: 0.96.0, 0.94.5
Attachments: HBASE-5664_94.patch, HBASE-5664_94_V2.patch,
HBASE-5664_94_V3.patch, HBASE-5664_Trunk.patch, HBASE-5664_Trunk_V2.patch
In HRegion.nextInternal(int limit, String metric)
We have while(true) loop so as to fetch a next result which satisfies
filter condition. When Filter filters out the current fetched row we call
nextRow(byte [] currentRow) before going with the next row.
{code}
if (results.isEmpty() || filterRow()) {
// this seems like a redundant step - we already consumed the row
// there're no left overs.
// the reasons for calling this method are:
// 1. reset the filters.
// 2. provide a hook to fast forward the row (used by subclasses)
nextRow(currentRow);
{code}
// 2. provide a hook to fast forward the row (used by subclasses)
We can provide same feature of fast forward support for the CP also.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5664) CP hooks in Scan flow for fast forward when filter filters out a row
[
https://issues.apache.org/jira/browse/HBASE-5664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13562330#comment-13562330
]
Anoop Sam John commented on HBASE-5664:
---
[~lhofhansl] Can we integrate this to 0.94?
Once it is finalized can change the fix version (if needed) and close the issue.
CP hooks in Scan flow for fast forward when filter filters out a row
Key: HBASE-5664
URL: https://issues.apache.org/jira/browse/HBASE-5664
Project: HBase
Issue Type: Improvement
Components: Coprocessors, Filters
Affects Versions: 0.92.1
Reporter: Anoop Sam John
Assignee: Anoop Sam John
Fix For: 0.96.0, 0.94.5
Attachments: HBASE-5664_94.patch, HBASE-5664_94_V2.patch,
HBASE-5664_Trunk.patch, HBASE-5664_Trunk_V2.patch
In HRegion.nextInternal(int limit, String metric)
We have while(true) loop so as to fetch a next result which satisfies
filter condition. When Filter filters out the current fetched row we call
nextRow(byte [] currentRow) before going with the next row.
{code}
if (results.isEmpty() || filterRow()) {
// this seems like a redundant step - we already consumed the row
// there're no left overs.
// the reasons for calling this method are:
// 1. reset the filters.
// 2. provide a hook to fast forward the row (used by subclasses)
nextRow(currentRow);
{code}
// 2. provide a hook to fast forward the row (used by subclasses)
We can provide same feature of fast forward support for the CP also.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5664) CP hooks in Scan flow for fast forward when filter filters out a row
[
https://issues.apache.org/jira/browse/HBASE-5664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13562459#comment-13562459
]
Lars Hofhansl commented on HBASE-5664:
--
I was trying to play with the patch in 0.94 and do some performance tests, but
it doesn't apply anymore.
CP hooks in Scan flow for fast forward when filter filters out a row
Key: HBASE-5664
URL: https://issues.apache.org/jira/browse/HBASE-5664
Project: HBase
Issue Type: Improvement
Components: Coprocessors, Filters
Affects Versions: 0.92.1
Reporter: Anoop Sam John
Assignee: Anoop Sam John
Fix For: 0.96.0, 0.94.5
Attachments: HBASE-5664_94.patch, HBASE-5664_94_V2.patch,
HBASE-5664_Trunk.patch, HBASE-5664_Trunk_V2.patch
In HRegion.nextInternal(int limit, String metric)
We have while(true) loop so as to fetch a next result which satisfies
filter condition. When Filter filters out the current fetched row we call
nextRow(byte [] currentRow) before going with the next row.
{code}
if (results.isEmpty() || filterRow()) {
// this seems like a redundant step - we already consumed the row
// there're no left overs.
// the reasons for calling this method are:
// 1. reset the filters.
// 2. provide a hook to fast forward the row (used by subclasses)
nextRow(currentRow);
{code}
// 2. provide a hook to fast forward the row (used by subclasses)
We can provide same feature of fast forward support for the CP also.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5664) CP hooks in Scan flow for fast forward when filter filters out a row
[
https://issues.apache.org/jira/browse/HBASE-5664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13562468#comment-13562468
]
Anoop Sam John commented on HBASE-5664:
---
Oh.. I will rebase the patch then.
CP hooks in Scan flow for fast forward when filter filters out a row
Key: HBASE-5664
URL: https://issues.apache.org/jira/browse/HBASE-5664
Project: HBase
Issue Type: Improvement
Components: Coprocessors, Filters
Affects Versions: 0.92.1
Reporter: Anoop Sam John
Assignee: Anoop Sam John
Fix For: 0.96.0, 0.94.5
Attachments: HBASE-5664_94.patch, HBASE-5664_94_V2.patch,
HBASE-5664_Trunk.patch, HBASE-5664_Trunk_V2.patch
In HRegion.nextInternal(int limit, String metric)
We have while(true) loop so as to fetch a next result which satisfies
filter condition. When Filter filters out the current fetched row we call
nextRow(byte [] currentRow) before going with the next row.
{code}
if (results.isEmpty() || filterRow()) {
// this seems like a redundant step - we already consumed the row
// there're no left overs.
// the reasons for calling this method are:
// 1. reset the filters.
// 2. provide a hook to fast forward the row (used by subclasses)
nextRow(currentRow);
{code}
// 2. provide a hook to fast forward the row (used by subclasses)
We can provide same feature of fast forward support for the CP also.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5664) CP hooks in Scan flow for fast forward when filter filters out a row
[
https://issues.apache.org/jira/browse/HBASE-5664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13560560#comment-13560560
]
Anoop Sam John commented on HBASE-5664:
---
[~apurtell] Comments from your side?
CP hooks in Scan flow for fast forward when filter filters out a row
Key: HBASE-5664
URL: https://issues.apache.org/jira/browse/HBASE-5664
Project: HBase
Issue Type: Improvement
Components: Coprocessors, Filters
Affects Versions: 0.92.1
Reporter: Anoop Sam John
Assignee: Anoop Sam John
Fix For: 0.96.0, 0.94.5
Attachments: HBASE-5664_94.patch, HBASE-5664_94_V2.patch,
HBASE-5664_Trunk.patch, HBASE-5664_Trunk_V2.patch
In HRegion.nextInternal(int limit, String metric)
We have while(true) loop so as to fetch a next result which satisfies
filter condition. When Filter filters out the current fetched row we call
nextRow(byte [] currentRow) before going with the next row.
{code}
if (results.isEmpty() || filterRow()) {
// this seems like a redundant step - we already consumed the row
// there're no left overs.
// the reasons for calling this method are:
// 1. reset the filters.
// 2. provide a hook to fast forward the row (used by subclasses)
nextRow(currentRow);
{code}
// 2. provide a hook to fast forward the row (used by subclasses)
We can provide same feature of fast forward support for the CP also.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5664) CP hooks in Scan flow for fast forward when filter filters out a row
[
https://issues.apache.org/jira/browse/HBASE-5664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13561170#comment-13561170
]
Andrew Purtell commented on HBASE-5664:
---
+1 if this is what you need Anoop.
CP hooks in Scan flow for fast forward when filter filters out a row
Key: HBASE-5664
URL: https://issues.apache.org/jira/browse/HBASE-5664
Project: HBase
Issue Type: Improvement
Components: Coprocessors, Filters
Affects Versions: 0.92.1
Reporter: Anoop Sam John
Assignee: Anoop Sam John
Fix For: 0.96.0, 0.94.5
Attachments: HBASE-5664_94.patch, HBASE-5664_94_V2.patch,
HBASE-5664_Trunk.patch, HBASE-5664_Trunk_V2.patch
In HRegion.nextInternal(int limit, String metric)
We have while(true) loop so as to fetch a next result which satisfies
filter condition. When Filter filters out the current fetched row we call
nextRow(byte [] currentRow) before going with the next row.
{code}
if (results.isEmpty() || filterRow()) {
// this seems like a redundant step - we already consumed the row
// there're no left overs.
// the reasons for calling this method are:
// 1. reset the filters.
// 2. provide a hook to fast forward the row (used by subclasses)
nextRow(currentRow);
{code}
// 2. provide a hook to fast forward the row (used by subclasses)
We can provide same feature of fast forward support for the CP also.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5664) CP hooks in Scan flow for fast forward when filter filters out a row
[
https://issues.apache.org/jira/browse/HBASE-5664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13561203#comment-13561203
]
Ted Yu commented on HBASE-5664:
---
Integrated to trunk.
Thanks for the patch, Anoop.
Thanks for the review, Andy.
CP hooks in Scan flow for fast forward when filter filters out a row
Key: HBASE-5664
URL: https://issues.apache.org/jira/browse/HBASE-5664
Project: HBase
Issue Type: Improvement
Components: Coprocessors, Filters
Affects Versions: 0.92.1
Reporter: Anoop Sam John
Assignee: Anoop Sam John
Fix For: 0.96.0, 0.94.5
Attachments: HBASE-5664_94.patch, HBASE-5664_94_V2.patch,
HBASE-5664_Trunk.patch, HBASE-5664_Trunk_V2.patch
In HRegion.nextInternal(int limit, String metric)
We have while(true) loop so as to fetch a next result which satisfies
filter condition. When Filter filters out the current fetched row we call
nextRow(byte [] currentRow) before going with the next row.
{code}
if (results.isEmpty() || filterRow()) {
// this seems like a redundant step - we already consumed the row
// there're no left overs.
// the reasons for calling this method are:
// 1. reset the filters.
// 2. provide a hook to fast forward the row (used by subclasses)
nextRow(currentRow);
{code}
// 2. provide a hook to fast forward the row (used by subclasses)
We can provide same feature of fast forward support for the CP also.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5664) CP hooks in Scan flow for fast forward when filter filters out a row
[
https://issues.apache.org/jira/browse/HBASE-5664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13561249#comment-13561249
]
Hudson commented on HBASE-5664:
---
Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #369 (See
[https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/369/])
HBASE-5664 CP hooks in Scan flow for fast forward when filter filters out a
row (Anoop) (Revision 1437790)
Result = FAILURE
tedyu :
Files :
*
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/coprocessor/BaseRegionObserver.java
*
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/coprocessor/RegionObserver.java
*
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
*
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RegionCoprocessorHost.java
CP hooks in Scan flow for fast forward when filter filters out a row
Key: HBASE-5664
URL: https://issues.apache.org/jira/browse/HBASE-5664
Project: HBase
Issue Type: Improvement
Components: Coprocessors, Filters
Affects Versions: 0.92.1
Reporter: Anoop Sam John
Assignee: Anoop Sam John
Fix For: 0.96.0, 0.94.5
Attachments: HBASE-5664_94.patch, HBASE-5664_94_V2.patch,
HBASE-5664_Trunk.patch, HBASE-5664_Trunk_V2.patch
In HRegion.nextInternal(int limit, String metric)
We have while(true) loop so as to fetch a next result which satisfies
filter condition. When Filter filters out the current fetched row we call
nextRow(byte [] currentRow) before going with the next row.
{code}
if (results.isEmpty() || filterRow()) {
// this seems like a redundant step - we already consumed the row
// there're no left overs.
// the reasons for calling this method are:
// 1. reset the filters.
// 2. provide a hook to fast forward the row (used by subclasses)
nextRow(currentRow);
{code}
// 2. provide a hook to fast forward the row (used by subclasses)
We can provide same feature of fast forward support for the CP also.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5664) CP hooks in Scan flow for fast forward when filter filters out a row
[
https://issues.apache.org/jira/browse/HBASE-5664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13561294#comment-13561294
]
Hudson commented on HBASE-5664:
---
Integrated in HBase-TRUNK #3786 (See
[https://builds.apache.org/job/HBase-TRUNK/3786/])
HBASE-5664 CP hooks in Scan flow for fast forward when filter filters out a
row (Anoop) (Revision 1437790)
Result = FAILURE
tedyu :
Files :
*
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/coprocessor/BaseRegionObserver.java
*
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/coprocessor/RegionObserver.java
*
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
*
/hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RegionCoprocessorHost.java
CP hooks in Scan flow for fast forward when filter filters out a row
Key: HBASE-5664
URL: https://issues.apache.org/jira/browse/HBASE-5664
Project: HBase
Issue Type: Improvement
Components: Coprocessors, Filters
Affects Versions: 0.92.1
Reporter: Anoop Sam John
Assignee: Anoop Sam John
Fix For: 0.96.0, 0.94.5
Attachments: HBASE-5664_94.patch, HBASE-5664_94_V2.patch,
HBASE-5664_Trunk.patch, HBASE-5664_Trunk_V2.patch
In HRegion.nextInternal(int limit, String metric)
We have while(true) loop so as to fetch a next result which satisfies
filter condition. When Filter filters out the current fetched row we call
nextRow(byte [] currentRow) before going with the next row.
{code}
if (results.isEmpty() || filterRow()) {
// this seems like a redundant step - we already consumed the row
// there're no left overs.
// the reasons for calling this method are:
// 1. reset the filters.
// 2. provide a hook to fast forward the row (used by subclasses)
nextRow(currentRow);
{code}
// 2. provide a hook to fast forward the row (used by subclasses)
We can provide same feature of fast forward support for the CP also.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5664) CP hooks in Scan flow for fast forward when filter filters out a row
[
https://issues.apache.org/jira/browse/HBASE-5664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13561422#comment-13561422
]
Anoop Sam John commented on HBASE-5664:
---
Ted Lars This is can be added in 0.94 also?
Once it is finalized can change the fix version (if needed) and close the issue.
CP hooks in Scan flow for fast forward when filter filters out a row
Key: HBASE-5664
URL: https://issues.apache.org/jira/browse/HBASE-5664
Project: HBase
Issue Type: Improvement
Components: Coprocessors, Filters
Affects Versions: 0.92.1
Reporter: Anoop Sam John
Assignee: Anoop Sam John
Fix For: 0.96.0, 0.94.5
Attachments: HBASE-5664_94.patch, HBASE-5664_94_V2.patch,
HBASE-5664_Trunk.patch, HBASE-5664_Trunk_V2.patch
In HRegion.nextInternal(int limit, String metric)
We have while(true) loop so as to fetch a next result which satisfies
filter condition. When Filter filters out the current fetched row we call
nextRow(byte [] currentRow) before going with the next row.
{code}
if (results.isEmpty() || filterRow()) {
// this seems like a redundant step - we already consumed the row
// there're no left overs.
// the reasons for calling this method are:
// 1. reset the filters.
// 2. provide a hook to fast forward the row (used by subclasses)
nextRow(currentRow);
{code}
// 2. provide a hook to fast forward the row (used by subclasses)
We can provide same feature of fast forward support for the CP also.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5664) CP hooks in Scan flow for fast forward when filter filters out a row
[
https://issues.apache.org/jira/browse/HBASE-5664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13559803#comment-13559803
]
Ted Yu commented on HBASE-5664:
---
Patch v2 looks good.
Please add release notes.
CP hooks in Scan flow for fast forward when filter filters out a row
Key: HBASE-5664
URL: https://issues.apache.org/jira/browse/HBASE-5664
Project: HBase
Issue Type: Improvement
Components: Coprocessors, Filters
Affects Versions: 0.92.1
Reporter: Anoop Sam John
Assignee: Anoop Sam John
Fix For: 0.96.0, 0.94.5
Attachments: HBASE-5664_94.patch, HBASE-5664_94_V2.patch,
HBASE-5664_Trunk.patch, HBASE-5664_Trunk_V2.patch
In HRegion.nextInternal(int limit, String metric)
We have while(true) loop so as to fetch a next result which satisfies
filter condition. When Filter filters out the current fetched row we call
nextRow(byte [] currentRow) before going with the next row.
{code}
if (results.isEmpty() || filterRow()) {
// this seems like a redundant step - we already consumed the row
// there're no left overs.
// the reasons for calling this method are:
// 1. reset the filters.
// 2. provide a hook to fast forward the row (used by subclasses)
nextRow(currentRow);
{code}
// 2. provide a hook to fast forward the row (used by subclasses)
We can provide same feature of fast forward support for the CP also.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5664) CP hooks in Scan flow for fast forward when filter filters out a row
[
https://issues.apache.org/jira/browse/HBASE-5664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13559814#comment-13559814
]
Hadoop QA commented on HBASE-5664:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12565967/HBASE-5664_Trunk_V2.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:red}-1 findbugs{color}. The patch appears to introduce 3 new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.TestLocalHBaseCluster
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/4125//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4125//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4125//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4125//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4125//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4125//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4125//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4125//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/4125//console
This message is automatically generated.
CP hooks in Scan flow for fast forward when filter filters out a row
Key: HBASE-5664
URL: https://issues.apache.org/jira/browse/HBASE-5664
Project: HBase
Issue Type: Improvement
Components: Coprocessors, Filters
Affects Versions: 0.92.1
Reporter: Anoop Sam John
Assignee: Anoop Sam John
Fix For: 0.96.0, 0.94.5
Attachments: HBASE-5664_94.patch, HBASE-5664_94_V2.patch,
HBASE-5664_Trunk.patch, HBASE-5664_Trunk_V2.patch
In HRegion.nextInternal(int limit, String metric)
We have while(true) loop so as to fetch a next result which satisfies
filter condition. When Filter filters out the current fetched row we call
nextRow(byte [] currentRow) before going with the next row.
{code}
if (results.isEmpty() || filterRow()) {
// this seems like a redundant step - we already consumed the row
// there're no left overs.
// the reasons for calling this method are:
// 1. reset the filters.
// 2. provide a hook to fast forward the row (used by subclasses)
nextRow(currentRow);
{code}
// 2. provide a hook to fast forward the row (used by subclasses)
We can provide same feature of fast forward support for the CP also.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5664) CP hooks in Scan flow for fast forward when filter filters out a row
[
https://issues.apache.org/jira/browse/HBASE-5664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13558937#comment-13558937
]
Ted Yu commented on HBASE-5664:
---
{code}
+ * @return Returns whether more rows are available for the scanner or not.
{code}
Remove 'Returns'
{code}
+RegionScannerImpl(Scan scan, ListKeyValueScanner additionalScanners,
HRegion region) throws IOException {
{code}
Wrap long line above.
{code}
+ * @param s
+ * @param currentRow
+ * @return
+ * @throws IOException
+ */
+ public boolean postScannerFilterRow(final InternalScanner s, final byte[]
currentRow)
{code}
Please complete javadoc.
See if the test failures in TestFromClientSideWithCoprocessor and
TestFromClientSide are related to this patch.
CP hooks in Scan flow for fast forward when filter filters out a row
Key: HBASE-5664
URL: https://issues.apache.org/jira/browse/HBASE-5664
Project: HBase
Issue Type: Improvement
Components: Coprocessors, Filters
Affects Versions: 0.92.1
Reporter: Anoop Sam John
Assignee: Anoop Sam John
Fix For: 0.96.0, 0.94.5
Attachments: HBASE-5664_94.patch, HBASE-5664_Trunk.patch
In HRegion.nextInternal(int limit, String metric)
We have while(true) loop so as to fetch a next result which satisfies
filter condition. When Filter filters out the current fetched row we call
nextRow(byte [] currentRow) before going with the next row.
{code}
if (results.isEmpty() || filterRow()) {
// this seems like a redundant step - we already consumed the row
// there're no left overs.
// the reasons for calling this method are:
// 1. reset the filters.
// 2. provide a hook to fast forward the row (used by subclasses)
nextRow(currentRow);
{code}
// 2. provide a hook to fast forward the row (used by subclasses)
We can provide same feature of fast forward support for the CP also.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5664) CP hooks in Scan flow for fast forward when filter filters out a row
[
https://issues.apache.org/jira/browse/HBASE-5664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13558942#comment-13558942
]
Hadoop QA commented on HBASE-5664:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12565809/HBASE-5664_Trunk.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 1
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:red}-1 findbugs{color}. The patch appears to introduce 3 new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 lineLengths{color}. The patch introduces lines longer than
100
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.security.access.TestAccessController
org.apache.hadoop.hbase.client.TestFromClientSideWithCoprocessor
org.apache.hadoop.hbase.coprocessor.TestRowProcessorEndpoint
org.apache.hadoop.hbase.client.TestFromClientSide
org.apache.hadoop.hbase.coprocessor.TestAggregateProtocol
org.apache.hadoop.hbase.TestLocalHBaseCluster
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/4109//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4109//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4109//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4109//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4109//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4109//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4109//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/4109//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/4109//console
This message is automatically generated.
CP hooks in Scan flow for fast forward when filter filters out a row
Key: HBASE-5664
URL: https://issues.apache.org/jira/browse/HBASE-5664
Project: HBase
Issue Type: Improvement
Components: Coprocessors, Filters
Affects Versions: 0.92.1
Reporter: Anoop Sam John
Assignee: Anoop Sam John
Fix For: 0.96.0, 0.94.5
Attachments: HBASE-5664_94.patch, HBASE-5664_Trunk.patch
In HRegion.nextInternal(int limit, String metric)
We have while(true) loop so as to fetch a next result which satisfies
filter condition. When Filter filters out the current fetched row we call
nextRow(byte [] currentRow) before going with the next row.
{code}
if (results.isEmpty() || filterRow()) {
// this seems like a redundant step - we already consumed the row
// there're no left overs.
// the reasons for calling this method are:
// 1. reset the filters.
// 2. provide a hook to fast forward the row (used by subclasses)
nextRow(currentRow);
{code}
// 2. provide a hook to fast forward the row (used by subclasses)
We can provide same feature of fast forward support for the CP also.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5664) CP hooks in Scan flow for fast forward when filter filters out a row
[
https://issues.apache.org/jira/browse/HBASE-5664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13559376#comment-13559376
]
Anoop Sam John commented on HBASE-5664:
---
Thanks Ted.
Let me see the failures
CP hooks in Scan flow for fast forward when filter filters out a row
Key: HBASE-5664
URL: https://issues.apache.org/jira/browse/HBASE-5664
Project: HBase
Issue Type: Improvement
Components: Coprocessors, Filters
Affects Versions: 0.92.1
Reporter: Anoop Sam John
Assignee: Anoop Sam John
Fix For: 0.96.0, 0.94.5
Attachments: HBASE-5664_94.patch, HBASE-5664_Trunk.patch
In HRegion.nextInternal(int limit, String metric)
We have while(true) loop so as to fetch a next result which satisfies
filter condition. When Filter filters out the current fetched row we call
nextRow(byte [] currentRow) before going with the next row.
{code}
if (results.isEmpty() || filterRow()) {
// this seems like a redundant step - we already consumed the row
// there're no left overs.
// the reasons for calling this method are:
// 1. reset the filters.
// 2. provide a hook to fast forward the row (used by subclasses)
nextRow(currentRow);
{code}
// 2. provide a hook to fast forward the row (used by subclasses)
We can provide same feature of fast forward support for the CP also.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5664) CP hooks in Scan flow for fast forward when filter filters out a row
[
https://issues.apache.org/jira/browse/HBASE-5664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13559418#comment-13559418
]
Anoop Sam John commented on HBASE-5664:
---
Oops! Missed the instanceof check in RegionCoprocessorHost..I will attach both
the version patches after correcting this issue.
Internally we were adding the new hooks to another interface other than
RegionObserver. In our code the instance of check was against this new
interface. I removed that (While creating the patch) but forgot to add the
original instanceof check which is needed :(
CP hooks in Scan flow for fast forward when filter filters out a row
Key: HBASE-5664
URL: https://issues.apache.org/jira/browse/HBASE-5664
Project: HBase
Issue Type: Improvement
Components: Coprocessors, Filters
Affects Versions: 0.92.1
Reporter: Anoop Sam John
Assignee: Anoop Sam John
Fix For: 0.96.0, 0.94.5
Attachments: HBASE-5664_94.patch, HBASE-5664_Trunk.patch
In HRegion.nextInternal(int limit, String metric)
We have while(true) loop so as to fetch a next result which satisfies
filter condition. When Filter filters out the current fetched row we call
nextRow(byte [] currentRow) before going with the next row.
{code}
if (results.isEmpty() || filterRow()) {
// this seems like a redundant step - we already consumed the row
// there're no left overs.
// the reasons for calling this method are:
// 1. reset the filters.
// 2. provide a hook to fast forward the row (used by subclasses)
nextRow(currentRow);
{code}
// 2. provide a hook to fast forward the row (used by subclasses)
We can provide same feature of fast forward support for the CP also.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5664) CP hooks in Scan flow for fast forward when filter filters out a row
[
https://issues.apache.org/jira/browse/HBASE-5664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13549262#comment-13549262
]
Andrew Purtell commented on HBASE-5664:
---
bq. Internal patch that we had, ported to the latest 94 branch. Pls give your
comments.
It looks plausible to me. The new hook is only called once per row.
bq. What if RegionObserver1 returns false for hasMore and then RegionObserver2
returns true for hasMore ?
This kind of chaining is a typical pattern for CP hooks. CPs and the users who
use them have to deal with that in current framework design.
CP hooks in Scan flow for fast forward when filter filters out a row
Key: HBASE-5664
URL: https://issues.apache.org/jira/browse/HBASE-5664
Project: HBase
Issue Type: Improvement
Components: Coprocessors
Affects Versions: 0.92.1
Reporter: Anoop Sam John
Assignee: Anoop Sam John
Fix For: 0.96.0, 0.94.5
Attachments: HBASE-5664_94.patch
In HRegion.nextInternal(int limit, String metric)
We have while(true) loop so as to fetch a next result which satisfies
filter condition. When Filter filters out the current fetched row we call
nextRow(byte [] currentRow) before going with the next row.
{code}
if (results.isEmpty() || filterRow()) {
// this seems like a redundant step - we already consumed the row
// there're no left overs.
// the reasons for calling this method are:
// 1. reset the filters.
// 2. provide a hook to fast forward the row (used by subclasses)
nextRow(currentRow);
{code}
// 2. provide a hook to fast forward the row (used by subclasses)
We can provide same feature of fast forward support for the CP also.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5664) CP hooks in Scan flow for fast forward when filter filters out a row
[
https://issues.apache.org/jira/browse/HBASE-5664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13549342#comment-13549342
]
Anoop Sam John commented on HBASE-5664:
---
Andrew I can go with making a patch for trunk? Trunk this area of code had
undergone some changes. I will have a look at it.
bq.The new hook is only called once per row.
Yes dont think we need pre/post thing
CP hooks in Scan flow for fast forward when filter filters out a row
Key: HBASE-5664
URL: https://issues.apache.org/jira/browse/HBASE-5664
Project: HBase
Issue Type: Improvement
Components: Coprocessors
Affects Versions: 0.92.1
Reporter: Anoop Sam John
Assignee: Anoop Sam John
Fix For: 0.96.0, 0.94.5
Attachments: HBASE-5664_94.patch
In HRegion.nextInternal(int limit, String metric)
We have while(true) loop so as to fetch a next result which satisfies
filter condition. When Filter filters out the current fetched row we call
nextRow(byte [] currentRow) before going with the next row.
{code}
if (results.isEmpty() || filterRow()) {
// this seems like a redundant step - we already consumed the row
// there're no left overs.
// the reasons for calling this method are:
// 1. reset the filters.
// 2. provide a hook to fast forward the row (used by subclasses)
nextRow(currentRow);
{code}
// 2. provide a hook to fast forward the row (used by subclasses)
We can provide same feature of fast forward support for the CP also.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5664) CP hooks in Scan flow for fast forward when filter filters out a row
[
https://issues.apache.org/jira/browse/HBASE-5664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13544069#comment-13544069
]
Ted Yu commented on HBASE-5664:
---
{code}
+hasMore = ((RegionObserver)
env.getInstance()).postScannerFilterRow(ctx, s, currentRow,
+hasMore);
{code}
What if RegionObserver1 returns false for hasMore and then RegionObserver2
returns true for hasMore ?
CP hooks in Scan flow for fast forward when filter filters out a row
Key: HBASE-5664
URL: https://issues.apache.org/jira/browse/HBASE-5664
Project: HBase
Issue Type: Improvement
Components: Coprocessors
Affects Versions: 0.92.1
Reporter: Anoop Sam John
Assignee: Anoop Sam John
Fix For: 0.96.0, 0.94.5
Attachments: HBASE-5664_94.patch
In HRegion.nextInternal(int limit, String metric)
We have while(true) loop so as to fetch a next result which satisfies
filter condition. When Filter filters out the current fetched row we call
nextRow(byte [] currentRow) before going with the next row.
{code}
if (results.isEmpty() || filterRow()) {
// this seems like a redundant step - we already consumed the row
// there're no left overs.
// the reasons for calling this method are:
// 1. reset the filters.
// 2. provide a hook to fast forward the row (used by subclasses)
nextRow(currentRow);
{code}
// 2. provide a hook to fast forward the row (used by subclasses)
We can provide same feature of fast forward support for the CP also.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5664) CP hooks in Scan flow for fast forward when filter filters out a row
[
https://issues.apache.org/jira/browse/HBASE-5664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13544607#comment-13544607
]
Anoop Sam John commented on HBASE-5664:
---
I think in a scenario where there are more than one CP implementation dealing
with this hook, better the hook handle what was returned by the previous hook..
That is why I am passing the current value of the hasMore flag to the CP hook
itself.. So if a previous one returned hasMore as false, may be the next one
do not do any work and just return false ??
CP hooks in Scan flow for fast forward when filter filters out a row
Key: HBASE-5664
URL: https://issues.apache.org/jira/browse/HBASE-5664
Project: HBase
Issue Type: Improvement
Components: Coprocessors
Affects Versions: 0.92.1
Reporter: Anoop Sam John
Assignee: Anoop Sam John
Fix For: 0.96.0, 0.94.5
Attachments: HBASE-5664_94.patch
In HRegion.nextInternal(int limit, String metric)
We have while(true) loop so as to fetch a next result which satisfies
filter condition. When Filter filters out the current fetched row we call
nextRow(byte [] currentRow) before going with the next row.
{code}
if (results.isEmpty() || filterRow()) {
// this seems like a redundant step - we already consumed the row
// there're no left overs.
// the reasons for calling this method are:
// 1. reset the filters.
// 2. provide a hook to fast forward the row (used by subclasses)
nextRow(currentRow);
{code}
// 2. provide a hook to fast forward the row (used by subclasses)
We can provide same feature of fast forward support for the CP also.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5664) CP hooks in Scan flow for fast forward when filter filters out a row
[
https://issues.apache.org/jira/browse/HBASE-5664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13542734#comment-13542734
]
Anoop Sam John commented on HBASE-5664:
---
I will upload a patch based on the 94 branch soon..
CP hooks in Scan flow for fast forward when filter filters out a row
Key: HBASE-5664
URL: https://issues.apache.org/jira/browse/HBASE-5664
Project: HBase
Issue Type: Improvement
Components: Coprocessors
Affects Versions: 0.92.1
Reporter: Anoop Sam John
Assignee: Anoop Sam John
Fix For: 0.96.0
In HRegion.nextInternal(int limit, String metric)
We have while(true) loop so as to fetch a next result which satisfies
filter condition. When Filter filters out the current fetched row we call
nextRow(byte [] currentRow) before going with the next row.
{code}
if (results.isEmpty() || filterRow()) {
// this seems like a redundant step - we already consumed the row
// there're no left overs.
// the reasons for calling this method are:
// 1. reset the filters.
// 2. provide a hook to fast forward the row (used by subclasses)
nextRow(currentRow);
{code}
// 2. provide a hook to fast forward the row (used by subclasses)
We can provide same feature of fast forward support for the CP also.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5664) CP hooks in Scan flow for fast forward when filter filters out a row
[
https://issues.apache.org/jira/browse/HBASE-5664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13541965#comment-13541965
]
Andrew Purtell commented on HBASE-5664:
---
[~anoopsamjohn] Sounds ok given your reasoning but do you have a patch we can
look at?
CP hooks in Scan flow for fast forward when filter filters out a row
Key: HBASE-5664
URL: https://issues.apache.org/jira/browse/HBASE-5664
Project: HBase
Issue Type: Improvement
Components: Coprocessors
Affects Versions: 0.92.1
Reporter: Anoop Sam John
Assignee: Anoop Sam John
Fix For: 0.96.0
In HRegion.nextInternal(int limit, String metric)
We have while(true) loop so as to fetch a next result which satisfies
filter condition. When Filter filters out the current fetched row we call
nextRow(byte [] currentRow) before going with the next row.
{code}
if (results.isEmpty() || filterRow()) {
// this seems like a redundant step - we already consumed the row
// there're no left overs.
// the reasons for calling this method are:
// 1. reset the filters.
// 2. provide a hook to fast forward the row (used by subclasses)
nextRow(currentRow);
{code}
// 2. provide a hook to fast forward the row (used by subclasses)
We can provide same feature of fast forward support for the CP also.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5664) CP hooks in Scan flow for fast forward when filter filters out a row
[
https://issues.apache.org/jira/browse/HBASE-5664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13540312#comment-13540312
]
Anoop Sam John commented on HBASE-5664:
---
This is needed in the Secondary index solution
The scenario is as below
Having 2 conditions in FilterList with MUST_PASS_ALL
One SCVF on column cf1:c1 and another scvf on column cf1:c2
Suppose there is only one index on column cf1:c1
Now based on the data got from the index table we will seek to the exact row
where cf1:c1=?. But dont know what is the value of cf1:c2 there in that row.
Now the second filter may filter out this row.
As per the logic in HRegion.nextInternal() it will start a loop there and
taking next row onwards till finding a row satisfying both conditions
Now as the condition in FL is MUST_PASS_ALL, there is no point in continuing
with all the rows. The next row satisfying the condition on column cf1:c1 , we
can know from the index data and so we can make a seek to that row directly and
check that.
Now there is no way to do so. It will be good to provide a CP hook for this so
that within that hook I can reseek to the exact row key.
CP hooks in Scan flow for fast forward when filter filters out a row
Key: HBASE-5664
URL: https://issues.apache.org/jira/browse/HBASE-5664
Project: HBase
Issue Type: Improvement
Components: Coprocessors
Affects Versions: 0.92.1
Reporter: Anoop Sam John
Assignee: Anoop Sam John
Fix For: 0.96.0
In HRegion.nextInternal(int limit, String metric)
We have while(true) loop so as to fetch a next result which satisfies
filter condition. When Filter filters out the current fetched row we call
nextRow(byte [] currentRow) before going with the next row.
{code}
if (results.isEmpty() || filterRow()) {
// this seems like a redundant step - we already consumed the row
// there're no left overs.
// the reasons for calling this method are:
// 1. reset the filters.
// 2. provide a hook to fast forward the row (used by subclasses)
nextRow(currentRow);
{code}
// 2. provide a hook to fast forward the row (used by subclasses)
We can provide same feature of fast forward support for the CP also.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5664) CP hooks in Scan flow for fast forward when filter filters out a row
[
https://issues.apache.org/jira/browse/HBASE-5664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13540334#comment-13540334
]
Lars Hofhansl commented on HBASE-5664:
--
As long as the reseek is optional it's fine, I think.
A (re)seek() is potentially *much* slower than just a next(). So when the next
row is only a few next()'s away replacing that with (re)seek() can have
disastrous performance effects (as seen in HBASE-6577).
CP hooks in Scan flow for fast forward when filter filters out a row
Key: HBASE-5664
URL: https://issues.apache.org/jira/browse/HBASE-5664
Project: HBase
Issue Type: Improvement
Components: Coprocessors
Affects Versions: 0.92.1
Reporter: Anoop Sam John
Assignee: Anoop Sam John
Fix For: 0.96.0
In HRegion.nextInternal(int limit, String metric)
We have while(true) loop so as to fetch a next result which satisfies
filter condition. When Filter filters out the current fetched row we call
nextRow(byte [] currentRow) before going with the next row.
{code}
if (results.isEmpty() || filterRow()) {
// this seems like a redundant step - we already consumed the row
// there're no left overs.
// the reasons for calling this method are:
// 1. reset the filters.
// 2. provide a hook to fast forward the row (used by subclasses)
nextRow(currentRow);
{code}
// 2. provide a hook to fast forward the row (used by subclasses)
We can provide same feature of fast forward support for the CP also.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5664) CP hooks in Scan flow for fast forward when filter filters out a row
[
https://issues.apache.org/jira/browse/HBASE-5664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13540351#comment-13540351
]
Anoop Sam John commented on HBASE-5664:
---
Yes in core we do not do any seek to the next row(as in HBASE-6577)
This is just a hook so that an implementation like sec index can make use of
this.
// 2. provide a hook to fast forward the row (used by subclasses)
nextRow(currentRow);
The above comment in the code says calling nextRow() can potentially give a way
for fast forwarding. But not sure how to make use. The RegionScannerImpl is not
a public class and so can not be subclassed. We can only wrap this obj. So I
felt new hook is the only way.
[~andrew.purt...@gmail.com] You have some comments/suggestion ?
CP hooks in Scan flow for fast forward when filter filters out a row
Key: HBASE-5664
URL: https://issues.apache.org/jira/browse/HBASE-5664
Project: HBase
Issue Type: Improvement
Components: Coprocessors
Affects Versions: 0.92.1
Reporter: Anoop Sam John
Assignee: Anoop Sam John
Fix For: 0.96.0
In HRegion.nextInternal(int limit, String metric)
We have while(true) loop so as to fetch a next result which satisfies
filter condition. When Filter filters out the current fetched row we call
nextRow(byte [] currentRow) before going with the next row.
{code}
if (results.isEmpty() || filterRow()) {
// this seems like a redundant step - we already consumed the row
// there're no left overs.
// the reasons for calling this method are:
// 1. reset the filters.
// 2. provide a hook to fast forward the row (used by subclasses)
nextRow(currentRow);
{code}
// 2. provide a hook to fast forward the row (used by subclasses)
We can provide same feature of fast forward support for the CP also.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5664) CP hooks in Scan flow for fast forward when filter filters out a row
[
https://issues.apache.org/jira/browse/HBASE-5664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13465428#comment-13465428
]
Anoop Sam John commented on HBASE-5664:
---
This can be useful when the secondary index implementation is done.
[HBASE-2038?]
Suppose we have 2 conditions on the scan based on 2 columns. Only we have one
index on the 1st column. This index can give me some rowkeys. Using a wrapper
for the RegionScanner and using reseek() I can seek to correct rows. Now after
data fetch the filter needs to be applied on that as the second column
condition check also to happen. And here if this check filters out the row we
need to make a seek to the next row as found from the index data(which is on
the one column)
In HRegion#nextInternal(int limit) we have
while (true) {
if (results.isEmpty() || filterRow()) {
// this seems like a redundant step - we already consumed the row
// there're no left overs.
// the reasons for calling this method are:
// 1. reset the filters.
// 2. provide a hook to fast forward the row (used by subclasses)
boolean moreRows = nextRow(currentRow);
...
}
My idea is to give a CP hook here so that from that user can make a reseek()
Otherwise the normal code flow while() loop will take the next row from the
table and apply filter and so on and we wont get the advantage of the index data
[~andrew.purt...@gmail.com] Can you please comment on this?
We have a patch for this adding the new hook which we have used internally
CP hooks in Scan flow for fast forward when filter filters out a row
Key: HBASE-5664
URL: https://issues.apache.org/jira/browse/HBASE-5664
Project: HBase
Issue Type: Improvement
Components: Coprocessors
Affects Versions: 0.92.1
Reporter: Anoop Sam John
Assignee: Anoop Sam John
Fix For: 0.96.0
In HRegion.nextInternal(int limit, String metric)
We have while(true) loop so as to fetch a next result which satisfies
filter condition. When Filter filters out the current fetched row we call
nextRow(byte [] currentRow) before going with the next row.
{code}
if (results.isEmpty() || filterRow()) {
// this seems like a redundant step - we already consumed the row
// there're no left overs.
// the reasons for calling this method are:
// 1. reset the filters.
// 2. provide a hook to fast forward the row (used by subclasses)
nextRow(currentRow);
{code}
// 2. provide a hook to fast forward the row (used by subclasses)
We can provide same feature of fast forward support for the CP also.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5664) CP hooks in Scan flow for fast forward when filter filters out a row
[
https://issues.apache.org/jira/browse/HBASE-5664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13465433#comment-13465433
]
ramkrishna.s.vasudevan commented on HBASE-5664:
---
@Anoop
In HBASE-6577 see Lars comment
https://issues.apache.org/jira/browse/HBASE-6577?focusedCommentId=13433898page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13433898.
Anyway we can try to get this in as we are using the new hook internally in our
version based on consent from others.
CP hooks in Scan flow for fast forward when filter filters out a row
Key: HBASE-5664
URL: https://issues.apache.org/jira/browse/HBASE-5664
Project: HBase
Issue Type: Improvement
Components: Coprocessors
Affects Versions: 0.92.1
Reporter: Anoop Sam John
Assignee: Anoop Sam John
Fix For: 0.96.0
In HRegion.nextInternal(int limit, String metric)
We have while(true) loop so as to fetch a next result which satisfies
filter condition. When Filter filters out the current fetched row we call
nextRow(byte [] currentRow) before going with the next row.
{code}
if (results.isEmpty() || filterRow()) {
// this seems like a redundant step - we already consumed the row
// there're no left overs.
// the reasons for calling this method are:
// 1. reset the filters.
// 2. provide a hook to fast forward the row (used by subclasses)
nextRow(currentRow);
{code}
// 2. provide a hook to fast forward the row (used by subclasses)
We can provide same feature of fast forward support for the CP also.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5664) CP hooks in Scan flow for fast forward when filter filters out a row
[
https://issues.apache.org/jira/browse/HBASE-5664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13465651#comment-13465651
]
ramkrishna.s.vasudevan commented on HBASE-5664:
---
@Lars
Anoop has clearly mentioned the scenario. When i commented about this in
HBASE-6577 i did not mention the usecase where we are trying to add such hooks.
Here we are ideally not doing the forwarding that is done by the Scans, whereas
it is something we leave it to the CP to decide on what to reseek rather than
we reseeking on the kernel like in HBASE-6577. This hook is really useful for
us.
CP hooks in Scan flow for fast forward when filter filters out a row
Key: HBASE-5664
URL: https://issues.apache.org/jira/browse/HBASE-5664
Project: HBase
Issue Type: Improvement
Components: Coprocessors
Affects Versions: 0.92.1
Reporter: Anoop Sam John
Assignee: Anoop Sam John
Fix For: 0.96.0
In HRegion.nextInternal(int limit, String metric)
We have while(true) loop so as to fetch a next result which satisfies
filter condition. When Filter filters out the current fetched row we call
nextRow(byte [] currentRow) before going with the next row.
{code}
if (results.isEmpty() || filterRow()) {
// this seems like a redundant step - we already consumed the row
// there're no left overs.
// the reasons for calling this method are:
// 1. reset the filters.
// 2. provide a hook to fast forward the row (used by subclasses)
nextRow(currentRow);
{code}
// 2. provide a hook to fast forward the row (used by subclasses)
We can provide same feature of fast forward support for the CP also.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5664) CP hooks in Scan flow for fast forward when filter filters out a row
[
https://issues.apache.org/jira/browse/HBASE-5664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13242047#comment-13242047
]
Anoop Sam John commented on HBASE-5664:
---
Previously seeking was not possible on the RegionScanner. Now with HBASE-5520
CP can do a fast forward with reseek() on RegionScanner.
Use case of this new hook
CP hooks in Scan flow for fast forward when filter filters out a row
Key: HBASE-5664
URL: https://issues.apache.org/jira/browse/HBASE-5664
Project: HBase
Issue Type: Improvement
Components: coprocessors
Affects Versions: 0.92.1
Reporter: Anoop Sam John
Assignee: Anoop Sam John
Fix For: 0.96.0
In HRegion.nextInternal(int limit, String metric)
We have while(true) loop so as to fetch a next result which satisfies
filter condition. When Filter filters out the current fetched row we call
nextRow(byte [] currentRow) before going with the next row.
{code}
if (results.isEmpty() || filterRow()) {
// this seems like a redundant step - we already consumed the row
// there're no left overs.
// the reasons for calling this method are:
// 1. reset the filters.
// 2. provide a hook to fast forward the row (used by subclasses)
nextRow(currentRow);
{code}
// 2. provide a hook to fast forward the row (used by subclasses)
We can provide same feature of fast forward support for the CP also.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5664) CP hooks in Scan flow for fast forward when filter filters out a row
[
https://issues.apache.org/jira/browse/HBASE-5664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13242048#comment-13242048
]
Anoop Sam John commented on HBASE-5664:
---
Previosly there was no use to give CP hook as there were no seek available in
RegionScanner. Now with HBASE-5520 reseek() is provided in RegionScanner, which
the CP hook can use for fast forward.
CP hooks in Scan flow for fast forward when filter filters out a row
Key: HBASE-5664
URL: https://issues.apache.org/jira/browse/HBASE-5664
Project: HBase
Issue Type: Improvement
Components: coprocessors
Affects Versions: 0.92.1
Reporter: Anoop Sam John
Assignee: Anoop Sam John
Fix For: 0.96.0
In HRegion.nextInternal(int limit, String metric)
We have while(true) loop so as to fetch a next result which satisfies
filter condition. When Filter filters out the current fetched row we call
nextRow(byte [] currentRow) before going with the next row.
{code}
if (results.isEmpty() || filterRow()) {
// this seems like a redundant step - we already consumed the row
// there're no left overs.
// the reasons for calling this method are:
// 1. reset the filters.
// 2. provide a hook to fast forward the row (used by subclasses)
nextRow(currentRow);
{code}
// 2. provide a hook to fast forward the row (used by subclasses)
We can provide same feature of fast forward support for the CP also.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
