[
https://issues.apache.org/jira/browse/PHOENIX-6501?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17501638#comment-17501638
]
ASF GitHub Bot commented on PHOENIX-6501:
-----------------------------------------
kadirozde commented on pull request #1399:
URL: https://github.com/apache/phoenix/pull/1399#issuecomment-1059669953
> Can we do this for local indexes as well? There is also a significant cost
to seeking (even when done locally)
>
> In fact the only difference might be how we get the a table reference.
@lhofhansl, we can definitely do this for local indexes. There are some
differences. The first is as you pointed out table vs region. For local
indexes, both data and index are accessed via the local region while the global
index is accessed remotely so we need to get the table for the global index and
then get connections to access the table. The concern of which connection pool
to use is not applicable to the local index. The second difference is that
there is one table region to retrieve data rows for the local index. However,
for the global index, there can be many. So we discover the table region
boundaries and access them in parallel for the global indexes using a thread
pool, which is not necessary for the local index. The last difference is
handling the row key offset for local indexes, which is not necessary for the
global indexes. So, I thought instead of lumping local and global index
batching together, we should handle them separately. I suggest having a
separate Jira and PR for the local index.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
> Use batching when joining data table rows with uncovered index rows
> -------------------------------------------------------------------
>
> Key: PHOENIX-6501
> URL: https://issues.apache.org/jira/browse/PHOENIX-6501
> Project: Phoenix
> Issue Type: Improvement
> Affects Versions: 5.1.2
> Reporter: Kadir Ozdemir
> Assignee: Kadir OZDEMIR
> Priority: Major
>
> PHOENIX-6458 extends the existing uncovered local index support for global
> indexes. The current solution uses HBase get operations to join data table
> rows with uncovered index rows on the server side. Doing a separate RPC call
> for every data table row can be expensive. Instead, we can buffer lots of
> data row keys in memory, use a skip scan filter and even multiple threads to
> issue a separate scan for each data table region in parallel. This will
> reduce the cost of join and also improve the performance.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
