[
https://issues.apache.org/jira/browse/CALCITE-2979?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16810739#comment-16810739
]
Stamatis Zampetakis commented on CALCITE-2979:
----------------------------------------------
Arrow format would be great and I would like to work on it in the future but is
not directly related to the problem that I am trying to solve right now. As
[~sereda] , correctly guessed Scan(B) is an external source which is not
necessarily in Arrow convention. I would like to match Filter + Scan(B) and
generate for example the following plan
{noformat}
NestedLoop(blockSize=3)
Scan(A)
ElasticScan(table=B, query="OR(>(cor0[0],B.id), >(cor0[1],B.id),
>(cor0[2],B.id)")
{noformat}
that way instead of sending one query to Elastic for each tuple in A, I divide
the number of queries to |A|/|BLK_SIZE|.
> Add a block-based nested loop join algorithm
> --------------------------------------------
>
> Key: CALCITE-2979
> URL: https://issues.apache.org/jira/browse/CALCITE-2979
> Project: Calcite
> Issue Type: Improvement
> Components: core
> Affects Versions: 1.19.0
> Reporter: Stamatis Zampetakis
> Assignee: Stamatis Zampetakis
> Priority: Major
> Labels: performance
>
> Currently, Calcite provides a tuple-based nested loop join algorithm
> implemented through EnumerableCorrelate and EnumerableDefaults.correlateJoin.
> This means that for each tuple of the outer relation we probe (set variables)
> in the inner relation.
> The goal of this issue is to add new algorithm (or extend the correlateJoin
> method) which first gathers blocks (batches) of tuples from the outer
> relation and then probes the inner relation once per block.
> There are cases (eg., indexes) where the inner relation can be accessed by
> more than one value which can greatly improve the performance in particular
> when the outer relation is big.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
