[
https://issues.apache.org/jira/browse/DRILL-6616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16559284#comment-16559284
]
ASF GitHub Bot commented on DRILL-6616:
---------------------------------------
sohami commented on a change in pull request #1401: DRILL-6616: Batch
Processing for Lateral/Unnest
URL: https://github.com/apache/drill/pull/1401#discussion_r205669743
##########
File path:
exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/unnest/UnnestImpl.java
##########
@@ -48,24 +51,47 @@
private static final Logger logger =
LoggerFactory.getLogger(UnnestImpl.class);
private ImmutableList<TransferPair> transfers;
- private LateralContract lateral; // corresponding lateral Join (or other
operator implementing the Lateral Contract)
private SelectionVectorMode svMode;
private RepeatedValueVector fieldToUnnest;
private RepeatedValueVector.RepeatedAccessor accessor;
private RecordBatch outgoing;
+ private IntVector rowIdVector ; // Allocated and owned by the
UnnestRecordBatch
+ private IntVector.Mutator rowIdVectorMutator;
+
/**
* The output batch limit starts at OUTPUT_ROW_COUNT, but may be decreased
* if records are found to be large.
*/
private int outputLimit = ValueVector.MAX_ROW_COUNT;
- // The index in the unnest column that is being processed.We start at zero
and continue until
+
+ /**
+ * We maintain three indexes
+ *
+ *
+ *
+ valueIndex 0 1 2 3
+ |- - - - -|- - - -|- -|- - - -|
+ | | | | | | | | | | | | | | | |
+ |- - - - -|- - - -|- -|- - - -|
+ innerValueIndex 0 1 2 3 4 0 1 2 3 0 1 0 1 2 3 |
+ runningInnerValueIndex 0 1 2 3 4 5 6 7 8 9 ...
+ *
Review comment:
Thanks for ascii art :)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
> Batch Processing for Lateral/Unnest
> -----------------------------------
>
> Key: DRILL-6616
> URL: https://issues.apache.org/jira/browse/DRILL-6616
> Project: Apache Drill
> Issue Type: Improvement
> Components: Execution - Relational Operators
> Affects Versions: 1.14.0
> Reporter: Sorabh Hamirwasia
> Assignee: Sorabh Hamirwasia
> Priority: Major
> Fix For: 1.15.0
>
>
> Implement the execution and planner side changes for the batch processing
> done by lateral and unnest. Based on the prototype we found performance to be
> much better as compared to initial row-by-row execution.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
