[
https://issues.apache.org/jira/browse/DRILL-6678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16580555#comment-16580555
]
Sorabh Hamirwasia commented on DRILL-6678:
------------------------------------------
The idea is the 3rd proposal using Batch Sizing.
{quote}Or, is the idea to append rows from multiple incoming batches until the
output batch reaches the target size? That is, if A, B and C are incoming
batches, the output batch may have all selected rows from A and B, and, say,
have the selected rows from C.
{quote}
> Improve SelectionVectorRemover to pack output batch based on BatchSizing
> ------------------------------------------------------------------------
>
> Key: DRILL-6678
> URL: https://issues.apache.org/jira/browse/DRILL-6678
> Project: Apache Drill
> Issue Type: Improvement
> Components: Execution - Relational Operators
> Affects Versions: 1.14.0
> Reporter: Sorabh Hamirwasia
> Assignee: Sorabh Hamirwasia
> Priority: Major
>
> SelectionVectorRemover in most of the cases is downstream to Filter which
> reduces the number of records to be copied in output container. In those
> cases if SelectionVectorRemover can pack the output batch to maximum
> utilization that will reduce the number of output batches from it and will
> help to improve performance. During Lateral & Unnest Performance evaluation
> we have noticed a significant decrease in performance as number of batches
> increases for same number of rows (i.e. Batch is not fully packed)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
