[
https://issues.apache.org/jira/browse/DRILL-6113?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pritesh Maker updated DRILL-6113:
---------------------------------
Fix Version/s: (was: 1.14.0)
> Limit batch size for Merge Receiver
> -----------------------------------
>
> Key: DRILL-6113
> URL: https://issues.apache.org/jira/browse/DRILL-6113
> Project: Apache Drill
> Issue Type: Improvement
> Components: Execution - Flow
> Affects Versions: 1.12.0
> Reporter: Padma Penumarthy
> Assignee: Padma Penumarthy
> Priority: Major
>
> Merge receiver has hard coded limit of 32K rows as batch. Since rows can be
> of varying width, it is difficult to predict the output batch size (in terms
> of memory) for this operator. Change this to derive row count based on actual
> memory available. We are introducing a new option called outputBatchSize to
> limit the batch size of each operator. Use the memory configured from that.
> Figure out the average row width of outgoing batch based on averages of
> batches coming from incoming streams. Limit the row count based on memory
> available to use and average row width.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
