[ 
https://issues.apache.org/jira/browse/DRILL-6113?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Pritesh Maker updated DRILL-6113:
---------------------------------
    Fix Version/s:     (was: 1.13.0)
                   1.14.0

> Limit batch size for Merge Receiver
> -----------------------------------
>
>                 Key: DRILL-6113
>                 URL: https://issues.apache.org/jira/browse/DRILL-6113
>             Project: Apache Drill
>          Issue Type: Improvement
>          Components: Execution - Flow
>    Affects Versions: 1.12.0
>            Reporter: Padma Penumarthy
>            Assignee: Padma Penumarthy
>            Priority: Major
>             Fix For: 1.14.0
>
>
> Merge receiver has hard coded limit of 32K rows as batch. Since rows can be 
> of varying width, it is difficult to predict the output batch size (in terms 
> of memory) for this operator. Change this to derive row count based on actual 
> memory available. We are introducing a new option called outputBatchSize to 
> limit the batch size of each operator. Use the memory configured from that. 
> Figure out the average row width of outgoing batch based on averages of 
> batches coming from incoming streams. Limit the row count based on memory 
> available to use and average row width.
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Reply via email to