Pritesh Maker updated DRILL-6113:
    Issue Type: Improvement  (was: Bug)

> Limit batch size for Merge Receiver
> -----------------------------------
>                 Key: DRILL-6113
>                 URL: https://issues.apache.org/jira/browse/DRILL-6113
>             Project: Apache Drill
>          Issue Type: Improvement
>          Components: Execution - Flow
>    Affects Versions: 1.12.0
>            Reporter: Padma Penumarthy
>            Assignee: Padma Penumarthy
>            Priority: Major
>             Fix For: 1.14.0
> Merge receiver has hard coded limit of 32K rows as batch. Since rows can be 
> of varying width, it is difficult to predict the output batch size (in terms 
> of memory) for this operator. Change this to derive row count based on actual 
> memory available. We are introducing a new option called outputBatchSize to 
> limit the batch size of each operator. Use the memory configured from that. 
> Figure out the average row width of outgoing batch based on averages of 
> batches coming from incoming streams. Limit the row count based on memory 
> available to use and average row width.

This message was sent by Atlassian JIRA

Reply via email to