Padma Penumarthy commented on DRILL-6180:

[~paul-rogers] [~kkhatua] Thank you for your input. I will make it 16MB for all 
operators for now. Definitely, 32MB is not going to work and sort is at least 
tested well with 16MB. Once we start doing more testing for batch sizing, I 
will reduce this further to 8MB or 4MB if we don't see any performance 

> Use System Option "output_batch_size" for External Sort
> -------------------------------------------------------
>                 Key: DRILL-6180
>                 URL: https://issues.apache.org/jira/browse/DRILL-6180
>             Project: Apache Drill
>          Issue Type: Improvement
>          Components: Execution - Flow
>    Affects Versions: 1.12.0
>            Reporter: Padma Penumarthy
>            Assignee: Padma Penumarthy
>            Priority: Critical
>             Fix For: 1.13.0
> External Sort has boot time configuration for output batch size 
> "drill.exec.sort.external.spill.merge_batch_size" which is defaulted to 16M.
> To make batch sizing configuration uniform across all operators, change this 
> to use new system option that is added 
> "drill.exec.memory.operator.output_batch_size". This option has default value 
> of 32M.
> So, what are the implications if default is changed to 32M for external sort ?
> Instead, should we change the output batch size default to 16M for all 
> operators ?

This message was sent by Atlassian JIRA

Reply via email to