I have written a small design document explaining the batch sizing work we are doing for operators (other than scan).
https://issues.apache.org/jira/browse/DRILL-6238 Some of this work has already been done in 1.13. flatten, merge join and external sort are changed to adhere to batch size limits. The work will continue in future releases. I would like to get feedback on the approach being used. All comments are welcome. Please update the JIRA or respond to this email with your comments. Thanks Padma