[GitHub] drill issue #1075: DRILL-6030: Managed sort should minimize number of batche...

2017-12-21 Thread vrozov
Github user vrozov commented on the issue: https://github.com/apache/drill/pull/1075 @paul-rogers Please review ---

[GitHub] drill issue #1075: DRILL-6030: Managed sort should minimize number of batche...

2017-12-19 Thread vrozov
Github user vrozov commented on the issue: https://github.com/apache/drill/pull/1075 The scenario when all batches can be merged in memory is covered by 'if (canUseMemoryMerge())` check in `SortImpl.java:399`. The affected code path applies only to cases where merge between spilled

[GitHub] drill issue #1075: DRILL-6030: Managed sort should minimize number of batche...

2017-12-19 Thread paul-rogers
Github user paul-rogers commented on the issue: https://github.com/apache/drill/pull/1075 One additional thought. This bug was found when sorting 18 GB of data in 8 GB of memory. That is, a case in which the sort must spill. What happens in the case in which the 18 GB of data