Paul Rogers created DRILL-5026:
----------------------------------
Summary: ExternalSortBatch uses two memory allocators; one will do
Key: DRILL-5026
URL: https://issues.apache.org/jira/browse/DRILL-5026
Project: Apache Drill
Issue Type: Improvement
Affects Versions: 1.8.0
Reporter: Paul Rogers
Priority: Minor
The {{ExternalSortBatch}} (ESB) operator performs a sort while spilling to disk
to stay within a defined memory budget.
ESB uses two memory allocators:
* The operator allocator for most operations,
* The copier allocator (child of the operator allocator) for merge operations.
The copier allocator is used only when merging prior to spill.
However, since the ESB must manage within a single memory budget, the code is
made more complex by using two allocators, and keeping track of the memory used
in both.
Better to use a single allocator. When checking for the need to spill, the code
already checks if enough memory is available for an in-memory sort, if needed.
Similarly, the code should also check if sufficient memory is available for the
copy step when needed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
