cetra3 commented on code in PR #20642:
URL: https://github.com/apache/datafusion/pull/20642#discussion_r2916181718
##########
datafusion/physical-plan/src/sorts/multi_level_merge.rs:
##########
@@ -356,45 +363,57 @@ impl MultiLevelMergeBuilder {
) -> Result<(Vec<SortedSpillFile>, usize)> {
assert_ne!(buffer_len, 0, "Buffer length must be greater than 0");
let mut number_of_spills_to_read_for_current_phase = 0;
+ // Track total memory needed for spill file buffers. When the
Review Comment:
It feels like this whole method is ripe for a refactor, and introducing a
memory floor is making it even more complex. Is there a way to incorporate the
memory floor, but also simplify this a little bit
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
