2010YOUY01 commented on code in PR #17163:
URL: https://github.com/apache/datafusion/pull/17163#discussion_r2300781427
##########
datafusion/physical-plan/src/sorts/sort.rs:
##########
@@ -782,20 +843,62 @@ impl ExternalSorter {
&mut self,
input: &RecordBatch,
) -> Result<()> {
- let size = get_reserved_byte_for_record_batch(input);
+ let batch_size = get_reserved_byte_for_record_batch(input, None);
+ let sort_res = self.reservation.try_grow(batch_size);
+
+ // if cursor is smaller than half of original batch, we may say that
2x batch is enough for both sort and merge phase
+ let cursor_small = self
Review Comment:
Here it's doing: if `rows/original_batch < 1.0`, round up the `rows` size
estimation to `1.0` instead of the actual ratio. Why doing this round up?
BTW is this PR ready to review now?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
