2010YOUY01 commented on code in PR #21713:
URL: https://github.com/apache/datafusion/pull/21713#discussion_r3106167735
##########
datafusion/physical-plan/src/spill/spill_manager.rs:
##########
@@ -160,9 +160,22 @@ impl SpillManager {
Ok(file.map(|f| (f, max_record_batch_size)))
}
- /// Reads a spill file as a stream. The file must be created by the
current `SpillManager`.
- /// This method will generate output in FIFO order: the batch appended
first
- /// will be read first.
+ /// Reads a spill file as a stream. The file must be created by the current
Review Comment:
I looked at the code—it appears to succeed silently, and error would be
propagated to the stream consumer due to a schema mismatch.
I’ve updated the comment to indicate this may lead to undefined behavior.
We could be more defensive here; I’ll send a follow-up patch to make it fail
fast.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
