berkaysynnada commented on code in PR #12174:
URL: https://github.com/apache/datafusion/pull/12174#discussion_r1732718351
##########
datafusion/physical-plan/src/sorts/sort.rs:
##########
@@ -737,9 +738,30 @@ impl SortExec {
/// This can reduce the memory pressure required by the sort
/// operation since rows that are not going to be included
/// can be dropped.
- pub fn with_fetch(mut self, fetch: Option<usize>) -> Self {
- self.fetch = fetch;
- self
+ pub fn with_fetch(&self, fetch: Option<usize>) -> Self {
+ let mut cache = self.cache.clone();
+ if fetch.is_some() {
+ // When a theoretically unnecessary sort becomes a top-K (which
Review Comment:
> I don't fully understand how a top-k sort would become bounded. I may
misundersrtand what the
[ExecutionMode](https://docs.rs/datafusion/latest/datafusion/physical_plan/enum.ExecutionMode.html)
trait means, but it seems like `TopK` could not complete until its input
completed, but if its input was unbounded the sort itself therefore would also
be unbounded
I misassumed the implementation of top-k. Could you please take a look to
the new idea?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
