Dandandan commented on issue #3516: URL: https://github.com/apache/arrow-datafusion/issues/3516#issuecomment-1254006432
So it seems we *mostly* have the benefits of a TopK operator now by pushing down the limit to individual operations. There are a couple of followups possible (will create some tickets for them and close this one): * Avoiding spilling to disk by reinserting the to-spill data to a memory buffer * Use limit in `SortPreserveMergeExec` * Keeping memory under control by merging buffered batches once in a while -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
