suremarc commented on issue #10316: URL: https://github.com/apache/datafusion/issues/10316#issuecomment-2447792748
Hey @alamb, I am interested in using `ProgressiveEval` (or having the functionality merged into `SortPreservingMergeStream`) as my use case is optimizing unions where the children have non-overlapping ranges. For instance: ```sql SELECT * FROM t1 WHERE "timestamp" < cutoff UNION ALL SELECT * FROM t2 WHERE "timestamp" >= cutoff ORDER BY "timestamp" ``` In principle it should be possible to read results from `t1` first. I do think we can reuse the analysis in #9593, but ideally we would have statistics per partition as @ozankabak mentioned. This would allow us to implement the operator generically, without really having to inspect its children. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
