korowa commented on issue #14238: URL: https://github.com/apache/datafusion/issues/14238#issuecomment-2613214394
> However another path related to coalesce might help downstream nodes or direct consumer not to struggle because of swarm of small batches I don't have a strong opinion here -- intuitively it seems like embedding coalescer into filtering operators (not only joins) could be beneficial for query execution time just because there will be less operators in the pipeline, but it still should be checked and somehow measured. I'll try to come up with some POC during a weekend for coalescer in e.g. FilterExec (this one just seems to be the easiest to implement). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For additional commands, e-mail: github-h...@datafusion.apache.org
