LiaCastaneda commented on issue #19580: URL: https://github.com/apache/datafusion/issues/19580#issuecomment-3709862908
> but I'm not sure if that's a good idea: if the probe side opens a large batch of files with partial filters and then immediately the filters complete it won't fully pick them up until the next batch of files, which would make for a more inefficient scan. I'm not sure I'm understanding this correctly, would this downside only apply to `CollectLeft`? If it's `Partitioned`, I imagine, for example, partition 0 will be able to move on with the dynamic filter from build partition 0, and so on, which in theory is better than probe partition 0 waiting for the rest of the partitions? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
