stuhood commented on issue #23817: URL: https://github.com/apache/datafusion/issues/23817#issuecomment-5652821019
> I'd like to help drive this, and I can bring a concrete consumer to the table. I work on ParadeDB's pg_search, where right now every partitioned-mode join dynamic filter contributes zero pruning: our index pushdown only recurses AND chains, so a CASE root extracts no InLists, and our fast-field prefilter rejects the conjunct because CaseExpr is not in its allowlist. The sharp edge is that our range co-partitioned join rewrite is itself what flips eligible joins into Partitioned mode, so our own optimization currently disables our own pruning. This isn't accurate under `datafusion-distributed`, which is the only place where we should have more than 1 partition. So I do not think that this issue is relevant to ParadeDB. ---- But having said that, I still think that it is a very important issue. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
