yashrb24 opened a new issue, #25463:
URL: https://github.com/apache/datafusion/issues/25463
### Is your feature request related to a problem or challenge?
Ordinary joins on multiple equality columns can miss opportunities to skip
data. For example:
```sql
SELECT *
FROM fact f
JOIN dimension d
ON f.a = d.x
AND f.b = d.y;
```
When the build-side input is small enough, DataFusion collects its key
combinations and creates a filter like:
```sql
(f.a, f.b) IN ((1, 10), (2, 20))
```
This correctly filters rows, but Bloom-filter pruning cannot extract the
allowed values for each column. We can therefore read row groups that cannot
contain a
### Describe the solution you'd like
We can extract the allowed values for each column and use them
```sql
a IN (1, 2)
b IN (10, 20)
```
while also keeping the original filter so that we don't have any false
positive matches
### Describe alternatives you've considered
Adding separate row filters for each column. This would repeat checks when
the original tuple filter already checks the complete combination.
### Additional context
_No response_
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
