buraksenn commented on issue #20969:
URL: https://github.com/apache/datafusion/issues/20969#issuecomment-4093170754
I've looked into this and wanted to check-in before proceeding since I may
miss something @alamb.
Implementation can be like this imo as in the issue description:
```
fn instantiate_static_filter(
in_array: ArrayRef,
) -> Result<Arc<dyn StaticFilter + Send + Sync>> {
// Flatten dictionary-encoded haystacks to their value type so that
// specialized filters (e.g. Int32StaticFilter) are used instead of
// falling through to the generic ArrayStaticFilter.
// Dictionary needles are still handled at evaluation time via the
// `downcast_dictionary_array!` blocks in each `contains()`
implementation.
let in_array = match in_array.data_type() {
DataType::Dictionary(_, value_type) => cast(&in_array,
value_type.as_ref())?,
_ => in_array,
};
```
We would always flatten and thus be able to use specific filters such as
Int32StaticFilter etc instead of falling to ArrayStaticFilter. Then I can add
comments, relevant test cases and clarify contract etc.
Then as a next step I can introduce schema validation to
`try_new_from_array` same as `try_new` such that we can check type mismatches
early.
WDYT @alamb ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
