advancedxy commented on code in PR #184:
URL:
https://github.com/apache/arrow-datafusion-comet/pull/184#discussion_r1520009054
##########
core/src/execution/datafusion/planner.rs:
##########
@@ -496,7 +493,16 @@ impl PhysicalPlanner {
.iter()
.map(|x| self.create_expr(x,
input_schema.clone()).unwrap())
.collect::<Vec<_>>();
- Ok(Arc::new(InListExpr::new(value, list, expr.negated, None)))
+ // in_list doesn't handle value being dictionary type
correctly, so we need to fall
+ // back to InListExpr if in_list fails.
+ // TODO: remove the fallback when
https://github.com/apache/arrow-datafusion/issues/9530 is fixed
Review Comment:
> nit: maybe we can check if input_schema contains any dictionary type and
use InListExpr if so?
Sound good to me. I will address it in a following commit in probably ~8
hours as I'm going to bed now.
> Unfortunately dictionary type is treated as a logical type in DF, which
caused many issues like this before. We created
https://github.com/apache/arrow-datafusion/discussions/7421 but haven't got
resources to work on it yet.
yea.. I can imagine that a lot of issues/special handling of dictionary
array would be introduced.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
