jayzhan211 commented on PR #11896:
URL: https://github.com/apache/datafusion/pull/11896#issuecomment-2276997223
`DataFusion error: Arrow error: Invalid argument error: Nested comparison:
Struct\(\[Field \{ name: "c0", data_type: Utf8, nullable: true, dict_id: 0,
dict_is_ordered: false, metadata: \{\} \}, Field \{ name: "c1", data_type:
Utf8, nullable: true, dict_id: 0, dict_is_ordered: false, metadata: \{\} \}\]\)
== Struct\(\[Field \{ name: "c0", data_type: Utf8, nullable: true, dict_id: 0,
dict_is_ordered: false, metadata: \{\} \}, Field \{ name: "c1", data_type:
Utf8, nullable: true, dict_id: 0, dict_is_ordered: false, metadata: \{\} \}\]\)
\(hint: use make_comparator instead\)`
`make_comparator ` is introduced recently for comparing nested type.
However, InListExpr had not yet supported to compare nested type
```rust
fn evaluate(&self, batch: &RecordBatch) -> Result<ColumnarValue> {
let num_rows = batch.num_rows();
let value = self.expr.evaluate(batch)?;
let r = match &self.static_filter {
Some(f) => f.contains(value.into_array(num_rows)?.as_ref(),
self.negated)?,
None => {
let value = value.into_array(num_rows)?;
let found = self.list.iter().map(|expr|
expr.evaluate(batch)).try_fold(
BooleanArray::new(BooleanBuffer::new_unset(num_rows),
None),
|result, expr| -> Result<BooleanArray> {
Ok(or_kleene(
&result,
&eq(&value, &expr?.into_array(num_rows)?)?,
)?)
},
)?;
if self.negated {
not(&found)?
} else {
found
}
}
};
Ok(ColumnarValue::Array(Arc::new(r)))
}
```
We need to change the `eq` to new function that calls
`compare_op_for_nested` for nested type, and fallback to `eq` for non-nested
type.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org
For additional commands, e-mail: github-h...@datafusion.apache.org
