houqp edited a comment on pull request #792:
URL: https://github.com/apache/arrow-datafusion/pull/792#issuecomment-919743458
@waynexia I forced a stack trace using panic to pin point the issue:
```
thread '<unnamed>' panicked at 'explicit panic',
/home/houqp/Documents/code/delta/arrow-datafusion/datafusion/src/logical_plan/dfschema.rs:173:17
stack backtrace:
0: std::panicking::begin_panic
at
/rustc/1698e3cac54aa8691d4e9e207567672af8231cb6/library/std/src/panicking.rs:543:12
1: datafusion::logical_plan::dfschema::DFSchema::index_of_column_by_name
at
/home/houqp/Documents/code/delta/arrow-datafusion/datafusion/src/logical_plan/dfschema.rs:173:17
2: datafusion::logical_plan::dfschema::DFSchema::field_with_qualified_name
at
/home/houqp/Documents/code/delta/arrow-datafusion/datafusion/src/logical_plan/dfschema.rs:242:19
3: datafusion::logical_plan::dfschema::DFSchema::field_from_column
at
/home/houqp/Documents/code/delta/arrow-datafusion/datafusion/src/logical_plan/dfschema.rs:249:24
4: datafusion::logical_plan::expr::Expr::get_type
at
/home/houqp/Documents/code/delta/arrow-datafusion/datafusion/src/logical_plan/expr.rs:375:35
5: datafusion::logical_plan::expr::Expr::get_type
at
/home/houqp/Documents/code/delta/arrow-datafusion/datafusion/src/logical_plan/expr.rs:425:18
6: datafusion::optimizer::common_subexpr_eliminate::optimize
at
/home/houqp/Documents/code/delta/arrow-datafusion/datafusion/src/optimizer/common_subexpr_eliminate.rs:109:29
7:
<datafusion::optimizer::common_subexpr_eliminate::CommonSubexprEliminate as
datafusion::optimizer::optimizer::OptimizerRule>::optimize
at
/home/houqp/Documents/code/delta/arrow-datafusion/datafusion/src/optimizer/common_subexpr_eliminate.rs:61:9
8: datafusion::execution::context::ExecutionContext::optimize_internal
at
/home/houqp/Documents/code/delta/arrow-datafusion/datafusion/src/execution/context.rs:618:24
9: datafusion::execution::context::ExecutionContext::optimize
at
/home/houqp/Documents/code/delta/arrow-datafusion/datafusion/src/execution/context.rs:499:13
10: datafusion::dataframe::DataFrame::collect
at ./src/dataframe.rs:125:20
11: datafusion::dataframe::<impl
pyo3::class::impl_::PyMethods<datafusion::dataframe::DataFrame> for
pyo3::class::impl_::PyClassImplCollector<datafusion::dataframe::DataFrame>>::py_method
s::METHODS::__wrap::{{closure}}
at ./src/dataframe.rs:49:1
12: pyo3::callback::handle_panic::{{closure}}
at
/home/houqp/.cargo/registry/src/github.com-1ecc6299db9ec823/pyo3-0.14.5/src/callback.rs:247:9
13: std::panicking::try::do_call
at
/rustc/1698e3cac54aa8691d4e9e207567672af8231cb6/library/std/src/panicking.rs:403:40
14: __rust_try
15: std::panicking::try
at
/rustc/1698e3cac54aa8691d4e9e207567672af8231cb6/library/std/src/panicking.rs:367:19
16: std::panic::catch_unwind
at
/rustc/1698e3cac54aa8691d4e9e207567672af8231cb6/library/std/src/panic.rs:129:14
17: pyo3::callback::handle_panic
at
/home/houqp/.cargo/registry/src/github.com-1ecc6299db9ec823/pyo3-0.14.5/src/callback.rs:245:24
18: datafusion::dataframe::<impl
pyo3::class::impl_::PyMethods<datafusion::dataframe::DataFrame> for
pyo3::class::impl_::PyClassImplCollector<datafusion::dataframe::DataFrame>>::py_method
s::METHODS::__wrap
at ./src/dataframe.rs:49:1
```
More specifically, this line:
```rust
let data_type = predicate.get_type(input.schema())?;
```
I think we can't just use the input schema here. For this failing query, the
input (the projection plan) only contains those two compound expressions. You
may want to use `plan.all_schemas()` instead.
Would be good if we can create a reproducible test case using Rust's
dataframe API first.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
