[ https://issues.apache.org/jira/browse/ARROW-9809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Andy Grove resolved ARROW-9809. ------------------------------- Fix Version/s: 2.0.0 Resolution: Fixed Issue resolved by pull request 8024 [https://github.com/apache/arrow/pull/8024] > [Rust] [DataFusion] logical schema = physical schema is not true > ---------------------------------------------------------------- > > Key: ARROW-9809 > URL: https://issues.apache.org/jira/browse/ARROW-9809 > Project: Apache Arrow > Issue Type: Bug > Components: Rust, Rust - DataFusion > Reporter: Jorge > Priority: Major > Labels: pull-request-available > Fix For: 2.0.0 > > Time Spent: 1h > Remaining Estimate: 0h > > In tests/sql.rs, we test that the physical and the optimized schema must > match. However, this is not necessarily true for all our queries. An example: > {code:java} > #[test] > fn csv_query_sum_cast() { > let mut ctx = ExecutionContext::new(); > register_aggregate_csv_by_sql(&mut ctx); > // c8 = i32; c9 = i64 > let sql = "SELECT c8 + c9 FROM aggregate_test_100"; > // check that the physical and logical schemas are equal > execute(&mut ctx, sql); > } > {code} > The physical expression (and schema) of this operation, after optimization, > is {{CAST(c8 as Int64) Plus c9}} (this test fails). > AFAIK, the invariant of the optimizer is that the output types and > nullability are the same. > Also, note that the reason the optimized logical schema equals the logical > schema is that our type coercer does not change the output names of the > schema, even though it re-writes logical expressions. I.e. after the > optimization, `.to_field()` of an expression may no longer match the field > name nor type in the Plan's schema. IMO this is currently by (implicit?) > design, as we do not want our logical schema's column names to change during > optimizations, or all column references may point to non-existent columns. > This is something that brought up on the mailing list about polymorphism. -- This message was sent by Atlassian Jira (v8.3.4#803005)