[
https://issues.apache.org/jira/browse/ARROW-9778?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andy Grove resolved ARROW-9778.
-------------------------------
Fix Version/s: 2.0.0
Resolution: Fixed
Issue resolved by pull request 8005
[https://github.com/apache/arrow/pull/8005]
> [Rust] [DataFusion] Logical and physical schemas' nullability does not match
> in 8 out of 20 end-to-end tests
> ------------------------------------------------------------------------------------------------------------
>
> Key: ARROW-9778
> URL: https://issues.apache.org/jira/browse/ARROW-9778
> Project: Apache Arrow
> Issue Type: Bug
> Components: Rust, Rust - DataFusion
> Reporter: Jorge
> Assignee: Andy Grove
> Priority: Major
> Labels: pull-request-available
> Fix For: 2.0.0
>
> Time Spent: 50m
> Remaining Estimate: 0h
>
> In `tests/sql.rs`, if we re-write the ```execute``` function to test the end
> schemas, as
> ```
> /// Execute query and return result set as tab delimited string
> fn execute(ctx: &mut ExecutionContext, sql: &str) -> Vec<String> {
> let plan = ctx.create_logical_plan(&sql).unwrap();
> let plan = ctx.optimize(&plan).unwrap();
> let physical_plan = ctx.create_physical_plan(&plan).unwrap();
> let results = ctx.collect(physical_plan.as_ref()).unwrap();
> if results.len() > 0 {
> // results must match the logical schema
> assert_eq!(plan.schema().as_ref(), results[0].schema().as_ref());
> }
> result_str(&results)
> }
> ```
> we end up with 8 tests failing, which indicates that our physical and logical
> plans are not aligned. In all cases, the issue is nullability: our logical
> plan assumes nullability = true, while our physical plan may change the
> nullability field.
> If we do not plan to track nullability on the logical level, we could
> consider replacing Schema by a type that does not track nullability.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
