alamb commented on issue #617: URL: https://github.com/apache/arrow-datafusion/issues/617#issuecomment-868611882
Here is a reproducer (in projection_push_down.rs) -- it can also be found on https://github.com/alamb/arrow-datafusion/tree/alamb/repro_projection_pruning ```rust #[test] fn table_scan_projected_schema_non_qualified_relation() -> Result<()> { let table_scan = test_table_scan()?; let input_schema = table_scan.schema(); assert_eq!(3, input_schema.fields().len()); assert_fields_eq(&table_scan, vec!["a", "b", "c"]); // Build the LogicalPlan directly (don't use PlanBuilder), so // that the Column references are unqualified (e.g. their // relation is `None`). PlanBuilder resolves the expressions let expr = vec![col("a"), col("b")]; let projected_fields = exprlist_to_fields(&expr, input_schema).unwrap(); let projected_schema = DFSchema::new(projected_fields).unwrap(); let plan = LogicalPlan::Projection { expr, input: Arc::new(table_scan), schema: Arc::new(projected_schema), }; assert_fields_eq(&plan, vec!["a", "b"]); let expected = "Projection: #a, #b\ \n TableScan: test projection=Some([0, 1])"; assert_optimized_plan_eq(&plan, expected); Ok(()) } ``` It fails in the following way: ``` ---- optimizer::projection_push_down::tests::table_scan_projected_schema_non_qualified_relation stdout ---- thread 'optimizer::projection_push_down::tests::table_scan_projected_schema_non_qualified_relation' panicked at 'assertion failed: `(left == right)` left: `"Projection: #a, #b\n TableScan: test projection=Some([0])"`, right: `"Projection: #test.a, #test.b\n TableScan: test projection=Some([0, 1])"`', datafusion/src/optimizer/projection_push_down.rs:745:9 note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace ``` The problem is that the `TableScan` is only scanning one column, even though the `Projection` is using two columns. I will put up a PR with code to fix the problem -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
