alamb opened a new pull request #337: URL: https://github.com/apache/arrow-datafusion/pull/337
# Which issue does this PR close? https://github.com/apache/arrow-datafusion/issues/333 # Rationale for this change EXPLAIN output for physical plans is currently close to useless (in my opinion). # What changes are included in this PR? * Visitor pattern to traverse `ExecutionPlans` * New `displayable` function for displaying `ExecutionPlans` reasonably * Documentation and test Note I will hope to use the same basic infrastructure to implement graphviz plans https://github.com/apache/arrow-datafusion/issues/219 # Example new format ``` > explain verbose select * from foo where a < 4; +-----------------------------------------+------------------------------------------------------------------------+ | plan_type | plan | +-----------------------------------------+------------------------------------------------------------------------+ | logical_plan | Projection: #a, #b, #c | | | Filter: #a Lt Int64(4) | | | TableScan: foo projection=None | | logical_plan after projection_push_down | Projection: #a, #b, #c | | | Filter: #a Lt Int64(4) | | | TableScan: foo projection=Some([0, 1, 2]) | | logical_plan after projection_push_down | Projection: #a, #b, #c | | | Filter: #a Lt Int64(4) | | | TableScan: foo projection=Some([0, 1, 2]) | | physical_plan | ProjectionExec: expr=[a, b, c] | | | FilterExec: CAST(a AS Int64) < 4 | | | CsvExec: source=Path(/tmp/foo.csv: [/tmp/foo.csv]), has_header=false | +-----------------------------------------+------------------------------------------------------------------------+ ``` # Are there any user-facing changes? Yes: output format for `EXPLAIN VERBOSE` has changed New Output: #API changes All changes are backwards compatible # Example old format ``` > explain verbose select * from foo where a < 4; +-----------------------------------------+-------------------------------------------------+ | plan_type | plan | +-----------------------------------------+-------------------------------------------------+ | logical_plan | Projection: #a, #b, #c | | | Filter: #a Lt Int64(4) | | | TableScan: foo projection=None | | logical_plan after projection_push_down | Projection: #a, #b, #c | | | Filter: #a Lt Int64(4) | | | TableScan: foo projection=Some([0, 1, 2]) | | logical_plan after projection_push_down | Projection: #a, #b, #c | | | Filter: #a Lt Int64(4) | | | TableScan: foo projection=Some([0, 1, 2]) | | physical_plan | ProjectionExec { | | | expr: [ | | | ( | | | Column { | | | name: "a", | | | }, | | | "a", | | | ), | | | ( | | | Column { | | | name: "b", | | | }, | | | "b", | | | ), | | | ( | | | Column { | | | name: "c", | | | }, | | | "c", | | | ), | | | ], | | | schema: Schema { | | | fields: [ | | | Field { | | | name: "a", | | | data_type: Int32, | | | nullable: false, | | | dict_id: 0, | | | dict_is_ordered: false, | | | metadata: None, | | | }, | | | Field { | | | name: "b", | | | data_type: Int32, | | | nullable: false, | | | dict_id: 0, | | | dict_is_ordered: false, | | | metadata: None, | | | }, | | | Field { | | | name: "c", | | | data_type: Int32, | | | nullable: false, | | | dict_id: 0, | | | dict_is_ordered: false, | | | metadata: None, | | | }, | | | ], | | | metadata: {}, | | | }, | | | input: FilterExec { | | | predicate: BinaryExpr { | | | left: TryCastExpr { | | | expr: Column { | | | name: "a", | | | }, | | | cast_type: Int64, | | | }, | | | op: Lt, | | | right: Literal { | | | value: Int64(4), | | | }, | | | }, | | | input: CsvExec { | | | source: PartitionedFiles { | | | path: "/tmp/foo.csv", | | | filenames: [ | | | "/tmp/foo.csv", | | | ], | | | }, | | | schema: Schema { | | | fields: [ | | | Field { | | | name: "a", | | | data_type: Int32, | | | nullable: false, | | | dict_id: 0, | | | dict_is_ordered: false, | | | metadata: None, | | | }, | | | Field { | | | name: "b", | | | data_type: Int32, | | | nullable: false, | | | dict_id: 0, | | | dict_is_ordered: false, | | | metadata: None, | | | }, | | | Field { | | | name: "c", | | | data_type: Int32, | | | nullable: false, | | | dict_id: 0, | | | dict_is_ordered: false, | | | metadata: None, | | | }, | | | ], | | | metadata: {}, | | | }, | | | has_header: false, | | | delimiter: Some( | | | 44, | | | ), | | | file_extension: ".csv", | | | projection: Some( | | | [ | | | 0, | | | 1, | | | 2, | | | ], | | | ), | | | projected_schema: Schema { | | | fields: [ | | | Field { | | | name: "a", | | | data_type: Int32, | | | nullable: false, | | | dict_id: 0, | | | dict_is_ordered: false, | | | metadata: None, | | | }, | | | Field { | | | name: "b", | | | data_type: Int32, | | | nullable: false, | | | dict_id: 0, | | | dict_is_ordered: false, | | | metadata: None, | | | }, | | | Field { | | | name: "c", | | | data_type: Int32, | | | nullable: false, | | | dict_id: 0, | | | dict_is_ordered: false, | | | metadata: None, | | | }, | | | ], | | | metadata: {}, | | | }, | | | batch_size: 8192, | | | limit: None, | | | }, | | | }, | | | } | +-----------------------------------------+-------------------------------------------------+ ``` </details> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
