parthchandra opened a new pull request, #255: URL: https://github.com/apache/arrow-datafusion-comet/pull/255
## Which issue does this PR close? Closes #253 ## Rationale for this change Adds addition planning information to a Spark plan which allows us to track the reasons why a Spark plan was not fullay converted to Comet. ## What changes are included in this PR? This PR adds a `CometExplainInfo` structure that is returned by every step of the planning process. Eventually the Structure is attached to the Spark plan being returned by adding it as a Comet specific tag ## How are these changes tested? Additional unit test and tested with all TPCH and TPCDS queries with both Spark 3.4 and Spark 3.4 with the extended plan support. Also regenerated the `CometTPC*QueriesList` output for both TPCH and TPCDS run against a 1TB dataset. Note: Since the extended plan support does not exist before Spark 4, the output of `CometTPC*QueriesList` was updated with a build of Spark 3.4 with the feature backported. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
