rdblue edited a comment on issue #28026: [SPARK-31257][SQL] Unify create table syntax (WIP) URL: https://github.com/apache/spark/pull/28026#issuecomment-606969684 @HeartSaVioR, here's some additional info to give you context. When we convert plans to v2, we alter the parser to produce a `Statement` plan that contains what was parsed, without modification. The reason is that the v1 plans put a lot of logic about what was allowed in the parser itself, which required the parser to be broken across spark-catalyst and spark-sql modules and was difficult to maintain. Now, logic about what is allowed is in the analyzer. Statement plans are converted to either the exact same v1 logical plan as before, or a v2 plan. To validate that we are producing the exact same logical plans, we move tests from the spark-sql `DDLParserSuite` (only runs the parser) into `PlanResolutionSuite` that runs the parser and conversion rules. These tests should be nearly identical to the original and are as close to a copy & paste as possible to be easy to verify. I just completed this step, which is why tests should be passing. In addition, we add tests to spark-catalyst's `DDLParserSuite` that test SQL to `Statement` conversion. This is where we will also add new tests for the unified behavior, like rejecting `PARTITION BY` clauses with both expressions and column declarations. These are the last tests that I need to add for this PR and I'll work on them tomorrow.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
