pengzhiwei2018 edited a comment on pull request #2645: URL: https://github.com/apache/hudi/pull/2645#issuecomment-806465222
> Let me see how/if we can simplify the inputSchema vs writeSchema thing. > > I went over the PR now. LGTM at a high level. > Few questions though > > * I see we are introducing some antlr parsing and inject a custom parse for spark 2.x. Is this done for backwards compat with Spark 2 and will be eventually removed? > * Do we reuse the MERGE/DELETE keywords from Spark 3? Is Spark 3 and Spark 2 syntax different. Can you comment on how we are approaching all this. > * Have you done any production testing of this PR? > > cc @kwondw could you also please chime in. We would like to land something basic and iterate and get this out for 0.9.0 next month. Thanks for you review @vinothchandar ! > I see we are introducing some antlr parsing and inject a custom parse for spark 2.x. Is this done for backwards compat with Spark 2 and will be eventually removed? Yes, It is for backwards for Spark2 and will be eventually removed for spark3 if there are no other syntax extend for the spark3. > Do we reuse the MERGE/DELETE keywords from Spark 3? Is Spark 3 and Spark 2 syntax different. Can you comment on how we are approaching all this. Yes ,I reused the extended syntax( MERGE&DELETE) from spark 3. So they are the same between spark2 and spark3 in the syntax. For spark3, spark can recognize the MERGE/DELTE syntax and parser it to LogicalPlan. For spark2, our extended sql parser will also parser it to the some LogicalPlan. After the parser, the LogicalPlan will goes to the same Rules（In `HoodieAnalysis`) to resolve and rewrite to Hoodie Command. Hoodie Command will translate the logical plan to the hoodie api call.The Hoodie Command shared between spark2 & spark 3. So except the sql parser for spark2, other parts can share between spark2 & spark3. > Have you done any production testing of this PR? Yes, I have test it in Aliyun's EMR cluster. And more test case will be done this week. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
