yihua opened a new pull request, #20560: URL: https://github.com/apache/hudi/pull/20560
### Describe the issue this Pull Request addresses Closes #20559 Part of #16053. **Stacked on #20551**, which merges first; until then this PR also shows that PR's commits. ### Summary and Changelog The code copied across the Spark 4.1, 4.2 and 4.3 modules now lives once, in `hudi-spark4-common/src/main/spark4.1-plus`. That root is not part of the `hudi-spark4-common` jar: each 4.1+ module adds it with build-helper and compiles it against its own Spark version. This matters because `hudi-spark4-common` is published from a single release pass, so forks inside that jar would be compiled against one Spark version only. - **Shared:** both grammars, the Avro and Parquet forks, the Catalyst and schema utils, the variant inferrer, and abstract bases for the extended SQL parser and AST builder. - **Still per version:** the adapters, row wrappers and `Analysis`, plus thin parser classes for the 4.3-only `ParserInterface` methods and the `LINES TERMINATED BY` check (4.3 uses `QueryParsingErrors`, 4.1 and 4.2 keep `validate`). - `hudi-spark4-common`'s scalastyle run also lints the shared root, since the version modules only lint `src/main/scala`. Spark 4.0 is unchanged. Its grammar and forks differ too much from 4.1 to share. ### Impact None for users. The class names of the moved forks change (for example `Spark43ParquetReader` becomes `Spark41PlusParquetReader`), but they are internal and the bundles pick them up by module. Adding a Spark 4.x version now needs only the per-version classes. ### Risk Level Low. Each moved file matches its 4.1, 4.2 and 4.3 originals apart from class names and comments. CI's JDK 17 test lanes only run 4.3, so 4.1 and 4.2 were checked locally: the moved code's unit tests, the SQL DDL/DML suites, and a COW and MOR bundle smoke test on real Spark jars passed on all three versions. Spark 4.0 was compiled only. ### Documentation Update None. ### Contributor's checklist - [x] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute) - [x] Enough context is provided in the sections above - [x] Adequate tests were added if applicable -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
