yihua opened a new pull request, #20560:
URL: https://github.com/apache/hudi/pull/20560

   ### Describe the issue this Pull Request addresses
   
   Closes #20559
   
   Part of #16053. **Stacked on #20551**, which merges first; until then this 
PR also shows that PR's commits.
   
   ### Summary and Changelog
   
   The code copied across the Spark 4.1, 4.2 and 4.3 modules now lives once, in 
`hudi-spark4-common/src/main/spark4.1-plus`. That root is not part of the 
`hudi-spark4-common` jar: each 4.1+ module adds it with build-helper and 
compiles it against its own Spark version. This matters because 
`hudi-spark4-common` is published from a single release pass, so forks inside 
that jar would be compiled against one Spark version only.
   
   - **Shared:** both grammars, the Avro and Parquet forks, the Catalyst and 
schema utils, the variant inferrer, and abstract bases for the extended SQL 
parser and AST builder.
   - **Still per version:** the adapters, row wrappers and `Analysis`, plus 
thin parser classes for the 4.3-only `ParserInterface` methods and the `LINES 
TERMINATED BY` check (4.3 uses `QueryParsingErrors`, 4.1 and 4.2 keep 
`validate`).
   - `hudi-spark4-common`'s scalastyle run also lints the shared root, since 
the version modules only lint `src/main/scala`.
   
   Spark 4.0 is unchanged. Its grammar and forks differ too much from 4.1 to 
share.
   
   ### Impact
   
   None for users. The class names of the moved forks change (for example 
`Spark43ParquetReader` becomes `Spark41PlusParquetReader`), but they are 
internal and the bundles pick them up by module. Adding a Spark 4.x version now 
needs only the per-version classes.
   
   ### Risk Level
   
   Low. Each moved file matches its 4.1, 4.2 and 4.3 originals apart from class 
names and comments. CI's JDK 17 test lanes only run 4.3, so 4.1 and 4.2 were 
checked locally: the moved code's unit tests, the SQL DDL/DML suites, and a COW 
and MOR bundle smoke test on real Spark jars passed on all three versions. 
Spark 4.0 was compiled only.
   
   ### Documentation Update
   
   None.
   
   ### Contributor's checklist
   
   - [x] Read through [contributor's 
guide](https://hudi.apache.org/contribute/how-to-contribute)
   - [x] Enough context is provided in the sections above
   - [x] Adequate tests were added if applicable
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to