deepakpanda93 commented on issue #15302: URL: https://github.com/apache/hudi/issues/15302#issuecomment-5326045298
This issue was reviewed as part of the JIRA-migrated backlog triage and is being closed as already fixed. The ask was to stop using classes copied from the Spark 3.2 module and port them to the Spark 3.3 implementations, naming `Spark33NestedSchemaPruning`. That class no longer exists on master, and neither does the module it came from. Verified by listing `hudi-spark-datasource/` at four release tags: | tag | Spark 2.x / 3.2.x modules | |---|---| | release-0.12.0 | `hudi-spark2-common`, `hudi-spark2`, `hudi-spark3.2.x` | | release-0.14.0 | the above plus `hudi-spark3.2plus-common` | | release-1.0.0 | none | | release-1.2.0 | none | They were removed by `ef8a7ad4f17a` (*"[HUDI-7918] Remove support of Spark 3.0, 3.1, and 3.2"*, PR #11692). I checked containment explicitly - comparing that commit against `release-1.0.0` and `release-1.2.0` reports `ahead` for both. What replaced them is what this ticket wanted: per-version implementations (`Spark33ParquetReader.scala`, `Spark34ParquetReader.scala`, `Spark35ParquetReader.scala`, and the 4.0/4.1/4.2 equivalents) using `NestedSchemaPruning` natively. Closing as already fixed in **1.0.0**. Note no commit mentions HUDI-4467 - this was resolved as a side effect of the Spark version pruning. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
