rangareddy commented on issue #17008: URL: https://github.com/apache/hudi/issues/17008#issuecomment-5391443617
This issue was reviewed as part of the JIRA-migrated backlog triage (HUDI-9404). **Findings: confirmed not done - and the duplication has grown since this was filed.** The description lists four classes (Spark 3.3, 3.4, 3.5, 4.0). Today `hudi-spark-datasource/hudi-spark/src/main/scala/org/apache/spark/sql/hudi/analysis/HoodieAnalysis.scala` resolves **five** by fully-qualified name at `:61-69`: ```scala "org.apache.spark.sql.hudi.analysis.HoodieSpark42DataSourceV2ToV1Fallback" // :61 "org.apache.spark.sql.hudi.analysis.HoodieSpark41DataSourceV2ToV1Fallback" // :63 "org.apache.spark.sql.hudi.analysis.HoodieSpark40DataSourceV2ToV1Fallback" // :65 "org.apache.spark.sql.hudi.analysis.HoodieSpark35DataSourceV2ToV1Fallback" // :67 "org.apache.spark.sql.hudi.analysis.HoodieSpark34DataSourceV2ToV1Fallback" // :69 ``` Spark 4.1 and 4.2 variants were added; none were unified. Context that matters for the "if there is no difference, unify them" premise: the supported-version matrix is currently widening, not narrowing. Spark 3.3 support was **dropped** by PR #18786 (`facb517ef957`, 2026-05-21) and then **reintroduced** by PR #18956 (`7c934f10f89b`, 2026-06-13). So a unification now has to hold across 3.4 through 4.2, and any future line gets added to whatever shape it lands in. That argues for doing this sooner rather than later - each new Spark line makes it more expensive. Note the classes are resolved reflectively by FQN string, so a unified class would also let that lookup collapse. Keeping this open. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
