[
https://issues.apache.org/jira/browse/HUDI-3679?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Raymond Xu updated HUDI-3679:
-----------------------------
Priority: Blocker (was: Major)
> Spark Compatibility Model
> -------------------------
>
> Key: HUDI-3679
> URL: https://issues.apache.org/jira/browse/HUDI-3679
> Project: Apache Hudi
> Issue Type: Epic
> Reporter: Alexey Kudinkin
> Assignee: Alexey Kudinkin
> Priority: Blocker
> Fix For: 0.11.0
>
>
> Currently we have inconsistent models of aliasing Spark version in the bundle
> name ("spark3", "spark3.1.x", "spark3.2.1", etc).
> To avoid confusion in the community, and make things consistent following
> compatibility pledge is proposed:
> +Hudi bundles will stay compatible with _all patch versions_ of Spark w/in
> single minor version branch (ie Spark 3.1.x, 3.2.x, etc)+
>
> Such compatibility model will be correspondingly reflected in the bundle
> names: corresponding spark minor version branch will be aliased in the bundle
> name to signal its compatibility with Spark versions w/in such branch
> ("spark3.1" for Spark 3.1.x, etc)
> To transition to the new model following steps are proposed:
> # Add new artifacts carrying Spark minor versions in the bundle names (for
> ex, "spark3.2", "spark3.1", "spark2.4") in 0.11
> # Deprecate single and triple digit bundles ("spark3", "spark2",
> "spark3.2.1") in favor of double-digit in 0.11
> # Discontinue deprecated bundles (spark3, spark2, etc) from 0.12
>
> As such following bundles will be released in 0.11
> * "spark2.4" (Scala 2.11/2.12)
> * "spark3.1"
> * "spark3.2"
> * "spark3" (this is the same as "spark3.2")
> * "spark2" (Scala 2.11/2.12, this is the same as "spark2.4")
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
