[
https://issues.apache.org/jira/browse/SPARK-58537?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jungtaek Lim updated SPARK-58537:
---------------------------------
Affects Version/s: 4.4.0
(was: 4.3.0)
> Scheduled Maven workflows on branch-4.x build master instead of the branch
> --------------------------------------------------------------------------
>
> Key: SPARK-58537
> URL: https://issues.apache.org/jira/browse/SPARK-58537
> Project: Spark
> Issue Type: Bug
> Components: Project Infra
> Affects Versions: 4.4.0
> Reporter: Jungtaek Lim
> Priority: Major
>
> The scheduled Maven workflows on {{branch-4.x}} are named for {{branch-4.x}}
> but actually build {{master}}, because they invoke the reusable
> {{maven_test.yml}} without passing the {{branch}} input, and that workflow's
> default is {{master}}.
> h3. The bug
> {{.github/workflows/build_maven.yml}} on {{branch-4.x}} is titled:
> {code}
> name: "Build / Maven (branch-4.x, Scala 2.13, Hadoop 3, JDK 17)"
> {code}
> but its job body supplies no inputs:
> {code}
> jobs:
> run-build:
> permissions:
> packages: write
> name: Run
> uses: ./.github/workflows/maven_test.yml
> if: github.repository == 'apache/spark'
> {code}
> So {{maven_test.yml}} falls back to its declared default:
> {code}
> branch:
> description: Branch to run the build against
> required: false
> type: string
> default: master
> {code}
> and it then checks out {{ref: ${{ inputs.branch }}}} — i.e. {{master}}.
> h3. Affected workflows
> Only the three consumers of {{maven_test.yml}} on {{branch-4.x}}:
> * {{build_maven.yml}}
> * {{build_maven_java21.yml}}
> * {{build_maven_java25.yml}}
> The six SBT/Python workflows on the branch ({{build_java17/21/25}},
> {{build_non_ansi}}, {{build_python_3.11/3.14}}) also omit {{branch:}}, but
> they use {{build_and_test.yml}}, whose default on this branch is already
> {{branch-4.x}}. They are correct and need no change.
> The cleanest fix is therefore to change the {{branch}} default in
> {{maven_test.yml}} on {{branch-4.x}} from {{master}} to {{branch-4.x}},
> matching {{build_and_test.yml}} on the same branch. Alternatively, pass
> {{with: branch: branch-4.x}} explicitly from each of the three callers.
> h3. Evidence
> The most recent green "Build / Maven (branch-4.x, ...)" run (30840288268,
> 2026-08-03) produced {{master}}'s artifacts. Its Precompile log contains:
> {code}
> spark-connect-client-jdbc_2.13-5.0.0-SNAPSHOT.jar, unused-1.0.0.jar define 1
> overlapping resource
> {code}
> At the time, {{branch-4.x}} was at {{4.3.0-SNAPSHOT}} and {{master}} at
> {{5.0.0-SNAPSHOT}}, so those coordinates show {{master}} had been checked out.
> For contrast, the "Build (branch-4.x, ...)" JDK 17 run (30818354111), which
> goes through {{build_and_test.yml}}, logs only {{4.3.0-SNAPSHOT}} — correctly
> building the branch.
> h3. Impact
> {{branch-4.x}} has had no scheduled Maven coverage, and the green checks from
> these three workflows are misleading.
> This masked SPARK-57897, where {{branch-4.x}} could not load its Maven
> project model at all — {{mvn}} failed with
> {{'dependencies.dependency.version' for io.grpc:grpc-api:jar is missing}}
> before any test could run — while the branch's Maven workflow kept reporting
> success. The breakage was found only by building the branch locally.
> Note the release branches are handled correctly elsewhere: the
> {{build_branch42_*.yml}} / {{build_branch4x_*.yml}} workflows on {{master}}
> do pass {{branch:}} explicitly. The bug is specific to the copies of these
> workflows that live on {{branch-4.x}} itself, and would be inherited by any
> newly cut branch that copies them (e.g. {{branch-4.3}}).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
