[
https://issues.apache.org/jira/browse/SPARK-58537?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jungtaek Lim resolved SPARK-58537.
----------------------------------
Fix Version/s: 4.4.0
Resolution: Fixed
Issue resolved by pull request 57739
[https://github.com/apache/spark/pull/57739]
> Scheduled Maven workflows on branch-4.x build master instead of the branch
> --------------------------------------------------------------------------
>
> Key: SPARK-58537
> URL: https://issues.apache.org/jira/browse/SPARK-58537
> Project: Spark
> Issue Type: Bug
> Components: Project Infra
> Affects Versions: 4.4.0
> Reporter: Jungtaek Lim
> Assignee: Jungtaek Lim
> Priority: Major
> Labels: pull-request-available
> Fix For: 4.4.0
>
>
> The scheduled Maven workflows on {{branch-4.x}} are named for {{branch-4.x}}
> but actually build {{master}}, because they invoke the reusable
> {{maven_test.yml}} without passing the {{branch}} input, and that workflow's
> default is {{master}}.
> h3. The bug
> {{.github/workflows/build_maven.yml}} on {{branch-4.x}} is titled:
> {code}
> name: "Build / Maven (branch-4.x, Scala 2.13, Hadoop 3, JDK 17)"
> {code}
> but its job body supplies no inputs:
> {code}
> jobs:
> run-build:
> permissions:
> packages: write
> name: Run
> uses: ./.github/workflows/maven_test.yml
> if: github.repository == 'apache/spark'
> {code}
> So {{maven_test.yml}} falls back to its declared default:
> {code}
> branch:
> description: Branch to run the build against
> required: false
> type: string
> default: master
> {code}
> and it then checks out {{ref: ${{ inputs.branch }}}} — i.e. {{master}}.
> h3. Affected workflows
> The five consumers of {{maven_test.yml}} on {{branch-4.x}} that pass no
> {{branch:}} input:
> * {{build_maven.yml}}
> * {{build_maven_java21.yml}}
> * {{build_maven_java25.yml}}
> * {{build_maven_java21_arm.yml}}
> * {{build_maven_java21_macos26.yml}}
> The last two are additionally titled {{"Build / Maven (master, ...)"}}, which
> looks like deliberate master-targeting but is a stale artifact of being
> copied from {{master}} -- the same two names are stale on {{branch-4.2}},
> where the {{maven_test.yml}} default is correctly {{branch-4.2}}. Their
> titles should be corrected along with the default.
> Callers that target another branch pass {{branch:}} explicitly (e.g.
> {{build_branch41_maven.yml}} passes {{branch-4.1}}) and are unaffected.
> The SBT/Python workflows on the branch ({{build_java17/21/25}},
> {{build_non_ansi}}, {{build_python_3.11/3.14}}) also omit {{branch:}}, but
> they use {{build_and_test.yml}}, whose default on this branch is already
> {{branch-4.x}}. They are correct and need no change.
> The fix is therefore to change the {{branch}} default in {{maven_test.yml}}
> on {{branch-4.x}} from {{master}} to {{branch-4.x}}, matching
> {{build_and_test.yml}} on the same branch and both reusable workflows on
> {{branch-4.2}} / {{branch-4.1}} (each defaulting to their own branch, which
> is why their callers correctly pass no input).
> The same bug exists on {{branch-4.3}}, which inherited these files when it
> was cut; it is fixed there as part of SPARK-58538. {{master}} is unaffected:
> {{default: master}} is correct there.
> h3. Evidence
> The most recent green "Build / Maven (branch-4.x, ...)" run (30840288268,
> 2026-08-03) produced {{master}}'s artifacts. Its Precompile log contains:
> {code}
> spark-connect-client-jdbc_2.13-5.0.0-SNAPSHOT.jar, unused-1.0.0.jar define 1
> overlapping resource
> {code}
> At the time, {{branch-4.x}} was at {{4.3.0-SNAPSHOT}} and {{master}} at
> {{5.0.0-SNAPSHOT}}, so those coordinates show {{master}} had been checked out.
> For contrast, the "Build (branch-4.x, ...)" JDK 17 run (30818354111), which
> goes through {{build_and_test.yml}}, logs only {{4.3.0-SNAPSHOT}} — correctly
> building the branch.
> h3. Impact
> {{branch-4.x}} has had no scheduled Maven coverage, and the green checks from
> these three workflows are misleading.
> This masked SPARK-57897, where {{branch-4.x}} could not load its Maven
> project model at all — {{mvn}} failed with
> {{'dependencies.dependency.version' for io.grpc:grpc-api:jar is missing}}
> before any test could run — while the branch's Maven workflow kept reporting
> success. The breakage was found only by building the branch locally.
> Note the release branches are handled correctly elsewhere: the
> {{build_branch42_*.yml}} / {{build_branch4x_*.yml}} workflows on {{master}}
> do pass {{branch:}} explicitly. The bug is specific to the copies of these
> workflows that live on {{branch-4.x}} itself, and would be inherited by any
> newly cut branch that copies them (e.g. {{branch-4.3}}).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
