This is an automated email from the ASF dual-hosted git repository.
dongjoon-hyun pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new 2b418b00a871 [SPARK-57722][INFRA] Merge `binding-policy` GHA job into
the `lint` job
2b418b00a871 is described below
commit 2b418b00a871ce0cf2dcb422348752dcaab9e36f
Author: Dongjoon Hyun <[email protected]>
AuthorDate: Fri Jun 26 11:09:42 2026 -0700
[SPARK-57722][INFRA] Merge `binding-policy` GHA job into the `lint` job
### What changes were proposed in this pull request?
This PR moves the standalone `binding-policy` job into the `lint` job as a
`Config binding policy exceptions check` step, right after the `Scala linter`
step.
### Why are the changes needed?
To save the GitHub Actions resources by removing additional job triggering.
The `binding-policy` job duplicated the `lint` job's checkout and branch-sync
setup just to run a one-line `git diff` check. The `lint` job already sets
`$APACHE_SPARK_REF` via the same sync, so the new step reuses it.
**BEFORE**
<img width="627" height="314" alt="Screenshot 2026-06-26 at 10 44 43"
src="https://github.com/user-attachments/assets/70c8bcb3-3764-42e4-a892-6c87e0e7d085";
/>
**AFTER (This PR)**
<img width="618" height="273" alt="Screenshot 2026-06-26 at 10 45 33"
src="https://github.com/user-attachments/assets/289af4bb-8c65-4137-9c0b-818d619d9483";
/>
### Does this PR introduce _any_ user-facing change?
No. CI-only change.
### How was this patch tested?
Pass the CIs.
### Was this patch authored or co-authored using generative AI tooling?
Generated-by: Claude Opus 4.8
Closes #56820 from dongjoon-hyun/SPARK-57722.
Authored-by: Dongjoon Hyun <[email protected]>
Signed-off-by: Dongjoon Hyun <[email protected]>
---
.github/workflows/build_and_test.yml | 76 +++++++++++++-----------------------
1 file changed, 27 insertions(+), 49 deletions(-)
diff --git a/.github/workflows/build_and_test.yml
b/.github/workflows/build_and_test.yml
index 32086f6fd722..5decebdae876 100644
--- a/.github/workflows/build_and_test.yml
+++ b/.github/workflows/build_and_test.yml
@@ -1035,55 +1035,6 @@ jobs:
if: inputs.branch != 'branch-3.5'
run: ./dev/check-protos.py
- # The config binding policy exceptions file lists pre-existing configs that
were created before
- # the binding policy was introduced. It must never grow: new configs must
declare a binding
- # policy via .withBindingPolicy() instead of being added to the exceptions
file. This job fails
- # if a PR adds entries to the file. It only runs on forks because PR builds
run there, and the
- # diff against apache/spark master is exactly the PR's change.
- binding-policy:
- needs: [precondition]
- if: >-
- (!cancelled()) &&
- github.repository != 'apache/spark' &&
- fromJson(needs.precondition.outputs.required).lint == 'true'
- name: Config binding policy exceptions check
- runs-on: ubuntu-latest
- timeout-minutes: 10
- steps:
- - name: Checkout Spark repository
- uses: actions/checkout@v6
- with:
- fetch-depth: 0
- repository: apache/spark
- ref: ${{ needs.precondition.outputs.head_sha }}
- - name: Sync the current branch with the latest in Apache Spark
- run: |
- echo "APACHE_SPARK_REF=$(git rev-parse HEAD)" >> $GITHUB_ENV
- git fetch https://github.com/$GITHUB_REPOSITORY.git
${GITHUB_REF#refs/heads/}
- git -c user.name='Apache Spark Test Account' -c
user.email='[email protected]' merge --no-commit --progress --squash
FETCH_HEAD
- git -c user.name='Apache Spark Test Account' -c
user.email='[email protected]' commit -m "Merged commit" --allow-empty
- - name: Check no new entries in the binding policy exceptions file
- run: |
-
EXCEPTIONS_FILE="sql/hive/src/test/resources/conf/binding-policy-exceptions/configs-without-binding-policy-exceptions"
- if [ ! -f "$EXCEPTIONS_FILE" ]; then
- echo "$EXCEPTIONS_FILE does not exist on this branch, skipping."
- exit 0
- fi
- ADDED_ENTRIES=$(git diff "$APACHE_SPARK_REF" HEAD --
"$EXCEPTIONS_FILE" | grep '^+' | grep -v '^+++' || true)
- if [ -n "$ADDED_ENTRIES" ]; then
- echo "ERROR: This PR adds the following entries to $EXCEPTIONS_FILE:"
- echo "$ADDED_ENTRIES"
- echo ""
- echo "This file is a frozen list of configs that predate the binding
policy, and new"
- echo "entries must not be added to it. Instead, declare a binding
policy when building"
- echo "the config entry, e.g."
- echo "
.withBindingPolicy(ConfigBindingPolicy.SESSION/PERSISTED/NOT_APPLICABLE)"
- echo "See the scaladoc of
org.apache.spark.internal.config.ConfigBindingPolicy for the"
- echo "semantics of each policy and how to choose one."
- exit 1
- fi
- echo "No new entries added to the binding policy exceptions file."
-
# Static analysis
lint:
needs: [precondition, infra-image]
@@ -1156,6 +1107,33 @@ jobs:
run: ./dev/mima
- name: Scala linter
run: ./dev/lint-scala
+ # The config binding policy exceptions file lists pre-existing configs
that were created before
+ # the binding policy was introduced. It must never grow: new configs must
declare a binding
+ # policy via .withBindingPolicy() instead of being added to the exceptions
file. This step fails
+ # if a PR adds entries to the file. It only runs on forks because PR
builds run there, and the
+ # diff against apache/spark master is exactly the PR's change.
+ - name: Config binding policy exceptions check
+ if: github.repository != 'apache/spark'
+ run: |
+
EXCEPTIONS_FILE="sql/hive/src/test/resources/conf/binding-policy-exceptions/configs-without-binding-policy-exceptions"
+ if [ ! -f "$EXCEPTIONS_FILE" ]; then
+ echo "$EXCEPTIONS_FILE does not exist on this branch, skipping."
+ exit 0
+ fi
+ ADDED_ENTRIES=$(git diff "$APACHE_SPARK_REF" HEAD --
"$EXCEPTIONS_FILE" | grep '^+' | grep -v '^+++' || true)
+ if [ -n "$ADDED_ENTRIES" ]; then
+ echo "ERROR: This PR adds the following entries to $EXCEPTIONS_FILE:"
+ echo "$ADDED_ENTRIES"
+ echo ""
+ echo "This file is a frozen list of configs that predate the binding
policy, and new"
+ echo "entries must not be added to it. Instead, declare a binding
policy when building"
+ echo "the config entry, e.g."
+ echo "
.withBindingPolicy(ConfigBindingPolicy.SESSION/PERSISTED/NOT_APPLICABLE)"
+ echo "See the scaladoc of
org.apache.spark.internal.config.ConfigBindingPolicy for the"
+ echo "semantics of each policy and how to choose one."
+ exit 1
+ fi
+ echo "No new entries added to the binding policy exceptions file."
- name: Scala structured logging check
if: hashFiles('dev/structured_logging_style.py') != ''
shell: 'script -q -e -c "bash {0}"'
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
