This is an automated email from the ASF dual-hosted git repository.
dongjoon-hyun pushed a commit to branch branch-4.1
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-4.1 by this push:
new 8c5975984f43 [SPARK-56180][4.1][INFRA] `build/mvn` prefer to download
tgz from Google mirror of Maven Central
8c5975984f43 is described below
commit 8c5975984f4341c951e4fbb28da9bee7c100600b
Author: Cheng Pan <[email protected]>
AuthorDate: Fri Nov 6 14:13:28 2026 -0800
[SPARK-56180][4.1][INFRA] `build/mvn` prefer to download tgz from Google
mirror of Maven Central
### What changes were proposed in this pull request?
Migrate `build/mvn` default tgz downloading from ASF
[`closer.lua`](https://infra.apache.org/release-download-pages.html#custom) to
Maven Central ( `DEFAULT_ARTIFACT_REPOSITORY`, default points to Google mirror).
Also, improve other scripts that use `DEFAULT_ARTIFACT_REPOSITORY`, now it
works well with and without a trailing slash.
Change `dev/check-license` to use Google mirror instead of
https://repo1.maven.org/maven2 when `DEFAULT_ARTIFACT_REPOSITORY` is absent.
### Why are the changes needed?
ASF services have not been very stable recently, this change reduces load
on the ASF download site.
`closer.lua` routes the downloading to either the ASF dlcdn site (powered
by CDN) or archive site (slow and unstable), while Maven Central preserves [all
historical
versions](https://repo1.maven.org/maven2/org/apache/maven/apache-maven/), so
this also benefits older branches that use old Maven versions (if we backport
it to older branches).
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
Manually tested - fresh clone spark repo, and run `build/mvn`, `build/sbt`,
`dev/check-license`.
### Was this patch authored or co-authored using generative AI tooling?
No.
Closes #60187 from dongjoon-hyun/SPARK-56180.
Lead-authored-by: Cheng Pan <[email protected]>
Co-authored-by: Cheng Pan <[email protected]>
Signed-off-by: Dongjoon Hyun <[email protected]>
---
build/mvn | 13 +++++++------
build/sbt-launch-lib.bash | 3 ++-
dev/check-license | 3 ++-
3 files changed, 11 insertions(+), 8 deletions(-)
diff --git a/build/mvn b/build/mvn
index 0e03a3c7b14e..61dde324e460 100755
--- a/build/mvn
+++ b/build/mvn
@@ -125,21 +125,22 @@ install_mvn() {
fi
if [ $(version $MVN_DETECTED_VERSION) -ne $(version $MVN_VERSION) ]; then
local MVN_TARBALL="apache-maven-${MVN_VERSION}-bin.tar.gz"
- local FILE_PATH="maven/maven-3/${MVN_VERSION}/binaries/${MVN_TARBALL}"
- local
APACHE_MIRROR=${APACHE_MIRROR:-'https://www.apache.org/dyn/closer.lua'}
- local MIRROR_URL_QUERY="?action=download"
+ local
FILE_PATH="org/apache/maven/apache-maven/${MVN_VERSION}/${MVN_TARBALL}"
+ local
MIRROR_BASE=${DEFAULT_ARTIFACT_REPOSITORY:-https://maven-central.storage-download.googleapis.com/maven2/}
+ local MIRROR_URL_QUERY=""
if [ $(command -v curl) ]; then
- if ! curl -L --output /dev/null --silent --head --fail
"${APACHE_MIRROR}/${FILE_PATH}${MIRROR_URL_QUERY}" ; then
+ if ! curl -L --output /dev/null --silent --head --fail
"${MIRROR_BASE%/}/${FILE_PATH}${MIRROR_URL_QUERY}" ; then
# Fall back to archive.apache.org for older Maven
echo "Falling back to archive.apache.org to download Maven" 1>&2
- APACHE_MIRROR="https://archive.apache.org/dist"
+ FILE_PATH="maven/maven-3/${MVN_VERSION}/binaries/${MVN_TARBALL}"
+ MIRROR_BASE="https://archive.apache.org/dist"
MIRROR_URL_QUERY=""
fi
fi
install_app \
- "${APACHE_MIRROR}" \
+ "${MIRROR_BASE%/}" \
"${FILE_PATH}" \
"${MIRROR_URL_QUERY}" \
"sha512" \
diff --git a/build/sbt-launch-lib.bash b/build/sbt-launch-lib.bash
index 01ba6b929f92..84934379f8b9 100755
--- a/build/sbt-launch-lib.bash
+++ b/build/sbt-launch-lib.bash
@@ -43,7 +43,8 @@ acquire_sbt_jar () {
# artifacts from internal repos only.
# Ex:
# DEFAULT_ARTIFACT_REPOSITORY=https://artifacts.internal.com/libs-release/
-
URL1=${DEFAULT_ARTIFACT_REPOSITORY:-https://repo1.maven.org/maven2/}org/scala-sbt/sbt-launch/${SBT_VERSION}/sbt-launch-${SBT_VERSION}.jar
+
DEFAULT_ARTIFACT_REPOSITORY=${DEFAULT_ARTIFACT_REPOSITORY:-https://repo1.maven.org/maven2/}
+
URL1=${DEFAULT_ARTIFACT_REPOSITORY%/}/org/scala-sbt/sbt-launch/${SBT_VERSION}/sbt-launch-${SBT_VERSION}.jar
JAR=build/sbt-launch-${SBT_VERSION}.jar
sbt_jar=$JAR
diff --git a/dev/check-license b/dev/check-license
index 9836d8c464aa..6b6eb6ba1f08 100755
--- a/dev/check-license
+++ b/dev/check-license
@@ -20,7 +20,8 @@
acquire_rat_jar () {
-
URL="${DEFAULT_ARTIFACT_REPOSITORY:-https://repo1.maven.org/maven2/}org/apache/rat/apache-rat/${RAT_VERSION}/apache-rat-${RAT_VERSION}.jar"
+
DEFAULT_ARTIFACT_REPOSITORY=${DEFAULT_ARTIFACT_REPOSITORY:-https://maven-central.storage-download.googleapis.com/maven2/}
+
URL="${DEFAULT_ARTIFACT_REPOSITORY%/}/org/apache/rat/apache-rat/${RAT_VERSION}/apache-rat-${RAT_VERSION}.jar"
JAR="$rat_jar"
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]