This is an automated email from the ASF dual-hosted git repository.
dongjoon pushed a commit to branch branch-3.4
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-3.4 by this push:
new ada09838f9c [SPARK-42673][BUILD] Make `build/mvn` build Spark only
with the verified maven version
ada09838f9c is described below
commit ada09838f9c9de239b0204ac2a87647c3ea75598
Author: yangjie01 <[email protected]>
AuthorDate: Mon Mar 6 11:05:42 2023 -0800
[SPARK-42673][BUILD] Make `build/mvn` build Spark only with the verified
maven version
### What changes were proposed in this pull request?
`build/mvn` tends to use the new maven version to build Spark now, and GA
starts to use 3.9.0 as the default maven version.
But there may be some uncertain factors when building Spark with unverified
version.
For example, `java-11-17` GA build task build with maven 3.9.0 has many
error logs in master like follow:
```
Error: [ERROR] An error occurred attempting to read POM
org.codehaus.plexus.util.xml.pull.XmlPullParserException: UTF-8 BOM plus
xml decl of ISO-8859-1 is incompatible (position: START_DOCUMENT seen <?xml
version="1.0" encoding="ISO-8859-1"... 1:42)
at org.codehaus.plexus.util.xml.pull.MXParser.parseXmlDeclWithVersion
(MXParser.java:3423)
at org.codehaus.plexus.util.xml.pull.MXParser.parseXmlDecl
(MXParser.java:3345)
at org.codehaus.plexus.util.xml.pull.MXParser.parsePI
(MXParser.java:3197)
at org.codehaus.plexus.util.xml.pull.MXParser.parseProlog
(MXParser.java:1828)
at org.codehaus.plexus.util.xml.pull.MXParser.nextImpl
(MXParser.java:1757)
at org.codehaus.plexus.util.xml.pull.MXParser.next (MXParser.java:1375)
at org.apache.maven.model.io.xpp3.MavenXpp3Reader.read
(MavenXpp3Reader.java:3940)
at org.apache.maven.model.io.xpp3.MavenXpp3Reader.read
(MavenXpp3Reader.java:612)
at org.apache.maven.model.io.xpp3.MavenXpp3Reader.read
(MavenXpp3Reader.java:627)
at org.cyclonedx.maven.BaseCycloneDxMojo.readPom
(BaseCycloneDxMojo.java:759)
at org.cyclonedx.maven.BaseCycloneDxMojo.readPom
(BaseCycloneDxMojo.java:746)
at org.cyclonedx.maven.BaseCycloneDxMojo.retrieveParentProject
(BaseCycloneDxMojo.java:694)
at org.cyclonedx.maven.BaseCycloneDxMojo.getClosestMetadata
(BaseCycloneDxMojo.java:524)
at org.cyclonedx.maven.BaseCycloneDxMojo.convert
(BaseCycloneDxMojo.java:481)
at org.cyclonedx.maven.CycloneDxMojo.execute (CycloneDxMojo.java:70)
at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo
(DefaultBuildPluginManager.java:126)
at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute2
(MojoExecutor.java:342)
at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute
(MojoExecutor.java:330)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute
(MojoExecutor.java:213)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute
(MojoExecutor.java:175)
at org.apache.maven.lifecycle.internal.MojoExecutor.access$000
(MojoExecutor.java:76)
at org.apache.maven.lifecycle.internal.MojoExecutor$1.run
(MojoExecutor.java:163)
at org.apache.maven.plugin.DefaultMojosExecutionStrategy.execute
(DefaultMojosExecutionStrategy.java:39)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute
(MojoExecutor.java:160)
at
org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject
(LifecycleModuleBuilder.java:105)
at
org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject
(LifecycleModuleBuilder.java:73)
at
org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build
(SingleThreadedBuilder.java:53)
at org.apache.maven.lifecycle.internal.LifecycleStarter.execute
(LifecycleStarter.java:118)
at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:260)
at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:172)
at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:100)
at org.apache.maven.cli.MavenCli.execute (MavenCli.java:821)
at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:270)
at org.apache.maven.cli.MavenCli.main (MavenCli.java:192)
at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
at jdk.internal.reflect.NativeMethodAccessorImpl.invoke
(NativeMethodAccessorImpl.java:77)
at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke
(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke (Method.java:568)
at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced
(Launcher.java:282)
at org.codehaus.plexus.classworlds.launcher.Launcher.launch
(Launcher.java:225)
at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode
(Launcher.java:406)
at org.codehaus.plexus.classworlds.launcher.Launcher.main
(Launcher.java:347)
```
So this pr change the version check condition of `build/mvn` to make it
build Spark only with the verified maven version.
### Why are the changes needed?
Make `build/mvn` build Spark only with the verified maven version
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
- `java-11-17` GA build task pass and no error message as above
- Manual test:
1. Make the system use maven 3.9.0( > 3.8.7 ) by default:
run `mvn -version`
```
Apache Maven 3.9.0 (9b58d2bad23a66be161c4664ef21ce219c2c8584)
Maven home: /Users/${userName}/Tools/maven
Java version: 1.8.0_362, vendor: Azul Systems, Inc., runtime:
/Users/${userName}/Tools/zulu8/zulu-8.jdk/Contents/Home/jre
Default locale: zh_CN, platform encoding: UTF-8
OS name: "mac os x", version: "13.2.1", arch: "aarch64", family: "mac"
```
and run `build/mvn -version`
```
Using `mvn` from path: /${basedir}/spark/build/apache-maven-3.8.7/bin/mvn
Using SPARK_LOCAL_IP=localhost
Apache Maven 3.8.7 (b89d5959fcde851dcb1c8946a785a163f14e1e29)
Maven home: /${basedir}/spark/build/apache-maven-3.8.7
Java version: 1.8.0_362, vendor: Azul Systems, Inc., runtime:
/Users/${userName}/Tools/zulu8/zulu-8.jdk/Contents/Home/jre
Default locale: zh_CN, platform encoding: UTF-8
OS name: "mac os x", version: "13.2.1", arch: "aarch64", family: "mac"
```
We can see Spark use 3.8.7 in build directory when the system default maven
> 3.8.7
2. Make the system use maven 3.8.7 by default:
run `mvn -version`
```
mvn -version
Apache Maven 3.8.7 (b89d5959fcde851dcb1c8946a785a163f14e1e29)
Maven home: /Users/${userName}/Tools/maven
Java version: 1.8.0_362, vendor: Azul Systems, Inc., runtime:
/Users/${userName}/Tools/zulu8/zulu-8.jdk/Contents/Home/jre
Default locale: zh_CN, platform encoding: UTF-8
OS name: "mac os x", version: "13.2.1", arch: "aarch64", family: "mac"
```
and run `build/mvn -version`
```
Using `mvn` from path: /Users/${userName}/Tools/maven/bin/mvn
Using SPARK_LOCAL_IP=localhost
Apache Maven 3.8.7 (b89d5959fcde851dcb1c8946a785a163f14e1e29)
Maven home: /Users/${userName}/Tools/maven
Java version: 1.8.0_362, vendor: Azul Systems, Inc., runtime:
/Users/${userName}/Tools/zulu8/zulu-8.jdk/Contents/Home/jre
Default locale: zh_CN, platform encoding: UTF-8
OS name: "mac os x", version: "13.2.1", arch: "aarch64", family: "mac"
```
We can see Spark use system default maven 3.8.7 when the system default
maven is 3.8.7.
3. Make the system use maven 3.8.6( < 3.8.7 ) by default:
run `mvn -version`
```
mvn -version
Apache Maven 3.8.6 (84538c9988a25aec085021c365c560670ad80f63)
Maven home: /Users/${userName}/Tools/maven
Java version: 1.8.0_362, vendor: Azul Systems, Inc., runtime:
/Users/${userName}/Tools/zulu8/zulu-8.jdk/Contents/Home/jre
Default locale: zh_CN, platform encoding: UTF-8
OS name: "mac os x", version: "13.2.1", arch: "aarch64", family: "mac"
```
and run `build/mvn -version`
```
Using `mvn` from path: /Users/${userName}/Tools/maven/bin/mvn
Using SPARK_LOCAL_IP=localhost
Apache Maven 3.8.7 (b89d5959fcde851dcb1c8946a785a163f14e1e29)
Maven home: /Users/${userName}/Tools/maven
Java version: 1.8.0_362, vendor: Azul Systems, Inc., runtime:
/Users/${userName}/Tools/zulu8/zulu-8.jdk/Contents/Home/jre
Default locale: zh_CN, platform encoding: UTF-8
OS name: "mac os x", version: "13.2.1", arch: "aarch64", family: "mac"
```
We can see Spark use 3.8.7 in build directory when the system default maven
< 3.8.7.
Closes #40283 from LuciferYang/ban-maven-3.9.x.
Authored-by: yangjie01 <[email protected]>
Signed-off-by: Dongjoon Hyun <[email protected]>
(cherry picked from commit f70b8cf1a00002b6c6b96ec4e6ad4d6c2f0ab392)
Signed-off-by: Dongjoon Hyun <[email protected]>
---
build/mvn | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/build/mvn b/build/mvn
index f1283a0fff9..aee9358fe44 100755
--- a/build/mvn
+++ b/build/mvn
@@ -119,7 +119,7 @@ install_mvn() {
if [ "$MVN_BIN" ]; then
local MVN_DETECTED_VERSION="$(mvn --version | head -n1 | awk '{print $3}')"
fi
- if [ $(version $MVN_DETECTED_VERSION) -lt $(version $MVN_VERSION) ]; then
+ if [ $(version $MVN_DETECTED_VERSION) -ne $(version $MVN_VERSION) ]; then
local MVN_TARBALL="apache-maven-${MVN_VERSION}-bin.tar.gz"
local FILE_PATH="maven/maven-3/${MVN_VERSION}/binaries/${MVN_TARBALL}"
local
APACHE_MIRROR=${APACHE_MIRROR:-'https://www.apache.org/dyn/closer.lua'}
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
