This is an automated email from the ASF dual-hosted git repository.
huaxingao pushed a commit to branch branch-3.2
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-3.2 by this push:
new 1800bff498a [SPARK-42188][BUILD][3.2] Force SBT protobuf version to
match Maven
1800bff498a is described below
commit 1800bff498a0d52ddfdc6f376dd164ff379e7c82
Author: Steve Vaughan Jr <s_vaug...@apple.com>
AuthorDate: Wed Jan 25 18:23:00 2023 -0800
[SPARK-42188][BUILD][3.2] Force SBT protobuf version to match Maven
### What changes were proposed in this pull request?
Update `SparkBuild.scala` to force SBT use of `protobuf-java` to match the
Maven version. The Maven dependencyManagement section forces `protobuf-java`
to use `2.5.0`, but SBT is using `3.14.0`.
### Why are the changes needed?
Define `protoVersion` in `SparkBuild.scala` and use it in
`DependencyOverrides` to force the SBT version of `protobuf-java` to match the
setting defined in the Maven top-level `pom.xml`. Add comments to both
`pom.xml` and `SparkBuild.scala` to ensure that the values are kept in sync.
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
Before the update, SBT reported using `3.14.0`:
```
% build/sbt dependencyTree | grep proto | sed 's/^.*-com/com/' | sort |
uniq -c
8 com.google.protobuf:protobuf-java:2.5.0 (evicted by: 3.14.0)
70 com.google.protobuf:protobuf-java:3.14.0
```
After the patch is applied, SBT reports using `2.5.0`:
```
% build/sbt dependencyTree | grep proto | sed 's/^.*-com/com/' | sort |
uniq -c
70 com.google.protobuf:protobuf-java:2.5.0
```
Closes #39745 from snmvaughan/feature/SPARK-42188.
Authored-by: Steve Vaughan Jr <s_vaug...@apple.com>
Signed-off-by: huaxingao <huaxin_...@apple.com>
---
pom.xml | 1 +
project/SparkBuild.scala | 6 ++++++
2 files changed, 7 insertions(+)
diff --git a/pom.xml b/pom.xml
index 8127d69476c..99ed2308da3 100644
--- a/pom.xml
+++ b/pom.xml
@@ -121,6 +121,7 @@
<slf4j.version>1.7.30</slf4j.version>
<log4j.version>1.2.17</log4j.version>
<hadoop.version>3.3.1</hadoop.version>
+ <!-- SPARK-42188: When updating `protobuf.version`, also need to update
`protoVersion` in `SparkBuild.scala` -->
<protobuf.version>2.5.0</protobuf.version>
<yarn.version>${hadoop.version}</yarn.version>
<zookeeper.version>3.6.2</zookeeper.version>
diff --git a/project/SparkBuild.scala b/project/SparkBuild.scala
index 8c1dbceb8b3..435040344c3 100644
--- a/project/SparkBuild.scala
+++ b/project/SparkBuild.scala
@@ -79,6 +79,9 @@ object BuildCommons {
val testTempDir = s"$sparkHome/target/tmp"
val javaVersion = settingKey[String]("source and target JVM version for
javac and scalac")
+
+ // SPARK-42188: needs to be consistent with `protobuf.version` in `pom.xml`.
+ val protoVersion = "2.5.0"
}
object SparkBuild extends PomBuild {
@@ -676,9 +679,12 @@ object KubernetesIntegrationTests {
* Overrides to work around sbt's dependency resolution being different from
Maven's.
*/
object DependencyOverrides {
+ import BuildCommons.protoVersion
+
lazy val guavaVersion = sys.props.get("guava.version").getOrElse("14.0.1")
lazy val settings = Seq(
dependencyOverrides += "com.google.guava" % "guava" % guavaVersion,
+ dependencyOverrides += "com.google.protobuf" % "protobuf-java" %
protoVersion,
dependencyOverrides += "xerces" % "xercesImpl" % "2.12.0",
dependencyOverrides += "jline" % "jline" % "2.14.6",
dependencyOverrides += "org.apache.avro" % "avro" % "1.10.2")
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org
For additional commands, e-mail: commits-h...@spark.apache.org
