This is an automated email from the ASF dual-hosted git repository.
dongjoon pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new 43210fe71fa8 [SPARK-45331][BUILD] Upgrade Scala to 2.13.12
43210fe71fa8 is described below
commit 43210fe71fa86f4f11a5afee4075052c09a34f6e
Author: yangjie01 <[email protected]>
AuthorDate: Sat Sep 30 04:46:46 2023 -0700
[SPARK-45331][BUILD] Upgrade Scala to 2.13.12
### What changes were proposed in this pull request?
This pr aims to upgrade Scala from 2.13.11 to 2.13.12.
Additionally, this pr adds ``-Wconf:msg=legacy-binding:s`` to suppress
similar compiler warnings as below:
```
[ERROR]
/Users/yangjie01/SourceCode/git/spark-mine-13/core/src/main/scala/org/apache/spark/deploy/client/StandaloneAppClient.scala:171:
reference to stop is ambiguous;
it is both defined in the enclosing class StandaloneAppClient and inherited
in the enclosing class ClientEndpoint as method stop (defined in trait
RpcEndpoint, inherited through parent trait ThreadSafeRpcEndpoint)
In Scala 2, symbols inherited from a superclass shadow symbols defined in
an outer scope.
Such references are ambiguous in Scala 3. To continue using the inherited
symbol, write `this.stop`.
Or use `-Wconf:msg=legacy-binding:s` to silence this warning. [quickfixable]
Applicable -Wconf / nowarn filters for this fatal warning: msg=<part of the
message>, cat=other,
site=org.apache.spark.deploy.client.StandaloneAppClient.ClientEndpoint.receive
```
### Why are the changes needed?
The new version bring some regression fixes:
- https://github.com/scala/scala/pull/10422
- https://github.com/scala/scala/pull/10424
And a new feature: Quickfixes
```
For some errors and warnings, the compiler now suggests an edit that could
fix the issue. Tooling such as IDEs can then offer the edits, or the compiler
itself will make the change if run again with -quickfix.
```
- https://github.com/scala/scala/pull/10406
- https://github.com/scala/scala/pull/10482
- https://github.com/scala/scala/pull/10484
The release notes as follows:
- https://github.com/scala/scala/releases/tag/v2.13.12
### Does this PR introduce _any_ user-facing change?
Yes, Scala version changed.
### How was this patch tested?
Pass Github
### Was this patch authored or co-authored using generative AI tooling?
NO
Closes #43185 from LuciferYang/SPARK-45331.
Authored-by: yangjie01 <[email protected]>
Signed-off-by: Dongjoon Hyun <[email protected]>
---
dev/deps/spark-deps-hadoop-3-hive-2.3 | 6 +++---
pom.xml | 11 +++++++++--
project/SparkBuild.scala | 9 +++++++--
3 files changed, 19 insertions(+), 7 deletions(-)
diff --git a/dev/deps/spark-deps-hadoop-3-hive-2.3
b/dev/deps/spark-deps-hadoop-3-hive-2.3
index 085316832bb9..38f05e39dcd2 100644
--- a/dev/deps/spark-deps-hadoop-3-hive-2.3
+++ b/dev/deps/spark-deps-hadoop-3-hive-2.3
@@ -229,11 +229,11 @@ py4j/0.10.9.7//py4j-0.10.9.7.jar
remotetea-oncrpc/1.1.2//remotetea-oncrpc-1.1.2.jar
rocksdbjni/8.3.2//rocksdbjni-8.3.2.jar
scala-collection-compat_2.13/2.7.0//scala-collection-compat_2.13-2.7.0.jar
-scala-compiler/2.13.11//scala-compiler-2.13.11.jar
-scala-library/2.13.11//scala-library-2.13.11.jar
+scala-compiler/2.13.12//scala-compiler-2.13.12.jar
+scala-library/2.13.12//scala-library-2.13.12.jar
scala-parallel-collections_2.13/1.0.4//scala-parallel-collections_2.13-1.0.4.jar
scala-parser-combinators_2.13/2.3.0//scala-parser-combinators_2.13-2.3.0.jar
-scala-reflect/2.13.11//scala-reflect-2.13.11.jar
+scala-reflect/2.13.12//scala-reflect-2.13.12.jar
scala-xml_2.13/2.2.0//scala-xml_2.13-2.2.0.jar
slf4j-api/2.0.9//slf4j-api-2.0.9.jar
snakeyaml-engine/2.6//snakeyaml-engine-2.6.jar
diff --git a/pom.xml b/pom.xml
index 1a3fe2a93b8d..fc3297c96bd1 100644
--- a/pom.xml
+++ b/pom.xml
@@ -172,7 +172,7 @@
<!-- managed up from 3.2.1 for SPARK-11652 -->
<commons.collections.version>3.2.2</commons.collections.version>
<commons.collections4.version>4.4</commons.collections4.version>
- <scala.version>2.13.11</scala.version>
+ <scala.version>2.13.12</scala.version>
<scala.binary.version>2.13</scala.binary.version>
<scalatest-maven-plugin.version>2.2.0</scalatest-maven-plugin.version>
<!-- don't upgrade scala-maven-plugin to version 4.7.2 or higher, see
SPARK-45144 for details -->
@@ -2947,7 +2947,7 @@
<arg>-Wconf:cat=other-nullary-override:wv</arg>
<arg>-Wconf:cat=other-match-analysis&site=org.apache.spark.sql.catalyst.catalog.SessionCatalog.lookupFunction.catalogFunction:wv</arg>
<arg>-Wconf:cat=other-pure-statement&site=org.apache.spark.streaming.util.FileBasedWriteAheadLog.readAll.readFile:wv</arg>
-
<arg>-Wconf:cat=other-pure-statement&site=org.apache.spark.scheduler.OutputCommitCoordinatorSuite.<local
OutputCommitCoordinatorSuite>.futureAction:wv</arg>
+
<arg>-Wconf:cat=other-pure-statement&site=org.apache.spark.scheduler.OutputCommitCoordinatorSuite:wv</arg>
<!--
SPARK-33775 Suppress compilation warnings that contain the
following contents.
TODO(SPARK-33805): Undo the corresponding deprecated usage
suppression rule after fixed.
@@ -2978,6 +2978,13 @@
SPARK-40497 Upgrade Scala to 2.13.11 and suppress `Implicit
definition should have explicit type`
-->
<arg>-Wconf:msg=Implicit definition should have explicit
type:s</arg>
+ <!--
+ SPARK-45331 Upgrade Scala to 2.13.12 and suppress "In Scala 2,
symbols inherited
+ from a superclass shadow symbols defined in an outer scope.
Such references are
+ ambiguous in Scala 3. To continue using the inherited symbol,
write `this.stop`.
+ Or use `-Wconf:msg=legacy-binding:s` to silence this warning.
[quickfixable]"
+ -->
+ <arg>-Wconf:msg=legacy-binding:s</arg>
</args>
<jvmArgs>
<jvmArg>-Xss128m</jvmArg>
diff --git a/project/SparkBuild.scala b/project/SparkBuild.scala
index 13c92142d46b..d9a79f6aa82f 100644
--- a/project/SparkBuild.scala
+++ b/project/SparkBuild.scala
@@ -236,7 +236,7 @@ object SparkBuild extends PomBuild {
"-Wconf:cat=other-nullary-override:wv",
"-Wconf:cat=other-match-analysis&site=org.apache.spark.sql.catalyst.catalog.SessionCatalog.lookupFunction.catalogFunction:wv",
"-Wconf:cat=other-pure-statement&site=org.apache.spark.streaming.util.FileBasedWriteAheadLog.readAll.readFile:wv",
-
"-Wconf:cat=other-pure-statement&site=org.apache.spark.scheduler.OutputCommitCoordinatorSuite.<local
OutputCommitCoordinatorSuite>.futureAction:wv",
+
"-Wconf:cat=other-pure-statement&site=org.apache.spark.scheduler.OutputCommitCoordinatorSuite:wv",
"-Wconf:cat=other-pure-statement&site=org.apache.spark.sql.streaming.sources.StreamingDataSourceV2Suite.testPositiveCase.\\$anonfun:wv",
// SPARK-33775 Suppress compilation warnings that contain the
following contents.
// TODO(SPARK-33805): Undo the corresponding deprecated usage
suppression rule after
@@ -259,7 +259,12 @@ object SparkBuild extends PomBuild {
"-Wconf:cat=unchecked&msg=eliminated by erasure:s",
"-Wconf:msg=^(?=.*?a value of type)(?=.*?cannot also be).+$:s",
// SPARK-40497 Upgrade Scala to 2.13.11 and suppress `Implicit
definition should have explicit type`
- "-Wconf:msg=Implicit definition should have explicit type:s"
+ "-Wconf:msg=Implicit definition should have explicit type:s",
+ // SPARK-45331 Upgrade Scala to 2.13.12 and suppress "In Scala 2,
symbols inherited
+ // from a superclass shadow symbols defined in an outer scope. Such
references are
+ // ambiguous in Scala 3. To continue using the inherited symbol, write
`this.stop`.
+ // Or use `-Wconf:msg=legacy-binding:s` to silence this warning.
[quickfixable]"
+ "-Wconf:msg=legacy-binding:s"
)
}
)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
