This is an automated email from the ASF dual-hosted git repository.
dongjoon pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new 135bdeb2ba10 [SPARK-47064][SQL][TESTS] Use Scala 2.13 Spark
distribution in `HiveExternalCatalogVersionsSuite`
135bdeb2ba10 is described below
commit 135bdeb2ba10726fb8036f4312c6a40cea63a801
Author: Dongjoon Hyun <dh...@apple.com>
AuthorDate: Thu Feb 15 09:42:45 2024 -0800
[SPARK-47064][SQL][TESTS] Use Scala 2.13 Spark distribution in
`HiveExternalCatalogVersionsSuite`
### What changes were proposed in this pull request?
This PR aims to use `Scala 2.13` Spark binary in
`HiveExternalCatalogVersionsSuite`.
### Why are the changes needed?
SPARK-45314 makes Scala 2.13 is the default Scala version.
As one of migration paths, the users choose Apache Spark 3.5.0 (Scala 2.13)
and Apache Spark 3.4.2 (Scala 2.13).
We had better focus on Scala 2.13 testing.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
Pass the CIs and manual validation.
```
$ SBT_OPTS=-Dspark.test.cache-dir=/tmp/test-spark \
build/sbt "hive/testOnly *.HiveExternalCatalogVersionsSuite" -Phive
...
[info] HiveExternalCatalogVersionsSuite:
[info] - backward compatibility (11 seconds, 583 milliseconds)
[info] Run completed in 42 seconds, 143 milliseconds.
[info] Total number of tests run: 1
[info] Suites: completed 1, aborted 0
[info] Tests: succeeded 1, failed 0, canceled 0, ignored 0, pending 0
[info] All tests passed.
[success] Total time: 55 s, completed Feb 15, 2024, 9:34:54 AM
```
During the testing, `Scala 2.13.8` is used.
```
$ ls -al /tmp/test-spark/spark-3.4.2/jars/scala*
-rw-r--r-- 1 dongjoon wheel 5596 Nov 24 23:04
/tmp/test-spark/spark-3.4.2/jars/scala-collection-compat_2.13-2.7.0.jar
-rw-r--r-- 1 dongjoon wheel 12097183 Nov 24 23:04
/tmp/test-spark/spark-3.4.2/jars/scala-compiler-2.13.8.jar
-rw-r--r-- 1 dongjoon wheel 6003601 Nov 24 23:04
/tmp/test-spark/spark-3.4.2/jars/scala-library-2.13.8.jar
-rw-r--r-- 1 dongjoon wheel 1127123 Nov 24 23:04
/tmp/test-spark/spark-3.4.2/jars/scala-parallel-collections_2.13-1.0.4.jar
-rw-r--r-- 1 dongjoon wheel 189556 Nov 24 23:04
/tmp/test-spark/spark-3.4.2/jars/scala-parser-combinators_2.13-2.1.1.jar
-rw-r--r-- 1 dongjoon wheel 3772083 Nov 24 23:04
/tmp/test-spark/spark-3.4.2/jars/scala-reflect-2.13.8.jar
-rw-r--r-- 1 dongjoon wheel 483090 Nov 24 23:04
/tmp/test-spark/spark-3.4.2/jars/scala-xml_2.13-2.1.0.jar
```
### Was this patch authored or co-authored using generative AI tooling?
No.
Closes #45124 from dongjoon-hyun/SPARK-47064.
Authored-by: Dongjoon Hyun <dh...@apple.com>
Signed-off-by: Dongjoon Hyun <dh...@apple.com>
---
.../org/apache/spark/sql/hive/HiveExternalCatalogVersionsSuite.scala | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git
a/sql/hive/src/test/scala/org/apache/spark/sql/hive/HiveExternalCatalogVersionsSuite.scala
b/sql/hive/src/test/scala/org/apache/spark/sql/hive/HiveExternalCatalogVersionsSuite.scala
index 50cf4017bd1e..726341ffdf9e 100644
---
a/sql/hive/src/test/scala/org/apache/spark/sql/hive/HiveExternalCatalogVersionsSuite.scala
+++
b/sql/hive/src/test/scala/org/apache/spark/sql/hive/HiveExternalCatalogVersionsSuite.scala
@@ -95,7 +95,7 @@ class HiveExternalCatalogVersionsSuite extends
SparkSubmitTestUtils {
mirrors.distinct :+ "https://archive.apache.org/dist"; :+
PROCESS_TABLES.releaseMirror
logInfo(s"Trying to download Spark $version from $sites")
for (site <- sites) {
- val filename = s"spark-$version-bin-hadoop3.tgz"
+ val filename = s"spark-$version-bin-hadoop3-scala2.13.tgz"
val url = s"$site/spark/spark-$version/$filename"
logInfo(s"Downloading Spark $version from $url")
try {
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org
For additional commands, e-mail: commits-h...@spark.apache.org
