This is an automated email from the ASF dual-hosted git repository.
chengpan pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/kyuubi.git
The following commit(s) were added to refs/heads/master by this push:
new 7ba458a0d [KYUUBI #5291] Verify Spark engine is compatible with Spark
3.5.0
7ba458a0d is described below
commit 7ba458a0da3e990700acdc58fc52300e87a0af6d
Author: Cheng Pan <[email protected]>
AuthorDate: Fri Sep 15 11:35:40 2023 +0000
[KYUUBI #5291] Verify Spark engine is compatible with Spark 3.5.0
### _Why are the changes needed?_
Apache Spark 3.5.0 released in few days ago, this PR aims to add
cross-version verification to ensure that Kyuubi engine built against Spark
3.4.1 works on Spark 3.5.0 runtime.
https://spark.apache.org/releases/spark-release-3-5-0.html
For users who want to try Kyuubi with Spark 3.5.0, please compile
master/branch-1.8 via
```
./build/dist --tgz --web-ui --spark-provided --flink-provided
--hive-provided
```
And then follow the [Quick
Start](https://kyuubi.readthedocs.io/en/master/quick_start/quick_start.html) to
setup Kyuubi and Spark properly.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including
negative and positive cases if possible
- [x] Add screenshots for manual tests if appropriate
```log
Connected to: Spark SQL (version 3.5.0)
Driver: Kyuubi Project Hive JDBC Client (version 1.8.0-SNAPSHOT)
Beeline version 1.8.0-SNAPSHOT by Apache Kyuubi
0: jdbc:hive2://0.0.0.0:10009/default> select version();
+-------------------------------------------------+
| version() |
+-------------------------------------------------+
| 3.5.0 ce5ddad990373636e94071e7cef2f31021add07b |
+-------------------------------------------------+
1 row selected (0.39 seconds)
0: jdbc:hive2://0.0.0.0:10009/default> select kyuubi_version();
+-------------------+
| kyuubi_version() |
+-------------------+
| 1.8.0-SNAPSHOT |
+-------------------+
1 row selected (0.16 seconds)
```
- [x] [Run
test](https://kyuubi.readthedocs.io/en/master/contributing/code/testing.html#running-tests)
locally before make a pull request
### _Was this patch authored or co-authored using generative AI tooling?_
No.
Closes #5291 from pan3793/cross-3.5.
Closes #5291
24107a414 [Cheng Pan] Update .github/workflows/master.yml
1554e7131 [Cheng Pan] Verify Spark engine on Spark 3.5
Authored-by: Cheng Pan <[email protected]>
Signed-off-by: Cheng Pan <[email protected]>
---
.github/workflows/master.yml | 5 +++++
1 file changed, 5 insertions(+)
diff --git a/.github/workflows/master.yml b/.github/workflows/master.yml
index b2dd57a50..3b85530d4 100644
--- a/.github/workflows/master.yml
+++ b/.github/workflows/master.yml
@@ -71,6 +71,11 @@ jobs:
spark-archive:
'-Dspark.archive.mirror=https://archive.apache.org/dist/spark/spark-3.3.3
-Dspark.archive.name=spark-3.3.3-bin-hadoop3.tgz -Pzookeeper-3.6'
exclude-tags:
'-Dmaven.plugin.scalatest.exclude.tags=org.scalatest.tags.Slow,org.apache.kyuubi.tags.DeltaTest,org.apache.kyuubi.tags.IcebergTest,org.apache.kyuubi.tags.SparkLocalClusterTest'
comment: 'verify-on-spark-3.3-binary'
+ - java: 8
+ spark: '3.4'
+ spark-archive:
'-Dspark.archive.mirror=https://archive.apache.org/dist/spark/spark-3.5.0
-Dspark.archive.name=spark-3.5.0-bin-hadoop3.tgz -Pzookeeper-3.6'
+ exclude-tags:
'-Dmaven.plugin.scalatest.exclude.tags=org.scalatest.tags.Slow,org.apache.kyuubi.tags.DeltaTest,org.apache.kyuubi.tags.IcebergTest,org.apache.kyuubi.tags.SparkLocalClusterTest'
+ comment: 'verify-on-spark-3.5-binary'
exclude:
# SPARK-33772: Spark supports JDK 17 since 3.3.0
- java: 17
