This is an automated email from the ASF dual-hosted git repository.
chengpan pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/kyuubi.git
The following commit(s) were added to refs/heads/master by this push:
new 693d8a22f [KYUUBI #6508] Add the key-value pairs in optimizedConf to
session conf
693d8a22f is described below
commit 693d8a22f8a0a314a1028ec1a4095c9612e31cdf
Author: jiaoqingbo <[email protected]>
AuthorDate: Mon Jul 1 11:15:28 2024 +0800
[KYUUBI #6508] Add the key-value pairs in optimizedConf to session conf
# :mag: Description
## Issue References 🔗
This pull request fixes #6508
## Describe Your Solution 🔧
Add the key-value pairs in optimizedConf to session conf，to make the
Kyuubi.env.SPARK_HOME take effect
## Types of changes :bookmark:
- [x] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
## Test Plan 🧪
#### Behavior Without This Pull Request :coffin:
When I configure the following parameters in kyuubi-env.sh
Then submit a spark batch task through the restful interface and pass the
kyuubi.engineEnv.SPARK_HOME parameter to change the default value
```curl -H "Content-Type: application/json" -X POST -d '{"batchType":
"SPARK", "resource":"xxxx/spark-examples_2.12-3.3.2.jar", "name": "Spark-PI",
"conf":
{"spark.master":"yarn","hive.server2.proxy.user":"XXXX","kyuubi.engineEnv.SPARK_HOME":"XXXXX/cluster114/spark","kyuubi.engineEnv.HADOOP_CONF_DIR":"XXXXX/conf"},
"args": [10],"className": "org.apache.spark.examples.SparkPi"}'
http://XXXXX:XXXX/api/v1/batches```
Observe the log and find that SPARK_HOME is not set and takes effect. It
still uses the value in kyuubi-env.sh.
#### Behavior With This Pull Request :tada:
After this PR, the update was successful
#### Related Unit Tests
---
# Checklist 📝
- [x] This patch was not authored or co-authored using [Generative
Tooling](https://www.apache.org/legal/generative-tooling.html)
**Be nice. Be informative.**
Closes #6510 from jiaoqingbo/6508.
Closes #6508
e89268e4b [jiaoqingbo] [KYUUBI #6508] Add the key-value pairs in
optimizedConf to session conf
Authored-by: jiaoqingbo <[email protected]>
Signed-off-by: Cheng Pan <[email protected]>
---
.../src/main/scala/org/apache/kyuubi/session/KyuubiBatchSession.scala | 4 ++++
1 file changed, 4 insertions(+)
diff --git
a/kyuubi-server/src/main/scala/org/apache/kyuubi/session/KyuubiBatchSession.scala
b/kyuubi-server/src/main/scala/org/apache/kyuubi/session/KyuubiBatchSession.scala
index 4ac84c1d0..f648c39cb 100644
---
a/kyuubi-server/src/main/scala/org/apache/kyuubi/session/KyuubiBatchSession.scala
+++
b/kyuubi-server/src/main/scala/org/apache/kyuubi/session/KyuubiBatchSession.scala
@@ -93,6 +93,10 @@ class KyuubiBatchSession(
}
}
+ optimizedConf.foreach {
+ case (key, value) => sessionConf.set(key, value)
+ }
+
override lazy val name: Option[String] =
batchName.filterNot(_.trim.isEmpty).orElse(optimizedConf.get(KyuubiConf.SESSION_NAME.key))
