This is an automated email from the ASF dual-hosted git repository.
bowenliang pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/kyuubi.git
The following commit(s) were added to refs/heads/master by this push:
new 88c8cc312 [KYUUBI #5807] Simple norm the use of
`kyuubi.operation.result.arrow.timestampAsString` and
`kyuubi.operation.result.format`
88c8cc312 is described below
commit 88c8cc312b902dd3a4be767d2cd80834e9f43ded
Author: ไฝๅฟ้ญ <[email protected]>
AuthorDate: Sun Dec 3 20:16:24 2023 +0800
[KYUUBI #5807] Simple norm the use of
`kyuubi.operation.result.arrow.timestampAsString` and
`kyuubi.operation.result.format`
# :mag: Description
## Issue References ๐
This pull request norm the arrow conf.
## Describe Your Solution ๐ง
Using `kyuubi.operation.result.arrow.timestampAsString` and
`kyuubi.operation.result.format` string to get conf . To replace as
`ARROW_BASED_ROWSET_TIMESTAMP_AS_STRING.key` and `OPERATION_RESULT_FORMAT`.
Make it more norm.
## Types of changes :bookmark:
- [ ] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [x] Other
## Test Plan ๐งช
No need test
---
# Checklists
## ๐ Author Self Checklist
- [x] My code follows the [style
guidelines](https://kyuubi.readthedocs.io/en/master/contributing/code/style.html)
of this project
- [x] I have performed a self-review
- [ ] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [ ] My changes generate no new warnings
- [ ] I have added tests that prove my fix is effective or that my feature
works
- [ ] New and existing unit tests pass locally with my changes
- [x] This patch was not authored or co-authored using [Generative
Tooling](https://www.apache.org/legal/generative-tooling.html)
## ๐ Committer Pre-Merge Checklist
- [ ] Pull request title is okay.
- [ ] No license issues.
- [ ] Milestone correctly set?
- [ ] Test coverage is ok
- [ ] Assignees are selected.
- [ ] Minimum number of approvals
- [ ] No changes are requested
**Be nice. Be informative.**
Closes #5807 from zzzzming95/norm-conf.
Closes #5807
9e65bfafe [ไฝๅฟ้ญ] norm the arrow conf
Authored-by: ไฝๅฟ้ญ <[email protected]>
Signed-off-by: liangbowen <[email protected]>
---
.../org/apache/kyuubi/engine/spark/operation/SparkOperation.scala | 4 ++--
.../kyuubi/engine/spark/operation/SparkSQLOperationManager.scala | 2 +-
2 files changed, 3 insertions(+), 3 deletions(-)
diff --git
a/externals/kyuubi-spark-sql-engine/src/main/scala/org/apache/kyuubi/engine/spark/operation/SparkOperation.scala
b/externals/kyuubi-spark-sql-engine/src/main/scala/org/apache/kyuubi/engine/spark/operation/SparkOperation.scala
index f40f1d490..5799e0237 100644
---
a/externals/kyuubi-spark-sql-engine/src/main/scala/org/apache/kyuubi/engine/spark/operation/SparkOperation.scala
+++
b/externals/kyuubi-spark-sql-engine/src/main/scala/org/apache/kyuubi/engine/spark/operation/SparkOperation.scala
@@ -29,7 +29,7 @@ import org.apache.spark.sql.types.StructType
import org.apache.kyuubi.{KyuubiSQLException, Utils}
import org.apache.kyuubi.config.KyuubiConf
-import org.apache.kyuubi.config.KyuubiConf.{OPERATION_SPARK_LISTENER_ENABLED,
SESSION_PROGRESS_ENABLE, SESSION_USER_SIGN_ENABLED}
+import
org.apache.kyuubi.config.KyuubiConf.{ARROW_BASED_ROWSET_TIMESTAMP_AS_STRING,
OPERATION_SPARK_LISTENER_ENABLED, SESSION_PROGRESS_ENABLE,
SESSION_USER_SIGN_ENABLED}
import
org.apache.kyuubi.config.KyuubiReservedKeys.{KYUUBI_SESSION_SIGN_PUBLICKEY,
KYUUBI_SESSION_USER_KEY, KYUUBI_SESSION_USER_SIGN, KYUUBI_STATEMENT_ID_KEY}
import org.apache.kyuubi.engine.spark.KyuubiSparkUtil.{getSessionConf,
SPARK_SCHEDULER_POOL_KEY}
import org.apache.kyuubi.engine.spark.events.SparkOperationEvent
@@ -271,7 +271,7 @@ abstract class SparkOperation(session: Session)
protected def resultFormat: String = "thrift"
protected def timestampAsString: Boolean = {
- spark.conf.get("kyuubi.operation.result.arrow.timestampAsString",
"false").toBoolean
+ spark.conf.get(ARROW_BASED_ROWSET_TIMESTAMP_AS_STRING.key,
"false").toBoolean
}
protected def setSessionUserSign(): Unit = {
diff --git
a/externals/kyuubi-spark-sql-engine/src/main/scala/org/apache/kyuubi/engine/spark/operation/SparkSQLOperationManager.scala
b/externals/kyuubi-spark-sql-engine/src/main/scala/org/apache/kyuubi/engine/spark/operation/SparkSQLOperationManager.scala
index ab0828746..cd365c62a 100644
---
a/externals/kyuubi-spark-sql-engine/src/main/scala/org/apache/kyuubi/engine/spark/operation/SparkSQLOperationManager.scala
+++
b/externals/kyuubi-spark-sql-engine/src/main/scala/org/apache/kyuubi/engine/spark/operation/SparkSQLOperationManager.scala
@@ -86,7 +86,7 @@ class SparkSQLOperationManager private (name: String) extends
OperationManager(n
val incrementalCollect =
spark.conf.getOption(OPERATION_INCREMENTAL_COLLECT.key)
.map(_.toBoolean).getOrElse(operationIncrementalCollectDefault)
// TODO: respect the config of the operation ExecuteStatement,
if it was set.
- val resultFormat =
spark.conf.get("kyuubi.operation.result.format", "thrift")
+ val resultFormat = spark.conf.get(OPERATION_RESULT_FORMAT.key,
"thrift")
resultFormat.toLowerCase match {
case "arrow" =>
new ArrowBasedExecuteStatement(
