pan3793 commented on code in PR #6789:
URL: https://github.com/apache/kyuubi/pull/6789#discussion_r1825349910
##########
externals/kyuubi-spark-sql-engine/src/main/scala/org/apache/spark/sql/kyuubi/SparkDatasetHelper.scala:
##########
@@ -290,4 +290,13 @@ object SparkDatasetHelper extends Logging {
nodeName == "org.apache.spark.sql.execution.command.ExecutedCommandExec" ||
nodeName == "org.apache.spark.sql.execution.CommandResultExec"
}
+
+ /** SPARK-47270: Returns a optimized plan for CommandResult, convert to
`LocalRelation`. */
+ def commandResultOptimized[T](dataset: Dataset[T]): Dataset[T] = {
Review Comment:
I'm -0 on this change, I would consider it a Spark side issue. Spark’s
master branch is undergoing a major refactoring, I'm worried about accessing
Spark's non-public API in the engine's "core code path".
For users who want to avoid triggering executor launch, they can either
patch their spark or set init SQL as something like: `SET spark.app.id`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscr...@kyuubi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscr...@kyuubi.apache.org
For additional commands, e-mail: notifications-h...@kyuubi.apache.org
