[spark] branch branch-3.2 updated: [SPARK-37052][CORE] Spark should only pass --verbose argument to main class when is sql shell

[wenchen]

This is an automated email from the ASF dual-hosted git repository.

wenchen pushed a commit to branch branch-3.2
in repository https://gitbox.apache.org/repos/asf/spark.git


The following commit(s) were added to refs/heads/branch-3.2 by this push:
     new 34086b0  [SPARK-37052][CORE] Spark should only pass --verbose argument 
to main class when is sql shell
34086b0 is described below

commit 34086b0c4dff77deaa3e381e63bf4597f100166e
Author: Angerszhuuuu <angers....@gmail.com>
AuthorDate: Tue Oct 19 14:40:52 2021 +0800

    [SPARK-37052][CORE] Spark should only pass --verbose argument to main class 
when is sql shell
    
    ### What changes were proposed in this pull request?
    In https://github.com/apache/spark/pull/32163 spark pass `--verbose` to 
main class o support spark-sql shell can use verbose argument too.
    But for other shell main class such as saprk-shell, it's intercepter don't 
support `--verbose`, so we should only pass `--verbose` for sql shell
    
    ### Why are the changes needed?
    Fix bug
    
    ### Does this PR introduce _any_ user-facing change?
    No
    
    ### How was this patch tested?
    
    Closes #34322 from AngersZhuuuu/SPARK-37052.
    
    Authored-by: Angerszhuuuu <angers....@gmail.com>
    Signed-off-by: Wenchen Fan <wenc...@databricks.com>
    (cherry picked from commit a6d3a2c84e5cdc642ed57602612f0303585c4b6e)
    Signed-off-by: Wenchen Fan <wenc...@databricks.com>
---
 core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala 
b/core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala
index 8124650..67a601b 100644
--- a/core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala
+++ b/core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala
@@ -853,7 +853,7 @@ private[spark] class SparkSubmit extends Logging {
     }
     sparkConf.set(SUBMIT_PYTHON_FILES, formattedPyFiles.split(",").toSeq)
 
-    if (args.verbose) {
+    if (args.verbose && isSqlShell(childMainClass)) {
       childArgs ++= Seq("--verbose")
     }
     (childArgs.toSeq, childClasspath.toSeq, sparkConf, childMainClass)

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org
For additional commands, e-mail: commits-h...@spark.apache.org