This is an automated email from the ASF dual-hosted git repository.
gurwls223 pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new 02f45229171 [SPARK-39133][PYTHON][DOC] Mention log level setting in
PYSPARK_JVM_STACKTRACE_ENABLED
02f45229171 is described below
commit 02f452291712c32f167d7d9389b1fc375448a6de
Author: Xinrong Meng <[email protected]>
AuthorDate: Tue May 10 17:59:39 2022 +0900
[SPARK-39133][PYTHON][DOC] Mention log level setting in
PYSPARK_JVM_STACKTRACE_ENABLED
### What changes were proposed in this pull request?
Mention log level setting in PYSPARK_JVM_STACKTRACE_ENABLED.
### Why are the changes needed?
Even if `spark.sql.pyspark.jvmStacktrace.enabled` is set False, we should
set log level to FATAL to see Python exceptions only. The PR is proposed to
document that.
### Does this PR introduce _any_ user-facing change?
No. Documents change only.
### How was this patch tested?
Manual tests.
Closes #36490 from xinrong-databricks/doc_jvm_stacktrace.
Lead-authored-by: Xinrong Meng <[email protected]>
Co-authored-by: Xinrong Meng
<[email protected]>
Co-authored-by: Hyukjin Kwon <[email protected]>
Signed-off-by: Hyukjin Kwon <[email protected]>
---
.../src/main/scala/org/apache/spark/sql/internal/SQLConf.scala | 5 +++--
1 file changed, 3 insertions(+), 2 deletions(-)
diff --git
a/sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala
b/sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala
index 4c0eccbf35d..43d342fd06b 100644
--- a/sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala
+++ b/sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala
@@ -2577,8 +2577,9 @@ object SQLConf {
val PYSPARK_JVM_STACKTRACE_ENABLED =
buildConf("spark.sql.pyspark.jvmStacktrace.enabled")
.doc("When true, it shows the JVM stacktrace in the user-facing PySpark
exception " +
- "together with Python stacktrace. By default, it is disabled and hides
JVM stacktrace " +
- "and shows a Python-friendly exception only.")
+ "together with Python stacktrace. By default, it is disabled to hide
JVM stacktrace " +
+ "and shows a Python-friendly exception only. Note that this is
independent from log " +
+ "level settings.")
.version("3.0.0")
.booleanConf
// show full stacktrace in tests but hide in production by default.
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
