This is an automated email from the ASF dual-hosted git repository.
yao pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new 9130f78fb12e [SPARK-47607] Add documentation for Structured logging
framework
9130f78fb12e is described below
commit 9130f78fb12eed94f48e1fd9ccedb6fe651a4440
Author: Gengliang Wang <gengli...@apache.org>
AuthorDate: Thu May 16 14:13:13 2024 +0800
[SPARK-47607] Add documentation for Structured logging framework
### What changes were proposed in this pull request?
Add documentation for Structured logging framework
### Why are the changes needed?
Provide document for Spark developers
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
Doc preview:
<img width="797" alt="image"
src="https://github.com/apache/spark/assets/1097932/d3c4fcdc-57e4-4af2-8b05-6b4f6731a8c0";>
### Was this patch authored or co-authored using generative AI tooling?
No
Closes #46605 from gengliangwang/updateGuideline.
Authored-by: Gengliang Wang <gengli...@apache.org>
Signed-off-by: Kent Yao <y...@apache.org>
---
.../main/scala/org/apache/spark/internal/README.md | 33 ++++++++++++++++++++++
1 file changed, 33 insertions(+)
diff --git a/common/utils/src/main/scala/org/apache/spark/internal/README.md
b/common/utils/src/main/scala/org/apache/spark/internal/README.md
index c0190b965834..28d279485187 100644
--- a/common/utils/src/main/scala/org/apache/spark/internal/README.md
+++ b/common/utils/src/main/scala/org/apache/spark/internal/README.md
@@ -1,5 +1,38 @@
# Guidelines for the Structured Logging Framework
+## Scala Logging
+Use the `org.apache.spark.internal.Logging` trait for logging in Scala code:
+* **Logging Messages with Variables**: When logging a message with variables,
wrap all the variables with `MDC`s and they will be automatically added to the
Mapped Diagnostic Context (MDC). This allows for structured logging and better
log analysis.
+```scala
+logInfo(log"Trying to recover app: ${MDC(LogKeys.APP_ID, app.id)}")
+```
+* **Constant String Messages**: If you are logging a constant string message,
use the log methods that accept a constant string.
+```scala
+logInfo("StateStore stopped")
+```
+
+## Java Logging
+Use the `org.apache.spark.internal.SparkLoggerFactory` to get the logger
instance in Java code:
+* **Getting Logger Instance**: Instead of using `org.slf4j.LoggerFactory`, use
`org.apache.spark.internal.SparkLoggerFactory` to ensure structured logging.
+```java
+import org.apache.spark.internal.SparkLogger;
+import org.apache.spark.internal.SparkLoggerFactory;
+
+private static final SparkLogger logger =
SparkLoggerFactory.getLogger(JavaUtils.class);
+```
+* **Logging Messages with Variables**: When logging messages with variables,
wrap all the variables with `MDC`s and they will be automatically added to the
Mapped Diagnostic Context (MDC).
+```java
+import org.apache.spark.internal.LogKeys;
+import org.apache.spark.internal.MDC;
+
+logger.error("Unable to delete file for partition {}",
MDC.of(LogKeys.PARTITION_ID$.MODULE$, i));
+```
+
+* **Constant String Messages**: For logging constant string messages, use the
standard logging methods.
+```java
+logger.error("Failed to abort the writer after failing to write map output.",
e);
+```
+
## LogKey
`LogKey`s serve as identifiers for mapped diagnostic contexts (MDC) within
logs. Follow these guidelines when adding a new LogKey:
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org
For additional commands, e-mail: commits-h...@spark.apache.org
