[
https://issues.apache.org/jira/browse/SPARK-31769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17111955#comment-17111955
]
Izek Greenfield commented on SPARK-31769:
-----------------------------------------
Hi [~cloud_fan] Following previous discussions on
[PR-26624|https://github.com/apache/spark/pull/26624], I'd like us to agree on
the necessity of this feature, after which I'll create a dedicated PR.
What are your thoughts?
> Add support of MDC in spark driver logs.
> ----------------------------------------
>
> Key: SPARK-31769
> URL: https://issues.apache.org/jira/browse/SPARK-31769
> Project: Spark
> Issue Type: Improvement
> Components: Spark Core
> Affects Versions: 3.0.0
> Reporter: Izek Greenfield
> Priority: Major
>
> In order to align driver logs with spark executors logs, add support to log
> MDC state for properties that are shared between the driver the server
> executors (e.g. application name. task id)
> The use case is applicable in 2 cases:
> # streaming
> # running spark under job server (in this case you have a long-running spark
> context that handles many tasks from different sources).
> See also [SPARK-8981|https://issues.apache.org/jira/browse/SPARK-8981] that
> handles the MDC in executor context.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
