[
https://issues.apache.org/jira/browse/TEZ-4039?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17043386#comment-17043386
]
László Bodor edited comment on TEZ-4039 at 2/24/20 1:39 PM:
------------------------------------------------------------
[~prasanth_j], [~jeagles]: could you please take a quick look at
[^TEZ-4039.03.patch] ?
this patch is basically about the following:
1. Introduce CallableWithMdc, analogous to CallableWithNdc
2. sets up MDC values in both DAGAppMaster and tasks
3. automatically handles values which have meaning in tez: dagId, taskAttemptId
4. according to configuration idea above, lets upstream components inject new
MDC keys/values via configuration
+TODO:
1. unit tests for basic the util method
2. container mode cluster testing (I'm going to provide output logs when I'm
done)
was (Author: abstractdog):
[~prasanth_j], [~jeagles]: could you please take a quick look at
[^TEZ-4039.02.patch] ?
this patch is basically about the following:
1. Introduce CallableWithMdc, analogous to CallableWithNdc
2. sets up MDC values in both DAGAppMaster and tasks
3. automatically handles values which have meaning in tez: dagId, taskAttemptId
4. according to configuration idea above, lets upstream components inject new
MDC keys/values via configuration
+TODO:
1. unit tests for basic the util method
2. container mode cluster testing (I'm going to provide output logs when I'm
done)
> Tez should inject dag id, query id into MDC
> -------------------------------------------
>
> Key: TEZ-4039
> URL: https://issues.apache.org/jira/browse/TEZ-4039
> Project: Apache Tez
> Issue Type: Bug
> Reporter: Prasanth Jayachandran
> Assignee: László Bodor
> Priority: Major
> Attachments: TEZ-4039.01.patch, TEZ-4039.02.patch, TEZ-4039.03.patch
>
>
> Tez currently uses CallableWithNdc to store thread specific context. It
> should also inject the context into MDC so that pattern layout can dump the
> contexts from MDC (with NDC it is not possible to read the context in pattern
> lyaout).
> Hive for example, sets queryId in the MDC and pattern layout prints the
> queryId
>
> {code:java}
> %d{ISO8601} %-5p [%t (%X{queryId})] %c{2}: %m%n
> {code}
> Llap sets dagId, fragmentId and queryId into MDC which is used for queryId
> based routing of logging.
> Similarly, Tez AM should set dagId and queryId (if available) into MDC.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
