[
https://issues.apache.org/jira/browse/HIVE-16061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15935315#comment-15935315
]
Prasanth Jayachandran commented on HIVE-16061:
----------------------------------------------
*Configuration:*
https://github.com/apache/hive/blob/master/llap-server/src/main/resources/llap-daemon-log4j2.properties#L79-L100
*MDC put for new threads:*
https://github.com/apache/hive/blob/master/llap-server/src/java/org/apache/hadoop/hive/llap/io/api/impl/LlapRecordReader.java#L108-L111
*MDC put/reset for threadpools (cache/reuse of threads):*
This is required for inheriting MDC when new threads are spawned.
https://github.com/apache/hive/blob/master/llap-server/src/java/org/apache/hadoop/hive/llap/daemon/impl/LlapDaemon.java#L336
For the case where a threadpool already exists and MDC has to be inherited (in
case of cached thread pools or core thread reuse), we use custom threadpool
that copies the IDs to MDC and clears it after executing the thread. There is
some reflection happening in custom threadpool (NDC to MDC copying) which is
LLAP specific and is not required for HS2.
https://github.com/apache/hive/blob/master/llap-server/src/java/org/apache/hadoop/hive/llap/daemon/impl/StatsRecordingThreadPool.java#L108-L149
> When hive.async.log.enabled is set to true, some output is not printed to the
> beeline console
> ---------------------------------------------------------------------------------------------
>
> Key: HIVE-16061
> URL: https://issues.apache.org/jira/browse/HIVE-16061
> Project: Hive
> Issue Type: Bug
> Components: Logging
> Affects Versions: 2.1.1
> Reporter: Aihua Xu
> Assignee: Aihua Xu
> Attachments: HIVE-16061.1.patch, HIVE-16061.2.patch
>
>
> Run a hiveserver2 instance "hive --service hiveserver2".
> Then from another console, connect to hiveserver2 "beeline -u
> "jdbc:hive2://localhost:10000"
> When you run a MR job like "select t1.key from src t1 join src t2 on
> t1.key=t2.key", some of the console logs like MR job info are not printed to
> the console while it just print to the hiveserver2 console.
> When hive.async.log.enabled is set to false and restarts the HiveServer2,
> then the output will be printed to the beeline console.
> OperationLog implementation uses the ThreadLocal variable to store associated
> the log file. When the hive.async.log.enabled is set to true, the logs will
> be processed by a ThreadPool and the actual threads from the pool which
> prints the message won't be able to access the log file stored in the
> original thread.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
