[
https://issues.apache.org/jira/browse/HIVE-13517?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sahil Takiar updated HIVE-13517:
--------------------------------
Attachment: HIVE-13517.2.patch
Attaching updated patch that adds some testing. Instead of explicitly copying
the {{log4j2.properties}} file from {{data/conf/spark/}} into the Spark
distribution, the new config {{hive.spark.log4j.file}} is now used.
I'm not 100% sure this was the original intention of this JIRA, but it is a
useful feature. Users can now add custom log4j files without modifying the
Spark installation.
In a future patch, we could add a better default log4j properties file, which
has the thread-id.
[~kellyzly] what do you think of this patch?
> Hive logs in Spark Executor and Driver should show thread-id.
> -------------------------------------------------------------
>
> Key: HIVE-13517
> URL: https://issues.apache.org/jira/browse/HIVE-13517
> Project: Hive
> Issue Type: Bug
> Components: Spark
> Affects Versions: 1.2.1, 2.0.0
> Reporter: Szehon Ho
> Assignee: Sahil Takiar
> Attachments: executor-driver-log.PNG, HIVE-13517.1.patch,
> HIVE-13517.2.patch
>
>
> In Spark, there might be more than one task running in one executor.
> Similarly, there may be more than one thread running in Driver.
> This makes debugging through the logs a nightmare. It would be great if there
> could be thread-ids in the logs.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
