[jira] [Commented] (HIVE-13517) Hive logs in Spark Executor and Driver should show thread-id.
[ https://issues.apache.org/jira/browse/HIVE-13517?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15955462#comment-15955462 ] Sahil Takiar commented on HIVE-13517: - Sounds good [~xuefuz], thanks for spending time to take a look at this! > Hive logs in Spark Executor and Driver should show thread-id. > - > > Key: HIVE-13517 > URL: https://issues.apache.org/jira/browse/HIVE-13517 > Project: Hive > Issue Type: Bug > Components: Spark >Affects Versions: 1.2.1, 2.0.0 >Reporter: Szehon Ho >Assignee: Sahil Takiar > Attachments: executor-driver-log.PNG, HIVE-13517.1.patch, > HIVE-13517.2.patch > > > In Spark, there might be more than one task running in one executor. > Similarly, there may be more than one thread running in Driver. > This makes debugging through the logs a nightmare. It would be great if there > could be thread-ids in the logs. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HIVE-13517) Hive logs in Spark Executor and Driver should show thread-id.
[ https://issues.apache.org/jira/browse/HIVE-13517?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15953928#comment-15953928 ] Xuefu Zhang commented on HIVE-13517: [~stakiar], Thanks for following up with this. I read thru the comments in the Spark PR and wasn't surprised at the resistance there. This issue, if exists, is very minor. Personally, I haven't met a case where thread-id is needed either. Nevertheless, our production cluster has this option on, which was done by the infra folks rather easily. Thus, I think we might just leave it as it is today and can pick it up in the future when we have a strong use case. Thanks. > Hive logs in Spark Executor and Driver should show thread-id. > - > > Key: HIVE-13517 > URL: https://issues.apache.org/jira/browse/HIVE-13517 > Project: Hive > Issue Type: Bug > Components: Spark >Affects Versions: 1.2.1, 2.0.0 >Reporter: Szehon Ho >Assignee: Sahil Takiar > Attachments: executor-driver-log.PNG, HIVE-13517.1.patch, > HIVE-13517.2.patch > > > In Spark, there might be more than one task running in one executor. > Similarly, there may be more than one thread running in Driver. > This makes debugging through the logs a nightmare. It would be great if there > could be thread-ids in the logs. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HIVE-13517) Hive logs in Spark Executor and Driver should show thread-id.
[ https://issues.apache.org/jira/browse/HIVE-13517?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15951599#comment-15951599 ] Sahil Takiar commented on HIVE-13517: - [~xuefuz] I've opened a JIRA, and posted a patch to update this in Spark, not sure what the community response will be - SPARK-20161 > Hive logs in Spark Executor and Driver should show thread-id. > - > > Key: HIVE-13517 > URL: https://issues.apache.org/jira/browse/HIVE-13517 > Project: Hive > Issue Type: Bug > Components: Spark >Affects Versions: 1.2.1, 2.0.0 >Reporter: Szehon Ho >Assignee: Sahil Takiar > Attachments: executor-driver-log.PNG, HIVE-13517.1.patch, > HIVE-13517.2.patch > > > In Spark, there might be more than one task running in one executor. > Similarly, there may be more than one thread running in Driver. > This makes debugging through the logs a nightmare. It would be great if there > could be thread-ids in the logs. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HIVE-13517) Hive logs in Spark Executor and Driver should show thread-id.
[
https://issues.apache.org/jira/browse/HIVE-13517?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15941478#comment-15941478
]
Sahil Takiar commented on HIVE-13517:
-
Changing the {{log4j.properties}} file in the Spark installation is definitely
a better solution to the thread-id issue. So perhaps we should change the goal
of this JIRA to "Allow custom log4j.properties file for Hive-on-Spark Driver
and Executor". Sound good?
> Hive logs in Spark Executor and Driver should show thread-id.
> -
>
> Key: HIVE-13517
> URL: https://issues.apache.org/jira/browse/HIVE-13517
> Project: Hive
> Issue Type: Bug
> Components: Spark
>Affects Versions: 1.2.1, 2.0.0
>Reporter: Szehon Ho
>Assignee: Sahil Takiar
> Attachments: executor-driver-log.PNG, HIVE-13517.1.patch,
> HIVE-13517.2.patch
>
>
> In Spark, there might be more than one task running in one executor.
> Similarly, there may be more than one thread running in Driver.
> This makes debugging through the logs a nightmare. It would be great if there
> could be thread-ids in the logs.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HIVE-13517) Hive logs in Spark Executor and Driver should show thread-id.
[
https://issues.apache.org/jira/browse/HIVE-13517?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15941462#comment-15941462
]
Sahil Takiar commented on HIVE-13517:
-
[~xuefuz] I'll open a ticket on the Spark side for adding the thread id to the
default {{log4j.properties}} file. I think the patch is still useful for users
who want a workaround until the Spark support is added. Also, allowing for a
custom {{log4j.properties}} allows users to fully configure the log format,
control log levels, etc; and different users can use different
{{log4j.properties}} files.
> Hive logs in Spark Executor and Driver should show thread-id.
> -
>
> Key: HIVE-13517
> URL: https://issues.apache.org/jira/browse/HIVE-13517
> Project: Hive
> Issue Type: Bug
> Components: Spark
>Affects Versions: 1.2.1, 2.0.0
>Reporter: Szehon Ho
>Assignee: Sahil Takiar
> Attachments: executor-driver-log.PNG, HIVE-13517.1.patch,
> HIVE-13517.2.patch
>
>
> In Spark, there might be more than one task running in one executor.
> Similarly, there may be more than one thread running in Driver.
> This makes debugging through the logs a nightmare. It would be great if there
> could be thread-ids in the logs.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HIVE-13517) Hive logs in Spark Executor and Driver should show thread-id.
[ https://issues.apache.org/jira/browse/HIVE-13517?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15941350#comment-15941350 ] Xuefu Zhang commented on HIVE-13517: [~stakiar], thanks for working on this. The patch looks good to me. However, I'm a little concern about the usability of this. For an end user, I need to create a dedicated log4j file just in order to get the thread-id logged. Compared to directly modifying the log4j file in Spark installation, I am not sure of the advantage here. Instead, I'm wondering if we should change the default log4j file in Spark such that this comes out of box. I'd think the thread-id is useful across all spark applications. Thoughts? > Hive logs in Spark Executor and Driver should show thread-id. > - > > Key: HIVE-13517 > URL: https://issues.apache.org/jira/browse/HIVE-13517 > Project: Hive > Issue Type: Bug > Components: Spark >Affects Versions: 1.2.1, 2.0.0 >Reporter: Szehon Ho >Assignee: Sahil Takiar > Attachments: executor-driver-log.PNG, HIVE-13517.1.patch, > HIVE-13517.2.patch > > > In Spark, there might be more than one task running in one executor. > Similarly, there may be more than one thread running in Driver. > This makes debugging through the logs a nightmare. It would be great if there > could be thread-ids in the logs. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HIVE-13517) Hive logs in Spark Executor and Driver should show thread-id.
[ https://issues.apache.org/jira/browse/HIVE-13517?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15940966#comment-15940966 ] Sahil Takiar commented on HIVE-13517: - [~xuefuz], [~csun] any chance either of you could take a look at this patch? What do you think of the approach? > Hive logs in Spark Executor and Driver should show thread-id. > - > > Key: HIVE-13517 > URL: https://issues.apache.org/jira/browse/HIVE-13517 > Project: Hive > Issue Type: Bug > Components: Spark >Affects Versions: 1.2.1, 2.0.0 >Reporter: Szehon Ho >Assignee: Sahil Takiar > Attachments: executor-driver-log.PNG, HIVE-13517.1.patch, > HIVE-13517.2.patch > > > In Spark, there might be more than one task running in one executor. > Similarly, there may be more than one thread running in Driver. > This makes debugging through the logs a nightmare. It would be great if there > could be thread-ids in the logs. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HIVE-13517) Hive logs in Spark Executor and Driver should show thread-id.
[
https://issues.apache.org/jira/browse/HIVE-13517?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15940961#comment-15940961
]
Sahil Takiar commented on HIVE-13517:
-
Yes, they could just write their own log4j.properties file and specify it in
{{hive.spark.log4j.file}} - but I think having a log4j file with out-of-the-box
support for thread-id in the logs would improve usability.
> Hive logs in Spark Executor and Driver should show thread-id.
> -
>
> Key: HIVE-13517
> URL: https://issues.apache.org/jira/browse/HIVE-13517
> Project: Hive
> Issue Type: Bug
> Components: Spark
>Affects Versions: 1.2.1, 2.0.0
>Reporter: Szehon Ho
>Assignee: Sahil Takiar
> Attachments: executor-driver-log.PNG, HIVE-13517.1.patch,
> HIVE-13517.2.patch
>
>
> In Spark, there might be more than one task running in one executor.
> Similarly, there may be more than one thread running in Driver.
> This makes debugging through the logs a nightmare. It would be great if there
> could be thread-ids in the logs.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HIVE-13517) Hive logs in Spark Executor and Driver should show thread-id.
[
https://issues.apache.org/jira/browse/HIVE-13517?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15939618#comment-15939618
]
liyunzhang_intel commented on HIVE-13517:
-
[~stakiar]: LGTM, but what i am confused
{quote}
In a future patch, we could add a better default log4j properties file, which
has the thread-id.
{quote}
Based on HIVE-13517.2.patch, if users want a specified pattern(like contains
thread-id) in log4j.properties, they can write the pattern in
{{hive.spark.log4j.file}}, why need add a better default log4j.properties later?
> Hive logs in Spark Executor and Driver should show thread-id.
> -
>
> Key: HIVE-13517
> URL: https://issues.apache.org/jira/browse/HIVE-13517
> Project: Hive
> Issue Type: Bug
> Components: Spark
>Affects Versions: 1.2.1, 2.0.0
>Reporter: Szehon Ho
>Assignee: Sahil Takiar
> Attachments: executor-driver-log.PNG, HIVE-13517.1.patch,
> HIVE-13517.2.patch
>
>
> In Spark, there might be more than one task running in one executor.
> Similarly, there may be more than one thread running in Driver.
> This makes debugging through the logs a nightmare. It would be great if there
> could be thread-ids in the logs.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HIVE-13517) Hive logs in Spark Executor and Driver should show thread-id.
[
https://issues.apache.org/jira/browse/HIVE-13517?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15939048#comment-15939048
]
Hive QA commented on HIVE-13517:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12860189/HIVE-13517.2.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 10511 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[comments] (batchId=35)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/4319/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/4319/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-4319/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12860189 - PreCommit-HIVE-Build
> Hive logs in Spark Executor and Driver should show thread-id.
> -
>
> Key: HIVE-13517
> URL: https://issues.apache.org/jira/browse/HIVE-13517
> Project: Hive
> Issue Type: Bug
> Components: Spark
>Affects Versions: 1.2.1, 2.0.0
>Reporter: Szehon Ho
>Assignee: Sahil Takiar
> Attachments: executor-driver-log.PNG, HIVE-13517.1.patch,
> HIVE-13517.2.patch
>
>
> In Spark, there might be more than one task running in one executor.
> Similarly, there may be more than one thread running in Driver.
> This makes debugging through the logs a nightmare. It would be great if there
> could be thread-ids in the logs.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HIVE-13517) Hive logs in Spark Executor and Driver should show thread-id.
[
https://issues.apache.org/jira/browse/HIVE-13517?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15937958#comment-15937958
]
Hive QA commented on HIVE-13517:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12860011/HIVE-13517.1.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 2 failed/errored test(s), 10510 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[comments] (batchId=35)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainanalyze_2]
(batchId=95)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/4309/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/4309/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-4309/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 2 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12860011 - PreCommit-HIVE-Build
> Hive logs in Spark Executor and Driver should show thread-id.
> -
>
> Key: HIVE-13517
> URL: https://issues.apache.org/jira/browse/HIVE-13517
> Project: Hive
> Issue Type: Bug
> Components: Spark
>Affects Versions: 1.2.1, 2.0.0
>Reporter: Szehon Ho
>Assignee: Sahil Takiar
> Attachments: executor-driver-log.PNG, HIVE-13517.1.patch
>
>
> In Spark, there might be more than one task running in one executor.
> Similarly, there may be more than one thread running in Driver.
> This makes debugging through the logs a nightmare. It would be great if there
> could be thread-ids in the logs.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HIVE-13517) Hive logs in Spark Executor and Driver should show thread-id.
[
https://issues.apache.org/jira/browse/HIVE-13517?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15933988#comment-15933988
]
Sahil Takiar commented on HIVE-13517:
-
[~kellyzly] yes thats correct, but I think the default {{log4j.properties}}
file in Spark is
https://github.com/apache/spark/blob/master/conf/log4j.properties.template -
which doesn't contain %t (thread name) in the layout pattern.
I think the goal of this JIRA is to specify a different {{log4j.properties}}
for Spark executors such that the thread name is in the layout pattern by
default. This should make debugging HoS jobs much easier.
[~szehon] is my understanding correct?
> Hive logs in Spark Executor and Driver should show thread-id.
> -
>
> Key: HIVE-13517
> URL: https://issues.apache.org/jira/browse/HIVE-13517
> Project: Hive
> Issue Type: Bug
> Components: Spark
>Affects Versions: 1.2.1, 2.0.0
>Reporter: Szehon Ho
>Assignee: liyunzhang_intel
> Attachments: executor-driver-log.PNG
>
>
> In Spark, there might be more than one task running in one executor.
> Similarly, there may be more than one thread running in Driver.
> This makes debugging through the logs a nightmare. It would be great if there
> could be thread-ids in the logs.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HIVE-13517) Hive logs in Spark Executor and Driver should show thread-id.
[
https://issues.apache.org/jira/browse/HIVE-13517?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15933954#comment-15933954
]
liyunzhang_intel commented on HIVE-13517:
-
[~stakiar]: it is ok to assign it to you.
what i am confused is the log pattern in hive and spark are like following
spark/conf/log4j.properties
{code}
log4j.rootCategory=DEBUG, console$
log4j.appender.console=org.apache.log4j.ConsoleAppender$
log4j.appender.console.target=System.err$
log4j.appender.console.layout=org.apache.log4j.PatternLayout$
log4j.appender.console.layout.ConversionPattern=%d{yy/MM/dd HH:mm:ss} %p %t
%c{1}: %m%n$
{code}
hive/conf/hive-log4j.properties
{code}
appender.console.type = Console$
appender.console.name = console$
appender.console.target = SYSTEM_ERR$
appender.console.layout.type = PatternLayout$
appender.console.layout.pattern = %d{ISO8601} %5p [%t] %c{2}: %m%n$
{code}
thread-name( %t ) is included in the pattern, so current we can see the
thread-name in the log. Is my understand right?
> Hive logs in Spark Executor and Driver should show thread-id.
> -
>
> Key: HIVE-13517
> URL: https://issues.apache.org/jira/browse/HIVE-13517
> Project: Hive
> Issue Type: Bug
> Components: Spark
>Affects Versions: 1.2.1, 2.0.0
>Reporter: Szehon Ho
>Assignee: liyunzhang_intel
> Attachments: executor-driver-log.PNG
>
>
> In Spark, there might be more than one task running in one executor.
> Similarly, there may be more than one thread running in Driver.
> This makes debugging through the logs a nightmare. It would be great if there
> could be thread-ids in the logs.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HIVE-13517) Hive logs in Spark Executor and Driver should show thread-id.
[ https://issues.apache.org/jira/browse/HIVE-13517?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15933927#comment-15933927 ] Sahil Takiar commented on HIVE-13517: - [~kellyzly] I think I may have an approach to fix this. Are you actively working on this, or do you mind if I assign it to myself? > Hive logs in Spark Executor and Driver should show thread-id. > - > > Key: HIVE-13517 > URL: https://issues.apache.org/jira/browse/HIVE-13517 > Project: Hive > Issue Type: Bug > Components: Spark >Affects Versions: 1.2.1, 2.0.0 >Reporter: Szehon Ho >Assignee: liyunzhang_intel > Attachments: executor-driver-log.PNG > > > In Spark, there might be more than one task running in one executor. > Similarly, there may be more than one thread running in Driver. > This makes debugging through the logs a nightmare. It would be great if there > could be thread-ids in the logs. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HIVE-13517) Hive logs in Spark Executor and Driver should show thread-id.
[
https://issues.apache.org/jira/browse/HIVE-13517?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15624357#comment-15624357
]
liyunzhang_intel commented on HIVE-13517:
-
[~szehon]:
currently i view the driver and executor logs by url like
"http://xxx:18080/;(see the attached picture) and click the link to view the
detailed logs. The content of log is like
{noformat}
Using Spark's default log4j profile: org/apache/spark/log4j-defaults.properties
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in
[jar:file:/home/hadoop26/nm-local-dir/filecache/11/spark-assembly-1.6.2-SNAPSHOT-hadoop2.6.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in
[jar:file:/home/hadoop-2.6.0/share/hadoop/common/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
16/10/31 23:16:58 INFO CoarseGrainedExecutorBackend: Registered signal handlers
for [TERM, HUP, INT]
16/10/31 23:16:59 WARN NativeCodeLoader: Unable to load native-hadoop library
for your platform... using builtin-java classes where applicable
16/10/31 23:16:59 INFO SecurityManager: Changing view acls to: root
16/10/31 23:16:59 INFO SecurityManager: Changing modify acls to: root
16/10/31 23:16:59 INFO SecurityManager: SecurityManager: authentication
disabled; ui acls disabled; users with view permissions: Set(root); users with
modify permissions: Set(root)
16/10/31 23:16:59 INFO SecurityManager: Changing view acls to: root
16/10/31 23:16:59 INFO SecurityManager: Changing modify acls to: root
{noformat}
So what you want is 16/10/31 23:16:59 *Thread-Name* INFO SecurityManager:
Changing modify acls to: root? is my understanding right?
> Hive logs in Spark Executor and Driver should show thread-id.
> -
>
> Key: HIVE-13517
> URL: https://issues.apache.org/jira/browse/HIVE-13517
> Project: Hive
> Issue Type: Bug
> Components: Spark
>Affects Versions: 1.2.1, 2.0.0
>Reporter: Szehon Ho
>Assignee: liyunzhang_intel
>
> In Spark, there might be more than one task running in one executor.
> Similarly, there may be more than one thread running in Driver.
> This makes debugging through the logs a nightmare. It would be great if there
> could be thread-ids in the logs.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-13517) Hive logs in Spark Executor and Driver should show thread-id.
[ https://issues.apache.org/jira/browse/HIVE-13517?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15623323#comment-15623323 ] Szehon Ho commented on HIVE-13517: -- Yea if the thread name is there, that is great. I thought last time when I checked the Spark Executor and Driver logs that they were mixed, and there was no indication about the thread. I don't have an environment right now to check that, do you see the thread name now in those logs? > Hive logs in Spark Executor and Driver should show thread-id. > - > > Key: HIVE-13517 > URL: https://issues.apache.org/jira/browse/HIVE-13517 > Project: Hive > Issue Type: Bug > Components: Spark >Affects Versions: 1.2.1, 2.0.0 >Reporter: Szehon Ho >Assignee: liyunzhang_intel > > In Spark, there might be more than one task running in one executor. > Similarly, there may be more than one thread running in Driver. > This makes debugging through the logs a nightmare. It would be great if there > could be thread-ids in the logs. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-13517) Hive logs in Spark Executor and Driver should show thread-id.
[
https://issues.apache.org/jira/browse/HIVE-13517?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15622634#comment-15622634
]
liyunzhang_intel commented on HIVE-13517:
-
[~szehon]:
{quote}
It would be great if there could be thread-ids in the logs.
{quote}
currently hive log4j pattern is like {noformat}
appender.DRFA.layout.pattern = %d{ISO8601} %5p [%t] %c{2}: %m%n{noformat} Here
%t means thread's name. What you want is thread-id, is my understanding right?
> Hive logs in Spark Executor and Driver should show thread-id.
> -
>
> Key: HIVE-13517
> URL: https://issues.apache.org/jira/browse/HIVE-13517
> Project: Hive
> Issue Type: Bug
> Components: Spark
>Affects Versions: 1.2.1, 2.0.0
>Reporter: Szehon Ho
>Assignee: liyunzhang_intel
>
> In Spark, there might be more than one task running in one executor.
> Similarly, there may be more than one thread running in Driver.
> This makes debugging through the logs a nightmare. It would be great if there
> could be thread-ids in the logs.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
