[
https://issues.apache.org/jira/browse/YARN-8632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16589918#comment-16589918
]
Xianghao Lu edited comment on YARN-8632 at 8/23/18 8:49 AM:
------------------------------------------------------------
It seems that "setUncaughtExceptionHandler" is not suitable in thread pool,
refer to
https://issues.apache.org/jira/browse/HADOOP-12748
[http://literatejava.com/threading/silent-thread-death-unhandled-exceptions/|http://literatejava.com/threading/silent-thread-death-unhandled-exceptions/]
Using try catch block to catch Exception may not be a good method but it does
solve the problem in this jira, but I also find it is used in
[AppLevelAggregator|https://github.com/apache/hadoop/blob/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice/src/main/java/org/apache/hadoop/yarn/server/timelineservice/collector/AppLevelTimelineCollectorWithAgg.java#L139]
[EntityLogScanner|https://github.com/apache/hadoop/blob/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timeline-pluginstorage/src/main/java/org/apache/hadoop/yarn/server/timeline/EntityGroupFSTimelineStore.java#L865],
etc. which are related to function scheduleAtFixedRate.
Fortunately, I noticed HADOOP-12749 HADOOP-14966 provide a nice way to log
uncaught exception in thread pool, so the patch of branch brunk will be very
easy.
Attatched YARN-8632.003.patch
was (Author: luxianghao):
It seems that "setUncaughtExceptionHandler" is not suitable in thread pool,
refer to
https://issues.apache.org/jira/browse/HADOOP-12748
[http://literatejava.com/threading/silent-thread-death-unhandled-exceptions/|http://example.com]
Using try catch block to catch Exception may not be a good method but it does
solve the problem in this jira, but I also find it is used in
[AppLevelAggregator|https://github.com/apache/hadoop/blob/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice/src/main/java/org/apache/hadoop/yarn/server/timelineservice/collector/AppLevelTimelineCollectorWithAgg.java#L139]
[EntityLogScanner|https://github.com/apache/hadoop/blob/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timeline-pluginstorage/src/main/java/org/apache/hadoop/yarn/server/timeline/EntityGroupFSTimelineStore.java#L865],
etc. which are related to function scheduleAtFixedRate.
Fortunately, I noticed HADOOP-12749 HADOOP-14966 provide a nice way to log
uncaught exception in thread pool, so the patch of branch brunk will be very
easy.
Attatched YARN-8632.003.patch
> No data in file realtimetrack.json after running SchedulerLoadSimulator
> -----------------------------------------------------------------------
>
> Key: YARN-8632
> URL: https://issues.apache.org/jira/browse/YARN-8632
> Project: Hadoop YARN
> Issue Type: Bug
> Components: scheduler-load-simulator
> Reporter: Xianghao Lu
> Assignee: Xianghao Lu
> Priority: Major
> Attachments: YARN-8632-branch-2.7.2.001.patch, YARN-8632.001.patch,
> YARN-8632.002.patch, YARN-8632.003.patch
>
>
> Recently, I have been using
> [SchedulerLoadSimulator|https://hadoop.apache.org/docs/r2.7.2/hadoop-sls/SchedulerLoadSimulator.html]
> to validate the impact of changes on my FairScheduler. I encountered some
> problems.
> Firstly, I fix a npe bug with the patch in
> https://issues.apache.org/jira/browse/YARN-4302
> Secondly, everything seems to be ok, but I just get "[]" in file
> realtimetrack.json. Finally, I find the MetricsLogRunnable thread will exit
> because of npe,
> the reason is "wrapper.getQueueSet()" is still null when executing "String
> metrics = web.generateRealTimeTrackingMetrics();"
> So, we should put "String metrics = web.generateRealTimeTrackingMetrics();"
> in try section to avoid MetricsLogRunnable thread exit with unexpected
> exception.
> My hadoop version is 2.7.2, it seems that hadoop trunk branch also has the
> second problem and I have made a patch to solve it.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
