[
https://issues.apache.org/jira/browse/YARN-1409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13821380#comment-13821380
]
Jason Lowe commented on YARN-1409:
----------------------------------
Is it sufficient to only check for scheduler shutdown before scheduling? I'm
thinking of the multithreaded case where we check if the scheduler is shutdown,
and it isn't, but then another thread shuts it down just before we schedule.
Should we instead expect to catch RejectedExecutionException when we try to
schedule and act appropriately? Or alternatively if we don't care about
rejected executions, install an appropriate RejectedExecutionHandler that
doesn't throw when the thread pool executor is created.
> NonAggregatingLongHandler can throw RejectedExecutionException
> --------------------------------------------------------------
>
> Key: YARN-1409
> URL: https://issues.apache.org/jira/browse/YARN-1409
> Project: Hadoop YARN
> Issue Type: Bug
> Reporter: Tsuyoshi OZAWA
> Assignee: Tsuyoshi OZAWA
> Attachments: YARN-1409.1.patch
>
>
> This problem is caused by handling APPLICATION_FINISHED events after calling
> sched.shotdown() in NonAggregatingLongHandler#serviceStop().
> org.apache.hadoop.mapred.TestJobCleanup can fail because of
> RejectedExecutionException by NonAggregatingLogHandler.
> {code}
> 2013-11-13 10:53:06,970 FATAL [AsyncDispatcher event handler]
> event.AsyncDispatcher (AsyncDispatcher.java:dispatch(166)) - Error in
> dispatcher thread
> java.util.concurrent.RejectedExecutionException: Task
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask@d51df63
> rejected from
> java.util.concurrent.ScheduledThreadPoolExecutor@7a20e369[Shutting down, pool
> size = 4, active threads = 0, queued tasks = 7, completed tasks = 0]
> at
> java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2048)
> at
> java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:821)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor.delayedExecute(ScheduledThreadPoolExecutor.java:325)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor.schedule(ScheduledThreadPoolExecutor.java:530)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.loghandler.NonAggregatingLogHandler.handle(NonAggregatingLogHandler.java:121)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.loghandler.NonAggregatingLogHandler.handle(NonAggregatingLogHandler.java:49)
> at
> org.apache.hadoop.yarn.event.AsyncDispatcher.dispatch(AsyncDispatcher.java:159)
> at
> org.apache.hadoop.yarn.event.AsyncDispatcher$1.run(AsyncDispatcher.java:95)
> at java.lang.Thread.run(Thread.java:724)
> {code}
--
This message was sent by Atlassian JIRA
(v6.1#6144)
