[jira] [Updated] (YARN-1409) NonAggregatingLogHandler can throw RejectedExecutionException

2013-12-26 Thread Tsuyoshi OZAWA (JIRA) 

 [ 
https://issues.apache.org/jira/browse/YARN-1409?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tsuyoshi OZAWA updated YARN-1409:
-

Attachment: YARN-1409.3.patch

Thank you for the review, [~jlowe]. I apologize for the delay.
1. Added a test case to reproduce this issue
2. Fixed to avoid creating LogDeleterRunnable instances twice.

 NonAggregatingLogHandler can throw RejectedExecutionException
 -

 Key: YARN-1409
 URL: https://issues.apache.org/jira/browse/YARN-1409
 Project: Hadoop YARN
  Issue Type: Bug
Reporter: Tsuyoshi OZAWA
Assignee: Tsuyoshi OZAWA
 Attachments: YARN-1409.1.patch, YARN-1409.2.patch, YARN-1409.3.patch


 This problem is caused by handling APPLICATION_FINISHED events after calling 
 sched.shotdown() in NonAggregatingLongHandler#serviceStop(). 
 org.apache.hadoop.mapred.TestJobCleanup can fail because of 
 RejectedExecutionException by NonAggregatingLogHandler.
 {code}
 2013-11-13 10:53:06,970 FATAL [AsyncDispatcher event handler] 
 event.AsyncDispatcher (AsyncDispatcher.java:dispatch(166)) - Error in 
 dispatcher thread
 java.util.concurrent.RejectedExecutionException: Task 
 java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask@d51df63 
 rejected from 
 java.util.concurrent.ScheduledThreadPoolExecutor@7a20e369[Shutting down, pool 
 size = 4, active threads = 0, queued tasks = 7, completed tasks = 0]
 at 
 java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2048)
 at 
 java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:821)
 at 
 java.util.concurrent.ScheduledThreadPoolExecutor.delayedExecute(ScheduledThreadPoolExecutor.java:325)
 at 
 java.util.concurrent.ScheduledThreadPoolExecutor.schedule(ScheduledThreadPoolExecutor.java:530)
 at 
 org.apache.hadoop.yarn.server.nodemanager.containermanager.loghandler.NonAggregatingLogHandler.handle(NonAggregatingLogHandler.java:121)
 at 
 org.apache.hadoop.yarn.server.nodemanager.containermanager.loghandler.NonAggregatingLogHandler.handle(NonAggregatingLogHandler.java:49)
 at 
 org.apache.hadoop.yarn.event.AsyncDispatcher.dispatch(AsyncDispatcher.java:159)
 at 
 org.apache.hadoop.yarn.event.AsyncDispatcher$1.run(AsyncDispatcher.java:95)
 at java.lang.Thread.run(Thread.java:724)
 {code}



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Updated] (YARN-1409) NonAggregatingLogHandler can throw RejectedExecutionException

2013-11-22 Thread Tsuyoshi OZAWA (JIRA) 

 [ 
https://issues.apache.org/jira/browse/YARN-1409?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tsuyoshi OZAWA updated YARN-1409:
-

Summary: NonAggregatingLogHandler can throw RejectedExecutionException  
(was: NonAggregatingLongHandler can throw RejectedExecutionException)

 NonAggregatingLogHandler can throw RejectedExecutionException
 -

 Key: YARN-1409
 URL: https://issues.apache.org/jira/browse/YARN-1409
 Project: Hadoop YARN
  Issue Type: Bug
Reporter: Tsuyoshi OZAWA
Assignee: Tsuyoshi OZAWA
 Attachments: YARN-1409.1.patch


 This problem is caused by handling APPLICATION_FINISHED events after calling 
 sched.shotdown() in NonAggregatingLongHandler#serviceStop(). 
 org.apache.hadoop.mapred.TestJobCleanup can fail because of 
 RejectedExecutionException by NonAggregatingLogHandler.
 {code}
 2013-11-13 10:53:06,970 FATAL [AsyncDispatcher event handler] 
 event.AsyncDispatcher (AsyncDispatcher.java:dispatch(166)) - Error in 
 dispatcher thread
 java.util.concurrent.RejectedExecutionException: Task 
 java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask@d51df63 
 rejected from 
 java.util.concurrent.ScheduledThreadPoolExecutor@7a20e369[Shutting down, pool 
 size = 4, active threads = 0, queued tasks = 7, completed tasks = 0]
 at 
 java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2048)
 at 
 java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:821)
 at 
 java.util.concurrent.ScheduledThreadPoolExecutor.delayedExecute(ScheduledThreadPoolExecutor.java:325)
 at 
 java.util.concurrent.ScheduledThreadPoolExecutor.schedule(ScheduledThreadPoolExecutor.java:530)
 at 
 org.apache.hadoop.yarn.server.nodemanager.containermanager.loghandler.NonAggregatingLogHandler.handle(NonAggregatingLogHandler.java:121)
 at 
 org.apache.hadoop.yarn.server.nodemanager.containermanager.loghandler.NonAggregatingLogHandler.handle(NonAggregatingLogHandler.java:49)
 at 
 org.apache.hadoop.yarn.event.AsyncDispatcher.dispatch(AsyncDispatcher.java:159)
 at 
 org.apache.hadoop.yarn.event.AsyncDispatcher$1.run(AsyncDispatcher.java:95)
 at java.lang.Thread.run(Thread.java:724)
 {code}



--
This message was sent by Atlassian JIRA
(v6.1#6144)

[jira] [Updated] (YARN-1409) NonAggregatingLogHandler can throw RejectedExecutionException

2013-11-22 Thread Tsuyoshi OZAWA (JIRA) 

 [ 
https://issues.apache.org/jira/browse/YARN-1409?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tsuyoshi OZAWA updated YARN-1409:
-

Attachment: YARN-1409.2.patch

Sorry for delay. Updated a patch to handle RejectedExecutionException in the 
handler.

 NonAggregatingLogHandler can throw RejectedExecutionException
 -

 Key: YARN-1409
 URL: https://issues.apache.org/jira/browse/YARN-1409
 Project: Hadoop YARN
  Issue Type: Bug
Reporter: Tsuyoshi OZAWA
Assignee: Tsuyoshi OZAWA
 Attachments: YARN-1409.1.patch, YARN-1409.2.patch


 This problem is caused by handling APPLICATION_FINISHED events after calling 
 sched.shotdown() in NonAggregatingLongHandler#serviceStop(). 
 org.apache.hadoop.mapred.TestJobCleanup can fail because of 
 RejectedExecutionException by NonAggregatingLogHandler.
 {code}
 2013-11-13 10:53:06,970 FATAL [AsyncDispatcher event handler] 
 event.AsyncDispatcher (AsyncDispatcher.java:dispatch(166)) - Error in 
 dispatcher thread
 java.util.concurrent.RejectedExecutionException: Task 
 java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask@d51df63 
 rejected from 
 java.util.concurrent.ScheduledThreadPoolExecutor@7a20e369[Shutting down, pool 
 size = 4, active threads = 0, queued tasks = 7, completed tasks = 0]
 at 
 java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2048)
 at 
 java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:821)
 at 
 java.util.concurrent.ScheduledThreadPoolExecutor.delayedExecute(ScheduledThreadPoolExecutor.java:325)
 at 
 java.util.concurrent.ScheduledThreadPoolExecutor.schedule(ScheduledThreadPoolExecutor.java:530)
 at 
 org.apache.hadoop.yarn.server.nodemanager.containermanager.loghandler.NonAggregatingLogHandler.handle(NonAggregatingLogHandler.java:121)
 at 
 org.apache.hadoop.yarn.server.nodemanager.containermanager.loghandler.NonAggregatingLogHandler.handle(NonAggregatingLogHandler.java:49)
 at 
 org.apache.hadoop.yarn.event.AsyncDispatcher.dispatch(AsyncDispatcher.java:159)
 at 
 org.apache.hadoop.yarn.event.AsyncDispatcher$1.run(AsyncDispatcher.java:95)
 at java.lang.Thread.run(Thread.java:724)
 {code}



--
This message was sent by Atlassian JIRA
(v6.1#6144)