[jira] [Commented] (YARN-3090) DeletionService can silently ignore deletion task failures
[ https://issues.apache.org/jira/browse/YARN-3090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14323906#comment-14323906 ] Hudson commented on YARN-3090: -- SUCCESS: Integrated in Hadoop-Hdfs-trunk-Java8 #97 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/97/]) YARN-3090. DeletionService can silently ignore deletion task failures. Contributed by Varun Saxena (jlowe: rev 4eb5f7fa32bab1b9ce3fb58eca51e2cd2e194cd5) * hadoop-yarn-project/CHANGES.txt * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/DeletionService.java > DeletionService can silently ignore deletion task failures > -- > > Key: YARN-3090 > URL: https://issues.apache.org/jira/browse/YARN-3090 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Affects Versions: 2.1.1-beta >Reporter: Jason Lowe >Assignee: Varun Saxena > Fix For: 2.7.0 > > Attachments: YARN-3090.001.patch, YARN-3090.002.patch, > YARN-3090.003.patch, YARN-3090.04.patch > > > If a non-I/O exception occurs while the DeletionService is executing a > deletion task then it will be silently ignored. The exception bubbles up to > the thread workers of the ScheduledThreadPoolExecutor which simply attaches > the throwable to the Future that was returned when the task was scheduled. > However the thread pool is used as a fire-and-forget pool, so nothing ever > looks at the Future and therefore the exception is never logged. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3090) DeletionService can silently ignore deletion task failures
[ https://issues.apache.org/jira/browse/YARN-3090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14316357#comment-14316357 ] Hudson commented on YARN-3090: -- FAILURE: Integrated in Hadoop-Mapreduce-trunk #2052 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2052/]) YARN-3090. DeletionService can silently ignore deletion task failures. Contributed by Varun Saxena (jlowe: rev 4eb5f7fa32bab1b9ce3fb58eca51e2cd2e194cd5) * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/DeletionService.java * hadoop-yarn-project/CHANGES.txt > DeletionService can silently ignore deletion task failures > -- > > Key: YARN-3090 > URL: https://issues.apache.org/jira/browse/YARN-3090 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Affects Versions: 2.1.1-beta >Reporter: Jason Lowe >Assignee: Varun Saxena > Fix For: 2.7.0 > > Attachments: YARN-3090.001.patch, YARN-3090.002.patch, > YARN-3090.003.patch, YARN-3090.04.patch > > > If a non-I/O exception occurs while the DeletionService is executing a > deletion task then it will be silently ignored. The exception bubbles up to > the thread workers of the ScheduledThreadPoolExecutor which simply attaches > the throwable to the Future that was returned when the task was scheduled. > However the thread pool is used as a fire-and-forget pool, so nothing ever > looks at the Future and therefore the exception is never logged. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3090) DeletionService can silently ignore deletion task failures
[ https://issues.apache.org/jira/browse/YARN-3090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14316328#comment-14316328 ] Hudson commented on YARN-3090: -- FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #102 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/102/]) YARN-3090. DeletionService can silently ignore deletion task failures. Contributed by Varun Saxena (jlowe: rev 4eb5f7fa32bab1b9ce3fb58eca51e2cd2e194cd5) * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/DeletionService.java * hadoop-yarn-project/CHANGES.txt > DeletionService can silently ignore deletion task failures > -- > > Key: YARN-3090 > URL: https://issues.apache.org/jira/browse/YARN-3090 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Affects Versions: 2.1.1-beta >Reporter: Jason Lowe >Assignee: Varun Saxena > Fix For: 2.7.0 > > Attachments: YARN-3090.001.patch, YARN-3090.002.patch, > YARN-3090.003.patch, YARN-3090.04.patch > > > If a non-I/O exception occurs while the DeletionService is executing a > deletion task then it will be silently ignored. The exception bubbles up to > the thread workers of the ScheduledThreadPoolExecutor which simply attaches > the throwable to the Future that was returned when the task was scheduled. > However the thread pool is used as a fire-and-forget pool, so nothing ever > looks at the Future and therefore the exception is never logged. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3090) DeletionService can silently ignore deletion task failures
[ https://issues.apache.org/jira/browse/YARN-3090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14316274#comment-14316274 ] Hudson commented on YARN-3090: -- FAILURE: Integrated in Hadoop-Hdfs-trunk #2033 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/2033/]) YARN-3090. DeletionService can silently ignore deletion task failures. Contributed by Varun Saxena (jlowe: rev 4eb5f7fa32bab1b9ce3fb58eca51e2cd2e194cd5) * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/DeletionService.java * hadoop-yarn-project/CHANGES.txt > DeletionService can silently ignore deletion task failures > -- > > Key: YARN-3090 > URL: https://issues.apache.org/jira/browse/YARN-3090 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Affects Versions: 2.1.1-beta >Reporter: Jason Lowe >Assignee: Varun Saxena > Fix For: 2.7.0 > > Attachments: YARN-3090.001.patch, YARN-3090.002.patch, > YARN-3090.003.patch, YARN-3090.04.patch > > > If a non-I/O exception occurs while the DeletionService is executing a > deletion task then it will be silently ignored. The exception bubbles up to > the thread workers of the ScheduledThreadPoolExecutor which simply attaches > the throwable to the Future that was returned when the task was scheduled. > However the thread pool is used as a fire-and-forget pool, so nothing ever > looks at the Future and therefore the exception is never logged. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3090) DeletionService can silently ignore deletion task failures
[ https://issues.apache.org/jira/browse/YARN-3090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14316006#comment-14316006 ] Hudson commented on YARN-3090: -- FAILURE: Integrated in Hadoop-Yarn-trunk #835 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/835/]) YARN-3090. DeletionService can silently ignore deletion task failures. Contributed by Varun Saxena (jlowe: rev 4eb5f7fa32bab1b9ce3fb58eca51e2cd2e194cd5) * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/DeletionService.java * hadoop-yarn-project/CHANGES.txt > DeletionService can silently ignore deletion task failures > -- > > Key: YARN-3090 > URL: https://issues.apache.org/jira/browse/YARN-3090 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Affects Versions: 2.1.1-beta >Reporter: Jason Lowe >Assignee: Varun Saxena > Fix For: 2.7.0 > > Attachments: YARN-3090.001.patch, YARN-3090.002.patch, > YARN-3090.003.patch, YARN-3090.04.patch > > > If a non-I/O exception occurs while the DeletionService is executing a > deletion task then it will be silently ignored. The exception bubbles up to > the thread workers of the ScheduledThreadPoolExecutor which simply attaches > the throwable to the Future that was returned when the task was scheduled. > However the thread pool is used as a fire-and-forget pool, so nothing ever > looks at the Future and therefore the exception is never logged. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3090) DeletionService can silently ignore deletion task failures
[ https://issues.apache.org/jira/browse/YARN-3090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14315984#comment-14315984 ] Hudson commented on YARN-3090: -- FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #101 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/101/]) YARN-3090. DeletionService can silently ignore deletion task failures. Contributed by Varun Saxena (jlowe: rev 4eb5f7fa32bab1b9ce3fb58eca51e2cd2e194cd5) * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/DeletionService.java * hadoop-yarn-project/CHANGES.txt > DeletionService can silently ignore deletion task failures > -- > > Key: YARN-3090 > URL: https://issues.apache.org/jira/browse/YARN-3090 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Affects Versions: 2.1.1-beta >Reporter: Jason Lowe >Assignee: Varun Saxena > Fix For: 2.7.0 > > Attachments: YARN-3090.001.patch, YARN-3090.002.patch, > YARN-3090.003.patch, YARN-3090.04.patch > > > If a non-I/O exception occurs while the DeletionService is executing a > deletion task then it will be silently ignored. The exception bubbles up to > the thread workers of the ScheduledThreadPoolExecutor which simply attaches > the throwable to the Future that was returned when the task was scheduled. > However the thread pool is used as a fire-and-forget pool, so nothing ever > looks at the Future and therefore the exception is never logged. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3090) DeletionService can silently ignore deletion task failures
[ https://issues.apache.org/jira/browse/YARN-3090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14314470#comment-14314470 ] Hudson commented on YARN-3090: -- FAILURE: Integrated in Hadoop-trunk-Commit #7062 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/7062/]) YARN-3090. DeletionService can silently ignore deletion task failures. Contributed by Varun Saxena (jlowe: rev 4eb5f7fa32bab1b9ce3fb58eca51e2cd2e194cd5) * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/DeletionService.java * hadoop-yarn-project/CHANGES.txt > DeletionService can silently ignore deletion task failures > -- > > Key: YARN-3090 > URL: https://issues.apache.org/jira/browse/YARN-3090 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Affects Versions: 2.1.1-beta >Reporter: Jason Lowe >Assignee: Varun Saxena > Fix For: 2.7.0 > > Attachments: YARN-3090.001.patch, YARN-3090.002.patch, > YARN-3090.003.patch, YARN-3090.04.patch > > > If a non-I/O exception occurs while the DeletionService is executing a > deletion task then it will be silently ignored. The exception bubbles up to > the thread workers of the ScheduledThreadPoolExecutor which simply attaches > the throwable to the Future that was returned when the task was scheduled. > However the thread pool is used as a fire-and-forget pool, so nothing ever > looks at the Future and therefore the exception is never logged. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3090) DeletionService can silently ignore deletion task failures
[ https://issues.apache.org/jira/browse/YARN-3090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14314445#comment-14314445 ] Jason Lowe commented on YARN-3090: -- +1 lgtm. Committing this. > DeletionService can silently ignore deletion task failures > -- > > Key: YARN-3090 > URL: https://issues.apache.org/jira/browse/YARN-3090 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Affects Versions: 2.1.1-beta >Reporter: Jason Lowe >Assignee: Varun Saxena > Attachments: YARN-3090.001.patch, YARN-3090.002.patch, > YARN-3090.003.patch, YARN-3090.04.patch > > > If a non-I/O exception occurs while the DeletionService is executing a > deletion task then it will be silently ignored. The exception bubbles up to > the thread workers of the ScheduledThreadPoolExecutor which simply attaches > the throwable to the Future that was returned when the task was scheduled. > However the thread pool is used as a fire-and-forget pool, so nothing ever > looks at the Future and therefore the exception is never logged. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3090) DeletionService can silently ignore deletion task failures
[
https://issues.apache.org/jira/browse/YARN-3090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14314385#comment-14314385
]
Hadoop QA commented on YARN-3090:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12697794/YARN-3090.04.patch
against trunk revision e0ec071.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/6576//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/6576//console
This message is automatically generated.
> DeletionService can silently ignore deletion task failures
> --
>
> Key: YARN-3090
> URL: https://issues.apache.org/jira/browse/YARN-3090
> Project: Hadoop YARN
> Issue Type: Bug
> Components: nodemanager
>Affects Versions: 2.1.1-beta
>Reporter: Jason Lowe
>Assignee: Varun Saxena
> Attachments: YARN-3090.001.patch, YARN-3090.002.patch,
> YARN-3090.003.patch, YARN-3090.04.patch
>
>
> If a non-I/O exception occurs while the DeletionService is executing a
> deletion task then it will be silently ignored. The exception bubbles up to
> the thread workers of the ScheduledThreadPoolExecutor which simply attaches
> the throwable to the Future that was returned when the task was scheduled.
> However the thread pool is used as a fire-and-forget pool, so nothing ever
> looks at the Future and therefore the exception is never logged.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-3090) DeletionService can silently ignore deletion task failures
[ https://issues.apache.org/jira/browse/YARN-3090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14314344#comment-14314344 ] Varun Saxena commented on YARN-3090: Able to kick it. > DeletionService can silently ignore deletion task failures > -- > > Key: YARN-3090 > URL: https://issues.apache.org/jira/browse/YARN-3090 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Affects Versions: 2.1.1-beta >Reporter: Jason Lowe >Assignee: Varun Saxena > Attachments: YARN-3090.001.patch, YARN-3090.002.patch, > YARN-3090.003.patch, YARN-3090.04.patch > > > If a non-I/O exception occurs while the DeletionService is executing a > deletion task then it will be silently ignored. The exception bubbles up to > the thread workers of the ScheduledThreadPoolExecutor which simply attaches > the throwable to the Future that was returned when the task was scheduled. > However the thread pool is used as a fire-and-forget pool, so nothing ever > looks at the Future and therefore the exception is never logged. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3090) DeletionService can silently ignore deletion task failures
[ https://issues.apache.org/jira/browse/YARN-3090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14314328#comment-14314328 ] Varun Saxena commented on YARN-3090: Weird. Jenkins is not getting kicked. Can somebody do that manually ? > DeletionService can silently ignore deletion task failures > -- > > Key: YARN-3090 > URL: https://issues.apache.org/jira/browse/YARN-3090 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Affects Versions: 2.1.1-beta >Reporter: Jason Lowe >Assignee: Varun Saxena > Attachments: YARN-3090.001.patch, YARN-3090.002.patch, > YARN-3090.003.patch, YARN-3090.04.patch > > > If a non-I/O exception occurs while the DeletionService is executing a > deletion task then it will be silently ignored. The exception bubbles up to > the thread workers of the ScheduledThreadPoolExecutor which simply attaches > the throwable to the Future that was returned when the task was scheduled. > However the thread pool is used as a fire-and-forget pool, so nothing ever > looks at the Future and therefore the exception is never logged. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3090) DeletionService can silently ignore deletion task failures
[
https://issues.apache.org/jira/browse/YARN-3090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14313211#comment-14313211
]
Hadoop QA commented on YARN-3090:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12697529/YARN-3090.004.patch
against trunk revision aab459c.
{color:red}-1 @author{color}. The patch appears to contain @author tags
which the Hadoop community has agreed to not allow in code contributions.
{color:green}+1 tests included{color}. The patch appears to include new
or modified test files.
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/6561//console
This message is automatically generated.
> DeletionService can silently ignore deletion task failures
> --
>
> Key: YARN-3090
> URL: https://issues.apache.org/jira/browse/YARN-3090
> Project: Hadoop YARN
> Issue Type: Bug
> Components: nodemanager
>Affects Versions: 2.1.1-beta
>Reporter: Jason Lowe
>Assignee: Varun Saxena
> Attachments: YARN-3090.001.patch, YARN-3090.002.patch,
> YARN-3090.003.patch, YARN-3090.004.patch
>
>
> If a non-I/O exception occurs while the DeletionService is executing a
> deletion task then it will be silently ignored. The exception bubbles up to
> the thread workers of the ScheduledThreadPoolExecutor which simply attaches
> the throwable to the Future that was returned when the task was scheduled.
> However the thread pool is used as a fire-and-forget pool, so nothing ever
> looks at the Future and therefore the exception is never logged.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-3090) DeletionService can silently ignore deletion task failures
[ https://issues.apache.org/jira/browse/YARN-3090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14307207#comment-14307207 ] Varun Saxena commented on YARN-3090: [~jlowe], oh is it so. To verify if it works, had written a small program to verify it and it worked. Probably because I directly threw an exception in run() without any processing. Will modify as per your suggestion. > DeletionService can silently ignore deletion task failures > -- > > Key: YARN-3090 > URL: https://issues.apache.org/jira/browse/YARN-3090 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Affects Versions: 2.1.1-beta >Reporter: Jason Lowe >Assignee: Varun Saxena > Attachments: YARN-3090.001.patch, YARN-3090.002.patch > > > If a non-I/O exception occurs while the DeletionService is executing a > deletion task then it will be silently ignored. The exception bubbles up to > the thread workers of the ScheduledThreadPoolExecutor which simply attaches > the throwable to the Future that was returned when the task was scheduled. > However the thread pool is used as a fire-and-forget pool, so nothing ever > looks at the Future and therefore the exception is never logged. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3090) DeletionService can silently ignore deletion task failures
[ https://issues.apache.org/jira/browse/YARN-3090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14307204#comment-14307204 ] Varun Saxena commented on YARN-3090: [~jlowe], oh is it so. To verify if it works, had written a small program to verify it and it worked. Probably because I directly threw an exception in run() without any processing. Will modify as per your suggestion. > DeletionService can silently ignore deletion task failures > -- > > Key: YARN-3090 > URL: https://issues.apache.org/jira/browse/YARN-3090 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Affects Versions: 2.1.1-beta >Reporter: Jason Lowe >Assignee: Varun Saxena > Attachments: YARN-3090.001.patch, YARN-3090.002.patch > > > If a non-I/O exception occurs while the DeletionService is executing a > deletion task then it will be silently ignored. The exception bubbles up to > the thread workers of the ScheduledThreadPoolExecutor which simply attaches > the throwable to the Future that was returned when the task was scheduled. > However the thread pool is used as a fire-and-forget pool, so nothing ever > looks at the Future and therefore the exception is never logged. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3090) DeletionService can silently ignore deletion task failures
[
https://issues.apache.org/jira/browse/YARN-3090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14305514#comment-14305514
]
Jason Lowe commented on YARN-3090:
--
Thanks for the patch, Varun! I kicked the tires on this patch and discovered
it doesn't work. I forced the container executor to throw a runtime exception
but it wasn't logged.
It turns out I was mistaken in my earlier analysis. The afterExecute method
normally does not receive any exception because it only reports exceptions the
_FutureTask run method_ throws, and normally that doesn't throw anything. It
stores any exception thrown by the underlying Runnable and almost always throws
nothing itself. Therefore afterExecute sees no exception even if one escaped
during deletion, and we log nothing. I mistakenly assumed that the thread pool
executor would pass any exception stored by the FutureTask to afterExecute, but
that's not the case.
To log the escaped exceptions we'd need to do something like this in
afterExecute:
{code}
@Override
protected void afterExecute(Runnable task, Throwable exception) {
if (task instanceof FutureTask) {
FutureTask futureTask = (FutureTask) task;
if (!futureTask.isCancelled()) {
try {
futureTask.get();
} catch (ExecutionException e) {
exception = e.getCause();
}
}
}
if (exception != null) {
LOG.warn("Exception during execution of task in DeletionService",
exception);
}
}
{code}
A couple of other nits on the patch:
* The new class should be private
* The log message should be an ERROR rather than a WARN since this should only
occur when an exception escaped the exception handling already in place for
deletion. Those kinds of exceptions are usually pretty bad (like NPEs).
> DeletionService can silently ignore deletion task failures
> --
>
> Key: YARN-3090
> URL: https://issues.apache.org/jira/browse/YARN-3090
> Project: Hadoop YARN
> Issue Type: Bug
> Components: nodemanager
>Affects Versions: 2.1.1-beta
>Reporter: Jason Lowe
>Assignee: Varun Saxena
> Attachments: YARN-3090.001.patch, YARN-3090.002.patch
>
>
> If a non-I/O exception occurs while the DeletionService is executing a
> deletion task then it will be silently ignored. The exception bubbles up to
> the thread workers of the ScheduledThreadPoolExecutor which simply attaches
> the throwable to the Future that was returned when the task was scheduled.
> However the thread pool is used as a fire-and-forget pool, so nothing ever
> looks at the Future and therefore the exception is never logged.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-3090) DeletionService can silently ignore deletion task failures
[ https://issues.apache.org/jira/browse/YARN-3090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14304554#comment-14304554 ] Varun Saxena commented on YARN-3090: Kindly review this issue. Release Audit warning unrelated > DeletionService can silently ignore deletion task failures > -- > > Key: YARN-3090 > URL: https://issues.apache.org/jira/browse/YARN-3090 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Affects Versions: 2.1.1-beta >Reporter: Jason Lowe >Assignee: Varun Saxena > Attachments: YARN-3090.001.patch, YARN-3090.002.patch > > > If a non-I/O exception occurs while the DeletionService is executing a > deletion task then it will be silently ignored. The exception bubbles up to > the thread workers of the ScheduledThreadPoolExecutor which simply attaches > the throwable to the Future that was returned when the task was scheduled. > However the thread pool is used as a fire-and-forget pool, so nothing ever > looks at the Future and therefore the exception is never logged. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3090) DeletionService can silently ignore deletion task failures
[
https://issues.apache.org/jira/browse/YARN-3090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14299714#comment-14299714
]
Hadoop QA commented on YARN-3090:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12695736/YARN-3090.002.patch
against trunk revision 26c2de3.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:red}-1 release audit{color}. The applied patch generated 1
release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/6476//testReport/
Release audit warnings:
https://builds.apache.org/job/PreCommit-YARN-Build/6476//artifact/patchprocess/patchReleaseAuditProblems.txt
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/6476//console
This message is automatically generated.
> DeletionService can silently ignore deletion task failures
> --
>
> Key: YARN-3090
> URL: https://issues.apache.org/jira/browse/YARN-3090
> Project: Hadoop YARN
> Issue Type: Bug
> Components: nodemanager
>Affects Versions: 2.1.1-beta
>Reporter: Jason Lowe
>Assignee: Varun Saxena
> Attachments: YARN-3090.001.patch, YARN-3090.002.patch
>
>
> If a non-I/O exception occurs while the DeletionService is executing a
> deletion task then it will be silently ignored. The exception bubbles up to
> the thread workers of the ScheduledThreadPoolExecutor which simply attaches
> the throwable to the Future that was returned when the task was scheduled.
> However the thread pool is used as a fire-and-forget pool, so nothing ever
> looks at the Future and therefore the exception is never logged.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-3090) DeletionService can silently ignore deletion task failures
[
https://issues.apache.org/jira/browse/YARN-3090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14299700#comment-14299700
]
Hadoop QA commented on YARN-3090:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12695733/YARN-3090.001.patch
against trunk revision 26c2de3.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:red}-1 findbugs{color}. The patch appears to introduce 1 new
Findbugs (version 2.0.3) warnings.
{color:red}-1 release audit{color}. The applied patch generated 1
release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/6475//testReport/
Release audit warnings:
https://builds.apache.org/job/PreCommit-YARN-Build/6475//artifact/patchprocess/patchReleaseAuditProblems.txt
Findbugs warnings:
https://builds.apache.org/job/PreCommit-YARN-Build/6475//artifact/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-nodemanager.html
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/6475//console
This message is automatically generated.
> DeletionService can silently ignore deletion task failures
> --
>
> Key: YARN-3090
> URL: https://issues.apache.org/jira/browse/YARN-3090
> Project: Hadoop YARN
> Issue Type: Bug
> Components: nodemanager
>Affects Versions: 2.1.1-beta
>Reporter: Jason Lowe
>Assignee: Varun Saxena
> Attachments: YARN-3090.001.patch
>
>
> If a non-I/O exception occurs while the DeletionService is executing a
> deletion task then it will be silently ignored. The exception bubbles up to
> the thread workers of the ScheduledThreadPoolExecutor which simply attaches
> the throwable to the Future that was returned when the task was scheduled.
> However the thread pool is used as a fire-and-forget pool, so nothing ever
> looks at the Future and therefore the exception is never logged.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-3090) DeletionService can silently ignore deletion task failures
[ https://issues.apache.org/jira/browse/YARN-3090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14294144#comment-14294144 ] Jason Lowe commented on YARN-3090: -- It could be with some rework. The hadoop-common class would either have to use its own logger or be parameterized with the logger to use. Since this is related to the DeletionService arguably it should use that logger. Don't know if we need to overthink this. The derived class in question is just a few lines of code, so not sure we need to put another class in hadoop-common for this one case. If we know there are many places it can/should be used then it's probably worth it, otherwise I say we wait until we come up with another use case and commonize the code when it makes sense (i.e.: the YAGNI principal). > DeletionService can silently ignore deletion task failures > -- > > Key: YARN-3090 > URL: https://issues.apache.org/jira/browse/YARN-3090 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Affects Versions: 2.1.1-beta >Reporter: Jason Lowe >Assignee: Varun Saxena > > If a non-I/O exception occurs while the DeletionService is executing a > deletion task then it will be silently ignored. The exception bubbles up to > the thread workers of the ScheduledThreadPoolExecutor which simply attaches > the throwable to the Future that was returned when the task was scheduled. > However the thread pool is used as a fire-and-forget pool, so nothing ever > looks at the Future and therefore the exception is never logged. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3090) DeletionService can silently ignore deletion task failures
[ https://issues.apache.org/jira/browse/YARN-3090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14294124#comment-14294124 ] Varun Saxena commented on YARN-3090: Should this derived class be moved to hadoop-common ? > DeletionService can silently ignore deletion task failures > -- > > Key: YARN-3090 > URL: https://issues.apache.org/jira/browse/YARN-3090 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Affects Versions: 2.1.1-beta >Reporter: Jason Lowe >Assignee: Varun Saxena > > If a non-I/O exception occurs while the DeletionService is executing a > deletion task then it will be silently ignored. The exception bubbles up to > the thread workers of the ScheduledThreadPoolExecutor which simply attaches > the throwable to the Future that was returned when the task was scheduled. > However the thread pool is used as a fire-and-forget pool, so nothing ever > looks at the Future and therefore the exception is never logged. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3090) DeletionService can silently ignore deletion task failures
[ https://issues.apache.org/jira/browse/YARN-3090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14288218#comment-14288218 ] Jason Lowe commented on YARN-3090: -- An easy way to at least log a message when something terrible occurrs to a deletion task is to use a derived ScheduledThreadPoolExecutor that overrides the afterExecute method to log any throwable that was associated with the task. > DeletionService can silently ignore deletion task failures > -- > > Key: YARN-3090 > URL: https://issues.apache.org/jira/browse/YARN-3090 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Affects Versions: 2.1.1-beta >Reporter: Jason Lowe > > If a non-I/O exception occurs while the DeletionService is executing a > deletion task then it will be silently ignored. The exception bubbles up to > the thread workers of the ScheduledThreadPoolExecutor which simply attaches > the throwable to the Future that was returned when the task was scheduled. > However the thread pool is used as a fire-and-forget pool, so nothing ever > looks at the Future and therefore the exception is never logged. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
