[jira] [Commented] (MAPREDUCE-6895) Job end notification not send due to YarnRuntimeException
[
https://issues.apache.org/jira/browse/MAPREDUCE-6895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16048711#comment-16048711
]
yunjiong zhao commented on MAPREDUCE-6895:
--
Thanks for your time review this patch.
> Job end notification not send due to YarnRuntimeException
> -
>
> Key: MAPREDUCE-6895
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6895
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: applicationmaster
>Affects Versions: 2.4.1, 2.8.0, 2.7.3
>Reporter: yunjiong zhao
>Assignee: yunjiong zhao
> Fix For: 2.9.0, 3.0.0-alpha4
>
> Attachments: MAPREDUCE-6895.001.patch, MAPREDUCE-6895.002.patch
>
>
> MRAppMaster.this.stop() throw out YarnRuntimeException as below log shows, it
> caused job end notification not send.
> {quote}
> 2017-05-24 12:14:02,165 WARN [Thread-693]
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Graceful stop failed
> org.apache.hadoop.yarn.exceptions.YarnRuntimeException:
> java.nio.channels.ClosedChannelException
> at
> org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler.handleEvent(JobHistoryEventHandler.java:531)
> at
> org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler.serviceStop(JobHistoryEventHandler.java:360)
> at
> org.apache.hadoop.service.AbstractService.stop(AbstractService.java:221)
> at
> org.apache.hadoop.service.ServiceOperations.stop(ServiceOperations.java:52)
> at
> org.apache.hadoop.service.ServiceOperations.stopQuietly(ServiceOperations.java:80)
> at
> org.apache.hadoop.service.CompositeService.stop(CompositeService.java:157)
> at
> org.apache.hadoop.service.CompositeService.serviceStop(CompositeService.java:131)
> at
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster.serviceStop(MRAppMaster.java:1476)
> at
> org.apache.hadoop.service.AbstractService.stop(AbstractService.java:221)
> at
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster.stop(MRAppMaster.java:1090)
> at
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster.shutDownJob(MRAppMaster.java:554)
> at
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster$JobFinishEventHandler$1.run(MRAppMaster.java:605)
> Caused by: java.nio.channels.ClosedChannelException
> at
> org.apache.hadoop.hdfs.DFSOutputStream.checkClosed(DFSOutputStream.java:1528)
> at org.apache.hadoop.fs.FSOutputSummer.write(FSOutputSummer.java:98)
> at
> org.apache.hadoop.fs.FSDataOutputStream$PositionCache.write(FSDataOutputStream.java:58)
> at java.io.DataOutputStream.write(DataOutputStream.java:107)
> at
> org.codehaus.jackson.impl.Utf8Generator._flushBuffer(Utf8Generator.java:1754)
> at
> org.codehaus.jackson.impl.Utf8Generator.flush(Utf8Generator.java:1088)
> at org.apache.avro.io.JsonEncoder.flush(JsonEncoder.java:67)
> at
> org.apache.hadoop.mapreduce.jobhistory.EventWriter.write(EventWriter.java:67)
> at
> org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler$MetaInfo.writeEvent(JobHistoryEventHandler.java:886)
> at
> org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler.handleEvent(JobHistoryEventHandler.java:520)
> ... 11 more
> 2017-05-24 12:14:02,165 INFO [Thread-693]
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Exiting MR AppMaster..GoodBye!
> {quote}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Updated] (MAPREDUCE-6895) Job end notification not send due to YarnRuntimeException
[
https://issues.apache.org/jira/browse/MAPREDUCE-6895?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Junping Du updated MAPREDUCE-6895:
--
Resolution: Fixed
Hadoop Flags: Reviewed
Fix Version/s: 3.0.0-alpha4
2.9.0
Status: Resolved (was: Patch Available)
> Job end notification not send due to YarnRuntimeException
> -
>
> Key: MAPREDUCE-6895
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6895
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: applicationmaster
>Affects Versions: 2.4.1, 2.8.0, 2.7.3
>Reporter: yunjiong zhao
>Assignee: yunjiong zhao
> Fix For: 2.9.0, 3.0.0-alpha4
>
> Attachments: MAPREDUCE-6895.001.patch, MAPREDUCE-6895.002.patch
>
>
> MRAppMaster.this.stop() throw out YarnRuntimeException as below log shows, it
> caused job end notification not send.
> {quote}
> 2017-05-24 12:14:02,165 WARN [Thread-693]
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Graceful stop failed
> org.apache.hadoop.yarn.exceptions.YarnRuntimeException:
> java.nio.channels.ClosedChannelException
> at
> org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler.handleEvent(JobHistoryEventHandler.java:531)
> at
> org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler.serviceStop(JobHistoryEventHandler.java:360)
> at
> org.apache.hadoop.service.AbstractService.stop(AbstractService.java:221)
> at
> org.apache.hadoop.service.ServiceOperations.stop(ServiceOperations.java:52)
> at
> org.apache.hadoop.service.ServiceOperations.stopQuietly(ServiceOperations.java:80)
> at
> org.apache.hadoop.service.CompositeService.stop(CompositeService.java:157)
> at
> org.apache.hadoop.service.CompositeService.serviceStop(CompositeService.java:131)
> at
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster.serviceStop(MRAppMaster.java:1476)
> at
> org.apache.hadoop.service.AbstractService.stop(AbstractService.java:221)
> at
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster.stop(MRAppMaster.java:1090)
> at
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster.shutDownJob(MRAppMaster.java:554)
> at
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster$JobFinishEventHandler$1.run(MRAppMaster.java:605)
> Caused by: java.nio.channels.ClosedChannelException
> at
> org.apache.hadoop.hdfs.DFSOutputStream.checkClosed(DFSOutputStream.java:1528)
> at org.apache.hadoop.fs.FSOutputSummer.write(FSOutputSummer.java:98)
> at
> org.apache.hadoop.fs.FSDataOutputStream$PositionCache.write(FSDataOutputStream.java:58)
> at java.io.DataOutputStream.write(DataOutputStream.java:107)
> at
> org.codehaus.jackson.impl.Utf8Generator._flushBuffer(Utf8Generator.java:1754)
> at
> org.codehaus.jackson.impl.Utf8Generator.flush(Utf8Generator.java:1088)
> at org.apache.avro.io.JsonEncoder.flush(JsonEncoder.java:67)
> at
> org.apache.hadoop.mapreduce.jobhistory.EventWriter.write(EventWriter.java:67)
> at
> org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler$MetaInfo.writeEvent(JobHistoryEventHandler.java:886)
> at
> org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler.handleEvent(JobHistoryEventHandler.java:520)
> ... 11 more
> 2017-05-24 12:14:02,165 INFO [Thread-693]
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Exiting MR AppMaster..GoodBye!
> {quote}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-6895) Job end notification not send due to YarnRuntimeException
[
https://issues.apache.org/jira/browse/MAPREDUCE-6895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16048705#comment-16048705
]
Junping Du commented on MAPREDUCE-6895:
---
Just filed MAPREDUCE-6897 and commit the latest patch to trunk and branch-2.
Thanks [~zhaoyunjiong] for patch contribution and [~raviprak] and
[~leftnoteasy] for review and comments!
> Job end notification not send due to YarnRuntimeException
> -
>
> Key: MAPREDUCE-6895
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6895
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: applicationmaster
>Affects Versions: 2.4.1, 2.8.0, 2.7.3
>Reporter: yunjiong zhao
>Assignee: yunjiong zhao
> Attachments: MAPREDUCE-6895.001.patch, MAPREDUCE-6895.002.patch
>
>
> MRAppMaster.this.stop() throw out YarnRuntimeException as below log shows, it
> caused job end notification not send.
> {quote}
> 2017-05-24 12:14:02,165 WARN [Thread-693]
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Graceful stop failed
> org.apache.hadoop.yarn.exceptions.YarnRuntimeException:
> java.nio.channels.ClosedChannelException
> at
> org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler.handleEvent(JobHistoryEventHandler.java:531)
> at
> org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler.serviceStop(JobHistoryEventHandler.java:360)
> at
> org.apache.hadoop.service.AbstractService.stop(AbstractService.java:221)
> at
> org.apache.hadoop.service.ServiceOperations.stop(ServiceOperations.java:52)
> at
> org.apache.hadoop.service.ServiceOperations.stopQuietly(ServiceOperations.java:80)
> at
> org.apache.hadoop.service.CompositeService.stop(CompositeService.java:157)
> at
> org.apache.hadoop.service.CompositeService.serviceStop(CompositeService.java:131)
> at
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster.serviceStop(MRAppMaster.java:1476)
> at
> org.apache.hadoop.service.AbstractService.stop(AbstractService.java:221)
> at
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster.stop(MRAppMaster.java:1090)
> at
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster.shutDownJob(MRAppMaster.java:554)
> at
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster$JobFinishEventHandler$1.run(MRAppMaster.java:605)
> Caused by: java.nio.channels.ClosedChannelException
> at
> org.apache.hadoop.hdfs.DFSOutputStream.checkClosed(DFSOutputStream.java:1528)
> at org.apache.hadoop.fs.FSOutputSummer.write(FSOutputSummer.java:98)
> at
> org.apache.hadoop.fs.FSDataOutputStream$PositionCache.write(FSDataOutputStream.java:58)
> at java.io.DataOutputStream.write(DataOutputStream.java:107)
> at
> org.codehaus.jackson.impl.Utf8Generator._flushBuffer(Utf8Generator.java:1754)
> at
> org.codehaus.jackson.impl.Utf8Generator.flush(Utf8Generator.java:1088)
> at org.apache.avro.io.JsonEncoder.flush(JsonEncoder.java:67)
> at
> org.apache.hadoop.mapreduce.jobhistory.EventWriter.write(EventWriter.java:67)
> at
> org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler$MetaInfo.writeEvent(JobHistoryEventHandler.java:886)
> at
> org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler.handleEvent(JobHistoryEventHandler.java:520)
> ... 11 more
> 2017-05-24 12:14:02,165 INFO [Thread-693]
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Exiting MR AppMaster..GoodBye!
> {quote}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Created] (MAPREDUCE-6897) Add Unit Test to make sure Job end notification get sent even appMaster stop get YarnRuntimeException
Junping Du created MAPREDUCE-6897: - Summary: Add Unit Test to make sure Job end notification get sent even appMaster stop get YarnRuntimeException Key: MAPREDUCE-6897 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6897 Project: Hadoop Map/Reduce Issue Type: Bug Reporter: Junping Du Priority: Minor In MAPREDUCE-6895, we fix the issue that Job end notification not send due to YarnRuntimeException throw in appMaster stop. We need to add unit test to make sure we won't run into the same issue again in future. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-6896) Document wrong spelling in usage of MapredTestDriver tools.
[
https://issues.apache.org/jira/browse/MAPREDUCE-6896?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16048567#comment-16048567
]
LiXin Ge commented on MAPREDUCE-6896:
-
Thanks [~templedf], that's my pleasure.
> Document wrong spelling in usage of MapredTestDriver tools.
> ---
>
> Key: MAPREDUCE-6896
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6896
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: documentation
>Affects Versions: 3.0.0-alpha3
>Reporter: LiXin Ge
>Assignee: LiXin Ge
> Labels: easyfix
> Fix For: 3.0.0-alpha4
>
> Attachments: MAPREDUCE-6896.001.patch
>
>
> When the user run performance test of MapredTestDriver, a spelling mistake
> seems exposed to user:
> {quote}
> ./hadoop org.apache.hadoop.test.MapredTestDriver
> An example program must be given as the first argument.
> Valid program names are:
> ..
> timelineperformance: A job that launches mappers to test *timline* service
> performance.
> {quote}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-6895) Job end notification not send due to YarnRuntimeException
[
https://issues.apache.org/jira/browse/MAPREDUCE-6895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16048494#comment-16048494
]
Hudson commented on MAPREDUCE-6895:
---
SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #11864 (See
[https://builds.apache.org/job/Hadoop-trunk-Commit/11864/])
MAPREDUCE-6895. Job end notification not send due to (junping_du: rev
6ed54f3439ea9c7af6bf129ebe1938380febb5e2)
* (edit)
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/MRAppMaster.java
* (edit)
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/JobEndNotifier.java
> Job end notification not send due to YarnRuntimeException
> -
>
> Key: MAPREDUCE-6895
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6895
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: applicationmaster
>Affects Versions: 2.4.1, 2.8.0, 2.7.3
>Reporter: yunjiong zhao
>Assignee: yunjiong zhao
> Attachments: MAPREDUCE-6895.001.patch, MAPREDUCE-6895.002.patch
>
>
> MRAppMaster.this.stop() throw out YarnRuntimeException as below log shows, it
> caused job end notification not send.
> {quote}
> 2017-05-24 12:14:02,165 WARN [Thread-693]
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Graceful stop failed
> org.apache.hadoop.yarn.exceptions.YarnRuntimeException:
> java.nio.channels.ClosedChannelException
> at
> org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler.handleEvent(JobHistoryEventHandler.java:531)
> at
> org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler.serviceStop(JobHistoryEventHandler.java:360)
> at
> org.apache.hadoop.service.AbstractService.stop(AbstractService.java:221)
> at
> org.apache.hadoop.service.ServiceOperations.stop(ServiceOperations.java:52)
> at
> org.apache.hadoop.service.ServiceOperations.stopQuietly(ServiceOperations.java:80)
> at
> org.apache.hadoop.service.CompositeService.stop(CompositeService.java:157)
> at
> org.apache.hadoop.service.CompositeService.serviceStop(CompositeService.java:131)
> at
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster.serviceStop(MRAppMaster.java:1476)
> at
> org.apache.hadoop.service.AbstractService.stop(AbstractService.java:221)
> at
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster.stop(MRAppMaster.java:1090)
> at
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster.shutDownJob(MRAppMaster.java:554)
> at
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster$JobFinishEventHandler$1.run(MRAppMaster.java:605)
> Caused by: java.nio.channels.ClosedChannelException
> at
> org.apache.hadoop.hdfs.DFSOutputStream.checkClosed(DFSOutputStream.java:1528)
> at org.apache.hadoop.fs.FSOutputSummer.write(FSOutputSummer.java:98)
> at
> org.apache.hadoop.fs.FSDataOutputStream$PositionCache.write(FSDataOutputStream.java:58)
> at java.io.DataOutputStream.write(DataOutputStream.java:107)
> at
> org.codehaus.jackson.impl.Utf8Generator._flushBuffer(Utf8Generator.java:1754)
> at
> org.codehaus.jackson.impl.Utf8Generator.flush(Utf8Generator.java:1088)
> at org.apache.avro.io.JsonEncoder.flush(JsonEncoder.java:67)
> at
> org.apache.hadoop.mapreduce.jobhistory.EventWriter.write(EventWriter.java:67)
> at
> org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler$MetaInfo.writeEvent(JobHistoryEventHandler.java:886)
> at
> org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler.handleEvent(JobHistoryEventHandler.java:520)
> ... 11 more
> 2017-05-24 12:14:02,165 INFO [Thread-693]
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Exiting MR AppMaster..GoodBye!
> {quote}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-6896) Document wrong spelling in usage of MapredTestDriver tools.
[
https://issues.apache.org/jira/browse/MAPREDUCE-6896?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16048386#comment-16048386
]
Hudson commented on MAPREDUCE-6896:
---
SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #11863 (See
[https://builds.apache.org/job/Hadoop-trunk-Commit/11863/])
MAPREDUCE-6896. Document wrong spelling in usage of MapredTestDriver (templedf:
rev 036a24bab3f3ff89e3d52c5487fce0525c0d1d78)
* (edit)
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/test/MapredTestDriver.java
* (edit)
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/client/api/impl/TestTimelineClient.java
* (edit)
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/main/java/org/apache/hadoop/yarn/client/api/impl/YarnClientImpl.java
> Document wrong spelling in usage of MapredTestDriver tools.
> ---
>
> Key: MAPREDUCE-6896
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6896
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: documentation
>Affects Versions: 3.0.0-alpha3
>Reporter: LiXin Ge
>Assignee: LiXin Ge
> Labels: easyfix
> Fix For: 3.0.0-alpha4
>
> Attachments: MAPREDUCE-6896.001.patch
>
>
> When the user run performance test of MapredTestDriver, a spelling mistake
> seems exposed to user:
> {quote}
> ./hadoop org.apache.hadoop.test.MapredTestDriver
> An example program must be given as the first argument.
> Valid program names are:
> ..
> timelineperformance: A job that launches mappers to test *timline* service
> performance.
> {quote}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Updated] (MAPREDUCE-6896) Document wrong spelling in usage of MapredTestDriver tools.
[
https://issues.apache.org/jira/browse/MAPREDUCE-6896?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Daniel Templeton updated MAPREDUCE-6896:
Resolution: Fixed
Hadoop Flags: Reviewed
Fix Version/s: 3.0.0-alpha4
Status: Resolved (was: Patch Available)
Thanks for the patch, [~GeLiXin]. Committed to trunk.
> Document wrong spelling in usage of MapredTestDriver tools.
> ---
>
> Key: MAPREDUCE-6896
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6896
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: documentation
>Affects Versions: 3.0.0-alpha3
>Reporter: LiXin Ge
>Assignee: LiXin Ge
> Labels: easyfix
> Fix For: 3.0.0-alpha4
>
> Attachments: MAPREDUCE-6896.001.patch
>
>
> When the user run performance test of MapredTestDriver, a spelling mistake
> seems exposed to user:
> {quote}
> ./hadoop org.apache.hadoop.test.MapredTestDriver
> An example program must be given as the first argument.
> Valid program names are:
> ..
> timelineperformance: A job that launches mappers to test *timline* service
> performance.
> {quote}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-6895) Job end notification not send due to YarnRuntimeException
[
https://issues.apache.org/jira/browse/MAPREDUCE-6895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16047518#comment-16047518
]
Ravi Prakash commented on MAPREDUCE-6895:
-
Sure Junping.
> Job end notification not send due to YarnRuntimeException
> -
>
> Key: MAPREDUCE-6895
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6895
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: applicationmaster
>Affects Versions: 2.4.1, 2.8.0, 2.7.3
>Reporter: yunjiong zhao
>Assignee: yunjiong zhao
> Attachments: MAPREDUCE-6895.001.patch, MAPREDUCE-6895.002.patch
>
>
> MRAppMaster.this.stop() throw out YarnRuntimeException as below log shows, it
> caused job end notification not send.
> {quote}
> 2017-05-24 12:14:02,165 WARN [Thread-693]
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Graceful stop failed
> org.apache.hadoop.yarn.exceptions.YarnRuntimeException:
> java.nio.channels.ClosedChannelException
> at
> org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler.handleEvent(JobHistoryEventHandler.java:531)
> at
> org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler.serviceStop(JobHistoryEventHandler.java:360)
> at
> org.apache.hadoop.service.AbstractService.stop(AbstractService.java:221)
> at
> org.apache.hadoop.service.ServiceOperations.stop(ServiceOperations.java:52)
> at
> org.apache.hadoop.service.ServiceOperations.stopQuietly(ServiceOperations.java:80)
> at
> org.apache.hadoop.service.CompositeService.stop(CompositeService.java:157)
> at
> org.apache.hadoop.service.CompositeService.serviceStop(CompositeService.java:131)
> at
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster.serviceStop(MRAppMaster.java:1476)
> at
> org.apache.hadoop.service.AbstractService.stop(AbstractService.java:221)
> at
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster.stop(MRAppMaster.java:1090)
> at
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster.shutDownJob(MRAppMaster.java:554)
> at
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster$JobFinishEventHandler$1.run(MRAppMaster.java:605)
> Caused by: java.nio.channels.ClosedChannelException
> at
> org.apache.hadoop.hdfs.DFSOutputStream.checkClosed(DFSOutputStream.java:1528)
> at org.apache.hadoop.fs.FSOutputSummer.write(FSOutputSummer.java:98)
> at
> org.apache.hadoop.fs.FSDataOutputStream$PositionCache.write(FSDataOutputStream.java:58)
> at java.io.DataOutputStream.write(DataOutputStream.java:107)
> at
> org.codehaus.jackson.impl.Utf8Generator._flushBuffer(Utf8Generator.java:1754)
> at
> org.codehaus.jackson.impl.Utf8Generator.flush(Utf8Generator.java:1088)
> at org.apache.avro.io.JsonEncoder.flush(JsonEncoder.java:67)
> at
> org.apache.hadoop.mapreduce.jobhistory.EventWriter.write(EventWriter.java:67)
> at
> org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler$MetaInfo.writeEvent(JobHistoryEventHandler.java:886)
> at
> org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler.handleEvent(JobHistoryEventHandler.java:520)
> ... 11 more
> 2017-05-24 12:14:02,165 INFO [Thread-693]
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Exiting MR AppMaster..GoodBye!
> {quote}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
