[
https://issues.apache.org/jira/browse/MAPREDUCE-3614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13216227#comment-13216227
]
Daryn Sharp commented on MAPREDUCE-3614:
----------------------------------------
Is there a compelling need to maintain a parallel set of JobIds? Why not check
the {{fileMap}} values to see if the writer is open? The event construction
looks a bit odd, but I don't know much about event injection.
I'd suggest reverting all the fs manipulation code, and try this:
{code}
Runtime.getRuntime().addShutdownHook(
- new CompositeServiceShutdownHook(appMaster));
+ new CompositeServiceShutdownHook(appMaster) {
+ @Override
+ public void run() {
+ super.run();
+ try {
+ FileSystem.closeAll();
+ } catch (IOException e) {
+ // ignore?
+ }
+ }
+ });
YarnConfiguration conf = new YarnConfiguration(new JobConf());
conf.addResource(new Path(MRJobConfig.JOB_CONF_FILE));
String jobUserName = System
.getenv(ApplicationConstants.Environment.USER.name());
conf.set(MRJobConfig.USER_NAME, jobUserName);
+ conf.setBoolean("fs.automatic.close", false);
initAndStartAppMaster(appMaster, conf, jobUserName);
{code}
Maybe you already tried this, but it prevents {{FileSystem}} from closing the
fs objects automatically. Then you do the {{FS.closeAll()}} in your shutdown
handler.
> finalState UNDEFINED if AM is killed by hand
> ---------------------------------------------
>
> Key: MAPREDUCE-3614
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3614
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Ravi Prakash
> Assignee: Ravi Prakash
> Attachments: MAPREDUCE-3614.branch-0.23.patch, MAPREDUCE-3614.patch,
> MAPREDUCE-3614.patch, MAPREDUCE-3614.patch, MAPREDUCE-3614.patch
>
>
> Courtesy [~dcapwell]
> {quote}
> If the AM is running and you kill the process (sudo kill #pid), the State in
> Yarn would be FINISHED and FinalStatus is UNDEFINED. The Tracking UI would
> say "History" and point to the proxy url (which will redirect to the history
> server).
> The state should be more descriptive that the job failed and the tracker url
> shouldn't point to the history server.
> {quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
