Xi Fang created MAPREDUCE-5330:
----------------------------------
Summary: Killing M/R JVM's leads to metrics not being uploaded
Key: MAPREDUCE-5330
URL: https://issues.apache.org/jira/browse/MAPREDUCE-5330
Project: Hadoop Map/Reduce
Issue Type: Bug
Affects Versions: 1-win
Environment: Windows
Reporter: Xi Fang
Assignee: Xi Fang
In MapReduce, we sometimes kill a task's JVM before it naturally shuts down if
we want to launch other tasks (look in JvmManager$JvmManagerForType.reapJvm).
This behavior means that if the map task process is in the middle of doing some
cleanup/finalization after the task is done, it might be interrupted/killed
without giving it a chance.
In the Microsoft's Hadoop Service, after a Map/Reduce task is done and during
closing file systems in a special shutdown hook, we're typically uploading
storage (ASV in our context) usage metrics to Microsoft Azure Tables. So if
this kill happens these metrics get lost. The impact is that for many MR jobs
we don't see accurate metrics reported most of the time.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
