[
https://issues.apache.org/jira/browse/MAPREDUCE-5330?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xi Fang updated MAPREDUCE-5330:
-------------------------------
Attachment: MAPREDUCE-5330.patch
> Killing M/R JVM's leads to metrics not being uploaded
> -----------------------------------------------------
>
> Key: MAPREDUCE-5330
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-5330
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Affects Versions: 1-win
> Environment: Windows
> Reporter: Xi Fang
> Assignee: Xi Fang
> Attachments: MAPREDUCE-5330.patch
>
>
> In MapReduce, we sometimes kill a task's JVM before it naturally shuts down
> if we want to launch other tasks (look in
> JvmManager$JvmManagerForType.reapJvm). This behavior means that if the map
> task process is in the middle of doing some cleanup/finalization after the
> task is done, it might be interrupted/killed without giving it a chance.
> In the Microsoft's Hadoop Service, after a Map/Reduce task is done and during
> closing file systems in a special shutdown hook, we're typically uploading
> storage (ASV in our context) usage metrics to Microsoft Azure Tables. So if
> this kill happens these metrics get lost. The impact is that for many MR jobs
> we don't see accurate metrics reported most of the time.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
