[
https://issues.apache.org/jira/browse/MAPREDUCE-2037?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Arun C Murthy updated MAPREDUCE-2037:
-------------------------------------
Fix Version/s: 0.23.0
Status: Patch Available (was: Open)
> Capturing interim progress times, CPU usage, and memory usage, when tasks
> reach certain progress thresholds
> -----------------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-2037
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2037
> Project: Hadoop Map/Reduce
> Issue Type: New Feature
> Reporter: Dick King
> Assignee: Dick King
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-2037.patch
>
>
> We would like to capture the following information at certain progress
> thresholds as a task runs:
> * Time taken so far
> * CPU load [either at the time the data are taken, or exponentially
> smoothed]
> * Memory load [also either at the time the data are taken, or
> exponentially smoothed]
> This would be taken at intervals that depend on the task progress plateaus.
> For example, reducers have three progress ranges -- [0-1/3], (1/3-2/3], and
> (2/3-3/3] -- where fundamentally different activities happen. Mappers have
> different boundaries, I understand, that are not symmetrically placed. Data
> capture boundaries should coincide with activity boundaries. For the state
> information capture [CPU and memory] we should average over the covered
> interval.
> This data would flow in with the heartbeats. It would be placed in the job
> history as part of the task attempt completion event, so it could be
> processed by rumen or some similar tool and could drive a benchmark engine.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
