[
https://issues.apache.org/jira/browse/MAPREDUCE-3255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13135246#comment-13135246
]
Todd Lipcon commented on MAPREDUCE-3255:
----------------------------------------
Since it would be a framework counter, though, we have no ability to ensure
that different containers have the same resource utilization. And comparing
across applications (eg for organizational billing), we need a standardized
unit.
Using RAM-seconds now doesn't preclude us from adding more counters for other
resource types later. I imagine that, for any resource type that is
requested/reserved by a job, we'd have a counter for that resource.
This also allows us future improvements where the resource requests for a
container may even change throughout its life (eg for native apps where
changing heap usage is easy to do)
> MR2 needs a counter analogous to "slot seconds"
> -----------------------------------------------
>
> Key: MAPREDUCE-3255
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3255
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Todd Lipcon
>
> In older versions of MR, we have a SLOT_MILLIS counter which counts how many
> seconds each task (and job) used of map/reduce slots. Since we no longer have
> the concept of a "slot" in MR2, we should change this counter to be something
> like "RAM-megabyte-seconds" - ie each task is charged for its resource
> requirement * the number of seconds it occupied those resources.
> Ideally these would be collected by the NodeManager rather than the children,
> so that they could be used for billing purposes, but a first cut of the
> counters through the MR framework itself would probably still be useful in
> more trusting environments.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
