[
https://issues.apache.org/jira/browse/MAPREDUCE-3255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13135295#comment-13135295
]
Milind Bhandarkar commented on MAPREDUCE-3255:
----------------------------------------------
Container resource requirements are used by the RM to allocate them, and are
logged in the resource usage. Since the chargeback would be on the resource
consumption as determined by RM in allocation decisions, that is the unit to be
exposed, rather than the actual usage of that resource.
I consider hadoop metering similar to freight / post office model. There is a
per unit volume (or weight) charge, and a per distance traveled charge. Each of
those have difference cost models (i.e. not a straight line, but a curve). One
needs to expose both of these as individual metrics, so that the billing
software can do the C1(v)*C2(d) multiplications, rather than having to be
restricted by a single C(v*d) function.
> MR2 needs a counter analogous to "slot seconds"
> -----------------------------------------------
>
> Key: MAPREDUCE-3255
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3255
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Todd Lipcon
>
> In older versions of MR, we have a SLOT_MILLIS counter which counts how many
> seconds each task (and job) used of map/reduce slots. Since we no longer have
> the concept of a "slot" in MR2, we should change this counter to be something
> like "RAM-megabyte-seconds" - ie each task is charged for its resource
> requirement * the number of seconds it occupied those resources.
> Ideally these would be collected by the NodeManager rather than the children,
> so that they could be used for billing purposes, but a first cut of the
> counters through the MR framework itself would probably still be useful in
> more trusting environments.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
