[
https://issues.apache.org/jira/browse/HADOOP-3759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12622284#action_12622284
]
Hemanth Yamijala commented on HADOOP-3759:
------------------------------------------
Ari, I had looked at the ResourceEstimator class that you defined in
HADOOP-657. In this patch we are not estimating the used memory - rather using
a configured value. If we go to a model where we estimate memory per task, then
we should definitely use the ResourceEstimator. But for now, we are going with
a simpler approach of letting the user specify this to us in configuration.
On the other hand, to communicate the used disk space, you are following
roughly the same mechanism as what is defined here IIRC - which is to compute
the free disk space in the heartbeat and communicate it via the
TaskTrackerStatus. Vinod's comment was to unify your change as part of the
resource map defined here so that it would be done in a like manner.
Makes sense ?
> Provide ability to run memory intensive jobs without affecting other running
> tasks on the nodes
> -----------------------------------------------------------------------------------------------
>
> Key: HADOOP-3759
> URL: https://issues.apache.org/jira/browse/HADOOP-3759
> Project: Hadoop Core
> Issue Type: Improvement
> Components: mapred
> Reporter: Hemanth Yamijala
> Assignee: Hemanth Yamijala
> Fix For: 0.19.0
>
> Attachments: HADOOP-3759.patch, HADOOP-3759.patch, HADOOP-3759.patch
>
>
> In HADOOP-3581, we are discussing how to prevent memory intensive tasks from
> affecting Hadoop daemons and other tasks running on a node. A related
> requirement is that users be provided an ability to run jobs which are memory
> intensive. The system must provide enough knobs to allow such jobs to be run
> while still maintaining the requirements of HADOOP-3581.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
