[
https://issues.apache.org/jira/browse/HADOOP-2227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12547119
]
Amareshwari Sri Ramadasu commented on HADOOP-2227:
--------------------------------------------------
bq. when one of the disks has enough space for a task, then
TaskTracker.enoughFreeSpace should return true.
With current behaviour , if we do this, localizeJob() and localizeTask() can
fail because they use getLocalPath() for localizing. In getLocalPath(), there
is no space availability check. And local file names are hashed to determine
which device to store the file on.
One solution could be to change getLocalPath to use localDirAllocator as
proposed in HADOOP-1991 and then do this change. But there would be some cases
where we do not know size of files apriori, to check for disk space.
Thoughts?
> wrong usage of mapred.local.dir.minspacestart
> ----------------------------------------------
>
> Key: HADOOP-2227
> URL: https://issues.apache.org/jira/browse/HADOOP-2227
> Project: Hadoop
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.15.1
> Reporter: Christian Kunz
> Assignee: Amareshwari Sri Ramadasu
> Priority: Critical
>
> As I understand it, mapred.local.dir.minspacestart should be used to specify
> that a map or reduce task requires a minimum disk space to be executed. When
> several disks are available, and mapred.local.dir lists all of them, then the
> free space should be added up. But TaskTracker.enoughFreeSpace returns false
> whenever one of the disks listed in mapred.local.dir has less than the space
> specified by mapred.local.dir.minspacestart.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
