[ http://issues.apache.org/jira/browse/HADOOP-137?page=all ]
Sameer Paranjpye updated HADOOP-137:
------------------------------------
Fix Version: 0.2
Version: 0.2
Description:
In the TaskTracker#Initialize method, the following line assigns task tracker
name (id):
this.taskTrackerName = "tracker_" + (Math.abs(r.nextInt()) % 100000);
For a fair size cluster, it is possible that different task trackers to get the
same id, causing name conflict.
I encountered this problem with a cluster of 274 nodes. Once such conflict
happens, a lot of strange things may happen.
For example, a reducer task tried to copy from a machine (task tracker) a map
output file that was actually produced
on another machine.
was:
In the TaskTracker#Initialize method, the following line assigns task tracker
name (id):
this.taskTrackerName = "tracker_" + (Math.abs(r.nextInt()) % 100000);
For a fair size cluster, it is possible that different task trackers to get the
same id, causing name conflict.
I encountered this problem with a cluster of 274 nodes. Once such conflict
happens, a lot of strange things may happen.
For example, a reducer task tried to copy from a machine (task tracker) a map
output file that was actually produced
on another machine.
Assign To: Owen O'Malley
This was fixed in the patch submitted for HADOOP-134
> Different TaskTrackers may get the same task tracker id, thus cause many
> problems.
> ----------------------------------------------------------------------------------
>
> Key: HADOOP-137
> URL: http://issues.apache.org/jira/browse/HADOOP-137
> Project: Hadoop
> Type: Bug
> Components: mapred
> Versions: 0.2
> Reporter: Runping Qi
> Assignee: Owen O'Malley
> Priority: Critical
> Fix For: 0.2
>
> In the TaskTracker#Initialize method, the following line assigns task tracker
> name (id):
> this.taskTrackerName = "tracker_" + (Math.abs(r.nextInt()) % 100000);
> For a fair size cluster, it is possible that different task trackers to get
> the same id, causing name conflict.
> I encountered this problem with a cluster of 274 nodes. Once such conflict
> happens, a lot of strange things may happen.
> For example, a reducer task tried to copy from a machine (task tracker) a map
> output file that was actually produced
> on another machine.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
