[
https://issues.apache.org/jira/browse/MAPREDUCE-438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12733355#action_12733355
]
Raghu Angadi commented on MAPREDUCE-438:
----------------------------------------
I think it would be a very useful fix. Many times, the users that are affected
by this don't know the actual cause.
> When connecting to HDFS using an IP Task MapRed gets confused when checking
> the output path.
> ----------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-438
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-438
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Environment: AIX 6.1 and Solaris
> Reporter: Bill Habermaas
> Priority: Minor
>
> The exception appearing below happens if you are using an IP address to
> reference a HDFS file system path. Although the URI authority for either
> path (whether referenced by hostname:port or IP-address:port) refer to the
> same filesystem, Task throws an exception because the stringified comparisons
> of the authority in the URIs are not equal. The FileSystem holds the network
> location of the namenode as a host name regardless as to whether an IP
> address or hostname is used when the file system is connected (refer to
> HADOOP-5191 for an earlier fix specifically for HDFS only).
> 2009-03-27 04:15:45,045 WARN [Thread-145]
> org.apache.hadoop.mapred.LocalJobRunner: job_local_0002
> java.io.IOException: Can not get the relative path: base =
> hdfs://10.120.16.68:9000/mydata/2009/03/27/0bab100a-1bf1-499a-935d-bc4b4e94f44c/_temporary/_attempt_local_0002_r_000000_0
> child =
> hdfs://p520aix61.mydomain.com:9000/mydata/2009/03/27/0bab100a-1bf1-499a-935d-bc4b4e94f44c/_temporary/_attempt_local_0002_r_000000_0/part-00000
> at org.apache.hadoop.mapred.Task.getFinalPath(Task.java:586)
> at org.apache.hadoop.mapred.Task.moveTaskOutputs(Task.java:599)
> at org.apache.hadoop.mapred.Task.moveTaskOutputs(Task.java:617)
> at org.apache.hadoop.mapred.Task.saveTaskOutput(Task.java:561)
> at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:202)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
