[
https://issues.apache.org/jira/browse/HDFS-1085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12888070#action_12888070
]
Dmytro Molkov commented on HDFS-1085:
-------------------------------------
I think it has everything to do with the JVM bug and not the best way of jetty
handling it. More details in https://issues.apache.org/jira/browse/HDFS-1194
We updated the jetty version in our production cluster and we do not see
secondary namenode issues anymore (we are always able to fetch the full file).
It should also help this case.
> hftp read failing silently
> ---------------------------
>
> Key: HDFS-1085
> URL: https://issues.apache.org/jira/browse/HDFS-1085
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: data-node
> Reporter: Koji Noguchi
> Assignee: Tsz Wo (Nicholas), SZE
> Attachments: h1085_20100713.patch
>
>
> When performing a massive distcp through hftp, we saw many tasks fail with
> {quote}
> 2010-04-06 17:56:43,005 INFO org.apache.hadoop.tools.DistCp: FAIL
> 2010/0/part-00032 : java.io.IOException: File size not matched: copied
> 193855488 bytes (184.9m) to tmpfile
> (=hdfs://omehost.com:8020/somepath/part-00032)
> but expected 1710327403 bytes (1.6g) from
> hftp://someotherhost/somepath/part-00032
> at
> org.apache.hadoop.tools.DistCp$CopyFilesMapper.copy(DistCp.java:435)
> at org.apache.hadoop.tools.DistCp$CopyFilesMapper.map(DistCp.java:543)
> at org.apache.hadoop.tools.DistCp$CopyFilesMapper.map(DistCp.java:310)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50)
> at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:358)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
> at org.apache.hadoop.mapred.Child.main(Child.java:159)
> {quote}
> This means that read itself didn't fail but the resulted file was somehow
> smaller.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
