[
https://issues.apache.org/jira/browse/HDFS-3318?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Eli Collins updated HDFS-3318:
------------------------------
Hadoop Flags: Incompatible change,Reviewed (was: Reviewed)
Marking this as an incompatible change as the following code broke distcp from
0.20.2 and 0.21 releases. The code below requires the content length header be
present however that field wasn't introduced in trunk until 22 (by HDFS-1085),
it came into branch-1 via the YDH merge. Filed HDFS-3671 to fix this.
{code}
- filelength = (cl == null) ? -1 : Long.parseLong(cl);
- in = connection.getInputStream();
+ if (cl == null) {
+ throw new IOException(StreamFile.CONTENT_LENGTH+" header is missing");
+ }
{code}
> Hftp hangs on transfers >2GB
> ----------------------------
>
> Key: HDFS-3318
> URL: https://issues.apache.org/jira/browse/HDFS-3318
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: hdfs client
> Affects Versions: 0.24.0, 0.23.3, 2.0.0-alpha
> Reporter: Daryn Sharp
> Assignee: Daryn Sharp
> Priority: Blocker
> Fix For: 0.23.3
>
> Attachments: HDFS-3318-1.patch, HDFS-3318.patch
>
>
> Hftp transfers >2GB hang after the transfer is complete. The problem appears
> to be caused by java internally using an int for the content length. When it
> overflows 2GB, it won't check the bounds of the reads on the input stream.
> The client continues reading after all data is received, and the client
> blocks until the server times out the connection -- _many_ minutes later. In
> conjunction with hftp timeouts, all transfers >2G fail with a read timeout.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
