[
https://issues.apache.org/jira/browse/HDFS-3577?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13414469#comment-13414469
]
Tsz Wo (Nicholas), SZE commented on HDFS-3577:
----------------------------------------------
It turns that the size of each chunk is very small (tens of bytes) when it uses
chunked transfer encoding. The client keeps opening new socket for each chunk
and then it leads to SocketException. The problem disappear if StreamingOutput
is replaced with MessageBodyWriter since it can specify the response size.
> webHdfsFileSystem fails to read files with chunked transfer encoding
> --------------------------------------------------------------------
>
> Key: HDFS-3577
> URL: https://issues.apache.org/jira/browse/HDFS-3577
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: hdfs client
> Affects Versions: 2.0.0-alpha
> Reporter: Alejandro Abdelnur
> Assignee: Tsz Wo (Nicholas), SZE
> Priority: Blocker
> Attachments: h3577_20120705.patch, h3577_20120708.patch
>
>
> If reading a file large enough for which the httpserver running
> webhdfs/httpfs uses chunked transfer encoding (more than 24K in the case of
> webhdfs), then the WebHdfsFileSystem client fails with an IOException with
> message *Content-Length header is missing*.
> It looks like WebHdfsFileSystem is delegating opening of the inputstream to
> *ByteRangeInputStream.URLOpener* class, which checks for the *Content-Length*
> header, but when using chunked transfer encoding the *Content-Length* header
> is not present and the *URLOpener.openInputStream()* method thrown an
> exception.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
