[
https://issues.apache.org/jira/browse/HDFS-4549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13635687#comment-13635687
]
Mark Wagner commented on HDFS-4549:
-----------------------------------
bq. I think we should leave Jersey to decide what to use, chunk encoding or not.
I agree that that would usually be best, however this is causing significant
performance problems, so I think it needs to be handled somehow. I just took a
look at the memory usage when pulling a 1GB file (which does use
content-length) and didn't see any memory usage increase. I'll look into this
further though.
{quote}
Is the Jetty version use by Hadoop 1 that has issues with chunk encoding?
Also, if we don't see the problem in trunk/branch-2, why change it there? not
broken, don't fix it.
{quote}
The issue has been observed in 6.1.26, which is used by branch-1, branch-2, and
trunk. This problem was first seen (chunked encoding being used, leading to
performance loss) in branch-1 (1.0.4 to be precise) for files of any size. I
didn't see it on branch-2/trunk because I didn't test larger files. When I was
backporting though, I noticed that it was still slow above 2GB, and it turns
out the same is true for branch-2/trunk (I've attached logs for a demonstration
of this issue using 2.0.3-alpha).
> WebHDFS hits a Jetty performance issue
> --------------------------------------
>
> Key: HDFS-4549
> URL: https://issues.apache.org/jira/browse/HDFS-4549
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: webhdfs
> Affects Versions: 1.1.2
> Reporter: Mark Wagner
> Assignee: Mark Wagner
> Attachments: example4549.txt, HDFS-4549.1.patch, HDFS-4549.2.patch
>
>
> WebHDFS on branch-1 is hitting a Jetty issue for me when it does chunked
> transfers. This is the same Jetty issue as MAPREDUCE-4399. I have not
> observed this on trunk.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
