[
https://issues.apache.org/jira/browse/KNOX-157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13780690#comment-13780690
]
Dilli Arumugam commented on KNOX-157:
-------------------------------------
Investigated this.
I do not think partial buffering would work.
If we go with buffering we would have to buffer the whole entity.
Would be doing more investigations.
One thing though: we could skip buffering completely when we are using simple
(non secure) cluster.
The option, I am considering for the short term:
No buffering at all.
With this, simple cluster would work fine all time.
Some Put, Post requests to secure cluster would fail.
Which Put, Post requests would fail?
Very first Put, Post requests to each service in secure cluster would fail.
Subsequently whenever hadoop.auth coookie of a service expires, corresponding
one Put/Post request would fail.
In the next iteration, we could have a configuration switch to toggle whether
buffering is enabled Put, Post requests to secure cluster.
Another option is to check the presence and expiry time of hadoop.auth cookie
and fetch a new hadoop.auth cookie when hadoop.auth cookie is null or expired.
> Knox is not able to process PUT/POST requests with large payload
> ----------------------------------------------------------------
>
> Key: KNOX-157
> URL: https://issues.apache.org/jira/browse/KNOX-157
> Project: Apache Knox
> Issue Type: Bug
> Components: Server
> Affects Versions: 0.3.0
> Reporter: Vladimir Tkhir
> Assignee: Dilli Arumugam
> Priority: Critical
> Fix For: 0.3.0
>
> Attachments: KNOX-157.patch
>
>
> Getting OutOfMemory exception when trying to process PUT/POST requests with
> large bodies. As an example create 1GB file in HDFS via Knox.
> Issue is related to replaying bodies on requests dispatch.
> From Kevin:
> We need a special version of a BufferedHttpEntity that will buffer up to the
> first n bytes and then if getContent() is called again after that many bytes
> have been read it throws an exception.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
