[
http://jira.magnolia-cms.com/browse/MAGNOLIA-3390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jan Haderka reopened MAGNOLIA-3390:
-----------------------------------
Please correct me if I'm wrong.
Current fix sets chunked encoding (w/o making it really configurable) when file
length is more then {{2^31^-1}} which is 2,147,483,647B => that's roughly 2 GB
... might be too high value, plus making this switch fixed means Magnolia can
no longer run on any HTTP 1.0 compatible server but needs 1.1 at least
> Prevent OOME and GC load during activation of large data sets
> -------------------------------------------------------------
>
> Key: MAGNOLIA-3390
> URL: http://jira.magnolia-cms.com/browse/MAGNOLIA-3390
> Project: Magnolia
> Issue Type: Improvement
> Components: activation
> Affects Versions: 4.3.x
> Reporter: Jörg von Frantzius
> Assignee: Philipp Bärfuss
> Priority: Blocker
> Fix For: 4.4 Beta1
>
>
> During activation, the activated data is currently held completely in memory
> for each activation request sent from author to public.
> h4. The problem
> When e.g. 250MB are activated on an author instance with 4 subscribers, these
> 250MB are allocated 4 times in a row in RAM and garbage-collected afterwards.
> Even if no OutOfMemoryError occurs during this, a high load is put on the
> Garbage Collector, likely forcing the VM to perform "stop the world" full
> collections, leading to unresponsiveness of the author instance for editors.
> Given large enough binary data or simultaneous attempts at activating it, any
> maximum heap size can be exceeded.
> h4. Current implementation
> This seems to be due to the default behaviour of
> {{java.net.URLConnection.getOutputStream()}} used by
> {{info.magnolia.module.exchangesimple.Transporter}}, which returns a subclass
> of {{ByteArrayOutputStream}} that caches the whole GET request in memory.
> This probably happens in order to determine the content-length before
> actually sending the request.
> h4. Proposed solution
> The solution is to use "chunked transfer coding", as defined in [RFC
> 2616|http://www.w3.org/Protocols/rfc2616/rfc2616-sec3.html#sec3.6.1]. This
> needs to be explicitly enabled by calling
> {{java.net.HttpURLConnection.setChunkedStreamingMode(int)}} prior to
> {{getOutputStream()}}. I verified via debugger that doing so will result in a
> {{sun.net.www.protocol.http.HttpURLConnection$StreamingOutputStream extends
> FilterOutputStream}} instead of {{sun.net.www.http.PosterOutputStream extends
> ByteArrayOutputStream}}.
> Chunking requires the public server to be HTTP/1.1 compliant. In case
> HTTP/1.1 compliance poses a problem e.g. with proxied public servers or weird
> HTTP servers, chunking of activation requests should be configurable. There
> could e.g. be a configuration NodeData
> "server/activation/subscribers/<subscribername>/useRequestChunking" with
> default value "true".
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://jira.magnolia-cms.com/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
----------------------------------------------------------------
For list details see
http://www.magnolia-cms.com/home/community/mailing-lists.html
To unsubscribe, E-mail to: <dev-list-unsubscr...@magnolia-cms.com>
----------------------------------------------------------------
