[
https://issues.apache.org/jira/browse/TS-1457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13456572#comment-13456572
]
Alan M. Carroll edited comment on TS-1457 at 9/17/12 12:35 AM:
---------------------------------------------------------------
Commit a2d46114a918cf72a38d66ea66dea96dbb9b6920
This fix changes the way chunking output is handled to avoid the memory bloat.
The root problem was using IOBuffers that were too large, in some cases 2M
buffers to hold ~10 bytes.
1) Dechunking now uses 256 byte buffers, which is the cut off for copying bytes
vs. using IOBuffer references.
2) Chunking now uses minimum size buffers because the only data actually placed
in those buffers are the chunk header and trailer which vary from 5 to 12
bytes. All other data is passed via IOBufferBlock references.
In addition the chunk size handling was modified to default to the original
4096 but it can now be configured globally and on a per transaction basis. The
global config value is
proxy.config.http.chunking.size INT <maximum chunk size>
This value is also accessible per transaction by using the
TSHttpTxnConfigInt[SG]et functions with a key of TS_CONFIG_HTTP_CHUNKING_SIZE.
was (Author: amc):
Commit
This fix changes the way chunking output is handled to avoid the memory bloat.
The root problem was using IOBuffers that were too large, in some cases 2M
buffers to hold ~10 bytes.
1) Dechunking now uses 256 byte buffers, which is the cut off for copying bytes
vs. using IOBuffer references.
2) Chunking now uses minimum size buffers because the only data actually placed
in those buffers are the chunk header and trailer which vary from 5 to 12
bytes. All other data is passed via IOBufferBlock references.
In addition the chunk size handling was modified to default to the original
4096 but it can now be configured globally and on a per transaction basis. The
global config value is
proxy.config.http.chunking.size INT <maximum chunk size>
This value is also accessible per transaction by using the
TSHttpTxnConfigInt[SG]et functions with a key of TS_CONFIG_HTTP_CHUNKING_SIZE.
> transforms during video requests cause massive memory growth
> ------------------------------------------------------------
>
> Key: TS-1457
> URL: https://issues.apache.org/jira/browse/TS-1457
> Project: Traffic Server
> Issue Type: Bug
> Components: Core
> Affects Versions: 3.2.0
> Environment: Linux RHEL6.2
> Reporter: Aidan McGurn
> Priority: Critical
> Attachments: 3.0.3-null-massif.txt, 3.0.5-null-massif.txt,
> alan-chunk-fix-3.2.0-null.txt, alan-nofix-3.2.0-null.txt,
> no-null-3.2.0-massif.txt, null-3.0.2-massif.txt, null-3.2.0-massif.txt,
> ts-1457-patch-1.diff
>
>
> Running the null-transform example object over 3.2.0, shows massive memory
> growth -
> This could be seen on our system since using 3.2.0 with custom plugin which
> invokes transforms - under load ~6K and with video content (only Video -
> small files are ok), a huge growth was seen (GB's) such that it crashes the
> load test machine.
> heap profiles are attached for comparison:
> -null transform example so over 3.2.0
> -NO null transform over 3.2.0
> -null transform example so over 3.0.2
> As can be seen no issue in 3.0.2 -
> The profiles show clearly the call chain is different for both releases
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
