[
https://issues.apache.org/jira/browse/TS-3102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14158921#comment-14158921
]
Sudheer Vinukonda edited comment on TS-3102 at 10/4/14 3:49 AM:
----------------------------------------------------------------
Added some debugging code to compare the thread that allocates the MIOBuffer
and the one that's freeing it (in FetchSM object) and they seem to be different
in most cases - Not sure if this fact is somehow playing a role.
The IOBufBlocks within the MIOBuffer are themselves from proxy allocator (so,
should be fine, I presume?), but, there are other buffers in it which may be
from per thread pool and discussing with the wise on the IRC, it seems this may
cause some issues, although, I am still struggling to come up with an accurate
explanation.
{code}
[Oct 2 04:57:31.568] Server {0x2b9333e9c700} ERROR: freeing req_buffer from
thread 0x2b9331a79010, allcoated from 47914215280656
[Oct 2 04:57:31.661] Server {0x2b9338302700} ERROR: freeing req_buffer from
thread 0x2b9331e7d010, allcoated from 47914216333328
[Oct 2 04:57:31.809] Server {0x2b9338908700} ERROR: freeing req_buffer from
thread 0x2b9332483010, allcoated from 47914251071504
[Oct 2 04:57:32.005] Server {0x2b9338f0e700} ERROR: freeing req_buffer from
thread 0x2b9332a89010, allcoated from 47914227912720
[Oct 2 04:57:32.087] Server {0x2b9333b99700} ERROR: freeing req_buffer from
thread 0x2b9331776010, allcoated from 47914251071504
[Oct 2 04:57:32.488] Server {0x2b9333896700} ERROR: freeing req_buffer from
thread 0x2b9331473010, allcoated from 47914227912720
[Oct 2 04:57:33.499] Server {0x2b9338b0a700} ERROR: freeing req_buffer from
thread 0x2b9332685010, allcoated from 47914251071504
[Oct 2 04:57:34.130] Server {0x2b9333e9c700} ERROR: freeing req_buffer from
thread 0x2b9331a79010, allcoated from 47914250018832
[Oct 2 04:57:34.524] Server {0x2b9338c0b700} ERROR: freeing req_buffer from
thread 0x2b9332786010, allcoated from 47914225807376
[Oct 2 04:57:35.241] Server {0x2b9338807700} ERROR: freeing req_buffer from
thread 0x2b9332382010, allcoated from 47914225807376
[Oct 2 04:57:35.650] Server {0x2b9339716700} ERROR: freeing req_buffer from
thread 0x2b9333291010, allcoated from 47914239492112
[Oct 2 04:57:36.687] Server {0x2b933900f700} ERROR: freeing req_buffer from
thread 0x2b9332b8a010, allcoated from 47914244755472
[Oct 2 04:57:36.887] Server {0x2b9338403700} ERROR: freeing req_buffer from
thread 0x2b9331f7e010, allcoated from 47914228965392
[Oct 2 04:57:37.173] Server {0x2b9338d0c700} ERROR: freeing req_buffer from
thread 0x2b9332887010, allcoated from 47914220544016
[Oct 2 04:57:37.296] Server {0x2b9338706700} ERROR: freeing req_buffer from
thread 0x2b9332281010, allcoated from 47914228965392
[Oct 2 04:57:41.361] Server {0x2b9339a19700} ERROR: freeing req_buffer from
thread 0x2b9333594010, allcoated from 47914239492112
[Oct 2 04:57:41.491] Server {0x2b9333896700} ERROR: freeing req_buffer from
thread 0x2b9331473010, allcoated from 47914239492112
[Oct 2 04:57:42.576] Server {0x2b9333b99700} ERROR: freeing req_buffer from
thread 0x2b9331776010, allcoated from 47914224754704
[Oct 2 04:57:43.468] Server {0x2b9338100700} ERROR: freeing req_buffer from
thread 0x2b9331c7b010, allcoated from 47914237386768
[Oct 2 04:57:46.013] Server {0x2b933900f700} ERROR: freeing req_buffer from
thread 0x2b9332b8a010, allcoated from 47914238439440
[Oct 2 04:57:46.150] Server {0x2b9333c9a700} ERROR: freeing req_buffer from
thread 0x2b9331877010, allcoated from 47914223702032
[Oct 2 04:57:46.211] Server {0x2b9339716700} ERROR: freeing req_buffer from
thread 0x2b9333291010, allcoated from 47914246860816
[Oct 2 04:57:47.603] Server {0x2b9338e0d700} ERROR: freeing req_buffer from
thread 0x2b9332988010, allcoated from 47914225807376
[Oct 2 04:57:49.061] Server {0x2b9338706700} ERROR: freeing req_buffer from
thread 0x2b9332281010, allcoated from 47914250018832
[Oct 2 04:57:51.423] Server {0x2b9338d0c700} ERROR: freeing req_buffer from
thread 0x2b9332887010, allcoated from 47914225807376
[Oct 2 04:57:51.617] Server {0x2b9333d9b700} ERROR: freeing req_buffer from
thread 0x2b9331978010, allcoated from 47914219491344
[Oct 2 04:57:52.292] Server {0x2b9338201700} ERROR: freeing req_buffer from
thread 0x2b9331d7c010, allcoated from 47914233176080
{code}
was (Author: sudheerv):
Added some debugging code to compare the thread that allocates the MIOBuffer
and the one that's freeing it and they seem to be different in most cases - Not
sure if this fact is somehow playing a role.
The IOBufBlocks within the MIOBuffer are themselves from proxy allocator (so,
should be fine, I presume?), but, there are other buffers in it which may be
from per thread pool and discussing with the wise on the IRC, it seems this may
cause some issues, although, I am still struggling to come up with an accurate
explanation.
{code}
[Oct 2 04:57:31.568] Server {0x2b9333e9c700} ERROR: freeing req_buffer from
thread 0x2b9331a79010, allcoated from 47914215280656
[Oct 2 04:57:31.661] Server {0x2b9338302700} ERROR: freeing req_buffer from
thread 0x2b9331e7d010, allcoated from 47914216333328
[Oct 2 04:57:31.809] Server {0x2b9338908700} ERROR: freeing req_buffer from
thread 0x2b9332483010, allcoated from 47914251071504
[Oct 2 04:57:32.005] Server {0x2b9338f0e700} ERROR: freeing req_buffer from
thread 0x2b9332a89010, allcoated from 47914227912720
[Oct 2 04:57:32.087] Server {0x2b9333b99700} ERROR: freeing req_buffer from
thread 0x2b9331776010, allcoated from 47914251071504
[Oct 2 04:57:32.488] Server {0x2b9333896700} ERROR: freeing req_buffer from
thread 0x2b9331473010, allcoated from 47914227912720
[Oct 2 04:57:33.499] Server {0x2b9338b0a700} ERROR: freeing req_buffer from
thread 0x2b9332685010, allcoated from 47914251071504
[Oct 2 04:57:34.130] Server {0x2b9333e9c700} ERROR: freeing req_buffer from
thread 0x2b9331a79010, allcoated from 47914250018832
[Oct 2 04:57:34.524] Server {0x2b9338c0b700} ERROR: freeing req_buffer from
thread 0x2b9332786010, allcoated from 47914225807376
[Oct 2 04:57:35.241] Server {0x2b9338807700} ERROR: freeing req_buffer from
thread 0x2b9332382010, allcoated from 47914225807376
[Oct 2 04:57:35.650] Server {0x2b9339716700} ERROR: freeing req_buffer from
thread 0x2b9333291010, allcoated from 47914239492112
[Oct 2 04:57:36.687] Server {0x2b933900f700} ERROR: freeing req_buffer from
thread 0x2b9332b8a010, allcoated from 47914244755472
[Oct 2 04:57:36.887] Server {0x2b9338403700} ERROR: freeing req_buffer from
thread 0x2b9331f7e010, allcoated from 47914228965392
[Oct 2 04:57:37.173] Server {0x2b9338d0c700} ERROR: freeing req_buffer from
thread 0x2b9332887010, allcoated from 47914220544016
[Oct 2 04:57:37.296] Server {0x2b9338706700} ERROR: freeing req_buffer from
thread 0x2b9332281010, allcoated from 47914228965392
[Oct 2 04:57:41.361] Server {0x2b9339a19700} ERROR: freeing req_buffer from
thread 0x2b9333594010, allcoated from 47914239492112
[Oct 2 04:57:41.491] Server {0x2b9333896700} ERROR: freeing req_buffer from
thread 0x2b9331473010, allcoated from 47914239492112
[Oct 2 04:57:42.576] Server {0x2b9333b99700} ERROR: freeing req_buffer from
thread 0x2b9331776010, allcoated from 47914224754704
[Oct 2 04:57:43.468] Server {0x2b9338100700} ERROR: freeing req_buffer from
thread 0x2b9331c7b010, allcoated from 47914237386768
[Oct 2 04:57:46.013] Server {0x2b933900f700} ERROR: freeing req_buffer from
thread 0x2b9332b8a010, allcoated from 47914238439440
[Oct 2 04:57:46.150] Server {0x2b9333c9a700} ERROR: freeing req_buffer from
thread 0x2b9331877010, allcoated from 47914223702032
[Oct 2 04:57:46.211] Server {0x2b9339716700} ERROR: freeing req_buffer from
thread 0x2b9333291010, allcoated from 47914246860816
[Oct 2 04:57:47.603] Server {0x2b9338e0d700} ERROR: freeing req_buffer from
thread 0x2b9332988010, allcoated from 47914225807376
[Oct 2 04:57:49.061] Server {0x2b9338706700} ERROR: freeing req_buffer from
thread 0x2b9332281010, allcoated from 47914250018832
[Oct 2 04:57:51.423] Server {0x2b9338d0c700} ERROR: freeing req_buffer from
thread 0x2b9332887010, allcoated from 47914225807376
[Oct 2 04:57:51.617] Server {0x2b9333d9b700} ERROR: freeing req_buffer from
thread 0x2b9331978010, allcoated from 47914219491344
[Oct 2 04:57:52.292] Server {0x2b9338201700} ERROR: freeing req_buffer from
thread 0x2b9331d7c010, allcoated from 47914233176080
{code}
> Improve memory reuse for SPDY contexts by reusing memory released by streams
> within a client session
> ----------------------------------------------------------------------------------------------------
>
> Key: TS-3102
> URL: https://issues.apache.org/jira/browse/TS-3102
> Project: Traffic Server
> Issue Type: Improvement
> Components: SPDY
> Affects Versions: 5.0.1
> Reporter: Sudheer Vinukonda
> Assignee: Sudheer Vinukonda
> Labels: yahoo
> Fix For: 5.2.0
>
> Attachments: TS-3102.diff
>
>
> In the present SPDY implementation in ATS, there is no client context reuse.
> Though the spdy session is reused, each stream (even the non-concurrent ones)
> is allocated a set of new/separate context buffers (including internal
> plugin_vc, client_session, SM object, header heap, transaction buffers,
> server session objects etc). Some of these objects are allocated from global
> pool, while some are from per-thread pool. The context memory is not reused
> unlike the non-spdy session, where there can be at most one transaction on a
> given connection/session at a time.
> Besides being very inefficient (the allocation/deallocation) this also leads
> to larger memory foot print over time, due to the relatively poor reuse of
> per thread pool objects (especially, when there are a high number of threads
> - e.g 100+ like we have).
> I am currently testing a patch that does not deallocate the streams when the
> transaction completes and reuses those for new/subsequent streams.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
