[
https://issues.apache.org/jira/browse/TS-3102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14157868#comment-14157868
]
Sudheer Vinukonda commented on TS-3102:
---------------------------------------
After running for almost 24 hours, here's how the no-reuse and reuse versions
compare:
Without reuse:
4507828224 | 1107984384 | 32768 | memory/ioBufAllocator[8]
1114374144 | 239607808 | 8192 |
memory/ioBufAllocator[6]
1307574272 | 216780800 | 4096 | memory/ioBufAllocator[5]
59375616 | 2073600 | 512 |
memory/FetchSMAllocator
29417472 | 7640640 | 672 |
memory/httpClientSessionAllocator
153640960 | 1440384 | 7744 |
memory/httpSMAllocator
25976832 | 909440 | 224 |
memory/spdyRequestAllocator
With reuse:
1068498944 | 744751104 | 32768 |
memory/ioBufAllocator[8]
265027584 | 228196352 | 8192 |
memory/ioBufAllocator[6]
847249408 | 301707264 | 4096 |
memory/ioBufAllocator[5]
24051712 | 7769600 | 512 |
memory/FetchSMAllocator
43352064 | 18711168 | 672 |
memory/httpClientSessionAllocator
39649280 | 1184832 | 7744 |
memory/httpSMAllocator
10522624 | 3399200 | 224 |
memory/spdyRequestAllocator
Basically, without reuse, there's large number of buffers allocated upfront
with the % of them in-use being much lower compared to the version with reuse.
For example, for the 32k buffer, at this point, only 1.1g out of 4.5g (<25%)
allocated is in-use without reuse, while 0.74mb out 1.06g (~75%) is in-use with
reuse. The reuse becomes much higher during peak hours.
progression of 32k buffer allocation/in-use without reuse:
4507828224 | 603062272 | 32768 | memory/ioBufAllocator[8]
4507828224 | 975634432 | 32768 | memory/ioBufAllocator[8]
4507828224 | 756744192 | 32768 | memory/ioBufAllocator[8]
4507828224 | 810156032 | 32768 | memory/ioBufAllocator[8]
4507828224 | 838467584 | 32768 | memory/ioBufAllocator[8]
4507828224 | 840499200 | 32768 | memory/ioBufAllocator[8]
4507828224 | 852459520 | 32768 | memory/ioBufAllocator[8]
4507828224 | 881491968 | 32768 | memory/ioBufAllocator[8]
4507828224 | 940867584 | 32768 | memory/ioBufAllocator[8]
4507828224 | 1107984384 | 32768 | memory/ioBufAllocator[8]
progression of 32k buffer allocation/in-use with reuse:
929038336 | 905445376 | 32768 | memory/ioBufAllocator[8]
929038336 | 872742912 | 32768 | memory/ioBufAllocator[8]
932184064 | 928776192 | 32768 | memory/ioBufAllocator[8]
939524096 | 930906112 | 32768 | memory/ioBufAllocator[8]
1068498944 | 917078016 | 32768 | memory/ioBufAllocator[8]
1068498944 | 980189184 | 32768 | memory/ioBufAllocator[8]
1068498944 | 1001750528 | 32768 | memory/ioBufAllocator[8]
1068498944 | 744751104 | 32768 | memory/ioBufAllocator[8]
Note that the version without reuse has been running a few additional hours
before I collected the dump, so, the 32K allocation numbers I have for that
version start at 4.5g (unfortunately, didn't capture how quickly it reached
there). But, the fact remains that, there's a large number of unused buffers
allocated, while the version with reuse didn't get beyond 1.1g so far in 24
hours.
Below's complete dump memory:
without reuse:
{code}
67108864 | 0 | 2097152 |
memory/ioBufAllocator[14]
134217728 | 82837504 | 1048576 |
memory/ioBufAllocator[13]
83886080 | 50855936 | 524288 |
memory/ioBufAllocator[12]
125829120 | 93323264 | 262144 |
memory/ioBufAllocator[11]
155189248 | 147849216 | 131072 |
memory/ioBufAllocator[10]
77594624 | 63373312 | 65536 | memory/ioBufAllocator[9]
4507828224 | 1107984384 | 32768 | memory/ioBufAllocator[8]
527958016 | 478429184 | 16384 | memory/ioBufAllocator[7]
1114374144 | 239607808 | 8192 | memory/ioBufAllocator[6]
1307574272 | 216780800 | 4096 | memory/ioBufAllocator[5]
5767168 | 5154816 | 2048 | memory/ioBufAllocator[4]
2621440 | 2380800 | 1024 | memory/ioBufAllocator[3]
1245184 | 1216512 | 512 | memory/ioBufAllocator[2]
66551808 | 2122240 | 256 | memory/ioBufAllocator[1]
180224 | 136832 | 128 | memory/ioBufAllocator[0]
59375616 | 2073600 | 512 | memory/FetchSMAllocator
0 | 0 | 112 |
memory/ICPPeerReadContAllocator
0 | 0 | 432 |
memory/PeerReadDataAllocator
0 | 0 | 240 | memory/INKVConnAllocator
2789376 | 26784 | 96 | memory/INKContAllocator
2514944 | 22400 | 32 | memory/apiHookAllocator
0 | 0 | 96 |
memory/prefetchLockHandlerAllocator
0 | 0 | 320 |
memory/PrefetchBlasterAllocator
0 | 0 | 192 |
memory/prefetchUrlEntryAllocator
0 | 0 | 128 |
memory/socksProxyAllocator
29417472 | 7640640 | 672 |
memory/httpClientSessionAllocator
153640960 | 1440384 | 7744 | memory/httpSMAllocator
4214784 | 74368 | 224 |
memory/httpServerSessionAllocator
0 | 0 | 48 |
memory/CacheLookupHttpConfigAllocator
0 | 0 | 7808 |
memory/httpUpdateSMAllocator
25976832 | 909440 | 224 |
memory/spdyRequestAllocator
4366336 | 1440192 | 208 |
memory/spdyClientSessionAllocator
0 | 0 | 48 |
memory/CongestRequestParamAllocator
0 | 0 | 144 |
memory/CongestionDBContAllocator
32768 | 0 | 256 |
memory/httpCacheAltAllocator
0 | 0 | 112 |
memory/OneWayTunnelAllocator
3538944 | 4608 | 2304 |
memory/hostDBContAllocator
270848 | 33856 | 33856 | memory/dnsBufAllocator
163840 | 1280 | 1280 | memory/dnsEntryAllocator
0 | 0 | 16 |
memory/DNSRequestDataAllocator
0 | 0 | 576 |
memory/cacheContAllocator
0 | 0 | 112 |
memory/inControlAllocator
0 | 0 | 112 |
memory/outControlAllocator
0 | 0 | 32 | memory/byteBankAllocator
0 | 0 | 592 |
memory/clusterVCAllocator
35893248 | 10376864 | 736 | memory/sslNetVCAllocator
23777280 | 7768896 | 688 | memory/netVCAllocator
0 | 0 | 128 |
memory/udpReadContAllocator
0 | 0 | 160 |
memory/udpPacketAllocator
0 | 0 | 384 | memory/socksAllocator
0 | 0 | 128 |
memory/UDPIOEventAllocator
94707712 | 8758336 | 64 | memory/ioBlockAllocator
42289152 | 8396544 | 48 | memory/ioDataAllocator
158699520 | 20956080 | 240 | memory/ioAllocator
16998400 | 4244000 | 80 | memory/mutexAllocator
9265152 | 2797344 | 96 | memory/eventAllocator
{code}
with reuse:
{code}
67108864 | 4194304 | 2097152 |
memory/ioBufAllocator[14]
100663296 | 58720256 | 1048576 |
memory/ioBufAllocator[13]
83886080 | 59244544 | 524288 |
memory/ioBufAllocator[12]
125829120 | 103022592 | 262144 |
memory/ioBufAllocator[11]
150994944 | 148373504 | 131072 |
memory/ioBufAllocator[10]
81788928 | 64749568 | 65536 | memory/ioBufAllocator[9]
1068498944 | 744751104 | 32768 | memory/ioBufAllocator[8]
417857536 | 408813568 | 16384 | memory/ioBufAllocator[7]
265027584 | 228196352 | 8192 | memory/ioBufAllocator[6]
847249408 | 301707264 | 4096 | memory/ioBufAllocator[5]
2359296 | 2285568 | 2048 | memory/ioBufAllocator[4]
1179648 | 1157120 | 1024 | memory/ioBufAllocator[3]
589824 | 521216 | 512 | memory/ioBufAllocator[2]
2785280 | 1500672 | 256 | memory/ioBufAllocator[1]
49152 | 34432 | 128 | memory/ioBufAllocator[0]
24051712 | 7769600 | 512 | memory/FetchSMAllocator
0 | 0 | 112 |
memory/ICPPeerReadContAllocator
0 | 0 | 432 |
memory/PeerReadDataAllocator
0 | 0 | 240 | memory/INKVConnAllocator
503808 | 24288 | 96 | memory/INKContAllocator
651264 | 19424 | 32 | memory/apiHookAllocator
0 | 0 | 96 |
memory/prefetchLockHandlerAllocator
0 | 0 | 320 |
memory/PrefetchBlasterAllocator
0 | 0 | 192 |
memory/prefetchUrlEntryAllocator
0 | 0 | 128 |
memory/socksProxyAllocator
43352064 | 18711168 | 672 |
memory/httpClientSessionAllocator
39649280 | 1184832 | 7744 | memory/httpSMAllocator
1175552 | 53984 | 224 |
memory/httpServerSessionAllocator
0 | 0 | 48 |
memory/CacheLookupHttpConfigAllocator
0 | 0 | 7808 |
memory/httpUpdateSMAllocator
10522624 | 3399200 | 224 |
memory/spdyRequestAllocator
5271552 | 1930464 | 288 |
memory/spdyClientSessionAllocator
0 | 0 | 48 |
memory/CongestRequestParamAllocator
0 | 0 | 144 |
memory/CongestionDBContAllocator
32768 | 0 | 256 |
memory/httpCacheAltAllocator
0 | 0 | 112 |
memory/OneWayTunnelAllocator
1179648 | 2304 | 2304 |
memory/hostDBContAllocator
203136 | 33856 | 33856 | memory/dnsBufAllocator
163840 | 0 | 1280 | memory/dnsEntryAllocator
0 | 0 | 16 |
memory/DNSRequestDataAllocator
0 | 0 | 576 |
memory/cacheContAllocator
0 | 0 | 112 |
memory/inControlAllocator
0 | 0 | 112 |
memory/outControlAllocator
0 | 0 | 32 | memory/byteBankAllocator
0 | 0 | 592 |
memory/clusterVCAllocator
24211456 | 8544960 | 736 | memory/sslNetVCAllocator
14354432 | 7298304 | 688 | memory/netVCAllocator
0 | 0 | 128 |
memory/udpReadContAllocator
0 | 0 | 160 |
memory/udpPacketAllocator
0 | 0 | 384 | memory/socksAllocator
0 | 0 | 128 |
memory/UDPIOEventAllocator
22347776 | 10121536 | 64 | memory/ioBlockAllocator
14671872 | 8224944 | 48 | memory/ioDataAllocator
70901760 | 29300640 | 240 | memory/ioAllocator
10240000 | 4883680 | 80 | memory/mutexAllocator
4448256 | 2273664 | 96 | memory/eventAllocator
{code}
> Reuse context memory for SPDY streams within a SPDY connection
> --------------------------------------------------------------
>
> Key: TS-3102
> URL: https://issues.apache.org/jira/browse/TS-3102
> Project: Traffic Server
> Issue Type: Improvement
> Components: SPDY
> Affects Versions: 5.0.1
> Reporter: Sudheer Vinukonda
> Assignee: Sudheer Vinukonda
> Labels: yahoo
> Fix For: 5.2.0
>
> Attachments: TS-3102.diff
>
>
> In the present SPDY implementation in ATS, there is no client context reuse.
> Though the spdy session is reused, each stream (even the non-concurrent ones)
> is allocated a set of new/separate context buffers (including internal
> plugin_vc, client_session, SM object, header heap, transaction buffers,
> server session objects etc). Some of these objects are allocated from global
> pool, while some are from per-thread pool. The context memory is not reused
> unlike the non-spdy session, where there can be at most one transaction on a
> given connection/session at a time.
> Besides being very inefficient (the allocation/deallocation) this also leads
> to larger memory foot print over time, due to the relatively poor reuse of
> per thread pool objects (especially, when there are a high number of threads
> - e.g 100+ like we have).
> I am currently testing a patch that does not deallocate the streams when the
> transaction completes and reuses those for new/subsequent streams.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
