[
https://issues.apache.org/jira/browse/TS-3102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14162116#comment-14162116
]
Sudheer Vinukonda edited comment on TS-3102 at 10/7/14 5:01 PM:
----------------------------------------------------------------
Have been running two hosts (started at the same time) with and without the
local SpdyRequest free list for the past 48 hours. Haven't noticed a big
difference in the memory consumption between the two and infact, like [~zwoop]
expected see a (very very slightly) worser memory foot print (74.6% vs 73.9%)
with the local free list version. Will continue the tests to see how the
versions compare if there was an origin outage.
On a related note, I noticed something interesting that may be a "potential"
delay (or even leak??) with Spdy. So, the scenario is that, if there's an
active timeout, HttpSM/HttpClientSession cleanup themselves and are supposed to
clean the netVC, which works perfectly in the non-spdy case. In the SPDY
scenario, however, the HttpClientSession's netVC is actually the pluginVC, on
which also the HttpClientSession invokes do_io_close(). However, this only sets
the passive_vc side of the PluginVC to "deletable" and the PluginVC continues
to wait for the active_vc's "deletable" flag, while holding the iobuffers and
other resources. The active_vc in this case is the FetchSM, which is tied to
the SpdyRequest, which in turn is tied to the SpdyClientSession.
SpdyClientSession does have a timer (inactive/active) associated, but, that is
at the client session level. The problem here is that this timer is reset every
time there's an activity on the session (which could mean a different stream).
So, the request/stream that had the active timeout remains un-released and is
not cleaned up until the client session is eventually released.
Also, to simulate the active timeout, I had a hack to delay HttpSM (basically,
with a sleep), and initiated a POST request with spdycat client. The client
continued to transfer data while the server was delayed resulting in a active
timeout. The HttpSM/HttpClientSession were cleaned up following the active
timeout, but, the PluginVC/FetchSM/SpdyRequest were not cleaned up (as
described above) and this actually resulted in a garbage SYN_REPLY. This
garbage HTTP/0.9 version SYN_REPLY (with status 0) has been a long-standing
issue with SPDY (I think [~briang] also saw this).
[Oct 7 16:57:07.765] Server {0x2af02e95d700} DEBUG: (spdy)
----spdy_send_callback, length:45
[Oct 7 16:57:07.765] Server {0x2af02e95d700} DEBUG: (spdy) ----SEND SYN_REPLY
(sm_id:1, stream_id:1, flag:0, length:37)
[Oct 7 16:57:07.766] Server {0x2af02e95d700} DEBUG: (spdy) :status: 0
[Oct 7 16:57:07.766] Server {0x2af02e95d700} DEBUG: (spdy) :version:
HTTP/0.9
[Oct 7 16:57:07.766] Server {0x2af02e95d700} DEBUG: (spdy) stream_id:1,
call:0, length:4096, already:0
[Oct 7 16:57:07.766] Server {0x2af02e95d700} DEBUG: (spdy) ----Request[1:1]
https://dev1.stg.uff.corp.gq1.yahoo.com/ 64090 0
[Oct 7 16:57:07.766] Server {0x2af02e95d700} DEBUG: (spdy) ----recv md5sum:
was (Author: sudheerv):
Have been running two hosts (started at the same time) with and without the
local SpdyRequest free list for the past 48 hours. Haven't noticed a big
difference in the memory consumption between the two and infact, like [~zwoop]
expected see a (very very slightly) worser memory foot print (74.6% vs 73.9%)
with the local free list version. Will continue the tests to see how the
versions compare if there was an origin outage.
On a related note, I noticed something interesting that may be a "potential"
delay (or even leak??) with Spdy. So, the scenario is that, if there's an
active timeout, HttpSM/HttpClientSession cleanup themselves and are supposed to
clean the netVC in the non-spdy case. In the SPDY scenario, however, the
HttpClientSession's netVC is actually the pluginVC, on which also the
HttpClientSession invokes do_io_close(). However, this only sets the passive_vc
side of the PluginVC to "deletable" and the PluginVC continues to wait for the
active_vc's "deletable" flag, while holding the iobuffers and other resources.
The active_vc in this case is the FetchSM, which is tied to the SpdyRequest,
which in turn is tied to the SpdyClientSession. SpdyClientSession does have a
timer (inactive/active) associated, but, that is at the client session level.
The problem here is that this timer is reset every time there's an activity on
the session (which could mean a different stream). So, the request/stream that
had the active timeout remains un-released and is not cleaned up until the
client session is eventually released.
Also, to simulate the active timeout, I had a hack to delay HttpSM (basically,
with a sleep), and initiated a POST request with spdycat client. The client
continued to transfer data while the server was delayed resulting in a active
timeout. The HttpSM/HttpClientSession were cleaned up following the active
timeout, but, the PluginVC/FetchSM/SpdyRequest were not cleaned up (as
described above) and this actually resulted in a garbage SYN_REPLY. This
garbage HTTP/0.9 version SYN_REPLY (with status 0) has been a long-standing
issue with SPDY (I think [~briang] also saw this).
[Oct 7 16:57:07.765] Server {0x2af02e95d700} DEBUG: (spdy)
----spdy_send_callback, length:45
[Oct 7 16:57:07.765] Server {0x2af02e95d700} DEBUG: (spdy) ----SEND SYN_REPLY
(sm_id:1, stream_id:1, flag:0, length:37)
[Oct 7 16:57:07.766] Server {0x2af02e95d700} DEBUG: (spdy) :status: 0
[Oct 7 16:57:07.766] Server {0x2af02e95d700} DEBUG: (spdy) :version:
HTTP/0.9
[Oct 7 16:57:07.766] Server {0x2af02e95d700} DEBUG: (spdy) stream_id:1,
call:0, length:4096, already:0
[Oct 7 16:57:07.766] Server {0x2af02e95d700} DEBUG: (spdy) ----Request[1:1]
https://dev1.stg.uff.corp.gq1.yahoo.com/ 64090 0
[Oct 7 16:57:07.766] Server {0x2af02e95d700} DEBUG: (spdy) ----recv md5sum:
> Improve memory reuse for SPDY contexts by reusing memory released by streams
> within a client session
> ----------------------------------------------------------------------------------------------------
>
> Key: TS-3102
> URL: https://issues.apache.org/jira/browse/TS-3102
> Project: Traffic Server
> Issue Type: Improvement
> Components: SPDY
> Affects Versions: 5.0.1
> Reporter: Sudheer Vinukonda
> Assignee: Sudheer Vinukonda
> Labels: yahoo
> Fix For: 5.2.0
>
> Attachments: TS-3102.diff
>
>
> In the present SPDY implementation in ATS, there is no client context reuse.
> Though the spdy session is reused, each stream (even the non-concurrent ones)
> is allocated a set of new/separate context buffers (including internal
> plugin_vc, client_session, SM object, header heap, transaction buffers,
> server session objects etc). Some of these objects are allocated from global
> pool, while some are from per-thread pool. The context memory is not reused
> unlike the non-spdy session, where there can be at most one transaction on a
> given connection/session at a time.
> Besides being very inefficient (the allocation/deallocation) this also leads
> to larger memory foot print over time, due to the relatively poor reuse of
> per thread pool objects (especially, when there are a high number of threads
> - e.g 100+ like we have).
> I am currently testing a patch that does not deallocate the streams when the
> transaction completes and reuses those for new/subsequent streams.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
