[
https://issues.apache.org/jira/browse/TS-3456?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14382822#comment-14382822
]
Susan Hinrichs edited comment on TS-3456 at 3/27/15 6:22 PM:
-------------------------------------------------------------
[~amc] and I talked about this a bit this morning. With your change and
previous observations, it seems likely that the problem is due to some faulty
locking on the queue associated with the nethandler. The vast majority of the
time a single thread is working with a nethandler's queue. But in this case,
the direct reenable would have another thread add/remove items on the
nethandler's queue.
In the short term, we should commit a variant of Lev's fix. In addition, we
should add a thread check and do the direct reenable if the current thread is
the same as the ssl_vc->thread and the code was able to acquire the code. In
Lev's separate thread case it would always do the schedule_imm to get back on
the original thread. I can put up another patch tomorrow or later this evening
for Lev to verify.
In the longer term, we should discuss this issue at the ApacheCon Summit. I
don't think any of us fully understand the details of the threading and locking
requirements. We should at least pool our knowledge and perhaps do some
organized investigation to clarify.
was (Author: shinrich):
[~amc] and I talked about this a bit this morning. With your change and
previous observations, it seems clear that the problem is due to some faulty
locking on the queue associated with the nethandler. The vast majority of the
time a single thread is working with a nethandler's queue. But in this case,
the direct reenable would have another thread add/remove items on the
nethandler's queue.
In the short term, we should commit a variant of Lev's fix. In addition, we
should add a thread check and do the direct reenable if the current thread is
the same as the ssl_vc->thread and the code was able to acquire the code. In
Lev's separate thread case it would always do the schedule_imm to get back on
the original thread. I can put up another patch tomorrow or later this evening
for Lev to verify.
In the longer term, we should discuss this issue at the ApacheCon Summit. I
don't think any of us fully understand the details of the threading and locking
requirements. We should at least pool our knowledge and perhaps do some
organized investigation to clarify.
> SSL blind tunnel sometimes not created
> ---------------------------------------
>
> Key: TS-3456
> URL: https://issues.apache.org/jira/browse/TS-3456
> Project: Traffic Server
> Issue Type: Bug
> Components: Plugins, SSL
> Reporter: Lev Stipakov
> Assignee: Susan Hinrichs
> Fix For: 6.0.0
>
> Attachments: ts-tls.cc
>
>
> Hello,
> I made a simple plugin that sets up TS_SSL_SNI_HOOK and creates a
> blind tunnel from a separate thread. With low load everything works
> fine, but with moderate load (100 simultaneous users, each user sends
> 200 HTTPS requests) I see somewhat strange behavior.
> On a client side I use Tsung, which creates users and sends number of
> requests per user. For each user Tsung waits for a response before
> sending a new request, so if response never arrives, a particular user
> (and the whole test) stalls.
> So, with load mentioned above I see few 'stalled' connections on both
> client and proxy – netstat shows them as ”established”, ATS seems to
> have data structures for those (checked
> proxy.process.net.connections_currently_open value), but no traffic
> goes between proxy and client.
> Client side (.175):
> tcp 0 0 10.133.3.175:40737 10.133.3.250:443 ESTABLISHED 14332/beam.smp
> (more similar connections here)
> Proxy side (.250 is a server):
> tcp 0 0 10.133.3.250:443 10.133.3.175:40737 ESTABLISHED 28117/traffic_serve
> (more similar connections here)
> I checked traffic.out log and found out that
> ”SSLNextProtocolAccept:mainEvent” does not get called as many times as
> it should. This can probably be explained by the fact that client does
> not send requests for given user anymore if response to previous
> request hasn't been received. Which, in turn, may indicate that at
> some point tunnel has not been created.
> The interesting thing is that everything works fine if a tunnel is
> created directly from TS_SSL_SNI_HOOK but not from the separate
> thread.
> The plugin code is very simple – I set up TS_SSL_SNI_HOOK and start a
> thread with TSThreadCreate. When hook got called, I push TSVConn to a
> thread-safe queue. The thread wakes up when item has been pushed,
> calls TSVConnTunnel / TSVConnReenable for given vconn and then waits
> for the next item. I have attached the code.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
