Re: [Twisted-Python] HTTP Agent persistent connections not closed for some HTTPS sites

[Jean-Paul Calderone]

On Fri, Mar 10, 2017 at 3:26 AM, Cory Benfield  wrote:

>
> On 9 Mar 2017, at 22:01, Jean-Paul Calderone 
> wrote:
>
> Run the server with TLSMemoryBIOProtocol and use the transport's
> pauseProducing method before you attempt the shutdown?
>
> Jean-Paul
>
>
> Yup, that will work. Similarly, having two TLSMemoryBIOProtocols
> communicating together and pausing one before calling loseConnection on the
> other should also work. Essentially, any situation you can conjure where
> the closing connection will manage not to see either a TLS CloseNotify or a
> TCP FIN or RST will do the job.
>

Indeed.  For that matter, you can have them interact in-memory (maybe
StringTransport is good enough as-is, maybe not, I'm not sure) and then
just stop shuttling the bytes in one direction.  (I/O-free tests for the
win)

Jean-Paul
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] HTTP Agent persistent connections not closed for some HTTPS sites

[Cory Benfield]

> On 9 Mar 2017, at 22:01, Jean-Paul Calderone  
> wrote:
> 
> Run the server with TLSMemoryBIOProtocol and use the transport's 
> pauseProducing method before you attempt the shutdown?
> 
> Jean-Paul

Yup, that will work. Similarly, having two TLSMemoryBIOProtocols communicating 
together and pausing one before calling loseConnection on the other should also 
work. Essentially, any situation you can conjure where the closing connection 
will manage not to see either a TLS CloseNotify or a TCP FIN or RST will do the 
job.

Cory

___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] HTTP Agent persistent connections not closed for some HTTPS sites

[Jean-Paul Calderone]

On Thu, Mar 9, 2017 at 3:40 PM, Adi Roiban  wrote:

> On 7 March 2017 at 09:31, Cory Benfield  wrote:
> >
> > On 6 Mar 2017, at 09:05, Adi Roiban  wrote:
> >
> > I am not sure about which code are we talking here.
> > The specific HTTP11ClientProtocol which will fix only the HTTP client
> > part or the generic TLSMemoryBIOProtocol code which might fix any TLS
> > connection?
> >
> >
> > I’m talking about the fact that calling TLSMemoryBIOProtocol.
> loseConnection
> > may not ever actually drop the underlying FD is the surprising part of
> the
> > code. We had to work around it in the server side by adding a timeout
> after
> > loseConnection is called to end up calling abortConnection.
> >
> > Cory
>
> In order to submit a patch, I am trying to write an automated tests for
> this.
> Were you able to reproduce this issue in a controlled environment.
>
> I am still not fully understanding why I get this behaviour with
> sharepoint.com sites.
>
> I am thinking at having a rigged TLS server which will keep the socket
> open but will not
> respond to the shutdown request but I still don't know how :)
>
>
Run the server with TLSMemoryBIOProtocol and use the transport's
pauseProducing method before you attempt the shutdown?

Jean-Paul


> --
> Adi Roiban
>
> ___
> Twisted-Python mailing list
> Twisted-Python@twistedmatrix.com
> http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
>
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] HTTP Agent persistent connections not closed for some HTTPS sites

[Adi Roiban]

On 7 March 2017 at 09:31, Cory Benfield  wrote:
>
> On 6 Mar 2017, at 09:05, Adi Roiban  wrote:
>
> I am not sure about which code are we talking here.
> The specific HTTP11ClientProtocol which will fix only the HTTP client
> part or the generic TLSMemoryBIOProtocol code which might fix any TLS
> connection?
>
>
> I’m talking about the fact that calling TLSMemoryBIOProtocol.loseConnection
> may not ever actually drop the underlying FD is the surprising part of the
> code. We had to work around it in the server side by adding a timeout after
> loseConnection is called to end up calling abortConnection.
>
> Cory

In order to submit a patch, I am trying to write an automated tests for this.
Were you able to reproduce this issue in a controlled environment.

I am still not fully understanding why I get this behaviour with
sharepoint.com sites.

I am thinking at having a rigged TLS server which will keep the socket
open but will not
respond to the shutdown request but I still don't know how :)

-- 
Adi Roiban

___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] HTTP Agent persistent connections not closed for some HTTPS sites

[Cory Benfield]

> On 6 Mar 2017, at 09:05, Adi Roiban  wrote:
> 
> I am not sure about which code are we talking here.
> The specific HTTP11ClientProtocol which will fix only the HTTP client
> part or the generic TLSMemoryBIOProtocol code which might fix any TLS
> connection?

I’m talking about the fact that calling TLSMemoryBIOProtocol.loseConnection may 
not ever actually drop the underlying FD is the surprising part of the code. We 
had to work around it in the server side by adding a timeout after 
loseConnection is called to end up calling abortConnection.

Cory___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] HTTP Agent persistent connections not closed for some HTTPS sites

[Adi Roiban]

On 6 March 2017 at 08:17, Cory Benfield  wrote:
>
> On 5 Mar 2017, at 14:25, Tristan Seligmann  wrote:
>
> On Sun, 5 Mar 2017 at 15:36 Adi Roiban  wrote:
>>
>> I have observed this while running some end to end tests in which the
>> pool.closeCachedConnections() deferred was not called, even after a
>> generous amount of seconds :)
>
>
> The code to abort an HTTP client connection is here:
>
> https://github.com/twisted/twisted/blob/twisted-17.1.0/src/twisted/web/_newclient.py#L1657
>
> This calls loseConnection which for a TLS connection will try to do a TLS
> shutdown under most circumstances (in some cases it can't, and will
> abortConnection on the underlying transport instead). If the remote end has
> stopped responding to the connection, I think this may end up hanging
> forever.
>
> I think this code should either call abortConnection directly, or set a
> timer which will abort the connection after a little while if a clean
> shutdown from loseConnection has not completed yet. I’
>
>
> Yeah, this looks right. We hit it in the server side code too: it’s a very
> unintuitive API in that sense.

I am not sure about which code are we talking here.
The specific HTTP11ClientProtocol which will fix only the HTTP client
part or the generic TLSMemoryBIOProtocol code which might fix any TLS
connection?

I have observed this issue on the server side for the case in which a
TLS connection is opened and then closed  without transferring any
data ... and then the handshake will not have the chance to complete.

I think that this is the same issue which was reported for Scrapy

-- 
Adi Roiban

___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] HTTP Agent persistent connections not closed for some HTTPS sites

[Cory Benfield]

> On 5 Mar 2017, at 14:25, Tristan Seligmann  wrote:
> 
> On Sun, 5 Mar 2017 at 15:36 Adi Roiban  > wrote:
> I have observed this while running some end to end tests in which the
> pool.closeCachedConnections() deferred was not called, even after a
> generous amount of seconds :)
> 
> The code to abort an HTTP client connection is here:
> 
> https://github.com/twisted/twisted/blob/twisted-17.1.0/src/twisted/web/_newclient.py#L1657
>  
> 
> 
> This calls loseConnection which for a TLS connection will try to do a TLS 
> shutdown under most circumstances (in some cases it can't, and will 
> abortConnection on the underlying transport instead). If the remote end has 
> stopped responding to the connection, I think this may end up hanging forever.
> 
> I think this code should either call abortConnection directly, or set a timer 
> which will abort the connection after a little while if a clean shutdown from 
> loseConnection has not completed yet. I’

Yeah, this looks right. We hit it in the server side code too: it’s a very 
unintuitive API in that sense.

Cory

___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] HTTP Agent persistent connections not closed for some HTTPS sites

[Paweł Miech]

> It looks like when connected to some  HTTPS servers, the TLS
connection is not successfully closed.
> But with Office365 sharepoint.com sites the TLS shutdown is not completed.
> Has anyone else observed this behaviour ?

I observed something simillar a while ago, there was bug report created for
it in Scrapy. I'm not 100% sure this was same issue as you're discussing
here. Issue in Scrapy: https://github.com/scrapy/scrapy/issues/985 fix in
Scrapy https://github.com/scrapy/scrapy/pull/999 Author of this PR to
Scrapy also created Twisted ticket
https://twistedmatrix.com/trac/ticket/7738

Hope it helps

2017-03-06 7:37 GMT+01:00 Glyph Lefkowitz :

>
> On Mar 5, 2017, at 6:25 AM, Tristan Seligmann 
> wrote:
>
> I think this code should either call abortConnection directly, or set a
> timer which will abort the connection after a little while if a clean
> shutdown from loseConnection has not completed yet. I'
>
>
> Agreed - did someone file a bug? :)
>
> -g
>
> ___
> Twisted-Python mailing list
> Twisted-Python@twistedmatrix.com
> http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
>
>
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] HTTP Agent persistent connections not closed for some HTTPS sites

[Glyph Lefkowitz]

> On Mar 5, 2017, at 6:25 AM, Tristan Seligmann  wrote:
> 
> I think this code should either call abortConnection directly, or set a timer 
> which will abort the connection after a little while if a clean shutdown from 
> loseConnection has not completed yet. I'
> 

Agreed - did someone file a bug? :)

-g___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] HTTP Agent persistent connections not closed for some HTTPS sites

[Tristan Seligmann]

On Sun, 5 Mar 2017 at 15:36 Adi Roiban  wrote:

> I have observed this while running some end to end tests in which the
> pool.closeCachedConnections() deferred was not called, even after a
> generous amount of seconds :)
>

The code to abort an HTTP client connection is here:

https://github.com/twisted/twisted/blob/twisted-17.1.0/src/twisted/web/_newclient.py#L1657

This calls loseConnection which for a TLS connection will try to do a TLS
shutdown under most circumstances (in some cases it can't, and will
abortConnection on the underlying transport instead). If the remote end has
stopped responding to the connection, I think this may end up hanging
forever.

I think this code should either call abortConnection directly, or set a
timer which will abort the connection after a little while if a clean
shutdown from loseConnection has not completed yet. I'
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python