[issue37226] Asyncio Fatal Error on SSL Transport - IndexError Deque Index Out Of Range

[Ben Brown]

Change by Ben Brown :


--
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37226] Asyncio Fatal Error on SSL Transport - IndexError Deque Index Out Of Range

[Ben Brown]

Ben Brown  added the comment:

I can fully confirm the issue is due to flow control or lack of in my code the 
system runs out of resources and this is when it errors, I have implemented 
flow control in my protocol and it now works without errors. One thing I did 
find is that the documentation on flow control was lacking but luckily I found 
this post which makes implementation clearer 
https://medium.com/@pgjones/an-asyncio-socket-tutorial-5e6f3308b8b0

I am not sure if you would still classify this as a bug the error could be 
clearer but it could be expected behaviour when you run out of resources, I 
think this can now be closed I am just unsure of the etiquette as to who closes 
it and what resolution should be selected.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37226] Asyncio Fatal Error on SSL Transport - IndexError Deque Index Out Of Range

[Ben Brown]

Ben Brown  added the comment:

With some more research it looks like the issue is flow related, I experimented 
with the StreamWriter in a simple server and using that plus drain() appears to 
have worked an I no longer get the error. I have added my new server to the 
gist I posted above.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37226] Asyncio Fatal Error on SSL Transport - IndexError Deque Index Out Of Range

[Maayan Keshet]

Maayan Keshet  added the comment:

Hey Ben! Unfortunately our use case is too involved to distill into a neat 
example. We use a websocket library (autobahn) to connect to a server of a 3rd 
party which is outside our control (which is not written in python, but rather 
node.js). We receive the errors on our client.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37226] Asyncio Fatal Error on SSL Transport - IndexError Deque Index Out Of Range

[Ben Brown]

Ben Brown  added the comment:

Maayan Keshet do you have a minimal example it would be interesting to compare 
against our code and see if there is something we are both doing that could 
help narrow down the issue.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37226] Asyncio Fatal Error on SSL Transport - IndexError Deque Index Out Of Range

[Maayan Keshet]

Maayan Keshet  added the comment:

Any updates on this issue? We're also encountering it as part of our websocket 
clients.

--
nosy: +maayank

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37226] Asyncio Fatal Error on SSL Transport - IndexError Deque Index Out Of Range

[Ben Brown]

Ben Brown  added the comment:

Hi is there any update on this issue, were you able to replicate the error with 
the minimal example I provided or is there any additional information I can 
provide.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37226] Asyncio Fatal Error on SSL Transport - IndexError Deque Index Out Of Range

[Andrew Svetlov]

Andrew Svetlov  added the comment:

Thank you!
Yuri, you might be interested too

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37226] Asyncio Fatal Error on SSL Transport - IndexError Deque Index Out Of Range

[Ben Brown]

Ben Brown  added the comment:

I have created a minimal example if that helps to show the issue 
https://gist.github.com/bobthemac/031213b8e37960ee805f2ae1e6990b60

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37226] Asyncio Fatal Error on SSL Transport - IndexError Deque Index Out Of Range

[Ben Brown]

Ben Brown  added the comment:

I tested an older version of OpenSSL 1.0.2g and I get the same error on that

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37226] Asyncio Fatal Error on SSL Transport - IndexError Deque Index Out Of Range

[Ben Brown]

Ben Brown  added the comment:

I am using the version I mentioned above 1.1.0 and TLS 1.2, I am sorry I can't 
be of more help.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37226] Asyncio Fatal Error on SSL Transport - IndexError Deque Index Out Of Range

[Christian Heimes]

Christian Heimes  added the comment:

Is 1.1.0 also a problem? I have only seen issue with 1.1.1 and TLS 1.3.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37226] Asyncio Fatal Error on SSL Transport - IndexError Deque Index Out Of Range

[Ben Brown]

Ben Brown  added the comment:

That's a shame, which version should it work on I don't mind downgrading for 
now to fix the issue as a workaround.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37226] Asyncio Fatal Error on SSL Transport - IndexError Deque Index Out Of Range

[Andrew Svetlov]

Andrew Svetlov  added the comment:

That's what I afraid.
asyncio/sslproto.py fails on new OpenSSL at several places :(

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37226] Asyncio Fatal Error on SSL Transport - IndexError Deque Index Out Of Range

[Andrew Svetlov]

Andrew Svetlov  added the comment:

Just to clarify: what OpenSSL version is used?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37226] Asyncio Fatal Error on SSL Transport - IndexError Deque Index Out Of Range

[Ben Brown]

Ben Brown  added the comment:

The OpenSSL version is OpenSSL 1.1.0j  20 Nov 2018

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37226] Asyncio Fatal Error on SSL Transport - IndexError Deque Index Out Of Range

[Christian Heimes]

Christian Heimes  added the comment:

This looks more like an asyncio problem than an SSL problem. Andrew, Yury,  
please take a look.

--
components: +asyncio
nosy: +asvetlov, yselivanov

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37226] Asyncio Fatal Error on SSL Transport - IndexError Deque Index Out Of Range

[Ben Brown]

New submission from Ben Brown :

I have been getting an intermittent errors when using asyncio with SSL. The 
error always occurs in the _process_write_backlog method in asyncio's 
sslproto.py file. I have looked at lots of possibilities as to what the cause 
is and found that for some reason when in _process_write_backlog's loop the 
deque seems to be empty, I added some quick terrible hacky code to confirm it 
fixed the issue and checking at each point it is used wether it is empty fixes 
the issue, I am unusure as to what causes it to become empty but still run 
through the loop. The most frequent time it happens is after we have a 
successful message the client sends a request to join a data stream this 
request mostly causes the error but sometimes it happens while the client is 
receiving data. I am currently using python 3.7.1 but have also tested my code 
on 3.7.3 with the same result. 

NOTE: I am currently working on a minimal sample to show the issue easier.

Fatal error on SSL transport
protocol: 
transport: <_SelectorSocketTransport fd=38 read=polling write=>
Traceback (most recent call last):
File "/usr/local/lib/python3.7/asyncio/sslproto.py", line 689, in 
_process_write_backlog
del self._write_backlog[0]
IndexError: deque index out of range

Fatal error on SSL transport
protocol: 
transport: <_SelectorSocketTransport fd=29 read=polling write=>
Traceback (most recent call last):
File "/usr/local/lib/python3.7/asyncio/sslproto.py", line 664, in 
_process_write_backlog
data, offset = self._write_backlog[0]
IndexError: deque index out of range

--
assignee: christian.heimes
components: SSL
messages: 345201
nosy: ben.brown, christian.heimes
priority: normal
severity: normal
status: open
title: Asyncio Fatal Error on SSL Transport - IndexError Deque Index Out Of 
Range
type: behavior
versions: Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com