[issue37042] wait_for(coro, timeout=0) memleak

[ixje]

ixje  added the comment:

Perhaps also worth mentioning is that when we supply None as timeout value in 
the `wait_for()` in the minimal sample, then it still reports 60MB memory 
usage. Seems pretty steep for doing basically nothing but looping around.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37042] wait_for(coro, timeout=0) memleak

[Andrew Svetlov]

Andrew Svetlov  added the comment:

Nice, thanks!

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37042] wait_for(coro, timeout=0) memleak

[ixje]

ixje  added the comment:

This is the consumption I'm seeing.

--
Added file: 
https://bugs.python.org/file48357/test_leak_minimal_mem_consumption.png

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37042] wait_for(coro, timeout=0) memleak

[ixje]

ixje  added the comment:

Hi Andrew, 
There is an attached minimal example (that differs from the code given in the 
first comment). I couldn't attach 2 files. So I pasted the code of one file to 
showcase how we could run into the issue, then a minimal reproducible example 
without network code in the attachment.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37042] wait_for(coro, timeout=0) memleak

[Andrew Svetlov]

Andrew Svetlov  added the comment:

Thanks for the report.
If the problem  is in asyncio.wait_for() function the real network code can be 
stripped for the leakage example and replaced with `await asyncio.sleep()`.

Would you try to boil down the snippet by converting it into a code that I can 
execute on my laptop to reproduce the problem?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37042] wait_for(coro, timeout=0) memleak

[ixje]

New submission from ixje :

I have a networked process that looks somewhat like this in its most basic form

```
import asyncio

shutting_down = False

async def read_message(reader, timeout=30):
async def _read(reader: asyncio.StreamReader):
try:
d = await reader.readexactly(24)
# do something with the data
print("I'm never called")
except:
return None

try:
return await asyncio.wait_for(_read(reader), timeout)
except Exception:
return None


async def my_service():
reader, writer = await asyncio.open_connection('127.0.0.1', 20333)
while not shutting_down:
m = await read_message(reader, timeout=0)
if m is None:
continue
# else process message


if __name__ == "__main__":
loop = asyncio.get_event_loop()
loop.create_task(my_service())
loop.run_forever()
```

read_message() has a default timeout of 30, but I thought setting it to 0 
(instead of None) would be equal to blocking. This bleeds 16GB of memory in ~3 
minutes. A minimal example is provided.

I manually applied the patch of https://bugs.python.org/issue36613 to a self 
compiled build of 3.7.3 
(https://github.com/python/cpython/commit/ef4ec6ed12d6c6200a85068f60483723298b6ff4)
 on Ubuntu 18.04 and that did not solve the problem.

--
components: asyncio
files: test_leak_minimal.py
messages: 343470
nosy: asvetlov, ixje, yselivanov
priority: normal
severity: normal
status: open
title: wait_for(coro, timeout=0) memleak
type: resource usage
versions: Python 3.7
Added file: https://bugs.python.org/file48356/test_leak_minimal.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com