New submission from Marat Sharafutdinov <[email protected]>:
I want to schedule a lot of parallel tasks, but it becomes slow with loop
blocking:
```python
import asyncio
task_count = 10000
async def main():
for x in range(1, task_count + 1):
asyncio.ensure_future(f(x))
async def f(x):
if x % 1000 == 0 or x == task_count:
print(f'Run f({x})')
await asyncio.sleep(1)
loop.call_later(1, lambda: asyncio.ensure_future(f(x)))
loop = asyncio.get_event_loop()
loop.set_debug(True)
loop.run_until_complete(main())
loop.run_forever()
```
Outputs:
```
Executing <Task finished coro=<main() done, defined at test_aio.py:5>
result=None created at /usr/lib/python3.6/asyncio/base_events.py:446> took
0.939 seconds
...
Executing <TimerHandle when=1841384.785427177
_set_result_unless_cancelled(<Future finis...events.py:275>, None) at
/usr/lib/python3.6/asyncio/futures.py:339 created at
/usr/lib/python3.6/asyncio/tasks.py:480> took 0.113 seconds
...
Executing <Task pending coro=<f() running at test_aio.py:12> wait_for=<Future
pending cb=[<TaskWakeupMethWrapper object at 0x7fe89344fcd8>()] created at
/usr/lib/python3.6/asyncio/base_events.py:275> created at test_aio.py:13> took
0.100 seconds
...
```
What can be another way to schedule a lot of parallel tasks?
----------
components: asyncio
messages: 314207
nosy: asvetlov, decaz, yselivanov
priority: normal
severity: normal
status: open
title: Asyncio loop blocks with a lot of parallel tasks
type: resource usage
versions: Python 3.6
_______________________________________
Python tracker <[email protected]>
<https://bugs.python.org/issue33115>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
