[issue44738] io_uring as a new backend to selectors and asyncio

[Terry J. Reedy]

Terry J. Reedy  added the comment:

Continue the discussion, including adding new info, to the #41271

--
nosy: +terry.reedy
resolution:  -> duplicate
stage:  -> resolved
status: open -> closed
superseder:  -> Add support for io_uring to cpython

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44738] io_uring as a new backend to selectors and asyncio

[Joongi Kim]

Joongi Kim  added the comment:

As in the previous discussion, instead of tackling stdlib right away, it would 
be nice to evaluate the approach using 3rd-party libs, such as trio and/or 
async-tokio, or maybe a new library.

I have a strong feeling that we need to improve the async file I/O.
AFAIK, aiofiles is the only choice we have and it uses a thread pool, which 
involves many more context switches than required.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44738] io_uring as a new backend to selectors and asyncio

[Joongi Kim]

Joongi Kim  added the comment:

Ah, yes, but one year has passed so it may be another chance to discuss its 
adoption, as new advances like tokio_uring became available.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44738] io_uring as a new backend to selectors and asyncio

[Nathaniel Smith]

Nathaniel Smith  added the comment:

I think this is a dupe of issue41271?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44738] io_uring as a new backend to selectors and asyncio

[Joongi Kim]

New submission from Joongi Kim :

This is a rough early idea suggestion on adding io_uring as an alternative I/O 
multiplexing mechanism in Python (maybe selectors and asyncio). io_uring is a 
relatively new I/O mechanism introduced in Linux kernel 5.1 or later.

https://lwn.net/Articles/776703/
https://lwn.net/Articles/810414/
https://blogs.oracle.com/linux/post/an-introduction-to-the-io_uring-asynchronous-io-framework

The advantages of io_uring over epoll:
 - completion-based
 - less number of syscalls
 - higher performance (https://twitter.com/hielkedv/status/1218891982636027905)
 - file I/O support including read/write/stat/open/close

I'm not sure that io_uring would bring actual performance improvements to 
Python (and asyncio) or not yet.
We need some exploration and prototyping, but still technically it would be a 
nice-to-have feature, considering Python's recent speed-up optimizations.
Also io_uring is also intended to support high-speed storage devices such as 
NVMe, and may be a good addition to asyncio in terms of improved async file I/O 
support.

Here are existing attempts to incorporate uring in other languages:
 - liburing (C, https://github.com/axboe/liburing)
 - iou, tokio-uring (Rust, https://tokio.rs/blog/2021-07-tokio-uring)

I don't have any estimation on the efforts and time required to do the work,
but just want to spark the discussion. :)

--
components: asyncio
messages: 398215
nosy: achimnol, asvetlov, corona10, njs, yselivanov
priority: normal
severity: normal
status: open
title: io_uring as a new backend to selectors and asyncio
type: enhancement
versions: Python 3.11

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com