Sounds like a good target for Python 3.8 IMHO aiofiles is stable in terms of "not many api changes in last years". But I unlike the implementation: it uses heavy metaprogramming. Saves lines of code but makes understanding and debugging much harder.
On Wed, Apr 4, 2018 at 7:48 PM Yury Selivanov <[email protected]> wrote: > I'm not sure if aiofiles is stable or not, but I would like the actual > implementation to be slightly different. aiofiles simply proxies all file > IO operations through a thread pool, but there are better alternatives in > some cases. Using a thread pool is totally fine for vanilla asyncio, but > there should be a way for projects like uvloop/twisted to inject their own > implementation of asynchronous file IO. > > To do that we probably need to define an abstract file IO interface and > add a method to event loop to return an object that implements that > interface. For instance: > > > # in asyncio: > > class asyncio.AbstractFileIOImplementation: > > async def open(self): > raise NotImplementedError > > async def open(...): > return await get_current_loop(). > get_file_io_implementation().open(...) > > > # in uvloop or twisted or ...: > > class uvloop.FileIO(asyncio.AbstractFileIOImplementation): > # ... implementation of "open()" > > class uvloop.Loop: > def get_file_io_implementation(self): > return self > > > # later in user code: > > async def main(): > file = await asyncio.open(...) > > > A link to this code snippet in case email screws up formatting: > https://gist.github.com/1st1/a1936d2f283c1e2f1848c4ed42634ebf > > Thanks, > Yury > > On Apr 4, 2018, 12:17 PM -0400, Guido van Rossum <[email protected]>, > wrote: > > Is it stable enough to be merged into the stdlib? Is there anything we can > learn from trio's version? I was impressed by the docs: > > http://trio.readthedocs.io/en/latest/reference-io.html#asynchronous-filesystem-i-o > e.g. "Why is async file I/O useful? The answer may surprise you" > > On Wed, Apr 4, 2018 at 9:07 AM, Andrew Svetlov <[email protected]> > wrote: > >> aiofiles already exist >> >> On Wed, Apr 4, 2018 at 6:05 PM Guido van Rossum <[email protected]> >> wrote: >> >>> There's code in trio for treating the filesystem as async, and massive >>> docs about when to use and when not. (Thanks Nathaniel!) We should consider >>> adding that to asyncio, perhaps as a 3rd party package. >>> >>> On Wed, Apr 4, 2018, 01:22 INADA Naoki <[email protected]> wrote: >>> >>>> You're right. logging.FileHandler **may** block. >>>> >>>> More precisely, FileHanlder calls flush, but not fsync or fdatasync. >>>> So your application won't be blocked unless your application produce >>>> massive logs. When it is blocked is up to your system (OS setting, >>>> DISK I/O speed, etc...) >>>> >>>> To avoid potential blocking, I recommend using PIPE. >>>> For example, you can send your log to Apache rotatelogs through PIPE. >>>> >>>> >>>> >>>> >>>> On Tue, Apr 3, 2018 at 5:13 PM, cr0hn cr0hn <[email protected]> wrote: >>>> > Hi folks, >>>> > >>>> > After looking a lot on the internet I can't find an answer for that. I >>>> > expose my doubt: >>>> > >>>> > Asyncio can't access to disk without blocking the event-loop. right? >>>> For >>>> > example if I'm using aiohttp or Sanic, I can't write in a file in one >>>> of my >>>> > end-points, without block the loop. Then... Can I use standard logging >>>> > library with the FileHandler to log into a file without blocking the >>>> event >>>> > loop? >>>> > >>>> > Thanks in advance! >>>> >>>> >>>> >>>> -- >>>> INADA Naoki <[email protected]> >>>> >>> -- >> Thanks, >> Andrew Svetlov >> > > > > -- > --Guido van Rossum (python.org/~guido) > > -- Thanks, Andrew Svetlov
