I'm not sure if aiofiles is stable or not, but I would like the actual implementation to be slightly different. aiofiles simply proxies all file IO operations through a thread pool, but there are better alternatives in some cases. Using a thread pool is totally fine for vanilla asyncio, but there should be a way for projects like uvloop/twisted to inject their own implementation of asynchronous file IO.
To do that we probably need to define an abstract file IO interface and add a method to event loop to return an object that implements that interface. For instance: # in asyncio: class asyncio.AbstractFileIOImplementation: async def open(self): raise NotImplementedError async def open(...): return await get_current_loop(). get_file_io_implementation().open(...) # in uvloop or twisted or ...: class uvloop.FileIO(asyncio.AbstractFileIOImplementation): # ... implementation of "open()" class uvloop.Loop: def get_file_io_implementation(self): return self # later in user code: async def main(): file = await asyncio.open(...) A link to this code snippet in case email screws up formatting: https://gist.github.com/1st1/a1936d2f283c1e2f1848c4ed42634ebf Thanks, Yury On Apr 4, 2018, 12:17 PM -0400, Guido van Rossum <[email protected]>, wrote: > Is it stable enough to be merged into the stdlib? Is there anything we can > learn from trio's version? I was impressed by the docs: > http://trio.readthedocs.io/en/latest/reference-io.html#asynchronous-filesystem-i-o > e.g. "Why is async file I/O useful? The answer may surprise you" > > > On Wed, Apr 4, 2018 at 9:07 AM, Andrew Svetlov <[email protected]> > > wrote: > > > aiofiles already exist > > > > > > > On Wed, Apr 4, 2018 at 6:05 PM Guido van Rossum <[email protected]> > > > > wrote: > > > > > There's code in trio for treating the filesystem as async, and > > > > > massive docs about when to use and when not. (Thanks Nathaniel!) We > > > > > should consider adding that to asyncio, perhaps as a 3rd party > > > > > package. > > > > > > > > > > > On Wed, Apr 4, 2018, 01:22 INADA Naoki <[email protected]> > > > > > > wrote: > > > > > > > You're right. logging.FileHandler **may** block. > > > > > > > > > > > > > > More precisely, FileHanlder calls flush, but not fsync or > > > > > > > fdatasync. > > > > > > > So your application won't be blocked unless your application > > > > > > > produce > > > > > > > massive logs. When it is blocked is up to your system (OS > > > > > > > setting, > > > > > > > DISK I/O speed, etc...) > > > > > > > > > > > > > > To avoid potential blocking, I recommend using PIPE. > > > > > > > For example, you can send your log to Apache rotatelogs through > > > > > > > PIPE. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Tue, Apr 3, 2018 at 5:13 PM, cr0hn cr0hn <[email protected]> > > > > > > > wrote: > > > > > > > > Hi folks, > > > > > > > > > > > > > > > > After looking a lot on the internet I can't find an answer for > > > > > > > > that. I > > > > > > > > expose my doubt: > > > > > > > > > > > > > > > > Asyncio can't access to disk without blocking the event-loop. > > > > > > > > right? For > > > > > > > > example if I'm using aiohttp or Sanic, I can't write in a file > > > > > > > > in one of my > > > > > > > > end-points, without block the loop. Then... Can I use standard > > > > > > > > logging > > > > > > > > library with the FileHandler to log into a file without > > > > > > > > blocking the event > > > > > > > > loop? > > > > > > > > > > > > > > > > Thanks in advance! > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > INADA Naoki <[email protected]> > > > -- > > > Thanks, > > > Andrew Svetlov > > > > -- > --Guido van Rossum (python.org/~guido)
