Published [janus](https://pypi.python.org/pypi/janus/0.0.1) -- thread-safe and async-aware queue with limited capacity, as you requested :)
On Saturday, February 28, 2015 at 2:55:59 AM UTC+3, Tin Tvrtković wrote: > > I was contemplating this just now. > > Let's say you want to stream a (huge) file from disk. You can fire off a > reader in a separate thread, using call_in_executor. Now this reader needs > to be able to feed the client chunks of the file as they are read, and also > you'd want backpressure. A thread-safe and async-aware queue (with limited > capacity) seems like the simplest solution by far. It'd be great if > something like this was available from the standard library; I'm not > entirely sure how to create one myself. > > What would the API of this queue look like? Would you need a put/get > version that blocks the thread, and a separate put/get version that blocks > a task? > > On Wednesday, February 25, 2015 at 2:43:54 PM UTC+1, Victor Stinner wrote: >> >> Hi, >> >> On IRC, someone told me that it took him hours to understand that >> asyncio.Queue is not thread-safe and he expected asyncio.Queue to be >> thread-safe. I modified the asyncio documentation to mention in almost >> all classes that asyncio classes are not thread-safe. >> >> I didn't touch the doc of lock classes (ex: asyncio.Lock), because I >> don't know if they are thread safe or not. Since locks should be used >> with "yield from lock", it's not easy to combine them with >> loop.call_soon_threadsafe(). >> >> Maybe we should modify asyncio.Queue and asyncio.Lock to make them >> thread-safe? >> >> What do you think? >> >> Victor >> >
