On Jan 28, 2016 4:13 AM, "Frank Millman" <fr...@chagford.com> wrote: > > "Chris Angelico" wrote in message news:captjjmr162+k4lzefpxrur6wxrhxbr-_wkrclldyr7kst+k...@mail.gmail.com... >> >> >> On Thu, Jan 28, 2016 at 8:13 PM, Frank Millman <fr...@chagford.com> wrote: >> > Run the database handler in a separate thread. Use a queue.Queue to send >> > requests to the handler. Use an asyncio.Queue to send results back to > the >> > caller, which can call 'await q.get()'. >> > >> > I ran a quick test and it seems to work. What do you think? >> >> My gut feeling is that any queue can block at either get or put .... >> > > H'mm, I will have to think about that one, and figure out how to create a worst-case scenario. I will report back on that.
The get and put methods of asyncio queues are coroutines, so I don't think this would be a real issue. The coroutine might block, but it won't block the event loop. If the queue fills up, then effectively the waiting coroutines just become a (possibly unordered) extension of the queue. >> >> The other risk is that the wrong result will be queried (two async >> tasks put something onto the queue - which one gets the first >> result?), which could either be coped with by simple sequencing (maybe >> this happens automatically, although I'd prefer a >> mathematically-provable result to "it seems to work"), or by wrapping >> the whole thing up in a function/class. >> > > I *think* I have this one covered. When the caller makes a request, it creates an instance of an asyncio.Queue, and includes it with the request. The db handler uses this queue to send the result back. > > Do you see any problem with this? That seems reasonable to me. I assume that when you send the result back you would be queuing up individual rows and not just sending a single object across, which could be more easily with just a single future. The main risk of adding limited threads to an asyncio program is that threads make it harder to reason about concurrency. Just make sure the threads don't share any state and you should be good. Note that I can only see queues being used to move data in this direction, not in the opposite. It's unclear to me how queue.get would work from the blocking thread. Asyncio queues aren't threadsafe, but you couldn't just use call_soon_threadsafe since the result is important. You might want to use a queue.Queue instead in that case, but then you run back into the problem of queue.put being a blocking operation. -- https://mail.python.org/mailman/listinfo/python-list
