Re: asyncio and blocking - an update

2016-02-12 Thread Chris Angelico
On Fri, Feb 12, 2016 at 7:17 PM, Frank Millman wrote: > Using run_in_executor() and cur.fetchall(), I experience delays of up to 2 > seconds while the task is active. > > Using my approach, the maximum I saw was about a tenth of a second, and that > is because I was looking for it - a normal user

Re: asyncio and blocking - an update

2016-02-12 Thread Frank Millman
"Frank Millman" wrote in message news:n9hjfp$ad7$1...@ger.gmane.org... However, my concern is not to maximise database performance, but to ensure that in an asynchronous environment, one task does not block the others from responding. My tests simulate a number of tasks running concurrently

Re: asyncio and blocking - an update

2016-02-11 Thread Frank Millman
"Chris Angelico" wrote in message news:CAPTjJmor8dMv2TDtq8RHQgWeSAaZgAmxK9gFth=oojhidwh...@mail.gmail.com... So really, the question is: Is this complexity buying you enough performance that it's worthwhile? Indeed, that is the question. Actually, in my case it is not quite the question. F

Re: asyncio and blocking - an update

2016-02-10 Thread Chris Angelico
On Thu, Feb 11, 2016 at 5:36 PM, Frank Millman wrote: > "Chris Angelico" wrote in message > news:CAPTjJmrVCkKAEevc9TW8FYYTnZgRUMPHectz+bD=dqrphxy...@mail.gmail.com... >> >> >> Something worth checking would be real-world database performance metrics > > > [snip lots of valid questions] > > My app

Re: asyncio and blocking - an update

2016-02-10 Thread Frank Millman
"Chris Angelico" wrote in message news:CAPTjJmrVCkKAEevc9TW8FYYTnZgRUMPHectz+bD=dqrphxy...@mail.gmail.com... Something worth checking would be real-world database performance metrics [snip lots of valid questions] My approach is guided by something I read a long time ago, and I don't know h

Re: asyncio and blocking - an update

2016-02-10 Thread Chris Angelico
On Thu, Feb 11, 2016 at 4:45 PM, Frank Millman wrote: > I have come up with a plan that seems to provide a solution. > > Instead of 'putting' one row at a time, let the database handler build up a > block of rows, and then 'put' the block. > > I tried a block of 10, and it ran a lot faster. I incr

Re: asyncio and blocking - an update

2016-02-10 Thread Frank Millman
"Frank Millman" wrote in message news:n9c4p3$gmp$1...@ger.gmane.org... Some of you may have been following my attempts to modify my asyncio app so that it does not block when accessing the database. Here is an update. Here is an update to my update ... I came up with what felt like a good

asyncio and blocking - an update

2016-02-08 Thread Frank Millman
Hi all Some of you may have been following my attempts to modify my asyncio app so that it does not block when accessing the database. Here is an update. I came up with what felt like a good idea. Run the database handler in a separate thread, pass requests to it using a queue.Queue, and get