I was actually able to easily reproduce this. Just open up 2 web2py
consoles (which connect to the same DB), and do:
scheduler.queue_task('myfunc', pvars=dict(), immediate=True)
The first one will succeed, and the second one will hang there for some
time, and then give the deadlock exception above.
This kinda makes sense, looking at the above DB query & considering
record-level locking. We have conditional writes on all records in the
scheduler_worker table, so if two connections attempt this at the same
time, it would follow that there is a deadlock. (Correct me if I'm wrong.)
So I think the solution would be something like (on line 1303 of
scheduler.py):
if immediate:
- self.db(self.db.scheduler_worker.is_ticker ==
True).update(status=PICK)
+ scheduler_worker_tickers =
self.db(self.db.scheduler_worker.is_ticker == True).select()
+ for scheduler_worker_ticker in scheduler_worker_tickers:
+ scheduler_worker_ticker.update_record(status=PICK)
On Thursday, February 26, 2015 at 3:54:47 PM UTC-8, Osman Masood wrote:
>
> Thanks for the quick response!
>
> I'm using Amazon RDS (just basically MySQL), on a pretty beefy instance
> (db.m3.large), 1000 IOPS. Would be pretty surprised if that were the
> problem.
>
> Running 5 scheduler workers with the default heartbeat (3s).
>
> On Thursday, February 26, 2015 at 3:35:41 PM UTC-8, Niphlod wrote:
>>
>> it seems to me that there is database contention........ although the
>> exception **should** be trapped (i.e. try to set the status to "PICK", if
>> not, well.... it's not a fatal error), there's something wrong with your
>> setup or your database is too much "underpowered" to do what you're asking.
>> How many workers are running and with what heartbeat ?
>>
>> On Friday, February 27, 2015 at 12:28:28 AM UTC+1, Osman Masood wrote:
>>>
>>> Hello all,
>>>
>>> I keep getting this error when calling scheduler.queue_task() with
>>> immediate=True:
>>>
>>> <class 'gluon.contrib.pymysql.err.InternalError'> (1213, u'Deadlock
>>> found when trying to get lock; try restarting transaction')
>>>
>>> The stack trace shows this is the culprit:
>>>
>>> self.db(self.db.scheduler_worker.is_ticker == True).update(status=PICK)
>>>
>>> (This is a web application which uses the scheduler pretty extensively.)
>>> Seems like multiple connections are competing for access to the same
>>> scheduler_worker row.
>>>
>>> Seems to me like a web2py bug. Any help would be greatly appreciated.
>>> Thanks!
>>>
>>>
--
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
---
You received this message because you are subscribed to the Google Groups
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
For more options, visit https://groups.google.com/d/optout.
