well, you're definitely a power-user if you're submitting 1000 tasks/minute. If you're using a high number of workers you're a perfect candidate for a new incarnation of the scheduler (yet to come, but it's going to be "actual" soon) that uses redis as the coordinator instead of the scheduler_worker table, and relevies the db for a lot of pressure. I'm not going to put a +1 on the patch because the method you're choosing is not "atomic" (and I don't see it as the real issue), requiring two queries that could update the wrong records. queue_task() DOESN'T use the scheduler_worker table, unless you're using immediate=True. Queuing 1000 tasks/minute, immediate should be completely irrelevant. Are you using it? If yes, why ?
What you should be careful of is IMHO the number of workers and the heartbeat value. Can I have more details on those figures please ? PS: without proper "pruning" of the scheduler_task table, you'll probably benefit of indexes here and there to help the db alleviate the relative pressure of the queries it does. Without figures, though, I can't say more. -- Resources: - http://web2py.com - http://web2py.com/book (Documentation) - http://github.com/web2py/web2py (Source code) - https://code.google.com/p/web2py/issues/list (Report Issues) --- You received this message because you are subscribed to the Google Groups "web2py-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
