On Tuesday, November 12, 2013 11:23:25 PM UTC+1, LightOfMooN wrote:
>
> I don't want worker without timeout. I just want scheduler task not to be
> stopped because of timeout from one of thousands records. Why scheduler
> doesn't execute tasks with status=='TIMEOUT'?
>
because any task that times out is not a task that anyone should retry
automatically.
>
> I have some function like:
>
> thing = db.executesql(db(db.things.id>0)._select(...,
> orderby=db.things.up_date))[0]
> try:
> get url, parse, ...
> db.executesql(db(db.things.id==thing[0])._update(availability=True,
> up_date=request.now))
> except:
> db.executesql(db(db.things.id==thing[0])._update(availability=False,
> up_date=request.now))
>
> And one wrong URL or 404 break all update system because of timeout, which
> stops all task.
>
>
it should, because you're wanting all of those to happen in a single
transaction.
If your app is consistent with a single "fetch", just do
foreach thing in things:
.....fetch()
....insert()
.....commit()
....mark thing as fetched
and if you want the "fetch" to have a timeout so you can actually control
your task and not having it run away, set a timeout on the fetch, and a
proportional timeout on the task.
Even better, make a single task that fetches the single url, so your
timeoutted tasks will be only the ones that actually timed out.
@limedrop : me doesn't, given that is the most bad design choice a task
processor can implement. If you want to shoot yourself in the foot, a
single task that requeues timeoutted tasks is easy to accomplish, as
@lightofmoon found out.
--
Resources:
- http://web2py.com
- http://web2py.com/book (Documentation)
- http://github.com/web2py/web2py (Source code)
- https://code.google.com/p/web2py/issues/list (Report Issues)
---
You received this message because you are subscribed to the Google Groups
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
For more options, visit https://groups.google.com/groups/opt_out.
