> Personally, I disagree. If I have an outage (whether network outage or > worker machine catching fire), I absolutely do not want jobs stuck > waiting for the host or network to be restored. I think in the face of > an outage, it's entirely appropriate to recover the jobs for > delegation to a new worker to take over the processing.
But this may complete break the logic of someones application if jobs are not idempotent[1]. If you have a minor network outage you may end up with jobs being done multiple times which may really mess things up. This is not always an appropriate assumption to make. However the assumption of "if you make a mess you need to handle the mess" is safer on the side of beanstalkd (and makes beanstalkd much easier to program). >And it also > seems entirely appropriate that the connection to the job broker is as > good as any other form of heartbeat. If you do need disconnected > processing, it would seem that you could reserve, bury and disconnect, > process, and then reconnect to delete the job. I think the problem here is more philosophical. Should recovery logic and idempotency[1] be the job of beanstalkd, or of the application? beanstalkd is a generic software solution for queues, you can't stick business rules or logic in it (yet, and hopefully never). Should beanstalkd try to be more "clever"? To have the kind of recovery you are proposing we need to add a lot of business level logic (do we handle failure, how do we handle failure, what about all the scorner/special cases? do we give beanstalkd the ability to inspect and understand jobs? etc.), personally I think this is far better done in your application than in beanstalkd. If you need to architect intelligence into your message queue it's not a message queue anymore, it's an application. [1]http://en.wikipedia.org/wiki/Idempotence > -- William -- Kurt Seifried [email protected] tel: 1-703-879-3176 -- You received this message because you are subscribed to the Google Groups "beanstalk-talk" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/beanstalk-talk?hl=en.
