Rhesa - Thanks for the points... On Aug 16, 7:00 pm, Rhesa Rozendaal <[email protected]> wrote: > I prefer spawning a fixed number of workers, where each one does > "watch-reserve-work-delete". That way you know the amount of workers, even if > you receive a large burst of jobs. Your way might end up forking so much that > you run out of resources.
You are right that spawning a number of "watch-reserve-work-delete- exit" workers is a far better solution that a pre-fork kind of model. I am attempting the latter in the hope that it would require less monitoring than the first. Since I am using Parallel::ForkManager to manage the forking cycle, I am hoping that its natural process limit would prevent the situation of runaway forking. > I have several types of jobs that can take up to 3 days or more to complete > (they depend on highly variable amounts of input), so I use a very high TTR > for those. That works great. That gives me confidence. > I haven't needed that feature yet, but I think it allows you to put jobs away > for processing at some later point. I suppose it works better than > re-releasing with a delay. Since posting this question, I read the protocol docs and have figured that the buried state (whatever else it may be) is useful exactly in the kind of situation I want to solve - preventing the server from re- entering a timed-out task into the queue when you have exceeded its TTR whene large and unpredicatble run-times are involved. What I do is a "reserve-bury-process-delete". As I discovered, beanstalk will not add a buried task back on the queue even if its TTR has been exceeded. In the event you want to re-schedule a failed message processing, you just have to "kick" it and the server, I think, will put it right back if its TTR has been exceeded. Thanks for your help. --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "beanstalk-talk" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [email protected] For more options, visit this group at http://groups.google.com/group/beanstalk-talk?hl=en -~----------~----~----~----~------~----~------~--~---
