On Sat, May 1, 2010 at 3:32 PM, waratuman <[email protected]> wrote: > 1. Is it possible to get a list of jobs that are in a tube? > 4. Is it possible to get the current job that a worker is working on?
Short answer: client-side logging already provides this information, plus a whole lot more on top, so it's usually easiest and more helpful just to use that. In beanstalkd, to provide a fast system under high load, we've chosen to avoid operations that can bog down. If there are millions of jobs in the queue, you don't want other clients to have to wait while beanstalkd performs an Θ(n log n) sort and copy. Unfortunately, these features are hard to provide in a way that is always fast and still useful. (Note that a worker may have arbitrarily many current jobs.) Fortunately, these features aren't really necessary in the first place. In my experience, in production, introspection is superfluous -- the application and workers should not require hand-holding. If they do, something is wrong with the application design. In development, tail -f on the client's log file does a pretty good job. That having been said, I'm not opposed to adding more introspection abilities; I just want to do it in a way that's in keeping with our high standard for simplicity and speed. I'll send another message soon to discuss using a radix tree instead of a binary heap, so we can have efficient paging for the contents of a tube. > 2. What happens with failed jobs? (this may be client side) There are several options here. As Kurt said, it depends on how you define failure and what sort of failure you are concerned about. If a worker gets wedged (say, in an infinite loop), its reserved jobs will time out, be put back in the queue, and be run by another worker. This is often the desired behavior, but if you want to avoid this, a worker can choose to delete or bury its jobs immediately after reserving them, rather than deleting them after doing the work. For failures that the worker handles explicitly, it can do whatever it considers appropriate, including deleting the job, burying the job, and releasing the job to be tried again (possibly with a different priority or with a time delay). I started to write a bunch of advice here, but turned it into a blog post instead. http://xph.us/2010/05/02/how-to-handle-job-failures.html The gist is this: never clean up jobs by hand. If a failure happens once, it can happen again. Always write code to handle newly-discovered failure types automatically, then run the new code to do the cleanup. > 3. Is it possible to get a list of queues that a worker is watching? This is a perfectly reasonable feature, but we don't currently do it, in no small part because workers don't have names in beanstalkd, so there's no way to refer to them. A client can get a list of its own queues (tubes), but not someone else's. kr -- You received this message because you are subscribed to the Google Groups "beanstalk-talk" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/beanstalk-talk?hl=en.
