Hi Chad, Some of the jobs I have running in the background are jobs that pull data from various 3rd party web services. Many of those web services are a throttle limit. So what I do now is grab the job from the queue, then see what kind of job it is and if check if I have enough hits left before I get throttled. If I have enough hits left then I process the job, otherwise I have to put the job back on the queue.
On Dec 26 2011, 12:06 pm, Chad Kouse <[email protected]> wrote: > I think I'd need to know what types of situations make your workers > able/not able to process jobs. > There is the concept of delaying a job that could work if your > situation allows it. > > --chad > > On Dec 25, 2011, at 10:48 PM, Jason <[email protected]> wrote: > > > > > > > > > I have certain jobs that get put on to the queue and when they are > > pulled off the queue to be processed, my worker must check if the job > > can be processed based on the information contained in the job. Often > > times (10's of thousands of jobs per day) the worker will find out > > that the job can't be processed and so the job is placed back on the > > queue. This happens for many of my jobs. Is there a more efficient/ > > better way that I should be doing this instead of popping jobs off the > > queue, checking them, and then throwing them back onto the queue for > > later processing? > > > -- > > You received this message because you are subscribed to the Google Groups > > "beanstalk-talk" group. > > To post to this group, send email to [email protected]. > > To unsubscribe from this group, send email to > > [email protected]. > > For more options, visit this group > > athttp://groups.google.com/group/beanstalk-talk?hl=en. -- You received this message because you are subscribed to the Google Groups "beanstalk-talk" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/beanstalk-talk?hl=en.
