Why not use multiple queues? Q1 is the meta-data that describes the batch job. This triggers the worker to process a batch. Q2 is the list of items in the batch.
For Example: Q1 contains 'job' => '67654', 'queue' => 'q334343' The process creating the Q1 jobs and batch queues runs constantly and only puts a job in Q1 every X minutes, records, whatever. When it does that is starts a new queue for the next batch. Then Beanstalkd queue q334343 contains individual jobs making up the batch. When a batch processor runs (via cron or as a daemon checking Q1) it checks Q1 for a batch to process and processes it if there is one. What am I missing? Brian On Aug 26, 3:19 pm, Andy <[email protected]> wrote: > On Aug 26, 8:33 am, David Strauss <[email protected]> wrote: > > > On Thu, 2010-08-26 at 04:55 -0700, Andy wrote: > > > How do I use cron/Hudson to tell beanstalkd to run jobs? Can you show > > > me some code examples? > > > I was assuming you would put the jobs that need to be batch-processed in > > beanstalkd and have Hudson wrap the batch processing. > > Isn't Hudson a Continuous Integration server, or are you talking about > a different Hudson? > > How do you use it to "wrap" batch processing around beanstalkd? -- You received this message because you are subscribed to the Google Groups "beanstalk-talk" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/beanstalk-talk?hl=en.
