That surprises me. Maybe this is an edge case, but I could imagine that some jobs need to be processed in order, by a single worker that handles one job at a time. Other jobs can be handled in parallel with a big bunch of workers waiting to process jobs as quickly as they arrive. This seems like two separate tubes to me - one with a single worker and one with a load of workers standing by.
That doesn't contradict with what you are suggesting - that the data alone can tell a worker enough about what it needs to do with that data. The tubes can then just be set up to define which jobs are handled in parallel and which are handled serially. Back to the original question, if everything goes through one pipe, then your approach would be to have just one type of worker for all jobs, and that worker would inspect the data and pass it on to whatever framework method is needs to process it. This is the way I am leaning at the moment - keeping the OS configuration simple and building any complexity into the application. A standard wrapper or envelope for jobs pushed onto a tube would need to be defined for the application, so the worker has and "address label" for passing the job data on to the correct method or process. -- Jason On Thursday, 6 December 2012 00:59:30 UTC, Keith Rarick wrote: > > On Wed, Dec 5, 2012 at 4:01 AM, Jason Judge > <[email protected]<javascript:>> > wrote: > > I'm assuming a separate pipe for each type of task will be useful > > All else being equal, it's better to put all jobs through a single tube. > Each job should be a complete description of the work to be done, > so that a worker can pick it up and run it without, for example, > knowing which tube it came from. > > If you have two workers with different capabilities (say, they're > written in different programming languages with different codebases > and process disjoint subsets of jobs), that would be a good reason > to have a tube for each type of worker. > -- You received this message because you are subscribed to the Google Groups "beanstalk-talk" group. To view this discussion on the web visit https://groups.google.com/d/msg/beanstalk-talk/-/OMQENwrj-LAJ. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/beanstalk-talk?hl=en.
