Make them failover ciclycally, only one active at a time. Andre
On Thu, Oct 18, 2012 at 9:16 PM, B. Estrade <[email protected]> wrote: > > > On Wednesday, October 17, 2012 2:50:28 PM UTC-5, chadkouse wrote: >> >> I'm not sure about like 90% of what you said but we present our beanstalkd >> pool to our application as a single point of entry maintained by a set of >> haproxy's. works pretty well and is simple enough. >> > > Thanks, I will look at haproxy. > > Cheers, > Brett > >> >> -- >> Chad Kouse >> >> On Wednesday, October 17, 2012 at 12:43 PM, B. Estrade wrote: >> >> I am working on a project that needs a job queue, and beanstalkd is >> looking very attractive. The thing is that we really want to make the >> application very highly available, and the fact that federation of >> beanstalkd is left to the client (like memcached) has lead me to consider >> the following approach. I'd like to create a resource manager that >> maintains a pool of daemons, but presents to the client application the view >> as a single and highly available process that is managing these requests. >> >> I would like to basically create a resource manager that provides failover >> and redundancy via a pool of distributed beanstalkd instances. The >> application, however, will view things logically as a single logical >> instance of the queue. It looks like that the way to go would be to manage >> the pool of beanstalkd instances using Paxos. >> >> My primary question after reading the documentation is, how would one use >> the existing features (as atomic primitives, so to speak) of beanstalkd to >> create a resource manager that provides the isolation guarantees and atomic >> changes to the state of a queue? As far as I can tell, this would be >> possible if a) there was a way to provide a temporary change to the queue >> (e.g., a job or set of jobs) and b) a way to make the temporary changes >> visible in an atomic way ( I suppose a daemon would also need to know if >> it's OK to make the commit, too). I don't think that these needs are unique >> the to distributed commit protocol since this would be required by each >> daemon individually to facilitate the protocol itself. Leader elect, I >> think, can be handled outside of the daemon. >> >> I would like to provide strong consistency across all beanstalkd instances >> in the pool; I am not interested, really, in simply master/slave replication >> (which I suppose one could facilitate via the bin log of a master). I am >> interested in seeing if it is possible to implement a Paxos like protocol >> among all daemons in the pool using beanstalkd's current capabilities. Weak >> or eventual consistency is also an option, but I'd like to see how possible >> the ideal is at the onset. >> >> Thanks! >> Brett >> >> -- >> You received this message because you are subscribed to the Google Groups >> "beanstalk-talk" group. >> To view this discussion on the web visit >> https://groups.google.com/d/msg/beanstalk-talk/-/wnNdSeBC_GkJ. >> To post to this group, send email to [email protected]. >> To unsubscribe from this group, send email to >> [email protected]. >> For more options, visit this group at >> http://groups.google.com/group/beanstalk-talk?hl=en. >> >> > -- > You received this message because you are subscribed to the Google Groups > "beanstalk-talk" group. > To view this discussion on the web visit > https://groups.google.com/d/msg/beanstalk-talk/-/af9VMKZ8oEMJ. > > To post to this group, send email to [email protected]. > To unsubscribe from this group, send email to > [email protected]. > For more options, visit this group at > http://groups.google.com/group/beanstalk-talk?hl=en. -- Andre Ruiz <[email protected]> Curitiba, PR, Brasil Tel +55 (41) 8407-3847 -- You received this message because you are subscribed to the Google Groups "beanstalk-talk" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/beanstalk-talk?hl=en.
