Chad, are you able to provide your haproxy configuration for beanstalkd in high availability mode ?
On Thursday, 18 October 2012 08:50:28 UTC+13, chadkouse wrote: > > I'm not sure about like 90% of what you said but we present our > beanstalkd pool to our application as a single point of entry maintained by > a set of haproxy's. works pretty well and is simple enough. > > -- > Chad Kouse > > On Wednesday, October 17, 2012 at 12:43 PM, B. Estrade wrote: > > I am working on a project that needs a job queue, and beanstalkd is > looking very attractive. The thing is that we really want to make the > application very highly available, and the fact that federation of > beanstalkd is left to the client (like memcached) has lead me to consider > the following approach. I'd like to create a resource manager that > maintains a pool of daemons, but presents to the client application the > view as a single and highly available process that is managing these > requests. > > I would like to basically create a resource manager that provides failover > and redundancy via a pool of distributed beanstalkd instances. The > application, however, will view things logically as a single logical > instance of the queue. It looks like that the way to go would be to manage > the pool of beanstalkd instances using Paxos. > > My primary question after reading the documentation is, how would one use > the existing features (as atomic primitives, so to speak) of beanstalkd to > create a resource manager that provides the isolation guarantees and atomic > changes to the state of a queue? As far as I can tell, this would be > possible if a) there was a way to provide a temporary change to the queue > (e.g., a job or set of jobs) and b) a way to make the temporary changes > visible in an atomic way ( I suppose a daemon would also need to know if > it's OK to make the commit, too). I don't think that these needs are > unique the to distributed commit protocol since this would be required by > each daemon individually to facilitate the protocol itself. Leader elect, > I think, can be handled outside of the daemon. > > I would like to provide strong consistency across all beanstalkd instances > in the pool; I am not interested, really, in simply master/slave > replication (which I suppose one could facilitate via the bin log of a > master). I am interested in seeing if it is possible to implement a Paxos > like protocol among all daemons in the pool using beanstalkd's current > capabilities. Weak or eventual consistency is also an option, but I'd like > to see how possible the ideal is at the onset. > > Thanks! > Brett > > -- > You received this message because you are subscribed to the Google Groups > "beanstalk-talk" group. > To view this discussion on the web visit > https://groups.google.com/d/msg/beanstalk-talk/-/wnNdSeBC_GkJ. > To post to this group, send email to [email protected]<javascript:> > . > To unsubscribe from this group, send email to > [email protected] <javascript:>. > For more options, visit this group at > http://groups.google.com/group/beanstalk-talk?hl=en. > > > -- You received this message because you are subscribed to the Google Groups "beanstalk-talk" group. To view this discussion on the web visit https://groups.google.com/d/msg/beanstalk-talk/-/w25lHXeg7LcJ. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/beanstalk-talk?hl=en.
