Hi Eugene, Thanks for the patch. I have a question: it seems to me that this patch is a (good) starting point of a broader change to be able to use HA in active/active configuration with RMQ. As far as I know, with that configuration we need to add some extra logic for consumers to deal with "consumer cancellation notification" and duplicated messages due to a (potentially) re-send after a failover.
Is that correct? Is there some plan to have a blueprint for this change? Regards -- Andrea Rosa >-----Original Message----- >From: Eugene Kirpichov [mailto:ekirpic...@gmail.com] >Sent: 26 July 2012 00:46 >To: Alessandro Tagliapietra; rbry...@redhat.com >Cc: Rosa, Andrea (HP Cloud Services); OpenStack Development Mailing >List; openstack@lists.launchpad.net >Subject: Re: [openstack-dev] [Openstack] High Available queues in >rabbitmq > >Gentlemen, > >Here is my patch: https://review.openstack.org/#/c/10305/ >It also depends on another small patch >https://review.openstack.org/#/c/10197 > >I'd like to ask someone to review it. >Also, how to get these changes into nova? It seems that nova has a >copy-paste of openstack-common inside it, should I just mirror the >changes to nova once they're accepted in openstack-common? > >I'm cc'ing Russell Bryant because he originally created the >openstack-common module. > >On Wed, Jul 25, 2012 at 3:03 AM, Alessandro Tagliapietra ><tagliapietra.alessan...@gmail.com> wrote: >> Yup, using as resource is a "old" way as >http://www.rabbitmq.com/ha.html >> Active/active makes sure that you have no downtime and it's simple as >you >> don't need to use DRBD. >> >> 2012/7/25 Rosa, Andrea (HP Cloud Services) <andrea.r...@hp.com> >> >>> Sorry for my question, I have just seen from the original thread that >>> we are talking about HA with Active/Active solution. >>> -- >>> Andrea Rosa >>> >>> >-----Original Message----- >>> >From: Rosa, Andrea (HP Cloud Services) >>> >Sent: 25 July 2012 10:45 >>> >To: Eugene Kirpichov >>> >Cc: openstack-...@lists.openstack.org; Alessandro Tagliapietra; >>> >openstack@lists.launchpad.net >>> >Subject: Re: [openstack-dev] [Openstack] High Available queues in >>> >rabbitmq >>> > >>> >Hi >>> > >>> >Your patch doesn't use a Resource manager, so are you working on an >>> >Active/Active >>> >configuration using mirrored queues? Or are you working on a cluster >>> >configuration? >>> > >>> >I am really interested in that change, thanks for your help. >>> >Regards >>> >-- >>> >Andrea Rosa >>> > >>> >>-----Original Message----- >>> >>From: openstack-bounces+andrea.rosa=hp....@lists.launchpad.net >>> >>[mailto:openstack-bounces+andrea.rosa=hp....@lists.launchpad.net] >On >>> >>Behalf Of Alessandro Tagliapietra >>> >>Sent: 24 July 2012 17:58 >>> >>To: Eugene Kirpichov >>> >>Cc: openstack-...@lists.openstack.org; >openstack@lists.launchpad.net >>> >>Subject: Re: [Openstack] High Available queues in rabbitmq >>> >> >>> >>Oh, so without the need to put an IP floating between hosts. >>> >>Good job, thanks for helping >>> >> >>> >>Best >>> >> >>> >>Alessandro >>> >> >>> >>Il giorno 24/lug/2012, alle ore 17:49, Eugene Kirpichov ha scritto: >>> >> >>> >>> Hi Alessandro, >>> >>> >>> >>> My patch is about removing the need for pacemaker (and it's >pacemaker >>> >>> that I denoted with the term "TCP load balancer"). >>> >>> >>> >>> I didn't submit the patch yesterday because I underestimated the >>> >>> effort to write unit tests for it and found a few issues on the >way. >>> >I >>> >>> hope I'll finish today. >>> >>> >>> >>> On Tue, Jul 24, 2012 at 12:00 AM, Alessandro Tagliapietra >>> >>> <tagliapietra.alessan...@gmail.com> wrote: >>> >>>> Sorry for the delay, i was out from work. >>> >>>> Awesome work Eugene, I don't need the patch instantly as i'm >still >>> >>building the infrastructure. >>> >>>> Will it will take alot of time to go in Ubuntu repositories? >>> >>>> >>> >>>> Why you said you need load balancing? You can use only the >master >>> >>node and in case the rabbitmq-server dies, switch the ip to the new >>> >>master with pacemaker, that's how I would do. >>> >>>> >>> >>>> Best Regards >>> >>>> >>> >>>> Alessadro >>> >>>> >>> >>>> >>> >>>> Il giorno 23/lug/2012, alle ore 21:49, Eugene Kirpichov ha >scritto: >>> >>>> >>> >>>>> +openstack-dev@ >>> >>>>> >>> >>>>> To openstack-dev: this is a discussion of an upcoming patch >about >>> >>>>> native RabbitMQ H/A support in nova. I'll post the patch for >>> >>>>> codereview today. >>> >>>>> >>> >>>>> On Mon, Jul 23, 2012 at 12:46 PM, Eugene Kirpichov >>> >><ekirpic...@gmail.com> wrote: >>> >>>>>> Yup, that's basically the same thing that Jay suggested :) >Obvious >>> >>in >>> >>>>>> retrospect... >>> >>>>>> >>> >>>>>> On Mon, Jul 23, 2012 at 12:42 PM, Oleg Gelbukh >>> >><ogelb...@mirantis.com> wrote: >>> >>>>>>> Eugene, >>> >>>>>>> >>> >>>>>>> I suggest just add option 'rabbit_servers' that will override >>> >>>>>>> 'rabbit_host'/'rabbit_port' pair, if present. This won't >break >>> >>anything, in >>> >>>>>>> my understanding. >>> >>>>>>> >>> >>>>>>> -- >>> >>>>>>> Best regards, >>> >>>>>>> Oleg Gelbukh >>> >>>>>>> Mirantis, Inc. >>> >>>>>>> >>> >>>>>>> >>> >>>>>>> On Mon, Jul 23, 2012 at 10:58 PM, Eugene Kirpichov >>> >><ekirpic...@gmail.com> >>> >>>>>>> wrote: >>> >>>>>>>> >>> >>>>>>>> Hi, >>> >>>>>>>> >>> >>>>>>>> I'm working on a RabbitMQ H/A patch right now. >>> >>>>>>>> >>> >>>>>>>> It actually involves more than just using H/A queues (unless >>> >>you're >>> >>>>>>>> willing to add a TCP load balancer on top of your RMQ >cluster). >>> >>>>>>>> You also need to add support for multiple RabbitMQ's >directly to >>> >>nova. >>> >>>>>>>> This is not hard at all, and I have the patch ready and >tested >>> >in >>> >>>>>>>> production. >>> >>>>>>>> >>> >>>>>>>> Alessandro, if you need this urgently, I can send you the >patch >>> >>right >>> >>>>>>>> now before the discussion codereview for inclusion in core >nova. >>> >>>>>>>> >>> >>>>>>>> The only problem is, it breaks backward compatibility a bit: >my >>> >>patch >>> >>>>>>>> assumes you have a flag "rabbit_addresses" which should look >>> >like >>> >>>>>>>> "rmq-host1:5672,rmq-host2:5672" instead of the prior >rabbit_host >>> >>and >>> >>>>>>>> rabbit_port flags. >>> >>>>>>>> >>> >>>>>>>> Guys, can you advise on a way to do this without being ugly >and >>> >>>>>>>> without breaking compatibility? >>> >>>>>>>> Maybe have "rabbit_host", "rabbit_port" be ListOpt's? But >that >>> >>sounds >>> >>>>>>>> weird, as their names are in singular. >>> >>>>>>>> Maybe have "rabbit_host", "rabbit_port" and also >"rabbit_host2", >>> >>>>>>>> "rabbit_port2" (assuming we only have clusters of 2 nodes)? >>> >>>>>>>> Something else? >>> >>>>>>>> >>> >>>>>>>> On Mon, Jul 23, 2012 at 11:27 AM, Jay Pipes ><jaypi...@gmail.com> >>> >>wrote: >>> >>>>>>>>> On 07/23/2012 09:02 AM, Alessandro Tagliapietra wrote: >>> >>>>>>>>>> Hi guys, >>> >>>>>>>>>> >>> >>>>>>>>>> just an idea, i'm deploying Openstack trying to make it >HA. >>> >>>>>>>>>> The missing thing is rabbitmq, which can be easily started >in >>> >>>>>>>>>> active/active mode, but it needs to declare the queues >adding >>> >>an >>> >>>>>>>>>> x-ha-policy entry. >>> >>>>>>>>>> http://www.rabbitmq.com/ha.html >>> >>>>>>>>>> It would be nice to add a config entry to be able to >declare >>> >>the queues >>> >>>>>>>>>> in that way. >>> >>>>>>>>>> If someone know where to edit the openstack code, else >i'll >>> >try >>> >>to do >>> >>>>>>>>>> that in the next weeks maybe. >>> >>>>>>>>> >>> >>>>>>>>> >>> >>>>>>>>> https://github.com/openstack/openstack- >>> >>common/blob/master/openstack/common/rpc/impl_kombu.py >>> >>>>>>>>> >>> >>>>>>>>> You'll need to add the config options there and the queue >is >>> >>declared >>> >>>>>>>>> here with the options supplied to the ConsumerBase >constructor: >>> >>>>>>>>> >>> >>>>>>>>> >>> >>>>>>>>> https://github.com/openstack/openstack- >>> >>common/blob/master/openstack/common/rpc/impl_kombu.py#L114 >>> >>>>>>>>> >>> >>>>>>>>> Best, >>> >>>>>>>>> -jay >>> >>>>>>>>> >>> >>>>>>>>> _______________________________________________ >>> >>>>>>>>> Mailing list: https://launchpad.net/~openstack >>> >>>>>>>>> Post to : openstack@lists.launchpad.net >>> >>>>>>>>> Unsubscribe : https://launchpad.net/~openstack >>> >>>>>>>>> More help : https://help.launchpad.net/ListHelp >>> >>>>>>>> >>> >>>>>>>> >>> >>>>>>>> >>> >>>>>>>> -- >>> >>>>>>>> Eugene Kirpichov >>> >>>>>>>> http://www.linkedin.com/in/eugenekirpichov >>> >>>>>>>> >>> >>>>>>>> _______________________________________________ >>> >>>>>>>> Mailing list: https://launchpad.net/~openstack >>> >>>>>>>> Post to : openstack@lists.launchpad.net >>> >>>>>>>> Unsubscribe : https://launchpad.net/~openstack >>> >>>>>>>> More help : https://help.launchpad.net/ListHelp >>> >>>>>>> >>> >>>>>>> >>> >>>>>> >>> >>>>>> >>> >>>>>> >>> >>>>>> -- >>> >>>>>> Eugene Kirpichov >>> >>>>>> http://www.linkedin.com/in/eugenekirpichov >>> >>>>> >>> >>>>> >>> >>>>> >>> >>>>> -- >>> >>>>> Eugene Kirpichov >>> >>>>> http://www.linkedin.com/in/eugenekirpichov >>> >>>>> >>> >>>>> _______________________________________________ >>> >>>>> Mailing list: https://launchpad.net/~openstack >>> >>>>> Post to : openstack@lists.launchpad.net >>> >>>>> Unsubscribe : https://launchpad.net/~openstack >>> >>>>> More help : https://help.launchpad.net/ListHelp >>> >>>> >>> >>> >>> >>> >>> >>> >>> >>> -- >>> >>> Eugene Kirpichov >>> >>> http://www.linkedin.com/in/eugenekirpichov >>> >> >>> >> >>> >>_______________________________________________ >>> >>Mailing list: https://launchpad.net/~openstack >>> >>Post to : openstack@lists.launchpad.net >>> >>Unsubscribe : https://launchpad.net/~openstack >>> >>More help : https://help.launchpad.net/ListHelp >>> > >>> >_______________________________________________ >>> >OpenStack-dev mailing list >>> >openstack-...@lists.openstack.org >>> >http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> >> > > > >-- >Eugene Kirpichov >http://www.linkedin.com/in/eugenekirpichov _______________________________________________ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
