Hello Lahiru, I am willing to start a discussion on how to implement java broker replication, However I do agree with Rob, tho, we should spend some time first to address some basic questions.
Here's my initial feeling. Goal: What do we want to achieve with replication ? I'd suggest both HA & performance, that with some adaptation policies I do not consider nor mutually exclusive, nor requiring too much compromise. Approach: Active/passive ? I'd suggest, as long as the state of the replicate object(s) is clearly defined, go with passive replication of a serialized instance of the state. It can always be recreated at failover. How: Well, primary-backup by means of virtual synchrony via group communication. Architecture: Define a replication layer of middleware that provides replication services (obviously), plugged into the broker architecture, well, where it is more convenient. I do not know anything about Qpid architecture, is there any document anywhere ? However I do have some proposals, you can have a look here http://portal.acm.org/citation.cfm?id=1101143 As a starter, I'd need someone to help me understand Qpid architecture, at least an overview. Regards Etienne On Wed, Sep 15, 2010 at 11:20 AM, Lahiru Gunathilake <[email protected]> wrote: > Hi Etienne, > > I would like to start doing some work with Active/Passive clustering as a > initial clustering support for Java broker since according to Rob > implementing Active/Active clustering is bit tricky due to a design issues > in Java broker. For my requirements with our products we can live with > active/passive clustering since we can configure our Enterprise Service bus > to use the passive node to be used only in a failover conditions so we > really do not need to use active/active clustering to address the HA issue > when we are using Apache Qpid with our solutions. > > I would like to start a discussion on how to implement Active/Passive > clustering implementation for Java broker. > > Rob can you please initiate a discussion on this if you think this will be a > useful thing for Java broker. > > Lahiru > > On Tue, Sep 14, 2010 at 4:52 AM, Etienne Antoniutti < > [email protected]> wrote: > >> Hello Everybody, >> >> sorry, noticed this just now, while cleaning my inbox. >> >> Is there any further development on this topic ? >> I'd might be interested in being involved on java clustering. >> >> Regards >> Etienne >> >> >----- Original Message ----- >> >From: [email protected] >> >To: [email protected] >> >Data: Wed, 7 Jul 2010 14:59:43 +0530 >> >Subject: Re: Implementing clustering support for Qpid Java broker >> >> >Hi Rob, >> > >> >Yes true I agree that we need to look in to the architecture and look on >> how >> >should we implement clustering and I just post about the library since >> Sorin >> >just asked about the library and I am currently looking in to the broker >> >architecture before deciding anything and I hope to discuss every step in >> >the list before implementing anything. >> > >> >I am thinking of listing what are we going to achieve by clustering Java >> >broker. Then we can discuss on where should we start and what is the first >> >version of clustering support should do. Then we can work on it and make >> >sure all of them are successfully implemented. >> > >> >Currently what I am thinking is achieving HA without loosing messages >> which >> >is the basic requirement of any clustering approach. >> > >> >Thanks a lot Robert for your reply. can you please elaborate little bit on >> >why should we wait until the AMQP 1-0 support is finished, if there's a >> >pretty good reason for that we can wait until it comes up. >> > >> >Lahiru >> > >> >On Wed, Jul 7, 2010 at 2:47 PM, Robert Godfrey <[email protected] >> >wrote: >> > >> >> First of all we need to be clear about what we mean / want to achieve >> >> by "clustering". >> >> >> >> If we are talking about clustering for reliability/HA then we need to >> >> be very clear that the limitations AMQP 0-8/0-9/0-9-1 prevent us from >> >> actually implementing this unless we mandate that every message >> >> published into the broker is inside a transaction (due to the fact >> >> that published messages are not acknowledges for AMQP 0-x versions >> >> where x < 10). >> >> >> >> The next issue is that the Java broker has not really been designed >> >> with clustering in mind - it's therefore not clear inside the code >> >> which pieces of state need to be replicated / where the broker should >> >> wait to hear that state has been successfully replicated before >> >> proceeding. >> >> >> >> My personal view on clustering is that we need to spend a fair amount >> >> of time thinking about the internal architecture of the broker before >> >> we start worrying about what other libraries / tools we pull in. >> >> >> >> There are probably ways of doing and active-passive cluster relatively >> >> cheaply (essentially replicating the store to the passive broker) - >> >> I'm not sure if this would help your use case Lahiru. >> >> >> >> Having said all the above, I am very keen to add clustering >> >> functionality to the Java Broker, and I've some vague notion that I >> >> should start looking at this later in the year. Personally I'd be >> >> looking at doing it after whatever re-architecting is done for >> >> supporting AMQP 1-0. >> >> >> >> Sorry if I sound discouraging - and if someone can come along and show >> >> an easy way to add clustering, I'd be impressed. >> >> >> >> Cheers, >> >> Rob >> >> >> >> On 7 July 2010 00:30, Lahiru Gunathilake <[email protected]> wrote: >> >> > Hi devs, >> >> > >> >> > Last couple of weeks I was working with Qpid Java broker in an actual >> >> > production environment and I encounter a scenario where we should >> cluster >> >> > Qpid and since we do not have a native clustering support we found a >> >> > solution to the single point of failure of Qpid but it is not an >> actual >> >> > cluster since we do not replicate the messages among nodes. So I would >> be >> >> > really happy to work in a clustering implementation for Qpid. >> >> > >> >> > Appreciate your ideas on this and I will update the list once we >> discuss >> >> a >> >> > proper design for clustering implementation. >> >> > >> >> > Regards >> >> > Lahiru >> >> > >> >> > -- >> >> > Apache Qpid, Worlds dominant messaging middleware..!!! >> >> > >> >> > Senior Software Engineer >> >> > WSO2 Inc >> >> > >> >> >> >> --------------------------------------------------------------------- >> >> Apache Qpid - AMQP Messaging Implementation >> >> Project: http://qpid.apache.org >> >> Use/Interact: mailto:[email protected] >> >> >> >> >> > >> > >> >-- >> >Apache Qpid, Worlds dominant messaging middleware..!!! >> > >> >Senior Software Engineer >> >WSO2 Inc >> > >> >> > --------------------------------------------------------------------- Apache Qpid - AMQP Messaging Implementation Project: http://qpid.apache.org Use/Interact: mailto:[email protected]
