>Agreed. > >We've got a pluggable discovery mechanism; both for the broker >to >advertise itself or for brokers/clients to discover things, so I >guess >DNS just becomes another provider. >http://activemq.apache.org/discovery.html
Exactly ! The pluggable nature of ActiveMQ is great for enhancements like these. >The difference with DNS is you may use your existing DNS >provider or >might wanna run a custom implementation to get the timeout >stuff etc? Again, right on ! The timeout may not be that big a deal depending on the implementation ... for example, it might be a lazy cleanup process which may or may not be part of the DNS master server. For the clients (ie producers or consumers), if a broker in the list does not respond, it will just timeout to the next (note: 'next' could be as simple as next in the list or based on some policy). James.Strachan wrote: > > Agreed. > > We've got a pluggable discovery mechanism; both for the broker to > advertise itself or for brokers/clients to discover things, so I guess > DNS just becomes another provider. > http://activemq.apache.org/discovery.html > > The difference with DNS is you may use your existing DNS provider or > might wanna run a custom implementation to get the timeout stuff etc? > > 2008/5/19 Sridhar2008 <[EMAIL PROTECTED]>: >> >> >> Thanks for the link. I quickly glanced through the dnsjava website, and >> not >> clear if it requires a specific server for updates to work. >> >> Also, most large organizations have their own legacy DNS in place, with >> some >> home grown way of updating the DNS records. So we need to accommodate >> this >> use case (for example, making 'auto-registration' optional). >> >> >> James.Strachan wrote: >>> >>> Ah cool thanks for that! I was wondering if an existing DNS library >>> might be useful. e.g. >>> >>> http://www.dnsjava.org/ >>> >>> which has a DNS client and server. >>> >>> I guess ZeroConf is kinda similar too; just not requiring a new or >>> custom DNS server (though at the expense of requiring multicast >>> support) >>> >>> 2008/5/19 Sridhar2008 <[EMAIL PROTECTED]>: >>>>> Incidentally I'm intrigued by the DNS discovery stuff; it sounds >>>>> >great >>>>>:). What did you have in mind? That each broker on startup >would >>>>>register a DNS entry (with a timeout maybe so they get >removed) and >>>> >>>> Yes, auto-registration. Couple of things to note: >>>> >>>> - The 'timeout feature' again is DNS master specific, the existing ones >>>> need >>>> to be enhanced to support this aging out. The standard timeout you see >>>> in >>>> the DNS entry is for a different purpose: refresh between master and >>>> slave >>>> DNS servers. >>>> >>>> - There is no standard APY for web dns registration, so we will >>>> define >>>> an >>>> APY, with specific implementatioin to interact with the DNS server >>>> (folks >>>> can chose to change the implementation for their env). >>>> >>>> - Most messaging systems run in trusted env, but some use cases may >>>> need >>>> authentication before a broker is allowed to do 'auto-registration'. >>>> This >>>> can be addressed later. >>>> >>>>>clients would ping DNS when attempting to connect in the >>>>>FailoverTransport? >>>> >>>> Yes, we would specify the cluster name in client config and they would >>>> connect using 'ClusterTransport' :-) >>>> >>>> >>>> James.Strachan wrote: >>>>> >>>>> 2008/5/15 Sridhar2008 <[EMAIL PROTECTED]>: >>>>>> >>>>>> >>>>>> Rob/James, >>>>>> >>>>>> Thanks for the feedback. I will address both of your questions in a >>>>>> combined >>>>>> way :-) >>>>>> >>>>>> Rob> Do you need persistent messaging - or non-persistent only? >>>>>> James> or just pick one broker per operation/transaction? >>>>>> >>>>>> Initially, it is going to be the above. Since the messaging usage >>>>>> scenarios >>>>>> are many, I will just assume the use case of interest for now is >>>>>> durable >>>>>> transfer using cluster for high availability. In this case I am not >>>>>> sure >>>>>> that it buys much to send to multiple brokers simultaneously (see my >>>>>> next >>>>>> paragraph below) - adds additional headaches as James sites below >>>>>> (however, >>>>>> there may be other use cases where this might be useful), in addition >>>>>> to >>>>>> making client-side logic complex. >>>>> >>>>> Agreed. >>>>> >>>>> For sending to a topic (outside of a transaction which may include >>>>> other operations) then sending the message to all brokers is a no >>>>> brainer. I guess so long as acknowledgements only get sent to the >>>>> broker they came from & for queue sending we only send to one of the >>>>> brokers it should be fairly straight forward. >>>>> >>>>> (So a little bit of hacking to the FanOutTransport should do the trick >>>>> I >>>>> think). >>>>> >>>>> We could get clever going forward; where rather than randomly picking >>>>> one of the brokers (or round robbin) we kinda partition destinations >>>>> across the available brokers? >>>>> >>>>> >>>>> >>>>>> Some thoughts: Eventual goal is a broker down in the cluster implies >>>>>> capacity hit and not a service hit. So, I will rather solve this use >>>>>> case >>>>>> as >>>>>> a distributed storage issue (after 'cluster transport' is added) - >>>>>> investigate either a DHT based solution or perhaps something like >>>>>> Hadoop >>>>>> with a JDBC interface. The state is replicated in multiple nodes of >>>>>> the >>>>>> cluster so the broker that is down can be ignored (ie no need to >>>>>> sweat >>>>>> on >>>>>> recovering its state). >>>>> >>>>> Agreed. >>>>> >>>>> FWIW distributing the state is trivial - the FanOutTransport can do >>>>> this today really. The issue is ensuring consistency (so more to do >>>>> with locking & consistency than moving the messages around). >>>>> >>>>> Incidentally I'm intrigued by the DNS discovery stuff; it sounds great >>>>> :). What did you have in mind? That each broker on startup would >>>>> register a DNS entry (with a timeout maybe so they get removed) and >>>>> clients would ping DNS when attempting to connect in the >>>>> FailoverTransport? >>>>> >>>>> -- >>>>> James >>>>> ------- >>>>> http://macstrac.blogspot.com/ >>>>> >>>>> Open Source Integration >>>>> http://open.iona.com >>>>> >>>>> >>>> >>>> -- >>>> View this message in context: >>>> http://www.nabble.com/Cluster-transport-...-tp17221068s2354p17323314.html >>>> Sent from the ActiveMQ - Dev mailing list archive at Nabble.com. >>>> >>>> >>> >>> >>> >>> -- >>> James >>> ------- >>> http://macstrac.blogspot.com/ >>> >>> Open Source Integration >>> http://open.iona.com >>> >>> >> >> -- >> View this message in context: >> http://www.nabble.com/Cluster-transport-...-tp17221068s2354p17323601.html >> Sent from the ActiveMQ - Dev mailing list archive at Nabble.com. >> >> > > > > -- > James > ------- > http://macstrac.blogspot.com/ > > Open Source Integration > http://open.iona.com > > -- View this message in context: http://www.nabble.com/Cluster-transport-...-tp17221068s2354p17323803.html Sent from the ActiveMQ - Dev mailing list archive at Nabble.com.
