Hi Chamila, I'm planing to use some thing similar to DNS round robian algorithm and I can you disruptor to handle this.
On Fri, Aug 26, 2016 at 9:37 PM, Chamila De Alwis <[email protected]> wrote: > +1 to this approach, since IMO this is a great feature for the > andes-client itself to have, not a wrapper around it. We can allow several > load-balancing methods additional to round robin, and the user will be able > to set the load-balancing method through the initial properties (JNDI or > properties). > > > Regards, > Chamila de Alwis > Committer and PMC Member - Apache Stratos > Senior Software Engineer | WSO2 > Blog: https://medium.com/@chamilad > > > > On Fri, Aug 26, 2016 at 4:58 AM, Sidath Weerasinghe <[email protected]> > wrote: > >> Hi all, >> >> *-UPDATE-* >> >> Instead off writing OSGI component; modify Andes client and write a >> plug-able discovery class. When the user configured, this class is >> activated. >> >> *Dynamic Discovery* >> >> There are two use cases for discovering IP address. >> >> 1. When the connecting MB clients are on the same network as the server. >> >> The above would be the most typical case since, most often clients (ESB, >> DAS) would be on the same network and in the above case, only the list of >> private IPs would be required. I believe achieving this is straightforward. >> >> 2. When connecting MB clients are outside the network >> >> In this case, we might have to call an API which will be provided by the >> IaaS provided i.e EC2, OpenStack to get the public IPs of the nodes. so the >> implementation should be extensible and allow folks to write adapters for >> each new IaaS implementation and retrieve the list of public IPs. For the >> start, we could write for EC2 and gradually add depending on the time. >> >> >> Basic idea...... >> >> [image: Inline image 1] >> >> >> >> This Static, SameNetwork, AWS types can be configurable. The default can >> be static. >> >> Static - provide IP List >> SameNetwork - Broker nodes are in local network with the client and can >> discover IP address >> AWS - get IP list by calling API >> >> >> *Load Balancing* >> >> In load balancing case I'm going to modify Andes client. >> >> There is an interface called "BrokerDetails" and there is a java classed >> called "AMQBrokerDetails" Which is implements using BrokerDetails. Inside >> that class host and the port is set. >> I'm going insert some mechanism to inside those class for load balance. >> >> The mechanism should be capable of providing host and the port when >> creating the connection. >> >> [image: Inline image 1] >> >> >> >> >> Need further discussion ... >> >> On Fri, Aug 26, 2016 at 8:50 AM, Sidath Weerasinghe <[email protected]> >> wrote: >> >>> Hi Chamila, >>> >>> As we discussed, I am writing separate OSGI component by using Andes >>> client as a dependency. I'm not going to compose failover list, I get the >>> IP address by calling admin service and according to that load balance the >>> client per connections. The periodical task can be configurable. >>> >>> On Fri, Aug 26, 2016 at 12:15 AM, Chamila De Alwis <[email protected]> >>> wrote: >>> >>>> Hi Sidath, >>>> >>>> I'm a bit confused on the use of an OSGi component. AFAIU this "has" to >>>> be handled by the Andes client. Please correct me if I'm wrong. With this >>>> approach, >>>> >>>> 1. One broker URL will be provided >>>> 2. The andes-client will call the proposed admin service and >>>> compose the failover list >>>> 3. URL refresh time could be configurable >>>> >>>> Is this similar to your approach? If it is, what step does an OSGi >>>> component replace? >>>> >>>> >>>> >>>> Regards, >>>> Chamila de Alwis >>>> Committer and PMC Member - Apache Stratos >>>> Senior Software Engineer | WSO2 >>>> Blog: https://medium.com/@chamilad >>>> >>>> >>>> >>>> On Mon, Aug 22, 2016 at 12:05 PM, Prabath Ariyarathna < >>>> [email protected]> wrote: >>>> >>>>> > I think option 1 would work because you have to use andes client as >>>>> a dependency to initiate individual connections with message broker. Also, >>>>> OSGi component should wrap message publishing service and provide it to >>>>> the caller. Inside the OSGi component, you can have logic to load balance >>>>> messages. >>>>> >>>>> Why we need separate OSGI component to handle this because AFAIK >>>>> existing failover logic also inside the andese client. How we are going to >>>>> handle this scenario in the none OSGI environment if we are going to use >>>>> separate OSGi component for implementing the logic. >>>>> >>>>> >>>>> Thanks. >>>>> >>>>> >>>>> >>>>> On Mon, Aug 22, 2016 at 4:39 PM, Sidath Weerasinghe <[email protected]> >>>>> wrote: >>>>> >>>>>> Hi Sajini, >>>>>> >>>>>> The value should be configurable. Initially clients creates then >>>>>> calls the service and user can configure the time value. >>>>>> >>>>>> On Mon, Aug 22, 2016 at 12:17 PM, Sajini De Silva <[email protected]> >>>>>> wrote: >>>>>> >>>>>>> Hi Sidath, >>>>>>> >>>>>>> However since inside the implementation we are calling this service >>>>>>> periodically, shouldn't this time value be configurable? Depending on >>>>>>> the >>>>>>> cluster environment we should be able to tune this value. >>>>>>> >>>>>>> Thanks >>>>>>> >>>>>>> On Sat, Aug 20, 2016 at 8:14 PM, Sidath Weerasinghe <[email protected] >>>>>>> > wrote: >>>>>>> >>>>>>>> Hi Malaka, >>>>>>>> >>>>>>>> Im planing to handle this seamlessly. >>>>>>>> >>>>>>>> On Sat, Aug 20, 2016 at 6:50 AM, Malaka Silva <[email protected]> >>>>>>>> wrote: >>>>>>>> >>>>>>>>> Hi Sidath, >>>>>>>>> >>>>>>>>> Does consumers and/or producers needs to additionally call this >>>>>>>>> services periodically to do this or will it be handled seamlessly? >>>>>>>>> >>>>>>>>> >>>>>>>>> On Fri, Aug 19, 2016 at 8:57 PM, Sidath Weerasinghe < >>>>>>>>> [email protected]> wrote: >>>>>>>>> >>>>>>>>>> Hi Prabath, >>>>>>>>>> >>>>>>>>>> When client establish a connection with a one of the broker node >>>>>>>>>> then client calls this service and get the live IP address of broker >>>>>>>>>> nodes >>>>>>>>>> in cluster. For that we need to know one of the broker IP address. >>>>>>>>>> Then >>>>>>>>>> periodically calling this service and update live IP address >>>>>>>>>> automatically. For that there is no configurations. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On Fri, Aug 19, 2016 at 5:15 PM, Prabath Ariyarathna < >>>>>>>>>> [email protected]> wrote: >>>>>>>>>> >>>>>>>>>>> Hi Sidath. >>>>>>>>>>> >>>>>>>>>>> *Periodically calling this service we can get the dynamically >>>>>>>>>>> adding MB node IP addresses and can remove disconnected IP >>>>>>>>>>> addresses from >>>>>>>>>>> the list. So it will always shuffle the IPs and provide it for the >>>>>>>>>>> client.* >>>>>>>>>>> >>>>>>>>>>> Can you little bit elaborate this statement. >>>>>>>>>>> - What time your going to calling this service? >>>>>>>>>>> - Is this based on the configuration? >>>>>>>>>>> >>>>>>>>>>> Thanks. >>>>>>>>>>> >>>>>>>>>>> On Fri, Aug 19, 2016 at 4:59 PM, Indika Sampath < >>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>> >>>>>>>>>>>> I think option 1 would work because you have to use andes >>>>>>>>>>>> client as a dependency to initiate individual connections with >>>>>>>>>>>> message >>>>>>>>>>>> broker. Also, OSGi component should wrap message publishing >>>>>>>>>>>> service and >>>>>>>>>>>> provide it to the caller. Inside the OSGi component, you can have >>>>>>>>>>>> logic to >>>>>>>>>>>> load balance messages. >>>>>>>>>>>> >>>>>>>>>>>> On Fri, Aug 19, 2016 at 4:25 PM, Sidath Weerasinghe < >>>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> Hi all, >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Message Broker provides means to horizontally scale between >>>>>>>>>>>>> nodes. The client which communicates with the broker is notified >>>>>>>>>>>>> on the >>>>>>>>>>>>> availability of the broker nodes in the cluster via JMS >>>>>>>>>>>>> connection URL, >>>>>>>>>>>>> which allows defining of a failover string (the list of IPs of >>>>>>>>>>>>> the broker >>>>>>>>>>>>> nodes separated by comma). Also, When integrating with other >>>>>>>>>>>>> products (eg- >>>>>>>>>>>>> ESB, DAS); we specify the list of nodes in JNDI.properties file >>>>>>>>>>>>> in the >>>>>>>>>>>>> following manner. >>>>>>>>>>>>> >>>>>>>>>>>>> connectionfactory.QueueConnectionFactory = >>>>>>>>>>>>> amqp://admin:admin@clientID/carbon >>>>>>>>>>>>> failover='roundrobin'&cyclecount='2'&brokerlist='tcp://IP1:5672 >>>>>>>>>>>>> retries='5'&connectdelay='50';tcp://IP2:5672?retries='5'&con >>>>>>>>>>>>> nectdelay='50';tcp://IP3:5672?retries='5'&connectdelay='50'' >>>>>>>>>>>>> >>>>>>>>>>>>> Configurations to the JNDI.properties >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> The above approach would require the user to manually list out >>>>>>>>>>>>> IPs of the broker nodes to the client connection string or the >>>>>>>>>>>>> application >>>>>>>>>>>>> context, which in fact limits the user experience and achieving >>>>>>>>>>>>> the >>>>>>>>>>>>> scalability goals in the following way, >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> 1. >>>>>>>>>>>>> >>>>>>>>>>>>> The user/developers having to manually list out the IPs in >>>>>>>>>>>>> the connection URL could be prone to error. >>>>>>>>>>>>> 2. >>>>>>>>>>>>> >>>>>>>>>>>>> The change of IPs of the broker nodes would require >>>>>>>>>>>>> reconfiguring and restart the client applications connected to >>>>>>>>>>>>> it. >>>>>>>>>>>>> 3. >>>>>>>>>>>>> >>>>>>>>>>>>> The approach would also be a blocker for auto scaling >>>>>>>>>>>>> between broker nodes. (Since addition of IPs needs to be >>>>>>>>>>>>> manually reflected >>>>>>>>>>>>> in the client application.The manual process would only >>>>>>>>>>>>> address failover >>>>>>>>>>>>> between broker nodes. This limits the client applications to >>>>>>>>>>>>> load balance >>>>>>>>>>>>> between the broker nodes) >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> How existing system works >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Proposed solution >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> - >>>>>>>>>>>>> >>>>>>>>>>>>> Specify an admin service inside the MB. where, the broker >>>>>>>>>>>>> clients could query the service to get the list of IPs in the >>>>>>>>>>>>> broker >>>>>>>>>>>>> cluster. >>>>>>>>>>>>> - >>>>>>>>>>>>> >>>>>>>>>>>>> Via calling an admin service which is implemented inside >>>>>>>>>>>>> the MB; can discover available MB nodes in the cluster. >>>>>>>>>>>>> - >>>>>>>>>>>>> >>>>>>>>>>>>> Periodically calling this service we can get the >>>>>>>>>>>>> dynamically adding MB node IP addresses and can remove >>>>>>>>>>>>> disconnected IP >>>>>>>>>>>>> addresses from the list. So it will always shuffle the IPs and >>>>>>>>>>>>> provide it >>>>>>>>>>>>> for the client. >>>>>>>>>>>>> - >>>>>>>>>>>>> >>>>>>>>>>>>> According to the live MB node IP address, I will create >>>>>>>>>>>>> the connections and use this connection as round robin fashion >>>>>>>>>>>>> or use >>>>>>>>>>>>> disruptor for load balancing purpose. >>>>>>>>>>>>> >>>>>>>>>>>>> [image: pic 2] >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> There are several ideas for doing this. >>>>>>>>>>>>> >>>>>>>>>>>>> 1. Build separate OSGI component >>>>>>>>>>>>> 2. Modify the Andes client according to the above scenario. >>>>>>>>>>>>> >>>>>>>>>>>>> Any ideas? >>>>>>>>>>>>> >>>>>>>>>>>>> -- >>>>>>>>>>>>> Thank You, >>>>>>>>>>>>> Best Regards, >>>>>>>>>>>>> >>>>>>>>>>>>> Sidath Weerasinghe >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> *Intern* >>>>>>>>>>>>> >>>>>>>>>>>>> *WSO2, Inc. * >>>>>>>>>>>>> >>>>>>>>>>>>> *lean . enterprise . middleware * >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> *Mobile: +94719802550 <%2B94%20777764144>* >>>>>>>>>>>>> >>>>>>>>>>>>> *Email: *[email protected] <[email protected]> >>>>>>>>>>>>> >>>>>>>>>>>>> _______________________________________________ >>>>>>>>>>>>> Architecture mailing list >>>>>>>>>>>>> [email protected] >>>>>>>>>>>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> -- >>>>>>>>>>>> Indika Sampath >>>>>>>>>>>> Senior Software Engineer >>>>>>>>>>>> WSO2 Inc. >>>>>>>>>>>> http://wso2.com >>>>>>>>>>>> >>>>>>>>>>>> Phone: +94 716 424 744 >>>>>>>>>>>> Blog: http://indikasampath.blogspot.com/ >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> _______________________________________________ >>>>>>>>>>>> Architecture mailing list >>>>>>>>>>>> [email protected] >>>>>>>>>>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> -- >>>>>>>>>>> >>>>>>>>>>> *Prabath Ariyarathna* >>>>>>>>>>> >>>>>>>>>>> *Associate Technical Lead* >>>>>>>>>>> >>>>>>>>>>> *WSO2, Inc. * >>>>>>>>>>> >>>>>>>>>>> *lean . enterprise . middleware * >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> *Email: [email protected] <[email protected]>* >>>>>>>>>>> >>>>>>>>>>> *Blog: http://prabu-lk.blogspot.com >>>>>>>>>>> <http://prabu-lk.blogspot.com>* >>>>>>>>>>> >>>>>>>>>>> *Flicker : https://www.flickr.com/photos/47759189@N08 >>>>>>>>>>> <https://www.flickr.com/photos/47759189@N08>* >>>>>>>>>>> >>>>>>>>>>> *Mobile: +94 77 699 4730 * >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> _______________________________________________ >>>>>>>>>>> Architecture mailing list >>>>>>>>>>> [email protected] >>>>>>>>>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> -- >>>>>>>>>> Thank You, >>>>>>>>>> Best Regards, >>>>>>>>>> >>>>>>>>>> Sidath Weerasinghe >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> *Intern* >>>>>>>>>> >>>>>>>>>> *WSO2, Inc. * >>>>>>>>>> >>>>>>>>>> *lean . enterprise . middleware * >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> *Mobile: +94719802550 <%2B94%20777764144>* >>>>>>>>>> >>>>>>>>>> *Email: *[email protected] <[email protected]> >>>>>>>>>> >>>>>>>>>> _______________________________________________ >>>>>>>>>> Architecture mailing list >>>>>>>>>> [email protected] >>>>>>>>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> -- >>>>>>>>> >>>>>>>>> Best Regards, >>>>>>>>> >>>>>>>>> Malaka Silva >>>>>>>>> Senior Technical Lead >>>>>>>>> M: +94 777 219 791 >>>>>>>>> Tel : 94 11 214 5345 >>>>>>>>> Fax :94 11 2145300 >>>>>>>>> Skype : malaka.sampath.silva >>>>>>>>> LinkedIn : http://www.linkedin.com/pub/malaka-silva/6/33/77 >>>>>>>>> Blog : http://mrmalakasilva.blogspot.com/ >>>>>>>>> >>>>>>>>> WSO2, Inc. >>>>>>>>> lean . enterprise . middleware >>>>>>>>> https://wso2.com/signature >>>>>>>>> http://www.wso2.com/about/team/malaka-silva/ >>>>>>>>> <http://wso2.com/about/team/malaka-silva/> >>>>>>>>> https://store.wso2.com/store/ >>>>>>>>> >>>>>>>>> Don't make Trees rare, we should keep them with care >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> Thank You, >>>>>>>> Best Regards, >>>>>>>> >>>>>>>> Sidath Weerasinghe >>>>>>>> >>>>>>>> >>>>>>>> *Intern* >>>>>>>> >>>>>>>> *WSO2, Inc. * >>>>>>>> >>>>>>>> *lean . enterprise . middleware * >>>>>>>> >>>>>>>> >>>>>>>> *Mobile: +94719802550 <%2B94%20777764144>* >>>>>>>> >>>>>>>> *Email: *[email protected] <[email protected]> >>>>>>>> >>>>>>>> _______________________________________________ >>>>>>>> Architecture mailing list >>>>>>>> [email protected] >>>>>>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>>>>>>> >>>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> Sajini De SIlva >>>>>>> Senior Software Engineer; WSO2 Inc.; http://wso2.com , >>>>>>> Email: [email protected] >>>>>>> Blog: http://sajinid.blogspot.com/ >>>>>>> Git hub profile: https://github.com/sajinidesilva >>>>>>> >>>>>>> Phone: +94 712797729 >>>>>>> >>>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> Thank You, >>>>>> Best Regards, >>>>>> >>>>>> Sidath Weerasinghe >>>>>> >>>>>> >>>>>> *Intern* >>>>>> >>>>>> *WSO2, Inc. * >>>>>> >>>>>> *lean . enterprise . middleware * >>>>>> >>>>>> >>>>>> *Mobile: +94719802550 <%2B94719802550>* >>>>>> >>>>>> *Email: *[email protected] >>>>>> >>>>>> Blog: https://medium.com/@sidath >>>>>> >>>>>> Linkedin: https://lk.linkedin.com/in/sidathweerasinghe >>>>>> >>>>>> _______________________________________________ >>>>>> Architecture mailing list >>>>>> [email protected] >>>>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>>>>> >>>>>> >>>>> >>>>> >>>>> -- >>>>> >>>>> *Prabath Ariyarathna* >>>>> >>>>> *Associate Technical Lead* >>>>> >>>>> *WSO2, Inc. * >>>>> >>>>> *lean . enterprise . middleware * >>>>> >>>>> >>>>> *Email: [email protected] <[email protected]>* >>>>> >>>>> *Blog: http://prabu-lk.blogspot.com <http://prabu-lk.blogspot.com>* >>>>> >>>>> *Flicker : https://www.flickr.com/photos/47759189@N08 >>>>> <https://www.flickr.com/photos/47759189@N08>* >>>>> >>>>> *Mobile: +94 77 699 4730 * >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> _______________________________________________ >>>>> Architecture mailing list >>>>> [email protected] >>>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>>>> >>>>> >>>> >>>> _______________________________________________ >>>> Architecture mailing list >>>> [email protected] >>>> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >>>> >>>> >>> >>> >>> -- >>> Thank You, >>> Best Regards, >>> >>> Sidath Weerasinghe >>> >>> >>> *Intern* >>> >>> *WSO2, Inc. * >>> >>> *lean . enterprise . middleware * >>> >>> >>> *Mobile: +94719802550 <%2B94719802550>* >>> >>> *Email: *[email protected] >>> >>> Blog: https://medium.com/@sidath >>> >>> Linkedin: https://lk.linkedin.com/in/sidathweerasinghe >>> >> >> >> >> -- >> Thank You, >> Best Regards, >> >> Sidath Weerasinghe >> >> >> *Intern* >> >> *WSO2, Inc. * >> >> *lean . enterprise . middleware * >> >> >> *Mobile: +94719802550 <%2B94719802550>* >> >> *Email: *[email protected] >> >> Blog: https://medium.com/@sidath >> >> Linkedin: https://lk.linkedin.com/in/sidathweerasinghe >> >> _______________________________________________ >> Architecture mailing list >> [email protected] >> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >> >> > > _______________________________________________ > Architecture mailing list > [email protected] > https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture > > -- Thank You, Best Regards, Sidath Weerasinghe *Intern* *WSO2, Inc. * *lean . enterprise . middleware * *Mobile: +94719802550* *Email: *[email protected] Blog: https://medium.com/@sidath Linkedin: https://lk.linkedin.com/in/sidathweerasinghe
_______________________________________________ Architecture mailing list [email protected] https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
