Well....I used to define several endpoints for one and the same servicename. Now specifying for example the RoundRobin policy, you achieved a basic load balancing behaviour when only setting the targetService without the targetEndpoint.
But you are right. A basic load balancing can also be achieved by specifying several jms consumers with different targets. Regards Lars 2009/2/11 Guillaume Nodet <[email protected]>: > Those policies were only for in-NMR routing. I have hardly seen users > using those anyway. > But the clustering solution will offer automatic load-balancing > between smx instances, because of the very nature of JMS where > consumers will compete for the same mesage. > Load-balancing inside the NMR could still be implemented for example > if you have several endpoints implementing the same interface. But > users usually tend to wire things explicitely. Have you ever used > those policies ? > > On Wed, Feb 11, 2009 at 00:02, Lars Heinemann <[email protected]> wrote: >> Another point would be some endpoint chooser policy for such clustered >> environments. There were several strategies/ policies in SMX to >> configure in servicemix.xml (RoundRobinPolicy, RandomChoicePolicy etc.). >> That would be really needed to provide some basic load balancing logic. >> >> Regards >> Lars >> >> >> >> Gert Vanthienen schrieb: >>> Guillaume, >>> >>> Thank you for the clarifications! This kind of clustering support is >>> yet another good reason for people to migrate to ServiceMix 4, very >>> nice work indeed! >>> >>> Regards, >>> >>> Gert >>> >>> >>> Guillaume Nodet wrote: >>>> 2009/2/10 Gert Vanthienen <[email protected]>: >>>> >>>>> Guillaume, >>>>> >>>>> From a user perspective, registering the endpoint through an OSGi >>>>> bundle should be fine. For the JBI packaging, we should take care >>>>> that the implementation doesn't tie the components' build to >>>>> ServiceMix 4 or we run into the same problem we used to have with >>>>> ServiceMix 3. We should probably also take care of ServiceMix 3 >>>>> users by adding a warning or something to the log if they configure >>>>> that flag because it won't do anything there, right? Given that >>>>> these ClusterRegistrations are in the OSGi service registry, I >>>>> suppose we can also add some console commands to enable/disable >>>>> clustering at runtime. >>>>> >>>> >>>> You're referring to adding a property on the endpoint I suppose ? If >>>> we add that on the endpoint exporter (which is smx4 specific, we would >>>> not have to take care about smx3 users). Implementing a set of >>>> commands is a really nice idea. >>>> >>>> >>>>> Once we have this done, we can look into making this cluster engine >>>>> more intelligent, e.g. by automatically detecting exchanges to be >>>>> sent to the engine as you described and perhaps by adding a >>>>> communication mechanism for ServiceMix 4 instances to share >>>>> information about endpoints (and thus allow them to turn on/off >>>>> clustering if the endpoint is available on another node automatically). >>>>> >>>> >>>> Yes, we should be able to add more intelligence later. >>>> >>>> >>>>> Just for clarity: this feature is meant a replacement for the >>>>> JCA/JMS Flow in ServiceMix 3. Would this solution solve all of our >>>>> concerns with those flows or are there still situations where a set >>>>> of JMS endpoints on both ServiceMix instances would be better than >>>>> using the built-in clustering solution? >>>>> >>>> >>>> Yes, I think so. The cluster engine is more scalable, more efficient, >>>> and the fact that the user has to configure on explicitely on a given >>>> endpoint put a slight burden on the user at the benefit of not needing >>>> to send all exchanges in the NMR through the cluster. The engine has >>>> also some kind of throughput control where it can controls the maximum >>>> number of exchanges sent into the NMR at a given time to not overload >>>> it. I can't really think of some use case where it would be better to >>>> have explicit endpoints, unless you really need to have a really fine >>>> grained control over the jms messages sent: for example one could want >>>> some jms messages to be persistent and others not. This is currently >>>> not possible to configure that, though we should be able to enhance >>>> the ClusterRegistration later to hold such informations. >>>> >>>> >>>>> Regards, >>>>> >>>>> Gert >>>>> >>>>> Guillaume Nodet wrote: >>>>> >>>>>> On Mon, Feb 9, 2009 at 13:31, Jean-Baptiste Onofré >>>>>> <[email protected]> wrote: >>>>>> >>>>>> >>>>>>> Hi Guillaume, >>>>>>> >>>>>>> If I have right understood, the ClusterRegistration in the >>>>>>> registry provides a filter. After that, the user needs to define >>>>>>> if the endpoint is "member" of the cluster or not. It means that, >>>>>>> in a cluster, we can have : >>>>>>> - standalone endpoints (dealing only with the local SMX instance) >>>>>>> - clustered endpoints (dealing with all SMX cluster members). Does >>>>>>> it mean that this kind of endpoints are federate in all cluster >>>>>>> members (the endpoint is present in all cluster member) ? >>>>>>> >>>>>>> >>>>>> The ClusterRegistration provides a filter that will match an endpoint >>>>>> set. If the endpoint that created and sent the exchange matches, the >>>>>> exchanges will be re-routed. In such case, the cluster engine will >>>>>> send a JMS message, that will be consumed by another cluster engine. >>>>>> The consumption is based on JMS selectors, so it will only consume >>>>>> messages that will be transformed into a JBI exchange, which target >>>>>> endpoint exists locally. So you can achieve both load-balancing and >>>>>> remoting. >>>>>> >>>>>> Let's take a real example: >>>>>> HTTP consumer -> EIP routing slip -> (xslt transformation , HTTP >>>>>> provider, xslt transformation) >>>>>> This route defines a transformation on the request, send it to another >>>>>> web service, and transform the response back. >>>>>> If you configure the HTTP consumer to be "clustered", then, it will >>>>>> look somewhat like: >>>>>> HTTP -> JMS -> EIP >>>>>> If you deploy this whole application on two instances (assuming the >>>>>> underlying JMS brokers are connected together), an http request coming >>>>>> to the first smx instance could be processed (eip and the remaining of >>>>>> the flow) by another instance (load-balancing / some failover). You >>>>>> could also imagine deploying the HTTP consumer only on one instance, >>>>>> and the remaining of the application only on a second instance >>>>>> (remoting). >>>>>> >>>>>> >>>>>> >>>>>>> If I compare with application servers clustering (with the EJBs >>>>>>> session replication, entity turns, etc), when you setup the >>>>>>> application server in cluster mode, your application works in >>>>>>> cluster mode (you can't choose if it's cluster compliant or not). >>>>>>> Using JBoss Groups, you can register applications in cluster or not. >>>>>>> >>>>>>> Maybe it can be interesting to let the user to choose which >>>>>>> endpoints is cluster one or not with something like, defining a >>>>>>> ClusterRegistration service that store the cluster endpoints set. >>>>>>> Like this, we can manage OSGi and JBI packaging, implicit cluster >>>>>>> and manual cluster. But the user needs to define clustered endpoint. >>>>>>> >>>>>>> >>>>>> Not sure if we can really compare with app servers clustering. If you >>>>>> want true clustering, you need to make sure the whole state can failed >>>>>> over onto another node. Unfortunately, most of the components >>>>>> maintain some state. For example if an endpoint receives an exchange >>>>>> and create a copy of it to send it to another endpoint, wait for the >>>>>> response and then copy it as the response of the original exchange, >>>>>> this original exchange if smx crashes unless it is stored somewhere. >>>>>> The servicemix-eip component has a configurable storage that is used >>>>>> to store all such data. It defaults to an simple map without >>>>>> persistence, but you can configure if to use a database if you want. >>>>>> However, this downgrades performances a lot. >>>>>> Usually, the best way is to put a single cluster endpoint for a whole >>>>>> flow at the beginning of this flow. In our example above, clustering >>>>>> the HTTP consumer should be somewhat sufficient: if the something >>>>>> wrong happens when processing the eip, xslt transformations or the >>>>>> call to the other web service, the transaction can be rolled back and >>>>>> retried on another smx instance. However, if the http client is >>>>>> waiting for a response and the first smx instance (hosting the http >>>>>> consumer) crashes, there's really nothing we can do ... >>>>>> >>>>>> Anyway, an interesting option would be to have a more complex filter >>>>>> that automatically detects exchanges that need to be clustered if for >>>>>> a given flow, no exchange have gone through the cluster engine yet. >>>>>> It would automatically cluster all consumer endpoints, but not >>>>>> subsequent exchanges. >>>>>> >>>>>> >>>>>> >>>>>>> It's only a quick think about SMX cluster. The topic is very >>>>>>> interesting. >>>>>>> >>>>>>> Regards >>>>>>> JB >>>>>>> >>>>>>> On Mon 09/02/09 09:28, "Guillaume Nodet" [email protected] wrote: >>>>>>> >>>>>>> >>>>>>>> I've commited my ongoing work about servicemix 4 clustering at >>>>>>>> https://svn.apache.org/repos/asf/servicemix/smx4/nmr/trunk/jbi/ >>>>>>>> cluster/ >>>>>>>> >>>>>>>> >>>>>>> It's not in the build yet and I committed it for discussion purpose. >>>>>>> >>>>>>> >>>>>>>> This work has two goals: >>>>>>>> * provide some persistence in the JBI layer >>>>>>>> * provide some transparent remoting between JBI endpoints >>>>>>>> >>>>>>>> The way I've began implementing that is to use an >>>>>>>> ExchangeListener in >>>>>>>> the NMR to re-route exchanges to a cluster endpoint (I guess it >>>>>>>> should >>>>>>>> be renamed to something like "cluster engine" to avoid being >>>>>>>> confused >>>>>>>> with "clustered endpoints"). >>>>>>>> >>>>>>>> The org.apache.servicemix.cluster.requestor package dervies from the >>>>>>>> spring message listener container and implements a jms layer >>>>>>>> which is >>>>>>>> able to provide request / response in an asynchronous way. >>>>>>>> >>>>>>>> I've experimented different things, and the one i've been focusing >>>>>>>> lately >>>>>>>> >>>>>>>> >>>>>>> is to use a single JMS queue and selectors. Let me explain a bit. >>>>>>> >>>>>>> >>>>>>>> The JMS flow in servicemix 3 was using lots of different >>>>>>>> destinations >>>>>>>> (one per container + one per endpoint + one per service qname + one >>>>>>>> per >>>>>>>> >>>>>>>> >>>>>>> interface qname). The problem with such a design is that a jms >>>>>>> >>>>>>> >>>>>>>> consumer >>>>>>>> >>>>>>>> >>>>>>> can easily consume only from one destination (unless we use some >>>>>>> >>>>>>> >>>>>>>> specific >>>>>>>> >>>>>>>> >>>>>>> activemq features). Another problem is that if not using activemq, >>>>>>> >>>>>>> >>>>>>>> setting up lots of JMS destinations can be really tedious. The >>>>>>>> use of >>>>>>>> a >>>>>>>> >>>>>>>> >>>>>>> single destination leads to fewer consumers, at the expense of using >>>>>>> >>>>>>> >>>>>>>> jms >>>>>>>> >>>>>>>> >>>>>>> selectors. Previsouly, i've tried to use two queues (one for >>>>>>> >>>>>>> >>>>>>>> requests >>>>>>>> >>>>>>>> >>>>>>> and another one for responses) but there's no real benefits in doing >>>>>>> >>>>>>> >>>>>>>> this >>>>>>>> >>>>>>>> >>>>>>> imho. >>>>>>> >>>>>>> >>>>>>>> The other thing i've been focusing on is to make sure that >>>>>>>> processing >>>>>>>> a >>>>>>>> >>>>>>>> >>>>>>> jms message does not block a thread, and yet be able to use jms or xa >>>>>>> >>>>>>> >>>>>>>> transactions. This is not so easy. For example the spring jms >>>>>>>> listener >>>>>>>> >>>>>>>> >>>>>>> containers do use a thread for consuming the jms message and process >>>>>>> >>>>>>> >>>>>>>> it, >>>>>>>> >>>>>>>> >>>>>>> expecting the processing to happen synchronously. However, in >>>>>>> >>>>>>> >>>>>>>> servicemix >>>>>>>> >>>>>>>> >>>>>>> synchronous processing is a bad idea, as if it involves sending an >>>>>>> >>>>>>> >>>>>>>> http >>>>>>>> >>>>>>>> >>>>>>> request and waiting for a response, this means blocking a thread for >>>>>>> >>>>>>> >>>>>>>> nothing. >>>>>>>> >>>>>>>> >>>>>>> For scalability, we need to not block threads if possible. But >>>>>>> spring >>>>>>> >>>>>>> >>>>>>>> message listener containers only support synchronous processing, so >>>>>>>> I've >>>>>>>> >>>>>>>> >>>>>>> hacked two new containers, one being JMS compliant, and another one >>>>>>> >>>>>>> >>>>>>>> specific >>>>>>>> >>>>>>>> >>>>>>> to ActiveMQ which is much more performant. It uses a >>>>>>> >>>>>>> >>>>>>>> MessageAvailableListener >>>>>>>> >>>>>>>> >>>>>>> to be notified when consumers have messages to be processed >>>>>>> instead of >>>>>>> >>>>>>> >>>>>>>> wasting >>>>>>>> >>>>>>>> >>>>>>> threads to poll actively for messages. >>>>>>> >>>>>>> >>>>>>>> Anyway, both containers can support client ack, jms local >>>>>>>> transactions >>>>>>>> or >>>>>>>> >>>>>>>> >>>>>>> xa transactions in asynchronous mode. >>>>>>> >>>>>>> >>>>>>>> I haven't really worked on how to register such endpoints (from the >>>>>>>> user point of >>>>>>>> view). At the moment, we need to register a ClusterRegistration >>>>>>>> in the >>>>>>>> OSGi >>>>>>>> >>>>>>>> >>>>>>> registry. Such registrations contains a filter that will be used to >>>>>>> >>>>>>> >>>>>>>> decide if a >>>>>>>> new active / consumer exchange should be re-routed to the cluster >>>>>>>> engine or >>>>>>>> not. >>>>>>>> >>>>>>>> >>>>>>> The most simple filter would be a filter that checks the source >>>>>>> endpoint >>>>>>> >>>>>>> >>>>>>>> and >>>>>>>> >>>>>>>> >>>>>>> will thus cluster all exchanges outgoing from a given endpoint. >>>>>>> >>>>>>> >>>>>>>> As for how to register such objects, one way would be to put that on >>>>>>>> the endpoint >>>>>>>> exporter that is used in smx4 to register jbi endpoints with the >>>>>>>> OSGi >>>>>>>> packaging, >>>>>>>> >>>>>>>> >>>>>>> but this would not work with JBI packaging (such registrations >>>>>>> would have >>>>>>> >>>>>>> >>>>>>>> to be >>>>>>>> >>>>>>>> >>>>>>> deployed in a separate osgi bundle). I was also thinking about >>>>>>> adding a >>>>>>> >>>>>>> >>>>>>>> simple >>>>>>>> >>>>>>>> >>>>>>> boolean property on all endpoints, something like clustered="true". >>>>>>> >>>>>>> >>>>>>>> Sorry for the long rant, but I should have sent this email way >>>>>>>> earlier >>>>>>>> ... >>>>>>>> >>>>>>>> >>>>>>> Feedback welcome. >>>>>>> >>>>>>> >>>>>>>> -- >>>>>>>> Cheers, >>>>>>>> Guillaume Nodet >>>>>>>> ------------------------ >>>>>>>> Blog: http://gnodet.blogspot.com/ >>>>>>>> >>>>>>>> >>>>>>> ------------------------ >>>>>>> >>>>>>> >>>>>>>> Open Source SOA >>>>>>>> http://fusesource.com >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>>> >>>>>> >>>>> >>>> >>>> >>>> >>>> >>> >> >> > > > > -- > Cheers, > Guillaume Nodet > ------------------------ > Blog: http://gnodet.blogspot.com/ > ------------------------ > Open Source SOA > http://fusesource.com > -- http://lhein.blogspot.com
