Re: Throttling based on content in camel
Hi, Looking at the documentation says that ThrottlingInflightRoutePolicy does throttling that is approximate based, meaning that its more coarse grained and not explicit precise as the Throttler (did a poc and coul. Also it looks like the RoutePolicy applies to the entire Route when compared to the Throttler which could act only on a endpoint. Looking at the Throttler features of being precise and capable of acting like a funnel i am considering of extending this throttler for having the support for checking based on content. Any complications that the team thinks which perhaps i am not considering please do let me know.. Regards, -Yogesh -- View this message in context: http://camel.465427.n5.nabble.com/Throttling-based-on-content-in-camel-tp5760505p5760709.html Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Throttling based on content in camel
+1 to create a new ThrottlingContentBasedRoutePolicy, and we can do some refactoring to let ThrottlingContentBasedRoutePolicy and ThrottlingInflightRoutePolicy share some common logic in a super class. -- Willem Jiang Red Hat, Inc. Web: http://www.redhat.com Blog: http://willemjiang.blogspot.com (English) http://jnn.iteye.com (Chinese) Twitter: willemjiang Weibo: 姜宁willem On December 11, 2014 at 11:40:19 PM, yogu13 (yogesh@synechron.com) wrote: > Hello Williem, > > We would like a lock to be implemented instead of any async delay as the > requirement is to allow X number of processing threads to be allowed for a > particular customer at any given point in time. If the number of requests > exceed X then the spilled ones needs to be blocked till any one of the > ongoing threads/flows ends the processing. Also the blocked threads would be > blocked for a finite time exceeding which they would be returned back. > > On quick inspection of the current > org.apache.camel.impl.ThrottlingInflightRoutePolicy i see that instead of > boolean stop = maxInflightExchanges > 0 && size > maxInflightExchanges; > within throttle method if we can externalize the condition for user defined > implementation to return the value then it could do.. > > Also to maintain backward compatibility it would be better if a second > implementation of org.apache.camel.impl.ContentBasedRoutePolicy is made > available. > > However this is just the initial thought ... > > Let me know if you see any issues with this to begin with and would this be > desired ? > > Regards, > -Yogesh > > > > -- > View this message in context: > http://camel.465427.n5.nabble.com/Throttling-based-on-content-in-camel-tp5760505p5760581.html > > Sent from the Camel - Users mailing list archive at Nabble.com. >
Re: Throttling based on content in camel
Hello Daniel, We are looking for throttling but with a slight deviation that we limit the concurrent executions for every client and the exceeded ones will have to wait till a free slot is available. Looking at async model doesnt seem to do the trick for us. Thank you! Regards, -Yogesh -- View this message in context: http://camel.465427.n5.nabble.com/Throttling-based-on-content-in-camel-tp5760505p5760610.html Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Throttling based on content in camel
On 11/12/14 16:38, yogu13 wrote: > Hello Williem, > > We would like a lock to be implemented instead of any async delay as the > requirement is to allow X number of processing threads to be allowed for a > particular customer at any given point in time. If the number of requests > exceed X then the spilled ones needs to be blocked till any one of the > ongoing threads/flows ends the processing. Also the blocked threads would be > blocked for a finite time exceeding which they would be returned back. > Are you looking for throttling or can your requirement simply be met by putting a limit on the number of threads active at one time? I recently looked at the async feature of the Throttler component and I was concerned that it was not clear in the documentation whether messages processed asynchronously after a delay would still be processed within the transaction. One consequence if they are not processed within a transaction is that if the JVM stops abruptly while some messages are delayed then the messages waiting in RAM for asynchronous processing could be lost. One way to deal with this is to put the messages back into the message broker with a scheduled delivery header. The message broker is then responsible for persistence.
Re: Throttling based on content in camel
on further look at the existing code the startConsumer and stopConsumer can also be used for used instead of locks.. Regards, -Yogesh -- View this message in context: http://camel.465427.n5.nabble.com/Throttling-based-on-content-in-camel-tp5760505p5760582.html Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Throttling based on content in camel
Hello Williem, We would like a lock to be implemented instead of any async delay as the requirement is to allow X number of processing threads to be allowed for a particular customer at any given point in time. If the number of requests exceed X then the spilled ones needs to be blocked till any one of the ongoing threads/flows ends the processing. Also the blocked threads would be blocked for a finite time exceeding which they would be returned back. On quick inspection of the current org.apache.camel.impl.ThrottlingInflightRoutePolicy i see that instead of boolean stop = maxInflightExchanges > 0 && size > maxInflightExchanges; within throttle method if we can externalize the condition for user defined implementation to return the value then it could do.. Also to maintain backward compatibility it would be better if a second implementation of org.apache.camel.impl.ContentBasedRoutePolicy is made available. However this is just the initial thought ... Let me know if you see any issues with this to begin with and would this be desired ? Regards, -Yogesh -- View this message in context: http://camel.465427.n5.nabble.com/Throttling-based-on-content-in-camel-tp5760505p5760581.html Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Throttling based on content in camel
We love contribution, please submit your Pull Request once you finish the feature. -- Willem Jiang Red Hat, Inc. Web: http://www.redhat.com Blog: http://willemjiang.blogspot.com (English) http://jnn.iteye.com (Chinese) Twitter: willemjiang Weibo: 姜宁willem On December 11, 2014 at 7:23:22 PM, yogu13 (yogesh@synechron.com) wrote: > Any thoughts ?? or am i addressing a wrong group here ... > > We are looking out for this feature in camel and i would be happy to > contribute this. > > Regards, > -Yogesh > > > > -- > View this message in context: > http://camel.465427.n5.nabble.com/Throttling-based-on-content-in-camel-tp5760505p5760556.html > > Sent from the Camel - Users mailing list archive at Nabble.com. >
Re: Throttling based on content in camel
On December 11, 2014 at 12:55:03 AM, yogu13 (yogesh@synechron.com) wrote: > > Hi, > > We need to do throttling based on id which is received as part of the > request but what I see that the default implementation of > org.apache.camel.impl.ThrottlingInflightRoutePolicy is based on predefined > rule set in the xml, there was a request raised sometime back for a more > dynamic implementation but looks like nothing has happened on this... > wondering if an implementation like below can be desired. > > Two implementations to be a made available as part of camel > > 1. the existing implementation based on the rule set defined in xml > 2. camel to provide an interface having a method boolean > processRequest(Exchange) .. user implements this interface .. the return of > this method is used to obtain a lock… What kind of lock do you want to use? Using lock could introduce some performance issue. Maybe we can take the Asynchronous delaying[1] as an example not to block the calling thread. [1]http://camel.apache.org/delayer.html#Delayer-Asynchronousdelaying > > Let me know the thoughts so i can update this JIRA @ > https://issues.apache.org/jira/browse/CAMEL-5599 > > Regards, > -Yogesh > > > > -- > View this message in context: > http://camel.465427.n5.nabble.com/Throttling-based-on-content-in-camel-tp5760505.html > > Sent from the Camel - Users mailing list archive at Nabble.com. >
Re: Throttling based on content in camel
Any thoughts ?? or am i addressing a wrong group here ... We are looking out for this feature in camel and i would be happy to contribute this. Regards, -Yogesh -- View this message in context: http://camel.465427.n5.nabble.com/Throttling-based-on-content-in-camel-tp5760505p5760556.html Sent from the Camel - Users mailing list archive at Nabble.com.
Throttling based on content in camel
Hi, We need to do throttling based on id which is received as part of the request but what I see that the default implementation of org.apache.camel.impl.ThrottlingInflightRoutePolicy is based on predefined rule set in the xml, there was a request raised sometime back for a more dynamic implementation but looks like nothing has happened on this... wondering if an implementation like below can be desired. Two implementations to be a made available as part of camel 1. the existing implementation based on the rule set defined in xml 2. camel to provide an interface having a method boolean processRequest(Exchange) .. user implements this interface .. the return of this method is used to obtain a lock... Let me know the thoughts so i can update this JIRA @ https://issues.apache.org/jira/browse/CAMEL-5599 Regards, -Yogesh -- View this message in context: http://camel.465427.n5.nabble.com/Throttling-based-on-content-in-camel-tp5760505.html Sent from the Camel - Users mailing list archive at Nabble.com.