Re: Question on multicast to pipelines
HI, I created a jira (CAMEL-10442) for this in case it’s a bug. Thanks for your help. Best, Edoado > On 1 Nov 2016, at 19:30, Brad Johnson <brad.john...@mediadriver.com> wrote: > > Edoardo, > > I missed the second set of logging statements, sorry. I thought you'd said > they were outputting the same thing. And, yes, that's a bit > counter-intuitive. Personally I almost always do it with just to routes in > the mulitcast and then any further routing or changes I put in those. And I > use blueprint for this though I'm slowly switching the Java DSL. > > > ><multicast"> > > > > > > > >///modify the body here > > > > > > ///log message here > > > On Tue, Nov 1, 2016 at 12:27 PM, DariusX <dariuscoo...@gmail.com> wrote: > >> Ah, I didn't understand what you were saying before. >> Your point is that these two should be synonymous, even if they're enclosed >> in a multicast()...end(): >> >> 1) This... >> .*pipeline("direct:A", "direct:B")* >> >> 2) and this... >> .*pipeline().to("direct:C").to("direct:D").end()* >> >> but only the second one works the way you expect. >> The first one sends the same input to both routes (i.e. does not send the >> output of A to B) >> >> >> It does seem odd, and a brief test confirms the behavior you describe ( >> Sample code here >> <https://github.com/DariusX/CamelSandbox/blob/master/ >> CamelSandbox/src/main/java/com/zerses/camelsandbox/ >> MulticastPipelinesTest.java)> >> ) >> >> Looking at the Camel code - Line 1165 >> <https://github.com/apache/camel/blob/master/camel-core/ >> src/main/java/org/apache/camel/model/ProcessorDefinition.java> >> , that version of pipeline(String...uri) is simply a synonym for >> to(String...uri), which would explain how it is working. >> >> Someone else would need to speak to how it *ought* to work, but I agree it >> does not seem intuitive as-is. >> >> >> >> >> >> >> -- >> View this message in context: http://camel.465427.n5.nabble. >> com/Question-on-multicast-to-pipelines-tp5789396p5789593.html >> Sent from the Camel - Users mailing list archive at Nabble.com. >>
Re: Question on multicast to pipelines
Edoardo, I missed the second set of logging statements, sorry. I thought you'd said they were outputting the same thing. And, yes, that's a bit counter-intuitive. Personally I almost always do it with just to routes in the mulitcast and then any further routing or changes I put in those. And I use blueprint for this though I'm slowly switching the Java DSL. <multicast"> ///modify the body here ///log message here On Tue, Nov 1, 2016 at 12:27 PM, DariusX <dariuscoo...@gmail.com> wrote: > Ah, I didn't understand what you were saying before. > Your point is that these two should be synonymous, even if they're enclosed > in a multicast()...end(): > > 1) This... > .*pipeline("direct:A", "direct:B")* > > 2) and this... > .*pipeline().to("direct:C").to("direct:D").end()* > > but only the second one works the way you expect. > The first one sends the same input to both routes (i.e. does not send the > output of A to B) > > > It does seem odd, and a brief test confirms the behavior you describe ( > Sample code here > <https://github.com/DariusX/CamelSandbox/blob/master/ > CamelSandbox/src/main/java/com/zerses/camelsandbox/ > MulticastPipelinesTest.java)> > ) > > Looking at the Camel code - Line 1165 > <https://github.com/apache/camel/blob/master/camel-core/ > src/main/java/org/apache/camel/model/ProcessorDefinition.java> > , that version of pipeline(String...uri) is simply a synonym for > to(String...uri), which would explain how it is working. > > Someone else would need to speak to how it *ought* to work, but I agree it > does not seem intuitive as-is. > > > > > > > -- > View this message in context: http://camel.465427.n5.nabble. > com/Question-on-multicast-to-pipelines-tp5789396p5789593.html > Sent from the Camel - Users mailing list archive at Nabble.com. >
Re: Question on multicast to pipelines
Ah, I didn't understand what you were saying before. Your point is that these two should be synonymous, even if they're enclosed in a multicast()...end(): 1) This... .*pipeline("direct:A", "direct:B")* 2) and this... .*pipeline().to("direct:C").to("direct:D").end()* but only the second one works the way you expect. The first one sends the same input to both routes (i.e. does not send the output of A to B) It does seem odd, and a brief test confirms the behavior you describe ( Sample code here <https://github.com/DariusX/CamelSandbox/blob/master/CamelSandbox/src/main/java/com/zerses/camelsandbox/MulticastPipelinesTest.java)> ) Looking at the Camel code - Line 1165 <https://github.com/apache/camel/blob/master/camel-core/src/main/java/org/apache/camel/model/ProcessorDefinition.java> , that version of pipeline(String...uri) is simply a synonym for to(String...uri), which would explain how it is working. Someone else would need to speak to how it *ought* to work, but I agree it does not seem intuitive as-is. -- View this message in context: http://camel.465427.n5.nabble.com/Question-on-multicast-to-pipelines-tp5789396p5789593.html Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Question on multicast to pipelines
I don’t see how .pipeline().to(“A”).to(“B”).end() should not be equivalent to .pipeline(“A”, “B”), or to .to(“A”, “B”) which is - for what I understood from documentation - equivalent to the pipeline statement anyway. I am of course changing data in the A and C steps, please go look at the gist in my original email (and the output of the working example for that matter.) I understand the intuition that the multicast sends the same message downstream; what is counter-intuitive to me is that it reaches *into* the pipelines rather than just their heads. Please read my code and the logs I added in my last post and let me know if this is expected - albeit backwards - behavior or whether it’s a bug. Best, Edoardo > On 1 Nov 2016, at 14:59, Brad Johnson <brad.john...@mediadriver.com> wrote: > > That's what you should see unless you change the data in A or in C. A and > C should both receive START. It is a multicast. If you change the value in > A you'll see that change in B but that will not be shown in C since C is at > the root of the multicast. If you change the data in C you'll see it in D. > > Another way to think of the multicast, if this helps, is that the first > elements in the multicast are like a pub/sub or JMS topic where each of the > subscribers receive exactly the same message. > > On Tue, Nov 1, 2016 at 4:49 AM, Edoardo Causarano < > edoardo.causar...@gmail.com> wrote: > >> Hi all, >> >> these are the results I get, only the most explicit and verbose >> configuration returns the expected result. >> >> Working route: >> .pipeline().to("A").to("B").end() >> .pipeline().to("C").to("D").end() >> >> 10:41:12.644 [main] INFO route1 - after direct:start body=START >> 10:41:12.666 [main] DEBUG org.apache.camel.processor.SendProcessor - >>>> >> A Exchange[ID-Spitfire-local-50181-1477993271722-0-3] >> 10:41:12.667 [main] DEBUG com.esc.test.MulticastPipelinesTest - A got >> in=START >> 10:41:12.667 [main] DEBUG org.apache.camel.processor.SendProcessor - >>>> >> B Exchange[ID-Spitfire-local-50181-1477993271722-0-3] >> 10:41:12.667 [main] DEBUG com.esc.test.MulticastPipelinesTest - B got in=A >> 10:41:12.670 [main] DEBUG org.apache.camel.processor.SendProcessor - >>>> >> C Exchange[ID-Spitfire-local-50181-1477993271722-0-4] >> 10:41:12.671 [main] DEBUG com.esc.test.MulticastPipelinesTest - C got >> in=START >> 10:41:12.671 [main] DEBUG org.apache.camel.processor.SendProcessor - >>>> >> D Exchange[ID-Spitfire-local-50181-1477993271722-0-4] >> 10:41:12.671 [main] DEBUG com.esc.test.MulticastPipelinesTest - D got in=C >> >> >> Faulty routes: >> .pipeline("A", "B") >> .pipeline("C", "D”) >> >> or >> >> .to("A", "B") >> .to("C", "D") >> >> 10:43:46.383 [main] INFO route1 - after direct:start body=START >> 10:43:46.389 [main] DEBUG org.apache.camel.processor.SendProcessor - >>>> >> A Exchange[ID-Spitfire-local-50316-1477993425625-0-3] >> 10:43:46.389 [main] DEBUG com.esc.test.MulticastPipelinesTest - A got >> in=START >> 10:43:46.390 [main] DEBUG org.apache.camel.processor.SendProcessor - >>>> >> B Exchange[ID-Spitfire-local-50316-1477993425625-0-4] >> 10:43:46.390 [main] DEBUG com.esc.test.MulticastPipelinesTest - B got >> in=START >> 10:43:46.391 [main] DEBUG org.apache.camel.processor.SendProcessor - >>>> >> C Exchange[ID-Spitfire-local-50316-1477993425625-0-5] >> 10:43:46.391 [main] DEBUG com.esc.test.MulticastPipelinesTest - C got >> in=START >> 10:43:46.391 [main] DEBUG org.apache.camel.processor.SendProcessor - >>>> >> D Exchange[ID-Spitfire-local-50316-1477993425625-0-6] >> 10:43:46.391 [main] DEBUG com.esc.test.MulticastPipelinesTest - D got >> in=START >> >> >> Best, >> Edoardo >> >>> On 31 Oct 2016, at 15:04, DariusX <dariuscoo...@gmail.com> wrote: >>> >>> Your example was: >>> multicast() >>> .pipeline("A", "B") >>> .pipeline("C", "D") >>> .end() >>> >>> You send "START" as the body to this. So, you should expect "START" to be >>> the in.body for both "A" and "C". >>> >>> The in body for "B" will depend on what "A" does. Example: if "A" >> transforms >>> the body to a constant "Hello from A", then that is what "B" will get as >> its >>> in.body. >>> >>> Similarly, "D" will get whatever "C" decides to send along. >>> >>> If neither A nor C make any changes to the body, then you should expect >>> "START" to be the in.body for all four. >>> >>> >>> >>> >>> >>> -- >>> View this message in context: http://camel.465427.n5.nabble. >> com/Question-on-multicast-to-pipelines-tp5789396p5789518.html >>> Sent from the Camel - Users mailing list archive at Nabble.com. >> >>
Re: Question on multicast to pipelines
That's what you should see unless you change the data in A or in C. A and C should both receive START. It is a multicast. If you change the value in A you'll see that change in B but that will not be shown in C since C is at the root of the multicast. If you change the data in C you'll see it in D. Another way to think of the multicast, if this helps, is that the first elements in the multicast are like a pub/sub or JMS topic where each of the subscribers receive exactly the same message. On Tue, Nov 1, 2016 at 4:49 AM, Edoardo Causarano < edoardo.causar...@gmail.com> wrote: > Hi all, > > these are the results I get, only the most explicit and verbose > configuration returns the expected result. > > Working route: > .pipeline().to("A").to("B").end() > .pipeline().to("C").to("D").end() > > 10:41:12.644 [main] INFO route1 - after direct:start body=START > 10:41:12.666 [main] DEBUG org.apache.camel.processor.SendProcessor - >>>> > A Exchange[ID-Spitfire-local-50181-1477993271722-0-3] > 10:41:12.667 [main] DEBUG com.esc.test.MulticastPipelinesTest - A got > in=START > 10:41:12.667 [main] DEBUG org.apache.camel.processor.SendProcessor - >>>> > B Exchange[ID-Spitfire-local-50181-1477993271722-0-3] > 10:41:12.667 [main] DEBUG com.esc.test.MulticastPipelinesTest - B got in=A > 10:41:12.670 [main] DEBUG org.apache.camel.processor.SendProcessor - >>>> > C Exchange[ID-Spitfire-local-50181-1477993271722-0-4] > 10:41:12.671 [main] DEBUG com.esc.test.MulticastPipelinesTest - C got > in=START > 10:41:12.671 [main] DEBUG org.apache.camel.processor.SendProcessor - >>>> > D Exchange[ID-Spitfire-local-50181-1477993271722-0-4] > 10:41:12.671 [main] DEBUG com.esc.test.MulticastPipelinesTest - D got in=C > > > Faulty routes: > .pipeline("A", "B") > .pipeline("C", "D”) > > or > > .to("A", "B") > .to("C", "D") > > 10:43:46.383 [main] INFO route1 - after direct:start body=START > 10:43:46.389 [main] DEBUG org.apache.camel.processor.SendProcessor - >>>> > A Exchange[ID-Spitfire-local-50316-1477993425625-0-3] > 10:43:46.389 [main] DEBUG com.esc.test.MulticastPipelinesTest - A got > in=START > 10:43:46.390 [main] DEBUG org.apache.camel.processor.SendProcessor - >>>> > B Exchange[ID-Spitfire-local-50316-1477993425625-0-4] > 10:43:46.390 [main] DEBUG com.esc.test.MulticastPipelinesTest - B got > in=START > 10:43:46.391 [main] DEBUG org.apache.camel.processor.SendProcessor - >>>> > C Exchange[ID-Spitfire-local-50316-1477993425625-0-5] > 10:43:46.391 [main] DEBUG com.esc.test.MulticastPipelinesTest - C got > in=START > 10:43:46.391 [main] DEBUG org.apache.camel.processor.SendProcessor - >>>> > D Exchange[ID-Spitfire-local-50316-1477993425625-0-6] > 10:43:46.391 [main] DEBUG com.esc.test.MulticastPipelinesTest - D got > in=START > > > Best, > Edoardo > > > On 31 Oct 2016, at 15:04, DariusX <dariuscoo...@gmail.com> wrote: > > > > Your example was: > > multicast() > >.pipeline("A", "B") > >.pipeline("C", "D") > > .end() > > > > You send "START" as the body to this. So, you should expect "START" to be > > the in.body for both "A" and "C". > > > > The in body for "B" will depend on what "A" does. Example: if "A" > transforms > > the body to a constant "Hello from A", then that is what "B" will get as > its > > in.body. > > > > Similarly, "D" will get whatever "C" decides to send along. > > > > If neither A nor C make any changes to the body, then you should expect > > "START" to be the in.body for all four. > > > > > > > > > > > > -- > > View this message in context: http://camel.465427.n5.nabble. > com/Question-on-multicast-to-pipelines-tp5789396p5789518.html > > Sent from the Camel - Users mailing list archive at Nabble.com. > >
Re: Question on multicast to pipelines
Hi all, these are the results I get, only the most explicit and verbose configuration returns the expected result. Working route: .pipeline().to("A").to("B").end() .pipeline().to("C").to("D").end() 10:41:12.644 [main] INFO route1 - after direct:start body=START 10:41:12.666 [main] DEBUG org.apache.camel.processor.SendProcessor - >>>> A Exchange[ID-Spitfire-local-50181-1477993271722-0-3] 10:41:12.667 [main] DEBUG com.esc.test.MulticastPipelinesTest - A got in=START 10:41:12.667 [main] DEBUG org.apache.camel.processor.SendProcessor - >>>> B Exchange[ID-Spitfire-local-50181-1477993271722-0-3] 10:41:12.667 [main] DEBUG com.esc.test.MulticastPipelinesTest - B got in=A 10:41:12.670 [main] DEBUG org.apache.camel.processor.SendProcessor - >>>> C Exchange[ID-Spitfire-local-50181-1477993271722-0-4] 10:41:12.671 [main] DEBUG com.esc.test.MulticastPipelinesTest - C got in=START 10:41:12.671 [main] DEBUG org.apache.camel.processor.SendProcessor - >>>> D Exchange[ID-Spitfire-local-50181-1477993271722-0-4] 10:41:12.671 [main] DEBUG com.esc.test.MulticastPipelinesTest - D got in=C Faulty routes: .pipeline("A", "B") .pipeline("C", "D”) or .to("A", "B") .to("C", "D") 10:43:46.383 [main] INFO route1 - after direct:start body=START 10:43:46.389 [main] DEBUG org.apache.camel.processor.SendProcessor - >>>> A Exchange[ID-Spitfire-local-50316-1477993425625-0-3] 10:43:46.389 [main] DEBUG com.esc.test.MulticastPipelinesTest - A got in=START 10:43:46.390 [main] DEBUG org.apache.camel.processor.SendProcessor - >>>> B Exchange[ID-Spitfire-local-50316-1477993425625-0-4] 10:43:46.390 [main] DEBUG com.esc.test.MulticastPipelinesTest - B got in=START 10:43:46.391 [main] DEBUG org.apache.camel.processor.SendProcessor - >>>> C Exchange[ID-Spitfire-local-50316-1477993425625-0-5] 10:43:46.391 [main] DEBUG com.esc.test.MulticastPipelinesTest - C got in=START 10:43:46.391 [main] DEBUG org.apache.camel.processor.SendProcessor - >>>> D Exchange[ID-Spitfire-local-50316-1477993425625-0-6] 10:43:46.391 [main] DEBUG com.esc.test.MulticastPipelinesTest - D got in=START Best, Edoardo > On 31 Oct 2016, at 15:04, DariusX <dariuscoo...@gmail.com> wrote: > > Your example was: > multicast() >.pipeline("A", "B") >.pipeline("C", "D") > .end() > > You send "START" as the body to this. So, you should expect "START" to be > the in.body for both "A" and "C". > > The in body for "B" will depend on what "A" does. Example: if "A" transforms > the body to a constant "Hello from A", then that is what "B" will get as its > in.body. > > Similarly, "D" will get whatever "C" decides to send along. > > If neither A nor C make any changes to the body, then you should expect > "START" to be the in.body for all four. > > > > > > -- > View this message in context: > http://camel.465427.n5.nabble.com/Question-on-multicast-to-pipelines-tp5789396p5789518.html > Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Question on multicast to pipelines
Your example was: multicast() .pipeline("A", "B") .pipeline("C", "D") .end() You send "START" as the body to this. So, you should expect "START" to be the in.body for both "A" and "C". The in body for "B" will depend on what "A" does. Example: if "A" transforms the body to a constant "Hello from A", then that is what "B" will get as its in.body. Similarly, "D" will get whatever "C" decides to send along. If neither A nor C make any changes to the body, then you should expect "START" to be the in.body for all four. -- View this message in context: http://camel.465427.n5.nabble.com/Question-on-multicast-to-pipelines-tp5789396p5789518.html Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Question on multicast to pipelines
"...yet they all fail (in the sense that they all receive the original START incoming payload. Can anyone explain how this is expected behavior?" All or just A and C receive the same message. I'd expect A and C to receive the same message but not for B and D to (unless you don't change the message in A and C. The pipelines should all receive the same messages that is available between the multicast. Is that the behavior you are seeing? The results of the first pipeline are not propagated to the second pipeline. I'm not sure what the underlying mechanics are without cracking open the source code but it is as if the multicast is cloning the message it receives and sending the clone to each of the pipelines. "In any case I don't understand how the multicast could leak into a vararg pipeline(...). " How is it leaking? Each pipeline specified between the multicast tags is intended to receive exactly the same exchange as the others without mutation of the content based on the results of any of the others. Remember, the multicast is designed so that it can parallel process all the routes specified at the beginning of each pipeline. The only way it could do that is if it were sending the same message to all of them. Again, what exactly, are you trying to do with the processing. On Fri, Oct 28, 2016 at 4:40 PM, Edoardo Causarano < edoardo.causar...@gmail.com> wrote: > Hi Brad, > > yes that's the definition of multicast but the documentation also suggests > that pipelines are supposed to be implicitly derived from a vararg to(...) > statement. > > In any case I don't understand how the multicast could leak into a vararg > pipeline(...). It really feels like a bug to me, but I know little enough > of Camel to assume there's probably a good reason (in which case I'm happy > to understand it's logic) >
Re: Question on multicast to pipelines
Hi Brad, yes that's the definition of multicast but the documentation also suggests that pipelines are supposed to be implicitly derived from a vararg to(...) statement. In any case I don't understand how the multicast could leak into a vararg pipeline(...). It really feels like a bug to me, but I know little enough of Camel to assume there's probably a good reason (in which case I'm happy to understand it's logic)
Re: Question on multicast to pipelines
Hi Darius, that's correct, I'm expecting the START message to enter the pipelines from their heads. On Fri, 28 Oct 2016 at 20:43, DariusX <dariuscoo...@gmail.com> wrote: > It isn't clear what you want as the expected output. > From your example, it seems that you have two "pipelines" and you want to > multicast to send the message down both pipelines. > But, that's probably not what you really want. > > > > -- > View this message in context: > http://camel.465427.n5.nabble.com/Question-on-multicast-to-pipelines-tp5789396p5789416.html > Sent from the Camel - Users mailing list archive at Nabble.com. >
Re: Question on multicast to pipelines
The multicast sends the same message to all the to or pipeline elements it finds in its definition. That's why it is a "multicast". What is it you are trying to accomplish here and don't focus so much on the mechanics of doing it. Without knowing what your business case/problem definition is it's hard to say what will work for you. On Fri, Oct 28, 2016 at 1:42 PM, DariusX <dariuscoo...@gmail.com> wrote: > It isn't clear what you want as the expected output. > From your example, it seems that you have two "pipelines" and you want to > multicast to send the message down both pipelines. > But, that's probably not what you really want. > > > > -- > View this message in context: http://camel.465427.n5.nabble. > com/Question-on-multicast-to-pipelines-tp5789396p5789416.html > Sent from the Camel - Users mailing list archive at Nabble.com. >
Re: Question on multicast to pipelines
It isn't clear what you want as the expected output. >From your example, it seems that you have two "pipelines" and you want to multicast to send the message down both pipelines. But, that's probably not what you really want. -- View this message in context: http://camel.465427.n5.nabble.com/Question-on-multicast-to-pipelines-tp5789396p5789416.html Sent from the Camel - Users mailing list archive at Nabble.com.
Question on multicast to pipelines
Hi all, I had some trouble figuring out how to multicast to some pipelines. I eventually found a [working] definition, but I also expected other forms to work such as: .multicast().aggregationStrategy(AggregationStrategies.groupedExchange()) .pipeline("A", "B") .pipeline("C", "D") .end() or .multicast().aggregationStrategy(AggregationStrategies.groupedExchange()) .to("A", "B") .to("C", "D") .end() yet they all fail (in the sense that they all receive the original START incoming payload. Can anyone explain how this is expected behavior? Best, Edoardo [working]: https://gist.github.com/ecausarano/4b66294464741b9f626890b29ea0aec2