I didn't think you were appending the data. But realize that when you call
direct or direct-vm it is a request/response style of route. So you are
calling route 1 which calls route 2 which calls route 1 and are not
returning as far as I know. So you are essentially making the equivalent
of a recursive method call. You should end up with a stack overflow that
way eventually not because of the size of the individual messages but due
to the number of calls from 1 to 2 and 2 to 1 and 1 to 2 and so on.
Essentially you keep filling up the stack with each new call.
There is a chapter on the POJO approach in the Camel in Action book with
the same sort of recommendation. All of chapter 4 is about it and one
section, for example, is called "using beans the easy way" and another
covers annotations. Hopefully in the new book (V2) they'll emphasize it a
bit more. Most of the time you don't need to specify the annotations if you
are passing a bean in the body that Camel can identify as the type and
method to invoke. Otherwise you can annotate the method call and type with
@Body and if you need to pass in a header you use @Header. If you want to
specify the method to call I think it's @Handler (can't be sure as I don't
use it most of the time).
On Fri, Oct 14, 2016 at 4:07 PM, Goyal, Arpit <arpit.go...@sap.com> wrote:
> Hi Brad,
> TOPIC 1:
> "StackOverflowError" - this is what happened with the current Route 2
> calling Route 1 for LARGE # OF PAGES. It was not about the data, because
> the body is not getting appended and is always limited to one page (at
> least in most cases).
> JVM/stack-size exploded around about 200 calls to the loop :) But yeah a
> good learning of what not to do :) Moved the Loop outside the route
> completely and based on Route 2 response, triggered the Route 1 again.
> Route 1 to Route 2 linkage was done only for modularization purpose.
> TOPIC 2:
> Very interesting point you have raised here about using POJO and not
> Processors. I actually wanted to know is there some explanation of what you
> said in CAMEL IN ACTION book. Just got hand of hard-copy and see if I can
> understand that better.
> I did use split and bean binding, but not sure if that is the right POJO
> approach? I guess I should use @ExchangeProperty
> - public List<Message> splitMessage(@Body Object body, Exchange
> Current use cases which processor is doing:
> - set / get exchange headers (multiple)
> - set / get camel exchange property (so far single)
> - saw annotation ExchangeProperty, but it has String type return
> value. Whereas on exchange we are setting Objects.
> -----Original Message-----
> From: Brad Johnson [mailto:brad.john...@mediadriver.com]
> Sent: Friday, October 14, 2016 8:54 AM
> To: email@example.com
> Subject: Re: Looping in routes using Direct Component - Performance impact?
> Have you looked at splitter/tokenizer/streaming for this instead of having
> route 1 call route 2 which calls route 1? Interesting route setup as I've
> never used routes in a recursive fashion like that. When using direct the
> return doesn't happen until the end of the last route in the chain. In
> this case it appears that you are continually delaying the return from the
> route via this iterative approach. I'd have to check that via testing to
> see what's really happening in that case but I've never done it like that
> and would assume it would eventually run out of memory for large amounts of
> data. As far as I know you can't just erase the history though you could
> set the body to null or empty string. But I wouldn't do that.
> Just unmarshal your data and run it through the splitter in a streaming
> fashion. If you don't need to process these in serial order you could then
> drop them on a SEDA queue with multiple consumer threads. You can then also
> set the queue size to limit how many documents are coming through at a
> Also, unless you have a compelling reason don't use Processors as they
> couple you to the framework. Just use a POJO and let Camel use reflection
> to select the correct method for processing or use annotations. As a
> general rule of thumb I use, if you don't know exactly why you need to use
> a Processor you don't need to use it.
> On Thu, Oct 13, 2016 at 6:23 PM, Goyal, Arpit <arpit.go...@sap.com> wrote:
> > I can't compromise my business logic and there is hardly any data; but
> > each time camel Route 2 starts Route 1, the previous route processing
> > onto stack. I only have limited stack after that any operation can fail.
> > To me below approach seems risky...I need to go async or remove the
> > looping out of the route overall.
> > Regards,
> > Arpit.
> > -----Original Message-----
> > From: souciance [mailto:souciance.eqdam.ras...@gmail.com]
> > Sent: Thursday, October 13, 2016 3:45 PM
> > To: firstname.lastname@example.org
> > Subject: Re: Looping in routes using Direct Component - Performance
> > I think you should first run your route setup with a few pages, 10s of
> > pages, hundreds of pages and then thousands of pages and then do
> > performance tests to see where what is eating memory and optimise from
> > there.
> > On Fri, Oct 14, 2016 at 12:31 AM, Goyal, Arpit [via Camel] <
> > ml-node+s465427n5788744...@n5.nabble.com> wrote:
> > > Hi,
> > >
> > > We currently are on 2.16.3 version and can't use the loop function
> > > of 2.17). So we have the following routes based on which we create sort
> > of
> > > loop with switch case. Now we are worried about the performance (when
> > > see stacktrace of camel processing)
> > >
> > > Route 1: from("direct:s1).to("read one page of data").process("process
> > > one page").to("direct:s2");
> > >
> > > Route 2: from("direct:s2").toD("go to external source").when("are more
> > > pages left").to("direct:s1").otherwise().process("end processing");
> > >
> > > Now if there are thousands of pages, it is possible that this loop
> > creates
> > > lot of objects in memory & can create performance issue? Is there a way
> > for
> > > Camel to be told, before going to route 1 again, forget all the past?
> > > disabling HISTORY would solve that?
> > >
> > > Regards,
> > > Arpit.
> > >
> > >
> > > ------------------------------
> > > If you reply to this email, your message will be added to the
> > > below:
> > > http://camel.465427.n5.nabble.com/Looping-in-routes-using-
> > > Direct-Component-Performance-impact-tp5788744.html
> > > To start a new topic under Camel - Users, email
> > > ml-node+s465427n465428...@n5.nabble.com
> > > To unsubscribe from Camel - Users, click here
> > > <http://camel.465427.n5.nabble.com/template/NamlServlet.jtp?macro=
> > unsubscribe_by_code&node=465428&code=c291Y2lhbmNlLmVxZGFtLnJhc2h0aU
> > BnbWFpbC5jb218NDY1NDI4fDE1MzI5MTE2NTY=>
> > > .
> > > NAML
> > > <http://camel.465427.n5.nabble.com/template/
> > viewer&id=instant_html%21nabble%3Aemail.naml&base=
> > BasicNamespace-nabble.view.web.template.NabbleNamespace-
> > nabble.view.web.template.NodeNamespace&breadcrumbs=
> > notify_subscribers%21nabble%3Aemail.naml-instant_emails%
> > 21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
> > >
> > --
> > View this message in context: http://camel.465427.n5.nabble.
> > com/Looping-in-routes-using-Direct-Component-Performance-
> > impact-tp5788744p5788745.html
> > Sent from the Camel - Users mailing list archive at Nabble.com.