Re: How to make aggregator wait until all threads are completed

2016-07-16 Thread mailingl...@j-b-s.de
That's the funny thing about splitting and parallel: The last message with the split end marker is usually never the last message processed. We tried resequencing (via split index) without success, because you have to keep all messages in memory to guarantee sequencing works properly, otherwise

Re: Camel Large File Processing Issues

2016-04-17 Thread mailingl...@j-b-s.de
Hi, the bodyToString converts it, try body.tokenize() and add a marshall() call before splitt. Von meinem iPhone gesendet > Am 17.04.2016 um 23:02 schrieb h0mer : > > Hey Guys, > > So I'm trying to implement the following Route: > > FTP --> Get Large log file (3G) --> Copy to Local Working

Re: Beanio stream...

2016-04-04 Thread mailingl...@j-b-s.de
Hi Brad! To read and split a file you need a combination of streaming unmarshalling and splitter components. Is tmxPaymentTechIn something you wrote? You can try to read the file by camel as xml, splitting groups by xpath and unmarshall just the group from xml to a bean model... Unfortunately

Re: Best Strategy to process a large number of rows in File

2016-03-30 Thread mailingl...@j-b-s.de
Hi Michele! Your mem settings look odd to mee. > -server -Xms256M -Xss512M -Xmx512M -XX:+UnlockDiagnosticVMOptions > -XX:+UnsyncloadClass -XX:PermSize=512M -XX:MaxPermSize=1024M Your Perm Space is greater than your heap? Sure? I doubt you need 1G set your heap to 1G and Perm to 256m as a fi

Re: Request to multiple routes

2016-03-29 Thread mailingl...@j-b-s.de
Hi! What do mean by request/response, HTTP? So your routes listen on a particular URL and you have 8 different URL in total, one per route? Jens Von meinem iPhone gesendet > Am 30.03.2016 um 01:45 schrieb Raghuveer : > > I have around 8 routes to call one after another. Can I set request to f

Re: Custom fluent API in Java DSL

2015-12-08 Thread mailingl...@j-b-s.de
Hi! Can you give an example what type of method you want to add to the RouteBuilder? To my understanding if you need a custom DSL to lets say "reuse camel specific DSL sequences" its easier to write your own DSL completely hiding camel and rely on RouteBuilder internally as usual. But maybe I

Re: Camel ThreadPool maxQueueSize question

2015-11-21 Thread mailingl...@j-b-s.de
ook into > it. Note my previous post that in my dev environment the reason it stopped > was out of memory error...i doubt that is the same case in production as > the rest of the routes do not stop. > > -Dave > > On Sat, Nov 21, 2015 at 1:36 AM, mailingl...@j-b-s.de > wrote

Re: Camel ThreadPool maxQueueSize question

2015-11-21 Thread mailingl...@j-b-s.de
Well, an OoMemEx says "allocating new memory failed". This has afaik no effect to ather running process. Ok chances are high they will fail too in subsequent processing steps if memory is not avail in general. What is the root cause? Are you reading a hugh file into memory? Are you spltting the

Re: Camel ThreadPool maxQueueSize question

2015-11-21 Thread mailingl...@j-b-s.de
Hi! when your sftp read threads stopps the files are still in process? In our env we had something similar in conjunction with splitting large files because the initial message is pending until all processing is completed. We solved it using a seda queue (limited in size) in betweeen our sfpt

Re: splitter().tokenize(...) does not recognize new lines in file uploaded using Spring MVC MultipartFile

2015-10-28 Thread mailingl...@j-b-s.de
Hi! Can you check both files in a Hex Editor/Viewer? Text Editors usually understand all kind of line breaks but eventually the splitter does not recognize it (you are using the line sep from your hosting system eg unix and your file is created under windows)? Jens Von meinem iPhone gesendet

Re: action at the end of the split

2015-08-11 Thread mailingl...@j-b-s.de
Hi! Try the following: in your split route just set a new header field with the value of "splitsize". Ok, this assignment is made more than once redundantly per each splitted element, but header properties you create on the original message are retained (hopefully :-) Jens Von meinem iPhone g

Re: Aggregator - discard new messages

2015-08-05 Thread mailingl...@j-b-s.de
We had the same problem and aggrgator (same applies to sequencer) will not work due to a potential timeout splitting the result as you described it allready. We solved it by a) starting the route b) do all processing / aggregation in combination with the inflight repository to detect the end of

Re: knowing wenn splitted exchange is completed in distributed env

2015-07-06 Thread mailingl...@j-b-s.de
Hi! We had a similar problem when using "parallel processing" on a single machine and searched for a solution without using aggregator or sequencer. At the very end we sent the results processed in parallel back to a common "ack" queue (maybe you strip the body before) which runs in the same pr

Re: Event driven producer?

2015-04-09 Thread mailingl...@j-b-s.de
Hi Christopher! Why not using an aggregator? Or you simply emit your messages 1:1 and set the message body to 'null' if there is nothing to send from your business logic perspective. You can simply use a filter afterward to drop all messages having a 'null' body to get your expected behaviour.

Re: ServiceStatus isProcessing?

2015-01-26 Thread mailingl...@j-b-s.de
Hi Kranti, from our experience relying solely on the inflight repo won't work, because even it's empty you may have pending messages in other components eg aggregator or sequencer or queues. Afaik there is no central camel service existing to check if all work is entirely finished so it depends

Re: sftp - Exception during concurent execution of sftp endpoint

2014-12-24 Thread mailingl...@j-b-s.de
Hi! What about processing in a serial fashion by sending both messages to a seda queue first and from here to your sftp endpoint? Just to avoid any concurrency issues and I am aware this not exactly an answer for your question but maybe a workaround. Jens Von meinem iPhone gesendet > Am 24.1

Re: aggregation and completionPredicate

2014-12-07 Thread mailingl...@j-b-s.de
Hi Claus! Sorry but I do not understand. My correlation expression is simply returning the value of "key" (see class "CorrelationExpression" below of my initial mail). Debugging I see the first list contains two elements as expected. As soon the correlation key changes camel correctly starts wi

Re: Fastest Data Holder

2014-04-30 Thread mailingl...@j-b-s.de
What do you mean by different systems? Different Camel Processors? You can store your data as body/property and this is a pure instance pointer. As long the message is not deep cloned/serialized as bytestream there is no overhead passing this message through the camel routes (in one vm). Can yo

Re: Java DSL / GroovyExpression / Binding

2013-08-18 Thread mailingl...@j-b-s.de
iteye.com (http://jnn.javaeye.com/) (Chinese) > Twitter: willemjiang > Weibo: 姜宁willem > > > > > > On Monday, August 19, 2013 at 12:47 PM, mailingl...@j-b-s.de wrote: > >> Hi all! >> >> Is there any chance to extend the binding of a GroovyExpression?

Java DSL / GroovyExpression / Binding

2013-08-18 Thread mailingl...@j-b-s.de
Hi all! Is there any chance to extend the binding of a GroovyExpression? Currently (in camel 2.11.1) the Groovy binding is defined on runtime based on the exchange only and I did not find an extension point. My current solutions is to use a GroovyExpression2 class which is a copy of the original