Hi. Check out http://camel.apache.org/splitter.html, especially the example "Split aggregate request/reply sample" This example seems to show how, after parallel processing, a single message can be sent.
I am sure you read the documentation before, so I don't know if that helps. Cheers, Max -----Ursprüngliche Nachricht----- Von: Willem Jiang [mailto:[email protected]] Gesendet: Mittwoch, 9. Juni 2010 05:36 An: [email protected] Betreff: Re: Split an exchange, process pieces in parallel and then wait for all to complete Hi, Camel splitter supports to process the message parallely. You can use parallelProcessing() dsl to define it from("file://trigger").process(getListOfFilesInDirectoryProcessor()).split(body()).parallelProcessing() ... Willem jonathanq wrote: > Hello, > > I am trying to get my head around how to construct a specific route in camel > 2.3. I am creating a process that listens on a "trigger" directory for a > new file which starts the process. > > Once it receives the "go" message. It will then look at a directory and > start processing all of the files in that directory. It is going to split > those files into smaller files and compress them. > > Once it is finished processing all of the files - we need to send a single > output message saying its done. > > Here is the general idea: > > from("file://trigger").process(getListOfFilesInDirectoryProcessor()).split(body()).threads(10).process(getFileSplitProcessor()).log("processing > of all input files complete") > > Obviously in the above example - we get a log message per input file, which > we don't want. > > Another idea was to use SEDA queues and an aggregator. This works - but it > relies on knowing how many input files there were for the batchSize() in the > aggregator. It also introduces the batchtimeout as well. It is not ideal - > so I am wondering if there is a better way. > > from("file://trigger") > .process(getListFilesInDirectoryProcessor()) > .split(body()) > .to("seda:atomicFiles"); > > from("seda:atomicFiles") > .threads(10) > .process(getFileSplitProcessor()) > .log("File thread completed") > .to("seda:aggregate"); > > from("seda:aggregate") > .aggregate(header("extractId")) > .batchSize(2) // this needs to be the same as the number of input files > .batchTimeout(10000000) > .log("Processing of input files complete."); > > I have tried to get my head around the new async package in Camel 2.x - but > I am having trouble understanding how to get it to do a request/reply. > > Is there an easier way to do what I want? The Async documentation goes into > detail on how to use callbacks - none of which seem to be in DSL routes. > The threads() function seems to be the DSL version...but it doesn't describe > how to do the request/reply. The example route just splits to multiple > threads and its done. > > What I am doing seems to be a common problem - get 1 input message...split > the work into smaller chunks to be processed simultaneously and then when > all pieces are done - continue on. So this EIP: > http://camel.apache.org/composed-message-processor.html > > The difference here is that I want the same processing to occur..just > multi-threaded. Whereas that EIP is more for different processing on each > piece. > > Thanks! > > Jonathan
