Hi
A workaround I can think off right now
// read the big file and split it into lines and send each line to the seda
queue (async).
From("file://mybigfiles").process(new Processor() {
public void process(Exchange e) {
// need to get hold of a producer template to easily send the lines to the
sede queue
final ProducerTemplate template = e.getContext().createProducerTemplate();
// get the file from the body
File file = e.getIn().getBody(File.class);
// create the scanner that split lines without reading all into memory
// mind you can also pass in a encoding if you like to the scanner
Scanner scanner = new Scanner(file);
// use our token as delimiter
scanner.setDelimiter("\n\r");
// loop and send each line to the seda queue
while (scanner.hasNext()){
String line = scanner.next();
template.sendBody("seda:myfileline", line);
}
});
// then do what you normally would do next
from("seda:myfileline").to("xxxx");
Mind the code is pseudo code written in email editor.
Med venlig hilsen
Claus Ibsen
......................................
Silverbullet
Skovsgårdsvænget 21
8362 Hørning
Tlf. +45 2962 7576
Web: www.silverbullet.dk
-----Original Message-----
From: Claus Ibsen [mailto:[EMAIL PROTECTED]
Sent: 4. september 2008 06:10
To: camel-user@activemq.apache.org
Subject: RE: Splitter for big files
Hi
A workaround or solution if you might say is also to do the splitting you self.
You can do this in a processor or a POJO and use bean binding.
Camel is very flexible and if there is something you are missing then you can
always code it yourself and be integrated easily with camel. It never takes the
power of coding from you.
Med venlig hilsen
Claus Ibsen
......................................
Silverbullet
Skovsgårdsvænget 21
8362 Hørning
Tlf. +45 2962 7576
Web: www.silverbullet.dk
-----Original Message-----
From: Bart Frackiewicz [mailto:[EMAIL PROTECTED]
Sent: 3. september 2008 23:16
To: camel-user@activemq.apache.org
Subject: Re: Splitter for big files
Hi Claus,
Claus Ibsen schrieb:
>
> Could you create a JIRA ticket for this improvement?
Thank you for creating the JIRA tickets. Do CAMEL-876 cover the XPATH
issue? The batch functionality was new for me.
> Btw how big is the files you use?
We process old datasets up to 1gb, the XML files are about 250mb.
Can i create a workaround for this? A colleague of mine found a good
implementation in Servicemix, which splits the lines and send the new
exchanges direct on the message bus. I am not so experienced with Camel
to measure this. Maybe you can give me a hint how you would solve it.
Bart
