It could more easy for user to group the lines if camel-stream component
can expose the strategy processLine to the client.
I just fill a JIRA[1] for it
You may need to extends the StreamConsumer yourself to implement such of
feature at this time.
[1]https://issues.apache.org/jira/browse/CAMEL-5017
On Mon Feb 20 08:59:09 2012, Anurag Sharma wrote:
Hi All,
I have a file with million plus records. Each record is of CSV format. Now
instead of reading the whole file in memory I would like to stream it record
by record , or prefereably bunch N nbr of records in a single message
exchange.
Following is my route which works OK for streaming it line by line.
from("file:src/data?fileName=webservices_20090723_001_trunc.log&noop=true").split(body().tokenize("\n")).streaming()
.to("seda:input?concurrentConsumers=1")
.log("Processing ${id}");
from("seda:input?concurrentConsumers=1")
.convertBodyTo(String.class)
.unmarshal(csv)
.to("bean:LogService?method=doHandleCsvData");
Now if i want to bunch a group of lines together, it appears that there is
no group option while streaming from file component. So I decided to try the
streaming component as follows:
from("stream:file?fileName=src/data/webservices_20090723_001_trunc.log&groupLines=2")
Now this does group two lines together however it removes the new line
seperator. Consequently two records are concatenated in a single list entry
when the message arrives at doHandleCsvData.
I suppose I can write my own producer within the File component that takes
the file handle and streams data out. However I am keen on exploring the
capabilities of the existring components.
Would appreciate any help.
Thanks& Regards,
Anurag
--
View this message in context:
http://camel.465427.n5.nabble.com/Grouping-lines-while-streaming-tp5497878p5497878.html
Sent from the Camel - Users mailing list archive at Nabble.com.
--
Willem
----------------------------------
FuseSource
Web: http://www.fusesource.com
Blog: http://willemjiang.blogspot.com (English)
http://jnn.javaeye.com (Chinese)
Twitter: willemjiang
Weibo: willemjiang
