I am processing big file - line by line with camel. The average amount of
processed messages per second is 30k. When I do the same in java using
BufferedReader - the average amount of processed messages per second is
500k. I am processing the same file. This is significant performance lost.
Am I doing something wrong in Camel?
camel route :
from("file:C:/Test?fileName=test_file.txt&noop=true")
.split().tokenize("\n").streaming()
.to("log:INFO?groupSize=10000");
java code:
FileReader fr = new FileReader("C:/Test/test_file.txt");
BufferedReader br = new BufferedReader(fr);
long count = 0;
long start = System.currentTimeMillis();
while(br.readLine() != null)
{
count++;
if(count % 10000 == 0)
{
long now = System.currentTimeMillis();
long msgPerSecond = 1000*count/(now-start);
System.out.println(msgPerSecond);
}
}
br.close();
--
View this message in context:
http://camel.465427.n5.nabble.com/Message-Processing-Performance-while-splitting-tp5735824.html
Sent from the Camel - Users mailing list archive at Nabble.com.