Hi if you suspects it's file io what about testing reading and splitting the file only wo further processing. Should tell you if the time is taken reading and splitting the file or in XML processing at stage 1 and stage 2.
If it's io where is the file located. Is it a local file, network share … There is a similar sample at http://www.catify.com/2012/07/09/parsing-large-files-with-apache-camel/. You might be able to compare the numbers you get at your system with the numbers provided in the article. Maruan Sahyoun Am 22.02.2013 um 06:21 schrieb cristisor <cristisor...@yahoo.com>: > Many thanks for your reply. > > When I read from the file I read simple lines, not XML. It takes more than > one hour to read, process and insert into the db 20.000 lines so I took out > the service unit that does the db operations and I was left with 26 minutes > for reading from the file line by line, converting to a stage 1 XML and then > to a stage 2 XML. The machine is an i7 with 8 GB of RAM, so I don't think > that it's normal to take so long. This is why I suspected that the I/O > operations, when reading the file, are slowing me down very much. > > Do you think that it will take me 100 times more to read 100 lines instead > of just one? The performance hit might not be at this point? > > Is sending the exchange to the queue and reading it inside another service > unit slowing me down? > > I'm not using Stax Stream, or at least I don't think that I'm using, to deal > the XML but I will take a look on this problem also. > > > > -- > View this message in context: > http://camel.465427.n5.nabble.com/Large-file-processing-with-Apache-Camel-tp5727977p5727988.html > Sent from the Camel - Users mailing list archive at Nabble.com.
