Hi The file component forces the write, to ensure its persistent on disk. See IOHelper.force method
So when you do so many writes, then that ought to impact the performance. We could though allow to configure the behavior of force, so people can turn it off. On Fri, Nov 9, 2012 at 4:01 PM, Gonzalo Vasquez <[email protected]> wrote: > I'm running a route that basically adds a character per line to a plain text > file, but it's taking to long, and it seems that it's due to some kind of > buffering issue when reading/writing from disk. > > I'm processing a 5MB file (attached as DC_FACCL132_0000 > MORA_1075_16-10-2012_19-09-47_15.txt.zip), with the corresponding XSL > template (also attached). > > It's taking for ever to process such a file, I understand I'm tokenizing on > line breaks, which could be the source of the problem as there are many > lines in the file (48198 exactly), but when running jvisualvm (see attached > images/snapshot)I can see the writing op is invoked 20386 times, which seem > not related to the line count. Is there an output buffer size that I can > configure? Or something like that? > > This is the route: > <camel:route id="pager" autoStartup="true"> > <camel:from > uri="file:///tmp/in?charset=Windows-1252&move=${file:parent}/../paged/${file:name.noext}.paged.ack&preMove=${file:name.noext}-${date:now:yyyyMMddHHmmssSSS}.${file:ext}" > /> > <camel:split streaming="true" parallelProcessing="false"> > <camel:tokenize token="\n" /> > <camel:to uri="bean:pager" /> > <camel:to > uri="file:///tmp/paged?charset=utf8&fileName=${file:name.noext}.paged&fileExist=Append" > /> > </camel:split> > </camel:route> > > This is the referenced bean: > > <bean id="pager" class="cl.altiuz.reports.etl.TextProcessor"> > <property name="xsltPath" > value="/Users/gonzalovasquez/Documents/workspace/altiuz-reports/reports-etl/xsl/pager.xsl" > /> > <property name="param" value="C.*PAG.* 1" /> > </bean> > > Camel versión is 2,10.1, and happens both on OSX & MS Windows, so I think > isn't a platform dependent problem, but a configuration one. > > Any ideas? Any thing else that I should send? > > Thanks! > > Gonzalo Vásquez Sáez > Gerente Investigación y Desarrollo (R&D) > Altiuz Soluciones Tecnológicas de Negocios Ltda. > Av. Nueva Tajamar 555 Of. 802, Las Condes > (56-2) 335 2461 > [email protected] > http://www.altiuz.cl > > > > > > > > -- Claus Ibsen ----------------- Red Hat, Inc. FuseSource is now part of Red Hat Email: [email protected] Web: http://fusesource.com Twitter: davsclaus Blog: http://davsclaus.com Author of Camel in Action: http://www.manning.com/ibsen
