Processing large files in Camel

[gramanero]

I am attempting to understand how the Camel File/File2 component works as I
have a need to use this type of capability in an upcoming project.

Goal: Write a Camel spring-based route that will detect files in a folder
structure, wait until the entire file has been written, and place an entry
onto an ActiveMQ queue that indicates the file name, path, and length, and
leave the file in place (i.e. do not let Camel move it to a .camel folder).

Camel Route:
    <route id="newFileReceived" startupOrder="1">
      <from
uri="file:E:\NewDataReceived?recursive=true&amp;maxMessagesPerPoll=500&amp;noop=true&amp;delay=60000"/>
      <to uri="activemq:queue:data.received"/>
    </route>

Problem: For small files this route works as expected. For large files, I am
seeing a .camelLock file appear every N seconds (which corresponds to the
configured 'delay'setting) and then disappear. I am assuming that this is
Camel trying to get a read lock on the file, but I cannot find any docs on
this type of behavior. I am also guessing that it keeps appearing and
disappearing due to the file size?? The size of the files that I am dealing
with are anywhere from 7Gb and up. This may not be a realistic use case for
what we will be dealing with, but I want to verify that writing files will
only ever put an entry onto the queue once the file have been completely
written and not before.

Camel version: Whatever is baked into apache-servicemix-4.4.1-fuse-03-06

Any suggestions/help is greatly appreciated.

Thank you.





--
View this message in context: 
http://camel.465427.n5.nabble.com/Processing-large-files-in-Camel-tp5719376.html
Sent from the Camel - Users mailing list archive at Nabble.com.