Hi I wonder if using groupLines 1000 etc to work on a bulk of lines at a time, wont be fast enough with the fileExists=Append mode.
On Thu, May 1, 2014 at 5:17 PM, Ephemeris Lappis <[email protected]> wrote: > Hello. > > We have to produce some rather big volumes of data and generate output > files, in several steps, some of these steps using splitting to process > inputs. I've been looking for two ways... > > The first way is writing a file, setting an "append" mode, that could be a > nice solution to write batches of lines, but I suppose that the file if > closed and reopened for each exchange, and thus may be a quite bad solution > when writing millions of line one by one from a splitting loop. > > The second way is to use a stream, using "stream:file", but I've not found > any way to control actually how the file is closed. FYI, we use Camel with > Service Mix with an emdedded 2.10, and the "closeOnDone" option is not > available. The "autoCloseCount" seemed to be the beginning of a solution, > but as the value can't be set dynamically, using a property for example, > this doesn't give a full control on the file closing, and outputs might stay > opened, and make fail the following tasks. > > Any idea to write big files and control how to close them ? > > Thanks in advance. > > Regards. > > > > -- > View this message in context: > http://camel.465427.n5.nabble.com/Writing-big-files-stream-or-file-tp5750742.html > Sent from the Camel - Users mailing list archive at Nabble.com. -- Claus Ibsen ----------------- Red Hat, Inc. Email: [email protected] Twitter: davsclaus Blog: http://davsclaus.com Author of Camel in Action: http://www.manning.com/ibsen hawtio: http://hawt.io/ fabric8: http://fabric8.io/
