As far as I know, Directory Spooling Source will send the file line by line as an event, and File Roll Sink will receive these lines and roll up to a big file by a fixed interval. Is it right, and can we config it to send the whole file as an event?
On Tue, Jan 22, 2013 at 1:22 PM, Nitin Pawar <[email protected]>wrote: > why don't you use directory spooling ? > > > On Tue, Jan 22, 2013 at 7:15 AM, Henry Ma <[email protected]> wrote: > >> Hi, >> >> When using Flume to collect log files, we want to just COPY the original >> files from several servers to a central storage (unix file system), not to >> roll up to a big file. Because we must record some messages of the original >> file such as name, host, path, timestamp, etc. Besides, we want to >> guarantee total reliability: no file miss, no file reduplicated. >> >> It seems that, in Source, we must put a whole file (size may be between >> 100KB and 100MB) into a Flume event; and in Sink, we must write each event >> to a single file. >> >> Is it practicable? Thanks! >> >> -- >> Best Regards, >> Henry Ma >> > > > > -- > Nitin Pawar > -- Best Regards, Henry Ma
