The HDFS Sink syncs at the end of each batch or when the file rolls. On Wed, Jan 16, 2013 at 10:55 AM, Nitin Pawar <[email protected]> wrote: > you can configure it as you nee > number of events > rollover by time > and other ways as well > > > On Thu, Jan 17, 2013 at 12:17 AM, Mohit Anchlia <[email protected]> > wrote: >> >> Right. I was asking about sync to "sink". My sink is hdfs so does flume >> sync to hdfs on every write operation? >> >> >> On Wed, Jan 16, 2013 at 10:26 AM, Brock Noland <[email protected]> wrote: >>> >>> Memory Channel does not write to disk and as such never syncs to disk. >>> File Channel does sync to disk for each batch put on or taken off the >>> channel. >>> >>> On Wed, Jan 16, 2013 at 10:21 AM, Mohit Anchlia <[email protected]> >>> wrote: >>> > Thanks! What I am really trying to understand is when does flume sync >>> > to the >>> > sink. I am not using batch events. >>> > >>> > >>> > On Wed, Jan 16, 2013 at 9:55 AM, Hari Shreedharan >>> > <[email protected]> wrote: >>> >> >>> >> It means that the channel can store that many events. If it is full, >>> >> then >>> >> the put() calls (on the source side) will start throwing >>> >> ChannelException. >>> >> The put call will block only for keep-alive number of seconds, after >>> >> which >>> >> it will throw. >>> >> >>> >> >>> >> Hari >>> >> >>> >> -- >>> >> Hari Shreedharan >>> >> >>> >> On Wednesday, January 16, 2013 at 9:46 AM, Mohit Anchlia wrote: >>> >> >>> >> Could someone help me understand capacity attribute of memoryChannel? >>> >> Does >>> >> it mean that memoryChannel flushes to sink only when this capacity is >>> >> reached or does it mean that it's the max events stored in memory and >>> >> call >>> >> blocks until everything else gets freed? >>> >> >>> >> >>> >> http://flume.apache.org/FlumeUserGuide.html#memory-channel >>> >> >>> >> >>> >> >>> > >>> >>> >>> >>> -- >>> Apache MRUnit - Unit testing MapReduce - >>> http://incubator.apache.org/mrunit/ >> >> > > > > -- > Nitin Pawar
-- Apache MRUnit - Unit testing MapReduce - http://incubator.apache.org/mrunit/
