I think I mistook the mechanism how renaming is triggered. If I configure the rollInterval or rollSize, when it reach the time interval or reach the rollsize, it triggers the renaming. But if I only configure the rollSize and the file size never reach that large, rename will never be triggered. Before I think when I finished writing, it should trigger closing thus call the rename. Now I add rollInterval, problem resolved.
On Mon, Jul 8, 2013 at 11:09 AM, shangan chen <[email protected]>wrote: > hi, Lenin, > Thanks for your quick reply. I didn't find any error log, in collector, I > can only find the create file log as following, and no other logs... > > 08 Jul 2013 11:04:20,425 INFO [hdfs-sink_hdfs_0-call-runner-9] > (org.apache.flume.sink.hdfs.BucketWriter.doOpen:215) - Creating > /user/sankuai/flume_data/mobile_nginx/dt=20130708/hour=11/log_collector_srv03.1373252660403.lzo.tmp > > > On Sat, Jul 6, 2013 at 1:54 AM, Lenin Raj <[email protected]> wrote: > >> Hi Chen, >> >> Were you able to find any errors in the logs? >> >> -- >> Lenin >> On Jul 5, 2013 4:59 PM, "shangan chen" <[email protected]> wrote: >> >>> I use hdfs sink with lzo codec, when finished injecting data to flume >>> agent, I found ***.lzo.tmp file in hdfs. It seems the hdfs outputstream >>> failed to call close, so rename is thus not called. >>> Anyone can explain why ? thanks a lot ! >>> >>> -- >>> have a good day! >>> chenshang'an >>> >>> > > > -- > have a good day! > chenshang'an > > -- have a good day! chenshang'an
