Hi

I am using spooldir source, memory channel, hdfs sink to collect log files
and store into HDFS.

When I run the flume agent, it is creating very very small files with size
766 bytes.

Input file: test.log [11.4 KB]
Output files: sales_web_log.1443871052640.log, etc.[all are very very small
files with size 766 bytes]

*How to increase the output file size?*

*Thanks & Regards,*

*Shiva Ram*
*Website: http://datamaking.com <http://datamaking.com>Facebook Page:
www.facebook.com/datamaking <http://www.facebook.com/datamaking>*
2015-10-02 18:06:26 DEBUG HelloWorldDemo:20 - This is debug : mkyong||412121
2015-10-02 18:06:26 INFO  HelloWorldDemo:24 - This is info : mkyong||412121
2015-10-02 18:06:26 WARN  HelloWorldDemo:27 - This is warn : mkyong||412121
2015-10-02 18:06:26 ERROR HelloWorldDemo:28 - This is error : mkyong||412121
2015-10-02 18:06:26 FATAL HelloWorldDemo:29 - This is fatal : mkyong||412121
2015-10-02 18:06:26 DEBUG HelloWorldDemo:20 - This is debug : mkyong||412122
2015-10-02 18:06:26 INFO  HelloWorldDemo:24 - This is info : mkyong||412122
2015-10-02 18:06:26 WARN  HelloWorldDemo:27 - This is warn : mkyong||412122
2015-10-02 18:06:26 ERROR HelloWorldDemo:28 - This is error : mkyong||412122
2015-10-02 18:06:26 FATAL HelloWorldDemo:29 - This is fatal : mkyong||412122
2015-10-02 18:06:26 DEBUG HelloWorldDemo:20 - This is debug : mkyong||412123
2015-10-02 18:06:26 INFO  HelloWorldDemo:24 - This is info : mkyong||412123
2015-10-02 18:06:26 WARN  HelloWorldDemo:27 - This is warn : mkyong||412123
2015-10-02 18:06:26 ERROR HelloWorldDemo:28 - This is error : mkyong||412123
2015-10-02 18:06:26 FATAL HelloWorldDemo:29 - This is fatal : mkyong||412123
2015-10-02 18:06:26 DEBUG HelloWorldDemo:20 - This is debug : mkyong||412124
2015-10-02 18:06:26 INFO  HelloWorldDemo:24 - This is info : mkyong||412124
2015-10-02 18:06:26 WARN  HelloWorldDemo:27 - This is warn : mkyong||412124
2015-10-02 18:06:26 ERROR HelloWorldDemo:28 - This is error : mkyong||412124
2015-10-02 18:06:26 FATAL HelloWorldDemo:29 - This is fatal : mkyong||412124
2015-10-02 18:06:26 DEBUG HelloWorldDemo:20 - This is debug : mkyong||412125
2015-10-02 18:06:26 INFO  HelloWorldDemo:24 - This is info : mkyong||412125
2015-10-02 18:06:26 WARN  HelloWorldDemo:27 - This is warn : mkyong||412125
2015-10-02 18:06:26 ERROR HelloWorldDemo:28 - This is error : mkyong||412125
2015-10-02 18:06:26 FATAL HelloWorldDemo:29 - This is fatal : mkyong||412125
2015-10-02 18:06:26 DEBUG HelloWorldDemo:20 - This is debug : mkyong||412126
2015-10-02 18:06:26 INFO  HelloWorldDemo:24 - This is info : mkyong||412126
2015-10-02 18:06:26 WARN  HelloWorldDemo:27 - This is warn : mkyong||412126
2015-10-02 18:06:26 ERROR HelloWorldDemo:28 - This is error : mkyong||412126
2015-10-02 18:06:26 FATAL HelloWorldDemo:29 - This is fatal : mkyong||412126
2015-10-02 18:06:26 DEBUG HelloWorldDemo:20 - This is debug : mkyong||412127
2015-10-02 18:06:26 INFO  HelloWorldDemo:24 - This is info : mkyong||412127
2015-10-02 18:06:26 WARN  HelloWorldDemo:27 - This is warn : mkyong||412127
2015-10-02 18:06:26 ERROR HelloWorldDemo:28 - This is error : mkyong||412127
2015-10-02 18:06:26 FATAL HelloWorldDemo:29 - This is fatal : mkyong||412127
2015-10-02 18:06:26 DEBUG HelloWorldDemo:20 - This is debug : mkyong||412128
2015-10-02 18:06:26 INFO  HelloWorldDemo:24 - This is info : mkyong||412128
2015-10-02 18:06:26 WARN  HelloWorldDemo:27 - This is warn : mkyong||412128
2015-10-02 18:06:26 ERROR HelloWorldDemo:28 - This is error : mkyong||412128
2015-10-02 18:06:26 FATAL HelloWorldDemo:29 - This is fatal : mkyong||412128
2015-10-02 18:06:26 DEBUG HelloWorldDemo:20 - This is debug : mkyong||412129
2015-10-02 18:06:26 INFO  HelloWorldDemo:24 - This is info : mkyong||412129
2015-10-02 18:06:26 WARN  HelloWorldDemo:27 - This is warn : mkyong||412129
2015-10-02 18:06:26 ERROR HelloWorldDemo:28 - This is error : mkyong||412129
2015-10-02 18:06:26 FATAL HelloWorldDemo:29 - This is fatal : mkyong||412129
2015-10-02 18:06:26 DEBUG HelloWorldDemo:20 - This is debug : mkyong||412130
2015-10-02 18:06:26 INFO  HelloWorldDemo:24 - This is info : mkyong||412130
2015-10-02 18:06:26 WARN  HelloWorldDemo:27 - This is warn : mkyong||412130
2015-10-02 18:06:26 ERROR HelloWorldDemo:28 - This is error : mkyong||412130
2015-10-02 18:06:26 FATAL HelloWorldDemo:29 - This is fatal : mkyong||412130
2015-10-02 18:06:26 DEBUG HelloWorldDemo:20 - This is debug : mkyong||412131
2015-10-02 18:06:26 INFO  HelloWorldDemo:24 - This is info : mkyong||412131
2015-10-02 18:06:26 WARN  HelloWorldDemo:27 - This is warn : mkyong||412131
2015-10-02 18:06:26 ERROR HelloWorldDemo:28 - This is error : mkyong||412131
2015-10-02 18:06:26 FATAL HelloWorldDemo:29 - This is fatal : mkyong||412131
2015-10-02 18:06:26 DEBUG HelloWorldDemo:20 - This is debug : mkyong||412132
2015-10-02 18:06:26 INFO  HelloWorldDemo:24 - This is info : mkyong||412132
2015-10-02 18:06:26 WARN  HelloWorldDemo:27 - This is warn : mkyong||412132
2015-10-02 18:06:26 ERROR HelloWorldDemo:28 - This is error : mkyong||412132
2015-10-02 18:06:26 FATAL HelloWorldDemo:29 - This is fatal : mkyong||412132
2015-10-02 18:06:26 DEBUG HelloWorldDemo:20 - This is debug : mkyong||412133
2015-10-02 18:06:26 INFO  HelloWorldDemo:24 - This is info : mkyong||412133
2015-10-02 18:06:26 WARN  HelloWorldDemo:27 - This is warn : mkyong||412133
2015-10-02 18:06:26 ERROR HelloWorldDemo:28 - This is error : mkyong||412133
2015-10-02 18:06:26 FATAL HelloWorldDemo:29 - This is fatal : mkyong||412133
2015-10-02 18:06:26 DEBUG HelloWorldDemo:20 - This is debug : mkyong||412134
2015-10-02 18:06:26 INFO  HelloWorldDemo:24 - This is info : mkyong||412134
2015-10-02 18:06:26 WARN  HelloWorldDemo:27 - This is warn : mkyong||412134
2015-10-02 18:06:26 ERROR HelloWorldDemo:28 - This is error : mkyong||412134
2015-10-02 18:06:26 FATAL HelloWorldDemo:29 - This is fatal : mkyong||412134
2015-10-02 18:06:26 DEBUG HelloWorldDemo:20 - This is debug : mkyong||412135
2015-10-02 18:06:26 INFO  HelloWorldDemo:24 - This is info : mkyong||412135
2015-10-02 18:06:26 WARN  HelloWorldDemo:27 - This is warn : mkyong||412135
2015-10-02 18:06:26 ERROR HelloWorldDemo:28 - This is error : mkyong||412135
2015-10-02 18:06:26 FATAL HelloWorldDemo:29 - This is fatal : mkyong||412135
2015-10-02 18:06:26 DEBUG HelloWorldDemo:20 - This is debug : mkyong||412136
2015-10-02 18:06:26 INFO  HelloWorldDemo:24 - This is info : mkyong||412136
2015-10-02 18:06:26 WARN  HelloWorldDemo:27 - This is warn : mkyong||412136
2015-10-02 18:06:26 ERROR HelloWorldDemo:28 - This is error : mkyong||412136
2015-10-02 18:06:26 FATAL HelloWorldDemo:29 - This is fatal : mkyong||412136
2015-10-02 18:06:26 DEBUG HelloWorldDemo:20 - This is debug : mkyong||412137
2015-10-02 18:06:26 INFO  HelloWorldDemo:24 - This is info : mkyong||412137
2015-10-02 18:06:26 WARN  HelloWorldDemo:27 - This is warn : mkyong||412137
2015-10-02 18:06:26 ERROR HelloWorldDemo:28 - This is error : mkyong||412137
2015-10-02 18:06:26 FATAL HelloWorldDemo:29 - This is fatal : mkyong||412137
2015-10-02 18:06:26 DEBUG HelloWorldDemo:20 - This is debug : mkyong||412138
2015-10-02 18:06:26 INFO  HelloWorldDemo:24 - This is info : mkyong||412138
2015-10-02 18:06:26 WARN  HelloWorldDemo:27 - This is warn : mkyong||412138
2015-10-02 18:06:26 ERROR HelloWorldDemo:28 - This is error : mkyong||412138
2015-10-02 18:06:26 FATAL HelloWorldDemo:29 - This is fatal : mkyong||412138
2015-10-02 18:06:26 DEBUG HelloWorldDemo:20 - This is debug : mkyong||412139
2015-10-02 18:06:26 INFO  HelloWorldDemo:24 - This is info : mkyong||412139
2015-10-02 18:06:26 WARN  HelloWorldDemo:27 - This is warn : mkyong||412139
2015-10-02 18:06:26 ERROR HelloWorldDemo:28 - This is error : mkyong||412139
2015-10-02 18:06:26 FATAL HelloWorldDemo:29 - This is fatal : mkyong||412139
2015-10-02 18:06:26 DEBUG HelloWorldDemo:20 - This is debug : mkyong||412140
2015-10-02 18:06:26 INFO  HelloWorldDemo:24 - This is info : mkyong||412140
2015-10-02 18:06:26 WARN  HelloWorldDemo:27 - This is warn : mkyong||412140
2015-10-02 18:06:26 ERROR HelloWorldDemo:28 - This is error : mkyong||412140
2015-10-02 18:06:26 FATAL HelloWorldDemo:29 - This is fatal : mkyong||412140
2015-10-02 18:06:26 DEBUG HelloWorldDemo:20 - This is debug : mkyong||412141
2015-10-02 18:06:26 INFO  HelloWorldDemo:24 - This is info : mkyong||412141
2015-10-02 18:06:26 WARN  HelloWorldDemo:27 - This is warn : mkyong||412141
2015-10-02 18:06:26 ERROR HelloWorldDemo:28 - This is error : mkyong||412141
2015-10-02 18:06:26 FATAL HelloWorldDemo:29 - This is fatal : mkyong||412141
2015-10-02 18:06:26 DEBUG HelloWorldDemo:20 - This is debug : mkyong||412142
2015-10-02 18:06:26 INFO  HelloWorldDemo:24 - This is info : mkyong||412142
2015-10-02 18:06:26 WARN  HelloWorldDemo:27 - This is warn : mkyong||412142
2015-10-02 18:06:26 ERROR HelloWorldDemo:28 - This is error : mkyong||412142
2015-10-02 18:06:26 FATAL HelloWorldDemo:29 - This is fatal : mkyong||412142
2015-10-02 18:06:26 DEBUG HelloWorldDemo:20 - This is debug : mkyong||412143
2015-10-02 18:06:26 INFO  HelloWorldDemo:24 - This is info : mkyong||412143
2015-10-02 18:06:26 WARN  HelloWorldDemo:27 - This is warn : mkyong||412143
2015-10-02 18:06:26 ERROR HelloWorldDemo:28 - This is error : mkyong||412143
2015-10-02 18:06:26 FATAL HelloWorldDemo:29 - This is fatal : mkyong||412143
2015-10-02 18:06:26 DEBUG HelloWorldDemo:20 - This is debug : mkyong||412144
2015-10-02 18:06:26 INFO  HelloWorldDemo:24 - This is info : mkyong||412144
2015-10-02 18:06:26 WARN  HelloWorldDemo:27 - This is warn : mkyong||412144
2015-10-02 18:06:26 ERROR HelloWorldDemo:28 - This is error : mkyong||412144
2015-10-02 18:06:26 FATAL HelloWorldDemo:29 - This is fatal : mkyong||412144
2015-10-02 18:06:26 DEBUG HelloWorldDemo:20 - This is debug : mkyong||412145
2015-10-02 18:06:26 INFO  HelloWorldDemo:24 - This is info : mkyong||412145
2015-10-02 18:06:26 WARN  HelloWorldDemo:27 - This is warn : mkyong||412145
2015-10-02 18:06:26 ERROR HelloWorldDemo:28 - This is error : mkyong||412145
2015-10-02 18:06:26 FATAL HelloWorldDemo:29 - This is fatal : mkyong||412145
2015-10-02 18:06:26 DEBUG HelloWorldDemo:20 - This is debug : mkyong||412146
2015-10-02 18:06:26 INFO  HelloWorldDemo:24 - This is info : mkyong||412146
2015-10-02 18:06:26 WARN  HelloWorldDemo:27 - This is warn : mkyong||412146
2015-10-02 18:06:26 ERROR HelloWorldDemo:28 - This is error : mkyong||412146
2015-10-02 18:06:26 FATAL HelloWorldDemo:29 - This is fatal : mkyong||412146
2015-10-02 18:06:26 DEBUG HelloWorldDemo:20 - This is debug : mkyong||412147
2015-10-02 18:06:26 INFO  HelloWorldDemo:24 - This is info : mkyong||412147
2015-10-02 18:06:26 WARN  HelloWorldDemo:27 - This is warn : mkyong||412147
2015-10-02 18:06:26 ERROR HelloWorldDemo:28 - This is error : mkyong||412147
2015-10-02 18:06:26 FATAL HelloWorldDemo:29 - This is fatal : mkyong||412147
2015-10-02 18:06:26 DEBUG HelloWorldDemo:20 - This is debug : mkyong||412148
2015-10-02 18:06:26 INFO  HelloWorldDemo:24 - This is info : mkyong||412148
2015-10-02 18:06:26 WARN  HelloWorldDemo:27 - This is warn : mkyong||412148
2015-10-02 18:06:26 ERROR HelloWorldDemo:28 - This is error : mkyong||412148
2015-10-02 18:06:26 FATAL HelloWorldDemo:29 - This is fatal : mkyong||412148
2015-10-02 18:06:26 DEBUG HelloWorldDemo:20 - This is debug : mkyong||412149
2015-10-02 18:06:26 INFO  HelloWorldDemo:24 - This is info : mkyong||412149
2015-10-02 18:06:26 WARN  HelloWorldDemo:27 - This is warn : mkyong||412149
2015-10-02 18:06:26 ERROR HelloWorldDemo:28 - This is error : mkyong||412149
2015-10-02 18:06:26 FATAL HelloWorldDemo:29 - This is fatal : mkyong||412149
2015-10-02 18:06:26 DEBUG HelloWorldDemo:20 - This is debug : mkyong||412150
2015-10-02 18:06:26 INFO  HelloWorldDemo:24 - This is info : mkyong||412150
2015-10-02 18:06:26 WARN  HelloWorldDemo:27 - This is warn : mkyong||412150
2015-10-02 18:06:26 ERROR HelloWorldDemo:28 - This is error : mkyong||412150

2015-10-02 18:06:26 DEBUG HelloWorldDemo:20 - This is debug : mkyong||412121
2015-10-02 18:06:26 INFO  HelloWorldDemo:24 - This is info : mkyong||412121
2015-10-02 18:06:26 WARN  HelloWorldDemo:27 - This is warn : mkyong||412121
2015-10-02 18:06:26 ERROR HelloWorldDemo:28 - This is error : mkyong||412121
2015-10-02 18:06:26 FATAL HelloWorldDemo:29 - This is fatal : mkyong||412121
2015-10-02 18:06:26 DEBUG HelloWorldDemo:20 - This is debug : mkyong||412122
2015-10-02 18:06:26 INFO  HelloWorldDemo:24 - This is info : mkyong||412122
2015-10-02 18:06:26 WARN  HelloWorldDemo:27 - This is warn : mkyong||412122
2015-10-02 18:06:26 ERROR HelloWorldDemo:28 - This is error : mkyong||412122
2015-10-02 18:06:26 FATAL HelloWorldDemo:29 - This is fatal : mkyong||412122

Reply via email to