Hi, On Tue, May 8, 2012 at 8:51 AM, S Ahmed <[email protected]> wrote: > Could flume be used to store log data where payload sizes are 2-100K in > size? (with the average being 25KB).
Yes Flume could handle that size. > > This data is collected from a website, so each request there will be a > payload that needs to be stored (it is in XML format) and then every x > seconds it should be persisted from memory to mysql/hdfs/disk. > > Is this an appropriate use case for flume? Sounds like it to me. > > Can flume persist to things other than hdfs like mysql? You'd have to write a MySQLSink or JDBCSink but yes that should work. Brock -- Apache MRUnit - Unit testing MapReduce - http://incubator.apache.org/mrunit/
