Oh, so that github repo is stale?
On 5/10/12, Brock Noland <[email protected]> wrote: > Hi, > > That is FlumeOG (0.9.X). FlumeNG (1.X) as far as I am aware has no > limitation like that. > > Brock > > On Wed, May 9, 2012 at 11:22 PM, S Ahmed <[email protected]> wrote: >> Brock, >> >> Doesn't this mean it is limited to 32k? >> >> https://github.com/apache/flume/blob/branch-0.9.5/flume-core/src/main/java/com/cloudera/flume/core/EventImpl.java#L39 >> >> >> On Tue, May 8, 2012 at 11:01 AM, Brock Noland <[email protected]> wrote: >>> >>> Hi, >>> >>> On Tue, May 8, 2012 at 8:51 AM, S Ahmed <[email protected]> wrote: >>> > Could flume be used to store log data where payload sizes are 2-100K >>> > in >>> > size? (with the average being 25KB). >>> >>> Yes Flume could handle that size. >>> >>> > >>> > This data is collected from a website, so each request there will be a >>> > payload that needs to be stored (it is in XML format) and then every x >>> > seconds it should be persisted from memory to mysql/hdfs/disk. >>> > >>> > Is this an appropriate use case for flume? >>> >>> Sounds like it to me. >>> >>> > >>> > Can flume persist to things other than hdfs like mysql? >>> >>> You'd have to write a MySQLSink or JDBCSink but yes that should work. >>> >>> Brock >>> >>> -- >>> Apache MRUnit - Unit testing MapReduce - >>> http://incubator.apache.org/mrunit/ >> >> > > > > -- > Apache MRUnit - Unit testing MapReduce - > http://incubator.apache.org/mrunit/ >
