Patrick, Will and I have put together a couple of classes that do much (not all) of what you're describing for a Syslog TCP client and an Avro RPC client. I've been meaning to commit them to Flume SVN but I haven't gotten around to it yet. Here is the syslog one: https://github.com/mpercy/flume-load-gen/blob/master/src/main/java/org/apache/flume/tcphammer/Hammer.java
I can give you an Avro RPC one which is similar. Regards, Mike On Mon, Jul 16, 2012 at 9:28 PM, Patrick Wendell <[email protected]> wrote: > Hi All, > > I was trying to debug a memory leak and it doesn't seem like there is > a good way to generate workload for Flume at a configurable rate, or > with configurable characteristics like payload size and header size. > The sequence source is something like this, but you can't control the > rate of event generation at all. > > What do people do now when benchmarking or debugging Flume? I was > thinking of writing a source that generates event streams with > configurable properties. Would that be useful? Or is this need already > met somehow else? > > - Patrick >
