I think that we can come up with an initial version with little efforts.
The simplest scenario I can think of is running a Flume instance (with a
SeqGen source and a Null sink) for one minute, and then report the average
events per second.
On Thu, Oct 13, 2016 at 6:43 PM, Attila Simon <s...@cloudera.com> wrote:
> Good idea! What would be required to set up something similar for Flume?
> ie initial time cost for setting up the infrastructure and periodic time
> cost to add new use-cases.
> On Thu, Oct 13, 2016 at 5:19 PM, Lior Zeno <liorz...@gmail.com> wrote:
> > Hi All,
> > Monitoring Flume's performance over time is an important step in every
> > production-level application. Benchmarking Flume on a nightly basis has
> > the following advantages:
> > * Better understanding of Flume's bottlenecks.
> > * Allow users to compare the performance of different solutions, such as
> > Logstash and Fluentd.
> > * Better understanding of the influence of recent commits on performance.
> > Logstash already conducts various performance tests, more details in this
> > link:
> > http://logstash-benchmarks.elastic.co/
> > I propose adding a few micro-benchmarks showing Flume's TPS vs date (of
> > course, in the ideal case where the input and/or output do not bottleneck
> > the system), e.g. using the SeqGen source.
> > Thoughts?
> > Thanks