Thank you Dominik. I have two more points, please.1- What do you mean "able to
control message size"? Is it max-pending-spout parameter?2- Is there any
published benchmark like this old-one here:
https://github.com/stormprocessor/storm-benchmark/commit/22bd17a81020ceef71ed73168ac89d3f8eaf61e2
Best Regards,Walid
From: Dominik Safaric <[email protected]>
To: Walid Aljoby <[email protected]>
Cc: "[email protected]" <[email protected]>; "[email protected]"
<[email protected]>
Sent: Friday, November 4, 2016 4:53 PM
Subject: Re: Storm benchmarks
Well, this depends onto the aspects of the measurements.
You may for example define a topology consisting of a spout, transformation
bolt and sink that receives byte arrays from Kafka, transforms them and
outputs. The nice thing is that you’d be able to control for the size of the
messages.
In addition, if you care about the performance in conjunction to stateful
operations such as aggregations, your topology might look alike the for example
WordCount topology.
Regards,Dominik
On 4 Nov 2016, at 09:50, Walid Aljoby <[email protected]> wrote:
Hi Dominik,
Many thanks for details. Actually I am looking for a set typologies for my test.
Thank you again,--RegardsWalid
From: Dominik Safaric <[email protected]>
To: [email protected]; Walid Aljoby <[email protected]>
Cc: "[email protected]" <[email protected]>
Sent: Friday, November 4, 2016 4:41 PM
Subject: Re: Storm benchmarks
Hi Walid,
You may benchmark Storm’s performance in terms of throughput and end-to-end
latency for example. In addition, the investigation could also include
variances in the configurational settings, such as the parallelism, message
size, intra-worker and inter-worker buffer size which some of them have a
profound effect onto the performance of Storm.
There are already a few benchmarks of Storm’s performance such as:
https://developer.ibm.com/streamsdev/wp-content/uploads/sites/15/2014/04/Streams-and-Storm-April-2014-Final.pdf
In addition, you may want to take a look at the academic paper Storm@Twitter
and Twitter Heron: Stream processing at scale which describe among others
certain performance aspects of Storm that might be helpful to you when
designing the benchmark.
Regards,Dominik
On 4 Nov 2016, at 09:36, Walid Aljoby <[email protected]> wrote:
Hi Everyone,
Anyone please could tell what are the common benchmarks for testing Storm
performance?
Thank you,--Regards
WA
