Hi, everyone. I'm a researcher in the distributed system field. Now i'm going to find the bottlenecks of Storm and Spark Streaming. I'd like to know the popular benchmarks to test the thoughout and latency of Storm or Spark Streaming. If someone has experiences or professional knowledge, i'm happy for your advice.
Another question is how to have a stress testing in Storm. I create a topology that has a spout (emit sentence) and a bolt that discards the messages recevied. And the number of spout tasks is 28, of bolts is 28 too. Of course, the physical resource of servers can satisfy the scenario. In the running, i observerd the number of emitting sentences is just 500 per minute per task. I'm so amazed because it can be up to 243,783 messages in using netty for transferring messages. I didn't do any advanced configuration. I have no idea how to let the storm fully using the server resource and then the bottleneck can be found. Thank you! Andrew Lee
