About the performance of Storm, I have done a performance test. With write/read hbase and redis, my topology could still process about 3000 tuples per second in our StormCluster server.
Sent from myMail for iOS 2014年1月10日 星期五 23:54 +0800 from Michael Ritsema <[email protected]>: I deploy with LocalCluster mode. I get several thousands messages processed a second but expect my botteneck is not at the storm layer. It would be interesting to see a full explanation of the differences in LocalCluster and a real deployment. I expect quite a few people would benefit from having storm run in process in actual deployments. -Rits On Fri, Jan 10, 2014 at 9:46 AM, Michael Rose < [email protected] > wrote: >Post your code. Even Dev mode is far faster for us. >On Jan 10, 2014 8:44 AM, "Klausen Schaefersinho" < [email protected] > >wrote: >>> I've benched storm at 1.8 million tuples per second on a big (24 core) box >>>using local or shuffle grouping between a spout and bolt. >>Production or development mode? >> >>> you're only seeing 10 events per second make sure you don't have any sleeps >>> Yeah I checked for sleeps etc. and stripped down my code. Now my bolts do >>> nothing and the spout just creates random data... >> >> >> >> >>On Fri, Jan 10, 2014 at 4:37 PM, Nathan Leung < [email protected] > wrote: >>>I've benched storm at 1.8 million tuples per second on a big (24 core) box >>>using local or shuffle grouping between a spout and bolt. If you're only >>>seeing 10 events per second make sure you don't have any sleeps (whether in >>>your code or elsewhere e.g. a library or triggered due to lack of data in >>>the storm spout). Also note that the default sleep period in the spout when >>>there is no data (in storm 0.9) is I believe 1ms so even if you're hitting >>>this condition you should see much more than 10 events per second. >>>On Jan 10, 2014 10:19 AM, "Jon Logan" < [email protected] > wrote: >>>>Why would you benchmark local mode? It's intended for debugging and >>>>development purposes, not actual production use............... >>>> >>>> >>>>per the website, Storm itself is benchmarked at > 1 million tuples per >>>>second per node. >>>> >>>> >>>>On Fri, Jan 10, 2014 at 8:40 AM, Klausen Schaefersinho < >>>>[email protected] > wrote: >>>>>Hi, >>>>> >>>>>how is the performance of a real cluster compared to a local development >>>>>cluster? I was trying to benchmark Storm now and found that it only >>>>>manages to consume 10 events per second in a simple topology which only >>>>>consists out of i spout and one bolt where the spout creates random >>>>>objects and the bolt acknowledge the tuple. >>>>> >>>>>10 events terrible slow, and I am not sure if this is related to the >>>>>develop mode. >>>>> >>>>>Cheers, >>>>> >>>>>Klaus >>
