Brandon, How would you compare the quality of the reports and the ability to customize the reports?
I was very disappointed with how hard it was to understand the reports from Tsung. While the data was there I really struggled to make sense of it. Poorly designed, no prioritization, poor layout, etc. And while the charting is OK for a single test run (barely), I really wanted a visual way immediately compare two or more different runs so I could immediately see how performance was trending. The Tsung community needs an information designer and a visual designer to spend a couple of weeks working together on the product. I know it's a geek's tool but I've come to expect better. Perhaps I've been spoiled by the gorgeous reports out of google analytics. Sent via iPhone Please excuse any spelling errors or odd auto-corrections. On Jul 16, 2012, at 7:42 AM, Branden Visser <[email protected]> wrote: > Hi everyone, > > I've been putting some research into an appropriate load testing tool > for us to use, I have been focusing mainly on JMeter and Tsung. > > Tsung, as far as I can tell, has the following selling points: > > 1. Its erlang guts let it drive many concurrent users more efficiently > than its competitors > 2. Its XML structure seems quite leaner, making it easier build tests > from source > 3. There is existing work from rSmart that can be leveraged to drive > our performance tests [1] > > With JMeter, I see the following benefits: > > 1. Lower overhead in spinning up a test, once the tests are already > built (VIA a maven plugin) > 2. I think the increase in complexity of JMeter comes with the benefit > of extensibility (unless you know erlang, I guess..) > 3. There is an existing community effort that can be leveraged to > drive our performance tests [2] > > They both have exactly 3 advantages, I don't know what to do!? > > Just kidding. But, unless there is quantifiable evidence that suggests > JMeter's performance will not suffice to properly test OAE (I have not > been able to find such evidence yet, but others may have more data), I > propose that we move forward with JMeter. I see value in making the > JMeter tests executable from the same command-line on which OAE is > built. I think this moves towards making the JMeter tests an artifact > of the release and not some orthogonal set of scripts uploaded > elsewhere, which in my experience tend to become of questionable age > and relevance. I think it will become more valuable to our deployers, > and the deployers' performance test data (which would hopefully be > more abundant with the lower barrier of entry) will become more > valuable to the core team. > > [1] https://github.com/kcampos/Open-Performance-Automation-Framework > [2] https://confluence.sakaiproject.org/display/QA/CLE+Load+Test+Framework > > -- > Cheers, > Branden > _______________________________________________ > oae-dev mailing list > [email protected] > http://collab.sakaiproject.org/mailman/listinfo/oae-dev _______________________________________________ oae-dev mailing list [email protected] http://collab.sakaiproject.org/mailman/listinfo/oae-dev
