I was not aware that we were conducting a technology selection process for load testing tooling. I was working under the impression that we had already adopted tsung for that purpose.
Are there selection criteria available for driving this decision to conclusion? In what timeframe? I will caution the community that if you are around long enough you will see history trying to repeat itself. IMHO, we are dangerously close to repeating a historical failure pattern: Sakai realizes it needs load testing tooling. Some ill defined process kind of sort of happens and we choose jmeter. Time passes. Nothing happens. Rinse and repeat starting at step 1. And while I do have a horse in the race (tsung framework), I am fine choosing jmeter. What I NOT okay with is repeating this failure pattern which IIRC has happened at least three times in Sakai's short history. L PS - Branden this is not targeted at you directly, but at the larger concern. :) On Jul 16, 2012, at 4:42 AM, Branden Visser <[email protected]> wrote: > Hi everyone, > > I've been putting some research into an appropriate load testing tool > for us to use, I have been focusing mainly on JMeter and Tsung. > > Tsung, as far as I can tell, has the following selling points: > > 1. Its erlang guts let it drive many concurrent users more efficiently > than its competitors > 2. Its XML structure seems quite leaner, making it easier build tests > from source > 3. There is existing work from rSmart that can be leveraged to drive > our performance tests [1] > > With JMeter, I see the following benefits: > > 1. Lower overhead in spinning up a test, once the tests are already > built (VIA a maven plugin) > 2. I think the increase in complexity of JMeter comes with the benefit > of extensibility (unless you know erlang, I guess..) > 3. There is an existing community effort that can be leveraged to > drive our performance tests [2] > > They both have exactly 3 advantages, I don't know what to do!? > > Just kidding. But, unless there is quantifiable evidence that suggests > JMeter's performance will not suffice to properly test OAE (I have not > been able to find such evidence yet, but others may have more data), I > propose that we move forward with JMeter. I see value in making the > JMeter tests executable from the same command-line on which OAE is > built. I think this moves towards making the JMeter tests an artifact > of the release and not some orthogonal set of scripts uploaded > elsewhere, which in my experience tend to become of questionable age > and relevance. I think it will become more valuable to our deployers, > and the deployers' performance test data (which would hopefully be > more abundant with the lower barrier of entry) will become more > valuable to the core team. > > [1] https://github.com/kcampos/Open-Performance-Automation-Framework > [2] https://confluence.sakaiproject.org/display/QA/CLE+Load+Test+Framework > > -- > Cheers, > Branden > _______________________________________________ > oae-dev mailing list > [email protected] > http://collab.sakaiproject.org/mailman/listinfo/oae-dev
_______________________________________________ oae-dev mailing list [email protected] http://collab.sakaiproject.org/mailman/listinfo/oae-dev
