Correction: RE: #4, it is wrong to say "nothing" has ever happened. However, it is correct in saying we have never adopted load testing as part of our core development practices. L
On Jul 16, 2012, at 3:04 PM, Lance Speelmon <[email protected]> wrote: > I was not aware that we were conducting a technology selection process for > load testing tooling. I was working under the impression that we had already > adopted tsung for that purpose. > > Are there selection criteria available for driving this decision to > conclusion? In what timeframe? > > I will caution the community that if you are around long enough you will see > history trying to repeat itself. IMHO, we are dangerously close to repeating > a historical failure pattern: > > Sakai realizes it needs load testing tooling. > Some ill defined process kind of sort of happens and we choose jmeter. > Time passes. > Nothing happens. > Rinse and repeat starting at step 1. > > And while I do have a horse in the race (tsung framework), I am fine choosing > jmeter. What I NOT okay with is repeating this failure pattern which IIRC > has happened at least three times in Sakai's short history. L > > PS - Branden this is not targeted at you directly, but at the larger concern. > :) > > > On Jul 16, 2012, at 4:42 AM, Branden Visser <[email protected]> wrote: > >> Hi everyone, >> >> I've been putting some research into an appropriate load testing tool >> for us to use, I have been focusing mainly on JMeter and Tsung. >> >> Tsung, as far as I can tell, has the following selling points: >> >> 1. Its erlang guts let it drive many concurrent users more efficiently >> than its competitors >> 2. Its XML structure seems quite leaner, making it easier build tests >> from source >> 3. There is existing work from rSmart that can be leveraged to drive >> our performance tests [1] >> >> With JMeter, I see the following benefits: >> >> 1. Lower overhead in spinning up a test, once the tests are already >> built (VIA a maven plugin) >> 2. I think the increase in complexity of JMeter comes with the benefit >> of extensibility (unless you know erlang, I guess..) >> 3. There is an existing community effort that can be leveraged to >> drive our performance tests [2] >> >> They both have exactly 3 advantages, I don't know what to do!? >> >> Just kidding. But, unless there is quantifiable evidence that suggests >> JMeter's performance will not suffice to properly test OAE (I have not >> been able to find such evidence yet, but others may have more data), I >> propose that we move forward with JMeter. I see value in making the >> JMeter tests executable from the same command-line on which OAE is >> built. I think this moves towards making the JMeter tests an artifact >> of the release and not some orthogonal set of scripts uploaded >> elsewhere, which in my experience tend to become of questionable age >> and relevance. I think it will become more valuable to our deployers, >> and the deployers' performance test data (which would hopefully be >> more abundant with the lower barrier of entry) will become more >> valuable to the core team. >> >> [1] https://github.com/kcampos/Open-Performance-Automation-Framework >> [2] https://confluence.sakaiproject.org/display/QA/CLE+Load+Test+Framework >> >> -- >> Cheers, >> Branden >> _______________________________________________ >> oae-dev mailing list >> [email protected] >> http://collab.sakaiproject.org/mailman/listinfo/oae-dev >
_______________________________________________ oae-dev mailing list [email protected] http://collab.sakaiproject.org/mailman/listinfo/oae-dev
