My personal position would be to move towards allowing functional testing, but still keep performance testing a priority. That is: never make a change that improves functional testing ability if it degrades performance testing ability. The opposite could happen, although I guess we would always go great lengths to try to make both requirements compatible.This makes a lot of sense.
The rationale for this is:
- JMeter is a performance testing tool -- or at least that's its charter.
- Performance testing requires functional testing features: you need to see that function is not degraded under load.
The functional testing in JMeter seems OK even now, but performance and functional testing are very different.
A performance test plan may have as few as 10 or less requests repeated in loops and across threads.
The tester doesn't mind working a bit harder to set up each request to keep the client-side load light.
(eg using ugly __regexFunction syntax to send a hidden parameter from last response. -
${__regexFunction(<input type='hidden' name='someID' value='(.*)'>,$1$,1,,,)} Note also how brittle this is (case-sensitive, expects single-quotes and exact whitespace,and doesn't recognize comments or other false positives)
)
A Functional test plan for a web app may easily have 1000's or more requests testing every possible regression failure point.
Here the client-side speed is a small factor.
The human effort required for creation/Maintenance and the brittleness of the tests are much more important.
(in example above, you would expect a good HTTP Functional test app to get hidden parameters with some much simpler function -
eg getHiddenParameter("someID")
- you would also expect this to have parsed the HTML so it can eg ignore comments or other unwanted text that simple req exp would match.)
Also the level of technical expertise for the tester writing the plan can be very different.
Having said that, the pluggable architecture of JMeter may allow for "Clever but Slow" Components to be made that are useful for Functional Testing,
but should not be installed in performance tests?
--------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
