(I see I'm having trouble getting my point across about response data showing up in the results.jtl XML file, so I made a more comprehensive test case and included a fix.)
I think there is a bug in JMeter where sometimes response data is written to the results.jtl XML file when run in non-GUI mode. This causes problems because it is not correclty encoded (the UTF-8 fix Jordei implemented solves part of it, but still non-XML data shows up) and it does not seem to serve a useful purpose.
Let me start at the beginning. Because I did not want to attach files to a message to a mailing list, please download the file <http://www.klomp.org/~vinny/jmeter_response_data_problem.zip> and unzip it.
Run the first test plan as such:
> jmeter -n -t response_data_problem_1.jmx
-l response_data_problem_1.jtl
-Jjmeter.save.saveservice.assertion_results=all
and have a look at response_data_problem_1.jtl (or just look at the one I included). Because both assertions on the Jakarta logo (/images/jakarta-logo.gif) succeeded, the results file reports the successes of the assertions. To get those results the "-Jjmeter.save.saveservice.assertion_results=all" bit is necessary.
Now run the second test plan:
> jmeter -n -t response_data_problem_2.jmx
-l response_data_problem_2.jtl
-Jjmeter.save.saveservice.assertion_results=all
and have a look at the corresponding results file. You can see all kinds of binary gibberish appearing here. This is the failure message for the second (response) assertion, even though it was the _first_ (size) assertion that failed. It failed because I changed "=8584" to "!=8584".
Now contrast that with the the third test plan:
> jmeter -n -t response_data_problem_3.jmx
-l response_data_problem_3.jtl
-Jjmeter.save.saveservice.assertion_results=all
If you look at response_data_problem_3.jtl, you can see that the response assertion succeeds here, but the size assertion fails.
This behaviour does not make sense to me. It is caused by these lines in <http://cvs.apache.org/viewcvs.cgi/jakarta-jmeter/src/components/org/apache/jmeter/assertions/ResponseAssertion.java?view=markup>:
public AssertionResult getResult(SampleResult response)
{
AssertionResult result;
if (!response.isSuccessful())
{
result = new AssertionResult();
result.setError(true);
byte [] ba = response.getResponseData();
result.setFailureMessage(
ba == null ? "Unknown Error (responseData is empty)" : new String(ba)
);
return result;
}
result = evaluateResponse(response);
return result;
}
For some reason a check is first made to see whether the sample was succesful before the assertion is actually checked. If the sample was not succesfull (either because there was an error executing the request or because an earlier assertion failed), the failure message is set to the response data and a failure is reported.
Now I have two questions:
1) Why is this check made?
2) If this check is necessary, why is the failure message not set to something like "Cannot assert response when sample was not succesfull"?
Oh yeah, I promised a fix. My fix would be to remove the whole check and have the method be similar like the one in <http://cvs.apache.org/viewcvs.cgi/jakarta-jmeter/src/components/org/apache/jmeter/assertions/SizeAssertion.java?view=markup>:
public AssertionResult getResult(SampleResult response)
{
if (response.getResponseData() == null)
{
AssertionResult result = new AssertionResult();
result.setError(false);
result.setFailure(true);
result.setFailureMessage("Response was null");
return result;
}
return evaluateResponse(response);
}Well, sorry the very long message but there seems to be no shorter way to explain this and it is an annoying problem for me.
Regards, Vincent.
--------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
