Hi Vladimir, On Fri, Feb 6, 2015 at 1:38 PM, Vladimir Sitnikov < [email protected]> wrote:
> Hi, > > I believe JMeter support of tracking failed samples can be > significantly improved. > Each and every manual suggests using "Aggregate Report" ([1]) or > friends, however it silently returns wrong data without any warning. > > For instance: (sample, response time, status) > Sample1, 9 sec, OK > Sample1, 0 sec, ERR > Sample1, 0 sec, ERR > > JMeter would show "average response time" as 3 seconds, and same for > percentiles (median should be 9 seconds, not 0). > However, it is common for failed requests to run much faster. > It does not matter very much how fast you can crash, but it does > matter how fast the successful responses are. > > I see two problems here: > 1) Default configuration averages/computes quantiles for both OK and > ERR responses. So users get wrong values in the report. > Can you open a bugzilla, should be easy to fix > 2) There is no easy way to track OK and ERR separately. Well, one can > add _two_ copies of AggregateReport (one for success, another one for > fails), however is that ever suggested in the documentation? Is that a > good user experience? One would have to switch back and forth from one > to another. > > Any ideas/plans to fix that? > Have a look at that and give us your feedback. Your code contributions are also very welcome : - https://issues.apache.org/bugzilla/show_bug.cgi?id=57516 > Well, I do not suggest implementing full set of reporting yet, but it > is a pity to see how performance engineers report wrong results unless > you pinpoint all those JMeter pitfalls. > > [1]: > http://jmeter.apache.org/usermanual/component_reference.html#Aggregate_Report > > Not sure this can only be fixed by JMeter :-) > -- > Regards, > Vladimir Sitnikov > -- Cordialement. Philippe Mouawad.
