Without the actual data it is impossible to say, if the reports are
wrong, or if they follow different ways to calculate a percentile.

The problem here is, that the two reports are using different algorithms
to calculate the percentiles. We are working with discrete numbers and
here a percentile is more like a range than a single (correct or wrong)
number. Say, you have the values [1, 2, 4, 8, 16, 32, 64, 128, 256, 512]
want to calculate a 90% percentile. For this you take any number that
splits the (sorted) list into two lists, where one list has all
elements, that are smaller (or equal) to the number and one list
contains all elements that are bigger than the number. For this example,
any number between 256 and 511.99 would be a valid 90% percentile. (that
is of course a broad description only)

The Aggregate report currently uses a memory intensive way and stores a
list with all values and picks the smallest number, that fits the above
description. (This memory hogging behaviour should be fixed and would
lead to less accurate data and less OOMs or slow JMeter instances)

The HTML report gives you a value that would probably be a good 90%
percentile, if you would have more data like the one, you already have
by picking some value between the lower bound and the upper bound of the
described range (in our example something between 256 and 511.999).

As your report has very few samples (21 for the biggest difference in
the numbers), the skew between the reported percentiles can look big,
but are not necessarily wrong.

Note, that if the numbers that both reports tell you are far apart, that
is probably a sign of having either too few samples, or the samples
(durations of the samples) are distributed sparsely near theĀ  percentile
ranges.

We have discussed in the past to use the same algorithm for both
components, so that you get consistent values, but there were always
other issues, that seemed to be more important.

Felix

Am 02.06.21 um 15:28 schrieb Deepak Chaudhari:
> Hi,
>
> I collected JTL file during a test run and using that I'm generating
> HTML report and aggregate report. When I compare aggregate and HTML
> reports I found that 90th and 95th percentile values are wrong in HTML
> report.
> I tried to change "jmeter.reportgenerator.statistic_window" value in
> user.properties but still getting wrong values.
>
> *Aggregate report:*
> image.png
>
> *HTML report:*
> image.png

Attachment: OpenPGP_signature
Description: OpenPGP digital signature

Reply via email to