Indeed. :) On Fri, Jun 4, 2021 at 11:52 PM Philippe Mouawad < p.moua...@ubik-ingenierie.com> wrote:
> You rock Felix ! > > On Fri, Jun 4, 2021 at 5:39 PM Deepak Chaudhari <deepak.myp...@gmail.com> > wrote: > >> I downloaded JMeter 5.5 and compared aggregate and HTML reports. >> Now the results are exactly the same >> >> Thank you very much Felix for fixing it on such short notice. :) >> >> [image: image.png] >> >> [image: image.png] >> . >> >> On Thu, Jun 3, 2021 at 11:14 PM Felix Schumacher < >> felix.schumac...@internetallee.de> wrote: >> >>> You might find https://bz.apache.org/bugzilla/show_bug.cgi?id=65353 >>> interesting. >>> >>> With the next nightly build or build from trunk, you should be able to use >>> the new property "backend_metrics_percentile_estimator" with a value of >>> "R_3" to lessen the difference between both reports. But keep in mind, that >>> the HTML Report use a sliding window, while the Aggregate Report does not >>> (and will eventually OOM). >>> >>> Felix >>> >>> >>> Am 03.06.21 um 17:46 schrieb Deepak Chaudhari: >>> >>> I need to send both the reports to the client and surely the question >>> will come "Why is there so much difference?" >>> If there is any way with which we can generate almost similar aggregate >>> and HTML reports? >>> >>> On Thu, Jun 3, 2021 at 9:00 PM Felix Schumacher < >>> felix.schumac...@internetallee.de> wrote: >>> >>>> Without the actual data it is impossible to say, if the reports are >>>> wrong, or if they follow different ways to calculate a percentile. >>>> >>>> The problem here is, that the two reports are using different >>>> algorithms to calculate the percentiles. We are working with discrete >>>> numbers and here a percentile is more like a range than a single (correct >>>> or wrong) number. Say, you have the values [1, 2, 4, 8, 16, 32, 64, 128, >>>> 256, 512] want to calculate a 90% percentile. For this you take any number >>>> that splits the (sorted) list into two lists, where one list has all >>>> elements, that are smaller (or equal) to the number and one list contains >>>> all elements that are bigger than the number. For this example, any number >>>> between 256 and 511.99 would be a valid 90% percentile. (that is of course >>>> a broad description only) >>>> >>>> The Aggregate report currently uses a memory intensive way and stores a >>>> list with all values and picks the smallest number, that fits the above >>>> description. (This memory hogging behaviour should be fixed and would lead >>>> to less accurate data and less OOMs or slow JMeter instances) >>>> >>>> The HTML report gives you a value that would probably be a good 90% >>>> percentile, if you would have more data like the one, you already have by >>>> picking some value between the lower bound and the upper bound of the >>>> described range (in our example something between 256 and 511.999). >>>> >>>> As your report has very few samples (21 for the biggest difference in >>>> the numbers), the skew between the reported percentiles can look big, but >>>> are not necessarily wrong. >>>> >>>> Note, that if the numbers that both reports tell you are far apart, >>>> that is probably a sign of having either too few samples, or the samples >>>> (durations of the samples) are distributed sparsely near the percentile >>>> ranges. >>>> >>>> We have discussed in the past to use the same algorithm for both >>>> components, so that you get consistent values, but there were always other >>>> issues, that seemed to be more important. >>>> >>>> Felix >>>> Am 02.06.21 um 15:28 schrieb Deepak Chaudhari: >>>> >>>> Hi, >>>> >>>> I collected JTL file during a test run and using that I'm generating >>>> HTML report and aggregate report. When I compare aggregate and HTML reports >>>> I found that 90th and 95th percentile values are wrong in HTML report. >>>> I tried to change "jmeter.reportgenerator.statistic_window" value in >>>> user.properties but still getting wrong values. >>>> >>>> *Aggregate report:* >>>> [image: image.png] >>>> >>>> *HTML report:* >>>> [image: image.png] >>>> >>>> > > -- > Cordialement > Philippe M. > Ubik-Ingenierie >
