Is it implemented or not yet? I downloaded JMeter 5.4.1 and added property "backend_metrics_percentile_estimator = R_3" in user.properties Still same results.
[image: image.png] [image: image.png] On Thu, Jun 3, 2021 at 11:43 PM Felix Schumacher < felix.schumac...@internetallee.de> wrote: > > Am 03.06.21 um 20:10 schrieb Deepak Chaudhari: > > Thank you very much for your reply. > Will it be in user.properties? > Do we need to install the latest JMeter version to reflect the changes? > > A build from trunk can be found at > https://ci-builds.apache.org/job/JMeter/job/JMeter-trunk/ > > Yes, the property can be set via user.properties. > > Felix > > > On Thu, Jun 3, 2021 at 11:14 PM Felix Schumacher < > felix.schumac...@internetallee.de> wrote: > >> You might find https://bz.apache.org/bugzilla/show_bug.cgi?id=65353 >> interesting. >> >> With the next nightly build or build from trunk, you should be able to use >> the new property "backend_metrics_percentile_estimator" with a value of >> "R_3" to lessen the difference between both reports. But keep in mind, that >> the HTML Report use a sliding window, while the Aggregate Report does not >> (and will eventually OOM). >> >> Felix >> >> >> Am 03.06.21 um 17:46 schrieb Deepak Chaudhari: >> >> I need to send both the reports to the client and surely the question >> will come "Why is there so much difference?" >> If there is any way with which we can generate almost similar aggregate >> and HTML reports? >> >> On Thu, Jun 3, 2021 at 9:00 PM Felix Schumacher < >> felix.schumac...@internetallee.de> wrote: >> >>> Without the actual data it is impossible to say, if the reports are >>> wrong, or if they follow different ways to calculate a percentile. >>> >>> The problem here is, that the two reports are using different algorithms >>> to calculate the percentiles. We are working with discrete numbers and here >>> a percentile is more like a range than a single (correct or wrong) number. >>> Say, you have the values [1, 2, 4, 8, 16, 32, 64, 128, 256, 512] want to >>> calculate a 90% percentile. For this you take any number that splits the >>> (sorted) list into two lists, where one list has all elements, that are >>> smaller (or equal) to the number and one list contains all elements that >>> are bigger than the number. For this example, any number between 256 and >>> 511.99 would be a valid 90% percentile. (that is of course a broad >>> description only) >>> >>> The Aggregate report currently uses a memory intensive way and stores a >>> list with all values and picks the smallest number, that fits the above >>> description. (This memory hogging behaviour should be fixed and would lead >>> to less accurate data and less OOMs or slow JMeter instances) >>> >>> The HTML report gives you a value that would probably be a good 90% >>> percentile, if you would have more data like the one, you already have by >>> picking some value between the lower bound and the upper bound of the >>> described range (in our example something between 256 and 511.999). >>> >>> As your report has very few samples (21 for the biggest difference in >>> the numbers), the skew between the reported percentiles can look big, but >>> are not necessarily wrong. >>> >>> Note, that if the numbers that both reports tell you are far apart, that >>> is probably a sign of having either too few samples, or the samples >>> (durations of the samples) are distributed sparsely near the percentile >>> ranges. >>> >>> We have discussed in the past to use the same algorithm for both >>> components, so that you get consistent values, but there were always other >>> issues, that seemed to be more important. >>> >>> Felix >>> Am 02.06.21 um 15:28 schrieb Deepak Chaudhari: >>> >>> Hi, >>> >>> I collected JTL file during a test run and using that I'm generating >>> HTML report and aggregate report. When I compare aggregate and HTML reports >>> I found that 90th and 95th percentile values are wrong in HTML report. >>> I tried to change "jmeter.reportgenerator.statistic_window" value in >>> user.properties but still getting wrong values. >>> >>> *Aggregate report:* >>> [image: image.png] >>> >>> *HTML report:* >>> [image: image.png] >>> >>>
