https://bz.apache.org/bugzilla/show_bug.cgi?id=64487
--- Comment #3 from Eligiusz Ziółkowski <[email protected]> --- Hi Felix, I would allow only the behaviour of 'TIMED' mode and removed support for 'FIXED'. Except for the issue described originally in the ticket which was found during test, I have I suspect other problems by looking at the code of SamplerMetric class. I believe FIXED mode will result in inconsistency between results provided by getKoPercentile, getOkPercentile on one hand and getAllPercentile on the other because * get{Ko,Ok}Percentile method use {ok,ko}ResponseStats which store up to LARGE_SLIDING_WINDOW_SIZE samples (=5000 by default) and * getAllPercentile method uses pctResponseStats member which stores up to SLIDING_WINDOW_SIZE samples (=100 by default) E.g. we sent 5000 samples in the test, which had response times 1000ms for all but last 101 samples, which had elapsed time 500ms. For simplicity, let's assume there were only successful samples. getOkPercentile(90) in FIXED mode will return 1000ms for the last period of the test while getAllPercentile(90) will return 500ms although all requests were Ok in the test. This is because getAllPercentile will catch only last 100 samples, while getOkPercentile will catch 5000. Allowing for non successful request doesn't invalidate this reasoning, there will be only more special cases, when it could work. Ok, what if we enforced that pctResponseStats have always the same window as {ko,ok}ResponseStats. Now things improve if we have only successful or only failed samples, but again it doesn't work in general. E.g. Let's assume the unified window size is 100. We run during the test 200 requests. First request lasted 1ms, second 2ms, third 3ms,..., 200th one lasted 200ms. Each odd request was successful, each even was failing. So successful samples are 1, 3, 5, ..., 199 ms long and failing ones 2, 4, 6, ... 200 ms long. getKoPercentile(90) = 180ms getOkPercentile(90) = 179ms getAllPercentile(90) = 190ms > getKoPercentile(90) > getOkPercentile(90) which seems impossible unless we recall, that getAllPercentile covers only last 100 samples with times 101, 102 ..., 200. How about test plans with multiple samplers reported separately? E.g. with the window size 100, if the sampler A has throughput 100/min and sampler B has throughput 10/min, the percentile for sampler A will describe what happened in last minute, while for sampler B it will describe what happened in last 10 minutes (10 req/min x10min = 100 req = window size). A bit complicated to interpret. I think leaving 'TIMED' mode as the only option helps us avoid all this issues. On the other hand, if for some reason someone really needs to aggregate over fixed number of samples, custom listener can be coded or .jtl file used. -- You are receiving this mail because: You are the assignee for the bug.
