[
https://issues.apache.org/jira/browse/GEODE-8950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17307269#comment-17307269
]
Donal Evans commented on GEODE-8950:
------------------------------------
To expand on Wayne's comment above, while I do believe that there was a
performance degradation in this benchmark due to GEODE-8496: top up dependency
updates (#5772), SHA 217be41f, the process of determining that that commit was
the likely cause was extremely time-consuming and required 240 runs of the
benchmark across 19 separate commits to produce enough data to be somewhat
confident in the result. The level of inconsistency in the benchmark is such
that unless a catastrophic degradation was introduced, it would be difficult to
notice amongst the background noise. Given that such a degradation would almost
certainly be detected in other benchmarks or performance tests, it's difficult
to see what value the P2pPartitionedPutLongBenchmark is currently providing.
> Benchmark failure - P2pPartitionedPutLongBenchmark
> --------------------------------------------------
>
> Key: GEODE-8950
> URL: https://issues.apache.org/jira/browse/GEODE-8950
> Project: Geode
> Issue Type: Bug
> Components: benchmarks
> Affects Versions: 1.15.0
> Reporter: Donal Evans
> Assignee: Donal Evans
> Priority: Major
> Labels: blocks-1.14.0, blocks-1.15.0
>
> Multiple benchmark failures due to P2pPartitionedPutLongBenchmark have been
> seen recently.
> This run saw 3 out of the 5 repeats fail due to flagged degradations in
> P2pPartitionedPutLongBenchmark:
> [https://concourse.apachegeode-ci.info/teams/main/pipelines/apache-develop-main/jobs/Benchmark_base/builds/16|https://concourse.apachegeode-ci.info/teams/main/pipelines/apache-develop-main/jobs/Benchmark_base/builds/16#L601ed52d:5552]
> This run saw 1 out of the 5 repeats fail due to flagged degradations in
> P2pPartitionedPutLongBenchmark:
> [https://concourse.apachegeode-ci.info/teams/main/pipelines/apache-develop-main/jobs/Benchmark_base/builds/20]
> This run saw 4 out of the 5 repeats fail due to flagged degradations in
> P2pPartitionedPutLongBenchmark:
> [https://concourse.apachegeode-ci.info/teams/main/pipelines/apache-develop-main/jobs/Benchmark_base/builds/27]
> In all the above benchmarks, the other failed runs were due to EOFExceptions
> rather than flagged degradations.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
