Barrett Oglesby created GEODE-7276:
--------------------------------------
Summary: CI failure: Benchmark PartitionedPutBenchmark analysis
failed
Key: GEODE-7276
URL: https://issues.apache.org/jira/browse/GEODE-7276
Project: Geode
Issue Type: Bug
Components: benchmarks
Reporter: Barrett Oglesby
Benchmark test 615
(https://concourse.apachegeode-ci.info/teams/main/pipelines/apache-develop-main/jobs/Benchmark/builds/615)
failed with:
{noformat}
BENCHMARK FAILED: org.apache.geode.benchmark.tests.PartitionedPutBenchmark
missing result file.
BENCHMARK FAILED: org.apache.geode.benchmark.tests.PartitionedPutBenchmark
missing result file.
{noformat}
The Analyzer is attempting to compare the test with the baseline and not
finding a
{noformat}
for (BenchmarkRunResult.ProbeResult probeResult : benchmarkResult.probeResults)
{
if (isNaN(probeResult.baseline) || isNaN(probeResult.test)) {
errorMessage.append("BENCHMARK FAILED: ").append(benchmarkResult.name)
.append(" missing result file.\n");
{noformat}
The output for the PartitionedPutBenchmark is:
{noformat}
org.apache.geode.benchmark.tests.PartitionedPutBenchmark
average ops/second Baseline: 204975.67 Test: NaN
Difference: NaN%
ops/second standard error Baseline: 308.71 Test: NaN
Difference: NaN%
ops/second standard deviation Baseline: 5338.09 Test: -0.00
Difference: -100.0%
YS 99th percentile latency Baseline: 3808.00 Test: 3510.00
Difference: -7.8%
median latency Baseline: 575487.00 Test: 568319.00
Difference: -1.2%
90th percentile latency Baseline: 4476927.00 Test: 4378623.00
Difference: -2.2%
99th percentile latency Baseline: 9379839.00 Test: 9543679.00
Difference: +1.7%
99.9th percentile latency Baseline: 51118079.00 Test: 52002815.00
Difference: +1.7%
average latency Baseline: 1754859.17 Test: 1757899.99
Difference: +0.2%
latency standard deviation Baseline: 3494292.88 Test: 5794614.66
Difference: +65.8%
latency standard error Baseline: 445.63 Test: 2335.52
Difference: +424.1%
{noformat}
It would be nice if the BENCHMARK FAILED message printed the
probeResult.description, but I guess the two BENCHMARK FAILED messages
correspond to average ops/second and ops/second standard error since those are
the two NAN values.
There are several RuntimeException in both the baseline and branch client.log
files, but since both of the files contain those, it doesn't look related.
{noformat}
[RMI TCP Connection(1)-172.31.46.146] WARN org.reflections.Reflections - could
not create Dir using directory from url
file:/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/ext/libatk-wrapper.so. skipping.
java.lang.RuntimeException: cannot use dir
/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/ext/libatk-wrapper.so
[RMI TCP Connection(1)-172.31.46.146] WARN org.reflections.Reflections - could
not create Vfs.Dir from url. ignoring the exception and continuing
org.reflections.ReflectionsException: could not create Vfs.Dir from url, no
matching UrlType was found
[file:/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/ext/libatk-wrapper.so]
either use fromURL(final URL url, final List<UrlType> urlTypes) or use the
static setDefaultURLTypes(final List<UrlType> urlTypes) or
addDefaultURLTypes(UrlType urlType) with your specialized UrlType.
{noformat}
On the other hand, the branch client.log file contains this exception, but
unfortunately, the servers look to be configured with warn log level, so its
hard to tell why this exception occurred, and if its even relevant.
{noformat}
Finishing main test [ts=1570353668642, date=Sun Oct 06 09:21:08 UTC 2019]
ERROR: Shutting down benchmark driver to unexpected exception.
Type '--help' for usage.
org.apache.geode.cache.client.ServerConnectivityException: Pool unexpected
socket timed out on client connection=Pooled Connection to
ip-172-31-42-223.us-west-2.compute.internal:46801: Connection[DESTROYED]
attempt=2). Server unreachable: could not connect after 2 attempts
at
org.apache.geode.cache.client.internal.OpExecutorImpl.handleException(OpExecutorImpl.java:659)
at
org.apache.geode.cache.client.internal.OpExecutorImpl.handleException(OpExecutorImpl.java:501)
at
org.apache.geode.cache.client.internal.OpExecutorImpl.execute(OpExecutorImpl.java:153)
at
org.apache.geode.cache.client.internal.OpExecutorImpl.execute(OpExecutorImpl.java:108)
at
org.apache.geode.cache.client.internal.PoolImpl.execute(PoolImpl.java:771)
at org.apache.geode.cache.client.internal.PutOp.execute(PutOp.java:89)
at
org.apache.geode.cache.client.internal.ServerRegionProxy.put(ServerRegionProxy.java:159)
at
org.apache.geode.internal.cache.LocalRegion.serverPut(LocalRegion.java:3035)
at
org.apache.geode.internal.cache.LocalRegion.cacheWriteBeforePut(LocalRegion.java:3152)
at
org.apache.geode.internal.cache.ProxyRegionMap.basicPut(ProxyRegionMap.java:238)
at
org.apache.geode.internal.cache.LocalRegion.virtualPut(LocalRegion.java:5579)
at
org.apache.geode.internal.cache.LocalRegionDataView.putEntry(LocalRegionDataView.java:162)
at
org.apache.geode.internal.cache.LocalRegion.basicPut(LocalRegion.java:5035)
at
org.apache.geode.internal.cache.LocalRegion.validatedPut(LocalRegion.java:1635)
at
org.apache.geode.internal.cache.LocalRegion.put(LocalRegion.java:1622)
at
org.apache.geode.internal.cache.AbstractRegion.put(AbstractRegion.java:441)
at org.apache.geode.benchmark.tasks.PutTask.test(PutTask.java:52)
at
org.yardstickframework.impl.BenchmarkRunner$2.run(BenchmarkRunner.java:178)
at java.lang.Thread.run(Thread.java:748)
{noformat}
The Benchmark test 616 results for the PartitionedPutBenchmark looked much
better:
{noformat}
org.apache.geode.benchmark.tests.PartitionedPutBenchmark
average ops/second Baseline: 205572.82 Test: 201880.99
Difference: -1.8%
ops/second standard error Baseline: 339.40 Test: 326.04
Difference: -3.9%
ops/second standard deviation Baseline: 5868.77 Test: 5637.70
Difference: -3.9%
YS 99th percentile latency Baseline: 4106.00 Test: 4105.00
Difference: -0.0%
median latency Baseline: 687103.00 Test: 683519.00
Difference: -0.5%
90th percentile latency Baseline: 4136959.00 Test: 4210687.00
Difference: +1.8%
99th percentile latency Baseline: 8470527.00 Test: 8650751.00
Difference: +2.1%
99.9th percentile latency Baseline: 49643519.00 Test: 53215231.00
Difference: +7.2%
average latency Baseline: 1748928.87 Test: 1780801.76
Difference: +1.8%
latency standard deviation Baseline: 3313021.29 Test: 3533721.57
Difference: +6.7%
latency standard error Baseline: 421.90 Test: 454.12
Difference: +7.6%
{noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
