pivotal-jbarrett commented on pull request #150: URL: https://github.com/apache/geode-benchmarks/pull/150#issuecomment-841532362
@upthewaterspout > One general question - when this merges, will these redis benchmarks be run in CI automatically? Is that going to work running against the previous version? I'm not sure how we are choosing which benchmarks run in CI, what baseline version to use and what the threshold is for failure. I have been working this one out in my head. Maybe you have some thoughts. As it stands the default benchmark job runs all the benchmarks without any options. It assumes all benchmarks have a topology of 4 hosts. The Redis benchmarks will all fail because they have a topology of 8 hosts, or something, by default. If we set the `-Pbenchmark.withServerCount=2` option they would run when the Geode version at least 1.15.0.build.0 (meaning anything pre-release 1.15 to anything after 1.15 GA). Since current benchmarks compare against 1.13.2 (the last release) those comparisons will fail since there is no Redis support. We could set the CI to run `--tests o.a.g.benchmark.tests.*` but it turns out that matches recursively and all the Redis benchmarks are sub-package. My thought is to refactor the packaging a little, or even make a new module redis-benchmarks, so that the `--tests` won't match. My leaning is towards a new module. Thoughts? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
