Am 28.02.22 um 17:20 schrieb Vladimir Sitnikov:
We need to add a convention so launching Gradle with a property `jmeter.property.key=value` would pass `key=value` JMeter property (it should remove jmeter.property. prefix, and the prefix is there to avoid ambiguity) The relevant bits are here: https://github.com/apache/jmeter/blob/1d573158f2af7be882674e59c02490836abd0cde/build.gradle.kts#L599-L606 The command would be like ./gradlew test -Pjmeter.property.jmeter.regex_engine=oro
JMeterProperties default to SystemProperties. So currently I could start the test with -Djmeter.use_java_regex=true, right?
Or would I have to add a passProperty("jmeter.use_java_regex"|"jmeter.regex_engine" ) call?
The code in lines 599-606 seem to be able to set those, but how would I pass it as a JMeterProperty (so it could potentially overwrite a system property)? I have to admit, that I am really not a Gradle expert.
Then the engine variation could be added into the CI matrix builder: https://github.com/apache/jmeter/pull/693 Of course, it does not answer what should we do with test coverage, however: a) We could ensure CI always has both variations, and we could merge the coverage (Codecov supports it) b) It might make sense to create a targeted parameterized test (e.g. a test that parameterizes with regex_engine). I think we should not invest much time in building a specialized test. If there's a test that can easily be converted to a parameterized, then it is fine. If adding such a test is time-consuming, I would just go with a command-line property and the CI randomizer.
My original thought was to go by the route of parameterized tests, as they are something I have done before and know how to do :)
I will try to add a passProperty() call with the new property and add another build step to the CI system(s), even if it does a lot of work twice. That way, the tests will at least run with both settings.
Felix
Vladimir
OpenPGP_0xEA6C3728EA91C4AF.asc
Description: OpenPGP public key
OpenPGP_signature
Description: OpenPGP digital signature
