Re: Question on testing properties in JMeter
Am 01.03.22 um 12:03 schrieb Vladimir Sitnikov:
When I run ./gradlew test -Djmeter.use_java_regex=true, it seems to have
more effect
I have no idea how it works now, however, there are three JVMs:
1) The first JVM evaluates the build script (it basically runs
build.gradle.kts, and it is the JVM that parses the build command-line)
2) The second JVM runs @Test classes
3) Then a test class might fork another JVM with JMeter
Or would I have to add a
passProperty("jmeter.use_java_regex"|"jmeter.regex_engine" ) call?
passProperty is needed to pass properties (sorry for duplication) between
#1 and #2 above.
If @Test class forks a JVM for JMeter, then it should somehow pass
properties on its own.
Of course, we can follow the "defer to System.properties by default"
approach, and we could ensure
all JVM forks pass jmeter.regex_engine (e.g. all jmeter.* properties) to
the forked JVMs via their system.properties.
I am afraid it would create tight coupling on System.properties, thus it
would make it harder to
launch JMeter instances in "isolation" within a single JVM.
On the other hand, if we prefer that System.properties contain no
JMeter-specific properties,
then we might end up with a better design, where all the properties could
be passed explicitly (via the constructor or whatever),
and it would make it easier to test: test parallelism is easier to manage
when tests do not require mutating the global state like System.properties.
So I am inclined we should favor designs where all the properties are
passed explicitly (e.g. via mask),
instead of "all properties occurring in System.properties automagically".
I have implemented a passing of system properties that match those with
"jmeter.properties." as prefix in PR 700.
With that I can provide a system property named "jmeter.use_java_regex"
to the unit tests run by gradle with
./gradlew test -Djmeter.properties.jmeter.use_java_regex=true
I believe, that is what you tried to explain to me.
Felix
WDYT?
Vladimir
OpenPGP_0xEA6C3728EA91C4AF.asc
Description: OpenPGP public key
OpenPGP_signature
Description: OpenPGP digital signature
Re: Question on testing properties in JMeter
>When I run ./gradlew test -Djmeter.use_java_regex=true, it seems to have
>more effect
I have no idea how it works now, however, there are three JVMs:
1) The first JVM evaluates the build script (it basically runs
build.gradle.kts, and it is the JVM that parses the build command-line)
2) The second JVM runs @Test classes
3) Then a test class might fork another JVM with JMeter
>Or would I have to add a
>passProperty("jmeter.use_java_regex"|"jmeter.regex_engine" ) call?
passProperty is needed to pass properties (sorry for duplication) between
#1 and #2 above.
If @Test class forks a JVM for JMeter, then it should somehow pass
properties on its own.
Of course, we can follow the "defer to System.properties by default"
approach, and we could ensure
all JVM forks pass jmeter.regex_engine (e.g. all jmeter.* properties) to
the forked JVMs via their system.properties.
I am afraid it would create tight coupling on System.properties, thus it
would make it harder to
launch JMeter instances in "isolation" within a single JVM.
On the other hand, if we prefer that System.properties contain no
JMeter-specific properties,
then we might end up with a better design, where all the properties could
be passed explicitly (via the constructor or whatever),
and it would make it easier to test: test parallelism is easier to manage
when tests do not require mutating the global state like System.properties.
So I am inclined we should favor designs where all the properties are
passed explicitly (e.g. via mask),
instead of "all properties occurring in System.properties automagically".
WDYT?
Vladimir
Re: Question on testing properties in JMeter
Am 28.02.22 um 19:15 schrieb Felix Schumacher:
Am 28.02.22 um 17:20 schrieb Vladimir Sitnikov:
We need to add a convention so launching Gradle with a property
`jmeter.property.key=value` would pass
`key=value` JMeter property (it should remove jmeter.property.
prefix, and
the prefix is there to avoid ambiguity)
The relevant bits are here:
https://github.com/apache/jmeter/blob/1d573158f2af7be882674e59c02490836abd0cde/build.gradle.kts#L599-L606
The command would be like ./gradlew test
-Pjmeter.property.jmeter.regex_engine=oro
JMeterProperties default to SystemProperties. So currently I could
start the test with -Djmeter.use_java_regex=true, right?
When I run ./gradlew test -Djmeter.use_java_regex=true, it seems to have
more effect, than to add the property to bin/system.property. Now more
tests fail and I will look into those.
Felix
Or would I have to add a
passProperty("jmeter.use_java_regex"|"jmeter.regex_engine" ) call?
The code in lines 599-606 seem to be able to set those, but how would
I pass it as a JMeterProperty (so it could potentially overwrite a
system property)? I have to admit, that I am really not a Gradle expert.
Then the engine variation could be added into the CI matrix builder:
https://github.com/apache/jmeter/pull/693
Of course, it does not answer what should we do with test coverage,
however:
a) We could ensure CI always has both variations, and we could merge the
coverage (Codecov supports it)
b) It might make sense to create a targeted parameterized test (e.g.
a test
that parameterizes with regex_engine).
I think we should not invest much time in building a specialized
test. If
there's a test that can easily be converted to a parameterized,
then it is fine. If adding such a test is time-consuming, I would
just go
with a command-line property and the CI randomizer.
My original thought was to go by the route of parameterized tests, as
they are something I have done before and know how to do :)
I will try to add a passProperty() call with the new property and add
another build step to the CI system(s), even if it does a lot of work
twice. That way, the tests will at least run with both settings.
Felix
Vladimir
OpenPGP_0xEA6C3728EA91C4AF.asc
Description: OpenPGP public key
OpenPGP_signature
Description: OpenPGP digital signature
Re: Question on testing properties in JMeter
Am 28.02.22 um 17:20 schrieb Vladimir Sitnikov:
We need to add a convention so launching Gradle with a property
`jmeter.property.key=value` would pass
`key=value` JMeter property (it should remove jmeter.property. prefix, and
the prefix is there to avoid ambiguity)
The relevant bits are here:
https://github.com/apache/jmeter/blob/1d573158f2af7be882674e59c02490836abd0cde/build.gradle.kts#L599-L606
The command would be like ./gradlew test
-Pjmeter.property.jmeter.regex_engine=oro
JMeterProperties default to SystemProperties. So currently I could start
the test with -Djmeter.use_java_regex=true, right?
Or would I have to add a
passProperty("jmeter.use_java_regex"|"jmeter.regex_engine" ) call?
The code in lines 599-606 seem to be able to set those, but how would I
pass it as a JMeterProperty (so it could potentially overwrite a system
property)? I have to admit, that I am really not a Gradle expert.
Then the engine variation could be added into the CI matrix builder:
https://github.com/apache/jmeter/pull/693
Of course, it does not answer what should we do with test coverage, however:
a) We could ensure CI always has both variations, and we could merge the
coverage (Codecov supports it)
b) It might make sense to create a targeted parameterized test (e.g. a test
that parameterizes with regex_engine).
I think we should not invest much time in building a specialized test. If
there's a test that can easily be converted to a parameterized,
then it is fine. If adding such a test is time-consuming, I would just go
with a command-line property and the CI randomizer.
My original thought was to go by the route of parameterized tests, as
they are something I have done before and know how to do :)
I will try to add a passProperty() call with the new property and add
another build step to the CI system(s), even if it does a lot of work
twice. That way, the tests will at least run with both settings.
Felix
Vladimir
OpenPGP_0xEA6C3728EA91C4AF.asc
Description: OpenPGP public key
OpenPGP_signature
Description: OpenPGP digital signature
Re: Question on testing properties in JMeter
We need to add a convention so launching Gradle with a property `jmeter.property.key=value` would pass `key=value` JMeter property (it should remove jmeter.property. prefix, and the prefix is there to avoid ambiguity) The relevant bits are here: https://github.com/apache/jmeter/blob/1d573158f2af7be882674e59c02490836abd0cde/build.gradle.kts#L599-L606 The command would be like ./gradlew test -Pjmeter.property.jmeter.regex_engine=oro Then the engine variation could be added into the CI matrix builder: https://github.com/apache/jmeter/pull/693 Of course, it does not answer what should we do with test coverage, however: a) We could ensure CI always has both variations, and we could merge the coverage (Codecov supports it) b) It might make sense to create a targeted parameterized test (e.g. a test that parameterizes with regex_engine). I think we should not invest much time in building a specialized test. If there's a test that can easily be converted to a parameterized, then it is fine. If adding such a test is time-consuming, I would just go with a command-line property and the CI randomizer. Vladimir
