kamilwu commented on a change in pull request #12499:
URL: https://github.com/apache/beam/pull/12499#discussion_r467907301
##########
File path: .test-infra/jenkins/job_LoadTests_ParDo_Flink_Python.groovy
##########
@@ -161,12 +164,13 @@ def streamingScenarios = { datasetName ->
test : 'apache_beam.testing.load_tests.pardo_test',
runner : CommonTestProperties.Runner.PORTABLE,
pipelineOptions: [
- job_name : 'load-tests-python-flink-streaming-pardo-5-' +
now,
+ job_name : 'load-tests-python-flink-streaming-pardo-1-' +
now,
Review comment:
I have mixed feelings about it. This test is not
`load-tests-python-flink-batch-pardo-1` but on streaming. There are more
differences between them: batch-pardo-1 uses 10 iterations, this test uses 5
iterations. 0 counters in batch-pardo-1 vs. 3 counters right here. Because of
that, I think we should stay with the previous job_name:
`load-tests-python-flink-streaming-pardo-5`.
The general idea behind load tests is that we run the same configuration on
different runners, in different SDKs and in different mode (batch or
streaming). Grafana dashboards for load tests were designed with that
convention in mind. If you choose `java` and `streaming` from the list, Grafana
will pull data from these measurements: `java_streaming_pardo_1`,
`java_streaming_pardo_2` and so. Your streaming tests are a bit problematic,
because they are not being run on Dataflow and batch. Also, they have no Java
counterpart.
That being said, I think about two solutions:
1) Add more charts. We would end up with a total of six charts. The fifth
and the sixth chart would be empty in most cases (for Java and for batch).
2) Create a separate, more specific version of dashboard just for these two
tests (streaming-pardo-5 and streaming-pardo-6). Leave "ParDo Load Tests"
dashboard intact.
@mxm What do you think?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
