Lukasz Gajowy created BEAM-8826: ----------------------------------- Summary: Investigate possibility of system metrics usage in portable performance tests Key: BEAM-8826 URL: https://issues.apache.org/jira/browse/BEAM-8826 Project: Beam Issue Type: Task Components: testing Reporter: Lukasz Gajowy
We currently use [TimeMonitor.java|https://github.com/apache/beam/blob/master/sdks/java/testing/test-utils/src/main/java/org/apache/beam/sdk/testutils/metrics/TimeMonitor.java] and [MeasureTime.py|https://github.com/apache/beam/blob/1988284a89b10b60eea48325f8a3b370b551c77c/sdks/python/apache_beam/testing/load_tests/load_test_metrics_utils.py#L406] DoFns to collect runtime in both portable and non-portable performance tests. However, in portable tests it seems to be possible to use [TOTAL_TIME_MSECSĀ |https://github.com/apache/beam/blob/1988284a89b10b60eea48325f8a3b370b551c77c/model/pipeline/src/main/proto/metrics.proto#L130]for collecting execution time. Other system metrics are available as well (size, bundle size etc). It seems like a good way to simplify things and get more useful metrics from portable jobs so it is worth investigating ways of using it in performance tests. -- This message was sent by Atlassian Jira (v8.3.4#803005)