Lukasz Gajowy created BEAM-8826:
-----------------------------------
Summary: Investigate possibility of system metrics usage in
portable performance tests
Key: BEAM-8826
URL: https://issues.apache.org/jira/browse/BEAM-8826
Project: Beam
Issue Type: Task
Components: testing
Reporter: Lukasz Gajowy
We currently use
[TimeMonitor.java|https://github.com/apache/beam/blob/master/sdks/java/testing/test-utils/src/main/java/org/apache/beam/sdk/testutils/metrics/TimeMonitor.java]
and
[MeasureTime.py|https://github.com/apache/beam/blob/1988284a89b10b60eea48325f8a3b370b551c77c/sdks/python/apache_beam/testing/load_tests/load_test_metrics_utils.py#L406]
DoFns to collect runtime in both portable and non-portable performance tests.
However, in portable tests it seems to be possible to use [TOTAL_TIME_MSECS
|https://github.com/apache/beam/blob/1988284a89b10b60eea48325f8a3b370b551c77c/model/pipeline/src/main/proto/metrics.proto#L130]for
collecting execution time. Other system metrics are available as well (size,
bundle size etc).
It seems like a good way to simplify things and get more useful metrics from
portable jobs so it is worth investigating ways of using it in performance
tests.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
