[
https://issues.apache.org/jira/browse/BEAM-6627?focusedWorklogId=196168&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-196168
]
ASF GitHub Bot logged work on BEAM-6627:
----------------------------------------
Author: ASF GitHub Bot
Created on: 08/Feb/19 11:22
Start Date: 08/Feb/19 11:22
Worklog Time Spent: 10m
Work Description: lgajowy commented on pull request #7772: [BEAM-6627]
Added Metrics API processing time reporting to TextIOIT
URL: https://github.com/apache/beam/pull/7772#discussion_r255033872
##########
File path:
sdks/java/io/file-based-io-tests/src/test/java/org/apache/beam/sdk/io/common/FileBasedIOITHelper.java
##########
@@ -65,6 +71,36 @@ public void processElement(ProcessContext c) {
}
}
+ public static void publishTestMetrics(
Review comment:
There's no need to treat time metrics and gcs differently. Both can be read
and published using the same set of methods. I think both `publishGcsResults`
and `publishTestMetrics` should be refactored to:
- a method that takes all metrics from the `PipelineResult` and returns
`List<NamedTestResult>`
- a method that logs results to console (this is done in `publishGcsResults`
and should be done with other metrics as well
- a method that publishes to BigQuery if bq options are present
So this would give the following method signatures (a general idea):
1. `Collection<NamedTestResult> readMetrics(PipelineResult result)`
2. `void publishToConsole(Collection<NamedTestResult> results)`
3. `void publishToBigQuery(Collection<NamedTestResult> results)`
For 2 I think you can reuse `ConsoleResultPublisher` from `load-tests`
module but please move it to `test-utils) same as you're reusing
`BigQueryResultPublisher`.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 196168)
Time Spent: 1h (was: 50m)
> Use Metrics API in IO performance tests
> ---------------------------------------
>
> Key: BEAM-6627
> URL: https://issues.apache.org/jira/browse/BEAM-6627
> Project: Beam
> Issue Type: Improvement
> Components: testing
> Reporter: Michal Walenia
> Assignee: Michal Walenia
> Priority: Minor
> Time Spent: 1h
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
