[
https://issues.apache.org/jira/browse/BEAM-3803?focusedWorklogId=78922&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-78922
]
ASF GitHub Bot logged work on BEAM-3803:
----------------------------------------
Author: ASF GitHub Bot
Created on: 09/Mar/18 15:22
Start Date: 09/Mar/18 15:22
Worklog Time Spent: 10m
Work Description: iemejia commented on a change in pull request #4823:
[BEAM-3803] [Nexmark] Handle both committed and attempted metrics
URL: https://github.com/apache/beam/pull/4823#discussion_r173478469
##########
File path:
sdks/java/nexmark/src/main/java/org/apache/beam/sdk/nexmark/NexmarkLauncher.java
##########
@@ -199,33 +199,41 @@ private long getCounterMetric(PipelineResult result,
String namespace, String na
Iterable<MetricResult<Long>> counters = metrics.counters();
try {
MetricResult<Long> metricResult = counters.iterator().next();
- return metricResult.attempted();
- } catch (NoSuchElementException e) {
+ Long value = metricResult.committed();
+ if (value == null) {
+ value = metricResult.attempted();
+ }
+ return value;
+ } catch (NoSuchElementException | NullPointerException e) {
LOG.error("Failed to get metric {}, from namespace {}", name, namespace);
}
return defaultValue;
}
/**
* Return the current value for a long counter, or a default value if can't
be retrieved.
- * Note this uses only attempted metrics because some runners don't support
committed metrics.
+ * Note some runners don't support committed metrics and some don't support
attempted metrics.
*/
private long getDistributionMetric(PipelineResult result, String namespace,
String name,
DistributionType distType, long defaultValue) {
MetricQueryResults metrics = result.metrics().queryMetrics(
MetricsFilter.builder().addNameFilter(MetricNameFilter.named(namespace,
name)).build());
Iterable<MetricResult<DistributionResult>> distributions =
metrics.distributions();
try {
- MetricResult<DistributionResult> distributionResult =
distributions.iterator().next();
+ MetricResult<DistributionResult> distributionResultMetric =
distributions.iterator().next();
+ DistributionResult distributionResult =
distributionResultMetric.committed();
+ if (distributionResult == null) {
+ distributionResult = distributionResultMetric.attempted();
+ }
switch (distType) {
case MIN:
- return distributionResult.attempted().min();
+ return distributionResult.min();
case MAX:
- return distributionResult.attempted().max();
+ return distributionResult.max();
default:
return defaultValue;
}
- } catch (NoSuchElementException e) {
+ } catch (NoSuchElementException | NullPointerException e) {
Review comment:
As above maybe better to not catch NullPointerException, better to be
consistent with the UnsupportedOperationException contract.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 78922)
Time Spent: 2h 20m (was: 2h 10m)
> [Nexmark] Not all runners support attempted metrics
> ---------------------------------------------------
>
> Key: BEAM-3803
> URL: https://issues.apache.org/jira/browse/BEAM-3803
> Project: Beam
> Issue Type: Bug
> Components: runner-dataflow
> Reporter: Andrew Pilloud
> Assignee: Andrew Pilloud
> Priority: Major
> Labels: nexmark
> Time Spent: 2h 20m
> Remaining Estimate: 0h
>
> The dataflow runner only supports committed metrics for batch jobs and
> attempted metrics for streaming jobs. Nexmark should use the best available
> metric source.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
