[
https://issues.apache.org/jira/browse/BEAM-4775?focusedWorklogId=200188&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-200188
]
ASF GitHub Bot logged work on BEAM-4775:
----------------------------------------
Author: ASF GitHub Bot
Created on: 18/Feb/19 18:35
Start Date: 18/Feb/19 18:35
Worklog Time Spent: 10m
Work Description: ryan-williams commented on pull request #7868:
[BEAM-4775] MonitoringInfo URN tweaks
URL: https://github.com/apache/beam/pull/7868#discussion_r257795396
##########
File path: model/fn-execution/src/main/proto/beam_fn_api.proto
##########
@@ -339,14 +339,16 @@ message MonitoringInfoSpecs {
enum Enum {
// TODO(ajamato): Add the PTRANSFORM name as a required label after
// upgrading the python SDK.
- USER_COUNTER = 0 [(monitoring_info_spec) = {
+ USER_METRIC = 0 [(monitoring_info_spec) = {
Review comment:
Currently, metrics are either of "user" type (defined by {ptransform,
namespace, name}), or they're one of the 5 "system"-metric types below.
It's true that the "user" category is not a "specific URN", so there is
tension with e.g. [this docstring
above](https://github.com/apache/beam/pull/7868/files#diff-4d226ebf3f70cf18449c29e82511a67eR333).
I think of this table as enumerating the possible metrics-URN *categories*
(indeed, that's what it does, and what our design calls for, as opposed to e.g.
enumerating all allowed "user"-metric URNs here). The 5 "system" categories are
each inhabited by exactly one URN, so they are particularly easy to deal with
in SDKs, while the user-metric category requires special treatment (which,
conveniently, the SDKs already implement).
In this PR, I am laying some groundwork toward supporting more "types" of
metrics (both "user" and "system"). "User" metrics should be able to take on
the three types listed here (counter, distribution, gauge).
With that context, there's a few directions I can see taking your comment:
- split the "user" category into 3 type-specific user-categories
- leave as-is, but make the docs more clear about the special status of
`USER_METRIC`'s "spec"
- some larger overhaul involving removing user metrics from this table.
Let me know what you think!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 200188)
Time Spent: 6h 10m (was: 6h)
> JobService should support returning metrics
> -------------------------------------------
>
> Key: BEAM-4775
> URL: https://issues.apache.org/jira/browse/BEAM-4775
> Project: Beam
> Issue Type: Bug
> Components: beam-model
> Reporter: Eugene Kirpichov
> Assignee: Ryan Williams
> Priority: Major
> Labels: triaged
> Time Spent: 6h 10m
> Remaining Estimate: 0h
>
> [https://github.com/apache/beam/blob/master/model/job-management/src/main/proto/beam_job_api.proto]
> currently doesn't appear to have a way for JobService to return metrics to a
> user, even though
> [https://github.com/apache/beam/blob/master/model/fn-execution/src/main/proto/beam_fn_api.proto]
> includes support for reporting SDK metrics to the runner harness.
>
> Metrics are apparently necessary to run any ValidatesRunner tests because
> PAssert needs to validate that the assertions succeeded. However, this
> statement should be double-checked: perhaps it's possible to somehow work
> with PAssert without metrics support.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
