[
https://issues.apache.org/jira/browse/BEAM-3545?focusedWorklogId=84640&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-84640
]
ASF GitHub Bot logged work on BEAM-3545:
----------------------------------------
Author: ASF GitHub Bot
Created on: 27/Mar/18 01:14
Start Date: 27/Mar/18 01:14
Worklog Time Spent: 10m
Work Description: lostluck commented on a change in pull request #4899:
[BEAM-3545] Go SDK UserCounters
URL: https://github.com/apache/beam/pull/4899#discussion_r177280743
##########
File path: sdks/go/pkg/beam/core/metrics/metrics.go
##########
@@ -0,0 +1,447 @@
+// Licensed to the Apache Software Foundation (ASF) under one or more
+// contributor license agreements. See the NOTICE file distributed with
+// this work for additional information regarding copyright ownership.
+// The ASF licenses this file to You under the Apache License, Version 2.0
+// (the "License"); you may not use this file except in compliance with
+// the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+// Package metrics handles Beam metrics.
+//
+// Metrics in the Beam model are uniquely identified by a namespace, a name,
+// and the PTransform context in which they are used. Further, they are
+// reported as a delta against the bundle being processed, so that overcounting
+// doesn't occur if a bundle needs to be retried.
+//
+// Metric containers aren't initialized until their first mutation, which
+// follows from the Beam model design, where metrics are only sent for a bundle
+// if they have changed. This is particularly convenient for distributions
which
+// means their min and max fields can be set to the first value on creation
+// rather than have some marker of uninitialized state, which would otherwise
+// need to be checked for on every update.
+//
+// Metric values are implemented as lightweight proxies of the user provided
+// namespace and name. This allows them to be declared globally, and used in
+// any ParDo. To handle reporting deltas on the metrics by bundle, metrics
+// are keyed by bundleID,PTransformID,namespace, and name, so metrics that
+// are identical except for bundles are treated as distinct, effectively
+// providing per bundle deltas, since a new value container is used per bundle.
+package metrics
+
+import (
+ "context"
+ "fmt"
+ "sort"
+ "sync"
+ "time"
+
+ "github.com/apache/beam/sdks/go/pkg/beam/core/util/bundle"
+ "github.com/apache/beam/sdks/go/pkg/beam/core/util/ptransform"
+ "github.com/apache/beam/sdks/go/pkg/beam/log"
+ "github.com/apache/beam/sdks/go/pkg/beam/model/fnexecution_v1"
+ "github.com/golang/protobuf/ptypes"
+)
+
+// userMetricker knows how to convert it's value to a Metrics_User proto.
+type userMetricker interface {
Review comment:
Good point! I dropped the "ker" suffix on the type.
We can always work around not having the private interface. It certainly
doesn't make sense to make it public at this point. Having the interface at all
saves us from having N different metric container typed maps for each bundleID
and PtransformID segregated storage.
I feel it's a reasonable use of a private interface, since it's purely for
the convenience of this package specifically, and since 3 distinct counter
containers needed a way to convert to the proto type for exporting.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 84640)
Time Spent: 5.5h (was: 5h 20m)
> Fn API metrics in Go SDK harness
> --------------------------------
>
> Key: BEAM-3545
> URL: https://issues.apache.org/jira/browse/BEAM-3545
> Project: Beam
> Issue Type: Sub-task
> Components: sdk-go
> Reporter: Kenneth Knowles
> Assignee: Robert Burke
> Priority: Major
> Labels: portability
> Time Spent: 5.5h
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
