Following up here: A few PRs have gone in related to this proposal (see BEAM-4775 <https://issues.apache.org/jira/browse/BEAM-4775>), and Robert, Alex, Mikhail, and I are discussing some questions that have come up along the way.
I wrote up the state of things in this doc <https://docs.google.com/document/d/1m83TsFvJbOlcLfXVXprQm1B7vUakhbLZMzuRrOHWnTg/edit#heading=h.faqan9rjc6dm>. It describes PRs that are in-flight as well as some further downstream questions: - Should metrics' {namespace,name} be stored in the URN, or as MonitoringInfo "labels"? - Should "attempted"/"committed" be surfaced as proto fields, or also made into a MonitoringInfo label? - Also, is there a discrepancy in what those terms mean, between older "physical"/"logical" semantics and new FN-API "bundle progress" / "bundle result" ones? It's not really a design doc at this stage, but I wanted to post it in case others are interested to follow or discuss. If we get agreement on these questions, we can update the previous design doc of record covering this work (https://s.apache.org/get-metrics-api). Thanks, -Ryan On Mon, Feb 4, 2019 at 2:26 PM Robert Bradshaw <[email protected]> wrote: > To summarize for the list, the plan of record is: > > The MonitoringInfo proto will be used again in this querying API, so the > metric format SDKs report will also be used when extracting metrics for a > job. > > // Job Service for running RunnerAPI pipelines > service JobService { > ... > rpc GetJobMetrics (GetJobMetricsRequest) returns (GetJobMetricsResponse); > } > > message GetJobMetricsRequest { > string job_id = 1; // (required) > } > > message GetJobMetricsResponse { > // (Optional) The aggregated value of the metric based on the in > flight work. > // SDKs optionally report these metrics in the > ProcessBundleProgressResponse. > MonitoringInfo attempted_metric_result = 1; > // (Required) The aggregated value of the metric based on the > completed work. > // SDKs report these metrics in the ProcessBundleResponse. > MonitoringInfo committed_metric_result = 2; > } > The new rpc in beam_job_metrics.proto. > > SDKs will continue to implement filtering of Metrics, by providing their > own language specific convenience functions to filter and obtain metrics. > In Java for example, the MetricResult and MetricFilter format interfaces > will continue to exist as interface to filter metrics for a specific step, > user namespace, username, etc. > > > Looking forward to seeing this happen. > > > On Mon, Feb 4, 2019 at 8:17 PM Alex Amato <[email protected]> wrote: > >> Done,it's on the website now. >> >> Ryan and I will move forward with the plan in this plan. If there are any >> major objections to this plan, please let us know by weds. Suggestions will >> be welcome later as well, as we are happy to iterate on this. But we will >> be proceeding with some of Ryan's PRs based on this design. >> >> On Thu, Jan 31, 2019 at 12:54 PM Ismaël Mejía <[email protected]> wrote: >> >>> Please don't forget to add this document to the design documents webpage. >>> >>> On Thu, Jan 31, 2019 at 8:46 PM Alex Amato <[email protected]> wrote: >>> > >>> > Hello Beam, >>> > >>> > Robert Ryan and I have been designing a metric extraction API for >>> Beam. Please take a look at this design, I would love to get more feedback >>> on this to improve the design. >>> > >>> > https://s.apache.org/get-metrics-api >>> > >>> > The primary goal of this proposal is to offer a simple way to obtain >>> all the metrics for a job. The following issues are addressed: >>> > >>> > The current design requires implementing metric querying for every >>> runner+language combination. >>> > >>> > Duplication of MetricResult related classes in each language. >>> > >>> > The existing MetricResult format only allows querying metrics defined >>> by a namespace, name and step, and does not allow generalized labelling as >>> used by MonitoringInfos. >>> > >>> > Enhance Beam’s ability to integration test new metrics >>> > >>> > >>> > Thank for taking a look, >>> > Alex >>> >>
