On Tue, Dec 27, 2016 at 10:56 PM 陈竞 <[email protected]> wrote: > is there any process or plan about metrics? i saw metrics package in > beam's sdk, however the api is experimental, which means it may be removed > in a chance, Besides, beam's metrics only support counter and distribution, > which is not enough for streaming pipeline, streaming pipeline also needs > meter. >
https://issues.apache.org/jira/browse/BEAM-773 is tracking the Flink implementation. I'm not aware of anyone currently working on it. If you're interseted (you mentioned on the bug you might be) it may be worth starting a thread on [email protected] to discuss the implementation. As far as implementing more of the API from the document (including Meters) it would help if you could elaborate on your use case a bit. Our current thoughts are that: (1) for all kinds of pipelines, it would be useful if counters (and other metrics) could be viewed over some recent time period (eg., last 1 minute or last 5 minutes). This helps to understand both what a pipeline is currently doing (regardless of whether it is Batch or Streaming) (2) for all kinds of pipelines, a Meter is useful to measure rate of occurrences. This could be the rate of processing within a Batch or Streaming pipeline, or the rate that input elements are being received from a source or written to a sink.
