I totally agree with you, meter is useful in streaming pipeline, as well as counter, histogram. It measure rate of occurrences to let us know the input qps , and also help us know the pipeline's current state.
we usually use codahale's metrics in storm pipeline which is useful for us monitoring streaming pipeline 2016-12-30 2:26 GMT+08:00 Ben Chambers <[email protected]>: > On Tue, Dec 27, 2016 at 10:56 PM 陈竞 <[email protected]> wrote: > >> is there any process or plan about metrics? i saw metrics package in >> beam's sdk, however the api is experimental, which means it may be removed >> in a chance, Besides, beam's metrics only support counter and distribution, >> which is not enough for streaming pipeline, streaming pipeline also needs >> meter. >> > > https://issues.apache.org/jira/browse/BEAM-773 is tracking the Flink > implementation. I'm not aware of anyone currently working on it. If you're > interseted (you mentioned on the bug you might be) it may be worth starting > a thread on [email protected] to discuss the implementation. > > As far as implementing more of the API from the document (including > Meters) it would help if you could elaborate on your use case a bit. Our > current thoughts are that: > > (1) for all kinds of pipelines, it would be useful if counters (and other > metrics) could be viewed over some recent time period (eg., last 1 minute > or last 5 minutes). This helps to understand both what a pipeline is > currently doing (regardless of whether it is Batch or Streaming) > > (2) for all kinds of pipelines, a Meter is useful to measure rate of > occurrences. This could be the rate of processing within a Batch or > Streaming pipeline, or the rate that input elements are being received from > a source or written to a sink. > -- 陈竞，中科院计算技术研究所，高性能计算机中心 Jing Chen HPCC.ICT.AC China
