Re: [DISCUSS] Standard / Convention for common connector metrics

2019-02-11 Thread Becket Qin
Thanks for the feedback, Shaoxuan. I'll create a FLIP then. Good point about documentation. I'll put together a doc on the website once we agreed upon the detail metric specifications. Jiangjie (Becket) Qin On Mon, Feb 11, 2019 at 4:10 PM Shaoxuan Wang wrote: > Becket, > > Thanks for the

Re: [DISCUSS] Standard / Convention for common connector metrics

2019-02-11 Thread Shaoxuan Wang
Becket, Thanks for the proposal. I am much agree that we should only design the standard/common Flink-related connector metrics, while keep the original names for the ones provided by the connector itself. If everyone agrees to make these changes, we should do it asap, the earlier to make the

Re: [DISCUSS] Standard / Convention for common connector metrics

2019-02-01 Thread Becket Qin
Thanks for the feedback, Konstantin, Completely agree that exposing the original metrics from the clients of other systems is a good practice and should be encouraged. Thanks, Jiangjie (Becket) Qin On Fri, Feb 1, 2019 at 4:01 PM Konstantin Knauf wrote: > Hi Becket, > > I like the idea of

Re: [DISCUSS] Standard / Convention for common connector metrics

2019-02-01 Thread Konstantin Knauf
Hi Becket, I like the idea of providing a standard set of metrics, which sources can choose to implement/expose. In addition, I think, sources, like the Kafka or Kinesis Source, should continue to forward the original consumer metrics under their original names, so that users familiar with

Re: [DISCUSS] Standard / Convention for common connector metrics

2019-01-31 Thread Becket Qin
Thanks for the connector metric url, Chesnay :) @Robert, as you can see, the metrics from different connectors are quite different. And there are different names for similar metrics, which is a little frustrating when users want to do monitoring / alerting. Thanks, Jiangjie (Becket) Qin On

Re: [DISCUSS] Standard / Convention for common connector metrics

2019-01-31 Thread Chesnay Schepler
@Robert: https://ci.apache.org/projects/flink/flink-docs-master/monitoring/metrics.html#connectors On 31.01.2019 11:03, Robert Metzger wrote: Hey Becket, thanks a lot for your proposal! Do you have an overview over the current situation of the metrics in the connectors? Which connectors

Re: [DISCUSS] Standard / Convention for common connector metrics

2019-01-31 Thread Robert Metzger
Hey Becket, thanks a lot for your proposal! Do you have an overview over the current situation of the metrics in the connectors? Which connectors expose metrics at all? Are they different? On Thu, Jan 31, 2019 at 8:44 AM Becket Qin wrote: > Hi folks, > > I was trying to add some metrics to

[DISCUSS] Standard / Convention for common connector metrics

2019-01-30 Thread Becket Qin
Hi folks, I was trying to add some metrics to Kafka connectors and realized that right now Flink does not have a common metric definition for the connectors. This complicates the monitoring and operation because the monitoring / alerts need to be set case by case. To address this issue, I would