Becket, Thanks for the proposal. I am much agree that we should only design the standard/common Flink-related connector metrics, while keep the original names for the ones provided by the connector itself. If everyone agrees to make these changes, we should do it asap, the earlier to make the change the less impact on the user's system. Maybe we can start to unify the metrics on the existing connectors, and then slowly introduce the new common/standard metrics if necessary.
Since metric is also a kind of user interface, we should either open a Flip, or create a webpage to document the specifications, it's up to you. The point is that we want to create a good document, such that the future developers can easily find and follow these specifications. Regards, Shaoxuan On Fri, Feb 1, 2019 at 5:15 PM Becket Qin <[email protected]> wrote: > Thanks for the feedback, Konstantin, > > Completely agree that exposing the original metrics from the clients of > other systems is a good practice and should be encouraged. > > Thanks, > > Jiangjie (Becket) Qin > > On Fri, Feb 1, 2019 at 4:01 PM Konstantin Knauf < > [email protected]> > wrote: > > > Hi Becket, > > > > I like the idea of providing a standard set of metrics, which sources can > > choose to implement/expose. In addition, I think, sources, like the Kafka > > or Kinesis Source, should continue to forward the original consumer > metrics > > under their original names, so that users familiar with Kafka/Kinesis can > > relate to them easily. > > > > Cheers, > > > > Konstantin > > > > > > > > On Fri, Feb 1, 2019 at 3:54 AM Becket Qin <[email protected]> wrote: > > > > > Thanks for the connector metric url, Chesnay :) > > > > > > @Robert, as you can see, the metrics from different connectors are > quite > > > different. And there are different names for similar metrics, which is > a > > > little frustrating when users want to do monitoring / alerting. > > > > > > Thanks, > > > > > > Jiangjie (Becket) Qin > > > > > > On Thu, Jan 31, 2019 at 6:41 PM Chesnay Schepler <[email protected]> > > > wrote: > > > > > > > @Robert: > > > > > > > > > > > > > > https://ci.apache.org/projects/flink/flink-docs-master/monitoring/metrics.html#connectors > > > > > > > > On 31.01.2019 11:03, Robert Metzger wrote: > > > > > Hey Becket, > > > > > thanks a lot for your proposal! > > > > > > > > > > Do you have an overview over the current situation of the metrics > in > > > the > > > > > connectors? > > > > > Which connectors expose metrics at all? > > > > > Are they different? > > > > > > > > > > On Thu, Jan 31, 2019 at 8:44 AM Becket Qin <[email protected]> > > > wrote: > > > > > > > > > >> Hi folks, > > > > >> > > > > >> I was trying to add some metrics to Kafka connectors and realized > > that > > > > >> right now Flink does not have a common metric definition for the > > > > >> connectors. This complicates the monitoring and operation because > > the > > > > >> monitoring / alerts need to be set case by case. > > > > >> > > > > >> To address this issue, I would like to see if is possible to have > a > > > set > > > > of > > > > >> standardized common metrics for all sources and sinks. The > following > > > doc > > > > >> describes the proposal. Feedback is very welcome. > > > > >> > > > > >> > > > > >> > > > > > > > > > > https://docs.google.com/document/d/1q86bgj_3T6WFbSUoxLDJJXmUcBOUcvWfh2RZvHG-nPU/edit# > > > > >> > > > > >> Thanks, > > > > >> > > > > >> Jiangjie (Becket) Qin > > > > >> > > > > > > > > > > > > > > > > > -- > > -- > > > > Konstantin Knauf | Solution Architect > > > > > > -- > > > > Join Flink Forward <https://flink-forward.org/> - The Apache Flink > > Conference > > > > Stream Processing | Event Driven | Real Time > > > > -- > > > > Data Artisans GmbH | Stresemannstr. 121A,10963 Berlin, Germany > > < > > > https://maps.google.com/?q=Stresemannstr.+121A,10963+Berlin,+Germany&entry=gmail&source=g > > > > > data Artisans, Inc. | 1161 Mission Street, San Francisco, CA-94103, USA > > < > > > https://maps.google.com/?q=1161+Mission+Street,+San+Francisco,+CA-94103,+USA&entry=gmail&source=g > > > > > > > -- > > Data Artisans GmbH > > Registered at Amtsgericht Charlottenburg: HRB 158244 B > > Managing Directors: Dr. Kostas Tzoumas, Dr. Stephan Ewen > > >
