Re: Client for Monitoring API!
Ah, now I understand what exactly your requirement is. I don't think there is such a tool in Flink which could help you to fetch and store the content of rest api. It seems not to be a general requirement. But I'm really interested in the motivation behind your requirement. Could you share more about it? Why do you want to keep the history of checkpointing at subtask level? How do you use this history? Thanks, Biao /'bɪ.aʊ/ On Thu, 19 Sep 2019 at 16:12, Anis Nasir wrote: > Thanks Biao for your response. > > We would like to fetch metrics at subtask level for each checkpoint. This > information is not exposed via default metrics, but are available in rest > end point! > > Also, would like to persist the history of checkpoints. This information > is lost whenever we restart the job (or job manager is lost). > > > Cheers, > Anis > > > > On Thu, Sep 19, 2019 at 5:02 AM Biao Liu wrote: > >> Hi Anis, >> >> Have you tried Flink metric reporter? It's a better way to handle metrics >> than through rest api. >> Flink supports reporting metrics to external system. You could find the >> list of external systems supported here [1]. >> >> 1. >> https://ci.apache.org/projects/flink/flink-docs-release-1.9/monitoring/metrics.html#reporter >> >> Thanks, >> Biao /'bɪ.aʊ/ >> >> >> >> On Wed, 18 Sep 2019 at 19:36, Anis Nasir wrote: >> >>> Hey all, >>> >>> Is there any client library that we can use to fetch/store the metrics >>> expose through flink monitoring rest api ? >>> >>> >>> Regards, >>> Anis >>> >>
Re: Client for Monitoring API!
Thanks Biao for your response. We would like to fetch metrics at subtask level for each checkpoint. This information is not exposed via default metrics, but are available in rest end point! Also, would like to persist the history of checkpoints. This information is lost whenever we restart the job (or job manager is lost). Cheers, Anis On Thu, Sep 19, 2019 at 5:02 AM Biao Liu wrote: > Hi Anis, > > Have you tried Flink metric reporter? It's a better way to handle metrics > than through rest api. > Flink supports reporting metrics to external system. You could find the > list of external systems supported here [1]. > > 1. > https://ci.apache.org/projects/flink/flink-docs-release-1.9/monitoring/metrics.html#reporter > > Thanks, > Biao /'bɪ.aʊ/ > > > > On Wed, 18 Sep 2019 at 19:36, Anis Nasir wrote: > >> Hey all, >> >> Is there any client library that we can use to fetch/store the metrics >> expose through flink monitoring rest api ? >> >> >> Regards, >> Anis >> >
Re: Client for Monitoring API!
Hi Anis, Have you tried Flink metric reporter? It's a better way to handle metrics than through rest api. Flink supports reporting metrics to external system. You could find the list of external systems supported here [1]. 1. https://ci.apache.org/projects/flink/flink-docs-release-1.9/monitoring/metrics.html#reporter Thanks, Biao /'bɪ.aʊ/ On Wed, 18 Sep 2019 at 19:36, Anis Nasir wrote: > Hey all, > > Is there any client library that we can use to fetch/store the metrics > expose through flink monitoring rest api ? > > > Regards, > Anis >
Re: Client for Monitoring API!
yes. you can use prometheus+Grafana. https://ci.apache.org/projects/flink/flink-docs-stable/monitoring/metrics.html#prometheus-orgapacheflinkmetricsprometheusprometheusreporter https://felipeogutierrez.blogspot.com/2019/04/monitoring-apache-flink-with-prometheus.html Felipe On 2019/09/18 11:36:37, Anis Nasir wrote: > Hey all,> > > Is there any client library that we can use to fetch/store the metrics> > expose through flink monitoring rest api ?> > > > Regards,> > Anis> >
Client for Monitoring API!
Hey all, Is there any client library that we can use to fetch/store the metrics expose through flink monitoring rest api ? Regards, Anis
