Hi everyone, Thank you very much for the prompt responses and help. I was able to get the MetricFetcher connected with my thread and am successfully able to collect metrics needed for my research and experiments.
Thank you, Atul On Thu, Mar 9, 2023 at 7:53 PM Atul Lal <atul.lal...@gmail.com> wrote: > Hi everyone, > > I also think the reporter won't work for my use case, it could work but > then I am trying to modify the placement of operators on taskmanagers > manually based on my some decisions made from metric collection, reporter > makes it harder to do so. Right now, I have REST APIs, but I was thinking > if in some way I could create a new MetricFetcher object and use that for > collecting metrics instead of REST APIs. > > Is it possible to use MetricFetcher? > > Thanks a lot for the help, > Atul > > On Thu, Mar 9, 2023 at 7:39 PM Mason Chen <mas.chen6...@gmail.com> wrote: > >> Hi all, >> >> Metric reporter may be useful if you only need per component level metrics >> like jobmanager and each taskmanager since the metric reporter runs in >> each >> component. However, for an aggregated job level view of metrics, there is >> no better out-of-the-box/user-facing way to get metrics than the REST API. >> A good example in using the REST API is the k8s operator autoscaler which >> needs to scrape metrics: >> >> https://github.com/apache/flink-kubernetes-operator/tree/main/flink-kubernetes-operator-autoscaler/src/main/java/org/apache/flink/kubernetes/operator/autoscaler >> . >> >> Best, >> Mason >> >> On Thu, Mar 9, 2023 at 4:34 AM Hang Ruan <ruanhang1...@gmail.com> wrote: >> >> > Hi, Atul, >> > >> > I think the metric reporter[1] will be helpful for you. >> > >> > Best, >> > Hang >> > >> > [1] >> > >> > >> https://nightlies.apache.org/flink/flink-docs-release-1.16/docs/deployment/metric_reporters/ >> > >> > Atul Lal <atul.lal...@gmail.com> 于2023年3月9日周四 17:49写道： >> > >> > > Hi everyone, >> > > >> > > I am trying to do some experiments with Flink. I am trying to modify >> the >> > > source code of Flink for this experiment, and I am starting a process >> > > thread from JobMaster.java constructor whenever a new job is started >> in >> > > Flink. In this thread, I want to monitor a few metrics related to the >> job >> > > and make some decisions based on it. >> > > >> > > Is there any way to collect metrics related to a job without using >> REST >> > > endpoints? Because I think using REST endpoints here is pointless as >> this >> > > is internal code running on JobMaster. If there is no other way than >> > using >> > > REST endpoints, is there any easy way to serialize or parse JSON >> > responses >> > > from those endpoints as the response structures are already defined in >> > the >> > > classes. >> > > >> > > I would really appreciate it if someone could help me with this. >> > > >> > > Thank you, >> > > Atul >> > > >> > >> >
