Re: Flink metric collection

2023-03-12 Thread Atul Lal
Hi everyone, Thank you very much for the prompt responses and help. I was able to get the MetricFetcher connected with my thread and am successfully able to collect metrics needed for my research and experiments. Thank you, Atul On Thu, Mar 9, 2023 at 7:53 PM Atul Lal wrote: > Hi everyone, > >

Re: Flink metric collection

2023-03-09 Thread Atul Lal
Hi everyone, I also think the reporter won't work for my use case, it could work but then I am trying to modify the placement of operators on taskmanagers manually based on my some decisions made from metric collection, reporter makes it harder to do so. Right now, I have REST APIs, but I was thin

Re: Flink metric collection

2023-03-09 Thread Mason Chen
Hi all, Metric reporter may be useful if you only need per component level metrics like jobmanager and each taskmanager since the metric reporter runs in each component. However, for an aggregated job level view of metrics, there is no better out-of-the-box/user-facing way to get metrics than the

Re: Flink metric collection

2023-03-09 Thread Hang Ruan
Hi, Atul, I think the metric reporter[1] will be helpful for you. Best, Hang [1] https://nightlies.apache.org/flink/flink-docs-release-1.16/docs/deployment/metric_reporters/ Atul Lal 于2023年3月9日周四 17:49写道: > Hi everyone, > > I am trying to do some experiments with Flink. I am trying to modify

Flink metric collection

2023-03-09 Thread Atul Lal
Hi everyone, I am trying to do some experiments with Flink. I am trying to modify the source code of Flink for this experiment, and I am starting a process thread from JobMaster.java constructor whenever a new job is started in Flink. In this thread, I want to monitor a few metrics related to the