Hello. Thank you for sharing your issue. Is your pipeline Bounded or Unbounded?
Thank you. On 2024/10/22 12:05:07 Kenneth Knowles wrote: > Thanks for reporting this. I know there's problems with the plumbing of > metrics in the FlinkRunner, but I don't know the full extent. I'm actually > starting to work on that right now. I filed > https://github.com/apache/beam/issues/32895 with your report and I'll add > others as I gather them up. > > Kenn > > On Tue, Oct 22, 2024 at 12:43 AM Aditya Shirsath <ad...@gmail.com> > wrote: > > > Hi, > > > > I also tried to export all the metrics using flink configuration to the > > prometheus metric reporter. There I'm able to see only the metrics > > generated by Flink. > > > > On Tue, Oct 22, 2024 at 9:47 AM Ahmet Altay via user < us...@beam.apache.org> > > wrote: > > > >> Adding Jan & Ken as people with potential knowledge in this area. > >> > >> On Sun, Oct 20, 2024 at 10:43 PM Aditya Shirsath <ad...@gmail.com> > >> wrote: > >> > >>> Hello, > >>> > >>> I'm creating a beam pipeline to run on flink runner. I've created > >>> custom metrics in beam(JAVA SDK). But I'm unable to see it in flink > >>> dashboard or flink rest API response. Although I'm able to see under > >>> Accumulators section but not under metrics. Under accumulators I can see, > >>> > >>> { "metrics": { "attempted": [{ "urn": "beam:metric:user:sum_int64:v1", > >>> "type": "beam:metrics:sum_int64:v1", "payload": "FA==", "labels": { > >>> "NAMESPACE": "Bingo", "NAME": "bingotest", "PTRANSFORM": > >>> "log/ParMultiDo(LogOutput)" } }] } } > >>> > >>> I'm also able to log the metrics in beam part. > >>> I'm using Apache Beam 2.55.0 and Flink runner 1.16. > >>> > >>> Please let me know if this is the correct way to do this or am I missing > >>> something? > >>> > >>> Do I need to create a separate HTTP sink for this as mentioned here > >>> <https://beam.apache.org/documentation/programming-guide/#metrics> ?. > >>> But I couldn't find any other documentation to export data. Could you > >>> please help me here? > >>> > >>> Thank you! > >>> > >> > -- LEE SOMANG R&D Department Backend Development Team The Chosunilbo, 30, Sejong-daero 21-gil, Jung-gu, Seoul, Korea (zip code: 04519) M : +82 10-8940-5081 E : h...@chosunbiz.com
