Hi meghajit, I think it makes sense to extend the current metrics. Could you list all metrics you need? Thanks!
Best regards, Jing On Fri, Jun 10, 2022 at 5:06 PM Lijie Wang <wangdachui9...@gmail.com> wrote: > Hi Meghajit, > > As far as I know, currently, the FileSource does not have the metrics you > need. You can implement your own source, and register custom metrics via > `SplitEnumeratorContext#metricGroup` and `SourceReaderContext#metricGroup`. > > Best, > Lijie > > Meghajit Mazumdar <meghajit.mazum...@gojek.com> 于2022年6月10日周五 16:36写道： > >> Hello, >> >> We are working on a Flink project which uses FileSource to discover and >> read Parquet Files from GCS. ( using Flink 1.14) >> >> As part of this, we wanted to implement some health metrics around the >> code. >> I wanted to know whether Flink gathers some metrics by itself around >> FileSource, e;g, number of files discovered by the SplitEnumerator, number >> of files added back to SplitAssigner, time taken to process per split, etc ? >> >> I checked in the official documentation >> <https://nightlies.apache.org/flink/flink-docs-release-1.14/docs/connectors/datastream/filesystem/> >> but there doesn't appear to be. Is the solution then to implement >> custom metrics like this >> <https://nightlies.apache.org/flink/flink-docs-release-1.14/docs/ops/metrics/> >> ? >> >> >> *Regards,* >> *Meghajit* >> >
