Hello Eswar,

as far as I'm aware the general structure of the Flink's metric system is rather similar to DropWizard. You can use DropWizard metrics by creating a simple wrapper, we even ship one for Histograms. Furthermore, you can also use DropWizard reporters, you only have to extend the DropWizardReporter class, essentially providing a factory method for your reporter.


Using Flinks infrastructure provides the following benefits:
* better resource usage, as only a single reporter instance per taskmanager exists
* access to system metrics
* namespace stuff; you cannot access all variables yourselves from a UDF without modifying the source of Flink; whether this is an advantage is of course dependent on what you are interested in

Regards,
Chesnay

On 20.09.2016 08:29, Eswar Reddy wrote:
Hi,

I see Flink support's built-in metrics to monitor various components of Flink. In addition, one can register application specific(custom) metrics to Flink's built-in metrics infra. The problem with this is user has to develop his custom metrics using Flink's metrics framework/API rather than a generic framework such as dropwizard. Alternatively, user can follow this <http://www.michael-noll.com/blog/2013/11/06/sending-metrics-from-storm-to-graphite/#high-level-approach> approach where his dropwizard metrics push code is co-located with actual app code within each Task and metrics are directly pushed to a backend writer(say, Graphite) from each Task.

In this alternative, I am aware of having to handle mapping spatial granularity of Flink's run-time with metrics namespace, but doing it myself should not a big effort. Fault-tolerance comes automatically since app code and metrics push code are co-located in the Task. Is there anything else Flink's metrics infra handles automatically? Based on this I'd weigh using good old dropwizard vs Flink specific metrics framework.

Finally, I guess feasibility an automatic dropwizard-to-flinkmetrics translation utility can be checked out, but I would like to first understand additional benefits of using flink's infra for custom metrics.

Thanks,
Eswar.

Reply via email to