One library that might be worth investigating is Micrometer Core. It's an Apache library that provides vendor neutral API over various instrumentation clients. This could be a great tool to add as a Metrics Sink. https://micrometer.io/docs
But I think we still should update the Heron metrics API to match this functionality. Heron currently provides the metrics aggregation from the various bolts/spouts into a central location. I think this mechanism is still important. But once it's aggregated we can leverage Heron's pluggable metrics sink capability to export to whatever metrics provider is desired. But I think we should update Heron to support the concept of labels so that the information can make it from Spout/Bolt to the Metrics Manager. So this might entail an update to the Heron API and also the Metrics Manager itself. But I agree with Ning, the existing built in Heron metrics logic will complicate adding the feature. On Mon, Dec 6, 2021 at 8:25 AM Nicholas Nezis <[email protected]> wrote: > I don't think we should jump to an external solution. We should add > support in our API, out at least extend with an optional set of additional > calls. > > We should look at Flink's API and implementation as reference. Also we > should look at Prometheus. > > In both of these, you usually define the keys when creating the counter, > and then provide the values when submitting the new value (or increment > call). > > I'll share some links soon. > > > > On Mon, Dec 6, 2021, 4:45 AM Ning Wang <[email protected]> wrote: > >> Hello, >> >> Nick and I have been talking about supporting tags in the Heron metrics >> library. I was thinking about something like DataDog so you just need to >> provide a list of tags in the `incr()` functions. However, one tricky >> thing >> about Heron's library is that aggregation happens in the library. >> >> I am curious if anyone has used DataDog (or StatsD) with Heron and how do >> you hook it up. For DataDog, I think the aggregation happens in the agent >> instead of the library. >> >> Thanks in advance. >> --ning >> >
