OK. Thanks for the input. I will think start from the library first to make sure the API is consistent.
On Mon, Dec 6, 2021 at 7:25 PM Nicholas Nezis <[email protected]> wrote: > One library that might be worth investigating is Micrometer Core. It's an > Apache library that provides vendor neutral API over various > instrumentation clients. This could be a great tool to add as a Metrics > Sink. https://micrometer.io/docs > > But I think we still should update the Heron metrics API to match this > functionality. Heron currently provides the metrics aggregation from the > various bolts/spouts into a central location. I think this mechanism is > still important. But once it's aggregated we can leverage Heron's pluggable > metrics sink capability to export to whatever metrics provider is desired. > > But I think we should update Heron to support the concept of labels so that > the information can make it from Spout/Bolt to the Metrics Manager. So this > might entail an update to the Heron API and also the Metrics Manager > itself. But I agree with Ning, the existing built in Heron metrics logic > will complicate adding the feature. > > > On Mon, Dec 6, 2021 at 8:25 AM Nicholas Nezis <[email protected]> > wrote: > > > I don't think we should jump to an external solution. We should add > > support in our API, out at least extend with an optional set of > additional > > calls. > > > > We should look at Flink's API and implementation as reference. Also we > > should look at Prometheus. > > > > In both of these, you usually define the keys when creating the counter, > > and then provide the values when submitting the new value (or increment > > call). > > > > I'll share some links soon. > > > > > > > > On Mon, Dec 6, 2021, 4:45 AM Ning Wang <[email protected]> wrote: > > > >> Hello, > >> > >> Nick and I have been talking about supporting tags in the Heron metrics > >> library. I was thinking about something like DataDog so you just need to > >> provide a list of tags in the `incr()` functions. However, one tricky > >> thing > >> about Heron's library is that aggregation happens in the library. > >> > >> I am curious if anyone has used DataDog (or StatsD) with Heron and how > do > >> you hook it up. For DataDog, I think the aggregation happens in the > agent > >> instead of the library. > >> > >> Thanks in advance. > >> --ning > >> > > >
