It would be useful to provide a reference implementation for consuming this data. Can you also capture the goals in an issue (I scanned the PR quickly but there’s no corresponding issue).
Further now I think there are multiple ways of recording/reporting some of the metrics (log markers which are largely silenced by default, kamon metrics, and now kafka). Is that right? I think we’ll need to also document these and a guide for when to use which - I caution that we are proliferating multiple ways of doing similar things with no consistency or articulated long term vision. -r > On Apr 17, 2018, at 12:01 PM, Vadim Raskin <[email protected]> wrote: > > Hi Chetan, > >> Can you share some details on how this is currently being done with > CouchDB. Do we have any analytics view configured which computes these > numbers currently? > > To my knowledge we don't have the views that are shared anywhere in open > repos. > >> May be we also include a basic default implementation out of the box > which collect aggregated stats using Kamon metrics already being used > > Not a bad idea, I'll consider sharing the peace of code after finishing the > development (separate from the original PR), it might require some > post-processing to strip away the ibm specific parts, so it might bide some > time. > > regards, > Vadim. > > > > On Tue, Apr 17, 2018 at 8:29 AM Chetan Mehrotra <[email protected]> > wrote: > >> Hi Vadim, >> >> This looks helpful to get better insight in runtime operational stats! >> >>> It has some advantages over to the prior approach (sending them to >> CouchDB). >> >> Can you share some details on how this is currently being done with >> CouchDB. Do we have any analytics view configured which computes these >> numbers currently? >> >>> Now it would be possible to simply connect a custom micro service >> to Kafka and consume the activations in real-time. >> >> May be we also include a basic default implementation out of the box >> which collect aggregated stats using Kamon metrics already being used >> Chetan Mehrotra >> >> >> On Mon, Apr 16, 2018 at 9:51 PM, Vadim Raskin <[email protected]> >> wrote: >>> Hi everyone, >>> >>> >>> I’ve just opened a PR that enables sending activation metadata to Kafka. >>> It has some advantages over to the prior approach (sending them to >>> CouchDB). Now it would be possible to simply connect a custom micro >> service >>> to Kafka and consume the activations in real-time. Some of the use cases >> it >>> might cover: activation metrics - collect the data and push them into a >>> custom time-series database; user activity audit; activation analytics - >>> potentially get some insights with KSQL. >>> >>> >>> At the moment I’ve created a new kafka topic called events, which will >>> include messages from Controllers and Invokers. It encompasses the >>> following data collected from a single activation: >>> >>> >>> concurrentActivations >>> throttledActivations >>> statusCode >>> initTime >>> waitTime >>> duration >>> kind >>> >>> >>> Probably some more metadata will affiliate this list soon. >>> >>> >>> Just wanted to give a short heads up here. The PR I mentioned: >>> https://github.com/apache/incubator-openwhisk/pull/3552 >>> >>> >>> Thank you, >>> >>> >>> Vadim. >>
