I think Google Groups somehow managed to eat a *very* long and detailed reply here, possibly as a result of a message deletion somewhere (or it somehow got kicked into the spam filter) *sigh* so I'm going to give just the TLDR of it. Something on this has been on my want-to-have list for a long time though, and was planned as a future follow-on to the Pipeline Graph Analysis API Project (AKA Bismuth) that I did.
This makes a lot of sense to me in general. We'd want to make sure these new metrics take advantage of the existing Dropwizard Metrics APIs - it should be pretty trivial to create a plugin wrapping https://github.com/ReadyTalk/metrics-statsd so that it spits those metrics and all of our existing goodies out. This also lets all the existing metrics integrations take advantage. I have existing implementations for some of these in an un-hosted plugin that's used as a utility for my Pipeline scalability lab. One thing to know is that it's much trickier to track a *step* execution in Pipeline vs. looking at the FlowNode(s) it creates - generally I think it's "good enough" to do per-Flownode metrics and use the StepDescriptor.getFunctionName for the node to determine what Pipeline step created it. I'd like to break down granularly by step for each Pipeline: how many FlowNodes were of that step type, how long each took to run on average, and how long that step took (wall time) overall. This lets us see regressions per step type, where averages over all steps in a pipeline are not very useful. It also tells us which steps are used a lot and which aren't -- useful for prioritizing. Similarly to Liam's comment on cyclomatic complexity, it would be helpful to know how many GroovyCPS elements are created within the Pipeline (similar to the improvement I did to put an effective Stack Depth limit in) -- this helps show real runtime complexity to users. On Friday, March 16, 2018 at 4:28:29 PM UTC-4, R Tyler Croy wrote: > > The successful adoption and iterative improvement of Jenkins Essentials > [0] is > heavily contingent on a spectrum of automated feedback which I've been > referring to as "telemetry" in many of the design documents. I wanted to > start > discussing the prospect of collecting anonymized Pipeline usage telemetry > to > help Jenkins Essentials, Blue Ocean, and Pipeline teams understand how > users > are actually using Jenkins Pipeline. > > James Dumay has already prototyped some similar work[1] for collecting > behavioral telemetry in Blue Ocean, but what I'm proposing would be much > more > broad in scope[2]. The metrics I am interested in, to help us understand > how > Pipeline is being used, are: > > * Counts: > * configured Declarative Pipelines > * configured Script Pipelines > * Pipeline executions > * distinct built-in step invocations (i.e. not counting Global Variable > invocations) > * Global Shared Pipelines configured > * Folder-level Shared Pipelines configured > * Agents used per-Pipeline > * Post-directive breakdown (stable,unstable,changed,etc) > > * Timers > * Runtime duration per step invocation > * Runtime duration per Pipeline > > > I believe this is a sufficiently useful set of metrics to send along, but > the > two questions I could use help answering are: > > * What are other metrics which would positively impact the development of > Jenkins? > * Are there concerns about implementation feasibility for any of these? > > > I am planning on using statsd (sent to the project's Datadog account), so > sampling and controlling the volume of individual metrics is not something > I'm > terribly worried about. > > > > Happy to hear any feedback y'all are willing to share! > > > [0] https://github.com/jenkins-infra/evergreen#evergreen > [1] https://github.com/jenkinsci/blueocean-plugin/pull/1653 > [2] https://issues.jenkins-ci.org/browse/JENKINS-49852 > > > > Cheers > - R. Tyler Croy > > ------------------------------------------------------ > Code: <https://github.com/rtyler> > Chatter: <https://twitter.com/agentdero> > xmpp: [email protected] <javascript:> > > % gpg --keyserver keys.gnupg.net --recv-key 1426C7DC3F51E16F > ------------------------------------------------------ > -- You received this message because you are subscribed to the Google Groups "Jenkins Developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/9659500e-1aa6-4e37-a342-9a4708a79bbf%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
