Hi Enrico, As mentionned I think 5 would be an error, we should provide an in memory flavor with a dump at exist (-Dmaven.metrics.dumpOnExit=/tmp/metrics.csv or so?) otherwise it will be a noop for user so not sure it is worth having it at all. In other words, if you want it as a vendor you put a javaagent on the JVM and you are done with the added value of getting metrics. I don't mean I think we must not have it, I strongly think we must, but I also think it is only worth if usable directly, even if limited to in memory case.
Romain Manni-Bucau @rmannibucau <https://twitter.com/rmannibucau> | Blog <https://rmannibucau.metawerx.net/> | Old Blog <http://rmannibucau.wordpress.com> | Github <https://github.com/rmannibucau> | LinkedIn <https://www.linkedin.com/in/rmannibucau> | Book <https://www.packtpub.com/application-development/java-ee-8-high-performance> Le dim. 31 mai 2020 à 22:31, Enrico Olivelli <eolive...@gmail.com> a écrit : > Hey guys, > I feel this Metrics Runtime API will be very useful to Maven and Maven > plugins. > But I am not sure we are reaching consensus. > > Main points: > 1) We will introduce a Metrics Provider API, maintained by Maven project. > 2) Metrics are not "extension points", they are not "events", > implementations of the Provider API cannot alter the behaviour of Maven > 3) Metrics are just "values in time" (counters, gauges, summaries) , > usually exposed as time series, they are useful to inspect the runtime > low-level performances of internals of Maven core, internal libraries and > plugins > 4) The implementation of the API (the code that publishes and/or analyse > data) is loaded using the Extensions mechanism, by default there will be a > noop implementation bundled with Maven distribution > 5) We (Maven project) won't provide initially any implementation other than > the NOOP, in order not to add burden to Maven project. I can maintain a few > implementations on a separate repository, no need to add a burden to main > Maven project. > 6) Plugins will be able to register metric values using the Metrics API, > the will look up the MetricSystem component with @Inject or with explicit > lookup. > > I am glad to move forward with this project if we agree on all of the > aspects. > I am open to discuss again about all of the points. > > Thanks to everyone who already joined the discussion, I hope we could ship > v1 of the API in Maven 3.7 or Maven 3.8, the sooner the better > > Enrico > > > > Il giorno lun 11 mag 2020 alle ore 08:02 Romain Manni-Bucau < > rmannibu...@gmail.com> ha scritto: > > > Hmm, it shouldn't be: > > > > metricsSystem > > .getMetricsContext() > > .getSummary( "resolvePluginDependency", "Time to resolve dependencies of > a > > plugin (ms)" ) > > .add( MetricsUtils.elapsedMillis( startResolve ) ); > > > > but > > > > counter.add(duration). > > > > Resolution of the counter can be either (just in terms of inputs, then > api > > can be fluent or not, it is a detail for me): > > > > Counter counter = metricSystem.get(new CounterSpec("my counter", "ms")); > > > > or, since any operation in maven has a scope (mojo, artifact resolution, > > ...): > > > > @Inject @MetricSpec("my-counter", "ms") Counter counter; > > > > Concretely metrics system should enable to resolve a counter from a few > > meta (at least name, likely also the unit) and counter is just a long > (then > > in terms of impl it is a LongAdder to be concurrency friendly I guess but > > it is a detail). > > We likely don't want to pay any other overhead otherwise I fully agree > > events are almost 1-1 in terms of feature but totally opposed in terms of > > design: > > > > 1. A counter is a unified view of data where data are pushed from > > contributors > > 2. Event are an heterogeneous set of data where consumers are > interpreting > > the value > > > > If you want to be iso you create a Counter event but then you have this > > overhead we should avoid IMO + you just recreated metric system with > > another API (likely slower due to the bus usage which requires a lot of > > caching and JIT to be iso in terms of perf but it is really after some > > dozen of thousands of executions). > > > > Romain Manni-Bucau > > @rmannibucau <https://twitter.com/rmannibucau> | Blog > > <https://rmannibucau.metawerx.net/> | Old Blog > > <http://rmannibucau.wordpress.com> | Github < > > https://github.com/rmannibucau> | > > LinkedIn <https://www.linkedin.com/in/rmannibucau> | Book > > < > > > https://www.packtpub.com/application-development/java-ee-8-high-performance > > > > > > > > > Le dim. 10 mai 2020 à 22:23, Enrico Olivelli <eolive...@gmail.com> a > > écrit : > > > > > Il giorno dom 10 mag 2020 alle ore 22:10 Robert Scholte < > > > rfscho...@apache.org> ha scritto: > > > > > > > This looks more like a bug, so this should be analyzed. > > > > It sounds to me, that if this information was accurate, there > wouldn't > > be > > > > a need for the a separate MetricSystem. > > > > > > > > > > Sorry, I have added that metric just to have some data to retrieve and > > > monitor. > > > Maybe this is not interesting. > > > > > > The point is that if I want to track this duration, now I can do it > just > > by > > > adding a bunch of lines locally. > > > There is no need to introduce a new formal "event", that would be part > of > > > some public API. > > > If the information is not useful I can remove it in the next release, > > there > > > is no strong contract with the end user > > > > > > Enrico > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Robert > > > > On 10-5-2020 20:38:39, Romain Manni-Bucau <rmannibu...@gmail.com> > > wrote: > > > > Le dim. 10 mai 2020 à 20:11, Karl Heinz Marbaise a > > > > écrit : > > > > > > > > > > > > > > On 10.05.20 19:59, Romain Manni-Bucau wrote: > > > > > > Events vs metrics is an old topic. > > > > > > The choice between both is not only design but also about perf. > In > > > > terms > > > > > of > > > > > > design it brings the ability to export data without exposing it > in > > > code > > > > > > (events are public). It also avoid to expose a stable api of > events > > > and > > > > > > create coupling between plugin/exts and only require a single > > stable > > > > api > > > > > > which is very minimal (lot of projects kept their metric > > abstraction > > > > > > stable). > > > > > > > > > > > > Typically wagon should export metrics but if you fire an event > per > > > > > download > > > > > > progress it will be quite slow currently and you will allocate a > > lot > > > of > > > > > > objects (or create contentions) just to increase a monotonic > > counter > > > (a > > > > > > longadder concretely). > > > > > > > > > > > > So I think Enrico is right and we cant avoid metrics (I have to > > admit > > > > > > events are overcomplex to use for plugin/ext dev, see all > profiling > > > > > > extensions, > > > > > > > > > > > not a single one works and report accurate data whereas metrics > > > > > > can miss some drilldown data but would be right). > > > > > > > > > > Interesting point cause can you give some more details on which the > > > > > testimony is based that none of them works? > > > > > > > > > > > > > They all measure bus events, mainly mojo_start, mojo_end but at the > end > > > > several mojo have a duration of 0ms...even if they take minutes. > > > > > > > > What I saw last time I checked the two top rated by github (and > google > > I > > > > think), some events were missing (one ex where metrics are easier to > > > > handle, you push your data, no computation on consumer side). > > > > Out of my head in the project i was working on, gmaven plugin was > > totally > > > > missed. > > > > > > > > > > > > > Can you explain a accurate implementation? > > > > > > > > > > > > > If my build take 5mn and i sum all not overlapping times of the > report > > > of a > > > > monothreaded (T1) build then i should be close to 5mn (+- some sec > for > > > the > > > > boot, ioc, logging, final report). > > > > If I cant extract where time is spent these tools are uselesd. > > > > > > > > For the story i used a javaagent (based on sirona) to instrument the > > > build > > > > i was optimizing, was using in memory counters with dump at shutdown > > (in > > > > csv). Indeed bytecode instrumentation overhead was not low enough > but i > > > > managed to explain the build duration properly with such a tool. > > > > > > > > If we put these basic metrics in maven core (executor for ex) I'm > > pretty > > > > confident results will be less random than events interpretation and > > > > required code is very low and trivial to maintain for any dev so > guess > > it > > > > is worth it even if off by default (noop metrics system). > > > > > > > > > > > > > Kind regards > > > > > Karl Heinz Marbaise > > > > > > > > > > > > > > > > > > > > > > > > > > > Now we can limit the surfacing api and start only by counters and > > > > gauges > > > > > > maybe and later add histogram, meters etc only if needed (not > sure > > > for > > > > > mvn). > > > > > > > > > > > > Le dim. 10 mai 2020 à 19:32, Enrico Olivelli a > > > > > écrit : > > > > > > > > > > > >> Il Dom 10 Mag 2020, 17:49 Robert Scholte ha > > > > > >> scritto: > > > > > >> > > > > > >>> Hi Enrico, > > > > > >>> > > > > > >>> These could all be solved by firing events. > > > > > >>> > > > > > >> > > > > > >> Yes > > > > > >> I see your point. > > > > > >> Unfortunately those are the first points that I have > instrumented > > in > > > > > order > > > > > >> to see some data. > > > > > >> > > > > > >> > > > > > >>> And there's another benefit: since there won't be an API, we > > don't > > > > need > > > > > >> to > > > > > >>> maintain it,or be afraid of mistakes. > > > > > >>> > > > > > >> > > > > > >> Exposed metrics won't be an API, or at least something to be > > > > maintained. > > > > > >> Each Maven version, component version, plugin version may expose > > any > > > > > metric > > > > > >> is valuable for the component developer or for the users. > > > > > >> > > > > > >> Exposed metrics are not an extension point, it is not expected > > that > > > a > > > > > >> MetricsProvider is able to alter the behaviour of Maven. > > > > > >> Events/Extension point must be maintained forever, but you can > > > > drop/add > > > > > a > > > > > >> metric at each release. They are very low level tracking points > in > > > > code. > > > > > >> > > > > > >> This is because I say that metrics are like loggers: you add > logs > > > > where > > > > > you > > > > > >> think it is useful, you can change log point at every version, > > there > > > > is > > > > > not > > > > > >> strong contract with the user, logging implementation is totally > > > > > pluggable. > > > > > >> > > > > > >> The Metrics API is mostly the contract with Metrics Provider > > > > developers. > > > > > >> It will be very stable, I don't think there will be many changes > > in > > > > the > > > > > mid > > > > > >> term. > > > > > >> It is only about the ability to give a name to counters, > summaries > > > and > > > > > >> gauges. > > > > > >> > > > > > >> Having a simple implementation that dumps the final values at > the > > > end > > > > is > > > > > >> not the full power of this mechanism. > > > > > >> > > > > > >> You can attach Prometheus or any other time series based metrics > > > > > database > > > > > >> and see the evolution of the build, this will be very useful for > > > long > > > > > >> running builds and especially on CI or during component > > development. > > > > > >> > > > > > >> I will finish the implementation with Prometheus and try to > > create a > > > > > >> Grafana dashboard. > > > > > >> > > > > > >> It would be super useful that some Maven-internals experts could > > > > suggest > > > > > >> useful values to track during the execution of the build. > > > > > >> > > > > > >> I hope that explains better the value of the Metrics system. > > > > > >> > > > > > >> Thank you Robert for your feedback and help > > > > > >> > > > > > >> Enrico > > > > > >> > > > > > >> > > > > > >> To me this is the wrong approach to embed metrics. > > > > > >>> > > > > > 7 > > > > > > > > > > > > > > >
