marcingrzejszczak commented on issue #20845: URL: https://github.com/apache/pulsar/issues/20845#issuecomment-1861027222
I found [it](https://github.com/apache/pulsar/blob/master/pip/pip-264.md#why-not-other-libraries)! Thanks for creating such an extensive document! I will address parts of it below. > In this epic scope we will only use the Metrics API and SDK. Specifically we will use the Java SDK in Pulsar Broker, Proxy and Function Worker, but also the Go and Python SDK for the wrapper code which executes functions written in Go and Python. With [Micrometer Observation](https://micrometer.io/docs/observation) you can use 1 Java API to instrument for whatever observability pillars (metrics, traces etc.). With OTel you will need to use as many APIs as necessary to achieve the same goal. > One very big breaking change (there are several described in the High Level Design, and also summarized in the Backward Compatibility section) is the naming. We are changing all metric names due to several reasons: > Attributes names (a.k.a. Labels) will utilize the Semantic Conventions defined in OTel. > OpenTelemetry defined an agreed upon attribute names for many attributes in the industry. The status of semantic conventions in OTel is still mixed or experimental for many of its parts. In Micrometer, through the Micrometer Observation Convention mechanism you can gradually migrate from certain naming conventions to other ones. You can do the same just for metrics (through `MeterFilter`), just for tracing (through `SpanFilter`) or for all signals (via `ObservationConvention` or `ObservationFilter`). > OTel doesn’t force you to supply documentation. > We will create a static code analysis rule failing the build if it finds instrument creation without description. > We will optionally try to somehow create an automated way to export all metrics metadata - instrument name, documentation, in an easy to read format to be used for documentation purposes on Pulsar website. [Micrometer Docs Generator](https://github.com/micrometer-metrics/micrometer-docs-generator) already provides this feature. You can have your code scanned and produce documentation out of it. [Example for Spring Cloud Task](https://docs.spring.io/spring-cloud-task/reference/observability.html). > It’s the new emerging industry-wide standard for observability and specific metrics, as opposed to just a library or a standard adopted and promoted by a single entity/company. I would be cautious about statements like that following the history of OpenTracing (e.g. [Couchbase](https://issues.couchbase.com/browse/JCBC-2046?focusedId=656281&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-656281) went all in on OT and now they created their own facade). With Micrometer, Micrometer Observation and Micrometer Tracing you have facades created in such a way that the user can define which “standard” they want to use and when there’s the “next standard” they will be able to migrate to it with minimal code changes. > Micrometer had the vision of becoming the industry standard API like SLF4J is for logging in the Java ecosystem. In reality, it didn’t catch on, as can be seen in the Maven Central statistics: It’s used by ~1000 artifacts, compared to sl4fj-api which is used by 60k artifacts—as such, picking it as the standard for today, seems like “betting” on the wrong project. It sounds a little bit like comparing incomparable libraries. You can’t compare a logging library to a metrics library. We were stating that we are like Slf4j but for observability. We never claimed that we will be Slf4j nor that we will reach their download statistics. I’m pretty sure that vast majority of applications are doing logging. Not all of the applications are gathering observability data. BTW if you compare the same statistics that you mentioned between micrometer-core library and e.g. opentelemetry-api then you’ll get around 1 800 for Micrometer Core (1.12.0) and 220 for opentelemetry-api (1.32.0). > Micrometer architecture relies heavily on the library to implement all target systems like Datadog, Prometheus, Graphite, OTLP, and more. OTel relies on the collector to implement that as it has more power and can contain the state if one of those systems goes down for some time. I think it’s a smarter choice, and more vendors will likely appear and maintain their exporter in OTel collector as we advance. This makes it easier for operators to have one exporter code base (say to Cortex) across different languages of micro-services, so it makes sense people will lean towards this framework and request it soon. Micrometer has an [OTLP MeterRegistry](https://micrometer.io/docs/registry/otlp) so it can export metrics in the OTLP format. You can check the code [here](https://github.com/micrometer-metrics/micrometer/blob/v1.12.1/implementations/micrometer-registry-otlp/src/main/java/io/micrometer/registry/otlp/OtlpMeterRegistry.java). > OTel was built with instrumentation scope in mind, which gives a sort of namespace per library or section of the code (Called Meter in the API). For Pulsar, it can be used to have one per plugin. Micrometer doesn’t have that notion. It’s great especially if Pulsar and another plugin are using same library (e.g. Caffeine for caching), thus in Prometheus or other libraries the metrics will override each other, but in OTel the meter provides an attribute for name and version, thus provide a namespace. I don’t understand this point. Can you please elaborate? > OTel by design has an instrument that you report measurements for a given attribute set, meaning it has that design of instrument = map(attributes→values). In Micrometer, it’s designed in a way that each (instrument, attributes) is a metric on its own. Less elegant and more confusing. I don’t understand this point. Can you please elaborate? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
