On 18.11.21 16:36, 'Fabian Reinartz' via Prometheus Developers wrote: > > A central issue is that the remote APIs expose the Prometheus storage data > model. It is notably different from the Prometheus/OpenMetrics > instrumentation model and discards most of the structure known at scrape > time. > Structured data is critical to store and query data more effectively and > translate it to different underlying storage data models. With the current > API however the structure is very challenging and sometimes impossible to > restore.
Thanks for picking this up. These were precisely the concerns when remote write was sketched out in 2016 – and one of the reasons to mark it explicitly as experimental. “Sadly” (and also unsurprisingly), everyone jumped on the experimental specification, and a whole industry has evolved around it, so that we are essentially required to go for a v2 to address the concerns now. I can add from the Prometheus side that things are finally moving towards storing structured data natively in the TSDB, namely with the work on the new histograms. I expect that the same work will open up possibilities for more structured data and also for richer and better integrated meta-data. The implications for remote-write are twofold: For one, those changes motivate to change remote-write along with them. On the other hand, it also enables Prometheus to support a more structured remote-write protocol in the first place. (Interestingly, before remote-write, Prometheus had federation, and it deliberately uses the same format as for scraping. The plan back then was to “soon” enable the Prometheus TSDB to support all the structure and meta-data in the exposition format. But that hasn't happened yet, and federation still exposes all metrics as flat "untyped" metrics without any meta-data.) -- Björn Rabenstein [PGP-ID] 0x851C3DA17D748D03 [email] [email protected] -- You received this message because you are subscribed to the Google Groups "Prometheus Developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-developers/20211122121647.GT3660%40jahnn.
