Thanks for the responses everyone. Sounds like the existing remote-write will become stable as is, which is certainly a logical consequence after all this time.
I can certainly imagine some features being added to it, as the transactional RW design shows. Another semi-low-hanging fruit may be to send the metric type enum on regular requests, not just out of band, to support conversion on the fly. But I think at some point it becomes possible but not necessarily sensical to extend the current proto. If client and server implementations need extensive conditional logic to handle a single proto, e.g. negotiation "how structured" the data should be, a separate endpoint ultimately seems simpler. The point on TSDB becoming more structured is interesting – how firm are these plans at this point? Any rough timelines? My first hunch would've been to explore integrating directly at the scraping layer to directly stream OpenMetrics (or a proto-equivalent) from there, backed by a separate, per-write-target WAL. This wouldn't constrain it by the currently supported storage data model and generally decouple the two aspects, which also seems more aligned with recent developments like the agent mode. Any thoughts on that general direction? On Monday, November 22, 2021 at 1:16:50 PM UTC+1 [email protected] wrote: > On 18.11.21 16:36, 'Fabian Reinartz' via Prometheus Developers wrote: > > > > A central issue is that the remote APIs expose the Prometheus storage > data > > model. It is notably different from the Prometheus/OpenMetrics > > instrumentation model and discards most of the structure known at scrape > > time. > > Structured data is critical to store and query data more effectively and > > translate it to different underlying storage data models. With the > current > > API however the structure is very challenging and sometimes impossible to > > restore. > > Thanks for picking this up. These were precisely the concerns when > remote write was sketched out in 2016 – and one of the reasons to mark > it explicitly as experimental. “Sadly” (and also unsurprisingly), > everyone jumped on the experimental specification, and a whole > industry has evolved around it, so that we are essentially required to > go for a v2 to address the concerns now. > > I can add from the Prometheus side that things are finally moving > towards storing structured data natively in the TSDB, namely with the > work on the new histograms. I expect that the same work will open up > possibilities for more structured data and also for richer and better > integrated meta-data. The implications for remote-write are twofold: > For one, those changes motivate to change remote-write along with > them. On the other hand, it also enables Prometheus to support a more > structured remote-write protocol in the first place. > > (Interestingly, before remote-write, Prometheus had federation, and it > deliberately uses the same format as for scraping. The plan back then > was to “soon” enable the Prometheus TSDB to support all the structure > and meta-data in the exposition format. But that hasn't happened yet, > and federation still exposes all metrics as flat "untyped" metrics > without any meta-data.) > > -- > Björn Rabenstein > [PGP-ID] 0x851C3DA17D748D03 > [email] [email protected] > -- You received this message because you are subscribed to the Google Groups "Prometheus Developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-developers/dabd717f-8cd8-4b38-8a73-54f6dfc14084n%40googlegroups.com.
