Sounds good! Should we start a regular working group to focus on this? I know many people have been interested.
In the meantime I'll try and get that v1 spec published on the prometheus site so we can officially call it "done". One question I'm interested in discussing is to what extent the existing protocol can be evolved towards something that is e.g. transactional, more structured and lightweight vs throwing it away and building a complete new one. There is a ton of support for remote write baked into existing software and it'd be a shame to throw that away. Cheers Tom On Thu, Nov 18, 2021 at 6:33 PM Bartłomiej Płotka <[email protected]> wrote: > Hi Fabian! > > As Julian said, it sounds like we have to then talk about Remote Write v2. > We can totally start designing one. Looking forward to proposals in this > space! (: > > Kind Regards, > Bartek > > On Thursday, November 18, 2021 at 6:31:40 PM UTC+1 Julien Pivotto wrote: > >> On 18 Nov 16:36, 'Fabian Reinartz' via Prometheus Developers wrote: >> > Hi developers, >> > >> > We recently launched Google Cloud’s Prometheus metric backend based on >> > Monarch. We encountered some obstacles regarding the remote APIs, which >> we >> > believe to be common for backends that were not built for Prometheus >> > bottom-up. >> > >> > A central issue is that the remote APIs expose the Prometheus storage >> data >> > model. It is notably different from the Prometheus/OpenMetrics >> > instrumentation model and discards most of the structure known at >> scrape >> > time. >> > Structured data is critical to store and query data more effectively >> and >> > translate it to different underlying storage data models. With the >> current >> > API however the structure is very challenging and sometimes impossible >> to >> > restore. >> > >> > We're also interested in potential new features, like first-class >> support >> > for HA deduplication and write atomicity. >> > >> > >> > We’d like to explore evolving the remote APIs so that interoperability >> and >> > compliance become more practically attainable for independently >> developed >> > backends. >> > But there should be substantial opportunities for backends that largely >> > reuse Prometheus code as well. >> > >> > >> > If I recall correctly from years ago, the current remote API was always >> > meant as a starting point, rather than the final solution. Is now a >> good >> > time to revisit its fundamentals? >> > >> > >> > Are there any recent discussions in this area to read up on and >> participate >> > in? >> >> >> Hello Fabian, >> >> I am not a remote write expert but we are about to mark the remote write >> specification as stable. Here is the proposal: >> >> https://docs.google.com/document/d/1LPhVRSFkGNSuU1fBd81ulhsCPR4hkSZyyBj1SZ8fWOM/edit#heading=h.3p42p5s8n0ui >> >> We are also working on transactional remote write: >> >> https://docs.google.com/document/d/1UgSNnQYB1TJKVHkrUybEZPDsDl6MYCSiABm71v73Uvs/edit#heading=h.ih79uqrsv2dl >> >> Regards, >> >> >> > >> > >> > >> > Thanks, >> > Fabian >> > >> > -- >> > You received this message because you are subscribed to the Google >> Groups "Prometheus Developers" group. >> > To unsubscribe from this group and stop receiving emails from it, send >> an email to [email protected]. >> > To view this discussion on the web visit >> https://groups.google.com/d/msgid/prometheus-developers/CAG97UEngEdrkrcGnH5J%2BRQ4ThT8GW530JcLOgdJ2Jf13A_RAjA%40mail.gmail.com. >> >> >> -- >> Julien Pivotto >> @roidelapluie >> > -- > You received this message because you are subscribed to the Google Groups > "Prometheus Developers" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/prometheus-developers/3fd49985-0509-4882-8d02-3199be0d51afn%40googlegroups.com > <https://groups.google.com/d/msgid/prometheus-developers/3fd49985-0509-4882-8d02-3199be0d51afn%40googlegroups.com?utm_medium=email&utm_source=footer> > . > -- You received this message because you are subscribed to the Google Groups "Prometheus Developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-developers/CAB58Z119epxRMfmXpajvS%2BGSPT_ZTe3UABR0Ho9nO8O3JoEhGA%40mail.gmail.com.
