Update on the alert generator compliance: Some vendors cannot expose some of the APIs required by the Prometheus alert generator compliance test suite.
*We modified the test suite so it can be run by vendors themselves and submit results to Prometheus/CNCF. *CNCF will contractually protect against submitting wrong results on purpose. If there are any questions, please feel free to ask in this thread. Thanks, Jean-Philippe Quéméner (johnnyqqqq) On Thursday, March 3, 2022 at 1:17:48 AM UTC+1 [email protected] wrote: > Thanks, created https://github.com/cortexproject/cortex/issues/4656 to > track the work > > On Wednesday, March 2, 2022 at 12:35:06 AM UTC-8 [email protected] wrote: > >> Hello Cortex team, >> >> I hope this email finds you well. >> >> As a part of Prometheus conformance program >> <https://github.com/cncf/prometheus-conformance>, we had floated a doc >> on "Prometheus Alert-Generator Compliance" many months ago and had >> finalized the specification for that (see here >> <https://github.com/prometheus/compliance/blob/main/alert_generator/specification.md> >> ). >> >> The test suite to test the specification is now ready and instructions on >> how to run the test suite are present here >> <https://github.com/prometheus/compliance/blob/main/alert_generator/README.md> >> . >> >> If you wish Cortex to be compliant with Prometheus Alert-Generator, >> please test your software with the above test suite and report back the >> results (results being the entire log output of the test run). >> >> If you do need any additional help from the test suite (for example >> setting custom headers to some requests), please let me know and I will add >> those abilities to the test suite. >> >> If you face issues in running the test suite or understanding any error >> messages, I am happy to answer your queries. >> >> We plan to publish the results during the third week of May 2022 (during >> KubeCon EU). >> >> PS: I already tested Cortex locally since I was familiar with it and it >> almost passed. It needs a Prometheus upgrade so that it can support a >> couple of new template functions that were introduced after the Prometheus >> version that is vendored in Cortex (Prometheus v2.33 is the latest that the >> compliance is looking at). >> >> Thanks, >> Ganesh (codesome) >> Prometheus team >> > -- You received this message because you are subscribed to the Google Groups "Prometheus Developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-developers/6ab2d4a3-00ee-4b08-b2ff-eff3f88fa140n%40googlegroups.com.
