Update on the alert generator compliance: Some vendors cannot expose some of the APIs required by the Prometheus alert generator compliance test suite.
*We modified the test suite so it can be run by vendors themselves and submit results to Prometheus/CNCF. *CNCF will contractually protect against submitting wrong results on purpose. If there are any questions, please feel free to ask in this thread. Thanks, Jean-Philippe Quéméner (johnnyqqqq) On Sunday, March 6, 2022 at 10:45:44 PM UTC+1 [email protected] wrote: > Hey Ganesh, > > Thanks for the heads up and to all the work that has gone into this from > the various contributors, you will hear from us in a bit. > > While I have you I’m looking for a review to update the OpenMetrics > conformance instructions to mention the scrape validator tool which Chao > authored last year: > https://github.com/prometheus/compliance/pull/77/files > > Best, > Rob > > On Wed, Mar 2, 2022 at 3:44 AM Ganesh Vernekar <[email protected]> wrote: > >> Hello Chronosphere team, >> >> I hope this email finds you well. >> >> As a part of Prometheus conformance program >> <https://github.com/cncf/prometheus-conformance>, we had floated a doc >> on "Prometheus Alert-Generator Compliance" many months ago and had >> finalized the specification for that (see here >> <https://github.com/prometheus/compliance/blob/main/alert_generator/specification.md> >> ). >> >> The test suite to test the specification is now ready and instructions on >> how to run the test suite are present here >> <https://github.com/prometheus/compliance/blob/main/alert_generator/README.md> >> . >> >> If you wish Chronosphere to be compliant with Prometheus Alert-Generator, >> please test your software with the above test suite and report back the >> results by replying to this thread (results being the entire log output of >> the test run). >> >> If you do need any additional help from the test suite (for example >> setting custom headers to some requests), please let me know and I will add >> those abilities to the test suite. >> >> Going forward, we would like to automate this process. We ask you to add >> your test-suite config template by opening a PR against >> prometheus/compliance <https://github.com/prometheus/compliance/> and >> create test-chronosphere.yaml in the alert_generator directory. See >> test-.*.yaml files here >> <https://github.com/prometheus/compliance/tree/main/alert_generator> for >> example. Please also add instructions on how to set up as comments in the >> same file. >> >> If you face issues in running the test suite or understanding any error >> messages, I am happy to answer your queries. >> >> We plan to publish the results during the third week of May 2022 (during >> KubeCon EU). >> >> Thanks, >> Ganesh (codesome) >> Prometheus team >> > -- You received this message because you are subscribed to the Google Groups "Prometheus Developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-developers/dd3d2482-3c86-4d36-89b7-d52a84ea183bn%40googlegroups.com.
