Great! I think as we start taking compatibility more seriously, we could even create a home for remote_write / PromQL / ... compatibility tests in either the main Prometheus org, or in a dedicated "Prometheus Compatibility Project" org?
On Tue, Apr 13, 2021 at 2:12 PM Tom Wilkie <[email protected]> wrote: > Hello Prometheans! > > In preparation for my PromCon talk, I build a little test suite the checks > various Prometheus Remote Write compatible "agents": > > https://github.com/tomwilkie/remote-write-compliance > > There are ~13 pretty simple tests, and each one exports some metrics, > forks off the "agent" configured to scrape those metrics and send remote > write requests back to the test, and then validates the results > it receives. Its still a WIP but there are a few interesting results > (including a bug in the Grafana Agent! that has already been fixed...) > > I guess I'm sending this email both as an FYI, to see if anyone has any > questions, and to see what people think of transferring this to the > prometheus-community github org? > > Cheers > > Tom > > --- FAIL: TestRemoteWrite/vmagent (0.00s) > > --- FAIL: TestRemoteWrite/vmagent/Invalid (10.21s) > > --- PASS: TestRemoteWrite/vmagent/Histogram (10.21s) > > --- FAIL: TestRemoteWrite/vmagent/NameLabel (10.21s) > > --- PASS: TestRemoteWrite/vmagent/SortedLabels (10.25s) > > --- PASS: TestRemoteWrite/vmagent/Counter (10.25s) > > --- PASS: TestRemoteWrite/vmagent/Gauge (10.25s) > > --- PASS: TestRemoteWrite/vmagent/InstanceLabel (10.27s) > > --- PASS: TestRemoteWrite/vmagent/Up (10.32s) > > --- PASS: TestRemoteWrite/vmagent/EmptyLabels (10.18s) > > --- FAIL: TestRemoteWrite/vmagent/Staleness (10.27s) > > --- PASS: TestRemoteWrite/vmagent/Timestamp (10.27s) > > --- FAIL: TestRemoteWrite/vmagent/RepeatedLabels (10.30s) > > --- PASS: TestRemoteWrite/vmagent/JobLabel (10.32s) > > --- PASS: TestRemoteWrite/prometheus (0.00s) > > --- PASS: TestRemoteWrite/prometheus/Timestamp (10.14s) > > --- PASS: TestRemoteWrite/prometheus/NameLabel (10.14s) > > --- PASS: TestRemoteWrite/prometheus/Counter (10.14s) > > --- PASS: TestRemoteWrite/prometheus/EmptyLabels (10.14s) > > --- PASS: TestRemoteWrite/prometheus/Staleness (10.14s) > > --- PASS: TestRemoteWrite/prometheus/InstanceLabel (10.14s) > > --- PASS: TestRemoteWrite/prometheus/RepeatedLabels (10.14s) > > --- PASS: TestRemoteWrite/prometheus/SortedLabels (10.17s) > > --- PASS: TestRemoteWrite/prometheus/Gauge (10.07s) > > --- PASS: TestRemoteWrite/prometheus/JobLabel (10.07s) > > --- PASS: TestRemoteWrite/prometheus/Up (10.07s) > > --- PASS: TestRemoteWrite/prometheus/Invalid (10.08s) > > --- PASS: TestRemoteWrite/prometheus/Histogram (10.08s) > > --- FAIL: TestRemoteWrite/otelcollector (0.00s) > > --- PASS: TestRemoteWrite/otelcollector/Timestamp (10.02s) > > --- FAIL: TestRemoteWrite/otelcollector/NameLabel (10.02s) > > --- PASS: TestRemoteWrite/otelcollector/EmptyLabels (10.02s) > > --- FAIL: TestRemoteWrite/otelcollector/Staleness (10.02s) > > --- FAIL: TestRemoteWrite/otelcollector/Up (10.02s) > > --- FAIL: TestRemoteWrite/otelcollector/InstanceLabel (10.03s) > > --- FAIL: TestRemoteWrite/otelcollector/Counter (10.03s) > > --- FAIL: TestRemoteWrite/otelcollector/RepeatedLabels (10.04s) > > --- PASS: TestRemoteWrite/otelcollector/Gauge (10.03s) > > --- PASS: TestRemoteWrite/otelcollector/SortedLabels (10.03s) > > --- FAIL: TestRemoteWrite/otelcollector/Invalid (10.03s) > > --- FAIL: TestRemoteWrite/otelcollector/JobLabel (10.03s) > > --- FAIL: TestRemoteWrite/otelcollector/Histogram (10.03s) > > --- FAIL: TestRemoteWrite/telegraf (0.00s) > > --- PASS: TestRemoteWrite/telegraf/Counter (10.03s) > > --- FAIL: TestRemoteWrite/telegraf/RepeatedLabels (10.03s) > > --- PASS: TestRemoteWrite/telegraf/SortedLabels (10.03s) > > --- FAIL: TestRemoteWrite/telegraf/EmptyLabels (10.03s) > > --- PASS: TestRemoteWrite/telegraf/Timestamp (10.03s) > > --- FAIL: TestRemoteWrite/telegraf/Staleness (10.03s) > > --- PASS: TestRemoteWrite/telegraf/Histogram (10.03s) > > --- FAIL: TestRemoteWrite/telegraf/NameLabel (10.03s) > > --- PASS: TestRemoteWrite/telegraf/JobLabel (10.03s) > > --- FAIL: TestRemoteWrite/telegraf/InstanceLabel (10.03s) > > --- FAIL: TestRemoteWrite/telegraf/Up (10.03s) > > --- PASS: TestRemoteWrite/telegraf/Gauge (10.03s) > > --- FAIL: TestRemoteWrite/telegraf/Invalid (10.03s) > > --- PASS: TestRemoteWrite/grafana (0.00s) > > --- PASS: TestRemoteWrite/grafana/Histogram (10.17s) > > --- PASS: TestRemoteWrite/grafana/InstanceLabel (10.17s) > > --- PASS: TestRemoteWrite/grafana/Invalid (10.18s) > > --- PASS: TestRemoteWrite/grafana/JobLabel (10.19s) > > --- PASS: TestRemoteWrite/grafana/Up (10.19s) > > --- PASS: TestRemoteWrite/grafana/Timestamp (10.19s) > > --- PASS: TestRemoteWrite/grafana/Gauge (10.19s) > > --- PASS: TestRemoteWrite/grafana/Counter (10.20s) > > --- PASS: TestRemoteWrite/grafana/Staleness (10.12s) > > --- PASS: TestRemoteWrite/grafana/EmptyLabels (10.13s) > > --- PASS: TestRemoteWrite/grafana/NameLabel (10.11s) > > --- PASS: TestRemoteWrite/grafana/RepeatedLabels (10.11s) > > --- PASS: TestRemoteWrite/grafana/SortedLabels (10.12s) > > -- > You received this message because you are subscribed to the Google Groups > "Prometheus Developers" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/prometheus-developers/CAB58Z120jLW%3D17H9RZwcMmGMfUAyVKH3O0MTvBqW45gMwo0oYA%40mail.gmail.com > <https://groups.google.com/d/msgid/prometheus-developers/CAB58Z120jLW%3D17H9RZwcMmGMfUAyVKH3O0MTvBqW45gMwo0oYA%40mail.gmail.com?utm_medium=email&utm_source=footer> > . > -- You received this message because you are subscribed to the Google Groups "Prometheus Developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-developers/CA%2BT6YoxmeQN2QO%3DHcg0euC6d%2BWn24fkwhRn8ZBG9k1x5p4KsDw%40mail.gmail.com.
