bking added a comment.
> It is possible that this metric is polluted with monitoring queries that do not relate to serving user traffic I did a little checking around this. Prometheus blackbox checks are defined here <https://gerrit.wikimedia.org/r/plugins/gitiles/operations/puppet/+/refs/heads/production/modules/profile/manifests/query_service/monitor/wikidata_public.pp> . Their frequency is defined by the `scrape_interval` setting for the probes-custom_puppet-http.yaml job <https://gerrit.wikimedia.org/r/plugins/gitiles/operations/puppet/+/refs/heads/production/modules/prometheus/manifests/blackbox/check/http.pp#90> , which is currently 15 seconds. Since we require a unique check per team, that should be around 8 queries per minute per Prometheus host. Note that there are separate Prometheus checks that use the jmx exporter instead of blackbox, each with a frequency of 60 seconds. Looking at `prometheus1006`, I see the following: - jmx_wdqs_updater - jmx_query_service_streaming_updater - jmx_wdqs_blazegraph My next step is to figure out how many of the Prometheus hosts are actively polling each WDQS endpoint. TASK DETAIL https://phabricator.wikimedia.org/T360993 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: bking Cc: bking, Aklapper, dcausse, Danny_Benjafield_WMDE, S8321414, Astuthiodit_1, AWesterinen, karapayneWMDE, Invadibot, maantietaja, ItamarWMDE, Akuckartz, Nandana, Namenlos314, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, KimKelting, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331
_______________________________________________ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
