Gehel has submitted this change and it was merged. ( https://gerrit.wikimedia.org/r/403933 )
Change subject: prometheus blazegraph exporter should not fail when blazegraph is down ...................................................................... prometheus blazegraph exporter should not fail when blazegraph is down prometheus client collects all metrics on startup to enumerate the metric names. If blazegraph isn't available at that time, it crashes. Note that the same issue occurs if blazegraph goes down while the exporter is running. Arguably, the prometheus python client itself should be more robust. But we can already make metric collection more robust by managing at least the most obvious exceptions. Bug: T184434 Change-Id: I8bfd360b2ca864e8c21ec8b03bde58b16a2839ae --- M prometheus-blazegraph-exporter 1 file changed, 30 insertions(+), 21 deletions(-) Approvals: jenkins-bot: Verified Filippo Giunchedi: Looks good to me, but someone else must approve Gehel: Looks good to me, approved diff --git a/prometheus-blazegraph-exporter b/prometheus-blazegraph-exporter index 9f9a23e..a75ffa2 100755 --- a/prometheus-blazegraph-exporter +++ b/prometheus-blazegraph-exporter @@ -27,6 +27,7 @@ from datetime import timedelta, tzinfo from dateutil.parser import parse +from urllib2 import URLError from xml.etree import ElementTree from prometheus_client import start_http_server, Summary @@ -52,9 +53,13 @@ url = self.url + "counters?depth=10&" + \ urllib.urlencode({'path': cnt_name}) - req = urllib2.Request(url) - req.add_header('Accept', 'application/xml') - response = urllib2.urlopen(req) + try: + req = urllib2.Request(url) + req.add_header('Accept', 'application/xml') + response = urllib2.urlopen(req) + except URLError: + return None + el = ElementTree.fromstring(response.read()) last_name = cnt_name.split('/')[-1] @@ -102,33 +107,37 @@ try: value = float(metric_value) - except ValueError: + except (ValueError, TypeError): value = float('nan') metric_family.add_metric([], value) - sparql_query = """ prefix schema: <http://schema.org/> - SELECT * WHERE { { - SELECT ( COUNT( * ) AS ?count ) { ?s ?p ?o } - } UNION { - SELECT * WHERE { <http://www.wikidata.org> schema:dateModified ?y } - } }""" - data = self.execute_sparql(sparql_query) - triple_metric = CounterMetricFamily('blazegraph_triples', '') lag_metric = CounterMetricFamily('blazegraph_lastupdated', '') + try: + sparql_query = """ prefix schema: <http://schema.org/> + SELECT * WHERE { { + SELECT ( COUNT( * ) AS ?count ) { ?s ?p ?o } + } UNION { + SELECT * WHERE { <http://www.wikidata.org> schema:dateModified ?y } + } }""" - for binding in data['results']['bindings']: - if 'count' in binding: - triple_count = binding['count']['value'] - triple_metric.add_metric([], float(triple_count)) + data = self.execute_sparql(sparql_query) - elif 'y' in binding: - lastUpdated = parse(binding['y']['value']) - lag_metric.add_metric([], float(lastUpdated.strftime('%s'))) - else: - raise ValueError('SPARQL binding returned with unexpected key') + for binding in data['results']['bindings']: + if 'count' in binding: + triple_count = binding['count']['value'] + triple_metric.add_metric([], float(triple_count)) + elif 'y' in binding: + lastUpdated = parse(binding['y']['value']) + lag_metric.add_metric([], float(lastUpdated.strftime('%s'))) + else: + raise ValueError('SPARQL binding returned with unexpected key') + + except URLError: + triple_metric.add_metric([], float('nan')) + lag_metric.add_metric([], float('nan')) yield triple_metric yield lag_metric -- To view, visit https://gerrit.wikimedia.org/r/403933 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: merged Gerrit-Change-Id: I8bfd360b2ca864e8c21ec8b03bde58b16a2839ae Gerrit-PatchSet: 3 Gerrit-Project: operations/debs/prometheus-blazegraph-exporter Gerrit-Branch: master Gerrit-Owner: Gehel <guillaume.leder...@wikimedia.org> Gerrit-Reviewer: Filippo Giunchedi <fgiunch...@wikimedia.org> Gerrit-Reviewer: Gehel <guillaume.leder...@wikimedia.org> Gerrit-Reviewer: Muehlenhoff <mmuhlenh...@wikimedia.org> Gerrit-Reviewer: jenkins-bot <> _______________________________________________ MediaWiki-commits mailing list MediaWiki-commits@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits