mforns added a comment.
I've looked a bit into this and I think I found what's happening.
Indeed the metrics query is able to gather the data correctly, but the
metrics do not reach Graphite.
The reason is the HiveToGraphite Spark job is failing when sending the
metrics to Graphite, because the values of the metrics are doubles.
22/09/07 00:33:14 ERROR HiveToGraphite: java.lang.Double cannot be cast to
java.lang.Long. Failed to send message to Graphite.
HiveToGraphite expects that the metric values are longs, and not doubles.
Although the queries do not explicitly specify the double type, my suspicion
is that the `percentile_approx` calculation in some metrics outputs a double,
percentile_approx(time_firstbyte, 0.5) as metric_count,
which after the `UNION` statement affects all the results (all the metric
values become doubles since they share the same column). But maybe I'm wrong!
In any case, we have to modify the query file to make sure the output values
are compatible with the type `long`.
---
PS: One question is, why did Spark finish with final app status `SUCCEEDED`
when it recorded a `HiveToGraphite ERROR`??? We Data Engineering should look
into that, as well...
TASK DETAIL
https://phabricator.wikimedia.org/T314131
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: mforns
Cc: Aklapper, Michael, Astuthiodit_1, EChetty, BTullis, karapayneWMDE,
Invadibot, maantietaja, ItamarWMDE, Akuckartz, Nandana, Lahi, Gq86,
GoranSMilovanovic, QZanden, LawExplorer, _jensen, rosalieper, Scott_WUaS,
Wikidata-bugs, aude, Mbch331
_______________________________________________
Wikidata-bugs mailing list -- [email protected]
To unsubscribe send an email to [email protected]
