[Wikidata-bugs] [Maniphest] T292601: Confirm Wikibase LuaFunctionCallTracker to statsd naming convention

[Lucas_Werkmeister_WMDE]

Lucas_Werkmeister_WMDE added a comment.


  This is probably superseded by T359253: Migrate 
MediaWiki.$prefix.wikibase.client.scribunto.* to statslib 
 now?

TASK DETAIL
  https://phabricator.wikimedia.org/T292601

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Lucas_Werkmeister_WMDE
Cc: Lucas_Werkmeister_WMDE, Krinkle, Aklapper, Danny_Benjafield_WMDE, 
LennardHofmann, Astuthiodit_1, karapayneWMDE, Invadibot, maantietaja, 
ItamarWMDE, Akuckartz, Nandana, lucamauri, Lahi, Gq86, GoranSMilovanovic, 
QZanden, KimKelting, LawExplorer, _jensen, rosalieper, Scott_WUaS, 
Wikidata-bugs, aude, Mbch331
___
Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org
To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org

[Wikidata-bugs] [Maniphest] T292601: Confirm Wikibase LuaFunctionCallTracker to statsd naming convention

[Michael]

Michael moved this task from Backlog to Tech on the 
MediaWiki-extensions-WikibaseClient board.
Michael edited projects, added wmde-wikidata-tech; removed [DEPRECATED] 
wdwb-tech.

TASK DETAIL
  https://phabricator.wikimedia.org/T292601

WORKBOARD
  https://phabricator.wikimedia.org/project/board/114/

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Michael
Cc: Lucas_Werkmeister_WMDE, Krinkle, Aklapper, Danny_Benjafield_WMDE, 
LennardHofmann, Astuthiodit_1, karapayneWMDE, Invadibot, maantietaja, 
ItamarWMDE, Akuckartz, Nandana, lucamauri, Lahi, Gq86, GoranSMilovanovic, 
QZanden, KimKelting, LawExplorer, _jensen, rosalieper, Scott_WUaS, 
Wikidata-bugs, aude, Mbch331, Ullasoff
___
Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org
To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org

[Wikidata-bugs] [Maniphest] T292601: Confirm Wikibase LuaFunctionCallTracker to statsd naming convention

[Krinkle]

Krinkle added a comment.


  It's not documented to my knowledge, but it is a pattern that is followed 
quite consistently throughout our software and elsewhere in the industry as 
relating to how StatsD, Graphite and Prometheus are used.
  
  Metrics follow a left-to-right hierarchy. In Prometheus this is done with 
underscores, in StatsD this is done with dots. There is generally one main 
segment at the start for the overal service or component, then a segment for 
the metric being measured, and then labels (Prometheus), hash tags (dogstatsd), 
or subcomponents (statsd/Graphite).
  
  The labels/subcomponents should only be used to represent subslices of the 
same metric, not to create entirely different metrics. So, for example, you 
would have `foo_bar_request_count.wiki` and `foo_bar_request_size.wiki` and not 
`foo_bar_request.wiki.request_count` and `foo_bar_request.wiki .request_size. 
This is also reflected on https://prometheus.io/docs/practices/naming/, because 
incompatible data structures would otherwise clash and result in non-sensical 
visualisations when later segments are left out of the query with a wildcard.
  
  This ensures maxium benefit of the wider ecosystem through Grafana, such as 
autocompletion, wildcard queries, and templates. And also reduces chances of 
conflicts with unrelated metrics clashing in the same timeseries database 
directory.  Remember that for Graphite, each dot-component is a subdirectory on 
disk where a file is stored, so 
`enwiki.wikibase.client.scribunto.wikibase.getEntityStatements.cache_miss` gets 
stored in 
`/var/lib/carbon/enwiki/wikibase/client/scribunto/wikibase/getEntityStatements/cache_miss/rate.wsp`,
 for example.

TASK DETAIL
  https://phabricator.wikimedia.org/T292601

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Krinkle
Cc: Lucas_Werkmeister_WMDE, Krinkle, Aklapper, Invadibot, maantietaja, 
Akuckartz, Nandana, lucamauri, Lahi, Gq86, GoranSMilovanovic, QZanden, 
LawExplorer, Vali.matei, _jensen, rosalieper, Scott_WUaS, Wikidata-bugs, aude, 
Addshore, Mbch331
___
Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org
To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org

[Wikidata-bugs] [Maniphest] T292601: Confirm Wikibase LuaFunctionCallTracker to statsd naming convention

[Lucas_Werkmeister_WMDE]

Lucas_Werkmeister_WMDE added a comment.


  Is this “statsd naming convention” documented anywhere? To me this task seems 
counterintuitive.

TASK DETAIL
  https://phabricator.wikimedia.org/T292601

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Lucas_Werkmeister_WMDE
Cc: Lucas_Werkmeister_WMDE, Krinkle, Aklapper, Invadibot, maantietaja, 
Akuckartz, Nandana, lucamauri, Lahi, Gq86, GoranSMilovanovic, QZanden, 
LawExplorer, Vali.matei, _jensen, rosalieper, Scott_WUaS, Wikidata-bugs, aude, 
Addshore, Mbch331
___
Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org
To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org

[Wikidata-bugs] [Maniphest] T292601: Confirm Wikibase LuaFunctionCallTracker to statsd naming convention

[Lydia_Pintscher]

Lydia_Pintscher added a project: wdwb-tech.

TASK DETAIL
  https://phabricator.wikimedia.org/T292601

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Lydia_Pintscher
Cc: Krinkle, Aklapper, Invadibot, maantietaja, Akuckartz, Nandana, lucamauri, 
Lahi, Gq86, GoranSMilovanovic, QZanden, LawExplorer, Vali.matei, _jensen, 
rosalieper, Scott_WUaS, Wikidata-bugs, aude, Addshore, Mbch331
___
Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org
To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org

[Wikidata-bugs] [Maniphest] T292601: Confirm Wikibase LuaFunctionCallTracker to statsd naming convention

[Maintenance_bot]

Maintenance_bot added a project: Wikidata.

TASK DETAIL
  https://phabricator.wikimedia.org/T292601

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Maintenance_bot
Cc: Krinkle, Aklapper, Invadibot, maantietaja, Akuckartz, Nandana, lucamauri, 
Lahi, Gq86, GoranSMilovanovic, QZanden, LawExplorer, Vali.matei, _jensen, 
rosalieper, Scott_WUaS, Wikidata-bugs, aude, Mbch331
___
Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org
To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org

[Wikidata-bugs] [Maniphest] T292601: Confirm Wikibase LuaFunctionCallTracker to statsd naming convention

[Krinkle]

Krinkle updated the task description.

TASK DETAIL
  https://phabricator.wikimedia.org/T292601

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Krinkle
Cc: Krinkle, Aklapper, Akuckartz, lucamauri, Vali.matei, _jensen, Wikidata-bugs
___
Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org
To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org

[Wikidata-bugs] [Maniphest] T292601: Confirm Wikibase LuaFunctionCallTracker to statsd naming convention

[Krinkle]

Krinkle created this task.
Krinkle added projects: MediaWiki-extensions-WikibaseClient, Wikibase-Lua, 
Performance-Team (Radar).
Restricted Application added a subscriber: Aklapper.

TASK DESCRIPTION
  The stats from LuaFunctionCallTracker are perhaps a bit too deep and 
integrated into the global namespace of stats messages. For example:
  

enwiki.wikibase.client.scribunto.wikibase.getEntityStatements.cache_miss:100|c
enwiki.wikibase.client.scribunto.wikibase.getEntityStatements.call:100|c

wikipedia.wikibase.client.scribunto.wikibase.getEntity.call:100|c
wikipedia.wikibase.client.scribunto.wikibase.getEntityStatements.call:100|c
  
  Compared to:
  
resourceloader_build.all:0.12707710266113|ms
resourceloader_build.user_options:0.12707710266113|ms

wanobjectcache.resourceloader_titleinfo.hit.volatile:0.41294097900391|ms
wanobjectcache.filerepo_file.hit.good:0.50878524780273|ms
  
  In particular:
  
  - The fragments (sitegroup, wikiid) are top-level metrics whereas these 
should go to the end to avoid conflicts with other metrics.
  - There is no containing metric for LuaFunctionCallTracker which makes these 
difficult to discover and organize.
  
  Source code:
  
  - Wikibase: LuaFunctionCallTracker.php 

  - Wikibase: mw.wikibase.lua 

 (call example)
  
  I suggest the following metric pattern instead:
  
wikibase_client_lua_tracker_by_wiki .  . 
wikibase_client_lua_tracker_by_sitegroup .  . 
  
  Where `key` would be a single segment, eg. dots converted to underscores 
using ` strtr( $key, '.', '_' )` . The Grafana dashboard can continue to query 
and plot multiple of these using wildcards, which are supported on segments as 
well. We can even copy over the existing data if preferred, using ad-hoc 
scripting on the Graphite which I can help with (and we've done this few times 
before).
  
  This is also in preparation for Prometheus, to make the metric ready for the 
conversion. From a 5 minute capture in production, this was the only metric 
source I found that was not grouping its metrics or that was using many more 
subsegments than seem to logically expected from the underlyng source code.

TASK DETAIL
  https://phabricator.wikimedia.org/T292601

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Krinkle
Cc: Krinkle, Aklapper, Akuckartz, lucamauri, Vali.matei, _jensen, Wikidata-bugs
___
Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org
To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org