[Wikidata-bugs] [Maniphest] T360891: Wikibase Lua tracking sampling is broken

[Maintenance_bot]

Maintenance_bot removed a project: Patch-For-Review.

TASK DETAIL
  https://phabricator.wikimedia.org/T360891

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: hoo, Maintenance_bot
Cc: tstarling, Lucas_Werkmeister_WMDE, Aklapper, Michael, 
Danny_Benjafield_WMDE, S8321414, LennardHofmann, Astuthiodit_1, karapayneWMDE, 
Invadibot, maantietaja, ItamarWMDE, Akuckartz, Dringsim, Nandana, lucamauri, 
Lahi, Gq86, GoranSMilovanovic, QZanden, KimKelting, LawExplorer, _jensen, 
rosalieper, Scott_WUaS, Wikidata-bugs, aude, Mbch331, Isabelladantes1983, 
Themindcoder, Adamm71, Jersione, Hellket777, LisafBia6531, 786, Biggs657, 
Juan90264, Alter-paule, Beast1978, Un1tY, Hook696, Kent7301, joker88john, 
CucyNoiD, Gaboe420, Giuliamocci, Cpaulf30, Af420, Bsandipan, Lewizho99, 
Maathavan, Neuronton
___
Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org
To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org

[Wikidata-bugs] [Maniphest] T360891: Wikibase Lua tracking sampling is broken

[ReleaseTaggerBot]

ReleaseTaggerBot added a project: MW-1.43-notes (1.43.0-wmf.3; 2024-04-30).

TASK DETAIL
  https://phabricator.wikimedia.org/T360891

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: hoo, ReleaseTaggerBot
Cc: tstarling, Lucas_Werkmeister_WMDE, Aklapper, Michael, 
Danny_Benjafield_WMDE, Isabelladantes1983, Themindcoder, Adamm71, S8321414, 
Jersione, Hellket777, LennardHofmann, LisafBia6531, Astuthiodit_1, 786, 
Biggs657, karapayneWMDE, Invadibot, maantietaja, Juan90264, Alter-paule, 
Beast1978, ItamarWMDE, Un1tY, Akuckartz, Dringsim, Hook696, Kent7301, 
joker88john, CucyNoiD, Nandana, Gaboe420, lucamauri, Giuliamocci, Cpaulf30, 
Lahi, Gq86, Af420, Bsandipan, GoranSMilovanovic, QZanden, KimKelting, 
LawExplorer, Lewizho99, Maathavan, _jensen, rosalieper, Neuronton, Scott_WUaS, 
Wikidata-bugs, aude, Mbch331
___
Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org
To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org

[Wikidata-bugs] [Maniphest] T360891: Wikibase Lua tracking sampling is broken

[gerritbot]

gerritbot added a comment.


  Change #1021429 **merged** by jenkins-bot:
  
  [mediawiki/extensions/Wikibase@master] Fix sampling of Lua function tracking
  
  https://gerrit.wikimedia.org/r/1021429

TASK DETAIL
  https://phabricator.wikimedia.org/T360891

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: hoo, gerritbot
Cc: tstarling, Lucas_Werkmeister_WMDE, Aklapper, Michael, 
Danny_Benjafield_WMDE, Isabelladantes1983, Themindcoder, Adamm71, S8321414, 
Jersione, Hellket777, LennardHofmann, LisafBia6531, Astuthiodit_1, 786, 
Biggs657, karapayneWMDE, Invadibot, maantietaja, Juan90264, Alter-paule, 
Beast1978, ItamarWMDE, Un1tY, Akuckartz, Dringsim, Hook696, Kent7301, 
joker88john, CucyNoiD, Nandana, Gaboe420, lucamauri, Giuliamocci, Cpaulf30, 
Lahi, Gq86, Af420, Bsandipan, GoranSMilovanovic, QZanden, KimKelting, 
LawExplorer, Lewizho99, Maathavan, _jensen, rosalieper, Neuronton, Scott_WUaS, 
Wikidata-bugs, aude, Mbch331
___
Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org
To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org

[Wikidata-bugs] [Maniphest] T360891: Wikibase Lua tracking sampling is broken

[gerritbot]

gerritbot added a comment.


  Change #1021429 had a related patch set uploaded (by Hoo man; author: Hoo 
man):
  
  [mediawiki/extensions/Wikibase@master] Fix sampling of Lua function tracking
  
  https://gerrit.wikimedia.org/r/1021429

TASK DETAIL
  https://phabricator.wikimedia.org/T360891

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: hoo, gerritbot
Cc: tstarling, Lucas_Werkmeister_WMDE, Aklapper, Michael, 
Danny_Benjafield_WMDE, Isabelladantes1983, Themindcoder, Adamm71, S8321414, 
Jersione, Hellket777, LennardHofmann, LisafBia6531, Astuthiodit_1, 786, 
Biggs657, karapayneWMDE, Invadibot, maantietaja, Juan90264, Alter-paule, 
Beast1978, ItamarWMDE, Un1tY, Akuckartz, Hook696, Kent7301, joker88john, 
CucyNoiD, Nandana, Gaboe420, lucamauri, Giuliamocci, Cpaulf30, Lahi, Gq86, 
Af420, Bsandipan, GoranSMilovanovic, QZanden, KimKelting, LawExplorer, 
Lewizho99, Maathavan, _jensen, rosalieper, Neuronton, Scott_WUaS, 
Wikidata-bugs, aude, Mbch331
___
Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org
To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org

[Wikidata-bugs] [Maniphest] T360891: Wikibase Lua tracking sampling is broken

[gerritbot]

gerritbot added a comment.


  Change #1014000 **abandoned** by Michael Große:
  
  [mediawiki/extensions/Wikibase@master] Fix Lua tracking by initializing 
random generator
  
  Reason:
  
  I'm in a different team now and do not have the capacity to work on this 
anymore. Please see for T360891 for the problems with the change in its current 
form and possible alternatives.
  
  https://gerrit.wikimedia.org/r/1014000

TASK DETAIL
  https://phabricator.wikimedia.org/T360891

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: hoo, gerritbot
Cc: tstarling, Lucas_Werkmeister_WMDE, Aklapper, Michael, 
Danny_Benjafield_WMDE, Isabelladantes1983, Themindcoder, Adamm71, S8321414, 
Jersione, Hellket777, LennardHofmann, LisafBia6531, Astuthiodit_1, 786, 
Biggs657, karapayneWMDE, Invadibot, maantietaja, Juan90264, Alter-paule, 
Beast1978, ItamarWMDE, Un1tY, Akuckartz, Hook696, Kent7301, joker88john, 
CucyNoiD, Nandana, Gaboe420, lucamauri, Giuliamocci, Cpaulf30, Lahi, Gq86, 
Af420, Bsandipan, GoranSMilovanovic, QZanden, KimKelting, LawExplorer, 
Lewizho99, Maathavan, _jensen, rosalieper, Neuronton, Scott_WUaS, 
Wikidata-bugs, aude, Mbch331
___
Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org
To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org

[Wikidata-bugs] [Maniphest] T360891: Wikibase Lua tracking sampling is broken

[Lucas_Werkmeister_WMDE]

Lucas_Werkmeister_WMDE added a comment.


  In T360891#9664129 , 
@tstarling wrote:
  
  > Or just increment a counter and report every time the counter mod N is 
zero. Offset the counter at startup by a random number.
  
  I think that sounds good enough – since the counter is offset for each page 
parse, the same module will still have different calls sampled each time it’s 
used on a page, so that should be random enough. Thanks for your detailed reply!

TASK DETAIL
  https://phabricator.wikimedia.org/T360891

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: hoo, Lucas_Werkmeister_WMDE
Cc: tstarling, Lucas_Werkmeister_WMDE, Aklapper, Michael, 
Danny_Benjafield_WMDE, Isabelladantes1983, Themindcoder, Adamm71, S8321414, 
Jersione, Hellket777, LennardHofmann, LisafBia6531, Astuthiodit_1, 786, 
Biggs657, karapayneWMDE, Invadibot, maantietaja, Juan90264, Alter-paule, 
Beast1978, ItamarWMDE, Un1tY, Akuckartz, Hook696, Kent7301, joker88john, 
CucyNoiD, Nandana, Gaboe420, lucamauri, Giuliamocci, Cpaulf30, Lahi, Gq86, 
Af420, Bsandipan, GoranSMilovanovic, QZanden, KimKelting, LawExplorer, 
Lewizho99, Maathavan, _jensen, rosalieper, Neuronton, Scott_WUaS, 
Wikidata-bugs, aude, Mbch331
___
Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org
To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org

[Wikidata-bugs] [Maniphest] T360891: Wikibase Lua tracking sampling is broken

[tstarling]

tstarling added a comment.


  We (Brad Jorsch and I) didn't want random numbers in Scribunto because it 
encourages an inefficient implementation of things like "spotlight" templates 
that show a random featured article from a list of such articles. We want to 
cache the output from Scribunto but then people will see the same random 
selection for months at a time, so users will inevitably try to defeat caching 
or incentivize purge requests.
  
  For this application I would suggest passing a random number in $setupOptions 
to LuaEngine::registerInterface(). Implement your own LCG 
 which should be 
doable in 2 lines of code even though you only have floating point numbers. The 
multiplication needs to fit in the 53-bit mantissa of the floating point number 
without rounding, which is not a problem for say MINSTD m=2^31-1, a=48271, 
which gives a maximum base 2 logarithm of 47.
  
  Or just increment a counter and report every time the counter mod N is zero. 
Offset the counter at startup by a random number.
  
  If you actually wanted to unconditionally seed the global RNG, this would not 
be the right way to do it. Better to seed it in mw.executeModule() based on an 
additional parameter passed from PHP.
  
  os.clock() only looks attractive as a random number source because we've 
deliberately avoided providing better random number sources to Lua.

TASK DETAIL
  https://phabricator.wikimedia.org/T360891

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: hoo, tstarling
Cc: tstarling, Lucas_Werkmeister_WMDE, Aklapper, Michael, 
Danny_Benjafield_WMDE, Isabelladantes1983, Themindcoder, Adamm71, S8321414, 
Jersione, Hellket777, LennardHofmann, LisafBia6531, Astuthiodit_1, 786, 
Biggs657, karapayneWMDE, Invadibot, maantietaja, Juan90264, Alter-paule, 
Beast1978, ItamarWMDE, Un1tY, Akuckartz, Hook696, Kent7301, joker88john, 
CucyNoiD, Nandana, Gaboe420, lucamauri, Giuliamocci, Cpaulf30, Lahi, Gq86, 
Af420, Bsandipan, GoranSMilovanovic, QZanden, KimKelting, LawExplorer, 
Lewizho99, Maathavan, _jensen, rosalieper, Neuronton, Scott_WUaS, 
Wikidata-bugs, aude, Mbch331
___
Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org
To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org

[Wikidata-bugs] [Maniphest] T360891: Wikibase Lua tracking sampling is broken

[hoo]

hoo claimed this task.
hoo moved this task from Ready for Peer Review to In Peer Review on the 
Wikidata Dev Team (Wikidata.org Slice) board.

TASK DETAIL
  https://phabricator.wikimedia.org/T360891

WORKBOARD
  https://phabricator.wikimedia.org/project/board/6751/

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: hoo
Cc: Lucas_Werkmeister_WMDE, Aklapper, Michael, Danny_Benjafield_WMDE, 
Isabelladantes1983, Themindcoder, Adamm71, S8321414, Jersione, Hellket777, 
LennardHofmann, LisafBia6531, Astuthiodit_1, 786, Biggs657, karapayneWMDE, 
Invadibot, maantietaja, Juan90264, Alter-paule, Beast1978, ItamarWMDE, Un1tY, 
Akuckartz, Hook696, Kent7301, joker88john, CucyNoiD, Nandana, Gaboe420, 
lucamauri, Giuliamocci, Cpaulf30, Lahi, Gq86, Af420, Bsandipan, 
GoranSMilovanovic, QZanden, KimKelting, LawExplorer, Lewizho99, Maathavan, 
_jensen, rosalieper, Neuronton, Scott_WUaS, Wikidata-bugs, aude, Mbch331
___
Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org
To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org

[Wikidata-bugs] [Maniphest] T360891: Wikibase Lua tracking sampling is broken

[Lucas_Werkmeister_WMDE]

Lucas_Werkmeister_WMDE added a project: Wikidata Dev Team (Wikidata.org Slice).
Lucas_Werkmeister_WMDE added a comment.


  Pulling directly into our review column as an “external” change to review 
(implemented / submitted by the Wikidata Integrations Team).

TASK DETAIL
  https://phabricator.wikimedia.org/T360891

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Lucas_Werkmeister_WMDE
Cc: Lucas_Werkmeister_WMDE, Aklapper, Michael, Danny_Benjafield_WMDE, 
Isabelladantes1983, Themindcoder, Adamm71, S8321414, Jersione, Hellket777, 
LennardHofmann, LisafBia6531, Astuthiodit_1, 786, Biggs657, karapayneWMDE, 
Invadibot, maantietaja, Juan90264, Alter-paule, Beast1978, ItamarWMDE, Un1tY, 
Akuckartz, Hook696, Kent7301, joker88john, CucyNoiD, Nandana, Gaboe420, 
lucamauri, Giuliamocci, Cpaulf30, Lahi, Gq86, Af420, Bsandipan, 
GoranSMilovanovic, QZanden, KimKelting, LawExplorer, Lewizho99, Maathavan, 
_jensen, rosalieper, Neuronton, Scott_WUaS, Wikidata-bugs, aude, Mbch331
___
Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org
To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org

[Wikidata-bugs] [Maniphest] T360891: Wikibase Lua tracking sampling is broken

[Michael]

Michael updated the task description.

TASK DETAIL
  https://phabricator.wikimedia.org/T360891

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Michael
Cc: Aklapper, Michael, Danny_Benjafield_WMDE, Isabelladantes1983, Themindcoder, 
Adamm71, S8321414, Jersione, Hellket777, LennardHofmann, LisafBia6531, 
Astuthiodit_1, 786, Biggs657, karapayneWMDE, Invadibot, maantietaja, Juan90264, 
Alter-paule, Beast1978, ItamarWMDE, Un1tY, Akuckartz, Hook696, Kent7301, 
joker88john, CucyNoiD, Nandana, Gaboe420, lucamauri, Giuliamocci, Cpaulf30, 
Lahi, Gq86, Af420, Bsandipan, GoranSMilovanovic, QZanden, KimKelting, 
LawExplorer, Lewizho99, Maathavan, _jensen, rosalieper, Neuronton, Scott_WUaS, 
Wikidata-bugs, aude, Mbch331
___
Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org
To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org

[Wikidata-bugs] [Maniphest] T360891: Wikibase Lua tracking sampling is broken

[gerritbot]

gerritbot added a project: Patch-For-Review.

TASK DETAIL
  https://phabricator.wikimedia.org/T360891

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: gerritbot
Cc: Aklapper, Michael, Danny_Benjafield_WMDE, Isabelladantes1983, Themindcoder, 
Adamm71, S8321414, Jersione, Hellket777, LennardHofmann, LisafBia6531, 
Astuthiodit_1, 786, Biggs657, karapayneWMDE, Invadibot, maantietaja, Juan90264, 
Alter-paule, Beast1978, ItamarWMDE, Un1tY, Akuckartz, Hook696, Kent7301, 
joker88john, CucyNoiD, Nandana, Gaboe420, lucamauri, Giuliamocci, Cpaulf30, 
Lahi, Gq86, Af420, Bsandipan, GoranSMilovanovic, QZanden, KimKelting, 
LawExplorer, Lewizho99, Maathavan, _jensen, rosalieper, Neuronton, Scott_WUaS, 
Wikidata-bugs, aude, Mbch331
___
Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org
To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org

[Wikidata-bugs] [Maniphest] T360891: Wikibase Lua tracking sampling is broken

[gerritbot]

gerritbot added a comment.


  Change #1014000 had a related patch set uploaded (by Michael Große; author: 
Michael Große):
  
  [mediawiki/extensions/Wikibase@master] Fix Lua tracking by initializing 
random generator
  
  https://gerrit.wikimedia.org/r/1014000

TASK DETAIL
  https://phabricator.wikimedia.org/T360891

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: gerritbot
Cc: Aklapper, Michael, Danny_Benjafield_WMDE, S8321414, LennardHofmann, 
Astuthiodit_1, karapayneWMDE, Invadibot, maantietaja, ItamarWMDE, Akuckartz, 
Nandana, lucamauri, Lahi, Gq86, GoranSMilovanovic, QZanden, KimKelting, 
LawExplorer, _jensen, rosalieper, Scott_WUaS, Wikidata-bugs, aude, Mbch331
___
Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org
To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org

[Wikidata-bugs] [Maniphest] T360891: Wikibase Lua tracking sampling is broken

[Michael]

Michael updated the task description.

TASK DETAIL
  https://phabricator.wikimedia.org/T360891

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Michael
Cc: Aklapper, Michael, Danny_Benjafield_WMDE, S8321414, LennardHofmann, 
Astuthiodit_1, karapayneWMDE, Invadibot, maantietaja, ItamarWMDE, Akuckartz, 
Nandana, lucamauri, Lahi, Gq86, GoranSMilovanovic, QZanden, KimKelting, 
LawExplorer, _jensen, rosalieper, Scott_WUaS, Wikidata-bugs, aude, Mbch331
___
Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org
To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org

[Wikidata-bugs] [Maniphest] T360891: Wikibase Lua tracking sampling is broken

[Michael]

Michael created this task.
Michael added projects: Wikidata, Wikibase-Lua, wmde-wikidata-tech.
Restricted Application added a subscriber: Aklapper.

TASK DESCRIPTION
  **Steps to replicate the issue** (include links if applicable):
  
  - Go to 
https://grafana.wikimedia.org/d/jO1rYjwmz/wikidata-lua-function-usage?from=now-30d=1=now=enwiki=7
  
  **What happens?**:
  Notice that the cache-miss rate is above 2, that is above 200%!
  
  **What should have happened instead?**:
  It should be something that is actually possible.
  
  **Other information** (browser name/version, screenshots, etc.):
  This seems to be caused by the fact that for the last 3 years, we only track 
a sample of the lua events for performance reason, about 1 in every 1000 
events. That was introduced in Add a sample rate for Lua function call tracking 
. That 
is achieved by comparing the sampling rate to a randomly generated number. 
**However, the random number generator is never explicitly initialized and so 
every request gets the exact same list of random numbers!**
  
  In practice, this would sadly seem to mean that the Lua tracking data of the 
last 3 years is almost completely worthless.

TASK DETAIL
  https://phabricator.wikimedia.org/T360891

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Michael
Cc: Aklapper, Michael, Danny_Benjafield_WMDE, S8321414, LennardHofmann, 
Astuthiodit_1, karapayneWMDE, Invadibot, maantietaja, ItamarWMDE, Akuckartz, 
Nandana, lucamauri, Lahi, Gq86, GoranSMilovanovic, QZanden, KimKelting, 
LawExplorer, _jensen, rosalieper, Scott_WUaS, Wikidata-bugs, aude, Mbch331
___
Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org
To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org