[Wikidata-bugs] [Maniphest] [Commented On] T184948: limit page creation and edit rate on Wikidata
gerritbot added a comment. Change 408629 abandoned by Ladsgroup: Add edit and create rate limit for wikidatawiki Reason: Duplicate https://gerrit.wikimedia.org/r/408629TASK DETAILhttps://phabricator.wikimedia.org/T184948EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Ladsgroup, gerritbotCc: Legoktm, Lea_Lacroix_WMDE, Dhx1, Stashbot, Edgars2007, Daniel_Mietchen, Pasleim, Magnus, abian, Lucas_Werkmeister_WMDE, ArtixKreiger, Multichill, Mahir256, Framawiki, hoo, Sjoerddebruin, Addshore, Ladsgroup, gerritbot, Aklapper, Lydia_Pintscher, Lahi, Gq86, GoranSMilovanovic, lisong, Jayprakash12345, QZanden, Zoranzoki21, LawExplorer, DatGuy, Devwaker, Niklitov, Urbanecm, JEumerus, Tulsi_Bhagat, Wong128hk, Luke081515, SimmeD, Wikidata-bugs, Snowolf, aude, Dcljr, Jdforrester-WMF, Matanya, Mbch331, Rxy, Jay8g, Krenair___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T184948: limit page creation and edit rate on Wikidata
Lydia_Pintscher added a comment. @Multichill let's look at the remaining issues at the hackathon together. I've put it on my list of things to discuss.TASK DETAILhttps://phabricator.wikimedia.org/T184948EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Ladsgroup, Lydia_PintscherCc: Liuxinyu970226, Legoktm, Lea_Lacroix_WMDE, Dhx1, Stashbot, Edgars2007, Daniel_Mietchen, Pasleim, Magnus, abian, Lucas_Werkmeister_WMDE, ArtixKreiger, Multichill, Mahir256, Framawiki, hoo, Sjoerddebruin, Addshore, Ladsgroup, gerritbot, Aklapper, Lydia_Pintscher, Lahi, Gq86, GoranSMilovanovic, lisong, Jayprakash12345, QZanden, Zoranzoki21, LawExplorer, DatGuy, Devwaker, Niklitov, Urbanecm, JEumerus, Tulsi_Bhagat, Wong128hk, Luke081515, SimmeD, biplabanand, Wikidata-bugs, Snowolf, aude, Dcljr, Jdforrester-WMF, Matanya, Mbch331, Rxy, Jay8g, Krenair___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T184948: limit page creation and edit rate on Wikidata
Multichill added a comment. @Lydia_Pintscher @Ladsgroup any status update on this?TASK DETAILhttps://phabricator.wikimedia.org/T184948EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Ladsgroup, MultichillCc: Liuxinyu970226, Legoktm, Lea_Lacroix_WMDE, Dhx1, Stashbot, Edgars2007, Daniel_Mietchen, Pasleim, Magnus, abian, Lucas_Werkmeister_WMDE, ArtixKreiger, Multichill, Mahir256, Framawiki, hoo, Sjoerddebruin, Addshore, Ladsgroup, gerritbot, Aklapper, Lydia_Pintscher, Lahi, Gq86, GoranSMilovanovic, lisong, Jayprakash12345, QZanden, Zoranzoki21, LawExplorer, DatGuy, Devwaker, Niklitov, Urbanecm, JEumerus, Tulsi_Bhagat, Wong128hk, Luke081515, SimmeD, biplabanand, Wikidata-bugs, Snowolf, aude, Dcljr, Jdforrester-WMF, Matanya, Mbch331, Rxy, Jay8g, Krenair___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T184948: limit page creation and edit rate on Wikidata
Sjoerddebruin added a comment. Seems like this is affecting the MassMessage extension: https://www.wikidata.org/w/index.php?title=Special:Log/massmessage&offset=&limit=250&type=massmessage&user=TASK DETAILhttps://phabricator.wikimedia.org/T184948EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Ladsgroup, SjoerddebruinCc: Dhx1, Stashbot, Edgars2007, Daniel_Mietchen, Pasleim, Magnus, abian, Lucas_Werkmeister_WMDE, ArtixKreiger, Multichill, Mahir256, Framawiki, hoo, Sjoerddebruin, Addshore, Ladsgroup, gerritbot, Aklapper, Lydia_Pintscher, Lahi, Gq86, GoranSMilovanovic, lisong, Jayprakash12345, QZanden, Zoranzoki21, LawExplorer, DatGuy, Devwaker, Niklitov, Urbanecm, JEumerus, Tulsi_Bhagat, Wong128hk, Luke081515, SimmeD, biplabanand, Wikidata-bugs, Snowolf, aude, Dcljr, Jdforrester-WMF, Matanya, Mbch331, Rxy, Jay8g, Krenair___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T184948: limit page creation and edit rate on Wikidata
Magnus added a comment. @Multichill agreed. The replag solution works AFAICT, why not duplicate that.TASK DETAILhttps://phabricator.wikimedia.org/T184948EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Ladsgroup, MagnusCc: Dhx1, Stashbot, Edgars2007, Daniel_Mietchen, Pasleim, Magnus, abian, Lucas_Werkmeister_WMDE, ArtixKreiger, Multichill, Mahir256, Framawiki, hoo, Sjoerddebruin, Addshore, Ladsgroup, gerritbot, Aklapper, Lydia_Pintscher, Lahi, Gq86, GoranSMilovanovic, lisong, Jayprakash12345, QZanden, Zoranzoki21, LawExplorer, DatGuy, Devwaker, Niklitov, Urbanecm, JEumerus, Tulsi_Bhagat, Wong128hk, Luke081515, SimmeD, biplabanand, Wikidata-bugs, Snowolf, aude, Dcljr, Jdforrester-WMF, Matanya, Mbch331, Rxy, Jay8g, Krenair___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T184948: limit page creation and edit rate on Wikidata
Dhx1 added a comment. The main issue I see with the hard limit of 40 creations/minute and 80 edits/minute is that it prevents bots from speeding up during quieter times of each day. RFC 6585 provides the "429 Too Many Requests" HTTP status code (see https://tools.ietf.org/html/rfc6585#page-3) The API could return a 429 status code if the server determines that the edit rate is too high, including a "Retry-After: x" header in the response to make bots pause for a few seconds. This removes the burden on the bots to do anything other than check for a 429 status code and check the "Retry-After: x" header. The bots no longer have to decide what lag/delays are acceptable at a given time of the day, and don't need to be updated each time the hard limits are changed.TASK DETAILhttps://phabricator.wikimedia.org/T184948EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Ladsgroup, Dhx1Cc: Dhx1, Stashbot, Edgars2007, Daniel_Mietchen, Pasleim, Magnus, abian, Lucas_Werkmeister_WMDE, ArtixKreiger, Multichill, Mahir256, Framawiki, hoo, Sjoerddebruin, Addshore, Ladsgroup, gerritbot, Aklapper, Lydia_Pintscher, Lahi, Gq86, GoranSMilovanovic, lisong, Jayprakash12345, QZanden, Zoranzoki21, LawExplorer, DatGuy, Devwaker, Niklitov, Urbanecm, JEumerus, Tulsi_Bhagat, Wong128hk, Luke081515, SimmeD, biplabanand, Wikidata-bugs, Snowolf, aude, Dcljr, Jdforrester-WMF, Matanya, Mbch331, Rxy, Jay8g, Krenair___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T184948: limit page creation and edit rate on Wikidata
Addshore added a comment. In T184948#4139228, @hoo wrote: Why is there a line at 60 edits per minute? Creation is at 40 per minute and edits are at 80 per minute. That would be because I miss read the patch *fixes now*TASK DETAILhttps://phabricator.wikimedia.org/T184948EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Ladsgroup, AddshoreCc: Stashbot, Edgars2007, Daniel_Mietchen, Pasleim, Magnus, abian, Lucas_Werkmeister_WMDE, ArtixKreiger, Multichill, Mahir256, Framawiki, hoo, Sjoerddebruin, Addshore, Ladsgroup, gerritbot, Aklapper, Lydia_Pintscher, Lahi, Gq86, GoranSMilovanovic, lisong, Jayprakash12345, QZanden, Zoranzoki21, LawExplorer, DatGuy, Devwaker, Niklitov, Urbanecm, JEumerus, Tulsi_Bhagat, Wong128hk, Luke081515, SimmeD, biplabanand, Wikidata-bugs, Snowolf, aude, Dcljr, Jdforrester-WMF, Matanya, Mbch331, Rxy, Jay8g, Krenair___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T184948: limit page creation and edit rate on Wikidata
hoo added a comment. In T184948#4139171, @Lucas_Werkmeister_WMDE wrote: Did we announce this limit to the community? I don't think so. In T184948#4139221, @Addshore wrote: I have added some lines to the graph on grafana representing these limits. F17086190: image.png Why is there a line at 60 edits per minute? Creation is at 40 per minute and edits are at 80 per minute.TASK DETAILhttps://phabricator.wikimedia.org/T184948EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Ladsgroup, hooCc: Stashbot, Edgars2007, Daniel_Mietchen, Pasleim, Magnus, abian, Lucas_Werkmeister_WMDE, ArtixKreiger, Multichill, Mahir256, Framawiki, hoo, Sjoerddebruin, Addshore, Ladsgroup, gerritbot, Aklapper, Lydia_Pintscher, Versusxo, Majesticalreaper22, Giuliamocci, Adrian1985, Cpaulf30, Lahi, Gq86, Baloch007, Darkminds3113, Lordiis, GoranSMilovanovic, lisong, Adik2382, Jayprakash12345, Th3d3v1ls, Ramalepe, Liugev6, QZanden, Zoranzoki21, LawExplorer, Lewizho99, Maathavan, DatGuy, Devwaker, Niklitov, Urbanecm, JEumerus, Tulsi_Bhagat, Wong128hk, Luke081515, SimmeD, biplabanand, Wikidata-bugs, Snowolf, aude, Dcljr, Jdforrester-WMF, Matanya, Mbch331, Rxy, Jay8g, Krenair___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T184948: limit page creation and edit rate on Wikidata
Addshore added a comment. I have added some lines to the graph on grafana representing these limits. F17086190: image.pngTASK DETAILhttps://phabricator.wikimedia.org/T184948EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Ladsgroup, AddshoreCc: Stashbot, Edgars2007, Daniel_Mietchen, Pasleim, Magnus, abian, Lucas_Werkmeister_WMDE, ArtixKreiger, Multichill, Mahir256, Framawiki, hoo, Sjoerddebruin, Addshore, Ladsgroup, gerritbot, Aklapper, Lydia_Pintscher, Versusxo, Majesticalreaper22, Giuliamocci, Adrian1985, Cpaulf30, Lahi, Gq86, Baloch007, Darkminds3113, Lordiis, GoranSMilovanovic, lisong, Adik2382, Jayprakash12345, Th3d3v1ls, Ramalepe, Liugev6, QZanden, Zoranzoki21, LawExplorer, Lewizho99, Maathavan, DatGuy, Devwaker, Niklitov, Urbanecm, JEumerus, Tulsi_Bhagat, Wong128hk, Luke081515, SimmeD, biplabanand, Wikidata-bugs, Snowolf, aude, Dcljr, Jdforrester-WMF, Matanya, Mbch331, Rxy, Jay8g, Krenair___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T184948: limit page creation and edit rate on Wikidata
Lucas_Werkmeister_WMDE added a comment. Did we announce this limit to the community?TASK DETAILhttps://phabricator.wikimedia.org/T184948EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Ladsgroup, Lucas_Werkmeister_WMDECc: Stashbot, Edgars2007, Daniel_Mietchen, Pasleim, Magnus, abian, Lucas_Werkmeister_WMDE, ArtixKreiger, Multichill, Mahir256, Framawiki, hoo, Sjoerddebruin, Addshore, Ladsgroup, gerritbot, Aklapper, Lydia_Pintscher, Versusxo, Majesticalreaper22, Giuliamocci, Adrian1985, Cpaulf30, Lahi, Gq86, Baloch007, Darkminds3113, Lordiis, GoranSMilovanovic, lisong, Adik2382, Jayprakash12345, Th3d3v1ls, Ramalepe, Liugev6, QZanden, Zoranzoki21, LawExplorer, Lewizho99, Maathavan, DatGuy, Devwaker, Niklitov, Urbanecm, JEumerus, Tulsi_Bhagat, Wong128hk, Luke081515, SimmeD, biplabanand, Wikidata-bugs, Snowolf, aude, Dcljr, Jdforrester-WMF, Matanya, Mbch331, Rxy, Jay8g, Krenair___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T184948: limit page creation and edit rate on Wikidata
Stashbot added a comment. Mentioned in SAL (#wikimedia-operations) [2018-04-18T13:16:53Z] Synchronized wmf-config/InitialiseSettings.php: [[gerrit:427156|Limit page creation and edit rate on Wikidata (T184948)]] (duration: 01m 17s)TASK DETAILhttps://phabricator.wikimedia.org/T184948EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Ladsgroup, StashbotCc: Stashbot, Edgars2007, Daniel_Mietchen, Pasleim, Magnus, abian, Lucas_Werkmeister_WMDE, ArtixKreiger, Multichill, Mahir256, Framawiki, hoo, Sjoerddebruin, Addshore, Ladsgroup, gerritbot, Aklapper, Lydia_Pintscher, Versusxo, Majesticalreaper22, Giuliamocci, Adrian1985, Cpaulf30, Lahi, Gq86, Baloch007, Darkminds3113, Lordiis, GoranSMilovanovic, lisong, Adik2382, Jayprakash12345, Th3d3v1ls, Ramalepe, Liugev6, QZanden, Zoranzoki21, LawExplorer, Lewizho99, Maathavan, DatGuy, Devwaker, Niklitov, Urbanecm, JEumerus, Tulsi_Bhagat, Wong128hk, Luke081515, SimmeD, biplabanand, Wikidata-bugs, Snowolf, aude, Dcljr, Jdforrester-WMF, Matanya, Mbch331, Rxy, Jay8g, Krenair___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T184948: limit page creation and edit rate on Wikidata
gerritbot added a comment. Change 427156 merged by jenkins-bot: [operations/mediawiki-config@master] Limit page creation and edit rate on Wikidata https://gerrit.wikimedia.org/r/427156TASK DETAILhttps://phabricator.wikimedia.org/T184948EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Ladsgroup, gerritbotCc: Edgars2007, Daniel_Mietchen, Pasleim, Magnus, abian, Lucas_Werkmeister_WMDE, ArtixKreiger, Multichill, Mahir256, Framawiki, hoo, Sjoerddebruin, Addshore, Ladsgroup, gerritbot, Aklapper, Lydia_Pintscher, Versusxo, Majesticalreaper22, Giuliamocci, Adrian1985, Cpaulf30, Lahi, Gq86, Baloch007, Darkminds3113, Lordiis, GoranSMilovanovic, lisong, Adik2382, Jayprakash12345, Th3d3v1ls, Ramalepe, Liugev6, QZanden, Zoranzoki21, LawExplorer, Lewizho99, Maathavan, DatGuy, Devwaker, Niklitov, Urbanecm, JEumerus, Tulsi_Bhagat, Wong128hk, Luke081515, SimmeD, biplabanand, Wikidata-bugs, Snowolf, aude, Dcljr, Jdforrester-WMF, Matanya, Mbch331, Rxy, Jay8g, Krenair___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T184948: limit page creation and edit rate on Wikidata
gerritbot added a comment. Change 427156 had a related patch set uploaded (by Ladsgroup; owner: Amir Sarabadani): [operations/mediawiki-config@master] Limit page creation and edit rate on Wikidata https://gerrit.wikimedia.org/r/427156TASK DETAILhttps://phabricator.wikimedia.org/T184948EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Ladsgroup, gerritbotCc: Edgars2007, Daniel_Mietchen, Pasleim, Magnus, abian, Lucas_Werkmeister_WMDE, ArtixKreiger, Multichill, Mahir256, Framawiki, hoo, Sjoerddebruin, Addshore, Ladsgroup, gerritbot, Aklapper, Lydia_Pintscher, Versusxo, Majesticalreaper22, Giuliamocci, Adrian1985, Cpaulf30, Lahi, Gq86, Baloch007, Darkminds3113, Lordiis, GoranSMilovanovic, lisong, Adik2382, Jayprakash12345, Th3d3v1ls, Ramalepe, Liugev6, QZanden, Zoranzoki21, LawExplorer, Lewizho99, Maathavan, DatGuy, Devwaker, Niklitov, Urbanecm, JEumerus, Tulsi_Bhagat, Wong128hk, Luke081515, SimmeD, biplabanand, Wikidata-bugs, Snowolf, aude, Dcljr, Jdforrester-WMF, Matanya, Mbch331, Rxy, Jay8g, Krenair___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T184948: limit page creation and edit rate on Wikidata
Lydia_Pintscher added a comment. @hoo, @Ladsgroup and I sat down together and talked it through. I updated the task description accordingly with what I believe is the minimum requirement to keep the infrastructure sane.TASK DETAILhttps://phabricator.wikimedia.org/T184948EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Lydia_PintscherCc: Edgars2007, Daniel_Mietchen, Pasleim, Magnus, abian, Lucas_Werkmeister_WMDE, ArtixKreiger, Multichill, Mahir256, Framawiki, hoo, Sjoerddebruin, Addshore, Ladsgroup, gerritbot, Aklapper, Lydia_Pintscher, Versusxo, Majesticalreaper22, Giuliamocci, Adrian1985, Cpaulf30, Lahi, Gq86, Baloch007, Darkminds3113, Lordiis, GoranSMilovanovic, Adik2382, Jayprakash12345, Th3d3v1ls, Ramalepe, Liugev6, QZanden, Zoranzoki21, LawExplorer, Lewizho99, Maathavan, DatGuy, Devwaker, Niklitov, Urbanecm, JEumerus, Tulsi_Bhagat, Wong128hk, Luke081515, SimmeD, biplabanand, Wikidata-bugs, Snowolf, aude, Dcljr, Jdforrester-WMF, Matanya, Mbch331, Rxy, Jay8g, Krenair___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T184948: limit page creation and edit rate on Wikidata
hoo added a comment. I just added documentation for the &can-bypass key: https://www.mediawiki.org/w/index.php?diff=2757174&oldid=2675575TASK DETAILhttps://phabricator.wikimedia.org/T184948EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: hooCc: Edgars2007, Daniel_Mietchen, Pasleim, Magnus, abian, Lucas_Werkmeister_WMDE, ArtixKreiger, Multichill, Mahir256, Framawiki, hoo, Sjoerddebruin, Addshore, Ladsgroup, gerritbot, Aklapper, Lydia_Pintscher, Versusxo, Majesticalreaper22, Giuliamocci, Adrian1985, Cpaulf30, Lahi, Gq86, Baloch007, Darkminds3113, Lordiis, GoranSMilovanovic, Adik2382, Jayprakash12345, Th3d3v1ls, Ramalepe, Liugev6, QZanden, Zoranzoki21, LawExplorer, Lewizho99, Maathavan, DatGuy, Devwaker, Niklitov, Urbanecm, JEumerus, Tulsi_Bhagat, Wong128hk, Luke081515, SimmeD, biplabanand, Wikidata-bugs, Snowolf, aude, Dcljr, Jdforrester-WMF, Matanya, Mbch331, Rxy, Jay8g, Krenair___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T184948: limit page creation and edit rate on Wikidata
hoo added a comment. Actually, it seems we could also have only specific limits that are not bypassable. While this is not documented (as far as I can tell), one can add '&can-bypass' => false to a specific rate limit action, like the following: > print_r($wgRateLimits); Array ( … [badoath] => Array ( [&can-bypass] => [user] => Array ( [0] => 10 [1] => 60 ) ) ) (for details, see User::pingLimiter) That would mean that we can limit only the number of edits… if we find reasonable numbers.TASK DETAILhttps://phabricator.wikimedia.org/T184948EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: hooCc: Edgars2007, Daniel_Mietchen, Pasleim, Magnus, abian, Lucas_Werkmeister_WMDE, ArtixKreiger, Multichill, Mahir256, Framawiki, hoo, Sjoerddebruin, Addshore, Ladsgroup, gerritbot, Aklapper, Lydia_Pintscher, Versusxo, Majesticalreaper22, Giuliamocci, Adrian1985, Cpaulf30, Lahi, Gq86, Baloch007, Darkminds3113, Lordiis, GoranSMilovanovic, Adik2382, Jayprakash12345, Th3d3v1ls, Ramalepe, Liugev6, QZanden, Zoranzoki21, LawExplorer, Lewizho99, Maathavan, DatGuy, Devwaker, Niklitov, Urbanecm, JEumerus, Tulsi_Bhagat, Wong128hk, Luke081515, SimmeD, biplabanand, Wikidata-bugs, Snowolf, aude, Dcljr, Jdforrester-WMF, Matanya, Mbch331, Rxy, Jay8g, Krenair___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T184948: limit page creation and edit rate on Wikidata
hoo added a comment. Note: We currently also have dispatch problems while no one is going faster than 75 edits per minute (as far as I can tell)TASK DETAILhttps://phabricator.wikimedia.org/T184948EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: hooCc: Edgars2007, Daniel_Mietchen, Pasleim, Magnus, abian, Lucas_Werkmeister_WMDE, ArtixKreiger, Multichill, Mahir256, Framawiki, hoo, Sjoerddebruin, Addshore, Ladsgroup, gerritbot, Aklapper, Lydia_Pintscher, Versusxo, Majesticalreaper22, Giuliamocci, Adrian1985, Cpaulf30, Lahi, Gq86, Baloch007, Darkminds3113, Lordiis, GoranSMilovanovic, Adik2382, Jayprakash12345, Th3d3v1ls, Ramalepe, Liugev6, QZanden, Zoranzoki21, LawExplorer, Lewizho99, Maathavan, DatGuy, Devwaker, Niklitov, Urbanecm, JEumerus, Tulsi_Bhagat, Wong128hk, Luke081515, SimmeD, biplabanand, Wikidata-bugs, Snowolf, aude, Dcljr, Jdforrester-WMF, Matanya, Mbch331, Rxy, Jay8g, Krenair___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T184948: limit page creation and edit rate on Wikidata
Addshore added a comment. In T184948#4100013, @Magnus wrote: I like the single-shot API request, but additionally I'd like a notification in the edit reply about server-side issues, as is done for replag at the moment. That could save me the additional API query before. Should be possible to to add something like wblag=1 in the same way we have maxlag=1 IMO. Also in terms of Mediawiki, perhaps it would be a good time to introduce other types of lag / reasons to slow down to core.TASK DETAILhttps://phabricator.wikimedia.org/T184948EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: AddshoreCc: Edgars2007, Daniel_Mietchen, Pasleim, Magnus, abian, Lucas_Werkmeister_WMDE, ArtixKreiger, Multichill, Mahir256, Framawiki, hoo, Sjoerddebruin, Addshore, Ladsgroup, gerritbot, Aklapper, Lydia_Pintscher, Versusxo, Majesticalreaper22, Giuliamocci, Adrian1985, Cpaulf30, Lahi, Gq86, Baloch007, Darkminds3113, Lordiis, GoranSMilovanovic, Adik2382, Jayprakash12345, Th3d3v1ls, Ramalepe, Liugev6, QZanden, Zoranzoki21, LawExplorer, Lewizho99, Maathavan, DatGuy, Devwaker, Niklitov, Urbanecm, JEumerus, Tulsi_Bhagat, Wong128hk, Luke081515, SimmeD, biplabanand, Wikidata-bugs, Snowolf, aude, Dcljr, Jdforrester-WMF, Matanya, Mbch331, Rxy, Jay8g, Krenair___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T184948: limit page creation and edit rate on Wikidata
Magnus added a comment. @Addshore It depends on what the actual issue with creating/editing is. If it's database-related (most likely), then having a few sleeping processes for load-dependent server-side delays might be a viable solution. I like the single-shot API request, but additionally I'd like a notification in the edit reply about server-side issues, as is done for replag at the moment. That could save me the additional API query before.TASK DETAILhttps://phabricator.wikimedia.org/T184948EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: MagnusCc: Edgars2007, Daniel_Mietchen, Pasleim, Magnus, abian, Lucas_Werkmeister_WMDE, ArtixKreiger, Multichill, Mahir256, Framawiki, hoo, Sjoerddebruin, Addshore, Ladsgroup, gerritbot, Aklapper, Lydia_Pintscher, Versusxo, Majesticalreaper22, Giuliamocci, Adrian1985, Cpaulf30, Lahi, Gq86, Baloch007, Darkminds3113, Lordiis, GoranSMilovanovic, Adik2382, Jayprakash12345, Th3d3v1ls, Ramalepe, Liugev6, QZanden, Zoranzoki21, LawExplorer, Lewizho99, Maathavan, DatGuy, Devwaker, Niklitov, Urbanecm, JEumerus, Tulsi_Bhagat, Wong128hk, Luke081515, SimmeD, biplabanand, Wikidata-bugs, Snowolf, aude, Dcljr, Jdforrester-WMF, Matanya, Mbch331, Rxy, Jay8g, Krenair___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T184948: limit page creation and edit rate on Wikidata
ArtixKreiger added a comment. So can i get back to running QuickStatement batches?TASK DETAILhttps://phabricator.wikimedia.org/T184948EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: ArtixKreigerCc: Edgars2007, Daniel_Mietchen, Pasleim, Magnus, abian, Lucas_Werkmeister_WMDE, ArtixKreiger, Multichill, Mahir256, Framawiki, hoo, Sjoerddebruin, Addshore, Ladsgroup, gerritbot, Aklapper, Lydia_Pintscher, Versusxo, Majesticalreaper22, Giuliamocci, Adrian1985, Cpaulf30, Lahi, Gq86, Baloch007, Darkminds3113, Lordiis, GoranSMilovanovic, Adik2382, Jayprakash12345, Th3d3v1ls, Ramalepe, Liugev6, QZanden, Zoranzoki21, LawExplorer, Lewizho99, Maathavan, DatGuy, Devwaker, Niklitov, Urbanecm, JEumerus, Tulsi_Bhagat, Wong128hk, Luke081515, SimmeD, biplabanand, Wikidata-bugs, Snowolf, aude, Dcljr, Jdforrester-WMF, Matanya, Mbch331, Rxy, Jay8g, Krenair___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T184948: limit page creation and edit rate on Wikidata
Multichill added a comment. In T184948#4089557, @Magnus wrote: So, either WMF expose the "overload status" so we can throttle on demand, or WMF delay the API response accordingly, so we don't have to do anything, or WMF scale to demand :-) Yep. WMF or WMDE I guess. @Lydia_Pintscher can you discus the best approach with your team and file the relevant bugs?TASK DETAILhttps://phabricator.wikimedia.org/T184948EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: MultichillCc: Edgars2007, Daniel_Mietchen, Pasleim, Magnus, abian, Lucas_Werkmeister_WMDE, ArtixKreiger, Multichill, Mahir256, Framawiki, hoo, Sjoerddebruin, Addshore, Ladsgroup, gerritbot, Aklapper, Lydia_Pintscher, Versusxo, Majesticalreaper22, Giuliamocci, Adrian1985, Cpaulf30, Lahi, Gq86, Baloch007, Darkminds3113, Lordiis, GoranSMilovanovic, Adik2382, Jayprakash12345, Th3d3v1ls, Ramalepe, Liugev6, QZanden, Zoranzoki21, LawExplorer, Lewizho99, Maathavan, DatGuy, Devwaker, Niklitov, Urbanecm, JEumerus, Tulsi_Bhagat, Wong128hk, Luke081515, SimmeD, biplabanand, Wikidata-bugs, Snowolf, aude, Dcljr, Jdforrester-WMF, Matanya, Mbch331, Rxy, Jay8g, Krenair___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T184948: limit page creation and edit rate on Wikidata
Magnus added a comment. So, either WMF expose the "overload status" so we can throttle on demand, or WMF delay the API response accordingly, so we don't have to do anything, or WMF scale to demand :-) TASK DETAILhttps://phabricator.wikimedia.org/T184948EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: MagnusCc: Edgars2007, Daniel_Mietchen, Pasleim, Magnus, abian, Lucas_Werkmeister_WMDE, ArtixKreiger, Multichill, Mahir256, Framawiki, hoo, Sjoerddebruin, Addshore, Ladsgroup, gerritbot, Aklapper, Lydia_Pintscher, Versusxo, Majesticalreaper22, Giuliamocci, Adrian1985, Cpaulf30, Lahi, Gq86, Baloch007, Darkminds3113, Lordiis, GoranSMilovanovic, Adik2382, Jayprakash12345, Th3d3v1ls, Ramalepe, Liugev6, QZanden, Zoranzoki21, LawExplorer, Lewizho99, Maathavan, DatGuy, Devwaker, Niklitov, Urbanecm, JEumerus, Tulsi_Bhagat, Wong128hk, Luke081515, SimmeD, biplabanand, Wikidata-bugs, Snowolf, aude, Dcljr, Jdforrester-WMF, Matanya, Mbch331, Rxy, Jay8g, Krenair___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T184948: limit page creation and edit rate on Wikidata
Multichill added a comment. Hi Magnus, In T184948#4087139, @Magnus wrote: That is a total of three different rate limitations (two dynamic, one hardcoded fallback). Personally, I consider that beyond due diligence, verging on paranoia. Therefore, I consider this matter resolved, as far as my tools are concerned. All three of them are lag based. In T184948#4075653, @Ladsgroup wrote: Too many edits in too small amount of time can affect infra in several ways, one is replag, one is dispatch lag, one is jobqueue size. If edit is about creating new items it can also affect the infra in some other ways too as we want to be consistent about Q-ids and do not assign same Q-ids to two items. If I understand this correctly, the lag can be low, but the systems can still be overloaded.TASK DETAILhttps://phabricator.wikimedia.org/T184948EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: MultichillCc: Edgars2007, Daniel_Mietchen, Pasleim, Magnus, abian, Lucas_Werkmeister_WMDE, ArtixKreiger, Multichill, Mahir256, Framawiki, hoo, Sjoerddebruin, Addshore, Ladsgroup, gerritbot, Aklapper, Lydia_Pintscher, Versusxo, Majesticalreaper22, Giuliamocci, Adrian1985, Cpaulf30, Lahi, Gq86, Baloch007, Darkminds3113, Lordiis, GoranSMilovanovic, Adik2382, Jayprakash12345, Th3d3v1ls, Ramalepe, Liugev6, QZanden, Zoranzoki21, LawExplorer, Lewizho99, Maathavan, DatGuy, Devwaker, Niklitov, Urbanecm, JEumerus, Tulsi_Bhagat, Wong128hk, Luke081515, SimmeD, biplabanand, Wikidata-bugs, Snowolf, aude, Dcljr, Jdforrester-WMF, Matanya, Mbch331, Rxy, Jay8g, Krenair___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T184948: limit page creation and edit rate on Wikidata
Magnus added a comment. OK, let me summarize my situation, as of today: Essentially, all my Wikidata edits (both bots and _javascript_ on Toolforge) run through the same OAuth PHP class I wrote Every single edit (including "create item") runs with a "maxlag=5" parameter. If the edit fails because of maxlag, it will wait 5 seconds, then try again, until it succeeds. This mechanism has been in place for months. As of today, after every edit (including "create item"), it will check the current lag on Wikidata. If the current lag is >1 sec, it will sleep three times the lag (in int seconds; so, if lag is 1.5 sec, it will sleep int(3*1.5)=4 seconds) before returning. If that check fails for some reason, it will sleep a pre-defined time for each edit type (create item=2sec, edit=1 sec) That is a total of three different rate limitations (two dynamic, one hardcoded fallback). Personally, I consider that beyond due diligence, verging on paranoia. Therefore, I consider this matter resolved, as far as my tools are concerned.TASK DETAILhttps://phabricator.wikimedia.org/T184948EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: MagnusCc: Edgars2007, Daniel_Mietchen, Pasleim, Magnus, abian, Lucas_Werkmeister_WMDE, ArtixKreiger, Multichill, Mahir256, Framawiki, hoo, Sjoerddebruin, Addshore, Ladsgroup, gerritbot, Aklapper, Lydia_Pintscher, Versusxo, Majesticalreaper22, Giuliamocci, Adrian1985, Cpaulf30, Lahi, Gq86, Baloch007, Darkminds3113, Lordiis, GoranSMilovanovic, Adik2382, Jayprakash12345, Th3d3v1ls, Ramalepe, Liugev6, QZanden, Zoranzoki21, LawExplorer, Lewizho99, Maathavan, DatGuy, Devwaker, Niklitov, Urbanecm, JEumerus, Tulsi_Bhagat, Wong128hk, Luke081515, SimmeD, biplabanand, Wikidata-bugs, Snowolf, aude, Dcljr, Jdforrester-WMF, Matanya, Mbch331, Rxy, Jay8g, Krenair___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T184948: limit page creation and edit rate on Wikidata
ArtixKreiger added a comment. this guy artix waits for the rate limit to implemented so he can go back and finish is batch job.TASK DETAILhttps://phabricator.wikimedia.org/T184948EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: ArtixKreigerCc: Edgars2007, Daniel_Mietchen, Pasleim, Magnus, abian, Lucas_Werkmeister_WMDE, ArtixKreiger, Multichill, Mahir256, Framawiki, hoo, Sjoerddebruin, Addshore, Ladsgroup, gerritbot, Aklapper, Lydia_Pintscher, Versusxo, Majesticalreaper22, Tamgue, Giuliamocci, Adrian1985, Cpaulf30, Lahi, Gq86, Baloch007, Darkminds3113, Lordiis, GoranSMilovanovic, Adik2382, Jayprakash12345, Th3d3v1ls, Ramalepe, Liugev6, QZanden, Zoranzoki21, LawExplorer, Lewizho99, Maathavan, DatGuy, Devwaker, Niklitov, Urbanecm, JEumerus, Tulsi_Bhagat, Wong128hk, Luke081515, SimmeD, biplabanand, Wikidata-bugs, Snowolf, aude, Dcljr, Jdforrester-WMF, Matanya, Mbch331, Rxy, Jay8g, Krenair___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T184948: limit page creation and edit rate on Wikidata
Daniel_Mietchen added a comment. I like the suggestion of adding a load-dependent server-side delay for API edits.TASK DETAILhttps://phabricator.wikimedia.org/T184948EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Daniel_MietchenCc: Daniel_Mietchen, Pasleim, Magnus, abian, Lucas_Werkmeister_WMDE, ArtixKreiger, Multichill, Mahir256, Framawiki, hoo, Sjoerddebruin, Addshore, Ladsgroup, gerritbot, Aklapper, Lydia_Pintscher, Versusxo, Majesticalreaper22, Tamgue, Giuliamocci, Adrian1985, Cpaulf30, Lahi, Gq86, Baloch007, Darkminds3113, Lordiis, GoranSMilovanovic, Adik2382, Jayprakash12345, Th3d3v1ls, Ramalepe, Liugev6, QZanden, Zoranzoki21, LawExplorer, Lewizho99, Maathavan, DatGuy, Devwaker, Niklitov, Urbanecm, JEumerus, Tulsi_Bhagat, Wong128hk, Luke081515, SimmeD, biplabanand, Wikidata-bugs, Snowolf, aude, Dcljr, Jdforrester-WMF, Matanya, Mbch331, Rxy, Jay8g, Krenair___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T184948: limit page creation and edit rate on Wikidata
Multichill added a comment. In T184948#4079500, @Magnus wrote: I have added a 5 second delay after item creations. I also added a mechanism to have a delay after any other edit type. Note that this limits single threads only. QuickStatements, various SourceMD webtools and bots, Mix'n'match sync etc. can still add up. Also, this does not effect my _javascript_ tools on Wikidata proper. Edit: Rate limiter diff Thanks for that. As for the edit delay. Sleep is always in seconds and the subsecond sleep functions all seem to have issues. I suggest you use rand() to sleep a second every n edits. That way you can do something between no throttle and waiting a second after every edit.TASK DETAILhttps://phabricator.wikimedia.org/T184948EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: MultichillCc: Pasleim, Magnus, abian, Lucas_Werkmeister_WMDE, ArtixKreiger, Multichill, Mahir256, Framawiki, hoo, Sjoerddebruin, Addshore, Ladsgroup, gerritbot, Aklapper, Lydia_Pintscher, Versusxo, Majesticalreaper22, Tamgue, Giuliamocci, Adrian1985, Cpaulf30, Lahi, Gq86, Baloch007, Darkminds3113, Lordiis, GoranSMilovanovic, Adik2382, Jayprakash12345, Th3d3v1ls, Ramalepe, Liugev6, QZanden, Zoranzoki21, LawExplorer, Lewizho99, Maathavan, DatGuy, Devwaker, Niklitov, Urbanecm, JEumerus, Tulsi_Bhagat, Wong128hk, Luke081515, SimmeD, biplabanand, Wikidata-bugs, Snowolf, aude, Dcljr, Jdforrester-WMF, Matanya, Mbch331, Rxy, Jay8g, Krenair___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T184948: limit page creation and edit rate on Wikidata
Magnus added a comment. For one tool, per user, per tab? Sure. I'd just set both item creation and edit to 1 second delay. For one tool, all users? No. At least, not easily. Question: WMF knows best when their servers are overloaded. Why not just add a delay for the API response when the servers are under duress? No need for every volunteer to scramble and sink time into altering their tools to preserve the WMF from users trying to get stuff done. I imagine adding something like 0.2 or 0.5 seconds sleep to *every* API edit when under duress might ease the load quickly and quietly. Why make WMFs problem my problem without need?TASK DETAILhttps://phabricator.wikimedia.org/T184948EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: MagnusCc: Pasleim, Magnus, abian, Lucas_Werkmeister_WMDE, ArtixKreiger, Multichill, Mahir256, Framawiki, hoo, Sjoerddebruin, Addshore, Ladsgroup, gerritbot, Aklapper, Lydia_Pintscher, Versusxo, Majesticalreaper22, Tamgue, Giuliamocci, Adrian1985, Cpaulf30, Lahi, Gq86, Baloch007, Darkminds3113, Lordiis, GoranSMilovanovic, Adik2382, Jayprakash12345, Th3d3v1ls, Ramalepe, Liugev6, QZanden, Zoranzoki21, LawExplorer, Lewizho99, Maathavan, DatGuy, Devwaker, Niklitov, Urbanecm, JEumerus, Tulsi_Bhagat, Wong128hk, Luke081515, SimmeD, biplabanand, Wikidata-bugs, Snowolf, aude, Dcljr, Jdforrester-WMF, Matanya, Mbch331, Rxy, Jay8g, Krenair___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T184948: limit page creation and edit rate on Wikidata
ArtixKreiger added a comment. Magnus, so what about the edit rate limit of 100/min? Is it hard coded?TASK DETAILhttps://phabricator.wikimedia.org/T184948EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: ArtixKreigerCc: Magnus, abian, Lucas_Werkmeister_WMDE, ArtixKreiger, Multichill, Mahir256, Framawiki, hoo, Sjoerddebruin, Addshore, Ladsgroup, gerritbot, Aklapper, Lydia_Pintscher, Versusxo, Majesticalreaper22, Tamgue, Giuliamocci, Adrian1985, Cpaulf30, Lahi, Gq86, Baloch007, Darkminds3113, Lordiis, GoranSMilovanovic, Adik2382, Jayprakash12345, Th3d3v1ls, Ramalepe, Liugev6, QZanden, Zoranzoki21, LawExplorer, Lewizho99, Maathavan, DatGuy, Devwaker, Niklitov, Urbanecm, JEumerus, Tulsi_Bhagat, Wong128hk, Luke081515, SimmeD, biplabanand, Wikidata-bugs, Snowolf, aude, Dcljr, Jdforrester-WMF, Matanya, Mbch331, Rxy, Jay8g, Krenair___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T184948: limit page creation and edit rate on Wikidata
Magnus added a comment. I have added a 5 second delay after item creations. I also added a mechanism to have a delay after any other edit type. Note that this limits single threads only. QuickStatements, various SourceMD webtools and bots, Mix'n'match sync etc. can still add up. Also, this does not effect my _javascript_ tools on Wikidata proper.TASK DETAILhttps://phabricator.wikimedia.org/T184948EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: MagnusCc: Magnus, abian, Lucas_Werkmeister_WMDE, ArtixKreiger, Multichill, Mahir256, Framawiki, hoo, Sjoerddebruin, Addshore, Ladsgroup, gerritbot, Aklapper, Lydia_Pintscher, Versusxo, Majesticalreaper22, Tamgue, Giuliamocci, Adrian1985, Cpaulf30, Lahi, Gq86, Baloch007, Darkminds3113, Lordiis, GoranSMilovanovic, Adik2382, Jayprakash12345, Th3d3v1ls, Ramalepe, Liugev6, QZanden, Zoranzoki21, LawExplorer, Lewizho99, Maathavan, DatGuy, Devwaker, Niklitov, Urbanecm, JEumerus, Tulsi_Bhagat, Wong128hk, Luke081515, SimmeD, biplabanand, Wikidata-bugs, Snowolf, aude, Dcljr, Jdforrester-WMF, Matanya, Mbch331, Rxy, Jay8g, Krenair___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T184948: limit page creation and edit rate on Wikidata
Magnus added a comment. Thanks to @Multichill for pointing out this discussion to me... As for [[Bus factor]], I have always welcomes co-maintainers on my tools, and have repeatedly suggested WMF involvement, anywhere from taking over/rewriting to just becoming co-maintainers of important tools. (Others are welcome too, bu I understand why other volunteers don't want to add themselves the burden of co-maintaining). Now, for the original post: Does this concern just item creation, or any kind of edit? I am quite willing to limit the big guns (in fact, I already have some of them limited) if WMF hard-/software are not up to adding a few rows to the database... ;-)TASK DETAILhttps://phabricator.wikimedia.org/T184948EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: MagnusCc: Magnus, abian, Lucas_Werkmeister_WMDE, ArtixKreiger, Multichill, Mahir256, Framawiki, hoo, Sjoerddebruin, Addshore, Ladsgroup, gerritbot, Aklapper, Lydia_Pintscher, Versusxo, Majesticalreaper22, Tamgue, Giuliamocci, Adrian1985, Cpaulf30, Lahi, Gq86, Baloch007, Darkminds3113, Lordiis, GoranSMilovanovic, Adik2382, Jayprakash12345, Th3d3v1ls, Ramalepe, Liugev6, QZanden, Zoranzoki21, LawExplorer, Lewizho99, Maathavan, DatGuy, Devwaker, Niklitov, Urbanecm, JEumerus, Tulsi_Bhagat, Wong128hk, Luke081515, SimmeD, biplabanand, Wikidata-bugs, Snowolf, aude, Dcljr, Jdforrester-WMF, Matanya, Mbch331, Rxy, Jay8g, Krenair___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T184948: limit page creation and edit rate on Wikidata
ArtixKreiger added a comment. Addshore, it seems Magnus is the only one who maintains the tool. If he feels like sleeping or do something else in his life, there would be a big set of shoes to fix, and not an easy pair of shoes to fix.TASK DETAILhttps://phabricator.wikimedia.org/T184948EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: ArtixKreigerCc: abian, Lucas_Werkmeister_WMDE, ArtixKreiger, Multichill, Mahir256, Framawiki, hoo, Sjoerddebruin, Addshore, Ladsgroup, gerritbot, Aklapper, Lydia_Pintscher, Versusxo, Majesticalreaper22, Tamgue, Giuliamocci, Adrian1985, Cpaulf30, Lahi, Gq86, Baloch007, Darkminds3113, Lordiis, GoranSMilovanovic, Adik2382, Jayprakash12345, Th3d3v1ls, Ramalepe, Liugev6, QZanden, Zoranzoki21, LawExplorer, Lewizho99, Maathavan, DatGuy, Devwaker, Niklitov, Urbanecm, JEumerus, Tulsi_Bhagat, Wong128hk, Luke081515, SimmeD, biplabanand, Wikidata-bugs, Snowolf, aude, Dcljr, Jdforrester-WMF, Matanya, Mbch331, Rxy, Jay8g, Krenair___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T184948: limit page creation and edit rate on Wikidata
Multichill added a comment. In T184948#4078538, @Lydia_Pintscher wrote: There are other tools that are going too fast like fatameh as well. This isn't something we should have to correct on a tool-by-tool basis. That's what you getting from being a bit more liberal for a long time. Tool rights have to be approved, bot rights have to be approved, all can be revoked when needed. If this really is a problem, take action or clearly identify the problematic tools/bots so admins like me can take action.TASK DETAILhttps://phabricator.wikimedia.org/T184948EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: MultichillCc: abian, Lucas_Werkmeister_WMDE, ArtixKreiger, Multichill, Mahir256, Framawiki, hoo, Sjoerddebruin, Addshore, Ladsgroup, gerritbot, Aklapper, Lydia_Pintscher, Versusxo, Majesticalreaper22, Tamgue, Giuliamocci, Adrian1985, Cpaulf30, Lahi, Gq86, Baloch007, Darkminds3113, Lordiis, GoranSMilovanovic, Adik2382, Jayprakash12345, Th3d3v1ls, Ramalepe, Liugev6, QZanden, Zoranzoki21, LawExplorer, Lewizho99, Maathavan, DatGuy, Devwaker, Niklitov, Urbanecm, JEumerus, Tulsi_Bhagat, Wong128hk, Luke081515, SimmeD, biplabanand, Wikidata-bugs, Snowolf, aude, Dcljr, Jdforrester-WMF, Matanya, Mbch331, Rxy, Jay8g, Krenair___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T184948: limit page creation and edit rate on Wikidata
Lydia_Pintscher added a comment. There are other tools that are going too fast like fatameh as well. This isn't something we should have to correct on a tool-by-tool basis.TASK DETAILhttps://phabricator.wikimedia.org/T184948EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Lydia_PintscherCc: abian, Lucas_Werkmeister_WMDE, ArtixKreiger, Multichill, Mahir256, Framawiki, hoo, Sjoerddebruin, Addshore, Ladsgroup, gerritbot, Aklapper, Lydia_Pintscher, Versusxo, Majesticalreaper22, Tamgue, Giuliamocci, Adrian1985, Cpaulf30, Lahi, Gq86, Baloch007, Darkminds3113, Lordiis, GoranSMilovanovic, Adik2382, Jayprakash12345, Th3d3v1ls, Ramalepe, Liugev6, QZanden, Zoranzoki21, LawExplorer, Lewizho99, Maathavan, DatGuy, Devwaker, Niklitov, Urbanecm, JEumerus, Tulsi_Bhagat, Wong128hk, Luke081515, SimmeD, biplabanand, Wikidata-bugs, Snowolf, aude, Dcljr, Jdforrester-WMF, Matanya, Mbch331, Rxy, Jay8g, Krenair___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T184948: limit page creation and edit rate on Wikidata
Multichill added a comment. In T184948#4075653, @Ladsgroup wrote: In T184948#4059447, @Multichill wrote: In T184948#4052662, @Ladsgroup wrote: I personally would like to strip the right from the bots and enforce a rate limit for them as they just don't care about what we say. If bots ignore replag ( https://meta.wikimedia.org/wiki/Bot_policy#Edit_throttle_and_peak_hours ) these bots should just be stripped of the botflag. The good shouldn't suffer because of a small number of problematic users. Too many edits in too small amount of time can affect infra in several ways, one is replag, one is dispatch lag, one is jobqueue size. If edit is about creating new items it can also affect the infra in some other ways too as we want to be consistent about Q-ids and do not assign same Q-ids to two items. Overall, if community stick to a rule of thumb like *max* 60 edits / min. and *max* 10 new items/min. and community actually enforces that (and don't wait until the whole infra falls down and we ping them so they stop), there wouldn't be a need to have such ticket in here in the first place but this continues to happen all the time. It's basically Widar (quick statements) that is going to fast, see last comment at https://www.wikidata.org/w/index.php?title=Wikidata:Project_chat&oldid=655772082#Rate_speed_for_QuickStatements on how to fix that. Or are you aware of any other tools that are going to fast. Nuclear option is to disable Widar and enable it again after the speed issues have been fixed.TASK DETAILhttps://phabricator.wikimedia.org/T184948EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: MultichillCc: abian, Lucas_Werkmeister_WMDE, ArtixKreiger, Multichill, Mahir256, Framawiki, hoo, Sjoerddebruin, Addshore, Ladsgroup, gerritbot, Aklapper, Lydia_Pintscher, Versusxo, Majesticalreaper22, Tamgue, Giuliamocci, Adrian1985, Cpaulf30, Lahi, Gq86, Baloch007, Darkminds3113, Lordiis, GoranSMilovanovic, Adik2382, Jayprakash12345, Th3d3v1ls, Ramalepe, Liugev6, QZanden, Zoranzoki21, LawExplorer, Lewizho99, Maathavan, DatGuy, Devwaker, Niklitov, Urbanecm, JEumerus, Tulsi_Bhagat, Wong128hk, Luke081515, SimmeD, biplabanand, Wikidata-bugs, Snowolf, aude, Dcljr, Jdforrester-WMF, Matanya, Mbch331, Rxy, Jay8g, Krenair___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T184948: limit page creation and edit rate on Wikidata
Ladsgroup added a comment. In T184948#4059447, @Multichill wrote: In T184948#4052662, @Ladsgroup wrote: I personally would like to strip the right from the bots and enforce a rate limit for them as they just don't care about what we say. If bots ignore replag ( https://meta.wikimedia.org/wiki/Bot_policy#Edit_throttle_and_peak_hours ) these bots should just be stripped of the botflag. The good shouldn't suffer because of a small number of problematic users. Too many edits in too small amount of time can affect infra in several ways, one is replag, one is dispatch lag, one is jobqueue size. If edit is about creating new items it can also affect the infra in some other ways too as we want to be consistent about Q-ids and do not assign same Q-ids to two items. Overall, if community stick to a rule of thumb like *max* 60 edits / min. and *max* 10 new items/min. and community actually enforces that (and don't wait until the whole infra falls down and we ping them so they stop), there wouldn't be a need to have such ticket in here in the first place but this continues to happen all the time.TASK DETAILhttps://phabricator.wikimedia.org/T184948EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: LadsgroupCc: abian, Lucas_Werkmeister_WMDE, ArtixKreiger, Multichill, Mahir256, Framawiki, hoo, Sjoerddebruin, Addshore, Ladsgroup, gerritbot, Aklapper, Lydia_Pintscher, Versusxo, Majesticalreaper22, Tamgue, Giuliamocci, Adrian1985, Cpaulf30, Lahi, Gq86, Baloch007, Darkminds3113, Lordiis, GoranSMilovanovic, Adik2382, Jayprakash12345, Th3d3v1ls, Ramalepe, Liugev6, QZanden, Zoranzoki21, LawExplorer, Lewizho99, Maathavan, DatGuy, Devwaker, Niklitov, Urbanecm, JEumerus, Tulsi_Bhagat, Wong128hk, Luke081515, SimmeD, biplabanand, Wikidata-bugs, Snowolf, aude, Dcljr, Jdforrester-WMF, Matanya, Mbch331, Rxy, Jay8g, Krenair___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T184948: limit page creation and edit rate on Wikidata
Addshore added a comment. In T184948#4064358, @ArtixKreiger wrote: Whats the point of noratelimit if admins and bots are rate limited? We can always have multiple limits. Also noratelimit is part of mediawiki, we didn't create it for wikibase, but we are now finding out that perhaps it is a bad idea to have this on Wikidata. However, yes. I agree with everyone that there should be some controllable edit rate. This is a random suggestion, but feel free to ignore. Perhaps, for "everyone", there is an enforced 100/min edit rate. For admins and "trusted users", 100/min is default since they KNOW not to do it and thus they get an option to set it higher. I think a blanket limit should work for everyone, unless we can come up with a concrete reason that any group would need to edit faster. I also suggest a co-share agreement between Magnus and WMF team due to [[Bus factor]]. I don't really understand this commentTASK DETAILhttps://phabricator.wikimedia.org/T184948EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: AddshoreCc: Lucas_Werkmeister_WMDE, ArtixKreiger, Multichill, Mahir256, Framawiki, hoo, Sjoerddebruin, Addshore, Ladsgroup, gerritbot, Aklapper, Lydia_Pintscher, Versusxo, Majesticalreaper22, Tamgue, Giuliamocci, Adrian1985, Cpaulf30, Lahi, Gq86, Baloch007, Darkminds3113, Lordiis, GoranSMilovanovic, Adik2382, Jayprakash12345, Th3d3v1ls, Ramalepe, Liugev6, QZanden, Zoranzoki21, LawExplorer, Lewizho99, Maathavan, DatGuy, Devwaker, Niklitov, Urbanecm, JEumerus, Tulsi_Bhagat, Wong128hk, Luke081515, SimmeD, biplabanand, Wikidata-bugs, Snowolf, aude, Dcljr, Jdforrester-WMF, Matanya, Mbch331, Rxy, Jay8g, Krenair___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T184948: limit page creation and edit rate on Wikidata
Addshore added a comment. In T184948#4059447, @Multichill wrote: If bots ignore replag ( https://meta.wikimedia.org/wiki/Bot_policy#Edit_throttle_and_peak_hours ) these bots should just be stripped of the botflag. The good shouldn't suffer because of a small number of problematic users. The issue is that the infrastructure limits that we are hitting are not linked to maxlag in any way.TASK DETAILhttps://phabricator.wikimedia.org/T184948EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: AddshoreCc: Lucas_Werkmeister_WMDE, ArtixKreiger, Multichill, Mahir256, Framawiki, hoo, Sjoerddebruin, Addshore, Ladsgroup, gerritbot, Aklapper, Lydia_Pintscher, Versusxo, Majesticalreaper22, Tamgue, Giuliamocci, Adrian1985, Cpaulf30, Lahi, Gq86, Baloch007, Darkminds3113, Lordiis, GoranSMilovanovic, Adik2382, Jayprakash12345, Th3d3v1ls, Ramalepe, Liugev6, QZanden, Zoranzoki21, LawExplorer, Lewizho99, Maathavan, DatGuy, Devwaker, Niklitov, Urbanecm, JEumerus, Tulsi_Bhagat, Wong128hk, Luke081515, SimmeD, biplabanand, Wikidata-bugs, Snowolf, aude, Dcljr, Jdforrester-WMF, Matanya, Mbch331, Rxy, Jay8g, Krenair___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T184948: limit page creation and edit rate on Wikidata
ArtixKreiger added a comment. Whats the point of noratelimit if admins and bots are rate limited? However, yes. I agree with everyone that there should be some controllable edit rate. This is a random suggestion, but feel free to ignore. Perhaps, for "everyone", there is an enforced 100/min edit rate. For admins and "trusted users", 100/min is default since they KNOW not to do it and thus they get an option to set it higher.TASK DETAILhttps://phabricator.wikimedia.org/T184948EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: ArtixKreigerCc: ArtixKreiger, Multichill, Mahir256, Framawiki, hoo, Sjoerddebruin, Addshore, Ladsgroup, gerritbot, Aklapper, Lydia_Pintscher, Versusxo, Majesticalreaper22, Tamgue, Giuliamocci, Adrian1985, Cpaulf30, Lahi, Gq86, Baloch007, Darkminds3113, Lordiis, GoranSMilovanovic, Adik2382, Jayprakash12345, Th3d3v1ls, Ramalepe, Liugev6, QZanden, Zoranzoki21, LawExplorer, Lewizho99, Maathavan, DatGuy, Devwaker, Niklitov, Urbanecm, JEumerus, Tulsi_Bhagat, Wong128hk, Luke081515, SimmeD, biplabanand, Wikidata-bugs, Snowolf, aude, Dcljr, Jdforrester-WMF, Matanya, Mbch331, Rxy, Jay8g, Krenair___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T184948: limit page creation and edit rate on Wikidata
Multichill added a comment. In T184948#4052662, @Ladsgroup wrote: I personally would like to strip the right from the bots and enforce a rate limit for them as they just don't care about what we say. If bots ignore replag ( https://meta.wikimedia.org/wiki/Bot_policy#Edit_throttle_and_peak_hours ) these bots should just be stripped of the botflag. The good shouldn't suffer because of a small number of problematic users.TASK DETAILhttps://phabricator.wikimedia.org/T184948EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: MultichillCc: Multichill, Mahir256, Framawiki, hoo, Sjoerddebruin, Addshore, Ladsgroup, gerritbot, Aklapper, Lydia_Pintscher, Versusxo, Majesticalreaper22, Ahmed123, Giuliamocci, Adrian1985, Cpaulf30, Lahi, Gq86, Baloch007, Darkminds3113, Lordiis, GoranSMilovanovic, Adik2382, Jayprakash12345, Th3d3v1ls, Ramalepe, Liugev6, QZanden, Zoranzoki21, LawExplorer, Lewizho99, Maathavan, DatGuy, Devwaker, Niklitov, Urbanecm, JEumerus, Tulsi_Bhagat, Wong128hk, Luke081515, SimmeD, biplabanand, Wikidata-bugs, Snowolf, aude, Dcljr, Jdforrester-WMF, Matanya, Mbch331, Rxy, Jay8g, Krenair___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T184948: limit page creation and edit rate on Wikidata
Ladsgroup added a comment. I personally would like to strip the right from the bots and enforce a rate limit for them as they just don't care about what we say.TASK DETAILhttps://phabricator.wikimedia.org/T184948EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: LadsgroupCc: Framawiki, hoo, Sjoerddebruin, Addshore, Ladsgroup, gerritbot, Aklapper, Lydia_Pintscher, Majesticalreaper22, Giuliamocci, Adrian1985, Cpaulf30, Lahi, Gq86, Baloch007, Darkminds3113, Lordiis, GoranSMilovanovic, Adik2382, Jayprakash12345, Th3d3v1ls, Ramalepe, Liugev6, QZanden, Zoranzoki21, LawExplorer, Lewizho99, Maathavan, DatGuy, Devwaker, Niklitov, Urbanecm, JEumerus, Tulsi_Bhagat, Wong128hk, Luke081515, SimmeD, biplabanand, Wikidata-bugs, Snowolf, aude, Dcljr, Jdforrester-WMF, Matanya, Mbch331, Rxy, Jay8g, Krenair___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
