[Wikidata-bugs] [Maniphest] [Commented On] T209504: Perform more constraint type checks in PHP before falling back to SPARQL
Lucas_Werkmeister_WMDE added a comment. That’s possible, but I’m not sure why that should happen. Doing more checks shouldn’t really affect the mean or average time of individual checks… unless we’re already overloading the internal cluster and it’s taking longer to respond to queries? I think the limit is more or less fine, for now. Let’s finish the job queue rollout first. Should we take another look at this afterwards, or leave it alone?TASK DETAILhttps://phabricator.wikimedia.org/T209504EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Lucas_Werkmeister_WMDECc: Tarrow, Lydia_Pintscher, Addshore, Stashbot, Agabi10, gerritbot, Michael, Lucas_Werkmeister_WMDE, Aklapper, CucyNoiD, Nandana, NebulousIris, Gaboe420, Versusxo, Majesticalreaper22, Giuliamocci, Adrian1985, Cpaulf30, Lahi, Gq86, Baloch007, Darkminds3113, Bsandipan, Lordiis, GoranSMilovanovic, Adik2382, Th3d3v1ls, Ramalepe, Liugev6, QZanden, merbst, LawExplorer, Lewizho99, Maathavan, _jensen, Jonas, Wikidata-bugs, aude, Mbch331___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T209504: Perform more constraint type checks in PHP before falling back to SPARQL
Stashbot added a comment. Mentioned in SAL (#wikimedia-operations) [2019-01-22T10:15:37Z] Synchronized wmf-config/InitialiseSettings.php: T209504 Decrease WBQualityConstraintsTypeCheckMaxEntities from 300 to 150 (duration: 00m 47s)TASK DETAILhttps://phabricator.wikimedia.org/T209504EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Lucas_Werkmeister_WMDE, StashbotCc: Tarrow, Lydia_Pintscher, Addshore, Stashbot, Agabi10, gerritbot, Michael, Lucas_Werkmeister_WMDE, Aklapper, CucyNoiD, Nandana, NebulousIris, Gaboe420, Versusxo, Majesticalreaper22, Giuliamocci, Adrian1985, Cpaulf30, Lahi, Gq86, Baloch007, Darkminds3113, Bsandipan, Lordiis, GoranSMilovanovic, Adik2382, Th3d3v1ls, Ramalepe, Liugev6, QZanden, merbst, LawExplorer, Lewizho99, Maathavan, _jensen, Jonas, Wikidata-bugs, aude, Mbch331___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T209504: Perform more constraint type checks in PHP before falling back to SPARQL
gerritbot added a comment. Change 485628 merged by jenkins-bot: [operations/mediawiki-config@master] Decrease WBQualityConstraintsTypeCheckMaxEntities https://gerrit.wikimedia.org/r/485628TASK DETAILhttps://phabricator.wikimedia.org/T209504EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Lucas_Werkmeister_WMDE, gerritbotCc: Tarrow, Lydia_Pintscher, Addshore, Stashbot, Agabi10, gerritbot, Michael, Lucas_Werkmeister_WMDE, Aklapper, CucyNoiD, Nandana, NebulousIris, Gaboe420, Versusxo, Majesticalreaper22, Giuliamocci, Adrian1985, Cpaulf30, Lahi, Gq86, Baloch007, Darkminds3113, Bsandipan, Lordiis, GoranSMilovanovic, Adik2382, Th3d3v1ls, Ramalepe, Liugev6, QZanden, merbst, LawExplorer, Lewizho99, Maathavan, _jensen, Jonas, Wikidata-bugs, aude, Mbch331___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T209504: Perform more constraint type checks in PHP before falling back to SPARQL
gerritbot added a comment. Change 485628 had a related patch set uploaded (by Lucas Werkmeister (WMDE); owner: Lucas Werkmeister (WMDE)): [operations/mediawiki-config@master] Decrease WBQualityConstraintsTypeCheckMaxEntities https://gerrit.wikimedia.org/r/485628TASK DETAILhttps://phabricator.wikimedia.org/T209504EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Lucas_Werkmeister_WMDE, gerritbotCc: Tarrow, Lydia_Pintscher, Addshore, Stashbot, Agabi10, gerritbot, Michael, Lucas_Werkmeister_WMDE, Aklapper, CucyNoiD, Nandana, NebulousIris, Gaboe420, Versusxo, Majesticalreaper22, Giuliamocci, Adrian1985, Cpaulf30, Lahi, Gq86, Baloch007, Darkminds3113, Bsandipan, Lordiis, GoranSMilovanovic, Adik2382, Th3d3v1ls, Ramalepe, Liugev6, QZanden, merbst, LawExplorer, Lewizho99, Maathavan, _jensen, Jonas, Wikidata-bugs, aude, Mbch331___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T209504: Perform more constraint type checks in PHP before falling back to SPARQL
Lucas_Werkmeister_WMDE added a comment. Probably, but I’d wait for a few days just to see more data, it’s not a catastrophic increase by any means.TASK DETAILhttps://phabricator.wikimedia.org/T209504EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Addshore, Lucas_Werkmeister_WMDECc: Tarrow, Lydia_Pintscher, Addshore, Stashbot, Agabi10, gerritbot, Michael, Lucas_Werkmeister_WMDE, Aklapper, CucyNoiD, Nandana, NebulousIris, Gaboe420, Versusxo, Majesticalreaper22, Giuliamocci, Adrian1985, Cpaulf30, Lahi, Gq86, Baloch007, Darkminds3113, Bsandipan, Lordiis, GoranSMilovanovic, Adik2382, Th3d3v1ls, Ramalepe, Liugev6, QZanden, merbst, LawExplorer, Lewizho99, Maathavan, _jensen, D3r1ck01, Jonas, Wikidata-bugs, aude, Mbch331___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T209504: Perform more constraint type checks in PHP before falling back to SPARQL
Addshore added a comment. In T209504#4883887, @Lucas_Werkmeister_WMDE wrote: Hm, looking at the last 2 days on the new dashboard (permalink) I feel like the runtime slightly deteriorated since the last config change? But there’s also some odd-looking patterns there, so perhaps we should observe for a bit longer. Yup, looking at more data than when I wrote T209504#4880277 now it does look worse. Perhaps we should tweak it down a little @Lucas_Werkmeister_WMDE ?TASK DETAILhttps://phabricator.wikimedia.org/T209504EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: AddshoreCc: Tarrow, Lydia_Pintscher, Addshore, Stashbot, Agabi10, gerritbot, Michael, Lucas_Werkmeister_WMDE, Aklapper, CucyNoiD, Nandana, NebulousIris, Gaboe420, Versusxo, Majesticalreaper22, Giuliamocci, Adrian1985, Cpaulf30, Lahi, Gq86, Baloch007, Darkminds3113, Bsandipan, Lordiis, GoranSMilovanovic, Adik2382, Th3d3v1ls, Ramalepe, Liugev6, QZanden, merbst, LawExplorer, Lewizho99, Maathavan, _jensen, D3r1ck01, Jonas, Wikidata-bugs, aude, Mbch331___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T209504: Perform more constraint type checks in PHP before falling back to SPARQL
Lucas_Werkmeister_WMDE added a comment. Hm, looking at the last 2 days on the new dashboard (permalink) I feel like the runtime slightly deteriorated since the last config change? But there’s also some odd-looking patterns there, so perhaps we should observe for a bit longer.TASK DETAILhttps://phabricator.wikimedia.org/T209504EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Addshore, Lucas_Werkmeister_WMDECc: Tarrow, Lydia_Pintscher, Addshore, Stashbot, Agabi10, gerritbot, Michael, Lucas_Werkmeister_WMDE, Aklapper, CucyNoiD, Nandana, NebulousIris, Gaboe420, Versusxo, Majesticalreaper22, Giuliamocci, Adrian1985, Cpaulf30, Lahi, Gq86, Baloch007, Darkminds3113, Bsandipan, Lordiis, GoranSMilovanovic, Adik2382, Th3d3v1ls, Ramalepe, Liugev6, QZanden, merbst, LawExplorer, Lewizho99, Maathavan, _jensen, D3r1ck01, Jonas, Wikidata-bugs, aude, Mbch331___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T209504: Perform more constraint type checks in PHP before falling back to SPARQL
Lucas_Werkmeister_WMDE added a comment. It feels absurd that loading 300 entities from the database and deserializing them is faster than a single SPARQL query, but if that’s the way it is… looking at the last 90 days on that dashboard, the mean, p75 and p95 type check runtime went down significantly when the we bumped the value from 20 to 100 and haven’t changed very much since then, so perhaps that’s the best we can do. Let’s wait a few more days to see the results of the latest config change, but after that I guess we can close this task.TASK DETAILhttps://phabricator.wikimedia.org/T209504EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Addshore, Lucas_Werkmeister_WMDECc: Tarrow, Lydia_Pintscher, Addshore, Stashbot, Agabi10, gerritbot, Michael, Lucas_Werkmeister_WMDE, Aklapper, CucyNoiD, Nandana, NebulousIris, Gaboe420, Versusxo, Majesticalreaper22, Giuliamocci, Adrian1985, Cpaulf30, Lahi, Gq86, Baloch007, Darkminds3113, Bsandipan, Lordiis, GoranSMilovanovic, Adik2382, Th3d3v1ls, Ramalepe, Liugev6, QZanden, merbst, LawExplorer, Lewizho99, Maathavan, _jensen, D3r1ck01, Jonas, Wikidata-bugs, aude, Mbch331___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T209504: Perform more constraint type checks in PHP before falling back to SPARQL
Stashbot added a comment. Mentioned in SAL (#wikimedia-operations) [2019-01-15T09:10:07Z] Synchronized wmf-config/InitialiseSettings.php: wgWBQualityConstraintsTypeCheckMaxEntities 300, T209504 (duration: 00m 46s)TASK DETAILhttps://phabricator.wikimedia.org/T209504EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Addshore, StashbotCc: Tarrow, Lydia_Pintscher, Addshore, Stashbot, Agabi10, gerritbot, Michael, Lucas_Werkmeister_WMDE, Aklapper, CucyNoiD, Nandana, NebulousIris, Gaboe420, Versusxo, Majesticalreaper22, Giuliamocci, Adrian1985, Cpaulf30, Lahi, Gq86, Baloch007, Darkminds3113, Bsandipan, Lordiis, GoranSMilovanovic, Adik2382, Th3d3v1ls, Ramalepe, Liugev6, QZanden, merbst, LawExplorer, Lewizho99, Maathavan, _jensen, D3r1ck01, Jonas, Wikidata-bugs, aude, Mbch331___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T209504: Perform more constraint type checks in PHP before falling back to SPARQL
gerritbot added a comment. Change 484370 merged by jenkins-bot: [operations/mediawiki-config@master] wgWBQualityConstraintsTypeCheckMaxEntities 300 https://gerrit.wikimedia.org/r/484370TASK DETAILhttps://phabricator.wikimedia.org/T209504EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Addshore, gerritbotCc: Tarrow, Lydia_Pintscher, Addshore, Stashbot, Agabi10, gerritbot, Michael, Lucas_Werkmeister_WMDE, Aklapper, CucyNoiD, Nandana, NebulousIris, Gaboe420, Versusxo, Majesticalreaper22, Giuliamocci, Adrian1985, Cpaulf30, Lahi, Gq86, Baloch007, Darkminds3113, Bsandipan, Lordiis, GoranSMilovanovic, Adik2382, Th3d3v1ls, Ramalepe, Liugev6, QZanden, merbst, LawExplorer, Lewizho99, Maathavan, _jensen, D3r1ck01, Jonas, Wikidata-bugs, aude, Mbch331___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T209504: Perform more constraint type checks in PHP before falling back to SPARQL
gerritbot added a comment. Change 484370 had a related patch set uploaded (by Addshore; owner: Addshore): [operations/mediawiki-config@master] wgWBQualityConstraintsTypeCheckMaxEntities 300 https://gerrit.wikimedia.org/r/484370TASK DETAILhttps://phabricator.wikimedia.org/T209504EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Addshore, gerritbotCc: Tarrow, Lydia_Pintscher, Addshore, Stashbot, Agabi10, gerritbot, Michael, Lucas_Werkmeister_WMDE, Aklapper, CucyNoiD, Nandana, NebulousIris, Gaboe420, Versusxo, Majesticalreaper22, Giuliamocci, Adrian1985, Cpaulf30, Lahi, Gq86, Baloch007, Darkminds3113, Bsandipan, Lordiis, GoranSMilovanovic, Adik2382, Th3d3v1ls, Ramalepe, Liugev6, QZanden, merbst, LawExplorer, Lewizho99, Maathavan, _jensen, D3r1ck01, Jonas, Wikidata-bugs, aude, Mbch331___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T209504: Perform more constraint type checks in PHP before falling back to SPARQL
Michael added a comment. In T209504#4869194, @Addshore wrote: It looks like we could probably bump this up another level, or a few more levels? Could we have a graph showing the changes over time in Q21510865_ValueTypeChecker and Q21503250_TypeChecker from the Constraint Check Mean Runtime Graph? As I understand, these are the metrics we really care about, and we should maybe stop when we see that they get worse again.TASK DETAILhttps://phabricator.wikimedia.org/T209504EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Tarrow, MichaelCc: Lydia_Pintscher, Addshore, Stashbot, Agabi10, gerritbot, Michael, Lucas_Werkmeister_WMDE, Aklapper, CucyNoiD, Nandana, NebulousIris, Gaboe420, Versusxo, Majesticalreaper22, Giuliamocci, Adrian1985, Cpaulf30, Lahi, Gq86, Baloch007, Darkminds3113, Bsandipan, Lordiis, GoranSMilovanovic, Adik2382, Th3d3v1ls, Ramalepe, Liugev6, QZanden, merbst, LawExplorer, Lewizho99, Maathavan, _jensen, D3r1ck01, Jonas, Wikidata-bugs, aude, Mbch331___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T209504: Perform more constraint type checks in PHP before falling back to SPARQL
Addshore added a comment. It looks like we could probably bump this up another level, or a few more levels?TASK DETAILhttps://phabricator.wikimedia.org/T209504EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Tarrow, AddshoreCc: Lydia_Pintscher, Addshore, Stashbot, Agabi10, gerritbot, Michael, Lucas_Werkmeister_WMDE, Aklapper, CucyNoiD, Nandana, NebulousIris, Gaboe420, Versusxo, Majesticalreaper22, Giuliamocci, Adrian1985, Cpaulf30, Lahi, Gq86, Baloch007, Darkminds3113, Bsandipan, Lordiis, GoranSMilovanovic, Adik2382, Th3d3v1ls, Ramalepe, Liugev6, QZanden, merbst, LawExplorer, Lewizho99, Maathavan, _jensen, D3r1ck01, Jonas, Wikidata-bugs, aude, Mbch331___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T209504: Perform more constraint type checks in PHP before falling back to SPARQL
Stashbot added a comment. Mentioned in SAL (#wikimedia-operations) [2019-01-09T17:39:07Z] That last one was SWAT: [[gerrit:483185|T209504 Increase PHP constraint check entities to 150]]TASK DETAILhttps://phabricator.wikimedia.org/T209504EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Tarrow, StashbotCc: Lydia_Pintscher, Addshore, Stashbot, Agabi10, gerritbot, Michael, Lucas_Werkmeister_WMDE, Aklapper, CucyNoiD, Nandana, NebulousIris, Gaboe420, Versusxo, Majesticalreaper22, Giuliamocci, Adrian1985, Cpaulf30, Lahi, Gq86, Baloch007, Darkminds3113, Bsandipan, Lordiis, GoranSMilovanovic, Adik2382, Th3d3v1ls, Ramalepe, Liugev6, QZanden, merbst, LawExplorer, Lewizho99, Maathavan, _jensen, D3r1ck01, Jonas, Wikidata-bugs, aude, Mbch331___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T209504: Perform more constraint type checks in PHP before falling back to SPARQL
gerritbot added a comment. Change 483185 merged by jenkins-bot: [operations/mediawiki-config@master] Increase PHP constraint check entities to 150 https://gerrit.wikimedia.org/r/483185TASK DETAILhttps://phabricator.wikimedia.org/T209504EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Tarrow, gerritbotCc: Lydia_Pintscher, Addshore, Stashbot, Agabi10, gerritbot, Michael, Lucas_Werkmeister_WMDE, Aklapper, CucyNoiD, Nandana, NebulousIris, Gaboe420, Versusxo, Majesticalreaper22, Giuliamocci, Adrian1985, Cpaulf30, Lahi, Gq86, Baloch007, Darkminds3113, Bsandipan, Lordiis, GoranSMilovanovic, Adik2382, Th3d3v1ls, Ramalepe, Liugev6, QZanden, merbst, LawExplorer, Lewizho99, Maathavan, _jensen, D3r1ck01, Jonas, Wikidata-bugs, aude, Mbch331___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T209504: Perform more constraint type checks in PHP before falling back to SPARQL
gerritbot added a comment. Change 483185 had a related patch set uploaded (by Tarrow; owner: Tarrow): [operations/mediawiki-config@master] Increase PHP constraint check entities to 150 https://gerrit.wikimedia.org/r/483185TASK DETAILhttps://phabricator.wikimedia.org/T209504EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Tarrow, gerritbotCc: Lydia_Pintscher, Addshore, Stashbot, Agabi10, gerritbot, Michael, Lucas_Werkmeister_WMDE, Aklapper, CucyNoiD, Nandana, NebulousIris, Gaboe420, Versusxo, Majesticalreaper22, Giuliamocci, Adrian1985, Cpaulf30, Lahi, Gq86, Baloch007, Darkminds3113, Bsandipan, Lordiis, GoranSMilovanovic, Adik2382, Th3d3v1ls, Ramalepe, Liugev6, QZanden, merbst, LawExplorer, Lewizho99, Maathavan, _jensen, D3r1ck01, Jonas, Wikidata-bugs, aude, Mbch331___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T209504: Perform more constraint type checks in PHP before falling back to SPARQL
Tarrow added a comment. @Addshore From wmf-config/InitialiseSettings.php: 'wgWBQualityConstraintsTypeCheckMaxEntities' => [ 'default' => 1000, 'wikidatawiki' => 100, ], I can bump to 150 as a similar 50% increase to what you suggested.TASK DETAILhttps://phabricator.wikimedia.org/T209504EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: TarrowCc: Lydia_Pintscher, Addshore, Stashbot, Agabi10, gerritbot, Michael, Lucas_Werkmeister_WMDE, Aklapper, CucyNoiD, Nandana, NebulousIris, Gaboe420, Versusxo, Majesticalreaper22, Giuliamocci, Adrian1985, Cpaulf30, Lahi, Gq86, Baloch007, Darkminds3113, Bsandipan, Lordiis, GoranSMilovanovic, Adik2382, Th3d3v1ls, Ramalepe, Liugev6, QZanden, merbst, LawExplorer, Lewizho99, Maathavan, _jensen, D3r1ck01, Jonas, Wikidata-bugs, aude, Mbch331___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T209504: Perform more constraint type checks in PHP before falling back to SPARQL
Addshore added a comment. In T209504#4851665, @Lydia_Pintscher wrote: It's the new year. Should we unstall this? I don't think deployments really start again until next week, so we can action this quite yet. Next week we could bump the value from 20 to 30? Thoughts @Lucas_Werkmeister_WMDE ? If that still prove to be too small we can continue increasing it throughout next week.TASK DETAILhttps://phabricator.wikimedia.org/T209504EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: AddshoreCc: Lydia_Pintscher, Addshore, Stashbot, Agabi10, gerritbot, Michael, Lucas_Werkmeister_WMDE, Aklapper, CucyNoiD, Nandana, NebulousIris, Gaboe420, Versusxo, Majesticalreaper22, Giuliamocci, Adrian1985, Cpaulf30, Lahi, Gq86, Baloch007, Darkminds3113, Bsandipan, Lordiis, GoranSMilovanovic, Adik2382, Th3d3v1ls, Ramalepe, Liugev6, QZanden, merbst, LawExplorer, Lewizho99, Maathavan, _jensen, D3r1ck01, Jonas, Wikidata-bugs, aude, Mbch331___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T209504: Perform more constraint type checks in PHP before falling back to SPARQL
Lydia_Pintscher added a comment. It's the new year. Should we unstall this?TASK DETAILhttps://phabricator.wikimedia.org/T209504EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Lydia_PintscherCc: Lydia_Pintscher, Addshore, Stashbot, Agabi10, gerritbot, Michael, Lucas_Werkmeister_WMDE, Aklapper, CucyNoiD, Nandana, NebulousIris, Gaboe420, Versusxo, Majesticalreaper22, Giuliamocci, Adrian1985, Cpaulf30, Lahi, Gq86, Baloch007, Darkminds3113, Bsandipan, Lordiis, GoranSMilovanovic, Adik2382, Th3d3v1ls, Ramalepe, Liugev6, QZanden, merbst, LawExplorer, Lewizho99, Maathavan, _jensen, D3r1ck01, Jonas, Wikidata-bugs, aude, Mbch331___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T209504: Perform more constraint type checks in PHP before falling back to SPARQL
Addshore added a comment. In T209504#4840623, @Lucas_Werkmeister_WMDE wrote: Well, none of those changes actually tell us if that’s a good thing or not, I think :) but the overall mean and median times for “type” and “value type” checks also went down noticeably, which is great. That said, I’m still not sure if we shouldn’t increase the limit even further? Even now, type checks running into the limit are still much faster than SPARQL checks (roughly 20 vs. 140 ms). There were not specific acceptance criteria so I was simply validating that "Perform more constraint type checks in PHP before falling back to SPARQL" had been done. I also think we could increase the limit more.TASK DETAILhttps://phabricator.wikimedia.org/T209504EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: AddshoreCc: Addshore, Stashbot, Agabi10, gerritbot, Michael, Lucas_Werkmeister_WMDE, Aklapper, CucyNoiD, Nandana, NebulousIris, Gaboe420, Versusxo, Majesticalreaper22, Giuliamocci, Adrian1985, Cpaulf30, Lahi, Gq86, Baloch007, Darkminds3113, Bsandipan, Lordiis, GoranSMilovanovic, Adik2382, Th3d3v1ls, Ramalepe, Liugev6, QZanden, merbst, LawExplorer, Lewizho99, Maathavan, _jensen, D3r1ck01, Jonas, Wikidata-bugs, aude, Lydia_Pintscher, Mbch331___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T209504: Perform more constraint type checks in PHP before falling back to SPARQL
Lucas_Werkmeister_WMDE added a comment. Well, none of those changes actually tell us if that’s a good thing or not, I think :) but the overall mean and median times for “type” and “value type” checks also went down, which is great. That said, I’m still not sure if we shouldn’t increase the limit even further? Even now, type checks running into the limit are still much faster than SPARQL checks (roughly 20 vs. 140 ms).TASK DETAILhttps://phabricator.wikimedia.org/T209504EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Addshore, Lucas_Werkmeister_WMDECc: Addshore, Stashbot, Agabi10, gerritbot, Michael, Lucas_Werkmeister_WMDE, Aklapper, CucyNoiD, Nandana, NebulousIris, Gaboe420, Versusxo, Majesticalreaper22, Giuliamocci, Adrian1985, Cpaulf30, Lahi, Gq86, Baloch007, Darkminds3113, Bsandipan, Lordiis, GoranSMilovanovic, Adik2382, Th3d3v1ls, Ramalepe, Liugev6, QZanden, merbst, LawExplorer, Lewizho99, Maathavan, _jensen, D3r1ck01, Jonas, Wikidata-bugs, aude, Lydia_Pintscher, Mbch331___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T209504: Perform more constraint type checks in PHP before falling back to SPARQL
Addshore added a comment. In T209504#4830822, @Michael wrote: @Lucas_Werkmeister_WMDE Interestingly, and independently from today's deployment, https://www.wikidata.org/wiki/Special:ConstraintReport/Q41 is broken o.O It looks like that might have just been a runtime issue and this page renders for me now.TASK DETAILhttps://phabricator.wikimedia.org/T209504EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: AddshoreCc: Addshore, Stashbot, Agabi10, gerritbot, Michael, Lucas_Werkmeister_WMDE, Aklapper, CucyNoiD, Nandana, NebulousIris, Gaboe420, Versusxo, Majesticalreaper22, Giuliamocci, Adrian1985, Cpaulf30, Lahi, Gq86, Baloch007, Darkminds3113, Bsandipan, Lordiis, GoranSMilovanovic, Adik2382, Th3d3v1ls, Ramalepe, Liugev6, QZanden, merbst, LawExplorer, Lewizho99, Maathavan, _jensen, D3r1ck01, Jonas, Wikidata-bugs, aude, Lydia_Pintscher, Mbch331___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T209504: Perform more constraint type checks in PHP before falling back to SPARQL
Michael added a comment. @Lucas_Werkmeister_WMDE Interestingly, and independently from today's deployment, https://www.wikidata.org/wiki/Special:ConstraintReport/Q41 is broken o.OTASK DETAILhttps://phabricator.wikimedia.org/T209504EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: MichaelCc: Stashbot, Agabi10, gerritbot, Michael, Lucas_Werkmeister_WMDE, Aklapper, CucyNoiD, Nandana, NebulousIris, Gaboe420, Versusxo, Majesticalreaper22, Giuliamocci, Adrian1985, Cpaulf30, Lahi, Gq86, Baloch007, Darkminds3113, Bsandipan, Lordiis, GoranSMilovanovic, Adik2382, Th3d3v1ls, Ramalepe, Liugev6, QZanden, merbst, LawExplorer, Lewizho99, Maathavan, _jensen, D3r1ck01, Jonas, Wikidata-bugs, aude, Lydia_Pintscher, Mbch331___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T209504: Perform more constraint type checks in PHP before falling back to SPARQL
Stashbot added a comment. Mentioned in SAL (#wikimedia-operations) [2018-12-18T12:17:13Z] Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:478630|Perform even more PHP constraint checks before falling back (T209504)]] (duration: 00m 46s)TASK DETAILhttps://phabricator.wikimedia.org/T209504EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: StashbotCc: Stashbot, Agabi10, gerritbot, Michael, Lucas_Werkmeister_WMDE, Aklapper, CucyNoiD, Nandana, NebulousIris, Gaboe420, Versusxo, Majesticalreaper22, Giuliamocci, Adrian1985, Cpaulf30, Lahi, Gq86, Baloch007, Darkminds3113, Bsandipan, Lordiis, GoranSMilovanovic, Adik2382, Th3d3v1ls, Ramalepe, Liugev6, QZanden, merbst, LawExplorer, Lewizho99, Maathavan, _jensen, D3r1ck01, Jonas, Wikidata-bugs, aude, Lydia_Pintscher, Mbch331___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T209504: Perform more constraint type checks in PHP before falling back to SPARQL
gerritbot added a comment. Change 478630 merged by jenkins-bot: [operations/mediawiki-config@master] Perform even more PHP constraint checks before falling back https://gerrit.wikimedia.org/r/478630TASK DETAILhttps://phabricator.wikimedia.org/T209504EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: gerritbotCc: Stashbot, Agabi10, gerritbot, Michael, Lucas_Werkmeister_WMDE, Aklapper, CucyNoiD, Nandana, NebulousIris, Gaboe420, Versusxo, Majesticalreaper22, Giuliamocci, Adrian1985, Cpaulf30, Lahi, Gq86, Baloch007, Darkminds3113, Bsandipan, Lordiis, GoranSMilovanovic, Adik2382, Th3d3v1ls, Ramalepe, Liugev6, QZanden, merbst, LawExplorer, Lewizho99, Maathavan, _jensen, D3r1ck01, Jonas, Wikidata-bugs, aude, Lydia_Pintscher, Mbch331___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T209504: Perform more constraint type checks in PHP before falling back to SPARQL
Lucas_Werkmeister_WMDE added a comment. Currently, we are checking about 65%-70% of entities in PHP. I would like to understand if that 20% that I would like to shift to PHP comes from entities with typically 20-40 checks each or from entities with 20-110 checks each. Then we can adjust the config accordingly. I’m afraid we just don’t have that information. When we fall back to SPARQL, we just ask the query service “is X a subclass of Y?” – there’s no way to know how many entities it looked at to arrive at the answer.TASK DETAILhttps://phabricator.wikimedia.org/T209504EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Lucas_Werkmeister_WMDECc: Stashbot, Agabi10, gerritbot, Michael, Lucas_Werkmeister_WMDE, Aklapper, CucyNoiD, Nandana, NebulousIris, Gaboe420, Versusxo, Majesticalreaper22, Giuliamocci, Adrian1985, Cpaulf30, Lahi, Gq86, Baloch007, Darkminds3113, Bsandipan, Lordiis, GoranSMilovanovic, Adik2382, Th3d3v1ls, Ramalepe, Liugev6, QZanden, merbst, LawExplorer, Lewizho99, Maathavan, _jensen, D3r1ck01, Jonas, Wikidata-bugs, aude, Lydia_Pintscher, Mbch331___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T209504: Perform more constraint type checks in PHP before falling back to SPARQL
Lucas_Werkmeister_WMDE added a comment. In T209504#4823532, @Agabi10 wrote: Isn't this one the rate? Which, by the way, could have a better color choice to make it more accessible. No, sorry, I meant the rate of SPARQL checks over time, not compared to PHP checks. (Though that rate should also shift towards PHP checks, yes – I mentioned that in the task description.) The colors are picked by Grafana, I didn’t do anything special there.TASK DETAILhttps://phabricator.wikimedia.org/T209504EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Lucas_Werkmeister_WMDECc: Stashbot, Agabi10, gerritbot, Michael, Lucas_Werkmeister_WMDE, Aklapper, CucyNoiD, Nandana, NebulousIris, Gaboe420, Versusxo, Majesticalreaper22, Giuliamocci, Adrian1985, Cpaulf30, Lahi, Gq86, Baloch007, Darkminds3113, Bsandipan, Lordiis, GoranSMilovanovic, Adik2382, Th3d3v1ls, Ramalepe, Liugev6, QZanden, merbst, LawExplorer, Lewizho99, Maathavan, _jensen, D3r1ck01, Jonas, Wikidata-bugs, aude, Lydia_Pintscher, Mbch331___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T209504: Perform more constraint type checks in PHP before falling back to SPARQL
Michael added a comment. In T209504#4823419, @Lucas_Werkmeister_WMDE wrote: However, what would be much more useful to know, would be the distribution of checks that exceeded that limit and were handled by SPARQL. Well, that’s the “Type Check Runtimes” and “Type Check Distribution” panels… which information exactly do you want to have? The number of entities checked doesn’t make sense in SPARQL. I would like to change this configuration to a value so that about 90% of entities are successfully checked in PHP and see if that changes the timings the way we want. Currently, we are checking about 65%-70% of entities in PHP. I would like to understand if that 20% that I would like to shift to PHP comes from entities with typically 20-40 checks each or from entities with 20-110 checks each. Then we can adjust the config accordingly. Also, as you mentioned in a comment further above, the total number of entities checked would be interesting.TASK DETAILhttps://phabricator.wikimedia.org/T209504EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: MichaelCc: Stashbot, Agabi10, gerritbot, Michael, Lucas_Werkmeister_WMDE, Aklapper, CucyNoiD, Nandana, NebulousIris, Gaboe420, Versusxo, Majesticalreaper22, Giuliamocci, Adrian1985, Cpaulf30, Lahi, Gq86, Baloch007, Darkminds3113, Bsandipan, Lordiis, GoranSMilovanovic, Adik2382, Th3d3v1ls, Ramalepe, Liugev6, QZanden, merbst, LawExplorer, Lewizho99, Maathavan, _jensen, D3r1ck01, Jonas, Wikidata-bugs, aude, Lydia_Pintscher, Mbch331___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T209504: Perform more constraint type checks in PHP before falling back to SPARQL
Agabi10 added a comment. In T209504#4823419, @Lucas_Werkmeister_WMDE wrote: In the “Type Check Runtime” panel, I would expect the times of PHP checks to increase slightly, and the times of SPARQL checks probably wouldn’t change (their rate should decrease, but we don’t currently plot that). Isn't this one the rate? Which, by the way, could have a better color choice to make it more accessible.TASK DETAILhttps://phabricator.wikimedia.org/T209504EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Agabi10Cc: Stashbot, Agabi10, gerritbot, Michael, Lucas_Werkmeister_WMDE, Aklapper, CucyNoiD, Nandana, NebulousIris, Gaboe420, Versusxo, Majesticalreaper22, Giuliamocci, Adrian1985, Cpaulf30, Lahi, Gq86, Baloch007, Darkminds3113, Bsandipan, Lordiis, GoranSMilovanovic, Adik2382, Th3d3v1ls, Ramalepe, Liugev6, QZanden, merbst, LawExplorer, Lewizho99, Maathavan, _jensen, D3r1ck01, Jonas, Wikidata-bugs, aude, Lydia_Pintscher, Mbch331___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T209504: Perform more constraint type checks in PHP before falling back to SPARQL
Lucas_Werkmeister_WMDE added a comment. However, what would be much more useful to know, would be the distribution of checks that exceeded that limit and were handled by SPARQL. Well, that’s the “Type Check Runtimes” and “Type Check Distribution” panels… which information exactly do you want to have? The number of entities checked doesn’t make sense in SPARQL. What kind of change are we expecting there? In the “Type Check Runtime” panel, I would expect the times of PHP checks to increase slightly, and the times of SPARQL checks probably wouldn’t change (their rate should decrease, but we don’t currently plot that). However, the overall time of “type constraint” and “value type constraint” checks, as shown in the “Constraint Check Upper Runtime” panel (first panel on the dashboard), should hopefully decrease somewhat. (Perhaps we should the overall runtime of constraint checks with those two types into a separate panel, and show there not just the upper runtime but also some percentiles.)TASK DETAILhttps://phabricator.wikimedia.org/T209504EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Lucas_Werkmeister_WMDECc: Stashbot, Agabi10, gerritbot, Michael, Lucas_Werkmeister_WMDE, Aklapper, CucyNoiD, Nandana, NebulousIris, Gaboe420, Versusxo, Majesticalreaper22, Giuliamocci, Adrian1985, Cpaulf30, Lahi, Gq86, Baloch007, Darkminds3113, Bsandipan, Lordiis, GoranSMilovanovic, Adik2382, Th3d3v1ls, Ramalepe, Liugev6, QZanden, merbst, LawExplorer, Lewizho99, Maathavan, _jensen, D3r1ck01, Jonas, Wikidata-bugs, aude, Lydia_Pintscher, Mbch331___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T209504: Perform more constraint type checks in PHP before falling back to SPARQL
Michael added a comment. In T209504#4821248, @Lucas_Werkmeister_WMDE wrote: New tracking has been deployed and added to Grafana (“PHP Type Check Entities” panel), so we should be able to see how much these numbers move up when we bump the limit. I see now that this gives us only the distribution among the number of checks on entities that are successfully completed in PHP, i.e. those that have <= 20 checks. However, what would be much more useful to know, would be the distribution of checks that exceeded that limit and were handled by SPARQL. Also, while I saw a small change in the type check distribution from the change we did, I didn't the see any change in the type check runtime. What kind of change are we expecting there?TASK DETAILhttps://phabricator.wikimedia.org/T209504EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: MichaelCc: Stashbot, Agabi10, gerritbot, Michael, Lucas_Werkmeister_WMDE, Aklapper, CucyNoiD, Nandana, NebulousIris, Gaboe420, Versusxo, Majesticalreaper22, Giuliamocci, Adrian1985, Cpaulf30, Lahi, Gq86, Baloch007, Darkminds3113, Bsandipan, Lordiis, GoranSMilovanovic, Adik2382, Th3d3v1ls, Ramalepe, Liugev6, QZanden, merbst, LawExplorer, Lewizho99, Maathavan, _jensen, D3r1ck01, Jonas, Wikidata-bugs, aude, Lydia_Pintscher, Mbch331___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T209504: Perform more constraint type checks in PHP before falling back to SPARQL
gerritbot added a comment. Change 478701 merged by jenkins-bot: [mediawiki/extensions/WikibaseQualityConstraints@master] Track number of entities checked in PHP https://gerrit.wikimedia.org/r/478701TASK DETAILhttps://phabricator.wikimedia.org/T209504EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Lucas_Werkmeister_WMDE, gerritbotCc: Stashbot, Agabi10, gerritbot, Michael, Lucas_Werkmeister_WMDE, Aklapper, CucyNoiD, Nandana, NebulousIris, Gaboe420, Versusxo, Majesticalreaper22, Giuliamocci, Adrian1985, Cpaulf30, Lahi, Gq86, Baloch007, Darkminds3113, Bsandipan, Lordiis, GoranSMilovanovic, Adik2382, Th3d3v1ls, Ramalepe, Liugev6, QZanden, merbst, LawExplorer, Lewizho99, Maathavan, _jensen, D3r1ck01, Jonas, Wikidata-bugs, aude, Lydia_Pintscher, Mbch331___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T209504: Perform more constraint type checks in PHP before falling back to SPARQL
gerritbot added a comment. Change 478701 had a related patch set uploaded (by Lucas Werkmeister (WMDE); owner: Lucas Werkmeister (WMDE)): [mediawiki/extensions/WikibaseQualityConstraints@master] Track number of entities checked in PHP https://gerrit.wikimedia.org/r/478701TASK DETAILhttps://phabricator.wikimedia.org/T209504EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Lucas_Werkmeister_WMDE, gerritbotCc: Stashbot, Agabi10, gerritbot, Michael, Lucas_Werkmeister_WMDE, Aklapper, CucyNoiD, Nandana, NebulousIris, Gaboe420, Versusxo, Majesticalreaper22, Giuliamocci, Adrian1985, Cpaulf30, Lahi, Gq86, Baloch007, Darkminds3113, Bsandipan, Lordiis, GoranSMilovanovic, Adik2382, Th3d3v1ls, Ramalepe, Liugev6, QZanden, merbst, LawExplorer, Lewizho99, Maathavan, _jensen, D3r1ck01, Jonas, Wikidata-bugs, aude, Lydia_Pintscher, Mbch331___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T209504: Perform more constraint type checks in PHP before falling back to SPARQL
Michael added a comment. TASK DETAILhttps://phabricator.wikimedia.org/T209504EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Lucas_Werkmeister_WMDE, MichaelCc: Stashbot, Agabi10, gerritbot, Michael, Lucas_Werkmeister_WMDE, Aklapper, CucyNoiD, Nandana, NebulousIris, Gaboe420, Versusxo, Majesticalreaper22, Giuliamocci, Adrian1985, Cpaulf30, Lahi, Gq86, Baloch007, Darkminds3113, Bsandipan, Lordiis, GoranSMilovanovic, Adik2382, Th3d3v1ls, Ramalepe, Liugev6, QZanden, merbst, LawExplorer, Lewizho99, Maathavan, _jensen, D3r1ck01, Jonas, Wikidata-bugs, aude, Lydia_Pintscher, Mbch331___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T209504: Perform more constraint type checks in PHP before falling back to SPARQL
Michael added a comment. @Lucas_Werkmeister_WMDE Good idea. Number of entities processed might be interesting. But also how the number of constraint checks per checked entity is distributed. Maybe we can calculate/track how some key aspects of the distribution change, e.g. median, third quartile, 95th percentile and max ? That should give as a better idea where to put the limit.TASK DETAILhttps://phabricator.wikimedia.org/T209504EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: MichaelCc: Stashbot, Agabi10, gerritbot, Michael, Lucas_Werkmeister_WMDE, Aklapper, CucyNoiD, Nandana, NebulousIris, Gaboe420, Versusxo, Majesticalreaper22, Giuliamocci, Adrian1985, Cpaulf30, Lahi, Gq86, Baloch007, Darkminds3113, Bsandipan, Lordiis, GoranSMilovanovic, Adik2382, Th3d3v1ls, Ramalepe, Liugev6, QZanden, merbst, LawExplorer, Lewizho99, Maathavan, _jensen, D3r1ck01, Jonas, Wikidata-bugs, aude, Lydia_Pintscher, Mbch331___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T209504: Perform more constraint type checks in PHP before falling back to SPARQL
Lucas_Werkmeister_WMDE added a comment. Hm, perhaps it would be good to have some tracking on the number of entities processed in PHP before we increase the limit further. What do you think?TASK DETAILhttps://phabricator.wikimedia.org/T209504EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Michael, Lucas_Werkmeister_WMDECc: Stashbot, Agabi10, gerritbot, Michael, Lucas_Werkmeister_WMDE, Aklapper, CucyNoiD, Nandana, NebulousIris, Gaboe420, Versusxo, Majesticalreaper22, Giuliamocci, Adrian1985, Cpaulf30, Lahi, Gq86, Baloch007, Darkminds3113, Bsandipan, Lordiis, GoranSMilovanovic, Adik2382, Th3d3v1ls, Ramalepe, Liugev6, QZanden, merbst, LawExplorer, Lewizho99, Maathavan, _jensen, D3r1ck01, Jonas, Wikidata-bugs, aude, Lydia_Pintscher, Mbch331___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T209504: Perform more constraint type checks in PHP before falling back to SPARQL
gerritbot added a comment. Change 478630 had a related patch set uploaded (by Michael Große; owner: Michael Große): [operations/mediawiki-config@master] Perform even more PHP constraint checks before falling back https://gerrit.wikimedia.org/r/478630TASK DETAILhttps://phabricator.wikimedia.org/T209504EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Michael, gerritbotCc: Stashbot, Agabi10, gerritbot, Michael, Lucas_Werkmeister_WMDE, Aklapper, CucyNoiD, Nandana, NebulousIris, Gaboe420, Versusxo, Majesticalreaper22, Giuliamocci, Adrian1985, Cpaulf30, Lahi, Gq86, Baloch007, Darkminds3113, Bsandipan, Lordiis, GoranSMilovanovic, Adik2382, Th3d3v1ls, Ramalepe, Liugev6, QZanden, merbst, LawExplorer, Lewizho99, Maathavan, _jensen, D3r1ck01, Jonas, Wikidata-bugs, aude, Lydia_Pintscher, Mbch331___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T209504: Perform more constraint type checks in PHP before falling back to SPARQL
Stashbot added a comment. Mentioned in SAL (#wikimedia-operations) [2018-12-03T12:19:53Z] Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:476822|Perform more PHP constraint checks before falling back (T209504)]] (duration: 00m 48s)TASK DETAILhttps://phabricator.wikimedia.org/T209504EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Michael, StashbotCc: Stashbot, Agabi10, gerritbot, Michael, Lucas_Werkmeister_WMDE, Aklapper, CucyNoiD, Nandana, NebulousIris, Gaboe420, Versusxo, Majesticalreaper22, Giuliamocci, Adrian1985, Cpaulf30, Lahi, Gq86, Baloch007, Darkminds3113, Bsandipan, Lordiis, GoranSMilovanovic, Adik2382, Th3d3v1ls, Ramalepe, Liugev6, QZanden, merbst, LawExplorer, Lewizho99, Maathavan, _jensen, D3r1ck01, Jonas, Wikidata-bugs, aude, Lydia_Pintscher, Mbch331___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T209504: Perform more constraint type checks in PHP before falling back to SPARQL
gerritbot added a comment. Change 476822 merged by jenkins-bot: [operations/mediawiki-config@master] Perform more PHP constraint checks before falling back https://gerrit.wikimedia.org/r/476822TASK DETAILhttps://phabricator.wikimedia.org/T209504EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Michael, gerritbotCc: Agabi10, gerritbot, Michael, Lucas_Werkmeister_WMDE, Aklapper, CucyNoiD, Nandana, NebulousIris, Gaboe420, Versusxo, Majesticalreaper22, Giuliamocci, Adrian1985, Cpaulf30, Lahi, Gq86, Baloch007, Darkminds3113, Bsandipan, Lordiis, GoranSMilovanovic, Adik2382, Th3d3v1ls, Ramalepe, Liugev6, QZanden, merbst, LawExplorer, Lewizho99, Maathavan, _jensen, D3r1ck01, Jonas, Wikidata-bugs, aude, Lydia_Pintscher, Mbch331___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T209504: Perform more constraint type checks in PHP before falling back to SPARQL
Michael added a comment. No, the config change should not be harmful, it might just no be as effective as "double the limit" might seem. This would be possible to tell from the distribution. But we should be able to deploy the change now and see whether it brings us from ~70% to ~80% or to ~90% of checks handled by PHP.TASK DETAILhttps://phabricator.wikimedia.org/T209504EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: MichaelCc: Agabi10, gerritbot, Michael, Lucas_Werkmeister_WMDE, Aklapper, CucyNoiD, Nandana, NebulousIris, Gaboe420, Versusxo, Majesticalreaper22, Giuliamocci, Adrian1985, Cpaulf30, Lahi, Gq86, Baloch007, Darkminds3113, Bsandipan, Lordiis, GoranSMilovanovic, Adik2382, Th3d3v1ls, Ramalepe, Liugev6, QZanden, merbst, LawExplorer, Lewizho99, Maathavan, _jensen, D3r1ck01, Jonas, Wikidata-bugs, aude, Lydia_Pintscher, Mbch331___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T209504: Perform more constraint type checks in PHP before falling back to SPARQL
Lucas_Werkmeister_WMDE added a comment. The only thing we have right now is the number of checks that stayed within the threshold vs. the number of checks that exceeded it. Do you think we should add additional tracking before deploying the config change?TASK DETAILhttps://phabricator.wikimedia.org/T209504EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Michael, Lucas_Werkmeister_WMDECc: Agabi10, gerritbot, Michael, Lucas_Werkmeister_WMDE, Aklapper, CucyNoiD, Nandana, NebulousIris, Gaboe420, Versusxo, Majesticalreaper22, Giuliamocci, Adrian1985, Cpaulf30, Lahi, Gq86, Baloch007, Darkminds3113, Bsandipan, Lordiis, GoranSMilovanovic, Adik2382, Th3d3v1ls, Ramalepe, Liugev6, QZanden, merbst, LawExplorer, Lewizho99, Maathavan, _jensen, D3r1ck01, Jonas, Wikidata-bugs, aude, Lydia_Pintscher, Mbch331___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T209504: Perform more constraint type checks in PHP before falling back to SPARQL
gerritbot added a comment. Change 476822 had a related patch set uploaded (by Michael Große; owner: Michael Große): [operations/mediawiki-config@master] Perform more PHP constraint checks before falling back https://gerrit.wikimedia.org/r/476822TASK DETAILhttps://phabricator.wikimedia.org/T209504EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Michael, gerritbotCc: gerritbot, Michael, Lucas_Werkmeister_WMDE, Aklapper, Nandana, Lahi, Gq86, GoranSMilovanovic, QZanden, merbst, LawExplorer, _jensen, Agabi10, D3r1ck01, Jonas, Wikidata-bugs, aude, Lydia_Pintscher, Mbch331___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T209504: Perform more constraint type checks in PHP before falling back to SPARQL
Michael added a comment. It would be helpful to have a plot showing the distribution of the number of checks per request. Either as normal histogram or as CDF. Maybe a 3D plot to show this over time?TASK DETAILhttps://phabricator.wikimedia.org/T209504EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: MichaelCc: Michael, Lucas_Werkmeister_WMDE, Aklapper, Nandana, Lahi, Gq86, GoranSMilovanovic, QZanden, merbst, LawExplorer, _jensen, Agabi10, D3r1ck01, Jonas, Wikidata-bugs, aude, Lydia_Pintscher, Mbch331___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
