[Wikidata-bugs] [Maniphest] [Commented On] T189409: Strong reduction of computing time at Wikivoyage needed
Anomie added a comment. FYI, Scribunto's profiler now reports Scribunto_LuaSandboxCallback::callParserFunction 1560 ms 45.1% Scribunto_LuaSandboxCallback::getEntity 580 ms 16.8% Scribunto_LuaSandboxCallback::gsub 320 ms 9.2% Scribunto_LuaSandboxCallback::addStatementUsage 300 ms 8.7% ? 100 ms 2.9% Scribunto_LuaSandboxCallback::preprocess 100 ms 2.9% recursiveClone 60 ms 1.7% Scribunto_LuaSandboxCallback::getExpandedArgument 60 ms 1.7% Scribunto_LuaSandboxCallback::getFileInfo 40 ms 1.2% Scribunto_LuaSandboxCallback::fullUrl 40 ms 1.2% [others] 300 ms 8.7% for a preview of that page.TASK DETAILhttps://phabricator.wikimedia.org/T189409EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: AnomieCc: hoo, Lucas_Werkmeister_WMDE, gerritbot, Agabi10, Marostegui, Anomie, Reedy, thiemowmde, Lydia_Pintscher, MaxSem, RolandUnger, Lahi, Gq86, Darkminds3113, GoranSMilovanovic, Jrbranaa, QZanden, LawExplorer, Vali.matei, SundanceRaphael, Volker_E, Wong128hk, Wikidata-bugs, aude, GWicke, Dinoguy1000, jayvdb, MrStradivarius, Jackmcbarn, Mbch331, Jay8g___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T189409: Strong reduction of computing time at Wikivoyage needed
RolandUnger added a comment. Today the computing time was similar to the values a few weeks ago. For https://de.wikivoyage.org/wiki/Halle_%28Saale%29, we have now 3700-3900 ms as compared to about yesterday's 8000 ms for 237 vCard templates. Comparing with other articles I estimated a computing time of about 11 ms for a vCard without Wikidata access and about 20 ms for a vCard with Wikidata access. The difference seems to be mainly attributed to mw.wikibase.getEntity( id ) calls.TASK DETAILhttps://phabricator.wikimedia.org/T189409EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: RolandUngerCc: hoo, Lucas_Werkmeister_WMDE, gerritbot, Agabi10, Marostegui, Anomie, Reedy, thiemowmde, Lydia_Pintscher, MaxSem, RolandUnger, Majesticalreaper22, Giuliamocci, Adrian1985, Cpaulf30, Lahi, Gq86, Baloch007, Darkminds3113, Lordiis, GoranSMilovanovic, Adik2382, Jrbranaa, Th3d3v1ls, Ramalepe, Liugev6, QZanden, LawExplorer, Vali.matei, Lewizho99, Maathavan, SundanceRaphael, Volker_E, Wong128hk, Wikidata-bugs, aude, GWicke, Dinoguy1000, jayvdb, MrStradivarius, Jackmcbarn, Mbch331, Jay8g___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T189409: Strong reduction of computing time at Wikivoyage needed
gerritbot added a comment. Change 418906 merged by jenkins-bot: [mediawiki/extensions/Wikibase@master] Test UsageDeduplicator not being called more than once https://gerrit.wikimedia.org/r/418906TASK DETAILhttps://phabricator.wikimedia.org/T189409EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: gerritbotCc: hoo, Lucas_Werkmeister_WMDE, gerritbot, Agabi10, Marostegui, Anomie, Reedy, thiemowmde, Lydia_Pintscher, MaxSem, RolandUnger, Giuliamocci, Adrian1985, Cpaulf30, Lahi, Gq86, Baloch007, Darkminds3113, Lordiis, GoranSMilovanovic, Adik2382, Jrbranaa, Th3d3v1ls, Ramalepe, Liugev6, QZanden, LawExplorer, Vali.matei, Lewizho99, Maathavan, SundanceRaphael, Volker_E, Wong128hk, Wikidata-bugs, aude, GWicke, Dinoguy1000, jayvdb, MrStradivarius, Jackmcbarn, Mbch331, Jay8g___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T189409: Strong reduction of computing time at Wikivoyage needed
RolandUnger added a comment. @thiemowmde Of course I can wait until Wednesday/Thursday to test it again. The report will follow.TASK DETAILhttps://phabricator.wikimedia.org/T189409EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: RolandUngerCc: hoo, Lucas_Werkmeister_WMDE, gerritbot, Agabi10, Marostegui, Anomie, Reedy, thiemowmde, Lydia_Pintscher, MaxSem, RolandUnger, Giuliamocci, Adrian1985, Cpaulf30, Lahi, Gq86, Baloch007, Darkminds3113, Lordiis, GoranSMilovanovic, Adik2382, Jrbranaa, Th3d3v1ls, Ramalepe, Liugev6, QZanden, LawExplorer, Vali.matei, Lewizho99, Minhnv-2809, Maathavan, SundanceRaphael, Volker_E, Wong128hk, Luke081515, Wikidata-bugs, aude, GWicke, Dinoguy1000, jayvdb, MrStradivarius, Jackmcbarn, Mbch331, Jay8g, Krenair___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T189409: Strong reduction of computing time at Wikivoyage needed
gerritbot added a comment. Change 418900 merged by jenkins-bot: [mediawiki/extensions/Wikibase@master] Micro-optimization of EntityUsage::getIdentityString https://gerrit.wikimedia.org/r/418900TASK DETAILhttps://phabricator.wikimedia.org/T189409EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: gerritbotCc: hoo, Lucas_Werkmeister_WMDE, gerritbot, Agabi10, Marostegui, Anomie, Reedy, thiemowmde, Lydia_Pintscher, MaxSem, RolandUnger, Giuliamocci, Adrian1985, Cpaulf30, Lahi, Gq86, Baloch007, Darkminds3113, Lordiis, GoranSMilovanovic, Adik2382, Th3d3v1ls, Ramalepe, Liugev6, QZanden, LawExplorer, Vali.matei, Lewizho99, Minhnv-2809, Maathavan, SundanceRaphael, Volker_E, Luke081515, Wikidata-bugs, aude, GWicke, Dinoguy1000, jayvdb, MrStradivarius, Jackmcbarn, Mbch331, Jay8g, Krenair___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T189409: Strong reduction of computing time at Wikivoyage needed
Lucas_Werkmeister_WMDE added a comment. dewikivoyage is in group1, so as far as I understand it will only be updated on Wednesday, not Tuesday.TASK DETAILhttps://phabricator.wikimedia.org/T189409EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Lucas_Werkmeister_WMDECc: hoo, Lucas_Werkmeister_WMDE, gerritbot, Agabi10, Marostegui, Anomie, Reedy, thiemowmde, Lydia_Pintscher, MaxSem, RolandUnger, Giuliamocci, Adrian1985, Cpaulf30, Lahi, Gq86, Baloch007, Darkminds3113, Lordiis, GoranSMilovanovic, Adik2382, Th3d3v1ls, Ramalepe, Liugev6, QZanden, LawExplorer, Vali.matei, Lewizho99, Minhnv-2809, Maathavan, SundanceRaphael, Volker_E, Luke081515, Wikidata-bugs, aude, GWicke, Dinoguy1000, jayvdb, MrStradivarius, Jackmcbarn, Mbch331, Jay8g, Krenair___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T189409: Strong reduction of computing time at Wikivoyage needed
gerritbot added a comment. Change 418887 merged by jenkins-bot: [mediawiki/extensions/Wikibase@master] Fix UsageDeduplicator accidentially being called way to often https://gerrit.wikimedia.org/r/418887TASK DETAILhttps://phabricator.wikimedia.org/T189409EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: gerritbotCc: hoo, Lucas_Werkmeister_WMDE, gerritbot, Agabi10, Marostegui, Anomie, Reedy, thiemowmde, Lydia_Pintscher, MaxSem, RolandUnger, Giuliamocci, Adrian1985, Cpaulf30, Lahi, Gq86, Baloch007, Darkminds3113, Lordiis, GoranSMilovanovic, Adik2382, Th3d3v1ls, Ramalepe, Liugev6, QZanden, LawExplorer, Vali.matei, Lewizho99, Minhnv-2809, Maathavan, SundanceRaphael, Volker_E, Luke081515, Wikidata-bugs, aude, GWicke, Dinoguy1000, jayvdb, MrStradivarius, Jackmcbarn, Mbch331, Jay8g, Krenair___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T189409: Strong reduction of computing time at Wikivoyage needed
gerritbot added a comment. Change 418906 had a related patch set uploaded (by Thiemo Kreuz (WMDE); owner: Thiemo Kreuz (WMDE)): [mediawiki/extensions/Wikibase@master] Test UsageDeduplicator not beeing called more than once https://gerrit.wikimedia.org/r/418906TASK DETAILhttps://phabricator.wikimedia.org/T189409EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: gerritbotCc: Lucas_Werkmeister_WMDE, gerritbot, Agabi10, Marostegui, Anomie, Reedy, thiemowmde, Lydia_Pintscher, MaxSem, RolandUnger, Giuliamocci, Adrian1985, Cpaulf30, Lahi, Gq86, Baloch007, Darkminds3113, Lordiis, GoranSMilovanovic, Adik2382, Th3d3v1ls, Ramalepe, Liugev6, QZanden, LawExplorer, Vali.matei, Lewizho99, Minhnv-2809, Maathavan, SundanceRaphael, Volker_E, Luke081515, Wikidata-bugs, aude, GWicke, Dinoguy1000, jayvdb, MrStradivarius, Jackmcbarn, Mbch331, Jay8g, Krenair___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T189409: Strong reduction of computing time at Wikivoyage needed
gerritbot added a comment. Change 418900 had a related patch set uploaded (by Thiemo Kreuz (WMDE); owner: Thiemo Kreuz (WMDE)): [mediawiki/extensions/Wikibase@master] Micro-optimization of EntityUsage::getIdentityString https://gerrit.wikimedia.org/r/418900TASK DETAILhttps://phabricator.wikimedia.org/T189409EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: gerritbotCc: gerritbot, Agabi10, Marostegui, Anomie, Reedy, thiemowmde, Lydia_Pintscher, MaxSem, RolandUnger, Giuliamocci, Adrian1985, Cpaulf30, Lahi, Gq86, Baloch007, Darkminds3113, Lordiis, GoranSMilovanovic, Adik2382, Th3d3v1ls, Ramalepe, Liugev6, QZanden, LawExplorer, Vali.matei, Lewizho99, Minhnv-2809, Maathavan, SundanceRaphael, Volker_E, Luke081515, Wikidata-bugs, aude, GWicke, Dinoguy1000, jayvdb, MrStradivarius, Jackmcbarn, Mbch331, Jay8g, Krenair___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T189409: Strong reduction of computing time at Wikivoyage needed
gerritbot added a comment. Change 418887 had a related patch set uploaded (by Thiemo Kreuz (WMDE); owner: Thiemo Kreuz (WMDE)): [mediawiki/extensions/Wikibase@master] Fix UsageDeduplicator accidentially being called way to often https://gerrit.wikimedia.org/r/418887TASK DETAILhttps://phabricator.wikimedia.org/T189409EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: gerritbotCc: gerritbot, Agabi10, Marostegui, Anomie, Reedy, thiemowmde, Lydia_Pintscher, MaxSem, RolandUnger, Lahi, Gq86, Darkminds3113, GoranSMilovanovic, QZanden, LawExplorer, Vali.matei, Minhnv-2809, SundanceRaphael, Volker_E, Luke081515, Wikidata-bugs, aude, GWicke, Dinoguy1000, jayvdb, MrStradivarius, Jackmcbarn, Mbch331, Jay8g, Krenair___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T189409: Strong reduction of computing time at Wikivoyage needed
Marostegui added a comment. The DB servers serving wikivoyage databases and wikidata have the same HW. We haven't replaced/change any HW for any of those sections: The servers in s3 (wikivoyage) are the same (512GB + SSDs): the vslow server is the slowest one, but it only receives 5% of the main traffic (and 100% of the vslow/dump traffic). The servers in s8 (wikidata) are all the same (512GB + SSDs). Again, we have not changed/replaced HW in any of those sections in the past weeks.TASK DETAILhttps://phabricator.wikimedia.org/T189409EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: MarosteguiCc: Marostegui, Anomie, Reedy, thiemowmde, Lydia_Pintscher, MaxSem, RolandUnger, Lahi, Gq86, Darkminds3113, GoranSMilovanovic, QZanden, LawExplorer, Vali.matei, Minhnv-2809, SundanceRaphael, Volker_E, Luke081515, Wikidata-bugs, aude, GWicke, Dinoguy1000, jayvdb, MrStradivarius, Jackmcbarn, Mbch331, Jay8g, Krenair___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T189409: Strong reduction of computing time at Wikivoyage needed
RolandUnger added a comment. Thanks very much for your answers. As I told now parsing takes now between 8 to 10 s, a few weeks ago it was only 4 s. I have no explanations for this phenomenon of the 100 % increase of the computing time. Is there anybody who knows the reason? I estimated the timings from the profiling data in the html script comparing vCard templates with and without Wikidata usage. If I add Scribunto_LuaSandboxCallback::getEntity and Scribunto_LuaSandboxCallback::addStatementUsage times then I get about 5000 ms as I estimated it above. But I thought that most of the time was used for Scribunto_LuaSandboxCallback::getEntity not for accessing the claims table. Our intention is to use Wikidata as much as possible. This is necessary for multiple use in a multi-language project like Wikivoyage. For instance, we are fetching almost all data from data sets like https://www.wikidata.org/wiki/Q42016015. So we have of course many (sometimes up to 20) accesses to the claims table. As it was told to me by the Wikidata team this is less expensive than using mw.wikibase.getBestStatements( id, p ) for each property. Maybe there was a change of the anEntity:getBestStatements( p ) function causing that considerable increase in the computing time. Of course, we are using frame:extensionTag() once per template call. It is done in Module:Map generating a tag. There is no other way to do this.TASK DETAILhttps://phabricator.wikimedia.org/T189409EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: RolandUngerCc: Anomie, Reedy, thiemowmde, Lydia_Pintscher, MaxSem, RolandUnger, Lahi, Gq86, Darkminds3113, GoranSMilovanovic, QZanden, Marostegui, LawExplorer, Vali.matei, Minhnv-2809, SundanceRaphael, Volker_E, Luke081515, Wikidata-bugs, aude, GWicke, Dinoguy1000, jayvdb, MrStradivarius, Jackmcbarn, Mbch331, Jay8g, Krenair___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
