[Wikidata-bugs] [Maniphest] [Commented On] T187521: Optimize recentchanges and wbc_entity_usage table across wikis

2018-07-11 Thread Stashbot
Stashbot added a comment.
Mentioned in SAL (#wikimedia-operations) [2018-07-12T05:01:23Z]  Optimize wbc_entity_usage on bewiki cewiki dawiki hywiki ttwiki on db1075 (s3 primary master) - T187521TASK DETAILhttps://phabricator.wikimedia.org/T187521EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Marostegui, StashbotCc: Stashbot, Base, Cosine02, jcrespo, GoranSMilovanovic, Marostegui, Aklapper, revi, Lydia_Pintscher, Ladsgroup, AndyTan, Lahi, Gq86, QZanden, LawExplorer, Minhnv-2809, Luke081515, Wikidata-bugs, aude, Mbch331, Jay8g, Krenair___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs


[Wikidata-bugs] [Maniphest] [Commented On] T187521: Optimize recentchanges and wbc_entity_usage table across wikis

2018-07-11 Thread Stashbot
Stashbot added a comment.
Mentioned in SAL (#wikimedia-operations) [2018-07-12T04:48:28Z]  Optimize wbc_entity_usage on bewiki cewiki dawiki hywiki ttwiki on db1123 - T187521TASK DETAILhttps://phabricator.wikimedia.org/T187521EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Marostegui, StashbotCc: Stashbot, Base, Cosine02, jcrespo, GoranSMilovanovic, Marostegui, Aklapper, revi, Lydia_Pintscher, Ladsgroup, AndyTan, Lahi, Gq86, QZanden, LawExplorer, Minhnv-2809, Luke081515, Wikidata-bugs, aude, Mbch331, Jay8g, Krenair___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs


[Wikidata-bugs] [Maniphest] [Commented On] T187521: Optimize recentchanges and wbc_entity_usage table across wikis

2018-07-11 Thread Stashbot
Stashbot added a comment.
Mentioned in SAL (#wikimedia-operations) [2018-07-11T08:14:27Z]  Optimize wbc_entity_usage on bewiki cewiki dawiki hywiki ttwiki on db1077 with replication, this will generate lag on s3 labs hosts - T187521TASK DETAILhttps://phabricator.wikimedia.org/T187521EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Marostegui, StashbotCc: Stashbot, Base, Cosine02, jcrespo, GoranSMilovanovic, Marostegui, Aklapper, revi, Lydia_Pintscher, Ladsgroup, AndyTan, Lahi, Gq86, QZanden, LawExplorer, Minhnv-2809, Luke081515, Wikidata-bugs, aude, Mbch331, Jay8g, Krenair___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs


[Wikidata-bugs] [Maniphest] [Commented On] T187521: Optimize recentchanges and wbc_entity_usage table across wikis

2018-07-11 Thread Stashbot
Stashbot added a comment.
Mentioned in SAL (#wikimedia-operations) [2018-07-11T07:51:29Z]  Optimize wbc_entity_usage on bewiki cewiki dawiki hywiki ttwiki on dbstore1002:s3, db1078 - T187521TASK DETAILhttps://phabricator.wikimedia.org/T187521EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Marostegui, StashbotCc: Stashbot, Base, Cosine02, jcrespo, GoranSMilovanovic, Marostegui, Aklapper, revi, Lydia_Pintscher, Ladsgroup, AndyTan, Lahi, Gq86, QZanden, LawExplorer, Minhnv-2809, Luke081515, Wikidata-bugs, aude, Mbch331, Jay8g, Krenair___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs


[Wikidata-bugs] [Maniphest] [Commented On] T187521: Optimize recentchanges and wbc_entity_usage table across wikis

2018-07-11 Thread Stashbot
Stashbot added a comment.
Mentioned in SAL (#wikimedia-operations) [2018-07-11T06:59:01Z]  Optimize wbc_entity_usage on bewiki cewiki dawiki hywiki ttwiki on db2043 (s3 codfw master), this will generate lag on s3 codfw - T187521TASK DETAILhttps://phabricator.wikimedia.org/T187521EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Marostegui, StashbotCc: Stashbot, Base, Cosine02, jcrespo, GoranSMilovanovic, Marostegui, Aklapper, revi, Lydia_Pintscher, Ladsgroup, AndyTan, Lahi, Gq86, QZanden, LawExplorer, Minhnv-2809, Luke081515, Wikidata-bugs, aude, Mbch331, Jay8g, Krenair___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs


[Wikidata-bugs] [Maniphest] [Commented On] T187521: Optimize recentchanges and wbc_entity_usage table across wikis

2018-07-11 Thread Stashbot
Stashbot added a comment.
Mentioned in SAL (#wikimedia-operations) [2018-07-11T06:30:30Z]  Optimize wbc_entity_usage on arwiki cawiki huwiki rowiki ukwiki on db1062 (s7 primary master) - T187521TASK DETAILhttps://phabricator.wikimedia.org/T187521EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Marostegui, StashbotCc: Stashbot, Base, Cosine02, jcrespo, GoranSMilovanovic, Marostegui, Aklapper, revi, Lydia_Pintscher, Ladsgroup, AndyTan, Lahi, Gq86, QZanden, LawExplorer, Minhnv-2809, Luke081515, Wikidata-bugs, aude, Mbch331, Jay8g, Krenair___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs


[Wikidata-bugs] [Maniphest] [Commented On] T187521: Optimize recentchanges and wbc_entity_usage table across wikis

2018-07-11 Thread Stashbot
Stashbot added a comment.
Mentioned in SAL (#wikimedia-operations) [2018-07-11T06:12:32Z]  Optimize wbc_entity_usage on arwiki cawiki huwiki rowiki ukwiki on db1086 - T187521TASK DETAILhttps://phabricator.wikimedia.org/T187521EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Marostegui, StashbotCc: Stashbot, Base, Cosine02, jcrespo, GoranSMilovanovic, Marostegui, Aklapper, revi, Lydia_Pintscher, Ladsgroup, AndyTan, Lahi, Gq86, QZanden, LawExplorer, Minhnv-2809, Luke081515, Wikidata-bugs, aude, Mbch331, Jay8g, Krenair___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs


[Wikidata-bugs] [Maniphest] [Commented On] T187521: Optimize recentchanges and wbc_entity_usage table across wikis

2018-07-10 Thread Stashbot
Stashbot added a comment.
Mentioned in SAL (#wikimedia-operations) [2018-07-11T05:49:10Z]  Optimize wbc_entity_usage on arwiki cawiki huwiki rowiki ukwiki on db1079 with replication, this will generate lag on s7 labs hosts - T187521TASK DETAILhttps://phabricator.wikimedia.org/T187521EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Marostegui, StashbotCc: Stashbot, Base, Cosine02, jcrespo, GoranSMilovanovic, Marostegui, Aklapper, revi, Lydia_Pintscher, Ladsgroup, AndyTan, Lahi, Gq86, QZanden, LawExplorer, Minhnv-2809, Luke081515, Wikidata-bugs, aude, Mbch331, Jay8g, Krenair___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs


[Wikidata-bugs] [Maniphest] [Commented On] T187521: Optimize recentchanges and wbc_entity_usage table across wikis

2018-07-10 Thread Stashbot
Stashbot added a comment.
Mentioned in SAL (#wikimedia-operations) [2018-07-11T05:23:34Z]  Optimize wbc_entity_usage on arwiki cawiki huwiki rowiki ukwiki on db1090 - T187521TASK DETAILhttps://phabricator.wikimedia.org/T187521EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Marostegui, StashbotCc: Stashbot, Base, Cosine02, jcrespo, GoranSMilovanovic, Marostegui, Aklapper, revi, Lydia_Pintscher, Ladsgroup, AndyTan, Lahi, Gq86, QZanden, LawExplorer, Minhnv-2809, Luke081515, Wikidata-bugs, aude, Mbch331, Jay8g, Krenair___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs


[Wikidata-bugs] [Maniphest] [Commented On] T187521: Optimize recentchanges and wbc_entity_usage table across wikis

2018-07-10 Thread Stashbot
Stashbot added a comment.
Mentioned in SAL (#wikimedia-operations) [2018-07-11T05:06:48Z]  Optimize wbc_entity_usage on arwiki cawiki huwiki rowiki ukwiki on db1094 - T187521TASK DETAILhttps://phabricator.wikimedia.org/T187521EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Marostegui, StashbotCc: Stashbot, Base, Cosine02, jcrespo, GoranSMilovanovic, Marostegui, Aklapper, revi, Lydia_Pintscher, Ladsgroup, AndyTan, Lahi, Gq86, QZanden, LawExplorer, Minhnv-2809, Luke081515, Wikidata-bugs, aude, Mbch331, Jay8g, Krenair___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs


[Wikidata-bugs] [Maniphest] [Commented On] T187521: Optimize recentchanges and wbc_entity_usage table across wikis

2018-07-10 Thread Stashbot
Stashbot added a comment.
Mentioned in SAL (#wikimedia-operations) [2018-07-10T06:32:37Z]  Optimize wbc_entity_usage on arwiki cawiki huwiki rowiki ukwiki on s7 codfw master (db2040) with replication, this will generate lag on s7 codfw - T187521TASK DETAILhttps://phabricator.wikimedia.org/T187521EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Marostegui, StashbotCc: Stashbot, Base, Cosine02, jcrespo, GoranSMilovanovic, Marostegui, Aklapper, revi, Lydia_Pintscher, Ladsgroup, AndyTan, Lahi, Gq86, QZanden, LawExplorer, Minhnv-2809, Luke081515, Wikidata-bugs, aude, Mbch331, Jay8g, Krenair___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs


[Wikidata-bugs] [Maniphest] [Commented On] T187521: Optimize recentchanges and wbc_entity_usage table across wikis

2018-07-09 Thread Stashbot
Stashbot added a comment.
Mentioned in SAL (#wikimedia-operations) [2018-07-10T05:17:21Z]  Optimize frwiki.wbc_entity_usage on s6 eqiad hosts T187521TASK DETAILhttps://phabricator.wikimedia.org/T187521EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Marostegui, StashbotCc: Stashbot, Base, Cosine02, jcrespo, GoranSMilovanovic, Marostegui, Aklapper, revi, Lydia_Pintscher, Ladsgroup, AndyTan, Lahi, Gq86, QZanden, LawExplorer, Minhnv-2809, Luke081515, Wikidata-bugs, aude, Mbch331, Jay8g, Krenair___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs


[Wikidata-bugs] [Maniphest] [Commented On] T187521: Optimize recentchanges and wbc_entity_usage table across wikis

2018-07-09 Thread Stashbot
Stashbot added a comment.
Mentioned in SAL (#wikimedia-operations) [2018-07-10T04:59:28Z]  Optimize frwiki.wbc_entity_usage on s6 codfw, this will generate lag on s6 codfw - T187521TASK DETAILhttps://phabricator.wikimedia.org/T187521EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Marostegui, StashbotCc: Stashbot, Base, Cosine02, jcrespo, GoranSMilovanovic, Marostegui, Aklapper, revi, Lydia_Pintscher, Ladsgroup, AndyTan, Lahi, Gq86, QZanden, LawExplorer, Minhnv-2809, Luke081515, Wikidata-bugs, aude, Mbch331, Jay8g, Krenair___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs


[Wikidata-bugs] [Maniphest] [Commented On] T187521: Optimize recentchanges and wbc_entity_usage table across wikis

2018-07-09 Thread Stashbot
Stashbot added a comment.
Mentioned in SAL (#wikimedia-operations) [2018-07-10T04:36:43Z]  Optimize bgwiki itwiki svwiki zhwiki wbc_entity_usage on db1066 (s2 primary master) - T187521TASK DETAILhttps://phabricator.wikimedia.org/T187521EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Marostegui, StashbotCc: Stashbot, Base, Cosine02, jcrespo, GoranSMilovanovic, Marostegui, Aklapper, revi, Lydia_Pintscher, Ladsgroup, AndyTan, Lahi, Gq86, QZanden, LawExplorer, Minhnv-2809, Luke081515, Wikidata-bugs, aude, Mbch331, Jay8g, Krenair___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs


[Wikidata-bugs] [Maniphest] [Commented On] T187521: Optimize recentchanges and wbc_entity_usage table across wikis

2018-07-09 Thread Stashbot
Stashbot added a comment.
Mentioned in SAL (#wikimedia-operations) [2018-07-09T08:31:07Z]  Optimize bgwiki itwiki svwiki zhwiki wbc_entity_usage on db1074 with replication, this will generate lag on s2 on labshosts - T187521TASK DETAILhttps://phabricator.wikimedia.org/T187521EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: StashbotCc: Stashbot, Base, Cosine02, jcrespo, GoranSMilovanovic, Marostegui, Aklapper, revi, Lydia_Pintscher, Ladsgroup, AndyTan, Lahi, Gq86, QZanden, LawExplorer, Minhnv-2809, Luke081515, Wikidata-bugs, aude, Mbch331, Jay8g, Krenair___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs


[Wikidata-bugs] [Maniphest] [Commented On] T187521: Optimize recentchanges and wbc_entity_usage table across wikis

2018-07-09 Thread Stashbot
Stashbot added a comment.
Mentioned in SAL (#wikimedia-operations) [2018-07-09T06:03:46Z]  Optimize bgwiki itwiki svwiki zhwiki wbc_entity_usage on dbstore1002:s2, db1122 and db1105 - T187521TASK DETAILhttps://phabricator.wikimedia.org/T187521EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: StashbotCc: Stashbot, Base, Cosine02, jcrespo, GoranSMilovanovic, Marostegui, Aklapper, revi, Lydia_Pintscher, Ladsgroup, AndyTan, Lahi, Gq86, QZanden, LawExplorer, Minhnv-2809, Luke081515, Wikidata-bugs, aude, Mbch331, Jay8g, Krenair___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs


[Wikidata-bugs] [Maniphest] [Commented On] T187521: Optimize recentchanges and wbc_entity_usage table across wikis

2018-07-08 Thread Stashbot
Stashbot added a comment.
Mentioned in SAL (#wikimedia-operations) [2018-07-09T05:41:25Z]  Optimize bgwiki itwiki svwiki zhwiki wbc_entity_usage on s2 codfw master with replication - lag will happen on s2 codfw - T187521TASK DETAILhttps://phabricator.wikimedia.org/T187521EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: StashbotCc: Stashbot, Base, Cosine02, jcrespo, GoranSMilovanovic, Marostegui, Aklapper, revi, Lydia_Pintscher, Ladsgroup, AndyTan, Lahi, Gq86, QZanden, LawExplorer, Minhnv-2809, Luke081515, Wikidata-bugs, aude, Mbch331, Jay8g, Krenair___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs


[Wikidata-bugs] [Maniphest] [Commented On] T187521: Optimize recentchanges and wbc_entity_usage table across wikis

2018-04-17 Thread Ladsgroup
Ladsgroup added a comment.
Thanks!TASK DETAILhttps://phabricator.wikimedia.org/T187521EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: LadsgroupCc: jcrespo, GoranSMilovanovic, Marostegui, Aklapper, revi, Lydia_Pintscher, Ladsgroup, Lahi, Gq86, QZanden, LawExplorer, Minhnv-2809, Luke081515, Wikidata-bugs, aude, Mbch331, Jay8g, Krenair___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs


[Wikidata-bugs] [Maniphest] [Commented On] T187521: Optimize recentchanges and wbc_entity_usage table across wikis

2018-04-17 Thread Ladsgroup
Ladsgroup added a comment.
hmm, I can see why. Mostly jobqueue stuff. Can you file a phabrictor ticket so we can pick it up and do something about it?TASK DETAILhttps://phabricator.wikimedia.org/T187521EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: LadsgroupCc: jcrespo, GoranSMilovanovic, Marostegui, Aklapper, revi, Lydia_Pintscher, Ladsgroup, Lahi, Gq86, QZanden, LawExplorer, Minhnv-2809, Luke081515, Wikidata-bugs, aude, Mbch331, Jay8g, Krenair___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs


[Wikidata-bugs] [Maniphest] [Commented On] T187521: Optimize recentchanges and wbc_entity_usage table across wikis

2018-04-16 Thread jcrespo
jcrespo added a comment.
commonswiki errors due to deadlocks on INSERT IGNORE INTO wbc_entity_usage  seem to be common (not too worrying, but on of the most comon database errors), could the code be optimized to avoid those? I am guessing that the same row is written many times (once per change on the same item), and maybe that could be simplified somehow. INSERT IGNORE is a bit of a bad trick here, and we may be writing multiple times the same data without need. Given the changes are done by the job queue and arrive in any order, maybe transaction serialization can be relaxed?TASK DETAILhttps://phabricator.wikimedia.org/T187521EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: jcrespoCc: jcrespo, GoranSMilovanovic, Marostegui, Aklapper, revi, Lydia_Pintscher, Ladsgroup, Lahi, Gq86, QZanden, LawExplorer, Minhnv-2809, Luke081515, Wikidata-bugs, aude, Mbch331, Jay8g, Krenair___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs


[Wikidata-bugs] [Maniphest] [Commented On] T187521: Optimize recentchanges and wbc_entity_usage table across wikis

2018-02-15 Thread Marostegui
Marostegui added a comment.
That's ok, it goes to dbstore1002 then. All good.TASK DETAILhttps://phabricator.wikimedia.org/T187521EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: MarosteguiCc: jcrespo, GoranSMilovanovic, Marostegui, Aklapper, revi, Lydia_Pintscher, Ladsgroup, Lahi, Gq86, QZanden, LawExplorer, Minhnv-2809, Luke081515, Wikidata-bugs, aude, Mbch331, Jay8g, Krenair___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs


[Wikidata-bugs] [Maniphest] [Commented On] T187521: Optimize recentchanges and wbc_entity_usage table across wikis

2018-02-15 Thread GoranSMilovanovic
GoranSMilovanovic added a comment.
@Marostegui The R script that orchestrates Apache Sqoop connects to analytics-store.eqiad.wmnet by using my analytics-research-client.cnf credentials from stat1004 - I wouldn't know exactly the server to which that resolves.TASK DETAILhttps://phabricator.wikimedia.org/T187521EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: GoranSMilovanovicCc: jcrespo, GoranSMilovanovic, Marostegui, Aklapper, revi, Lydia_Pintscher, Ladsgroup, Lahi, Gq86, QZanden, LawExplorer, Minhnv-2809, Luke081515, Wikidata-bugs, aude, Mbch331, Jay8g, Krenair___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs


[Wikidata-bugs] [Maniphest] [Commented On] T187521: Optimize recentchanges and wbc_entity_usage table across wikis

2018-02-15 Thread GoranSMilovanovic
GoranSMilovanovic added a comment.
@Marostegui m = 0, h = 0, dom = 7,14,21,29, mon = *, dow = *, i.e. every 7th, 14th, 21st, and 29th of the month, 00:00 UTC.TASK DETAILhttps://phabricator.wikimedia.org/T187521EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: GoranSMilovanovicCc: jcrespo, GoranSMilovanovic, Marostegui, Aklapper, revi, Lydia_Pintscher, Ladsgroup, Lahi, Gq86, QZanden, LawExplorer, Minhnv-2809, Luke081515, Wikidata-bugs, aude, Mbch331, Jay8g, Krenair___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs


[Wikidata-bugs] [Maniphest] [Commented On] T187521: Optimize recentchanges and wbc_entity_usage table across wikis

2018-02-15 Thread jcrespo
jcrespo added a comment.
Sqoops the wbc_entity_usage tables

Also, which db server is being used? analytics-replica/analytics store/dbstore1002 is ok to do that, others are not.TASK DETAILhttps://phabricator.wikimedia.org/T187521EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: jcrespoCc: jcrespo, GoranSMilovanovic, Marostegui, Aklapper, revi, Lydia_Pintscher, Ladsgroup, Lahi, Gq86, QZanden, LawExplorer, Minhnv-2809, Luke081515, Wikidata-bugs, aude, Mbch331, Jay8g, Krenair___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs


[Wikidata-bugs] [Maniphest] [Commented On] T187521: Optimize recentchanges and wbc_entity_usage table across wikis

2018-02-15 Thread Marostegui
Marostegui added a comment.

In T187521#3977723, @GoranSMilovanovic wrote:
@Ladsgroup  @Marostegui  I have a cron job on stat1004 that Sqoops the wbc_entity_usage tables for all projects into a HiveQL table for the Wikidata Concepts Monitor pre-processing. The cron job runs on a weekly schedule. Please let me know if you think it would be affected by whatever optimization you plan to do there. No serious problems would be caused if I would need to drop a weekly update or two if you predict any interactions. Thanks.


There should not be any issues with those optimizations and your cronjob.
Just for knowing it, which day/time does it run?TASK DETAILhttps://phabricator.wikimedia.org/T187521EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: MarosteguiCc: GoranSMilovanovic, Marostegui, Aklapper, revi, Lydia_Pintscher, Ladsgroup, Lahi, Gq86, QZanden, LawExplorer, Minhnv-2809, Luke081515, Wikidata-bugs, aude, Mbch331, Jay8g, Krenair___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs


[Wikidata-bugs] [Maniphest] [Commented On] T187521: Optimize recentchanges and wbc_entity_usage table across wikis

2018-02-15 Thread GoranSMilovanovic
GoranSMilovanovic added a comment.
@Ladsgroup  @Marostegui  I have a cron job on stat1004 that Sqoops the wbc_entity_usage tables for all projects into a HiveQL table for the Wikidata Concepts Monitor pre-processing. The cron job runs on a weekly schedule. Please let me know if you think it would be affected by whatever optimization you plan to do there. No serious problems would be caused if I would need to drop a weekly update or two if you predict any interactions. Thanks.TASK DETAILhttps://phabricator.wikimedia.org/T187521EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: GoranSMilovanovicCc: GoranSMilovanovic, Marostegui, Aklapper, revi, Lydia_Pintscher, Ladsgroup, Lahi, Gq86, QZanden, LawExplorer, Minhnv-2809, Luke081515, Wikidata-bugs, aude, Mbch331, Jay8g, Krenair___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs