[Wikidata-bugs] [Maniphest] [Updated] T208425: [EPIC] Kill the wb_terms table

[Maintenance_bot]

Maintenance_bot removed a project: Patch-For-Review.

TASK DETAIL
  https://phabricator.wikimedia.org/T208425

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Maintenance_bot
Cc: Mahir256, Pasleim, Jony, Lea_Lacroix_WMDE, Bugreporter, Ladsgroup, 
JAllemandou, ArielGlenn, jcrespo, Joe, Gehel, alaa_wmde, Marostegui, 
Jdforrester-WMF, Addshore, Aklapper, darthmon_wmde, Nandana, Lahi, Gq86, 
Pablo-WMDE, GoranSMilovanovic, QZanden, LawExplorer, _jensen, rosalieper, 
Scott_WUaS, Wikidata-bugs, aude, Lydia_Pintscher, Mbch331, Blissjay007, 
Oblanco79, Alter-paule, Beast1978, Un1tY, Hook696, Daryl-TTMG, RomaAmorRoma, 
E.S.A-Sheild, Kent7301, Meekrab2012, joker88john, CucyNoiD, NebulousIris, 
Gaboe420, Versusxo, Majesticalreaper22, Giuliamocci, Adrian1985, Cpaulf30, 
Af420, Darkminds3113, Bsandipan, Lordiis, Adik2382, Th3d3v1ls, Ramalepe, 
Liugev6, WSH1906, Lewizho99, Maathavan
___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Updated] T208425: [EPIC] Kill the wb_terms table

[ReleaseTaggerBot]

ReleaseTaggerBot added a project: MW-1.35-notes (1.35.0-wmf.34; 2020-05-26).

TASK DETAIL
  https://phabricator.wikimedia.org/T208425

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: ReleaseTaggerBot
Cc: Mahir256, Pasleim, Jony, Lea_Lacroix_WMDE, Bugreporter, Ladsgroup, 
JAllemandou, ArielGlenn, jcrespo, Joe, Gehel, alaa_wmde, Marostegui, 
Jdforrester-WMF, Addshore, Aklapper, Blissjay007, Oblanco79, Alter-paule, 
Beast1978, Un1tY, Hook696, Daryl-TTMG, RomaAmorRoma, E.S.A-Sheild, 
darthmon_wmde, Kent7301, Meekrab2012, joker88john, CucyNoiD, Nandana, 
NebulousIris, Gaboe420, Versusxo, Majesticalreaper22, Giuliamocci, Adrian1985, 
Cpaulf30, Lahi, Gq86, Af420, Darkminds3113, Bsandipan, Lordiis, Pablo-WMDE, 
GoranSMilovanovic, Adik2382, Th3d3v1ls, Ramalepe, Liugev6, QZanden, 
LawExplorer, WSH1906, Lewizho99, Maathavan, _jensen, rosalieper, Scott_WUaS, 
Wikidata-bugs, aude, Lydia_Pintscher, Mbch331
___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Updated] T208425: [EPIC] Kill the wb_terms table

[gerritbot]

gerritbot added a project: Patch-For-Review.

TASK DETAIL
  https://phabricator.wikimedia.org/T208425

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: gerritbot
Cc: Mahir256, Pasleim, Jony, Lea_Lacroix_WMDE, Bugreporter, Ladsgroup, 
JAllemandou, ArielGlenn, jcrespo, Joe, Gehel, alaa_wmde, Marostegui, 
Jdforrester-WMF, Addshore, Aklapper, Blissjay007, Oblanco79, Alter-paule, 
Beast1978, Un1tY, Hook696, Daryl-TTMG, RomaAmorRoma, E.S.A-Sheild, 
darthmon_wmde, Kent7301, Meekrab2012, joker88john, CucyNoiD, Nandana, 
NebulousIris, Gaboe420, Versusxo, Majesticalreaper22, Giuliamocci, Adrian1985, 
Cpaulf30, Lahi, Gq86, Af420, Darkminds3113, Bsandipan, Lordiis, Pablo-WMDE, 
GoranSMilovanovic, Adik2382, Th3d3v1ls, Ramalepe, Liugev6, QZanden, 
LawExplorer, WSH1906, Lewizho99, Maathavan, _jensen, rosalieper, Scott_WUaS, 
Wikidata-bugs, aude, Lydia_Pintscher, Mbch331
___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Updated] T208425: [EPIC] Kill the wb_terms table

[Maintenance_bot]

Maintenance_bot removed a project: Patch-For-Review.

TASK DETAIL
  https://phabricator.wikimedia.org/T208425

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Maintenance_bot
Cc: Mahir256, Pasleim, Jony, Lea_Lacroix_WMDE, Bugreporter, Ladsgroup, 
JAllemandou, ArielGlenn, jcrespo, Joe, Gehel, alaa_wmde, Marostegui, 
Jdforrester-WMF, Addshore, Aklapper, darthmon_wmde, Nandana, Lahi, Gq86, 
GoranSMilovanovic, QZanden, LawExplorer, _jensen, rosalieper, Scott_WUaS, 
Wikidata-bugs, aude, Lydia_Pintscher, Mbch331, Oblanco79, Alter-paule, 
Beast1978, Un1tY, Hook696, Daryl-TTMG, RomaAmorRoma, E.S.A-Sheild, Kent7301, 
Meekrab2012, joker88john, CucyNoiD, NebulousIris, Gaboe420, Versusxo, 
Majesticalreaper22, Giuliamocci, Adrian1985, Cpaulf30, Af420, Darkminds3113, 
Bsandipan, Lordiis, Adik2382, Th3d3v1ls, Ramalepe, Liugev6, WSH1906, Lewizho99, 
Maathavan
___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Updated] T208425: [EPIC] Kill the wb_terms table

[gerritbot]

gerritbot added a comment.


  Change 586480 had a related patch set uploaded (by Jforrester; owner: 
Ladsgroup):
  [mediawiki/extensions/Wikibase@wmf/1.35.0-wmf.26] Follow-up a956c655 
:
 Only avoid dropping wb_items_per_site so prod can be merged
  
  https://gerrit.wikimedia.org/r/586480

TASK DETAIL
  https://phabricator.wikimedia.org/T208425

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: gerritbot
Cc: Mahir256, Pasleim, Jony, Lea_Lacroix_WMDE, Bugreporter, Ladsgroup, 
JAllemandou, ArielGlenn, jcrespo, Joe, Gehel, alaa_wmde, Marostegui, 
Jdforrester-WMF, Addshore, Aklapper, darthmon_wmde, Nandana, Lahi, Gq86, 
GoranSMilovanovic, QZanden, LawExplorer, _jensen, rosalieper, Scott_WUaS, 
Wikidata-bugs, aude, Lydia_Pintscher, Mbch331
___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Updated] T208425: [EPIC] Kill the wb_terms table

[gerritbot]

gerritbot added a project: Patch-For-Review.

TASK DETAIL
  https://phabricator.wikimedia.org/T208425

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: gerritbot
Cc: Mahir256, Pasleim, Jony, Lea_Lacroix_WMDE, Bugreporter, Ladsgroup, 
JAllemandou, ArielGlenn, jcrespo, Joe, Gehel, alaa_wmde, Marostegui, 
Jdforrester-WMF, Addshore, Aklapper, Oblanco79, Alter-paule, Beast1978, Un1tY, 
Hook696, Daryl-TTMG, RomaAmorRoma, E.S.A-Sheild, darthmon_wmde, Kent7301, 
Meekrab2012, joker88john, CucyNoiD, Nandana, NebulousIris, Gaboe420, Versusxo, 
Majesticalreaper22, Giuliamocci, Adrian1985, Cpaulf30, Lahi, Gq86, Af420, 
Darkminds3113, Bsandipan, Lordiis, GoranSMilovanovic, Adik2382, Th3d3v1ls, 
Ramalepe, Liugev6, QZanden, LawExplorer, WSH1906, Lewizho99, Maathavan, 
_jensen, rosalieper, Scott_WUaS, Wikidata-bugs, aude, Lydia_Pintscher, Mbch331
___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Updated] T208425: [EPIC] Kill the wb_terms table

[Maintenance_bot]

Maintenance_bot removed a project: Patch-For-Review.

TASK DETAIL
  https://phabricator.wikimedia.org/T208425

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Maintenance_bot
Cc: Mahir256, Pasleim, Jony, Lea_Lacroix_WMDE, Bugreporter, Ladsgroup, 
JAllemandou, ArielGlenn, jcrespo, Joe, Gehel, alaa_wmde, Marostegui, 
Jdforrester-WMF, Addshore, Aklapper, darthmon_wmde, Nandana, Lahi, Gq86, 
GoranSMilovanovic, QZanden, LawExplorer, _jensen, rosalieper, Scott_WUaS, 
Wikidata-bugs, aude, Lydia_Pintscher, Mbch331, Oblanco79, Alter-paule, 
Beast1978, Un1tY, Hook696, Daryl-TTMG, RomaAmorRoma, E.S.A-Sheild, Kent7301, 
Meekrab2012, joker88john, CucyNoiD, NebulousIris, Gaboe420, Versusxo, 
Majesticalreaper22, Giuliamocci, Adrian1985, Cpaulf30, Af420, Darkminds3113, 
Bsandipan, Lordiis, Adik2382, Th3d3v1ls, Ramalepe, Liugev6, WSH1906, Lewizho99, 
Maathavan
___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Updated] T208425: [EPIC] Kill the wb_terms table

[Stashbot]

Stashbot added a comment.


  Mentioned in SAL (#wikimedia-operations) [2020-04-06T23:59:23Z] 
 Synchronized 
php-1.35.0-wmf.26/extensions/Wikibase/repo/includes/Store/Sql/DatabaseSchemaUpdater.php:
 Do not try to drop things when theres no wb_terms table T208425 
 T249565 
 (duration: 00m 59s)

TASK DETAIL
  https://phabricator.wikimedia.org/T208425

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Stashbot
Cc: Mahir256, Pasleim, Jony, Lea_Lacroix_WMDE, Bugreporter, Ladsgroup, 
JAllemandou, ArielGlenn, jcrespo, Joe, Gehel, alaa_wmde, Marostegui, 
Jdforrester-WMF, Addshore, Aklapper, Oblanco79, Alter-paule, Beast1978, Un1tY, 
Hook696, Daryl-TTMG, RomaAmorRoma, E.S.A-Sheild, darthmon_wmde, Kent7301, 
Meekrab2012, joker88john, CucyNoiD, Nandana, NebulousIris, Gaboe420, Versusxo, 
Majesticalreaper22, Giuliamocci, Adrian1985, Cpaulf30, Lahi, Gq86, Af420, 
Darkminds3113, Bsandipan, Lordiis, GoranSMilovanovic, Adik2382, Th3d3v1ls, 
Ramalepe, Liugev6, QZanden, LawExplorer, WSH1906, Lewizho99, Maathavan, 
_jensen, rosalieper, Scott_WUaS, Wikidata-bugs, aude, Lydia_Pintscher, Mbch331
___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Updated] T208425: [EPIC] Kill the wb_terms table

[gerritbot]

gerritbot added a project: Patch-For-Review.

TASK DETAIL
  https://phabricator.wikimedia.org/T208425

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: gerritbot
Cc: Mahir256, Pasleim, Jony, Lea_Lacroix_WMDE, Bugreporter, Ladsgroup, 
JAllemandou, ArielGlenn, jcrespo, Joe, Gehel, alaa_wmde, Marostegui, 
Jdforrester-WMF, Addshore, Aklapper, Oblanco79, Alter-paule, Beast1978, Un1tY, 
Hook696, Daryl-TTMG, RomaAmorRoma, E.S.A-Sheild, darthmon_wmde, Kent7301, 
Meekrab2012, joker88john, CucyNoiD, Nandana, NebulousIris, Gaboe420, Versusxo, 
Majesticalreaper22, Giuliamocci, Adrian1985, Cpaulf30, Lahi, Gq86, Af420, 
Darkminds3113, Bsandipan, Lordiis, GoranSMilovanovic, Adik2382, Th3d3v1ls, 
Ramalepe, Liugev6, QZanden, LawExplorer, WSH1906, Lewizho99, Maathavan, 
_jensen, rosalieper, Scott_WUaS, Wikidata-bugs, aude, Lydia_Pintscher, Mbch331
___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Updated] T208425: [EPIC] Kill the wb_terms table

[Addshore]

Addshore added a subtask: T221764: Overview of wb_terms redesign.

TASK DETAIL
  https://phabricator.wikimedia.org/T208425

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Addshore
Cc: Mahir256, Pasleim, Jony, Lea_Lacroix_WMDE, Bugreporter, Ladsgroup, 
JAllemandou, ArielGlenn, jcrespo, Joe, Gehel, alaa_wmde, Marostegui, 
Jdforrester-WMF, Addshore, Aklapper, darthmon_wmde, Nandana, Lahi, Gq86, 
GoranSMilovanovic, QZanden, LawExplorer, _jensen, rosalieper, Scott_WUaS, 
Wikidata-bugs, aude, Lydia_Pintscher, Mbch331
___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Updated] T208425: [EPIC] Kill the wb_terms table

[Addshore]

Addshore added a comment.


  In T208425#5992641 , 
@Mahir256 wrote:
  
  > I have a query, https://quarry.wmflabs.org/query/18763, that returns 
label/description/alias statistics for a set of languages (also mentioned in 
T197161#4300061 ). This was 
derived from a query used by @Pasleim to update this table 
 
(which worked until the end of May) and its South Asian counterpart 
 
(which worked up until wb_terms updates got turned off). I tried to rewrite the 
Quarry query to use the new databases (https://quarry.wmflabs.org/query/41692), 
but running this has not yet succeeded, either using Quarry or directly on 
tools-login. I am not sure whether this rewritten query can be simplified 
beyond what's written at present, so I was hoping there might be a better way 
of obtaining these statistics via SQL that does not presently time out.
  
  The best I can suggest with the new tables for quarry is 
https://quarry.wmflabs.org/query/43212
  You'll have to run a query per language in order to not be killed.
  
  Ideally we will calculate these numbers on the #analytics 
 cluster and publish them 
somewhere they can just be queried as we used to do.
  Probably needs T154601: Grafana: "wikidata-datamodel-terms" doesn't update 
anymore  to be fixed, thn each day 
you could grab these numbers in under a second!

TASK DETAIL
  https://phabricator.wikimedia.org/T208425

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Addshore
Cc: Mahir256, Pasleim, Jony, Lea_Lacroix_WMDE, Bugreporter, Ladsgroup, 
JAllemandou, ArielGlenn, jcrespo, Joe, Gehel, alaa_wmde, Marostegui, 
Jdforrester-WMF, Addshore, Aklapper, darthmon_wmde, Nandana, Lahi, Gq86, 
GoranSMilovanovic, QZanden, LawExplorer, _jensen, rosalieper, Scott_WUaS, 
Wikidata-bugs, aude, Lydia_Pintscher, Mbch331
___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Updated] T208425: [EPIC] Kill the wb_terms table

[Bugreporter]

Bugreporter added a comment.


  This is a bug - T248363: Haslabel treats aliases as labels 


TASK DETAIL
  https://phabricator.wikimedia.org/T208425

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Bugreporter
Cc: Mahir256, Pasleim, Jony, Lea_Lacroix_WMDE, Bugreporter, Ladsgroup, 
JAllemandou, ArielGlenn, jcrespo, Joe, Gehel, alaa_wmde, Marostegui, 
Jdforrester-WMF, Addshore, Aklapper, darthmon_wmde, Nandana, Lahi, Gq86, 
GoranSMilovanovic, QZanden, LawExplorer, _jensen, rosalieper, Scott_WUaS, 
Wikidata-bugs, aude, Lydia_Pintscher, Mbch331
___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Updated] T208425: [EPIC] Kill the wb_terms table

[Marostegui]

Marostegui removed a project: DBA.

TASK DETAIL
  https://phabricator.wikimedia.org/T208425

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Marostegui
Cc: Bugreporter, Ladsgroup, JAllemandou, ArielGlenn, jcrespo, Joe, Gehel, 
alaa_wmde, Marostegui, Jdforrester-WMF, Addshore, Aklapper, darthmon_wmde, 
Nandana, Lahi, Gq86, GoranSMilovanovic, QZanden, LawExplorer, _jensen, 
rosalieper, Scott_WUaS, Wikidata-bugs, aude, Lydia_Pintscher, Mbch331, Iflorez, 
Minhnv-2809, Jay8g, Krenair
___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Updated] T208425: [EPIC] Kill the wb_terms table

[Marostegui]

Marostegui added a project: DBA.

TASK DETAIL
  https://phabricator.wikimedia.org/T208425

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Marostegui
Cc: Bugreporter, Ladsgroup, JAllemandou, ArielGlenn, jcrespo, Joe, Gehel, 
alaa_wmde, Marostegui, Jdforrester-WMF, Addshore, Aklapper, Iflorez, 
darthmon_wmde, Nandana, Lahi, Gq86, GoranSMilovanovic, QZanden, LawExplorer, 
Minhnv-2809, _jensen, rosalieper, Scott_WUaS, Wikidata-bugs, aude, 
Lydia_Pintscher, Mbch331, Jay8g, Krenair
___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Updated] T208425: [EPIC] Kill the wb_terms table

[Maintenance_bot]

Maintenance_bot removed a project: Patch-For-Review.

TASK DETAIL
  https://phabricator.wikimedia.org/T208425

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Maintenance_bot
Cc: Ladsgroup, JAllemandou, ArielGlenn, jcrespo, Joe, Gehel, alaa_wmde, 
Marostegui, Jdforrester-WMF, Addshore, Aklapper, darthmon_wmde, Nandana, Lahi, 
Gq86, GoranSMilovanovic, QZanden, LawExplorer, _jensen, rosalieper, Scott_WUaS, 
Wikidata-bugs, aude, Lydia_Pintscher, Mbch331, Alter-paule, Beast1978, Un1tY, 
Hook696, Daryl-TTMG, RomaAmorRoma, E.S.A-Sheild, Meekrab2012, joker88john, 
CucyNoiD, NebulousIris, Gaboe420, Versusxo, Majesticalreaper22, Giuliamocci, 
Adrian1985, Cpaulf30, Af420, Darkminds3113, Bsandipan, Lordiis, Adik2382, 
Th3d3v1ls, Ramalepe, Liugev6, WSH1906, Lewizho99, Maathavan
___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Updated] T208425: [EPIC] Kill the wb_terms table

[gerritbot]

gerritbot added a project: Patch-For-Review.

TASK DETAIL
  https://phabricator.wikimedia.org/T208425

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: gerritbot
Cc: Ladsgroup, JAllemandou, ArielGlenn, jcrespo, Joe, Gehel, alaa_wmde, 
Marostegui, Jdforrester-WMF, Addshore, Aklapper, Alter-paule, Beast1978, Un1tY, 
Hook696, Daryl-TTMG, RomaAmorRoma, E.S.A-Sheild, darthmon_wmde, Meekrab2012, 
joker88john, CucyNoiD, Nandana, NebulousIris, Gaboe420, Versusxo, 
Majesticalreaper22, Giuliamocci, Adrian1985, Cpaulf30, Lahi, Gq86, Af420, 
Darkminds3113, Bsandipan, Lordiis, GoranSMilovanovic, Adik2382, Th3d3v1ls, 
Ramalepe, Liugev6, QZanden, LawExplorer, WSH1906, Lewizho99, Maathavan, 
_jensen, rosalieper, Scott_WUaS, Wikidata-bugs, aude, Lydia_Pintscher, Mbch331
___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Updated] T208425: [EPIC] Kill the wb_terms table

[Addshore]

Addshore added a subtask: T245592: EntityTermLookupBase imcrements wb_terms 
related metrics, but is now also used by the new term storage and should use 
different metric names..

TASK DETAIL
  https://phabricator.wikimedia.org/T208425

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Addshore
Cc: Ladsgroup, JAllemandou, ArielGlenn, jcrespo, Joe, Gehel, alaa_wmde, 
Marostegui, Jdforrester-WMF, Addshore, Aklapper, darthmon_wmde, Nandana, Lahi, 
Gq86, GoranSMilovanovic, QZanden, LawExplorer, _jensen, rosalieper, Scott_WUaS, 
Wikidata-bugs, aude, Lydia_Pintscher, Mbch331
___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Updated] T208425: [EPIC] Kill the wb_terms table

[Ladsgroup]

Ladsgroup added subscribers: JAllemandou, Ladsgroup.
Ladsgroup added a comment.


  In T208425#5836066 , 
@jcrespo wrote:
  
  > Thanks a lot for the work on this- may I suggest a step before the next 
step (after rebuilding) of "checking all data, old and new, is consistent". 
This is a lot of data, and even on well thought processes missing rows were 
discovered after I requested a comparison on other well-though migration, which 
happened due to mistakes/existing inconsistencies/aborts. May I request such a 
step, which could be as fast as a simple join query (<5m to run) between old an 
new to check no rows are missing or extra, and have equivalent data?
  
  We already do this with sqoop and analytics team (thanks to @JAllemandou). 
here's an example 
.
 That's how we discovered things like T243944: Really large holes in the new 
term store (again) 

TASK DETAIL
  https://phabricator.wikimedia.org/T208425

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Ladsgroup
Cc: Ladsgroup, JAllemandou, ArielGlenn, jcrespo, Joe, Gehel, alaa_wmde, 
Marostegui, Jdforrester-WMF, Addshore, Aklapper, darthmon_wmde, Nandana, Lahi, 
Gq86, GoranSMilovanovic, QZanden, LawExplorer, _jensen, rosalieper, Scott_WUaS, 
Wikidata-bugs, aude, Lydia_Pintscher, Mbch331
___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Updated] T208425: [EPIC] Kill the wb_terms table

[Addshore]

Addshore removed a subtask: T219144: Register new terms store schema schanges 
with MW update mechanism.

TASK DETAIL
  https://phabricator.wikimedia.org/T208425

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Addshore
Cc: ArielGlenn, jcrespo, Joe, Gehel, alaa_wmde, Marostegui, Jdforrester-WMF, 
Addshore, Aklapper, darthmon_wmde, Nandana, Lahi, Gq86, GoranSMilovanovic, 
QZanden, LawExplorer, _jensen, rosalieper, Scott_WUaS, Wikidata-bugs, aude, 
Lydia_Pintscher, Mbch331
___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Updated] T208425: [EPIC] Kill the wb_terms table

[Addshore]

Addshore removed a subtask: T236979: Fix remaining usecases still reading from 
old wb_terms schema.

TASK DETAIL
  https://phabricator.wikimedia.org/T208425

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Addshore
Cc: ArielGlenn, jcrespo, Joe, Gehel, alaa_wmde, Marostegui, Jdforrester-WMF, 
Addshore, Aklapper, darthmon_wmde, Nandana, Lahi, Gq86, GoranSMilovanovic, 
QZanden, LawExplorer, _jensen, rosalieper, Scott_WUaS, Wikidata-bugs, aude, 
Lydia_Pintscher, Mbch331
___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Updated] T208425: [EPIC] Kill the wb_terms table

[Addshore]

Addshore added a subtask: T219303: [Checkpoint 5] Update Read Logic.

TASK DETAIL
  https://phabricator.wikimedia.org/T208425

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Addshore
Cc: ArielGlenn, jcrespo, Joe, Gehel, alaa_wmde, Marostegui, Jdforrester-WMF, 
Addshore, Aklapper, darthmon_wmde, Nandana, Lahi, Gq86, GoranSMilovanovic, 
QZanden, LawExplorer, _jensen, rosalieper, Scott_WUaS, Wikidata-bugs, aude, 
Lydia_Pintscher, Mbch331
___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Updated] T208425: [EPIC] Kill the wb_terms table

[Addshore]

Addshore added a subtask: T242723: Use new terms store by default on Wikibase 
installs.

TASK DETAIL
  https://phabricator.wikimedia.org/T208425

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Addshore
Cc: ArielGlenn, jcrespo, Joe, Gehel, alaa_wmde, Marostegui, Jdforrester-WMF, 
Addshore, Aklapper, darthmon_wmde, Nandana, Lahi, Gq86, GoranSMilovanovic, 
QZanden, LawExplorer, _jensen, rosalieper, Scott_WUaS, Wikidata-bugs, aude, 
Lydia_Pintscher, Mbch331
___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Updated] T208425: [EPIC] Kill the wb_terms table

[Addshore]

Addshore added a subtask: T242644: New term store doesn't provide case 
insensitive search (for when elastic is not used).

TASK DETAIL
  https://phabricator.wikimedia.org/T208425

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Addshore
Cc: ArielGlenn, jcrespo, Joe, Gehel, alaa_wmde, Marostegui, Jdforrester-WMF, 
Addshore, Aklapper, darthmon_wmde, Nandana, Lahi, Gq86, GoranSMilovanovic, 
QZanden, LawExplorer, _jensen, rosalieper, Scott_WUaS, Wikidata-bugs, aude, 
Lydia_Pintscher, Mbch331
___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Updated] T208425: [EPIC] Kill the wb_terms table

[Addshore]

Addshore added a subtask: T241212: Investigate orphan rows in the new terms 
tables after T237984.

TASK DETAIL
  https://phabricator.wikimedia.org/T208425

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Addshore
Cc: ArielGlenn, jcrespo, Joe, Gehel, alaa_wmde, Marostegui, Jdforrester-WMF, 
Addshore, Aklapper, darthmon_wmde, Nandana, Lahi, Gq86, GoranSMilovanovic, 
QZanden, LawExplorer, _jensen, rosalieper, Scott_WUaS, Wikidata-bugs, aude, 
Lydia_Pintscher, Mbch331
___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Updated] T208425: [EPIC] Kill the wb_terms table

[Addshore]

Addshore added a subtask: T241211: Identify holes in the items terms tables for 
prod as of 17:00 on the 19th December for rebuilding.

TASK DETAIL
  https://phabricator.wikimedia.org/T208425

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Addshore
Cc: ArielGlenn, jcrespo, Joe, Gehel, alaa_wmde, Marostegui, Jdforrester-WMF, 
Addshore, Aklapper, darthmon_wmde, Nandana, Lahi, Gq86, GoranSMilovanovic, 
QZanden, LawExplorer, _jensen, rosalieper, Scott_WUaS, Wikidata-bugs, aude, 
Lydia_Pintscher, Mbch331
___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Updated] T208425: [EPIC] Kill the wb_terms table

[Addshore]

Addshore added subtasks: T241209: Full rebuild of new wikidata terms tables on 
test after T237984, T241210: Full rebuild of new wikidata terms tables on beta 
after T237984.

TASK DETAIL
  https://phabricator.wikimedia.org/T208425

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Addshore
Cc: ArielGlenn, jcrespo, Joe, Gehel, alaa_wmde, Marostegui, Jdforrester-WMF, 
Addshore, Aklapper, darthmon_wmde, Nandana, Lahi, Gq86, GoranSMilovanovic, 
QZanden, LawExplorer, _jensen, rosalieper, Scott_WUaS, Wikidata-bugs, aude, 
Lydia_Pintscher, Mbch331
___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Updated] T208425: [EPIC] Kill the wb_terms table

[Addshore]

Addshore added a subtask: T219302: wb_terms trailblazing wrapup - Prepare 
report, announcements and next step.

TASK DETAIL
  https://phabricator.wikimedia.org/T208425

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Addshore
Cc: ArielGlenn, jcrespo, Joe, Gehel, alaa_wmde, Marostegui, Jdforrester-WMF, 
Addshore, Aklapper, darthmon_wmde, DannyS712, Nandana, Lahi, Gq86, 
GoranSMilovanovic, QZanden, LawExplorer, _jensen, rosalieper, Scott_WUaS, 
Wikidata-bugs, aude, Lydia_Pintscher, Mbch331
___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Updated] T208425: [EPIC] Kill the wb_terms table

[alaa_wmde]

alaa_wmde added a subtask: T237723: PHP Warning: [data-update-failed]: A data 
update callback triggered an exception (Fail-safe exception. Number of ids to 
be restored is not equal to the number of records that are about to be inserted 
into master. This should never happen, except for an edge-case that was not 
detected during development or due to a race-condition that is not covered by 
this implementation.) [Called from 
Wikibase\Repo\Content\DataUpdateAdapter::doUpdate in 
/srv/mediawiki/php-1.35.0-wmf.5/extensions/Wikibase/repo/includes/Content/DataUpdateAdapter.php
 at line 62].

TASK DETAIL
  https://phabricator.wikimedia.org/T208425

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: alaa_wmde
Cc: ArielGlenn, jcrespo, Joe, Gehel, alaa_wmde, Marostegui, Jdforrester-WMF, 
Addshore, Aklapper, darthmon_wmde, DannyS712, Nandana, Lahi, Gq86, 
GoranSMilovanovic, QZanden, LawExplorer, _jensen, rosalieper, Scott_WUaS, 
Wikidata-bugs, aude, Lydia_Pintscher, Mbch331
___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Updated] T208425: [EPIC] Kill the wb_terms table

[Addshore]

Addshore added a subtask: T235117: Investigate possible optimizations writes to 
new term store.

TASK DETAIL
  https://phabricator.wikimedia.org/T208425

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Addshore
Cc: ArielGlenn, jcrespo, Joe, Gehel, alaa_wmde, Marostegui, Jdforrester-WMF, 
Addshore, Aklapper, darthmon_wmde, DannyS712, Nandana, Lahi, Gq86, 
GoranSMilovanovic, QZanden, LawExplorer, _jensen, rosalieper, Wikidata-bugs, 
aude, Lydia_Pintscher, Mbch331
___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Updated] T208425: [EPIC] Kill the wb_terms table

[Addshore]

Addshore added a subtask: T229407: Spikes in DB traffic and rows/s reads when 
reading from new terms store.

TASK DETAIL
  https://phabricator.wikimedia.org/T208425

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Addshore
Cc: ArielGlenn, jcrespo, Joe, Gehel, alaa_wmde, Marostegui, Jdforrester-WMF, 
Addshore, Aklapper, darthmon_wmde, DannyS712, Nandana, Lahi, Gq86, 
GoranSMilovanovic, QZanden, LawExplorer, _jensen, rosalieper, Wikidata-bugs, 
aude, Lydia_Pintscher, Mbch331
___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Updated] T208425: [EPIC] Kill the wb_terms table

[Addshore]

Addshore added a subtask: T219144: Register new terms store schema schanges 
with MW update mechanism.

TASK DETAIL
  https://phabricator.wikimedia.org/T208425

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Addshore
Cc: ArielGlenn, jcrespo, Joe, Gehel, alaa_wmde, Marostegui, Jdforrester-WMF, 
Addshore, Aklapper, darthmon_wmde, DannyS712, Nandana, Lahi, Gq86, 
GoranSMilovanovic, QZanden, LawExplorer, _jensen, rosalieper, Wikidata-bugs, 
aude, Lydia_Pintscher, Mbch331
___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Updated] T208425: [EPIC] Kill the wb_terms table

[Addshore]

Addshore added a subtask: T225345: Test read logic of new terms store on test 
node.

TASK DETAIL
  https://phabricator.wikimedia.org/T208425

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Addshore
Cc: ArielGlenn, jcrespo, Joe, Gehel, alaa_wmde, Marostegui, Jdforrester-WMF, 
Addshore, Aklapper, darthmon_wmde, DannyS712, Nandana, Lahi, Gq86, 
GoranSMilovanovic, QZanden, LawExplorer, _jensen, rosalieper, Wikidata-bugs, 
aude, Lydia_Pintscher, Mbch331
___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Updated] T208425: [EPIC] Kill the wb_terms table

[Addshore]

Addshore added a subtask: T236979: Fix remaining usecases for new termstore 
schema reading.

TASK DETAIL
  https://phabricator.wikimedia.org/T208425

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Addshore
Cc: ArielGlenn, jcrespo, Joe, Gehel, alaa_wmde, Marostegui, Jdforrester-WMF, 
Addshore, Aklapper, darthmon_wmde, DannyS712, Nandana, Lahi, Gq86, 
GoranSMilovanovic, QZanden, LawExplorer, _jensen, rosalieper, Wikidata-bugs, 
aude, Lydia_Pintscher, Mbch331
___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Updated] T208425: [EPIC] Kill the wb_terms table

[jcrespo]

jcrespo added subscribers: Gehel, Joe, jcrespo.
jcrespo added a comment.


  I see you are going to ask DBAs at T219145 
, that is great.
  
  As a heads up, because I saw there is a chance of starting using other data 
stores (which by itself is not an issue), instead of or in addition to MySQL to 
give a heads up to service operations SREs and probably search too- in the past 
there has been misunderstandings with sending important/data that has to be 
persisted to datastores that ops cannot guarantee persistance/are not properly 
replicated between dcs (e.g. redis). Please do not go beyond the research phase 
without taking SREs with your proposal to avoid future misunderstandings about 
storage requirements. CC @Joe @Gehel

TASK DETAIL
  https://phabricator.wikimedia.org/T208425

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: jcrespo
Cc: jcrespo, Joe, Gehel, alaa_wmde, Marostegui, Jdforrester-WMF, Addshore, 
Aklapper, Nandana, Lahi, Gq86, GoranSMilovanovic, QZanden, LawExplorer, 
_jensen, rosalieper, Wikidata-bugs, aude, Lydia_Pintscher, Mbch331
___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Updated] T208425: [EPIC] Kill the wb_terms table

[Addshore]

Addshore removed a subtask: T215902: Investigate normalization of data stored 
in wb_terms table.

TASK DETAIL
  https://phabricator.wikimedia.org/T208425

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Addshore
Cc: alaa_wmde, Marostegui, Jdforrester-WMF, Addshore, Aklapper, Nandana, Lahi, 
Gq86, GoranSMilovanovic, QZanden, LawExplorer, _jensen, rosalieper, 
Wikidata-bugs, aude, Lydia_Pintscher, Mbch331
___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Updated] T208425: [EPIC] Kill the wb_terms table

[Addshore]

Addshore removed a subtask: T217821: Investigate duplication of strings in 
wb_terms table for wikidatawiki.

TASK DETAIL
  https://phabricator.wikimedia.org/T208425

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Addshore
Cc: alaa_wmde, Marostegui, Jdforrester-WMF, Addshore, Aklapper, Nandana, Lahi, 
Gq86, GoranSMilovanovic, QZanden, LawExplorer, _jensen, rosalieper, 
Wikidata-bugs, aude, Lydia_Pintscher, Mbch331
___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Updated] T208425: [EPIC] Kill the wb_terms table

[Addshore]

Addshore added a subtask: T219175: [Mega] - Migrate data from wb_terms to new 
schema.

TASK DETAIL
  https://phabricator.wikimedia.org/T208425

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Addshore
Cc: alaa_wmde, Marostegui, Jdforrester-WMF, Addshore, Aklapper, Nandana, Lahi, 
Gq86, GoranSMilovanovic, QZanden, LawExplorer, _jensen, rosalieper, 
Wikidata-bugs, aude, Lydia_Pintscher, Mbch331
___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Updated] T208425: [EPIC] Kill the wb_terms table

[Addshore]

Addshore added a subtask: T217821: Investigate duplication of strings in 
wb_terms table for wikidatawiki.

TASK DETAIL
  https://phabricator.wikimedia.org/T208425

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Addshore
Cc: Jdforrester-WMF, Addshore, Aklapper, alaa_wmde, Nandana, Lahi, Gq86, 
GoranSMilovanovic, QZanden, LawExplorer, _jensen, rosalieper, Wikidata-bugs, 
aude, Lydia_Pintscher, Mbch331
___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs