[Wikidata-bugs] [Maniphest] [Created] T252068: WQDS Data Reload
RKemper created this task. RKemper added a project: Wikidata-Query-Service. Restricted Application added a subscriber: Aklapper. Restricted Application added a project: Wikidata. TASK DESCRIPTION We have already done the reload on `wdqs1010`, so now we need to xfer to the various other relevant servers TASK DETAIL https://phabricator.wikimedia.org/T252068 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: RKemper Cc: RKemper, Aklapper, CBogen, darthmon_wmde, Nandana, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T252068: WQDS Data Reload
RKemper added a comment. `wdqs1010` is one of our test servers Therefore can screw up `wdqs1010` as much as we want, but not the others --- we need to transfer that data to each server which implies we need to open a port for data transfer we have 2 primary DC: eqiad and codfw due to the magic of geo DNS, the servers in codfw see much less traffic than eqiad in each DC, we have 2 wdqs clusters, one public, one internal one place to find which server is where: https://github.com/wikimedia/puppet/blob/production/manifests/site.pp#L2207-L2242 https://github.com/wikimedia/puppet/blob/production/manifests/site.pp#L2207-L2242 to find where each server is TASK DETAIL https://phabricator.wikimedia.org/T252068 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: RKemper Cc: RKemper, Aklapper, CBogen, darthmon_wmde, Legado_Shulgin, Nandana, Davinaclare77, Qtn1293, Techguru.pc, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, Th3d3v1ls, Hfbn0, QZanden, EBjune, merbst, LawExplorer, Zppix, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, Wong128hk, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, faidon, Mbch331, Rxy, Jay8g, fgiunchedi ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T252068: WQDS Data Reload
RKemper added a comment. should also circle back and fix https://github.com/wikimedia/puppet/blob/production/manifests/site.pp#L2207-L2242,` wdqs200[78]` are declared twice TASK DETAIL https://phabricator.wikimedia.org/T252068 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: RKemper Cc: RKemper, Aklapper, CBogen, darthmon_wmde, Legado_Shulgin, Nandana, Davinaclare77, Qtn1293, Techguru.pc, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, Th3d3v1ls, Hfbn0, QZanden, EBjune, merbst, LawExplorer, Zppix, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, Wong128hk, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, faidon, Mbch331, Rxy, Jay8g, fgiunchedi ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Unblock] T230588: Wikidata Query Service is swapping items and properties
RKemper closed subtask T252068: WQDS Data Reload as Resolved. TASK DETAIL https://phabricator.wikimedia.org/T230588 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Zbyszko, RKemper Cc: Ghuron, Nikki, Toni_001, Dipsacus_fullonum, Larske, Mathew.onipe, Esc3300, agray, Jakub.klimek, Gehel, bennofs, Magnus, Robby, Mmarx, jeremyb, Lea_Lacroix_WMDE, Manu1400, Lucas_Werkmeister_WMDE, Smalyshev, Moebeus, Aklapper, Fnielsen, CBogen, darthmon_wmde, Nandana, Lahi, Gq86, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Unblock] T245135: Too many values for wikibase:geoLatitude and wikibase:geoLongitude in WDQS
RKemper closed subtask T252068: WQDS Data Reload as Resolved. TASK DETAIL https://phabricator.wikimedia.org/T245135 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: RKemper Cc: Lucas_Werkmeister_WMDE, Aklapper, Dipsacus_fullonum, CBogen, darthmon_wmde, Nandana, Lahi, Gq86, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Unblock] T240831: Sometimes Q77104211 shows up in sparql query as having a wrong value for P2397, however item is deleted since December 3rd 2019
RKemper closed subtask T252068: WQDS Data Reload as Resolved. TASK DETAIL https://phabricator.wikimedia.org/T240831 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: RKemper Cc: Gehel, Aklapper, Mbch331, CBogen, darthmon_wmde, Nandana, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Closed] T252068: WQDS Data Reload
RKemper closed this task as "Resolved". RKemper added a comment. Still need to circle back to dependent tickets and verify that those problems are solved TASK DETAIL https://phabricator.wikimedia.org/T252068 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: RKemper Cc: Gehel, RKemper, Aklapper, CBogen, darthmon_wmde, Legado_Shulgin, Nandana, Davinaclare77, Qtn1293, Techguru.pc, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, Th3d3v1ls, Hfbn0, QZanden, EBjune, merbst, LawExplorer, Zppix, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, Wong128hk, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, faidon, Mbch331, Rxy, Jay8g, fgiunchedi ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T252068: WQDS Data Reload
RKemper added a comment. Data transfer is done across all instances as of last friday, along with a wdqs-categories reload that we tacked on. Circling back to mark this ticket as done. TASK DETAIL https://phabricator.wikimedia.org/T252068 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: RKemper Cc: Gehel, RKemper, Aklapper, CBogen, darthmon_wmde, Legado_Shulgin, Nandana, Davinaclare77, Qtn1293, Techguru.pc, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, Th3d3v1ls, Hfbn0, QZanden, EBjune, merbst, LawExplorer, Zppix, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, Wong128hk, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, faidon, Mbch331, Rxy, Jay8g, fgiunchedi ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T261204: Wikidata lexeme ttl dumps should be in a "predictable" folder
RKemper added a comment. dumpsgen 58563 0.0 0.0 4276 700 ?Ss 20:50 0:00 /bin/sh -c python3 /srv/deployment/dumps/dumps/xmldumps-backup/generatemiscdumps.py --configfile /etc/dumps/confs/addschanges.conf --dumptype incrdumps --quiet dumpsgen 58565 0.0 0.0 57976 18128 ?S20:50 0:03 python3 /srv/deployment/dumps/dumps/xmldumps-backup/generatemiscdumps.py --configfile /etc/dumps/confs/addschanges.conf --dumptype incrdumps --quiet dumpsgen 63246 3.9 0.0 456952 64728 ?S21:08 2:51 /usr/bin/php7.2 /srv/mediawiki/multiversion/MWScript.php dumpTextPass.php --wiki=commonswiki --stub=gzip:/mnt/dumpsdata/otherdumps/incr/commonswiki/20200912/commonswiki-20200912-stubs-meta-hist-incr.xml.gz --quiet --spawn=/usr/bin/php7.2 --output=bzip2:/mnt/dumpsdata/otherdumps/incr/commonswiki/20200912/commonswiki-20200912-pages-meta-hist-incr.xml.bz2 dumpsgen 63248 0.0 0.0 4276 740 ?S21:08 0:00 sh -c bzip2 > '/mnt/dumpsdata/otherdumps/incr/commonswiki/20200912/commonswiki-20200912-pages-meta-hist-incr.xml.bz2' dumpsgen 63249 2.9 0.0 13604 8492 ?S21:08 2:08 bzip2 dumpsgen 63250 0.0 0.0 4276 736 ?S21:08 0:00 sh -c '/usr/bin/php7.2' '/srv/mediawiki/php-1.36.0-wmf.8/../multiversion/MWScript.php' 'fetchText.php' '--wiki' 'commonswiki' dumpsgen 63251 18.5 0.0 453812 63484 ?S21:08 13:24 /usr/bin/php7.2 /srv/mediawiki/php-1.36.0-wmf.8/../multiversion/MWScript.php fetchText.php --wiki commonswiki Dumps before merging. Nothing specific to `wikidata` is running so I just merged it now. TASK DETAIL https://phabricator.wikimedia.org/T261204 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: RKemper Cc: RKemper, ArielGlenn, dcausse, Alter-paule, jannee_e, Beast1978, CBogen, Un1tY, Akuckartz, Hook696, darthmon_wmde, Kent7301, joker88john, CucyNoiD, Nandana, Namenlos314, Gaboe420, Giuliamocci, Cpaulf30, Lahi, Gq86, Af420, Bsandipan, Lucas_Werkmeister_WMDE, GoranSMilovanovic, Lunewa, QZanden, EBjune, merbst, LawExplorer, Lewizho99, Maathavan, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, gnosygnu, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T261204: Wikidata lexeme ttl dumps should be in a "predictable" folder
RKemper moved this task from Needs review to Needs Reporting on the Discovery-Search (Current work) board. RKemper added a comment. We'll want to monitor across the next week to verify all looks good. TASK DETAIL https://phabricator.wikimedia.org/T261204 WORKBOARD https://phabricator.wikimedia.org/project/board/1227/ EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: RKemper Cc: RKemper, ArielGlenn, dcausse, Alter-paule, jannee_e, Beast1978, CBogen, Un1tY, Akuckartz, Hook696, darthmon_wmde, Kent7301, joker88john, CucyNoiD, Nandana, Namenlos314, Gaboe420, Giuliamocci, Cpaulf30, Lahi, Gq86, Af420, Bsandipan, Lucas_Werkmeister_WMDE, GoranSMilovanovic, Lunewa, QZanden, EBjune, merbst, LawExplorer, Lewizho99, Maathavan, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, gnosygnu, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T262009: Add an entry in the WDQS Runbook on killer queries
RKemper updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T262009 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: RKemper Cc: Aklapper, RKemper, Gehel, CBogen, Akuckartz, darthmon_wmde, Nandana, Namenlos314, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T265868: wdqs2002 out of disk space
RKemper moved this task from In Progress to Needs Reporting on the Discovery-Search (Current work) board. RKemper added a comment. This should be resolved now; the transfer is complete and the node matches others in the fleet. ryankemper@wdqs2002:~$ ls -lh /srv/wdqs/wikidata.jnl -rw-rw-r-- 1 blazegraph blazegraph 806G Oct 19 22:57 /srv/wdqs/wikidata.jnl vs ryankemper@wdqs2004:~$ ls -lh /srv/wdqs/wikidata.jnl -rw-rw-r-- 1 blazegraph blazegraph 806G Oct 19 22:57 /srv/wdqs/wikidata.jnl TASK DETAIL https://phabricator.wikimedia.org/T265868 WORKBOARD https://phabricator.wikimedia.org/project/board/1227/ EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Gehel, RKemper Cc: RKemper, dcausse, Gehel, Aklapper, CBogen, Akuckartz, darthmon_wmde, Nandana, Namenlos314, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T258835: Update Commons query service favicon
RKemper moved this task from Needs review to Needs Reporting on the Discovery-Search (Current work) board. RKemper added a comment. Applied this today TASK DETAIL https://phabricator.wikimedia.org/T258835 WORKBOARD https://phabricator.wikimedia.org/project/board/1227/ EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse, RKemper Cc: RKemper, Jarekt, CBogen, Gehel, Aklapper, Multichill, Akuckartz, darthmon_wmde, Nandana, Namenlos314, Lahi, Gq86, Ramsey-WMF, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, Poyekhali, _jensen, rosalieper, Taiwania_Justo, Scott_WUaS, Jonas, Xmlizer, Ixocactus, Wong128hk, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, El_Grafo, Dinoguy1000, Manybubbles, Isarra, Steinsplitter, Mbch331, Keegan ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T258739: wdqs admins should have access to nginx logs, jstack on wdqs machines
RKemper added a comment. We discussed this during this week's SRE meeting and resolved to enable full root access for wdqs admins, rather than granularly expanding access one file at a time. This also lines up better with how we currently manage Elasticsearch, where admins have root access. TASK DETAIL https://phabricator.wikimedia.org/T258739 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Joe, RKemper Cc: herron, RKemper, CDanis, dcausse, Aklapper, Dzahn, lmata, Alter-paule, Beast1978, CBogen, Un1tY, Akuckartz, Hook696, darthmon_wmde, Kent7301, joker88john, Legado_Shulgin, CucyNoiD, Nandana, Namenlos314, jijiki, Klaas_Z4us_V, Gaboe420, Giuliamocci, Davinaclare77, Qtn1293, Cpaulf30, Techguru.pc, Lahi, Gq86, Af420, Bsandipan, Lucas_Werkmeister_WMDE, GoranSMilovanovic, Th3d3v1ls, Hfbn0, QZanden, EBjune, merbst, LawExplorer, Lewizho99, Zppix, Maathavan, elukey, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, Wong128hk, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, mark, JanZerebecki, faidon, Mbch331, Rxy, Jay8g, Krenair, fgiunchedi ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T258739: wdqs admins should have access to nginx logs, jstack on wdqs machines
RKemper added a project: Sustainability (Incident Followup). TASK DETAIL https://phabricator.wikimedia.org/T258739 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: RKemper Cc: RKemper, CDanis, dcausse, Aklapper, Dzahn, lmata, Alter-paule, Beast1978, CBogen, Un1tY, Akuckartz, Hook696, darthmon_wmde, Kent7301, joker88john, Legado_Shulgin, CucyNoiD, Nandana, Namenlos314, jijiki, Klaas_Z4us_V, Gaboe420, Giuliamocci, Davinaclare77, Qtn1293, Cpaulf30, Techguru.pc, Lahi, Gq86, Af420, Bsandipan, Lucas_Werkmeister_WMDE, GoranSMilovanovic, Th3d3v1ls, Hfbn0, QZanden, EBjune, merbst, LawExplorer, Lewizho99, Zppix, Maathavan, elukey, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, Wong128hk, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, mark, JanZerebecki, faidon, Mbch331, Rxy, Jay8g, Krenair, fgiunchedi ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T258739: wdqs admins should have access to nginx logs, jstack on wdqs machines
RKemper renamed this task from "wdqs admins should have access to nginx logs on wdqs machines" to "wdqs admins should have access to nginx logs, jstack on wdqs machines". TASK DETAIL https://phabricator.wikimedia.org/T258739 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: RKemper Cc: RKemper, CDanis, dcausse, Aklapper, Dzahn, lmata, Alter-paule, Beast1978, CBogen, Un1tY, Akuckartz, Hook696, darthmon_wmde, Kent7301, joker88john, Legado_Shulgin, CucyNoiD, Nandana, Namenlos314, Gaboe420, Giuliamocci, Davinaclare77, Qtn1293, Cpaulf30, Techguru.pc, Lahi, Gq86, Af420, Bsandipan, Lucas_Werkmeister_WMDE, GoranSMilovanovic, Th3d3v1ls, Hfbn0, QZanden, EBjune, merbst, LawExplorer, Lewizho99, Zppix, Maathavan, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, Wong128hk, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, mark, JanZerebecki, faidon, Mbch331, Rxy, Jay8g, Krenair, fgiunchedi ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T271412: Investigate an alert on statements volume in Blazegraph instances
RKemper updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T271412 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: RKemper Cc: Aklapper, Zbyszko, MPhamWMF, CBogen, Akuckartz, Nandana, Namenlos314, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T269302: Many files on Commons cannot be found in WCQS
RKemper updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T269302 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: RKemper Cc: Lydia_Pintscher, Vojtech.dostal, Dipsacus_fullonum, Aklapper, Muchiri124, CBogen, Nintendofan885, Akuckartz, Nandana, JKSTNK, Namenlos314, Lahi, PDrouin-WMF, Gq86, E1presidente, Ramsey-WMF, Cparle, Anooprao, SandraF_WMF, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, Tramullas, Acer, merbst, LawExplorer, Salgo60, Silverfish, Poyekhali, _jensen, rosalieper, Taiwania_Justo, Scott_WUaS, Jonas, Xmlizer, Susannaanas, Ixocactus, Wong128hk, Jane023, jkroll, Wikidata-bugs, Jdouglas, Base, matthiasmullie, aude, Tobias1984, El_Grafo, Dinoguy1000, Manybubbles, Ricordisamoa, Wesalius, Fabrice_Florin, Raymond, Steinsplitter, Mbch331, Keegan ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T269204: Some wdqs metrics changed when switching to python3
RKemper added a comment. `sudo -i wmf-auto-reimage-host --conftool -p T269204 wdqs2004.codfw.wmnet` is an example of how to reimage hosts (run from cumin) TASK DETAIL https://phabricator.wikimedia.org/T269204 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: RKemper Cc: Gehel, RKemper, dcausse, Aklapper, lmata, CBogen, Akuckartz, Nandana, Namenlos314, Lahi, Gq86, herron, Lucas_Werkmeister_WMDE, GoranSMilovanovic, Chicocvenancio, QZanden, EBjune, merbst, LawExplorer, Volans, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331, fgiunchedi ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T269204: Some wdqs metrics changed when switching to python3
RKemper claimed this task. RKemper updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T269204 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: RKemper Cc: Gehel, RKemper, dcausse, Aklapper, lmata, CBogen, Akuckartz, Nandana, Namenlos314, Lahi, Gq86, herron, Lucas_Werkmeister_WMDE, GoranSMilovanovic, Chicocvenancio, QZanden, EBjune, merbst, LawExplorer, Volans, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331, fgiunchedi ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T259588: Reload categories once 1.36.0-wmf.3 is running on all groups
RKemper added a comment. Note: https://phabricator.wikimedia.org/T269331 was created due to an exception encountered every time we run the data reload cookbook. The exception only occurs at the very end, so it might not indicate an actual problem TASK DETAIL https://phabricator.wikimedia.org/T259588 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: RKemper Cc: RKemper, dcausse, Aklapper, Wilmanbeno, CBogen, Akuckartz, Nandana, Namenlos314, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, jayvdb, Mbch331, jeremyb ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T269331: wdqs.data-reload cookbook fails when switching
RKemper added a comment. Here's something interesting: On a server where I've recently run the `data-reload` for `categories` (`wdqs1006` in this case), the categories journal looks like so: `-rw-rw-r-- 1 blazegraph blazegraph 21G Dec 3 07:19 categories.jnl` Yet on a wdqs instance where I have not yet run the reload (`wdqs2007` for example), it looks like this: `-rw-rw-r-- 1 blazegraph blazegraph 23G Dec 3 07:32 categories.jnl` So the freshly reloaded instances have only a 21G `categories.jnl`, whereas the not-freshly-reloaded instances have a 23G `categories.jnl` - a full 2GB bigger. I'm not sure if that indicates a problem somewhere, but naively I would not expect the journal to be smaller after reloading. TASK DETAIL https://phabricator.wikimedia.org/T269331 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: RKemper Cc: RKemper, dcausse, Aklapper, CBogen, Akuckartz, Nandana, Namenlos314, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T259588: Reload categories once 1.36.0-wmf.3 is running on all groups
RKemper moved this task from In Progress to Needs Reporting on the Discovery-Search (Current work) board. RKemper added a comment. All categories reloads/transfers are complete TASK DETAIL https://phabricator.wikimedia.org/T259588 WORKBOARD https://phabricator.wikimedia.org/project/board/1227/ EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: RKemper Cc: RKemper, dcausse, Aklapper, Wilmanbeno, CBogen, Akuckartz, Nandana, Namenlos314, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, jayvdb, Mbch331, jeremyb ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T269693: mediawiki_job_wikidata-updateQueryServiceLag failing
RKemper added a comment. There's some context in the description of https://phabricator.wikimedia.org/T269204 that mentions that the counter metric `blazegraph_lastupdated` is now `blazegraph_lastupdated_total`, so if the`mediawiki_job_wikidata-updateQueryServiceLag` job has to do with that metric then the re-image is likely the source of the problem TASK DETAIL https://phabricator.wikimedia.org/T269693 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: RKemper Cc: Dzahn, RLazarus, RKemper, Aklapper, Ramtin0071, Devnull, lmata, Muchiri124, CBogen, Akuckartz, Legado_Shulgin, Nandana, Namenlos314, Davinaclare77, Qtn1293, Techguru.pc, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, Th3d3v1ls, Hfbn0, QZanden, EBjune, merbst, LawExplorer, Zppix, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, Wong128hk, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, faidon, Mbch331, Rxy, Jay8g, fgiunchedi ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T269693: mediawiki_job_wikidata-updateQueryServiceLag failing
RKemper added a comment. Job lives here: https://github.com/wikimedia/mediawiki-extensions-Wikidata.org/blob/60c5f96ebf424b792077bb7c6b533a68702e7aea/maintenance/updateQueryServiceLag.php#L70 I have a patch open here: https://gerrit.wikimedia.org/r/c/operations/puppet/+/646888 that addresses a different manifestation of this same issue with `blazegraph_lastupdated`. Currently it looks like we'll want to switch `blazegraph_lastupdated` from a `Counter` to a `Gauge`, but I need to figure out if changing the metric type will break the behavior in `updateQueryServiceLag.php` or elsewhere. TASK DETAIL https://phabricator.wikimedia.org/T269693 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: RKemper Cc: Dzahn, RLazarus, RKemper, Aklapper, Ramtin0071, Devnull, lmata, Muchiri124, CBogen, Akuckartz, Legado_Shulgin, Nandana, Namenlos314, Davinaclare77, Qtn1293, Techguru.pc, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, Th3d3v1ls, Hfbn0, QZanden, EBjune, merbst, LawExplorer, Zppix, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, Wong128hk, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, faidon, Mbch331, Rxy, Jay8g, fgiunchedi ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T259588: Reload categories once 1.36.0-wmf.3 is running on all groups
RKemper moved this task from Ready for Development to In Progress on the Discovery-Search (Current work) board. RKemper added a comment. Will start on this today TASK DETAIL https://phabricator.wikimedia.org/T259588 WORKBOARD https://phabricator.wikimedia.org/project/board/1227/ EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: RKemper Cc: RKemper, dcausse, Aklapper, Wilmanbeno, CBogen, Akuckartz, Nandana, Namenlos314, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, jayvdb, Mbch331, jeremyb ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T244753: Migrate WDQS to Debian Buster
RKemper moved this task from In Progress to Needs Reporting on the Discovery-Search (Current work) board. RKemper added a comment. This is all done. No issues currently. TASK DETAIL https://phabricator.wikimedia.org/T244753 WORKBOARD https://phabricator.wikimedia.org/project/board/1227/ EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: RKemper Cc: Aklapper, MoritzMuehlenhoff, Gehel, Ramtin0071, MPhamWMF, Devnull, lmata, Muchiri124, CBogen, Akuckartz, Legado_Shulgin, Nandana, Namenlos314, Davinaclare77, Qtn1293, Techguru.pc, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, Th3d3v1ls, Hfbn0, QZanden, EBjune, merbst, LawExplorer, Zppix, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, Wong128hk, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, faidon, Mbch331, Rxy, Jay8g, fgiunchedi ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T269872: wdqs-categories prometheus exporter failing on wdqs1011
RKemper created this task.
RKemper added projects: Wikidata-Query-Service, Discovery-Search (Current work).
Restricted Application added a subscriber: Aklapper.
Restricted Application added a project: Wikidata.
TASK DESCRIPTION
ryankemper@wdqs1011:~$ sudo systemctl status
prometheus-blazegraph-exporter-wdqs-categories.service
● prometheus-blazegraph-exporter-wdqs-categories.service - Prometheus
Blazegraph Exporter (wdqs-categories)
Loaded: loaded
(/lib/systemd/system/prometheus-blazegraph-exporter-wdqs-categories.service;
enabled; vendor preset: enabled)
Active: failed (Result: exit-code) since Thu 2020-12-10 17:10:22 UTC;
12min ago
Process: 21672 ExecStart=/usr/local/bin/prometheus-blazegraph-exporter
--listen :9194 --port 9990 (code=exited, status=1/FAILURE)
Main PID: 21672 (code=exited, status=1/FAILURE)
Dec 10 17:10:22 wdqs1011 systemd[1]:
prometheus-blazegraph-exporter-wdqs-categories.service: Service
RestartSec=100ms expired, scheduling restart.
Dec 10 17:10:22 wdqs1011 systemd[1]:
prometheus-blazegraph-exporter-wdqs-categories.service: Scheduled restart job,
restart counter is at 5.
Dec 10 17:10:22 wdqs1011 systemd[1]: Stopped Prometheus Blazegraph Exporter
(wdqs-categories).
Dec 10 17:10:22 wdqs1011 systemd[1]:
prometheus-blazegraph-exporter-wdqs-categories.service: Start request repeated
too quickly.
Dec 10 17:10:22 wdqs1011 systemd[1]:
prometheus-blazegraph-exporter-wdqs-categories.service: Failed with result
'exit-code'.
Dec 10 17:10:22 wdqs1011 systemd[1]: Failed to start Prometheus Blazegraph
Exporter (wdqs-categories).
This exporter relies on `localhost:9194`, which is refusing connections
currently. It seems like the actual categories service on the "other side" is
failing:
ryankemper@wdqs1011:~$ sudo systemctl status wdqs-categories.service
● wdqs-categories.service - Query Service - Blazegraph - wdqs-categories
Loaded: loaded (/lib/systemd/system/wdqs-categories.service; static;
vendor preset: enabled)
Active: active (running) since Thu 2020-12-10 17:04:48 UTC; 19min ago
Main PID: 18590 (java)
Tasks: 114 (limit: 1)
Memory: 2.3G
CGroup: /system.slice/wdqs-categories.service
└─18590 java -server -XX:+UseG1GC -Xmx8g
-Xloggc:/var/log/wdqs/wdqs-categories_jvm_gc.%p-%t.log -XX:+PrintGCDetails
-XX:+PrintGCDateStamps -XX:+PrintGCTimeStamps -XX:+PrintAdaptiveSizePolicy -X
Dec 10 17:10:21 wdqs1011 wdqs-categories[18590]:
SELECT ( COUNT( * ) AS ?count ) { ?s ?p ?o }
Dec 10 17:10:21 wdqs1011 wdqs-categories[18590]: }
UNION {
Dec 10 17:10:21 wdqs1011 wdqs-categories[18590]:
SELECT * WHERE { <http://www.wikidata.org> schema:dateModified ?y }
Dec 10 17:10:21 wdqs1011 wdqs-categories[18590]: }
} req.requestURI=/bigdata/namespace/wdq/sparql, req.xForwardedFor=null,
req.queryString=format=json=+prefix+schema%3A+%3Cht
Dec 10 17:10:21 wdqs1011 wdqs-categories[18590]:
com.bigdata.rdf.sail.webapp.DatasetNotFoundException: namespace=wdq
Dec 10 17:10:21 wdqs1011 wdqs-categories[18590]: at
com.bigdata.rdf.sail.BigdataSail$BigdataSailConnection.(BigdataSail.java:2071)
Dec 10 17:10:21 wdqs1011 wdqs-categories[18590]: Wrapped by:
java.lang.RuntimeException:
com.bigdata.rdf.sail.webapp.DatasetNotFoundException: namespace=wdq
Dec 10 17:10:21 wdqs1011 wdqs-categories[18590]: at
com.bigdata.rdf.sail.BigdataSail.getReadOnlyConnection(BigdataSail.java:1506)
Dec 10 17:10:21 wdqs1011 wdqs-categories[18590]: Wrapped by:
java.util.concurrent.ExecutionException: java.lang.RuntimeException:
com.bigdata.rdf.sail.webapp.DatasetNotFoundException: namespace=wdq
Dec 10 17:10:21 wdqs1011 wdqs-categories[18590]: at
java.util.concurrent.FutureTask.report(FutureTask.java:122)
TASK DETAIL
https://phabricator.wikimedia.org/T269872
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: RKemper
Cc: RKemper, Aklapper, CBogen, Akuckartz, Nandana, Namenlos314, Lahi, Gq86,
Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst,
LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll,
Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331
___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T269872: wdqs-categories prometheus exporter failing on select wdqs instances
RKemper renamed this task from "wdqs-categories prometheus exporter failing on wdqs1011" to "wdqs-categories prometheus exporter failing on select wdqs instances". RKemper updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T269872 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: RKemper Cc: RKemper, Aklapper, CBogen, Akuckartz, Nandana, Namenlos314, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T269204: Some wdqs metrics changed when switching to python3
RKemper added a comment. Restarting `prometheus-blazegraph-exporter-wdqs-blazegraph.service` after switching from Counter to Gauge now shows the correct `blazegraph_lastupdated` metric when running `curl localhost:9193` (9193 is the port for wdqs-blazegraph). Still waiting to see if the downstream consumers of this metric are working properly now, but that *should* have fixed it. TASK DETAIL https://phabricator.wikimedia.org/T269204 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: RKemper Cc: Gehel, RKemper, dcausse, Aklapper, lmata, Alter-paule, Beast1978, CBogen, Un1tY, Akuckartz, Hook696, Kent7301, joker88john, CucyNoiD, Nandana, Namenlos314, Gaboe420, Giuliamocci, Cpaulf30, Lahi, Gq86, Af420, Bsandipan, herron, Lucas_Werkmeister_WMDE, GoranSMilovanovic, Chicocvenancio, QZanden, EBjune, merbst, LawExplorer, Lewizho99, Volans, Maathavan, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331, fgiunchedi ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T269872: wdqs-categories prometheus exporter failing on select wdqs instances
RKemper moved this task from In Progress to Needs Reporting on the Discovery-Search (Current work) board. RKemper added a comment. The above two patches (really one patch with a followup patch to fix a typo) seem to have fixed the problem. I'll want to circle back to verify we're getting all the metrics we expect, but at a minimum the prometheus blazegraph exporter for wdqs-categories is working again. TASK DETAIL https://phabricator.wikimedia.org/T269872 WORKBOARD https://phabricator.wikimedia.org/project/board/1227/ EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: RKemper Cc: Gehel, RKemper, Aklapper, Alter-paule, Beast1978, CBogen, Un1tY, Akuckartz, Hook696, Kent7301, joker88john, CucyNoiD, Nandana, Namenlos314, Gaboe420, Giuliamocci, Cpaulf30, Lahi, Gq86, Af420, Bsandipan, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, Lewizho99, Maathavan, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T255399: Prepare wdqs1009 to run the streaming updater
RKemper moved this task from In Progress to Needs Reporting on the Discovery-Search (Current work) board. RKemper added a comment. We need to do another reload but not under this ticket TASK DETAIL https://phabricator.wikimedia.org/T255399 WORKBOARD https://phabricator.wikimedia.org/project/board/1227/ EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: RKemper Cc: Gehel, Ottomata, Aklapper, dcausse, Alter-paule, Beast1978, CBogen, Un1tY, Akuckartz, Hook696, Kent7301, joker88john, CucyNoiD, Nandana, Namenlos314, Gaboe420, Giuliamocci, Cpaulf30, Lahi, Gq86, Af420, Bsandipan, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, Lewizho99, Maathavan, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T271851: Clean up gui from the wdqs deploy repo and puppet
RKemper added a comment. In T271851#6765207 <https://phabricator.wikimedia.org/T271851#6765207>, @Ladsgroup wrote: > @RKemper I'm so sorry it was buggy, the cleanest way forward here is to deploy wcqs using microsites and we can easily drop everything from puppet then. Does that make sense? Makes sense! I'm working on getting the wcqs microsite working and then we can revert my earlier revert and nuke the remaining gui file at the same time. I'll tag you on the patch(es) when ready. TASK DETAIL https://phabricator.wikimedia.org/T271851 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Ladsgroup, RKemper Cc: Addshore, dcausse, RKemper, Gehel, Aklapper, Ladsgroup, MPhamWMF, Hazizibinmahdi, CBogen, Akuckartz, Nandana, Namenlos314, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, Mahir256, QZanden, EBjune, merbst, LawExplorer, Salgo60, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Lydia_Pintscher, Mbch331 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T267175: SPARQL-Query shows entries, which should be filter out; number of entries in result set might change when executed repeatedly (possible caching/indexing problem)
RKemper moved this task from In Progress to To Be Deployed on the Discovery-Search (Current work) board. RKemper added a comment. Deploying https://gerrit.wikimedia.org/r/656833 now TASK DETAIL https://phabricator.wikimedia.org/T267175 WORKBOARD https://phabricator.wikimedia.org/project/board/1227/ EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: RKemper Cc: Skim, Strepon, Multichill, Zbyszko, RKemper, Epidosis, dcausse, Tagishsimon, Lydia_Pintscher, CBogen, Z_thomas, agray, Gehel, Lucas_Werkmeister_WMDE, Aklapper, M2k_dewiki, MPhamWMF, Alter-paule, Beast1978, Un1tY, Akuckartz, Hook696, Kent7301, joker88john, CucyNoiD, Nandana, Namenlos314, Gaboe420, Giuliamocci, Cpaulf30, Lahi, Gq86, Af420, Bsandipan, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, Lewizho99, Maathavan, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T267175: SPARQL-Query shows entries, which should be filter out; number of entries in result set might change when executed repeatedly (possible caching/indexing problem)
RKemper added a comment. (Puppet run successful on wdqs nodes following deploy) TASK DETAIL https://phabricator.wikimedia.org/T267175 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: RKemper Cc: Skim, Strepon, Multichill, Zbyszko, RKemper, Epidosis, dcausse, Tagishsimon, Lydia_Pintscher, CBogen, Z_thomas, agray, Gehel, Lucas_Werkmeister_WMDE, Aklapper, M2k_dewiki, MPhamWMF, Alter-paule, Beast1978, Un1tY, Akuckartz, Hook696, Kent7301, joker88john, CucyNoiD, Nandana, Namenlos314, Gaboe420, Giuliamocci, Cpaulf30, Lahi, Gq86, Af420, Bsandipan, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, Lewizho99, Maathavan, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T267175: SPARQL-Query shows entries, which should be filter out; number of entries in result set might change when executed repeatedly (possible caching/indexing problem)
RKemper added a comment. Post deploy check: `wdqs-updater.service` is now running without the `--import-async` flag as expected: `ExecStart=/bin/bash /srv/deployment/wdqs/wdqs/runUpdate.sh -n wdq -- --kafka kafka-main1001.eqiad.wmnet:9092,kafka-main1002.eqiad.wmnet:9092,kafka-main1003.eqiad.wmnet:9092 --consumer wdqs1003 -b 700 --clusters eqiad,codfw --constraints --oldRevision 3 --entityNamespaces 0,120,146` I think now we just wait and see if the issue in the ticket description pops up again. TASK DETAIL https://phabricator.wikimedia.org/T267175 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: RKemper Cc: Skim, Strepon, Multichill, Zbyszko, RKemper, Epidosis, dcausse, Tagishsimon, Lydia_Pintscher, CBogen, Z_thomas, agray, Gehel, Lucas_Werkmeister_WMDE, Aklapper, M2k_dewiki, MPhamWMF, Alter-paule, Beast1978, Un1tY, Akuckartz, Hook696, Kent7301, joker88john, CucyNoiD, Nandana, Namenlos314, Gaboe420, Giuliamocci, Cpaulf30, Lahi, Gq86, Af420, Bsandipan, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, Lewizho99, Maathavan, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T270476: Linked Data Fragments endpoint returns IllegalStateException
RKemper added a comment. I responded to Maciej (the one who reported the problem originally) on the mailing list. Looks like we accidentally gave them the wrong ticket number earlier: https://lists.wikimedia.org/pipermail/wikidata/2021-January/014451.html Looks like this ticket hasn't been triaged/assigned yet TASK DETAIL https://phabricator.wikimedia.org/T270476 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: RKemper Cc: RKemper, Gehel, Aklapper, MPhamWMF, CBogen, Akuckartz, Nandana, Namenlos314, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T270236: wdqs1005: disk space critical on /srv/
RKemper added a project: Wikidata-Query-Service. Restricted Application added a project: Wikidata. TASK DETAIL https://phabricator.wikimedia.org/T270236 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: RKemper Cc: Aklapper, RKemper, MPhamWMF, CBogen, Akuckartz, Nandana, Namenlos314, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T270236: wdqs1005: disk space critical on /srv/
RKemper triaged this task as "High" priority. TASK DETAIL https://phabricator.wikimedia.org/T270236 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: RKemper Cc: Aklapper, RKemper, MPhamWMF, CBogen, Akuckartz, Nandana, Namenlos314, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T267927: Reload wikidata journal from fresh dumps
RKemper claimed this task. TASK DETAIL https://phabricator.wikimedia.org/T267927 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: RKemper Cc: Aklapper, dcausse, MPhamWMF, CBogen, Akuckartz, Nandana, Namenlos314, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T273097: Create Flink Base Image
RKemper added a comment. @akosiaris Is your concern with the idea of using a`flink` base image solution mainly just centered around the inefficiency/inconvenience of needing SRE to merge any flink version upgrades? Since we have an embedded SRE on search (me) and to a lesser extent Guillaume, I think it wouldn't be too much of a problem. In general having our dependencies managed by a docker image will make it easier for us to be explicit about what version we're using, and it seems like the default docker-y way of doing things. Is there a technical reason why a base image might not be a good idea? TASK DETAIL https://phabricator.wikimedia.org/T273097 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: RKemper Cc: RKemper, dcausse, Aklapper, Gehel, akosiaris, Mstyles, MPhamWMF, CBogen, Akuckartz, Nandana, Namenlos314, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T273636: Blazegraph journal for wcqs is too big
RKemper added a comment. Notified WikiData mailing list and also posted here: https://commons.wikimedia.org/wiki/Commons:SPARQL_query_service#Updates TASK DETAIL https://phabricator.wikimedia.org/T273636 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: RKemper Cc: RKemper, Aklapper, dcausse, MPhamWMF, CBogen, Akuckartz, Nandana, Namenlos314, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T267927: Reload wikidata journal from fresh dumps
RKemper added a comment. We'll want to reload these this Friday, because the latest dumps should be available thursday evening. TASK DETAIL https://phabricator.wikimedia.org/T267927 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: RKemper Cc: Aklapper, dcausse, MPhamWMF, CBogen, Akuckartz, Nandana, Namenlos314, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T273636: Blazegraph journal for wcqs is too big
RKemper added a comment. `wcqs-beta-01.eqiad.wmflabs` is running low on disk space due to its blazegraph journal dataset size. In order to free up space we will need to take the service down, delete the journal and re-import from the latest dump. Service interruption will begin at Feb 4 18:30 UTC and continue until the data reload is complete. TASK DETAIL https://phabricator.wikimedia.org/T273636 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: RKemper Cc: RKemper, Aklapper, dcausse, MPhamWMF, CBogen, Akuckartz, Nandana, Namenlos314, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T284445: Blazegraph journal too large on wdqs1012
RKemper claimed this task. TASK DETAIL https://phabricator.wikimedia.org/T284445 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: RKemper Cc: RKemper, Gehel, Aklapper, Invadibot, MPhamWMF, maantietaja, CBogen, Akuckartz, Nandana, Namenlos314, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T284445: Blazegraph journal too large on wdqs1012
RKemper updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T284445 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: RKemper Cc: RKemper, Gehel, Aklapper, Invadibot, MPhamWMF, maantietaja, CBogen, Akuckartz, Nandana, Namenlos314, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T284445: Blazegraph journal too large on wdqs1012
RKemper moved this task from Ready for Development to Needs Reporting on the Discovery-Search (Current work) board. RKemper added a comment. This is done. New `wikidata.jnl` is 975G as expected: `-rw-rw-r-- 1 blazegraph blazegraph 975G Jun 8 19:01 wikidata.jnl` TASK DETAIL https://phabricator.wikimedia.org/T284445 WORKBOARD https://phabricator.wikimedia.org/project/board/1227/ EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: RKemper Cc: RKemper, Gehel, Aklapper, Invadibot, MPhamWMF, maantietaja, CBogen, Akuckartz, Nandana, Namenlos314, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T281504: Degraded RAID on wdqs2007
RKemper closed this task as "Resolved". TASK DETAIL https://phabricator.wikimedia.org/T281504 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: RKemper Cc: RKemper, Aklapper, ops-monitoring-bot, Ramtin0071, MPhamWMF, Devnull, lmata, Muchiri124, CBogen, RhinosF1, Legado_Shulgin, ReaperDawn, Namenlos314, Davinaclare77, Qtn1293, Techguru.pc, Gq86, Lucas_Werkmeister_WMDE, Th3d3v1ls, Hfbn0, EBjune, merbst, Zppix, Jonas, Xmlizer, Wong128hk, Cmjohnson, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Southparkfan, faidon, Jay8g, fgiunchedi ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T281504: Degraded RAID on wdqs2007
RKemper added a comment. Re-image of `wdqs2007` was completed successfully. TASK DETAIL https://phabricator.wikimedia.org/T281504 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: RKemper Cc: RKemper, Aklapper, ops-monitoring-bot, Ramtin0071, MPhamWMF, Devnull, lmata, Muchiri124, CBogen, RhinosF1, Legado_Shulgin, ReaperDawn, Namenlos314, Davinaclare77, Qtn1293, Techguru.pc, Gq86, Lucas_Werkmeister_WMDE, Th3d3v1ls, Hfbn0, EBjune, merbst, Zppix, Jonas, Xmlizer, Wong128hk, Cmjohnson, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Southparkfan, faidon, Jay8g, fgiunchedi ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T281504: Degraded RAID on wdqs2007
RKemper claimed this task. TASK DETAIL https://phabricator.wikimedia.org/T281504 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: RKemper Cc: RKemper, Aklapper, ops-monitoring-bot, Ramtin0071, MPhamWMF, Devnull, lmata, Muchiri124, CBogen, RhinosF1, Legado_Shulgin, ReaperDawn, Namenlos314, Davinaclare77, Qtn1293, Techguru.pc, Gq86, Lucas_Werkmeister_WMDE, Th3d3v1ls, Hfbn0, EBjune, merbst, Zppix, Jonas, Xmlizer, Wong128hk, Cmjohnson, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Southparkfan, faidon, Jay8g, fgiunchedi ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T273636: Blazegraph journal for wcqs is too big
RKemper claimed this task. TASK DETAIL https://phabricator.wikimedia.org/T273636 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: RKemper Cc: MPhamWMF, RKemper, Aklapper, dcausse, CBogen, Akuckartz, Nandana, Namenlos314, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T273636: Blazegraph journal for wcqs is too big
RKemper moved this task from In Progress to Needs Reporting on the Discovery-Search (Current work) board. RKemper added a comment. WCQS is back in service; updating the notification channels right now and will comment back here after TASK DETAIL https://phabricator.wikimedia.org/T273636 WORKBOARD https://phabricator.wikimedia.org/project/board/1227/ EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: RKemper Cc: MPhamWMF, RKemper, Aklapper, dcausse, CBogen, Akuckartz, Nandana, Namenlos314, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T273636: Blazegraph journal for wcqs is too big
RKemper added a comment. Updated: [X] https://commons.wikimedia.org/wiki/Commons:SPARQL_query_service#Updates [X] https://commons.wikimedia.org/wiki/Commons:Village_pump#Unscheduled_maintenance%3A_Wikimedia_Commons_Query_Service [X] WikiData ML mailing list (note: A community member pointed out that sending an e-mail to a Commons mailing list would have made more sense, which is a great point. We'll do that in the future) TASK DETAIL https://phabricator.wikimedia.org/T273636 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: RKemper Cc: MPhamWMF, RKemper, Aklapper, dcausse, CBogen, Akuckartz, Nandana, Namenlos314, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T266495: Create Debian Package for Flink
RKemper added a comment. (See https://phabricator.wikimedia.org/T273097#6805355 for why this ticket has been closed) TASK DETAIL https://phabricator.wikimedia.org/T266495 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: RKemper Cc: RKemper, dcausse, elukey, EBernhardson, Aklapper, Gehel, Mstyles, Ottomata, MPhamWMF, CBogen, Akuckartz, Nandana, Namenlos314, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T267927: Reload wikidata journal from fresh dumps
RKemper added a comment. Still waiting for the latest dumps to be downloaded (few more hours), then need to reboot WDQS hosts as part of https://phabricator.wikimedia.org/T274213, then can do the actual data-reload TASK DETAIL https://phabricator.wikimedia.org/T267927 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: RKemper Cc: Aklapper, dcausse, MPhamWMF, CBogen, Akuckartz, Nandana, Namenlos314, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T266470: Expose wdqs1009 to wdqs users and gather feedback
RKemper triaged this task as "Medium" priority. RKemper claimed this task. TASK DETAIL https://phabricator.wikimedia.org/T266470 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: RKemper Cc: Gehel, Lea_Lacroix_WMDE, dcausse, Aklapper, MPhamWMF, CBogen, Akuckartz, Nandana, Namenlos314, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T273097: Create Flink Base Image
RKemper updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T273097 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: RKemper Cc: dcausse, Aklapper, Gehel, akosiaris, Mstyles, MPhamWMF, CBogen, Akuckartz, Nandana, Namenlos314, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T267927: Reload wikidata journal from fresh dumps
RKemper added a comment. `wdqs1009`, `wdqs1010`, and `wdqs2008` are done, so we need to `data-transfer` to the remaining instances. It looks like `wdqs1010` is warning (not critical) for disk space on `/srv`. Glancing at the box it's still got 44G left but is at 96% utilization: `/dev/mapper/vg0-srv 1.1T 999G 44G 96% /srv` This is essentially entirely attributable to `wikidata.jnl` which is **1 TB** in size. TASK DETAIL https://phabricator.wikimedia.org/T267927 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: RKemper Cc: ayounsi, jbond, Zbyszko, Gehel, Aklapper, dcausse, Invadibot, MPhamWMF, maantietaja, Alter-paule, Beast1978, CBogen, Un1tY, Akuckartz, Hook696, Kent7301, joker88john, CucyNoiD, Nandana, Namenlos314, Gaboe420, Giuliamocci, Cpaulf30, Lahi, Gq86, Af420, Bsandipan, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, Lewizho99, Maathavan, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T266470: Expose wdqs1009 to wdqs users and gather feedback
RKemper added a comment. I missed a step yesterday: I'd updated `/srv/private` as well as the public `labs/private` repo but missed the step for updating `operations/puppet` with the new pubkey: https://gerrit.wikimedia.org/r/c/operations/puppet/+/671267 TASK DETAIL https://phabricator.wikimedia.org/T266470 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: RKemper Cc: Gehel, Lea_Lacroix_WMDE, dcausse, Aklapper, MPhamWMF, maantietaja, Alter-paule, Beast1978, CBogen, Un1tY, Akuckartz, Hook696, Kent7301, joker88john, CucyNoiD, Nandana, Namenlos314, Gaboe420, Giuliamocci, Cpaulf30, Lahi, Gq86, Af420, Bsandipan, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, Lewizho99, Maathavan, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, abian, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T266470: Expose wdqs1009 to wdqs users and gather feedback
RKemper added a comment. The issues with envoy were resolved by running `sudo /usr/local/sbin/build-envoy-config -c /etc/envoy` to properly build `/etc/envoy/envoy.yaml`. That should have been done by puppet already, triggered upon a `sudo systemctl restart envoyproxy.service`, but it didn't - perhaps a race condition. See https://gerrit.wikimedia.org/g/operations/puppet/+/b7dacbca9fae42b32bb91fd485a3f2c70ff903b3/modules/envoyproxy/manifests/init.pp#81 and https://gerrit.wikimedia.org/g/operations/puppet/+/b7dacbca9fae42b32bb91fd485a3f2c70ff903b3/modules/envoyproxy/manifests/conf.pp#30 for the puppet code that normally does it automatically. **Envoy is working properly now and wdqs1009 is now reachable from the caching layer** TASK DETAIL https://phabricator.wikimedia.org/T266470 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: RKemper Cc: Gehel, Lea_Lacroix_WMDE, dcausse, Aklapper, MPhamWMF, maantietaja, Alter-paule, Beast1978, CBogen, Un1tY, Akuckartz, Hook696, Kent7301, joker88john, CucyNoiD, Nandana, Namenlos314, Gaboe420, Giuliamocci, Cpaulf30, Lahi, Gq86, Af420, Bsandipan, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, Lewizho99, Maathavan, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, abian, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T266470: Expose wdqs1009 to wdqs users and gather feedback
RKemper added a comment.
**DNS change logs**
ryankemper@authdns1001:~$ sudo authdns-update
Updating authdns1001.wikimedia.org (self)...
Pulling the current revision from
https://gerrit.wikimedia.org/r/operations/dns.git
Reviewing 85d9b49dc2ff0f8e3657f6f2cd91ce3df79bd1cf...
templates/wikidata.org | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git templates/wikidata.org templates/wikidata.org
index 9f1a5ed3..cea393b1 100644
--- templates/wikidata.org
+++ templates/wikidata.org
@@ -32,7 +32,8 @@
; Servers (alphabetic order)
; Service aliases
-query 1D IN CNAME dyna.wikimedia.org.
+query 1D IN CNAME dyna.wikimedia.org.
+query-preview 1D IN CNAME dyna.wikimedia.org.
; Wikis (alphabetic order)
Merge these changes? (yes/no)? yes
Updating 3a84bd1c..85d9b49d
Fast-forward
templates/wikidata.org | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
Deploying via utils/deploy-check.py...
Assembling and testing data in /tmp/dns-check.voy682v4
-- Generating zonefiles from zone templates
-- Processed 205 zones into directory /tmp/dns-check.voy682v4/zones
OK: No tabs
Summary of violations:
W001|MISSING_IP_FOR_NAME_AND_PTR: 35
W002|MISSING_PTR_FOR_NAME_AND_IP: 27
W103|MISSING_MGMT_FOR_NAME: 59
W105|TOO_MANY_PUBLIC_NAMES: 3
RESULT: 0 Errors, 124 Warnings, 0 Ignored violations, 0 Ignored lines
-- Copying automatically generated zone files under target tree
-- Copying repo-driven real config files and admin_state
-- Copying puppetized config and GeoIP from /etc/gdnsd
-- Checking for illegal tabs in zonefiles
-- Running zone_validator to check WMF rules
-- Running /usr/sbin/gdnsd checkconf on /tmp/dns-check.voy682v4
-- Preflight checkconf is OK
Deploying from /tmp/dns-check.voy682v4 to system dirs
-- Descending to subdirectory: netbox
-- Done with subdir: netbox
-- Zone changed: wikidata.org
Reloading gdnsd zonefiles
info: Zone data reloaded
OK
---
authdns2001.wikimedia.org,dns[1001-1002,2001-2002,3001-3002,4001-4002,5001-5002].wikimedia.org
(11)
---
OK - authdns updated successfully
OK - authdns-update successful on all nodes!
Validation looks good:
user@computer ~/wmf/gui-deploy [production]% for i in 0 1 2 ; do dig
@ns${i}.wikimedia.org -t any query-preview.wikidata.org ; done
; <<>> DiG 9.10.6 <<>> @ns0.wikimedia.org -t any query-preview.wikidata.org
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 58419
;; flags: qr aa rd; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1
;; WARNING: recursion requested but not available
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 1024
;; QUESTION SECTION:
;query-preview.wikidata.org.IN ANY
;; ANSWER SECTION:
query-preview.wikidata.org. 86400 INCNAME dyna.wikimedia.org.
;; Query time: 82 msec
;; SERVER: 208.80.154.238#53(208.80.154.238)
;; WHEN: Fri Mar 12 17:06:00 PST 2021
;; MSG SIZE rcvd: 84
; <<>> DiG 9.10.6 <<>> @ns1.wikimedia.org -t any query-preview.wikidata.org
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 3886
;; flags: qr aa rd; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1
;; WARNING: recursion requested but not available
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 1024
;; QUESTION SECTION:
;query-preview.wikidata.org.IN ANY
;; ANSWER SECTION:
query-preview.wikidata.org. 86400 INCNAME dyna.wikimedia.org.
;; Query time: 61 msec
;; SERVER: 208.80.153.231#53(208.80.153.231)
;; WHEN: Fri Mar 12 17:06:00 PST 2021
;; MSG SIZE rcvd: 84
; <<>> DiG 9.10.6 <<>> @ns2.wikimedia.org -t any query-preview.wikidata.org
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 53512
;; flags: qr aa rd; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1
;; WARNING: recursion requested but not available
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 1024
;; QUESTION SECTION:
;query-preview.wikidata.org.IN ANY
;; ANSWER SECTION:
query-preview.wikidata.org. 86400 INCNAME dyna.wikimedia.org.
;; Query time: 345 msec
;; SERVER: 91.198.174.239#53(91.198.174.239)
;; WHEN: Fri Mar 12 17:06:01 PST 2021
;; MSG SIZE rcvd: 84
TASK DETAIL
https://phabricator.wikimedia.org/T266470
EMAIL PREFERENCES
https://phabricator.wikimedi
[Wikidata-bugs] [Maniphest] T266470: Expose wdqs1009 to wdqs users and gather feedback
RKemper added a comment.
**DNS change logs**
ryankemper@authdns1001:~$ sudo authdns-update
Updating authdns1001.wikimedia.org (self)...
Pulling the current revision from
https://gerrit.wikimedia.org/r/operations/dns.git
Reviewing 85d9b49dc2ff0f8e3657f6f2cd91ce3df79bd1cf...
templates/wikidata.org | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git templates/wikidata.org templates/wikidata.org
index 9f1a5ed3..cea393b1 100644
--- templates/wikidata.org
+++ templates/wikidata.org
@@ -32,7 +32,8 @@
; Servers (alphabetic order)
; Service aliases
-query 1D IN CNAME dyna.wikimedia.org.
+query 1D IN CNAME dyna.wikimedia.org.
+query-preview 1D IN CNAME dyna.wikimedia.org.
; Wikis (alphabetic order)
Merge these changes? (yes/no)? yes
Updating 3a84bd1c..85d9b49d
Fast-forward
templates/wikidata.org | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
Deploying via utils/deploy-check.py...
Assembling and testing data in /tmp/dns-check.voy682v4
-- Generating zonefiles from zone templates
-- Processed 205 zones into directory /tmp/dns-check.voy682v4/zones
OK: No tabs
Summary of violations:
W001|MISSING_IP_FOR_NAME_AND_PTR: 35
W002|MISSING_PTR_FOR_NAME_AND_IP: 27
W103|MISSING_MGMT_FOR_NAME: 59
W105|TOO_MANY_PUBLIC_NAMES: 3
RESULT: 0 Errors, 124 Warnings, 0 Ignored violations, 0 Ignored lines
-- Copying automatically generated zone files under target tree
-- Copying repo-driven real config files and admin_state
-- Copying puppetized config and GeoIP from /etc/gdnsd
-- Checking for illegal tabs in zonefiles
-- Running zone_validator to check WMF rules
-- Running /usr/sbin/gdnsd checkconf on /tmp/dns-check.voy682v4
-- Preflight checkconf is OK
Deploying from /tmp/dns-check.voy682v4 to system dirs
-- Descending to subdirectory: netbox
-- Done with subdir: netbox
-- Zone changed: wikidata.org
Reloading gdnsd zonefiles
info: Zone data reloaded
OK
---
authdns2001.wikimedia.org,dns[1001-1002,2001-2002,3001-3002,4001-4002,5001-5002].wikimedia.org
(11)
---
OK - authdns updated successfully
OK - authdns-update successful on all nodes!
Validation looks good:
user@computer ~/wmf/gui-deploy [production]% for i in 0 1 2 ; do dig
@ns${i}.wikimedia.org -t any query-preview.wikidata.org ; done
; <<>> DiG 9.10.6 <<>> @ns0.wikimedia.org -t any query-preview.wikidata.org
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 58419
;; flags: qr aa rd; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1
;; WARNING: recursion requested but not available
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 1024
;; QUESTION SECTION:
;query-preview.wikidata.org.IN ANY
;; ANSWER SECTION:
query-preview.wikidata.org. 86400 INCNAME dyna.wikimedia.org.
;; Query time: 82 msec
;; SERVER: 208.80.154.238#53(208.80.154.238)
;; WHEN: Fri Mar 12 17:06:00 PST 2021
;; MSG SIZE rcvd: 84
; <<>> DiG 9.10.6 <<>> @ns1.wikimedia.org -t any query-preview.wikidata.org
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 3886
;; flags: qr aa rd; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1
;; WARNING: recursion requested but not available
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 1024
;; QUESTION SECTION:
;query-preview.wikidata.org.IN ANY
;; ANSWER SECTION:
query-preview.wikidata.org. 86400 INCNAME dyna.wikimedia.org.
;; Query time: 61 msec
;; SERVER: 208.80.153.231#53(208.80.153.231)
;; WHEN: Fri Mar 12 17:06:00 PST 2021
;; MSG SIZE rcvd: 84
; <<>> DiG 9.10.6 <<>> @ns2.wikimedia.org -t any query-preview.wikidata.org
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 53512
;; flags: qr aa rd; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1
;; WARNING: recursion requested but not available
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 1024
;; QUESTION SECTION:
;query-preview.wikidata.org.IN ANY
;; ANSWER SECTION:
query-preview.wikidata.org. 86400 INCNAME dyna.wikimedia.org.
;; Query time: 345 msec
;; SERVER: 91.198.174.239#53(91.198.174.239)
;; WHEN: Fri Mar 12 17:06:01 PST 2021
;; MSG SIZE rcvd: 84
TASK DETAIL
https://phabricator.wikimedia.org/T266470
EMAIL PREFERENCES
https://phabricator.wikimedi
[Wikidata-bugs] [Maniphest] T266470: Expose wdqs1009 to wdqs users and gather feedback
RKemper added a comment. Finished rolling back to the previous iteration of `wdqs.discovery.wmnet` cert since we're now going to create a net-new cert `wdqs1009.eqiad.wmnet` for `wdqs-test` TASK DETAIL https://phabricator.wikimedia.org/T266470 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: RKemper Cc: Gehel, Lea_Lacroix_WMDE, dcausse, Aklapper, MPhamWMF, maantietaja, Alter-paule, Beast1978, CBogen, Un1tY, Akuckartz, Hook696, Kent7301, joker88john, CucyNoiD, Nandana, Namenlos314, Gaboe420, Giuliamocci, Cpaulf30, Lahi, Gq86, Af420, Bsandipan, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, Lewizho99, Maathavan, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, abian, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T266470: Expose wdqs1009 to wdqs users and gather feedback
RKemper added a comment. Certificate(wdqs.discovery.wmnet, authorities=[PuppetCA(puppetmaster1001.eqiad.wmnet_8140)]): /srv/private/modules/secret/secrets/certificates/wdqs.discovery.wmnet/wdqs.discovery.wmnet.key.private.pem: PRESENT (mtime: 2019-10-21T08:11:52.741452) /srv/private/modules/secret/secrets/certificates/wdqs.discovery.wmnet/wdqs.discovery.wmnet.key.public.pem: PRESENT (mtime: 2019-10-21T08:11:52.745452) /srv/private/modules/secret/secrets/certificates/wdqs.discovery.wmnet/wdqs.discovery.wmnet.crt.pem: PRESENT (mtime: 2021-03-10T19:17:36.206506) /srv/private/modules/secret/secrets/certificates/wdqs.discovery.wmnet/ca.crt.pem: PRESENT (mtime: 2019-12-10T14:26:42.496697) /srv/private/modules/secret/secrets/certificates/wdqs.discovery.wmnet/wdqs.discovery.wmnet.keystore.p12: PRESENT (mtime: 2021-03-10T19:17:36.226506) /srv/private/modules/secret/secrets/certificates/wdqs.discovery.wmnet/wdqs.discovery.wmnet.keystore.jks: PRESENT (mtime: 2021-03-10T19:17:38.030503) /srv/private/modules/secret/secrets/certificates/wdqs.discovery.wmnet/truststore.jks: PRESENT (mtime: 2021-03-10T19:17:38.938502) ryankemper@puppetmaster1001:/srv/private$ git status On branch master Changes not staged for commit: (use "git add ..." to update what will be committed) (use "git checkout -- ..." to discard changes in working directory) modified: modules/secret/secrets/certificates/certificate.manifests.d/wdqs.certs.yaml modified: modules/secret/secrets/certificates/wdqs.discovery.wmnet/truststore.jks modified: modules/secret/secrets/certificates/wdqs.discovery.wmnet/wdqs.discovery.wmnet.crt.pem modified: modules/secret/secrets/certificates/wdqs.discovery.wmnet/wdqs.discovery.wmnet.csr.pem modified: modules/secret/secrets/certificates/wdqs.discovery.wmnet/wdqs.discovery.wmnet.keystore.jks modified: modules/secret/secrets/certificates/wdqs.discovery.wmnet/wdqs.discovery.wmnet.keystore.p12 TASK DETAIL https://phabricator.wikimedia.org/T266470 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: RKemper Cc: Gehel, Lea_Lacroix_WMDE, dcausse, Aklapper, MPhamWMF, maantietaja, Alter-paule, Beast1978, CBogen, Un1tY, Akuckartz, Hook696, Kent7301, joker88john, CucyNoiD, Nandana, Namenlos314, Gaboe420, Giuliamocci, Cpaulf30, Lahi, Gq86, Af420, Bsandipan, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, Lewizho99, Maathavan, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, abian, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T266470: Expose wdqs1009 to wdqs users and gather feedback
RKemper added a comment. Current status for when I pick this back up: - `query-preview.wikidata.org` (with `wdqs1009.eqiad.wmnet` as an `alt_name`) is generated - reverted https://gerrit.wikimedia.org/r/c/operations/puppet/+/670339/ for now, envoy wasn't happy: Error: /Stage[main]/Envoyproxy/Exec[verify-envoy-config]: Failed to call refresh: '/usr/local/sbin/build-envoy-config -c '/etc/envoy'' returned 1 instead of one of [0] Error: /Stage[main]/Envoyproxy/Exec[verify-envoy-config]: '/usr/local/sbin/build-envoy-config -c '/etc/envoy'' returned 1 instead of one of [0] Error: /Stage[main]/Profile::Tlsproxy::Envoy/Sslcert::Certificate[query-preview.wikidata.org]/File[/etc/ssl/localcerts/query-preview.wikidata.org.crt]: Could not evaluate: Could not retrieve information from environment production source(s) puppet:///files/ssl/query-preview.wikidata.org.crt TASK DETAIL https://phabricator.wikimedia.org/T266470 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: RKemper Cc: Gehel, Lea_Lacroix_WMDE, dcausse, Aklapper, MPhamWMF, maantietaja, Alter-paule, Beast1978, CBogen, Un1tY, Akuckartz, Hook696, Kent7301, joker88john, CucyNoiD, Nandana, Namenlos314, Gaboe420, Giuliamocci, Cpaulf30, Lahi, Gq86, Af420, Bsandipan, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, Lewizho99, Maathavan, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, abian, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T266470: Expose wdqs1009 to wdqs users and gather feedback
RKemper added a comment.
Hit a big blocker with the current proposed approach of using
`wdqs1009.eqiad.wmnet` as the cert name:
ryankemper@puppetmaster1001:/srv/private$ sudo cergen -c 'wdqs1009.*'
--generate --base-path /srv/private/modules/secret/secrets/certificates
/srv/private/modules/secret/secrets/certificates/certificate.manifests.d
2021-03-12 07:55:03,774 INFO cergen
Generating certificates ['wdqs1009.eqiad.wmnet'] with force=False
2021-03-12 07:55:03,774 INFO Certificate(wdqs1009.eqiad.wmnet)
Generating all files, force=False...
2021-03-12 07:55:03,776 INFO Certificate(wdqs1009.eqiad.wmnet)
Generating certificate file
/usr/lib/python3/dist-packages/urllib3/connection.py:362:
SubjectAltNameWarning: Certificate for puppetmaster1001.eqiad.wmnet has no
`subjectAltName`, falling back to check for a `commonName` for now. This
feature is being removed by major browsers and deprecated by RFC 2818. (See
https://github.com/shazow/urllib3/issues/497 for details.)
SubjectAltNameWarning
Traceback (most recent call last):
File "/usr/bin/cergen", line 11, in
load_entry_point('cergen==0.2.5', 'console_scripts', 'cergen')()
File "/usr/lib/python3/dist-packages/cergen/main.py", line 93, in main
certificate.generate(force=args['--force'])
File "/usr/lib/python3/dist-packages/cergen/certificate.py", line 304, in
generate
self.generate_crt(force=force)
File "/usr/lib/python3/dist-packages/cergen/certificate.py", line 344, in
generate_crt
self.x509_cert = self.authority.sign(csr, self.expiry)
File "/usr/lib/python3/dist-packages/cergen/puppet.py", line 193, in sign
self.name, common_name
RuntimeError: puppetmaster1001.eqiad.wmnet_8140 already has a signed
certificate for wdqs1009.eqiad.wmnet. If you are trying to regenerate a Puppet
CA signed certificate, you need to first remove the certificate from the Puppet
CA. On the puppetmaster, `puppet cert clean wdqs1009.eqiad.wmnet` should do it.
Corresponding to this cergen manifest file
`modules/secret/secrets/certificates/certificate.manifests.d/wdqs1009.certs.yaml`:
wdqs1009.eqiad.wmnet:
authority: puppet_ca
expiry: null
alt_names: ["wdqs1009.eqiad.wmnet","query-preview.wikidata.org"]
key:
password: REDACTED
algorithm: ec
I'm a bit lost on the right way to proceed. Thinking out loud here:
**Option 1** Just actually re-generate the cert with `puppet cert clean
wdqs1009.eqiad.wmnet` before running cergen. This feels wrong though, and I
think ultimately it comes down to using cergen in a way that wasn't intended:
name collision with the existing cert puppet has.
**Option 2** Do this instead:
query-preview.wikidata.org:
authority: puppet_ca
expiry: null
alt_names: ["query-preview.wikidata.org", "wdqs1009.eqiad.wmnet"]
key:
password: REDACTED
algorithm: ec
Might want to talk to traffic again. But in the interest of iteration speed I
think trying out Option **2** is not a bad idea; there's not really any risk
since this is pertaining to `wdqs1009.eqiad.wmnet` which is a test host.
TASK DETAIL
https://phabricator.wikimedia.org/T266470
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: RKemper
Cc: Gehel, Lea_Lacroix_WMDE, dcausse, Aklapper, MPhamWMF, maantietaja,
Alter-paule, Beast1978, CBogen, Un1tY, Akuckartz, Hook696, Kent7301,
joker88john, CucyNoiD, Nandana, Namenlos314, Gaboe420, Giuliamocci, Cpaulf30,
Lahi, Gq86, Af420, Bsandipan, Lucas_Werkmeister_WMDE, GoranSMilovanovic,
QZanden, EBjune, merbst, LawExplorer, Lewizho99, Maathavan, _jensen,
rosalieper, Scott_WUaS, Jonas, Xmlizer, abian, jkroll, Wikidata-bugs, Jdouglas,
aude, Tobias1984, Manybubbles, Mbch331
___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T266470: Expose wdqs1009 to wdqs users and gather feedback
RKemper added a comment. **Option 2 ** fails to even generate the cert. All the cergen documentation is written for a certificate like `query-preview.discovery.wmnet` and not `wdqs1009.eqiad.wmnet` or `query-preview.wikidata.org`. So I do think this just isn't what cergen is built to do. ryankemper@puppetmaster1001:/srv/private$ git status On branch master Untracked files: (use "git add ..." to include in what will be committed) modules/secret/secrets/certificates/certificate.manifests.d/wdqs1009.certs.yaml nothing added to commit but untracked files present (use "git add" to track) ryankemper@puppetmaster1001:/srv/private$ sudo cergen -c 'wdqs1009.*' --generate --base-path /srv/private/modules/secret/secrets/certificates /srv/private/modules/secret/secrets/certificates/certificate.manifests.d 2021-03-12 08:06:03,210 INFO cergen Generating certificates [] with force=False Status of certificates [] TASK DETAIL https://phabricator.wikimedia.org/T266470 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: RKemper Cc: Gehel, Lea_Lacroix_WMDE, dcausse, Aklapper, MPhamWMF, maantietaja, Alter-paule, Beast1978, CBogen, Un1tY, Akuckartz, Hook696, Kent7301, joker88john, CucyNoiD, Nandana, Namenlos314, Gaboe420, Giuliamocci, Cpaulf30, Lahi, Gq86, Af420, Bsandipan, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, Lewizho99, Maathavan, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, abian, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T266470: Expose wdqs1009 to wdqs users and gather feedback
RKemper added a comment. Ah, so poking around the `certificate.manifests.d` repo I see certs that don't necessarily follow the `discovery.wmnet` pattern. To me that implies **Option 2** should be working, so I might be missing something. Here's an example that doesn't use discovery: ryankemper@puppetmaster1001:/srv/private$ cat modules/secret/secrets/certificates/certificate.manifests.d/analytics_http_ui.certs.yaml yarn.wikimedia.org: authority: puppet_ca expiry: null alt_names: ["yarn.wikimedia.org", "hue.wikimedia.org", "hue-next.wikimedia.org", "superset.wikimedia.org", "pivot.wikimedia.org", "turnilo.wikimedia.org", "stats.wikimedia.org", "analytics.wikimedia.org", "piwik.wikimedia.org", "datasets.wikimedia.org"] key: password: REDACTED algorithm: ec TASK DETAIL https://phabricator.wikimedia.org/T266470 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: RKemper Cc: Gehel, Lea_Lacroix_WMDE, dcausse, Aklapper, MPhamWMF, maantietaja, Alter-paule, Beast1978, CBogen, Un1tY, Akuckartz, Hook696, Kent7301, joker88john, CucyNoiD, Nandana, Namenlos314, Gaboe420, Giuliamocci, Cpaulf30, Lahi, Gq86, Af420, Bsandipan, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, Lewizho99, Maathavan, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, abian, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T266470: Expose wdqs1009 to wdqs users and gather feedback
RKemper added a comment. Certificate(wdqs.discovery.wmnet, authorities=[PuppetCA(puppetmaster1001.eqiad.wmnet_8140)]): /srv/private/modules/secret/secrets/certificates/wdqs.discovery.wmnet/wdqs.discovery.wmnet.key.private.pem: PRESENT (mtime: 2019-10-21T08:11:52.741452) /srv/private/modules/secret/secrets/certificates/wdqs.discovery.wmnet/wdqs.discovery.wmnet.key.public.pem: PRESENT (mtime: 2019-10-21T08:11:52.745452) /srv/private/modules/secret/secrets/certificates/wdqs.discovery.wmnet/wdqs.discovery.wmnet.crt.pem: PRESENT (mtime: 2021-03-10T04:58:24.029629) /srv/private/modules/secret/secrets/certificates/wdqs.discovery.wmnet/ca.crt.pem: PRESENT (mtime: 2019-12-10T14:26:42.496697) /srv/private/modules/secret/secrets/certificates/wdqs.discovery.wmnet/wdqs.discovery.wmnet.keystore.p12: PRESENT (mtime: 2021-03-10T04:58:24.049629) /srv/private/modules/secret/secrets/certificates/wdqs.discovery.wmnet/wdqs.discovery.wmnet.keystore.jks: PRESENT (mtime: 2021-03-10T04:58:25.893621) /srv/private/modules/secret/secrets/certificates/wdqs.discovery.wmnet/truststore.jks: PRESENT (mtime: 2021-03-10T04:58:26.865617) ryankemper@puppetmaster1001:/srv/private$ git status On branch master Changes not staged for commit: (use "git add ..." to update what will be committed) (use "git checkout -- ..." to discard changes in working directory) modified: modules/secret/secrets/certificates/certificate.manifests.d/wdqs.certs.yaml modified: modules/secret/secrets/certificates/wdqs.discovery.wmnet/truststore.jks modified: modules/secret/secrets/certificates/wdqs.discovery.wmnet/wdqs.discovery.wmnet.crt.pem modified: modules/secret/secrets/certificates/wdqs.discovery.wmnet/wdqs.discovery.wmnet.csr.pem modified: modules/secret/secrets/certificates/wdqs.discovery.wmnet/wdqs.discovery.wmnet.keystore.jks modified: modules/secret/secrets/certificates/wdqs.discovery.wmnet/wdqs.discovery.wmnet.keystore.p12 no changes added to commit (use "git add" and/or "git commit -a") Relevant status after performing the cergen. (Git commit not created yet as seen by the dirty working directory) TASK DETAIL https://phabricator.wikimedia.org/T266470 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: RKemper Cc: Gehel, Lea_Lacroix_WMDE, dcausse, Aklapper, MPhamWMF, maantietaja, Alter-paule, Beast1978, CBogen, Un1tY, Akuckartz, Hook696, Kent7301, joker88john, CucyNoiD, Nandana, Namenlos314, Gaboe420, Giuliamocci, Cpaulf30, Lahi, Gq86, Af420, Bsandipan, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, Lewizho99, Maathavan, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, abian, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T267927: Reload wikidata journal from fresh dumps
RKemper added a comment. Downtimed `wdqs2008` until `2021-03-04 21:56:59` TASK DETAIL https://phabricator.wikimedia.org/T267927 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: RKemper Cc: Zbyszko, Gehel, Aklapper, dcausse, MPhamWMF, CBogen, Akuckartz, Nandana, Namenlos314, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T267927: Reload wikidata journal from fresh dumps
RKemper added a comment. Note: Puppet is still disabled on `wdqs2008` while the reload runs. It occurred to me that I'm not sure if puppet actually **needs** to be disabled during data reloads or if that's just a precaution we've historically taken - any insight here @Gehel? TASK DETAIL https://phabricator.wikimedia.org/T267927 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: RKemper Cc: Zbyszko, Gehel, Aklapper, dcausse, MPhamWMF, CBogen, Akuckartz, Nandana, Namenlos314, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T274751: Upgrade firmware on wdqs1009
RKemper added a comment. @Cmjohnson The data reload is complete on `wdqs1009`, so the host can now have its firmware upgraded and be rebooted at its convenience. Note this is an internal wdqs test host, so there is no public-facing service for us to worry about. Feel free to proceed when convenient for you - just shoot me a ping or drop a line in #wikimedia-discovery, but no need to schedule a formal window. TASK DETAIL https://phabricator.wikimedia.org/T274751 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Cmjohnson, RKemper Cc: Jclark-ctr, RKemper, Cmjohnson, Gehel, MoritzMuehlenhoff, Ramtin0071, MPhamWMF, Devnull, lmata, Muchiri124, CBogen, Akuckartz, Legado_Shulgin, Nandana, Namenlos314, Davinaclare77, Qtn1293, Techguru.pc, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, Th3d3v1ls, Hfbn0, QZanden, EBjune, merbst, LawExplorer, Zppix, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, Wong128hk, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Southparkfan, faidon, Addshore, Mbch331, Rxy, Jay8g, fgiunchedi ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T266470: Expose wdqs1009 to wdqs users and gather feedback
RKemper added a comment. Here's how the ats mapping looks afterdeploy of the `backend.yaml` changes: ryankemper@cp1075:~$ sudo cat /etc/trafficserver/remap.config | grep query map http://query-preview.wikidata.org/bigdata https://wdqs1009.eqiad.wmnet/bigdata map http://query-preview.wikidata.org/sparql https://wdqs1009.eqiad.wmnet/sparql map http://query-preview.wikidata.org https://webserver-misc-apps.discovery.wmnet map http://query.wikidata.org/bigdata/ldf https://wdqs1005.eqiad.wmnet/bigdata/ldf map http://query.wikidata.org/sparql https://wdqs.discovery.wmnet/sparql map http://query.wikidata.org/bigdata https://wdqs.discovery.wmnet/bigdata map http://query.wikidata.org https://webserver-misc-apps.discovery.wmnet TASK DETAIL https://phabricator.wikimedia.org/T266470 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: RKemper Cc: Gehel, Lea_Lacroix_WMDE, dcausse, Aklapper, MPhamWMF, maantietaja, Alter-paule, Beast1978, CBogen, Un1tY, Akuckartz, Hook696, Kent7301, joker88john, CucyNoiD, Nandana, Namenlos314, Gaboe420, Giuliamocci, Cpaulf30, Lahi, Gq86, Af420, Bsandipan, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, Lewizho99, Maathavan, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, abian, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T266470: Expose wdqs1009 to wdqs users and gather feedback
RKemper added a comment. Posting logs of our IRC convo from ~1 month ago for context when I tag people for review: [2021-02-04 11:31:36] bblack: around ? [2021-02-04 11:31:58] I'm here with ryankemper to talk about T266470, unless you want to jump in a meet [2021-02-04 11:31:59] T266470: Expose wdqs1009 to wdqs users and gather feedback - https://phabricator.wikimedia.org/T266470 [2021-02-04 11:32:51] taking notes in https://etherpad.wikimedia.org/p/expose-wdqs1009 [2021-02-04 11:33:08] gehel: hi :) [2021-02-04 11:33:15] o/ [2021-02-04 11:33:15] ryankemper: hi too :) [2021-02-04 11:33:23] \o [2021-02-04 11:33:40] so, I guess my best understanding of the present stuff is what I see in: [2021-02-04 11:33:47] context: we want to expose wdqs1009 as a test server so that our users can make sure we're not breaking anything with the new WDQS updater [2021-02-04 11:33:53] https://gerrit.wikimedia.org/r/plugins/gitiles/operations/puppet/+/refs/heads/production/hieradata/common/profile/trafficserver/backend.yaml#157 [2021-02-04 11:34:12] there's several related definitions there for https://query.wikidata.org/ [2021-02-04 11:34:24] for different subspaces of the URI routing to different backends [2021-02-04 11:34:40] /bigdata/ldf goes explicitly to wds1005 [2021-02-04 11:35:07] /bigdata and /sparql both go to the discovery service to a set of wdqs [2021-02-04 11:35:25] but the root url hits our misc webapps setup, for the JS powering the query UI [2021-02-04 11:35:29] yep, somewhat similar, but in this case, we want to have a different FQDN (probably query-preview.wikidata.org or something similar) and route to a different server that isn't part of the current WDQS pools [2021-02-04 11:36:45] so in this case, I would guess, you'd want no special exception for /bigdata/ldf, and you'd want all of /bigdata and /sparql -> wdqs1009 [2021-02-04 11:37:06] which is something we can just define in that file I linked, and adding a name at the DNS layer, and done [2021-02-04 11:37:16] but the root URL for the UI part, might be a little trickier [2021-02-04 11:37:39] I need to see where that's defined in git, and it might need a new site definition, and to template in a new place to send the UI queries... [2021-02-04 11:37:57] but there's no other special requirements I think/expect? [2021-02-04 11:37:57] we'll probably want to create another microsite for the JS part, that's fairly straightforward [2021-02-04 11:38:15] no other special requirements IIRC [2021-02-04 11:38:17] yep, that seems fairly simple to me [2021-02-04 11:38:22] so to resume: [2021-02-04 11:39:08] create a DNS entry (CNAME to dyna.wm.o), another set of entries in backend.yaml map, create another minisite (with the appropriate configuration) [2021-02-04 11:39:22] do we need to do anything for SSL certs? Or is that fully automated? [2021-02-04 11:39:48] that part's all shared between everything that runs through the cache, so long as you stick to our canonical domains and just change the leading hostname part [2021-02-04 11:40:04] to e.g. query-preview.wikidata.org [2021-02-04 11:40:05] "canonical" includes wikidata.org? [2021-02-04 11:40:11] ok, good! [2021-02-04 11:40:37] yeah for reference, the canonical set is: https://wikitech.wikimedia.org/wiki/HTTPS#For_the_Foundation's_canonical_domain_names [2021-02-04 11:41:13] ok, one last stupid and off topic question: what does "dyna" stands for in dyna.wm.o? [2021-02-04 11:42:14] it's not a "real" DNS RR-type you would see in the wild. It's a custom thing our authserver implements which provides dynamic A and records depending on geodns. that's how we route to the closest of our geographic edges for users [2021-02-04 11:42:36] https://github.com/gdnsd/gdnsd/wiki/GdnsdPluginGeoip [2021-02-04 11:42:41] ^ that, basically [2021-02-04 11:42:57] thanks! [2021-02-04 11:43:23] templates/wikimedia.org:dyna600 IN DYNA geoip!text-addrs [2021-02-04 11:43:36] ^ is what all the other production CNAMEs are pointing towards [2021-02-04 11:43:54] gehel: also I don't know if you've created new service hostnames since the move to netbox [2021-02-04 11:44:03] but we don't do commits on the ops/dns repo for that kind of thing anymore [2021-02-04 11:44:25] Oh no, I havent [2021-02-04 11:44:33] ryankemper: is the one who's goign to do it [2021-02-04 11:45:00] I assume it's documented [2021-02-04 11:45:01] oh wait, I'm getting ahead of ourselves [2021-02-04 11:45:03] — gehel is searching [2021-02-04 11:45:13] that part still is done in ops/dns ! :) [2021-02-04 11:45:25] now I'm curious anyway! [2021-02-04 11:45:43] https://gerrit.wikimedia.org/r/plugins/gitiles/operations/dns/+/refs/heads
[Wikidata-bugs] [Maniphest] T252504: Automate the smoke test of the canary deployment of WDQS
RKemper renamed this task from "Smoke test the canary deployment of WDQS" to "Automate the smoke test of the canary deployment of WDQS". RKemper updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T252504 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: RKemper Cc: Aklapper, Zbyszko, MPhamWMF, maantietaja, CBogen, Akuckartz, Nandana, Namenlos314, jijiki, Klaas_Z4us_V, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, elukey, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, abian, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Addshore, Mbch331, Jay8g ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T280108: WDQS data-transfer cookbook needs to wait for updater to catchup on lag
RKemper added a comment. Tested out the new cookbook, there's at least one small issue: Exception raised while executing cookbook sre.wdqs.data-transfer: Traceback (most recent call last): File "/usr/lib/python3/dist-packages/spicerack/_menu.py", line 234, in run raw_ret = runner.run() File "/usr/lib/python3/dist-packages/spicerack/_module_api.py", line 18, in run return self._run(self.args, self.spicerack) File "/srv/deployment/spicerack/cookbooks/sre/wdqs/data-transfer.py", line 179, in run wait_for_updater(prometheus, site(source), source) File "/usr/lib/python3/dist-packages/spicerack/decorators.py", line 104, in wrapper return func(*args, **kwargs) # type: ignore File "/srv/deployment/spicerack/cookbooks/sre/wdqs/__init__.py", line 25, in wait_for_updater last_updated = int(result['value'][1]) TypeError: list indices must be integers or slices, not str END (FAIL) - Cookbook sre.wdqs.data-transfer (exit_code=99) Error seems quite straightforward - we'll want to patch this. TASK DETAIL https://phabricator.wikimedia.org/T280108 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: RKemper Cc: RKemper, Gehel, Aklapper, Invadibot, MPhamWMF, maantietaja, CBogen, Akuckartz, Nandana, Namenlos314, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T267927: Reload wikidata journal from fresh dumps
RKemper updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T267927 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: RKemper Cc: jbond, Zbyszko, Gehel, Aklapper, dcausse, Invadibot, MPhamWMF, maantietaja, CBogen, Akuckartz, Nandana, Namenlos314, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T267927: Reload wikidata journal from fresh dumps
RKemper removed a project: Patch-For-Review. TASK DETAIL https://phabricator.wikimedia.org/T267927 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: RKemper Cc: jbond, Zbyszko, Gehel, Aklapper, dcausse, Invadibot, MPhamWMF, maantietaja, CBogen, Akuckartz, Nandana, Namenlos314, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331, Alter-paule, Beast1978, Un1tY, Hook696, Kent7301, joker88john, CucyNoiD, Gaboe420, Giuliamocci, Cpaulf30, Af420, Bsandipan, Lewizho99, Maathavan ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T267927: Reload wikidata journal from fresh dumps
RKemper updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T267927 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: RKemper Cc: jbond, Zbyszko, Gehel, Aklapper, dcausse, Invadibot, MPhamWMF, maantietaja, Alter-paule, Beast1978, CBogen, Un1tY, Akuckartz, Hook696, Kent7301, joker88john, CucyNoiD, Nandana, Namenlos314, Gaboe420, Giuliamocci, Cpaulf30, Lahi, Gq86, Af420, Bsandipan, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, Lewizho99, Maathavan, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T267927: Reload wikidata journal from fresh dumps
RKemper added a comment. Since we've seeded to `wdqs1004.eqiad.wmnet` we can start using that to transfer to other eqiad public nodes. Still need to seed to the other 3 (codfw public, codfw internal, eqiad internal) before we'll be at the peak "velocity". TASK DETAIL https://phabricator.wikimedia.org/T267927 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: RKemper Cc: jbond, Zbyszko, Gehel, Aklapper, dcausse, Invadibot, MPhamWMF, maantietaja, CBogen, Akuckartz, Nandana, Namenlos314, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T280108: WDQS data-transfer cookbook needs to wait for updater to catchup on lag
RKemper added a comment.
(For future reference)
import logging
logging.basicConfig(level=logging.DEBUG)
from spicerack import Spicerack
spicerack = Spicerack(verbose=True, dry_run=True)
def site(host):
"""Hacky way to get the site in which a host is located."""
if 'eqiad' in str(host):
return 'eqiad'
if 'codfw' in str(host):
return 'codfw'
raise ValueError('Site is unknown for {host}.'.format(host=host))
remote = spicerack.remote()
prometheus = spicerack.prometheus()
remote_host = remote.query('wdqs2008.codfw.wmnet')
host = remote_host.hosts[0].split(".")[0]
query = "scalar(time() - blazegraph_lastupdated{instance='%s:9193'})" % host
result = prometheus.query(query, site(remote_host))
last_updated = int(result[1])
if last_updated > 1200:
raise ValueError("Let's wait for updater to catch up, current
last_updated of {} is too high.".format(last_updated))
Snippet used to diagnose the above issue without having to actually run the
cookbook
TASK DETAIL
https://phabricator.wikimedia.org/T280108
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: RKemper
Cc: RKemper, Gehel, Aklapper, Invadibot, MPhamWMF, maantietaja, Alter-paule,
Beast1978, CBogen, Un1tY, Akuckartz, Hook696, Kent7301, joker88john, CucyNoiD,
Nandana, Namenlos314, Gaboe420, Giuliamocci, Cpaulf30, Lahi, Gq86, Af420,
Bsandipan, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst,
LawExplorer, Lewizho99, Maathavan, _jensen, rosalieper, Scott_WUaS, Jonas,
Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331
___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T280108: WDQS data-transfer cookbook needs to wait for updater to catchup on lag
RKemper added a comment. 100.0% (2/2) success ratio (>= 100.0% threshold) for command: 'systemctl start ...art wdqs-updater'. 100.0% (2/2) success ratio (>= 100.0% threshold) of nodes successfully executed all commands. [1/1000, retrying in 600.00s] Attempt to run 'cookbooks.sre.wdqs.wait_for_updater' raised: invalid literal for int() with base 10: 'NaN' [2/1000, retrying in 600.00s] Attempt to run 'cookbooks.sre.wdqs.wait_for_updater' raised: invalid literal for int() with base 10: '4458.16434332' [3/1000, retrying in 600.00s] Attempt to run 'cookbooks.sre.wdqs.wait_for_updater' raised: invalid literal for int() with base 10: '4230.269809265' [4/1000, retrying in 600.00s] Attempt to run 'cookbooks.sre.wdqs.wait_for_updater' raised: invalid literal for int() with base 10: '3850.3779346' [5/1000, retrying in 600.00s] Attempt to run 'cookbooks.sre.wdqs.wait_for_updater' raised: invalid literal for int() with base 10: '3175.483400543' [6/1000, retrying in 600.00s] Attempt to run 'cookbooks.sre.wdqs.wait_for_updater' raised: invalid literal for int() with base 10: '888.591801086' [7/1000, retrying in 600.00s] Attempt to run 'cookbooks.sre.wdqs.wait_for_updater' raised: invalid literal for int() with base 10: '39.694332428' [8/1000, retrying in 600.00s] Attempt to run 'cookbooks.sre.wdqs.wait_for_updater' raised: invalid literal for int() with base 10: '42.79952316284' Another small issue. Looks like we need to tell it to truncate (or just use doubles instead of ints) TASK DETAIL https://phabricator.wikimedia.org/T280108 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: RKemper Cc: RKemper, Gehel, Aklapper, Invadibot, MPhamWMF, maantietaja, CBogen, Akuckartz, Nandana, Namenlos314, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T267927: Reload wikidata journal from fresh dumps
RKemper updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T267927 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: RKemper Cc: jbond, Zbyszko, Gehel, Aklapper, dcausse, Invadibot, MPhamWMF, maantietaja, CBogen, Akuckartz, Nandana, Namenlos314, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T267927: Reload wikidata journal from fresh dumps
RKemper added a comment. [EQIAD PUBLIC] DONE sudo -i cookbook sre.wdqs.data-transfer --source wdqs2008.codfw.wmnet --dest wdqs1004.eqiad.wmnet --reason "transferring wikidata journal following reload from dumps" --blazegraph_instance blazegraph --task-id T267927 <https://phabricator.wikimedia.org/T267927> sudo -i cookbook sre.wdqs.data-transfer --source wdqs2008.codfw.wmnet --dest wdqs1005.eqiad.wmnet --reason "transferring wikidata journal following reload from dumps" --blazegraph_instance blazegraph --task-id T267927 <https://phabricator.wikimedia.org/T267927> sudo -i cookbook sre.wdqs.data-transfer --source wdqs2008.codfw.wmnet --dest wdqs1006.eqiad.wmnet --reason "transferring wikidata journal following reload from dumps" --blazegraph_instance blazegraph --task-id T267927 <https://phabricator.wikimedia.org/T267927> sudo -i cookbook sre.wdqs.data-transfer --source wdqs2008.codfw.wmnet --dest wdqs1007.eqiad.wmnet --reason "transferring wikidata journal following reload from dumps" --blazegraph_instance blazegraph --task-id T267927 <https://phabricator.wikimedia.org/T267927> sudo -i cookbook sre.wdqs.data-transfer --source wdqs2008.codfw.wmnet --dest wdqs1012.eqiad.wmnet --reason "transferring wikidata journal following reload from dumps" --blazegraph_instance blazegraph --task-id T267927 <https://phabricator.wikimedia.org/T267927> sudo -i cookbook sre.wdqs.data-transfer --source wdqs2008.codfw.wmnet --dest wdqs1013.eqiad.wmnet --reason "transferring wikidata journal following reload from dumps" --blazegraph_instance blazegraph --task-id T267927 <https://phabricator.wikimedia.org/T267927> [CODFW PUBLIC] DONE sudo -i cookbook sre.wdqs.data-transfer --source wdqs2008.codfw.wmnet --dest wdqs2001.codfw.wmnet --reason "transferring wikidata journal following reload from dumps" --blazegraph_instance blazegraph --task-id T267927 <https://phabricator.wikimedia.org/T267927> sudo -i cookbook sre.wdqs.data-transfer --source wdqs2008.codfw.wmnet --dest wdqs2002.codfw.wmnet --reason "transferring wikidata journal following reload from dumps" --blazegraph_instance blazegraph --task-id T267927 <https://phabricator.wikimedia.org/T267927> sudo -i cookbook sre.wdqs.data-transfer --source wdqs2008.codfw.wmnet --dest wdqs2003.codfw.wmnet --reason "transferring wikidata journal following reload from dumps" --blazegraph_instance blazegraph --task-id T267927 <https://phabricator.wikimedia.org/T267927> sudo -i cookbook sre.wdqs.data-transfer --source wdqs2008.codfw.wmnet --dest wdqs2007.codfw.wmnet --reason "transferring wikidata journal following reload from dumps" --blazegraph_instance blazegraph --task-id T267927 <https://phabricator.wikimedia.org/T267927> [EQIAD INTERNAL] DONE sudo -i cookbook sre.wdqs.data-transfer --source wdqs2008.codfw.wmnet --dest wdqs1003.eqiad.wmnet --reason "transferring wikidata journal following reload from dumps" --blazegraph_instance blazegraph --task-id T267927 <https://phabricator.wikimedia.org/T267927> sudo -i cookbook sre.wdqs.data-transfer --source wdqs2008.codfw.wmnet --dest wdqs1008.eqiad.wmnet --reason "transferring wikidata journal following reload from dumps" --blazegraph_instance blazegraph --task-id T267927 <https://phabricator.wikimedia.org/T267927> sudo -i cookbook sre.wdqs.data-transfer --source wdqs2008.codfw.wmnet --dest wdqs1011.eqiad.wmnet --reason "transferring wikidata journal following reload from dumps" --blazegraph_instance blazegraph --task-id T267927 <https://phabricator.wikimedia.org/T267927> [CODFW INTERNAL] DONE sudo -i cookbook sre.wdqs.data-transfer --source wdqs2008.codfw.wmnet --dest wdqs2004.codfw.wmnet --reason "transferring wikidata journal following reload from dumps" --blazegraph_instance blazegraph --task-id T267927 <https://phabricator.wikimedia.org/T267927> sudo -i cookbook sre.wdqs.data-transfer --source wdqs2008.codfw.wmnet --dest wdqs2005.codfw.wmnet --reason "transferring wikidata journal following reload from dumps" --blazegraph_instance blazegraph --task-id T267927 <https://phabricator.wikimedia.org/T267927> sudo -i cookbook sre.wdqs.data-transfer --source wdqs2008.codfw.wmnet --dest wdqs2006.codfw.wmnet --reason "transferring wikidata journal following reload from dumps" --blazegraph_instance blazegraph --task-id T267927 <https://phabricator.wikimedia.org/T267927> TASK DETAIL https://phabricator.wikimedia.org/T267927 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: RKemper Cc: jbond, Zbyszko, Gehel, Aklapper, dcausse, Invadibot, MPhamWMF, maantietaja, Alter-paule, Beast1978, CBogen, Un1tY, Akuckartz, Hook696, Kent7301, joker88john, CucyNoiD, Nanda
[Wikidata-bugs] [Maniphest] T267927: Reload wikidata journal from fresh dumps
RKemper added a comment. I was having trouble understanding what the restart status of blazegraph has to do with the transfer of `/srv/wdqs/wikidata.jnl`. (As an aside, the point that running multiple simultaneous transfers from the same source is bottlenecked by outgoing bandwidth anyway makes perfect sense to me). It just clicked for me though - the cookbook transfers the file while blazegraph is disabled, so if another cookbook instance re-enables blazegraph while another cookbook instance is still receiving `/srv/wdqs/wikidata.jnl`, the `wikidata.jnl` will be getting mutated during the transfer. Note: I updated my comment above to indicate that `wdqs100[3,4]` are not properly done due to the data corruption. TASK DETAIL https://phabricator.wikimedia.org/T267927 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: RKemper Cc: jbond, Zbyszko, Gehel, Aklapper, dcausse, Invadibot, MPhamWMF, maantietaja, Alter-paule, Beast1978, CBogen, Un1tY, Akuckartz, Hook696, Kent7301, joker88john, CucyNoiD, Nandana, Namenlos314, Gaboe420, Giuliamocci, Cpaulf30, Lahi, Gq86, Af420, Bsandipan, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, Lewizho99, Maathavan, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T267927: Reload wikidata journal from fresh dumps
RKemper updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T267927 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: RKemper Cc: jbond, Zbyszko, Gehel, Aklapper, dcausse, Invadibot, MPhamWMF, maantietaja, CBogen, Akuckartz, Nandana, Namenlos314, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T267927: Reload wikidata journal from fresh dumps
RKemper added a comment. We can't proceed any further on this ticket until we address https://phabricator.wikimedia.org/T280382, because the short-term fix for disk space will require re-imaging the affected `wdqs` hosts which will require starting the `data-transfer` process over (not the `data-reload`, fortunately). TASK DETAIL https://phabricator.wikimedia.org/T267927 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: RKemper Cc: jbond, Zbyszko, Gehel, Aklapper, dcausse, Invadibot, MPhamWMF, maantietaja, CBogen, Akuckartz, Nandana, Namenlos314, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T267927: Reload wikidata journal from fresh dumps
RKemper updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T267927 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: RKemper Cc: jbond, Zbyszko, Gehel, Aklapper, dcausse, Invadibot, MPhamWMF, maantietaja, CBogen, Akuckartz, Nandana, Namenlos314, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T267927: Reload wikidata journal from fresh dumps
RKemper added a comment.
`sudo cookbook sre.wdqs.data-reload wdqs1009.eqiad.wmnet
--reuse-downloaded-dump --reload-data wikidata --skolemize --reason 'T267927:
Reload wikidata jnl from fresh dumps' --task-id T267927 ` is failing with:
- OUTPUT of 'test -f /srv/wdq...test-all.ttl.bz2' -
- OUTPUT of 'test -f /srv/wdq...-lexemes.ttl.bz2' -
- OUTPUT of 'dump_size=`du /s...avail+$db_size))' -
- OUTPUT of 'rm -rf /srv/wdqs...d -- --skolemize' -
#logback.classic pattern: %d{HH:mm:ss.SSS} [%thread] %-5level %logger{36} -
%msg%n
22:46:13.025 [main] ERROR org.wikidata.query.rdf.tool.Munge - Fatal error
munging RDF
org.openrdf.rio.RDFParseException: Expected an RDF value here, found '{'
[line 2]
at
org.openrdf.rio.helpers.RDFParserHelper.reportFatalError(RDFParserHelper.java:440)
at
org.openrdf.rio.helpers.RDFParserBase.reportFatalError(RDFParserBase.java:685)
at
org.openrdf.rio.turtle.TurtleParser.reportFatalError(TurtleParser.java:1405)
at org.openrdf.rio.turtle.TurtleParser.parseValue(TurtleParser.java:662)
at
org.openrdf.rio.turtle.TurtleParser.parsePredicate(TurtleParser.java:505)
at
org.openrdf.rio.turtle.TurtleParser.parsePredicateObjectList(TurtleParser.java:420)
at
org.openrdf.rio.turtle.TurtleParser.parseImplicitBlank(TurtleParser.java:613)
at
org.openrdf.rio.turtle.TurtleParser.parseTriples(TurtleParser.java:394)
at
org.openrdf.rio.turtle.TurtleParser.parseStatement(TurtleParser.java:259)
at org.openrdf.rio.turtle.TurtleParser.parse(TurtleParser.java:214)
at org.wikidata.query.rdf.tool.Munge.run(Munge.java:105)
at org.wikidata.query.rdf.tool.Munge.main(Munge.java:59)
22:46:13.024
[org.wikidata.query.rdf.tool.rdf.AsyncRDFHandler$RDFActionsReplayer] INFO
org.wikidata.query.rdf.tool.Munge - Switching to
/srv/wdqs/munged/wikidump-1.ttl.gz
`sudo cookbook sre.wdqs.data-reload wdqs1010.eqiad.wmnet
--reuse-downloaded-dump --reload-data wikidata --reason 'T267927: Reload
wikidata jnl from fresh dumps' --task-id T267927`is failing with:
ryankemper@cumin1001:~$ sudo cookbook sre.wdqs.data-reload
wdqs1010.eqiad.wmnet --reuse-downloaded-dump --reload-data wikidata --reason
'T267927: Reload wikidata jnl from fresh dumps' --task-id T267927
START - Cookbook sre.wdqs.data-reload
- OUTPUT of 'test -f /srv/wdq...test-all.ttl.bz2' -
PASS
||
100% (1/1) [00:00<00:00, 1.92hosts/s]
FAIL |
| 0% (0/1)
[00:00= 100.0% threshold) for command: 'test -f
/srv/wdq...test-all.ttl.bz2'.
100.0% (1/1) success ratio (>= 100.0% threshold) of nodes successfully
executed all commands.
Detected dump (/srv/wdqs/latest-all.ttl.bz2). Skipping download
- OUTPUT of 'test -f /srv/wdq...-lexemes.ttl.bz2' -
PASS
||
100% (1/1) [00:00<00:00, 2.44hosts/s]
FAIL |
| 0% (0/1)
[00:00= 100.0% threshold) for command: 'test -f
/srv/wdq...-lexemes.ttl.bz2'.
100.0% (1/1) success ratio (>= 100.0% threshold) of nodes successfully
executed all commands.
Detected dump (/srv/wdqs/latest-lexemes.ttl.bz2). Skipping download
checking available disk space
- OUTPUT of 'dump_size=`du /s...avail+$db_size))' -
PASS
||
100% (1/1) [00:00<00:00, 2.45hosts/s]
FAIL |
| 0% (0/1)
[00:00= 100.0% threshold) for command:
'dump_size=`du /s...avail+$db_size))'.
100.0% (1/1) success ratio (>= 100.0% threshold) of nodes successfully
executed all commands.
Running munger for main database and then lexeme
munging /srv/wdqs/munged (skolemizaton: False)
- OUTPUT of 'rm -rf /srv/wdqs.../wdqs/munged -- ' -
#logback.classic pattern: %d{HH:mm:ss.SSS} [%thread] %-5level %logger{36} -
%msg%n
22:50:03.137 [main] ERROR org.wikidata.query.rdf.tool.Munge - Fatal error
munging RDF
org.openrdf.rio.RDFParseException: Expected an RDF value here, found '{'
[line 2]
at
org.openrdf.rio.helpers.RDFParserHelper.reportFatalError(RDFPar
[Wikidata-bugs] [Maniphest] T266470: Expose wdqs1009 to wdqs users and gather feedback
RKemper added a comment. TODO from IRC meeting with bblack/gehel: create a DNS entry (CNAME to dyna.wm.o), another set of entries in backend.yaml map, create another minisite (with the appropriate configuration) TASK DETAIL https://phabricator.wikimedia.org/T266470 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: RKemper Cc: Gehel, Lea_Lacroix_WMDE, dcausse, Aklapper, MPhamWMF, CBogen, Akuckartz, Nandana, Namenlos314, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T281454: Onboard teams with Prometheus-based alerts to AlertManager
RKemper added a subtask: T289077: Migrate wdqs prometheus-based alerts from Icinga to alert-manager. TASK DETAIL https://phabricator.wikimedia.org/T281454 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: RKemper Cc: Jdlrobson, fgiunchedi, Aklapper, Invadibot, MPhamWMF, maantietaja, lmata, CBogen, Akuckartz, Nandana, Namenlos314, Lahi, Gq86, herron, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T280001: Set up puppet configuration for new WCQS cluster
RKemper added a comment. With respect to https://gerrit.wikimedia.org/r/c/operations/puppet/+/713946, we'll need to know if `sdcquery01` is still in use. If it's not and we can "decommission" it (or at least not worry about breaking it), that will let us require `oauth_settings` to be defined here, instead of being an `Optional`: https://github.com/wikimedia/puppet/blob/ae7ad4bc10f65ce260c623a1f4c06835d26f1c52/modules/profile/manifests/query_service/wcqs.pp#L21 (This is because for WCQS we always want `oauth_settings` to be set, so we want to disallow the use of null/undefined) TASK DETAIL https://phabricator.wikimedia.org/T280001 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: RKemper Cc: EBernhardson, Aklapper, Zbyszko, Suran38, Biggs657, Invadibot, Lalamarie69, MPhamWMF, maantietaja, Juan90264, Alter-paule, Beast1978, CBogen, Un1tY, Akuckartz, Hook696, Kent7301, joker88john, CucyNoiD, Nandana, Namenlos314, Gaboe420, Giuliamocci, Cpaulf30, Lahi, Gq86, Af420, Bsandipan, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, Lewizho99, Maathavan, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T280001: Set up puppet configuration for new WCQS cluster
RKemper added a comment. Update: `sdcquery01` has been deleted, so we can proceed with removing the `Optional` type TASK DETAIL https://phabricator.wikimedia.org/T280001 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: RKemper Cc: EBernhardson, Aklapper, Zbyszko, Suran38, Biggs657, Invadibot, Lalamarie69, MPhamWMF, maantietaja, Juan90264, Alter-paule, Beast1978, CBogen, Un1tY, Akuckartz, Hook696, Kent7301, joker88john, CucyNoiD, Nandana, Namenlos314, Gaboe420, Giuliamocci, Cpaulf30, Lahi, Gq86, Af420, Bsandipan, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, Lewizho99, Maathavan, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T280006: Set up the application authentication for WCQS on commons-query.wikimedia.org
RKemper added a comment. Zbyszko, Erik, and I discussed the current status of this effort. We've realized that there's some open questions around how to actually implement the oauth, particularly with stuff like the callback URL given the way our routing logic works currently. Currently, for WDQS (which does not use oauth), users visit the "microsite" for the UI, but their requests end up hitting nginx (which forwards them to blazegraph). For WCQS, there won't be that pattern of a request hitting blazegraph directly. They'll be talking to the microsite, and the user is never going to do an interactive request against the backend server (e.g. nginx->blazegraph). ATS at the edge [*before* the request hits the internal loadbalancer (lvs)] will look at request and decide, based off the directory, whether it should go to the microsite or nginx. There's one special directory that will route to nginx, whereas everything else will go to the microsite. Where this breaks down is with oauth in the picture, this pattern won't work. - One approach would be for all traffic to route to nginx, and then have nginx decide what to forward to the microsite. As a general rule we were trying to avoid this approach though because we want this routing logic handled at the "infra" level, not the app layer. We'll need to figure out an approach here. TASK DETAIL https://phabricator.wikimedia.org/T280006 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: RKemper Cc: RKemper, hnowlan, sdkim, BPirkle, Zache, Aklapper, Gehel, CBogen, ttaylor, Zbyszko, Invadibot, MPhamWMF, maantietaja, Akuckartz, Nandana, Namenlos314, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T280247: Create WCQS UI microsite deployment
RKemper added a comment. @Ladsgroup https://gerrit.wikimedia.org/r/c/operations/puppet/+/719502 Okay the puppet change is officially merged, so the `gui-deploy` stuff should be unblocked now. TASK DETAIL https://phabricator.wikimedia.org/T280247 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: EBernhardson, RKemper Cc: WMDE-leszek, RKemper, Manuel, Ladsgroup, Addshore, Aklapper, Gehel, CBogen, ttaylor, Zbyszko, Suran38, Biggs657, Invadibot, Lalamarie69, MPhamWMF, maantietaja, Juan90264, Alter-paule, Beast1978, Un1tY, Akuckartz, Hook696, Kent7301, joker88john, CucyNoiD, Nandana, Namenlos314, Gaboe420, Giuliamocci, Cpaulf30, Lahi, Gq86, Af420, Bsandipan, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, Lewizho99, Maathavan, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T280247: Create WCQS UI microsite deployment
RKemper added a comment. @Ladsgroup Puppet patch is merged now. I haven't merged the gui-deploy patches yet so those two gui-deploy patches should be good to go whenever now that the puppet changes are in place TASK DETAIL https://phabricator.wikimedia.org/T280247 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: EBernhardson, RKemper Cc: RKemper, Manuel, Ladsgroup, Addshore, Aklapper, Gehel, CBogen, ttaylor, Zbyszko, Suran38, Biggs657, Invadibot, Lalamarie69, MPhamWMF, maantietaja, Juan90264, Alter-paule, Beast1978, Un1tY, Akuckartz, Hook696, Kent7301, joker88john, CucyNoiD, Nandana, Namenlos314, Gaboe420, Giuliamocci, Cpaulf30, Lahi, Gq86, Af420, Bsandipan, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, Lewizho99, Maathavan, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T290545: Wikidata Query Service inaccessible via GUI, only partially accessible via embedded queries
RKemper added a comment. @Lydia_Pintscher Thanks this is definitely related to that work. When I spot-checked query.wikidata.org after merging earlier it looks like the new config hadn't taken effect yet. Reverted the puppet change. Working on getting things back how they should be currently. (Thanks for opening this @Daniel_Mietchen) TASK DETAIL https://phabricator.wikimedia.org/T290545 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: RKemper Cc: Lydia_Pintscher, Ladsgroup, RKemper, Gnoeee, Fuzheado, Aklapper, Daniel_Mietchen, Invadibot, MPhamWMF, maantietaja, CBogen, Akuckartz, Nandana, Namenlos314, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T290545: Wikidata Query Service inaccessible via GUI, only partially accessible via embedded queries
RKemper added a comment. Config change was successfully reverted. We had to clear out the varnish cache with `echo 'https://query.wikidata.org/' | mwscript purgeList.php` per https://wikitech.wikimedia.org/wiki/Kafka_HTTP_purging. Everything should be / is back to normal now. TASK DETAIL https://phabricator.wikimedia.org/T290545 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: RKemper Cc: Lydia_Pintscher, Ladsgroup, RKemper, Gnoeee, Fuzheado, Aklapper, Daniel_Mietchen, Invadibot, MPhamWMF, maantietaja, CBogen, Akuckartz, Nandana, Namenlos314, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T290330: Wikidata Query Service unstable in codfw
RKemper added a comment. We're no longer doing hourly restarts; we should monitor availability over the weekend to make sure that we're capable of handling current load without the crutch of the hourly restarts. We still have the useragent we banned in the last full outage (dailymotion) banned, so I'd think we'll be okay on load, but I'll be checking Grafana over the next couple days. We can/should unban dailymotion early next week - perhaps tuesday - if wdqs has been stable over the weekend and during monday's load TASK DETAIL https://phabricator.wikimedia.org/T290330 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: RKemper Cc: RKemper, Addshore, LSobanski, RhinosF1, Zbyszko, WMDE-leszek, Mholloway, Gehel, Ladsgroup, Aklapper, Bugreporter, Invadibot, MPhamWMF, maantietaja, CBogen, Akuckartz, Nandana, Namenlos314, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T288501: Blazegraph journal too large on wdqs2003
RKemper added a comment. ryankemper@wdqs2003:~$ ls -lah /srv/wdqs total 2.8T drwxrwxr-x 3 blazegraph wikidev4.0K Aug 6 07:26 . drwxr-xr-x 5 root root 4.0K Jun 2 22:59 .. -rw-rw-r-- 1 blazegraph wikidev 54 Aug 6 07:26 aliases.map -rw-rw-r-- 1 blazegraph blazegraph 23G Aug 10 07:41 categories.jnl -rw-r--r-- 1 root root 0 Jun 3 01:25 data_loaded -rw-r--r-- 1 blazegraph blazegraph 83K Aug 6 07:26 dcatap-20210806.rdf drwxrwxr-x 2 blazegraph wikidev 68K Aug 10 07:41 dumps -rw-rw-r-- 1 blazegraph blazegraph 2.8T Aug 9 19:25 wikidata.jnl ryankemper@wdqs2003:~$ df -h Filesystem Size Used Avail Use% Mounted on udev 63G 0 63G 0% /dev tmpfs13G 1.3G 12G 11% /run /dev/md0 73G 15G 55G 22% / tmpfs63G 4.0K 63G 1% /dev/shm tmpfs 5.0M 0 5.0M 0% /run/lock tmpfs63G 0 63G 0% /sys/fs/cgroup tmpfs 1.0G 20K 1.0G 1% /var/lib/nginx /dev/md22.9T 2.8T 120K 100% /srv tmpfs13G 0 13G 0% /run/user/24320 `wikidata.jnl` is over-inflated. here's a healthy host to compare to: ryankemper@wdqs2002:~$ ls -lah /srv/wdqs total 998G drwxrwxr-x 3 blazegraph wikidev4.0K Aug 6 07:38 . drwxr-xr-x 5 root root 4.0K Jun 4 04:46 .. -rw-rw-r-- 1 blazegraph wikidev 54 Aug 6 07:38 aliases.map -rw-rw-r-- 1 blazegraph blazegraph 23G Aug 10 07:56 categories.jnl -rw-r--r-- 1 root root 0 Jun 4 06:46 data_loaded -rw-r--r-- 1 blazegraph blazegraph 83K Aug 6 07:38 dcatap-20210806.rdf drwxrwxr-x 2 blazegraph wikidev 64K Aug 10 07:56 dumps -rw-rw-r-- 1 blazegraph blazegraph 975G Aug 10 17:07 wikidata.jnl ryankemper@wdqs2002:~$ df -h Filesystem Size Used Avail Use% Mounted on udev 48G 0 48G 0% /dev tmpfs 9.5G 952M 8.6G 10% /run /dev/md0 73G 7.3G 62G 11% / tmpfs48G 4.0K 48G 1% /dev/shm tmpfs 5.0M 0 5.0M 0% /run/lock tmpfs48G 0 48G 0% /sys/fs/cgroup tmpfs 1.0G 20K 1.0G 1% /var/lib/nginx /dev/md22.9T 999G 1.8T 36% /srv tmpfs 9.5G 0 9.5G 0% /run/user/24320 --- Solution is pretty simple, nuke `wikidata.jnl` (w/ the host de-pooled) and then kick off a `data-transfer` from `wdqs2002` (external) or `wdqs2005` (internal) TASK DETAIL https://phabricator.wikimedia.org/T288501 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: RKemper Cc: Zbyszko, RKemper, ema, Aklapper, Invadibot, MPhamWMF, maantietaja, CBogen, Akuckartz, Nandana, Namenlos314, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
