[Wikidata-bugs] [Maniphest] T307869: Request for new search profile for Wikidata that boosts Items for languages
dcausse added a comment. No new profiles should be created for other wikibase installation as most of the wikidata specific options are managed in wmf specific config, not Wikibase nor CirrusSearch so the new Lexeme creation page should behave exactly as before. All the fixes we had to make in CirrusSearch should not impact anything except if other Wikibase installations had tuned such broken settings (but I doubt since they were totally broken and ineffective) The bugfix that might affect Wikibase installations relying on CirrusSearch is: - Fixed the handling of the configuration variable `wgWBCSStatementBoost` which was ignored. @Lea_WMDE @Evelien_WMDE do you have a link to such problems with Elastic in wbcloud? TASK DETAIL https://phabricator.wikimedia.org/T307869 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Lucas_Werkmeister_WMDE, dcausse Cc: Lea_WMDE, Evelien_WMDE, ItamarWMDE, dcausse, Lucas_Werkmeister_WMDE, MPhamWMF, Aklapper, Lydia_Pintscher, Hellket777, Astuthiodit_1, Nikospappas1312, 786, Biggs657, karapayneWMDE, Invadibot, Universal_Omega, maantietaja, Juan90264, Alter-paule, Beast1978, CBogen, Un1tY, Akuckartz, Hook696, Kent7301, joker88john, CucyNoiD, Nandana, Gaboe420, Giuliamocci, Cpaulf30, Lahi, Gq86, Af420, Bsandipan, GoranSMilovanovic, Mahir256, QZanden, EBjune, LawExplorer, Lewizho99, Maathavan, _jensen, rosalieper, Bodhisattwa, Neuronton, Scott_WUaS, Wikidata-bugs, aude, Gryllida, Mbch331 ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T307869: Request for new search profile for Wikidata that boosts Items for languages
dcausse added a comment. There is yet another problem (see patch above that should fix it). I'm sorry that deploying this profile is such a pain, it demonstrates a clear problem in the way we (the search team) deploy such features/profiles and I filed T311528 <https://phabricator.wikimedia.org/T311528> to discuss and hopefully improve the situation. TASK DETAIL https://phabricator.wikimedia.org/T307869 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Lucas_Werkmeister_WMDE, dcausse Cc: ItamarWMDE, dcausse, Lucas_Werkmeister_WMDE, MPhamWMF, Aklapper, Lydia_Pintscher, Hellket777, Astuthiodit_1, Nikospappas1312, 786, Biggs657, karapayneWMDE, Invadibot, Universal_Omega, maantietaja, Juan90264, Alter-paule, Beast1978, CBogen, Un1tY, Akuckartz, Hook696, Kent7301, joker88john, CucyNoiD, Nandana, Gaboe420, Giuliamocci, Cpaulf30, Lahi, Gq86, Af420, Bsandipan, GoranSMilovanovic, Mahir256, QZanden, EBjune, LawExplorer, Lewizho99, Maathavan, _jensen, rosalieper, Bodhisattwa, Neuronton, Scott_WUaS, Wikidata-bugs, aude, Gryllida, Mbch331 ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T307869: Request for new search profile for Wikidata that boosts Items for languages
dcausse added a comment. Sorry about that, there was yet another issue in the WikibaseCirrusSearch Hook that caused the config to be ignored and cause the language selector profile context to simply use exactly the same settings as the classic entity completion search. There was also a typo in mw-config fixed in one the attached patch. TASK DETAIL https://phabricator.wikimedia.org/T307869 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Lucas_Werkmeister_WMDE, dcausse Cc: ItamarWMDE, dcausse, Lucas_Werkmeister_WMDE, MPhamWMF, Aklapper, Lydia_Pintscher, Astuthiodit_1, Nikospappas1312, 786, Biggs657, karapayneWMDE, Invadibot, Universal_Omega, maantietaja, Juan90264, Alter-paule, Beast1978, CBogen, Un1tY, Akuckartz, Hook696, Kent7301, joker88john, CucyNoiD, Nandana, Gaboe420, Giuliamocci, Cpaulf30, Lahi, Gq86, Af420, Bsandipan, GoranSMilovanovic, Mahir256, QZanden, EBjune, LawExplorer, Lewizho99, Maathavan, _jensen, rosalieper, Bodhisattwa, Neuronton, Scott_WUaS, Wikidata-bugs, aude, Gryllida, Mbch331 ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T307869: Request for new search profile for Wikidata that boosts Items for languages
dcausse added a comment. The above patch should fix the issue, I forgot that profile repositories must have have unique names, sorry about that! TASK DETAIL https://phabricator.wikimedia.org/T307869 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Lucas_Werkmeister_WMDE, dcausse Cc: ItamarWMDE, dcausse, Lucas_Werkmeister_WMDE, MPhamWMF, Aklapper, Lydia_Pintscher, Astuthiodit_1, Nikospappas1312, 786, Biggs657, karapayneWMDE, Invadibot, Universal_Omega, maantietaja, Juan90264, Alter-paule, Beast1978, CBogen, Un1tY, Akuckartz, Hook696, Kent7301, joker88john, CucyNoiD, Nandana, Gaboe420, Giuliamocci, Cpaulf30, Lahi, Gq86, Af420, Bsandipan, GoranSMilovanovic, Mahir256, QZanden, EBjune, LawExplorer, Lewizho99, Maathavan, _jensen, rosalieper, Bodhisattwa, Neuronton, Scott_WUaS, Wikidata-bugs, aude, Gryllida, Mbch331 ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T307869: Request for new search profile for Wikidata that boosts Items for languages
dcausse added a comment. The patches above add few placeholder to allow tuning a custom profile meant to be use by the language selector on Special:NewLexeme: - https://gerrit.wikimedia.org/r/c/mediawiki/extensions/WikibaseLexemeCirrusSearch/+/801791/ adds a new `profile context` named `lexeme_new_lexeme_prefix` and few config options to allow to configure it - https://gerrit.wikimedia.org/r/c/operations/mediawiki-config/+/801793/ is an example patch configuring this `lexeme_new_lexeme_prefix` profile context (I took the first approach mentioned in the description) - https://gerrit.wikimedia.org/r/c/mediawiki/extensions/WikibaseCirrusSearch/+/801790/ is still WIP but adds the possibility to switch the `profile context` from the EntitySearchElastic constructor The open question is how to pass a new URI param added to `wbsearchentities` meant to switch between profiles back to `EntitySearchElastic`. TASK DETAIL https://phabricator.wikimedia.org/T307869 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: ItamarWMDE, dcausse, Lucas_Werkmeister_WMDE, MPhamWMF, Aklapper, Lydia_Pintscher, Fernandobacasegua34, Astuthiodit_1, Nikospappas1312, 786, Suran38, Biggs657, karapayneWMDE, Invadibot, Lalamarie69, maantietaja, Juan90264, Alter-paule, Beast1978, CBogen, Un1tY, Akuckartz, Hook696, Kent7301, joker88john, CucyNoiD, Nandana, Gaboe420, Giuliamocci, Cpaulf30, Lahi, Gq86, Af420, Bsandipan, GoranSMilovanovic, Mahir256, QZanden, EBjune, LawExplorer, Lewizho99, Maathavan, _jensen, rosalieper, Bodhisattwa, Neuronton, Scott_WUaS, Wikidata-bugs, aude, Gryllida, Mbch331 ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T268864: WikibaseCirrusSearch uses Elastica's Match class
dcausse changed the status of subtask T271777: Bump rufin/elastica (and related libraries) to versions that support PHP 8.0 from Stalled to Open. TASK DETAIL https://phabricator.wikimedia.org/T268864 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: Aklapper, Reedy, Astuthiodit_1, karapayneWMDE, Invadibot, MPhamWMF, maantietaja, Wilmanbeno, CBogen, ItamarWMDE, Akuckartz, DannyS712, Nandana, Lahi, Gq86, GoranSMilovanovic, QZanden, EBjune, LawExplorer, _jensen, rosalieper, Scott_WUaS, Wikidata-bugs, aude, jayvdb, Nikerabbit, Jdforrester-WMF, Addshore, MaxSem, Mbch331, jeremyb ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T268865: WikibaseLexemeCirrusSearch uses Elastica's Match class
dcausse changed the status of subtask T271777: Bump rufin/elastica (and related libraries) to versions that support PHP 8.0 from Stalled to Open. TASK DETAIL https://phabricator.wikimedia.org/T268865 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Reedy, dcausse Cc: Reedy, Astuthiodit_1, karapayneWMDE, Invadibot, MPhamWMF, maantietaja, Wilmanbeno, CBogen, ItamarWMDE, Akuckartz, DannyS712, Nandana, Lahi, Gq86, GoranSMilovanovic, Mahir256, QZanden, EBjune, LawExplorer, _jensen, rosalieper, Bodhisattwa, Scott_WUaS, Wikidata-bugs, aude, jayvdb, Nikerabbit, Jdforrester-WMF, Addshore, MaxSem, Mbch331, jeremyb ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T307635: Query service results are missing some variables on some servers
dcausse added a comment. This is extremely weird and I suspect a serious blazegraph bug that causes this. I could not reproduce the problem at the moment running the python script provided but it might certainly happen again in the future. I'm not sure how to proceed here but perhaps capturing the full blazegraph response when it occurs might help? TASK DETAIL https://phabricator.wikimedia.org/T307635 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: dcausse, LucasWerkmeister, Nikki, Aklapper, Astuthiodit_1, karapayneWMDE, Invadibot, MPhamWMF, maantietaja, CBogen, ItamarWMDE, Akuckartz, Nandana, Namenlos314, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T306054: Upgrade deployment-wdqs01 host to Buster
dcausse added a project: Discovery-Search (Current work). TASK DETAIL https://phabricator.wikimedia.org/T306054 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: dcausse, Lucas_Werkmeister_WMDE, Mathew.onipe, Aklapper, Majavah, Peachey88, Jdforrester-WMF, Astuthiodit_1, karapayneWMDE, Invadibot, MPhamWMF, maantietaja, CBogen, ItamarWMDE, Akuckartz, CptViraj, DannyS712, Nandana, Namenlos314, Lahi, Gq86, Bsandipan, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Addshore, Mbch331, Jay8g, Krenair ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T305983: query.wikidata.org/bigdata/ldf - Language string should include language tag
dcausse moved this task from Incoming to For Later on the Wikidata-Query-Service board. dcausse triaged this task as "Medium" priority. TASK DETAIL https://phabricator.wikimedia.org/T305983 WORKBOARD https://phabricator.wikimedia.org/project/board/891/ EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: Aklapper, OMrkvonO, Astuthiodit_1, karapayneWMDE, Invadibot, MPhamWMF, maantietaja, CBogen, ItamarWMDE, Akuckartz, Nandana, Namenlos314, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T306054: Upgrade deployment-wdqs01 host to Buster
dcausse added a comment. I can confirm, this host is not used. TASK DETAIL https://phabricator.wikimedia.org/T306054 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: dcausse, Lucas_Werkmeister_WMDE, Mathew.onipe, Aklapper, Majavah, Peachey88, Jdforrester-WMF, Astuthiodit_1, karapayneWMDE, Invadibot, MPhamWMF, maantietaja, CBogen, ItamarWMDE, Akuckartz, CptViraj, DannyS712, Nandana, Namenlos314, Lahi, Gq86, Bsandipan, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Addshore, Mbch331, Jay8g, Krenair ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T305818: Perform a data transfer to wdqs2004 & wdqs1004 to reclaim burnt allocators
dcausse created this task. dcausse added a project: Wikidata-Query-Service. Restricted Application added a subscriber: Aklapper. TASK DESCRIPTION wdqs2004 & wdqs1004 lost their free allocators too quickly (known issue that pops up //randomly//). We should do a data transfer from a sane source to reclaim these before other machines are affected by the same problem. AC: - wdqs2004 & wdqs1004 have their number of free allocators around 232k like other "sane" nodes. TASK DETAIL https://phabricator.wikimedia.org/T305818 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: dcausse, Aklapper, MPhamWMF, CBogen, Namenlos314, Gq86, Lucas_Werkmeister_WMDE, EBjune, merbst, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T302189: Regularly purge orphaned sitelink, value and reference nodes
dcausse added a comment. Reason is that this data //may// be referenced by other items and thus cannot be deleted blindly without asking blazegraph: //"is this data used by another item?"// which would be too costly to ask for every edit. Another approach is to reload blazegraph from the dumps at regular intervals (TBD: once, twice or four times a year). TASK DETAIL https://phabricator.wikimedia.org/T302189 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: MPhamWMF, Bugreporter, dcausse, Aklapper, Yair_rand, Astuthiodit_1, karapayneWMDE, Invadibot, maantietaja, CBogen, ItamarWMDE, Akuckartz, Nandana, Namenlos314, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T293862: Investigate using jvmquake to limit the time a JVM is unusable due to GC overhead
dcausse added a comment. Actually wdqs2007, wdqs2004 and wdqs2003 also triggered jvmquake, GC activity increased and wdqs2007 & wdqs2003 were unresponsive for a couple minutes. For wdqs2004 there are no visible blips in the various graph. I guess we should relax the settings a bit more. TASK DETAIL https://phabricator.wikimedia.org/T293862 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: bking, Aklapper, dcausse, Fernandobacasegua34, Astuthiodit_1, 786, Suran38, Biggs657, karapayneWMDE, Invadibot, Lalamarie69, MPhamWMF, maantietaja, Juan90264, Alter-paule, Beast1978, CBogen, ItamarWMDE, Un1tY, Akuckartz, Hook696, Kent7301, joker88john, CucyNoiD, Nandana, Namenlos314, Gaboe420, Giuliamocci, Cpaulf30, Lahi, Gq86, Af420, Bsandipan, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, Lewizho99, Maathavan, _jensen, rosalieper, Neuronton, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T293862: Investigate using jvmquake to limit the time a JVM is unusable due to GC overhead
dcausse added a comment. With the settings we properly detected wdqs1006 going down for 30minutes at `2022-04-22T12:30:00` (this 2minutes after the first blip in the graph). Unfortunately there was a false positive wdqs1012 at `2022-04-22T10:00:00` as this machine was unavailable from 2 minutes only. Unsure if it's still too sensitive or if we can accept having a couple false positives. TASK DETAIL https://phabricator.wikimedia.org/T293862 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: bking, Aklapper, dcausse, Fernandobacasegua34, Astuthiodit_1, 786, Suran38, Biggs657, karapayneWMDE, Invadibot, Lalamarie69, MPhamWMF, maantietaja, Juan90264, Alter-paule, Beast1978, CBogen, ItamarWMDE, Un1tY, Akuckartz, Hook696, Kent7301, joker88john, CucyNoiD, Nandana, Namenlos314, Gaboe420, Giuliamocci, Cpaulf30, Lahi, Gq86, Af420, Bsandipan, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, Lewizho99, Maathavan, _jensen, rosalieper, Neuronton, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T304365: Add property predicates to WCQS
dcausse moved this task from Incoming to Scaling on the Wikidata-Query-Service board. dcausse added a comment. I agree that federation is adding a lot of //boiler plate// and inspecting the shape of the IRIs is very fragile. But merging multiple graphs into the same store for ease of use is going against the recent discussions we had around the future of the WDQS architecture, it is also a bit more complex than it seems within the current data flows. Nevertheless I think the concern you raise is very valid and should be taken into account while we figure out if splitting the graph and building on top SPARQL federation is something we have to pursue and/or if some sub-graph are very central that they'd better be replicated to all sub-graphs. TASK DETAIL https://phabricator.wikimedia.org/T304365 WORKBOARD https://phabricator.wikimedia.org/project/board/891/ EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: dcausse, WikidataFacts, Aklapper, Dominicbm, Astuthiodit_1, karapayneWMDE, Invadibot, GFontenelle_WMF, MPhamWMF, maantietaja, FRomeo_WMF, CBogen, ItamarWMDE, Nintendofan885, Akuckartz, Nandana, JKSTNK, Namenlos314, Lahi, Gq86, E1presidente, Ramsey-WMF, Cparle, SandraF_WMF, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, Tramullas, Acer, merbst, LawExplorer, Salgo60, Silverfish, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, Susannaanas, Jane023, jkroll, Wikidata-bugs, Jdouglas, Base, matthiasmullie, aude, Tobias1984, Manybubbles, Ricordisamoa, Wesalius, Lydia_Pintscher, Raymond, Steinsplitter, Mbch331 ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T301147: The WDQS streaming updater went unstable for several hours (2022-02-06T23:00:00 - 2022-02-07T06:20:00)
dcausse updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T301147 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: elukey, akosiaris, Gehel, RKemper, bking, toan, Addshore, JMeybohm, Michael, Aklapper, dcausse, Astuthiodit_1, karapayneWMDE, Invadibot, MPhamWMF, maantietaja, CBogen, ItamarWMDE, Akuckartz, Nandana, Namenlos314, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T301147: The WDQS streaming updater went unstable for several hours (2022-02-06T23:00:00 - 2022-02-07T06:20:00)
dcausse added a comment. Thanks for the quick answer! (response inline) In T301147#7821582 <https://phabricator.wikimedia.org/T301147#7821582>, @JMeybohm wrote: >> - If the above is not possible could we mitigate this problem by over-allocating resources (increase the number of replicas) to the deployment to increase the chances of proper recovery if this situation happens again? > > If that makes sense from your POV you could do that ofc. I can't speak on how problematic this situation was compared to the potential waste of resources another pod means. But if the current workload is already maxing out the capacity of the 6 replicas you have, maybe bumping that to 7 might be smart anyways to account for peaks? The additional PODs won't be used as a flink job does not automatically scale so it would be a pure waste of resources (2.5G of reserved mem per additional POD). It would help I guess to improve redundancy in this scenario only if k8s assigns every POD to a distinct machine, in which case even with a single machine misbehaving flink would have enough redundancy to allocate the job to the spare POD. If k8s does do allocation randomly or that there are not enough k8s worker nodes (1 spare POD in our case would mean spreading the PODs over 8 different machines) then it's probably not worth the waste of resources. > In T301147#7821422 <https://phabricator.wikimedia.org/T301147#7821422>, @dcausse wrote: > >> @JMeybohm do you see any additional action items that would improve the resilience of k8s in such scenario? > > Unfortunately we don't have any data on what went wrong on the node. I think T277876 <https://phabricator.wikimedia.org/T277876> would be a step in the right direction but I also doubt it would have fully prevented this issue (ultimately I can't say). Thanks, I'm adding it to the ticket description as a possible improvement. TASK DETAIL https://phabricator.wikimedia.org/T301147 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: elukey, akosiaris, Gehel, RKemper, bking, toan, Addshore, JMeybohm, Michael, Aklapper, dcausse, Astuthiodit_1, karapayneWMDE, Invadibot, MPhamWMF, maantietaja, CBogen, ItamarWMDE, Akuckartz, Nandana, Namenlos314, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T301147: The WDQS streaming updater went unstable for several hours (2022-02-06T23:00:00 - 2022-02-07T06:20:00)
dcausse moved this task from Ready for Development to Needs review on the Discovery-Search (Current work) board. dcausse added a comment. Tentatively moving this ticket to //needs review// as I'm not sure sure we can do much more from the search team perspective. I think the last point to discuss was to investigate the reasons why a single k8s node that misbehaves could make a deployment unstable. @JMeybohm do you see any additional action items that would improve the resilience of k8s in such scenario? TASK DETAIL https://phabricator.wikimedia.org/T301147 WORKBOARD https://phabricator.wikimedia.org/project/board/1227/ EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: elukey, akosiaris, Gehel, RKemper, bking, toan, Addshore, JMeybohm, Michael, Aklapper, dcausse, Astuthiodit_1, karapayneWMDE, Invadibot, MPhamWMF, maantietaja, CBogen, ItamarWMDE, Akuckartz, Nandana, Namenlos314, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T305068: Alert when flink does not have the number of expected task managers
dcausse claimed this task. dcausse moved this task from Incoming to Needs review on the Discovery-Search (Current work) board. TASK DETAIL https://phabricator.wikimedia.org/T305068 WORKBOARD https://phabricator.wikimedia.org/project/board/1227/ EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: Aklapper, Michael, JMeybohm, Addshore, toan, bking, RKemper, Gehel, akosiaris, elukey, dcausse, Astuthiodit_1, karapayneWMDE, Invadibot, MPhamWMF, maantietaja, CBogen, ItamarWMDE, Akuckartz, Nandana, Namenlos314, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T209859: Wikidata autocomplete (wbsearchentities) results with score <= 0
dcausse removed EJoseph as the assignee of this task. dcausse added a subscriber: EJoseph. TASK DETAIL https://phabricator.wikimedia.org/T209859 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: EJoseph, Liuxinyu970226, dcausse, Smalyshev, EBernhardson, Aklapper, Astuthiodit_1, karapayneWMDE, Invadibot, MPhamWMF, maantietaja, CBogen, ItamarWMDE, Akuckartz, Nandana, Lahi, Gq86, GoranSMilovanovic, QZanden, EBjune, LawExplorer, _jensen, rosalieper, Scott_WUaS, Wikidata-bugs, aude, Mbch331 ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T209859: Wikidata autocomplete (wbsearchentities) results with score <= 0
dcausse moved this task from Wikibase Search to needs triage on the Discovery-Search board. dcausse assigned this task to EJoseph. TASK DETAIL https://phabricator.wikimedia.org/T209859 WORKBOARD https://phabricator.wikimedia.org/project/board/1849/ EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: EJoseph, dcausse Cc: Liuxinyu970226, dcausse, Smalyshev, EBernhardson, Aklapper, Astuthiodit_1, karapayneWMDE, Invadibot, MPhamWMF, maantietaja, CBogen, ItamarWMDE, Akuckartz, Nandana, Lahi, Gq86, GoranSMilovanovic, QZanden, EBjune, LawExplorer, _jensen, rosalieper, Scott_WUaS, Wikidata-bugs, aude, Mbch331 ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T238751: Only generate maxlag from pooled query service servers.
dcausse added a project: Discovery-Search. TASK DETAIL https://phabricator.wikimedia.org/T238751 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Joe, dcausse Cc: karapayneWMDE, Lucas_Werkmeister_WMDE, Ladsgroup, Gehel, Jheald, Joe, Addshore, Aklapper, Fernandobacasegua34, Astuthiodit_1, 786, Suran38, Biggs657, Invadibot, Lalamarie69, MPhamWMF, LSobanski, maantietaja, Juan90264, Alter-paule, Beast1978, CBogen, ItamarWMDE, Un1tY, Akuckartz, Hook696, Kent7301, joker88john, CucyNoiD, Nandana, jijiki, Klaas_Z4us_V, Gaboe420, Giuliamocci, Cpaulf30, Lahi, Gq86, Af420, Bsandipan, GoranSMilovanovic, QZanden, EBjune, LawExplorer, Lewizho99, Maathavan, elukey, _jensen, rosalieper, Neuronton, Scott_WUaS, Wikidata-bugs, aude, Mbch331, Jay8g ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T301147: The WDQS streaming updater went unstable for several hours (2022-02-06T23:00:00 - 2022-02-07T06:20:00)
dcausse updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T301147 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: elukey, akosiaris, Gehel, RKemper, bking, toan, Addshore, JMeybohm, Michael, Aklapper, dcausse, Astuthiodit_1, karapayneWMDE, Invadibot, MPhamWMF, maantietaja, CBogen, ItamarWMDE, Akuckartz, Nandana, Namenlos314, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T301147: The WDQS streaming updater went unstable for several hours (2022-02-06T23:00:00 - 2022-02-07T06:20:00)
dcausse updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T301147 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: elukey, akosiaris, Gehel, RKemper, bking, toan, Addshore, JMeybohm, Michael, Aklapper, dcausse, Astuthiodit_1, karapayneWMDE, Invadibot, MPhamWMF, maantietaja, CBogen, ItamarWMDE, Akuckartz, Nandana, Namenlos314, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T305068: Alert when flink does not have the number of expected task managers
dcausse created this task. dcausse added projects: Wikidata-Query-Service, Wikidata, Discovery-Search (Current work). TASK DESCRIPTION As a maintainer of a flink session cluster I want to be alerted when the number of taskmanagers is not what the deployment expects so that I can react quickly. It may happen that k8s is preferring to reboot containers on a broken k8s node rather than migrate the pod to a new pod (see parent ticket), for k8s this deployment may appear to be working properly but for flink the resources it expects are not available and the job it's supposed to run will remain in the SCHEDULED state. AC: - alert when the number of task managers is below a certain threshold TASK DETAIL https://phabricator.wikimedia.org/T305068 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: Aklapper, Michael, JMeybohm, Addshore, toan, bking, RKemper, Gehel, akosiaris, elukey, dcausse, Astuthiodit_1, karapayneWMDE, Invadibot, MPhamWMF, maantietaja, CBogen, ItamarWMDE, Akuckartz, Nandana, Namenlos314, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T294076: Blazegraph and MariaDB contain different sitelinks at Wikidata
dcausse moved this task from Waiting to Needs Reporting on the Discovery-Search (Current work) board. dcausse added a comment. The reconciliation process is running and should auto-correct missed updates couple hours after they're performed. I also fixed the inconsistencies listed here and other related tickets. Please let me know if you still find errors. TASK DETAIL https://phabricator.wikimedia.org/T294076 WORKBOARD https://phabricator.wikimedia.org/project/board/1227/ EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: Lucas_Werkmeister_WMDE, Sjoerddebruin, dcausse, William_Avery, RShigapov, Aklapper, Astuthiodit_1, karapayneWMDE, Invadibot, MPhamWMF, maantietaja, CBogen, ItamarWMDE, Akuckartz, Nandana, Namenlos314, Lahi, Gq86, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T302494: The WDQS Streaming Updater should use S3 to access thanos-swift instead of the native swift protocol
dcausse updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T302494 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: RKemper, dcausse Cc: bking, Aklapper, dcausse, Fernandobacasegua34, Astuthiodit_1, 786, Suran38, Biggs657, karapayneWMDE, Invadibot, Lalamarie69, MPhamWMF, maantietaja, Juan90264, Alter-paule, Beast1978, CBogen, ItamarWMDE, Un1tY, Akuckartz, Hook696, Kent7301, joker88john, CucyNoiD, Nandana, Namenlos314, Gaboe420, Giuliamocci, Cpaulf30, Lahi, Gq86, Af420, Bsandipan, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, Lewizho99, Maathavan, _jensen, rosalieper, Neuronton, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T302494: The WDQS Streaming Updater should use S3 to access thanos-swift instead of the native swift protocol
dcausse moved this task from In Progress to Needs Reporting on the Discovery-Search (Current work) board. dcausse added a comment. Moved remaining work in T304914 <https://phabricator.wikimedia.org/T304914>. TASK DETAIL https://phabricator.wikimedia.org/T302494 WORKBOARD https://phabricator.wikimedia.org/project/board/1227/ EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: RKemper, dcausse Cc: bking, Aklapper, dcausse, Fernandobacasegua34, Astuthiodit_1, 786, Suran38, Biggs657, karapayneWMDE, Invadibot, Lalamarie69, MPhamWMF, maantietaja, Juan90264, Alter-paule, Beast1978, CBogen, ItamarWMDE, Un1tY, Akuckartz, Hook696, Kent7301, joker88john, CucyNoiD, Nandana, Namenlos314, Gaboe420, Giuliamocci, Cpaulf30, Lahi, Gq86, Af420, Bsandipan, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, Lewizho99, Maathavan, _jensen, rosalieper, Neuronton, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T302494: The WDQS Streaming Updater should use S3 to access thanos-swift instead of the native swift protocol
dcausse updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T302494 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: RKemper, dcausse Cc: bking, Aklapper, dcausse, Fernandobacasegua34, Astuthiodit_1, 786, Suran38, Biggs657, karapayneWMDE, Invadibot, Lalamarie69, MPhamWMF, maantietaja, Juan90264, Alter-paule, Beast1978, CBogen, ItamarWMDE, Un1tY, Akuckartz, Hook696, Kent7301, joker88john, CucyNoiD, Nandana, Namenlos314, Gaboe420, Giuliamocci, Cpaulf30, Lahi, Gq86, Af420, Bsandipan, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, Lewizho99, Maathavan, _jensen, rosalieper, Neuronton, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T304914: Remove the presto client for swift from the flink image
dcausse created this task. dcausse added a project: Wikidata-Query-Service. Restricted Application added a subscriber: Aklapper. TASK DESCRIPTION As a maintainer of a flink session cluster I want to stop using the presto client for swift present in the flink image so that I can migrate to newer version of flink since it was removed. This is a followup of T302494 <https://phabricator.wikimedia.org/T302494> where we dropped this dependency from the jobs running in the flink session cluster. This task is about dropping this swift client from the image. Existing flink session clusters rely on this swift client to store their H/A related data (e.g. job jars). This means we must migrate existing clusters to using s3 as a simple drop-in replacement is unlikely to work. Suggested migration procedure: - For codfw - route wdqs & wcqs to eqiad only - adapt the wikidata maxlag to poll eqiad only - stop (with a savepoint) all the jobs (WDQS & WCQS) running on the codfw k8s wikikube cluster - undeploy all the k8s deployments under the `rdf-streaming-updater` namespace (dropping all flink generated configmaps might be necessary by e.g. recreating the k8s namespace) - delete the flink_ha_storage folder on the corresponding s3 bucket (`rdf-streaming-updater-codfw`) - drop presto-swift from https://gerrit.wikimedia.org/g/wikidata/query/flink-rdf-streaming-updater and create a new image - adapt the patch generated by PipelineLib when merging the patch above and remove all mentions to swift from `deployment-charts` (possibly adapting existing patch: https://gerrit.wikimedia.org/r/c/operations/deployment-charts/+/766123) - deploy the chart to the rdf-streaming-updater namespace in codfw (which should be empty) - deploy the flink jobs (WCQS & WDQS) from their corresponding savepoints - repool codfw & resume polling codfw for wikidata maxlag calculation - For eqiad (do all the above replacing eqiad with codfw and vice versa) Note that most of this procedure can be tested against the staging cluster (omitting the parts about the routing live traffic and wikidata maxlag) AC: - none of the flink session clusters are using the presto swift client TASK DETAIL https://phabricator.wikimedia.org/T304914 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: dcausse, Aklapper, MPhamWMF, CBogen, Namenlos314, Gq86, Lucas_Werkmeister_WMDE, EBjune, merbst, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T242453: Deadlock in blazegraph blocking all queries and updates
dcausse updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T242453 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: RKemper, bking, RLazarus, Legoktm, Gehel, William_Avery, CDanis, Addshore, dcausse, Aklapper, Astuthiodit_1, karapayneWMDE, Invadibot, MPhamWMF, maantietaja, CBogen, ItamarWMDE, Akuckartz, Nandana, Namenlos314, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T242453: Deadlock in blazegraph blocking all queries and updates
dcausse reopened this task as "Open". dcausse added a comment. re-opening, seems to happen more frequently TASK DETAIL https://phabricator.wikimedia.org/T242453 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: RLazarus, Legoktm, Gehel, William_Avery, CDanis, Addshore, dcausse, Aklapper, Astuthiodit_1, karapayneWMDE, Invadibot, MPhamWMF, maantietaja, CBogen, ItamarWMDE, Akuckartz, Nandana, Namenlos314, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T303256: WDQS servers should use skolem for wikibaseSomeValueMode
dcausse moved this task from Needs review to Needs Reporting on the Discovery-Search (Current work) board. dcausse added a comment. `wikibase:isSomeValue` is functioning properly again. TASK DETAIL https://phabricator.wikimedia.org/T303256 WORKBOARD https://phabricator.wikimedia.org/project/board/1227/ EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: seav, Aklapper, dcausse, Astuthiodit_1, karapayneWMDE, Invadibot, MPhamWMF, maantietaja, CBogen, Akuckartz, Nandana, Namenlos314, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T302396: Investigate EOFException when performing the first checkpoint after restoring from a savepoint
dcausse added a comment. S3 <https://phabricator.wikimedia.org/S3> is confirmed to have fixed this issue, all jobs are now running 0.3.104 of the streaming-updater and are using the s3 client to persist their durable state. TASK DETAIL https://phabricator.wikimedia.org/T302396 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: Gehel, EBernhardson, Aklapper, dcausse, Astuthiodit_1, karapayneWMDE, Invadibot, MPhamWMF, maantietaja, CBogen, Akuckartz, Nandana, Namenlos314, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T302494: The WDQS Streaming Updater should use S3 to access thanos-swift instead of the native swift protocol
dcausse updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T302494 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: RKemper, dcausse Cc: bking, Aklapper, dcausse, Fernandobacasegua34, Astuthiodit_1, 786, Suran38, Biggs657, karapayneWMDE, Invadibot, Lalamarie69, MPhamWMF, maantietaja, Juan90264, Alter-paule, Beast1978, CBogen, Un1tY, Akuckartz, Hook696, Kent7301, joker88john, CucyNoiD, Nandana, Namenlos314, Gaboe420, Giuliamocci, Cpaulf30, Lahi, Gq86, Af420, Bsandipan, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, Lewizho99, Maathavan, _jensen, rosalieper, Neuronton, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T302494: The WDQS Streaming Updater should use S3 to access thanos-swift instead of the native swift protocol
dcausse updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T302494 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: RKemper, dcausse Cc: bking, Aklapper, dcausse, Fernandobacasegua34, Astuthiodit_1, 786, Suran38, Biggs657, karapayneWMDE, Invadibot, Lalamarie69, MPhamWMF, maantietaja, Juan90264, Alter-paule, Beast1978, CBogen, Un1tY, Akuckartz, Hook696, Kent7301, joker88john, CucyNoiD, Nandana, Namenlos314, Gaboe420, Giuliamocci, Cpaulf30, Lahi, Gq86, Af420, Bsandipan, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, Lewizho99, Maathavan, _jensen, rosalieper, Neuronton, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T302830: query service: (Alert) Reduced availability for job jmx_wdqs_updater in eqiad
dcausse updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T302830 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: RKemper, dcausse Cc: Aklapper, RKemper, Astuthiodit_1, karapayneWMDE, Invadibot, MPhamWMF, maantietaja, CBogen, Akuckartz, Nandana, Namenlos314, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T303256: WDQS servers should use skolem for wikibaseSomeValueMode
dcausse added a comment. might be accidentally fixed by merging https://gerrit.wikimedia.org/r/c/operations/puppet/+/742670 (it's still unclear why it's broken in the first place) TASK DETAIL https://phabricator.wikimedia.org/T303256 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: seav, Aklapper, dcausse, Astuthiodit_1, karapayneWMDE, Invadibot, MPhamWMF, maantietaja, CBogen, Akuckartz, Nandana, Namenlos314, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T302494: The WDQS Streaming Updater should use S3 to access thanos-swift instead of the native swift protocol
dcausse updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T302494 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: RKemper, dcausse Cc: Aklapper, dcausse, Fernandobacasegua34, Astuthiodit_1, 786, Suran38, Biggs657, karapayneWMDE, Invadibot, Lalamarie69, MPhamWMF, maantietaja, Juan90264, Alter-paule, Beast1978, CBogen, Un1tY, Akuckartz, Hook696, Kent7301, joker88john, CucyNoiD, Nandana, Namenlos314, Gaboe420, Giuliamocci, Cpaulf30, Lahi, Gq86, Af420, Bsandipan, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, Lewizho99, Maathavan, _jensen, rosalieper, Neuronton, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T293063: Write and adapt Runbooks and cookbooks related to the WDQS Streaming Updater and kubernetes
dcausse updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T293063 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: JMeybohm, Jelto, Aklapper, jijiki, dcausse, Astuthiodit_1, Arnoldokoth, karapayneWMDE, Invadibot, MPhamWMF, GeminiAgaloos, maantietaja, wkandek, CBogen, Akuckartz, Nandana, Namenlos314, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Addshore, Mbch331, Dzahn ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T293063: Write and adapt Runbooks and cookbooks related to the WDQS Streaming Updater and kubernetes
dcausse updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T293063 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: JMeybohm, Jelto, Aklapper, jijiki, dcausse, Astuthiodit_1, Arnoldokoth, karapayneWMDE, Invadibot, MPhamWMF, GeminiAgaloos, maantietaja, wkandek, CBogen, Akuckartz, Nandana, Namenlos314, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Addshore, Mbch331, Dzahn ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T303256: WDQS servers should use skolem for wikibaseSomeValueMode
dcausse updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T303256 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: Aklapper, dcausse, MPhamWMF, CBogen, Namenlos314, Gq86, Lucas_Werkmeister_WMDE, EBjune, merbst, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T303256: WDQS servers should use skolem for wikibaseSomeValueMode
dcausse updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T303256 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: Aklapper, dcausse, MPhamWMF, CBogen, Namenlos314, Gq86, Lucas_Werkmeister_WMDE, EBjune, merbst, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T303256: WDQS servers should use skolem for wikibaseSomeValueMode
dcausse created this task. dcausse added a project: Wikidata-Query-Service. Restricted Application added a subscriber: Aklapper. TASK DESCRIPTION The function `wikibase:isSomeValue` should filter skolem and not blank nodes on wdqs servers. It seems that this setting was recently acctidentaly dropped from the server configurations. AC: - blazegraph should be started with `-DwikibaseSomeValueMode=skolem` TASK DETAIL https://phabricator.wikimedia.org/T303256 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: Aklapper, dcausse, MPhamWMF, CBogen, Namenlos314, Gq86, Lucas_Werkmeister_WMDE, EBjune, merbst, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T293862: Investigate using jvmquake to limit the time a JVM is unusable due to GC overhead
dcausse added a comment. Pushed https://gitlab.wikimedia.org/repos/search-platform/jvmquake/-/merge_requests/1 (up for review) to have a debian package that we could install on production machines. TASK DETAIL https://phabricator.wikimedia.org/T293862 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: Aklapper, dcausse, karapayneWMDE, Invadibot, MPhamWMF, maantietaja, CBogen, Akuckartz, Nandana, Namenlos314, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T302494: The WDQS Streaming Updater should use S3 to access thanos-swift instead of the native swift protocol
dcausse updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T302494 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: RKemper, dcausse Cc: Aklapper, dcausse, Fernandobacasegua34, 786, Suran38, Biggs657, karapayneWMDE, Invadibot, Lalamarie69, MPhamWMF, maantietaja, Juan90264, Alter-paule, Beast1978, CBogen, Un1tY, Akuckartz, Hook696, Kent7301, joker88john, CucyNoiD, Nandana, Namenlos314, Gaboe420, Giuliamocci, Cpaulf30, Lahi, Gq86, Af420, Bsandipan, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, Lewizho99, Maathavan, _jensen, rosalieper, Neuronton, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T279541: Add a reconciliation strategy to the wdqs streaming updater
dcausse added a comment. Deployment of this feature has been stopped due to T302340 <https://phabricator.wikimedia.org/T302340>. TASK DETAIL https://phabricator.wikimedia.org/T279541 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: EBernhardson, RShigapov, dcausse, Aklapper, Fernandobacasegua34, 786, Suran38, Biggs657, karapayneWMDE, Invadibot, Lalamarie69, MPhamWMF, maantietaja, Juan90264, Alter-paule, Beast1978, CBogen, Un1tY, Akuckartz, Hook696, Kent7301, joker88john, CucyNoiD, Nandana, Namenlos314, Gaboe420, Giuliamocci, Cpaulf30, Lahi, Gq86, Af420, Bsandipan, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, Lewizho99, Maathavan, _jensen, rosalieper, Neuronton, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T294076: Blazegraph and MariaDB contain different sitelinks at Wikidata
dcausse merged a task: T302458: Q108896181 keeps showing up as having zero statements. dcausse added subscribers: Sjoerddebruin, Lucas_Werkmeister_WMDE. TASK DETAIL https://phabricator.wikimedia.org/T294076 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: Lucas_Werkmeister_WMDE, Sjoerddebruin, dcausse, William_Avery, RShigapov, Aklapper, karapayneWMDE, Invadibot, MPhamWMF, maantietaja, CBogen, Akuckartz, Nandana, Namenlos314, Lahi, Gq86, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T302458: Q108896181 keeps showing up as having zero statements
dcausse closed this task as a duplicate of T294076: Blazegraph and MariaDB contain different sitelinks at Wikidata. TASK DETAIL https://phabricator.wikimedia.org/T302458 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: dcausse, Lucas_Werkmeister_WMDE, Aklapper, Sjoerddebruin, karapayneWMDE, Invadibot, MPhamWMF, maantietaja, CBogen, Akuckartz, Nandana, Namenlos314, Lahi, Gq86, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T302458: Q108896181 keeps showing up as having zero statements
dcausse added a comment. Thanks for the report, these inconsistencies are due to missed updates which will be automatically fixed once T279541 <https://phabricator.wikimedia.org/T279541> is deployed. I'm marking this ticket as a duplicate of T294076 <https://phabricator.wikimedia.org/T294076> since they share the same root cause and same fix. TASK DETAIL https://phabricator.wikimedia.org/T302458 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: dcausse, Lucas_Werkmeister_WMDE, Aklapper, Sjoerddebruin, karapayneWMDE, Invadibot, MPhamWMF, maantietaja, CBogen, Akuckartz, Nandana, Namenlos314, Lahi, Gq86, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T302494: The WDQS Streaming Updater should use S3 to access thanos-swift instead of the native swift protocol
dcausse created this task. dcausse added a project: Wikidata-Query-Service. Restricted Application added a subscriber: Aklapper. TASK DESCRIPTION Followup of T302396 <https://phabricator.wikimedia.org/T302396>. The thanos-swift cluster is S3 <https://phabricator.wikimedia.org/S3> compatible so we should use that instead of the native swift client which we customized to implement tmp auth and has been removed from the official flink distribution: https://issues.apache.org/jira/browse/FLINK-21819. AC: - WDQS Streaming Updater is thanos-swift through the s3 protocol TASK DETAIL https://phabricator.wikimedia.org/T302494 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: Aklapper, dcausse, MPhamWMF, CBogen, Namenlos314, Gq86, Lucas_Werkmeister_WMDE, EBjune, merbst, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T302396: Investigate EOFException when performing the first checkpoint after restoring from a savepoint
dcausse added a comment. Root cause seems swift related: Saw this in taskmanager logs: `Received IOException while reading 'swift://rdf-streaming-updater-codfw.thanos-swift/wikidata/savepoints/savepoint-0d1c37-86ed4cb29023/bc3ce8ed-70b2-4e91-a81b-07f585dd0f1f', attempting to reopen: org.apache.flink.fs.openstackhadoop.shaded.org.apache.hadoop.fs.swift.exceptions.SwiftConnectionClosedException: Connection to Swift service has been closed: read() -all data consumed` TASK DETAIL https://phabricator.wikimedia.org/T302396 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: EBernhardson, Aklapper, dcausse, karapayneWMDE, Invadibot, MPhamWMF, maantietaja, CBogen, Akuckartz, Nandana, Namenlos314, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T302396: Investigate EOFException when performing the first checkpoint after restoring from a savepoint
dcausse updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T302396 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: Aklapper, dcausse, karapayneWMDE, Invadibot, MPhamWMF, maantietaja, CBogen, Akuckartz, Nandana, Namenlos314, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T302396: Investigate EOFException when performing the first checkpoint after restoring from a savepoint
dcausse updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T302396 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: Aklapper, dcausse, karapayneWMDE, Invadibot, MPhamWMF, maantietaja, CBogen, Akuckartz, Nandana, Namenlos314, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T302396: Investigate EOFException when performing the first checkpoint after restoring from a savepoint
dcausse created this task. dcausse added a project: Wikidata-Query-Service. Restricted Application added a subscriber: Aklapper. TASK DESCRIPTION While deploying a new version of the streaming-updater (0.3.103) flink failed with: java.io.EOFException at java.base/java.io.DataInputStream.readFully(DataInputStream.java:202) at java.base/java.io.DataInputStream.readFully(DataInputStream.java:170) at org.apache.flink.api.common.typeutils.base.array.BytePrimitiveArraySerializer.deserialize(BytePrimitiveArraySerializer.java:82) at org.apache.flink.contrib.streaming.state.restore.RocksDBFullRestoreOperation.restoreKVStateData(RocksDBFullRestoreOperation.java:229) at org.apache.flink.contrib.streaming.state.restore.RocksDBFullRestoreOperation.restoreKeyGroupsInStateHandle(RocksDBFullRestoreOperation.java:158) at org.apache.flink.contrib.streaming.state.restore.RocksDBFullRestoreOperation.restore(RocksDBFullRestoreOperation.java:142) at org.apache.flink.contrib.streaming.state.RocksDBKeyedStateBackendBuilder.build(RocksDBKeyedStateBackendBuilder.java:284) at org.apache.flink.contrib.streaming.state.RocksDBStateBackend.createKeyedStateBackend(RocksDBStateBackend.java:587) at org.apache.flink.contrib.streaming.state.RocksDBStateBackend.createKeyedStateBackend(RocksDBStateBackend.java:93) at org.apache.flink.streaming.api.operators.StreamTaskStateInitializerImpl.lambda$keyedStatedBackend$1(StreamTaskStateInitializerImpl.java:328) at org.apache.flink.streaming.api.operators.BackendRestorerProcedure.attemptCreateAndRestore(BackendRestorerProcedure.java:168) at org.apache.flink.streaming.api.operators.BackendRestorerProcedure.createAndRestore(BackendRestorerProcedure.java:135) at org.apache.flink.streaming.api.operators.StreamTaskStateInitializerImpl.keyedStatedBackend(StreamTaskStateInitializerImpl.java:345) at org.apache.flink.streaming.api.operators.StreamTaskStateInitializerImpl.streamOperatorStateContext(StreamTaskStateInitializerImpl.java:163) at org.apache.flink.streaming.api.operators.AbstractStreamOperator.initializeState(AbstractStreamOperator.java:272) at org.apache.flink.streaming.runtime.tasks.OperatorChain.initializeStateAndOpenOperators(OperatorChain.java:425) at org.apache.flink.streaming.runtime.tasks.StreamTask.lambda$beforeInvoke$2(StreamTask.java:535) at org.apache.flink.streaming.runtime.tasks.StreamTaskActionExecutor$1.runThrowing(StreamTaskActionExecutor.java:50) at org.apache.flink.streaming.runtime.tasks.StreamTask.beforeInvoke(StreamTask.java:525) at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:565) at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:755) at org.apache.flink.runtime.taskmanager.Task.run(Task.java:570) at java.base/java.lang.Thread.run(Thread.java:834) notes: - the problem occurs when using two different savepoints: - (thanos) `rdf-streaming-updater-codfw/commons/savepoints/deploy_0_3_103/savepoint-c4b021-9c4cd6541ec7/` - (thanos) `rdf-streaming-updater-codfw/commons/savepoints/savepoint-c4b021-818dd669a47e/` - the exception is only visible on a taskmanager POD running on `kubernetes2003` (logs <https://logstash.wikimedia.org/app/discover#/?_g=(filters:!(),query:(language:lucene,query:kubernetes2003),refreshInterval:(pause:!t,value:0),time:(from:'2022-02-22T16:00:00.000Z',to:'2022-02-22T19:30:00.000Z'))&_a=(columns:!(log,host,message,kubernetes.pod_id),filters:!(),index:'logstash-*',interval:auto,query:(language:lucene,query:'kubernetes.master_url:%22https:%2F%2Fkubemaster.svc.codfw.wmnet%22%20AND%20kubernetes.namespace_name:%22rdf-streaming-updater%22%20AND%20java.io.EOFException%20AND%20kubernetes.labels.component:taskmanager'),sort:!())>) - the problem occurs when loading the savepoints using the previous version 0.3.99 - the deploy worked fine on staging for both wdqs and wcqs, fine as well on wcqs at codfw - the system was able to resume using 0.3.99 and a previous checkpoint `rdf-streaming-updater-codfw/wikidata/checkpoints/e245dd1e76d56d9ded351b27cf2d4c2a/chk-415014`. AC: - understand the root cause of the failure TASK DETAIL https://phabricator.wikimedia.org/T302396 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: Aklapper, dcausse, MPhamWMF, CBogen, Namenlos314, Gq86, Lucas_Werkmeister_WMDE, EBjune, merbst, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T302189: Query service retains orphaned sitelinks
dcausse closed this task as "Declined". dcausse added a comment. Sitelink orphans are not clean up at update time for performance reasons, same is done for orphaned `values` and `references`. The database is cleaned up during a full reload which we try to plan once a year. Declining as this is done "on-purpose" but please feel free to re-open if this causes any usability issue on your side. TASK DETAIL https://phabricator.wikimedia.org/T302189 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: dcausse, Aklapper, Yair_rand, karapayneWMDE, Invadibot, MPhamWMF, maantietaja, CBogen, Akuckartz, Nandana, Namenlos314, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T301695: Special:EntityData on test-commons.wikimedia.org produces wrong sdcdata prefix in its RDF output
dcausse created this task. dcausse added a project: SDC General. Restricted Application added a subscriber: Aklapper. TASK DESCRIPTION When I extract the RDF content using Special:EntityData of an image with a MediaInfo slot on test-commons-wikimedia.org I want it to be configured simarly than commons.wikimedia.org so that I can use test-commons as a test service. When calling https://test-commons.wikimedia.org/wiki/Special:EntityData/M410.ttl I see: @prefix sdcdata: <https://test-commons.wikimedia.org/wiki/testcommons:Special:EntityData/> . But I expect: @prefix sdcdata: <https://test-commons.wikimedia.org/wiki/Special:EntityData/> . TASK DETAIL https://phabricator.wikimedia.org/T301695 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: Aklapper, dcausse, GFontenelle_WMF, FRomeo_WMF, CBogen, Nintendofan885, JKSTNK, Lahi, E1presidente, Ramsey-WMF, Cparle, SandraF_WMF, Tramullas, Acer, Salgo60, Silverfish, Susannaanas, Jane023, Wikidata-bugs, Base, matthiasmullie, Ricordisamoa, Wesalius, Lydia_Pintscher, Raymond, Steinsplitter ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T301147: The WDQS streaming updater went unstable for several hours (2022-02-06T23:00:00 - 2022-02-07T06:20:00)
dcausse added a comment. In T301147#7692414 <https://phabricator.wikimedia.org/T301147#7692414>, @JMeybohm wrote: > In T301147#7689837 <https://phabricator.wikimedia.org/T301147#7689837>, @dcausse wrote: > >> @JMeybohm we're still investigating why the application did not properly recover while kubernetes1014 went down but if you have ideas on the two questions in the ticket description this would be very helpful, thanks! > > Unfortunately I'm not exactly sure what happened to the node. What I know is that the system load surged (potentially due to high iowait) on the system, leaving running processes practically starving but the system was still responding to ICMP and kubernetes status heartbeats still (mostly) worked. Leaving the node flipping between Ready/NotReady state. > That means the node was not actually down from k8s POV, which is why no new Pods where created until I drained the node respectively before I powercycled it (as evicting pods was actually hanging as well, as k8s tries to be nice and the node still was in it's overloaded state). Thanks! I've updated the task description with few action items, please let us know if you see something else we should do to improve this. TASK DETAIL https://phabricator.wikimedia.org/T301147 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: toan, Addshore, JMeybohm, Michael, Aklapper, dcausse, Invadibot, MPhamWMF, maantietaja, CBogen, Akuckartz, Nandana, Namenlos314, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T301147: The WDQS streaming updater went unstable for several hours (2022-02-06T23:00:00 - 2022-02-07T06:20:00)
dcausse updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T301147 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: toan, Addshore, JMeybohm, Michael, Aklapper, dcausse, Invadibot, MPhamWMF, maantietaja, CBogen, Akuckartz, Nandana, Namenlos314, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T293862: Investigate using jvmquake to limit the time a JVM is unusable due to GC overhead
dcausse claimed this task. dcausse moved this task from Ready for Development to In Progress on the Discovery-Search (Current work) board. TASK DETAIL https://phabricator.wikimedia.org/T293862 WORKBOARD https://phabricator.wikimedia.org/project/board/1227/ EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: Aklapper, dcausse, Invadibot, MPhamWMF, maantietaja, CBogen, Akuckartz, Nandana, Namenlos314, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T301147: The WDQS streaming updater went unstable for several hours (2022-02-06T23:00:00 - 2022-02-07T06:20:00)
dcausse added a comment. k8s seems to have tried to kill the container for the whole period according messages like: Container flink-session-cluster-main-taskmanager failed liveness probe, will be restarted <https://logstash.wikimedia.org/app/discover#/doc/logstash-*/logstash-syslog-2022.02.07?id=C_LI0n4B-VMqbJtQBjqU> (searching for `k8s_event.involvedObject.uid:"1db45eb6-2405-4aa3-bec1-71fcdbbe4f9a"`). TASK DETAIL https://phabricator.wikimedia.org/T301147 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: Addshore, JMeybohm, Michael, Aklapper, dcausse, Invadibot, MPhamWMF, maantietaja, CBogen, Akuckartz, Nandana, Namenlos314, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T301147: The WDQS streaming updater went unstable for several hours (2022-02-06T23:00:00 - 2022-02-07T06:20:00)
dcausse added a subscriber: JMeybohm. dcausse added a comment. @JMeybohm we're still investigating why the application did not properly recover while kubernetes1014 went down but if you have ideas on the two questions in the ticket description this would be very helpful, thanks! TASK DETAIL https://phabricator.wikimedia.org/T301147 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: JMeybohm, Michael, Aklapper, dcausse, Invadibot, MPhamWMF, maantietaja, CBogen, Akuckartz, Nandana, Namenlos314, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T301147: The WDQS streaming updater went unstable for several hours (2022-02-06T23:00:00 - 2022-02-07T06:20:00)
dcausse created this task.
dcausse added a project: Wikidata-Query-Service.
Restricted Application added a subscriber: Aklapper.
TASK DESCRIPTION
For 7 hours (`2022-02-06T23:00:00` to `2022-02-07T06:20:00`) the streaming
updater in `eqiad` stopped working properly preventing edits to flow to all the
wdqs machines in eqiad.
The lag started to rise in eqiad and caused edits to be throttled during this
period:
F34944091: Capture d’écran du 2022-02-07 11-40-08.png
<https://phabricator.wikimedia.org/F34944091>
Investigations:
- the streaming updater for WCQS went down from `2022-02-06T16:32:00` to
`2022-02-06T23:00:00`
- the streaming updater for WDQS went down from `2022-02-06T23:00:00` to
`2022-02-07T06:20:00`
- the number of total task slots went down to 20 from 24 (4tasks == 1pod)
between `2022-02-06T16:32:00` and `2022-02-07T06:20:00` causing resource
starvation and preventing both jobs from running at the same time
(`flink_jobmanager_taskSlotsTotal{kubernetes_namespace="rdf-streaming-updater"}`)
- kubernetes1014 (T301099 <https://phabricator.wikimedia.org/T301099>) seemed
to have showed problems during this same period (`2022-02-06T16:32:00` to
`2022-02-07T06:20:00`)
- the deployment used by the updater used one POD
(`1db45eb6-2405-4aa3-bec1-71fcdbbe4f9a`) from kubernetes1014
- the flink session cluster was able to regain its 24 slots after after
`1db45eb6-2405-4aa3-bec1-71fcdbbe4f9a` came back (at `2022-02-07T08:07:00`),
then this POD disappeared again in favor of another one and the service
successfully restarted.
- during the whole incident k8s metrics & flink metrics seem to disagree:
- flink says that it lost 4 task managers (1 POD)
- k8s always reports at least 6 PODS
(`count(container_memory_usage_bytes{namespace="rdf-streaming-updater",
container="flink-session-cluster-main-taskmanager"})`)
Questions:
- why do flink and k8s metrics disagree (active PODs vs number of task
manager)?
- why a new POD was not created after kubernetes1014 went down (making
`1db45eb6-2405-4aa3-bec1-71fcdbbe4f9a` unavailable to the deployment)?
What could we have done better:
- we could have route wdqs traffic to codfw during the outage and avoid
throttling edits
TASK DETAIL
https://phabricator.wikimedia.org/T301147
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: dcausse
Cc: Aklapper, dcausse, MPhamWMF, CBogen, Namenlos314, Gq86,
Lucas_Werkmeister_WMDE, EBjune, merbst, Jonas, Xmlizer, jkroll, Wikidata-bugs,
Jdouglas, aude, Tobias1984, Manybubbles
___
Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org
To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T289770: Add hints in response headers for 404 responses in Special:EntityData
dcausse closed this task as "Declined". dcausse added a comment. Thanks for the investigation on this! We don't plan to pursue this route given the added complexity and it's not clear if the benefit is worth the effort, esp. over tuning retries on 404 and the work on reconciliation. I'm tentatively declining and will reopen if we think it's worth reconsidering again. TASK DETAIL https://phabricator.wikimedia.org/T289770 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Michael, dcausse Cc: Michael, Lydia_Pintscher, Lucas_Werkmeister_WMDE, dcausse, Zbyszko, Addshore, Aklapper, 786, Suran38, Biggs657, Invadibot, Lalamarie69, MPhamWMF, maantietaja, Juan90264, Alter-paule, Beast1978, CBogen, Un1tY, Akuckartz, Hook696, Kent7301, joker88john, CucyNoiD, Nandana, Namenlos314, Gaboe420, Giuliamocci, Cpaulf30, Lahi, Gq86, Af420, Bsandipan, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, Lewizho99, Maathavan, _jensen, rosalieper, Neuronton, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T299290: Unexpected behavior in federated queries with LinguaLibre in WDQS
dcausse removed projects: Discovery-Search (Current work), Wikidata, Wikidata-Query-Service. dcausse added a comment. Tried to debug this a bit and I believe the problem is on the lingualibre side. I suspect a weird bug happening because of the query length. Query that passes: https://people.wikimedia.org/~dcausse/T299290-ok.sparql Query that fails: https://people.wikimedia.org/~dcausse/T299290-bad.sparql Difference is just one empty space. Command to run this manually is: `curl -X POST -H"Accept: application/sparql-results+xml" --data-urlencode query@T299290-bad.sparql -data-urlencode "queryId=de249fca-8362-11ec-a8a3-0242ac120002" https://lingualibre.org/sparql` Happy to help if LinguaLibre maintainers can obtain some server logs. Untagging WDQS as there's not much we can do on our side. TASK DETAIL https://phabricator.wikimedia.org/T299290 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: dcausse, Rdrg109, Eihel, Poslovitch, Pamputt, Base, Ltrlg, Invadibot, MPhamWMF, maantietaja, CBogen, Akuckartz, Nandana, Namenlos314, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T300240: Missing Wikidata RDF (ttl and nt) dumps for 20220117
dcausse added a comment. @ArielGlenn no problem! :) When these dumps fail we're informed couple days after and it might be interesting for us to be pro-active about that but not sure we have enough knowledge of the dump process/infra to be super useful in case of failures, if you feel it's accessible to a neophyte like me please feel free to add me :) TASK DETAIL https://phabricator.wikimedia.org/T300240 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: ArielGlenn, Aklapper, JAllemandou, AKhatun_WMF, dcausse, Invadibot, maantietaja, jannee_e, Akuckartz, holger.knust, Nandana, Lahi, Gq86, GoranSMilovanovic, Lunewa, QZanden, LawExplorer, _jensen, rosalieper, Scott_WUaS, gnosygnu, Wikidata-bugs, aude, Addshore, Mbch331 ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T300240: Missing Wikidata RDF (ttl and nt) dumps for 20220117
dcausse closed this task as "Invalid". dcausse added a comment. Thanks for checking! TASK DETAIL https://phabricator.wikimedia.org/T300240 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: ArielGlenn, Aklapper, JAllemandou, AKhatun_WMF, dcausse, Invadibot, maantietaja, jannee_e, Akuckartz, holger.knust, Nandana, Lahi, Gq86, GoranSMilovanovic, Lunewa, QZanden, LawExplorer, _jensen, rosalieper, Scott_WUaS, gnosygnu, Wikidata-bugs, aude, Addshore, Mbch331 ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T300240: Missing Wikidata RDF (ttl and nt) dumps for 20220117
dcausse renamed this task from "TaskInstance: import_wikidata_ttl.wait_for_all_ttl_dump 2022-01-14T03:00:00+00:00" to "Missing Wikidata RDF (ttl and nt) dumps for 20220117". dcausse updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T300240 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: Aklapper, JAllemandou, AKhatun_WMF, dcausse, Invadibot, MPhamWMF, maantietaja, CBogen, Akuckartz, Nandana, Namenlos314, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T300240: TaskInstance: import_wikidata_ttl.wait_for_all_ttl_dump 2022-01-14T03:00:00+00:00
dcausse added a comment. RDF dumps seem absent from https://dumps.wikimedia.org/wikidatawiki/entities/20220117/ but seems to be there again on 20220124. Everything is OK now so I suspect a transient failure. Pinging #dumps-generation <https://phabricator.wikimedia.org/tag/dumps-generation/> just in case there's something to investigate. TASK DETAIL https://phabricator.wikimedia.org/T300240 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: Aklapper, JAllemandou, AKhatun_WMF, dcausse, Invadibot, MPhamWMF, maantietaja, CBogen, Akuckartz, Nandana, Namenlos314, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T297454: WCQS gives "502 Bad Gateway Error"
dcausse closed this task as "Resolved". dcausse added a comment. Thanks for the report, blazegraph died I restarted it, should be available again now. TASK DETAIL https://phabricator.wikimedia.org/T297454 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcaro, dcausse Cc: dcausse, Theklan, Marsupium, Vojtech.dostal, Base, RhinosF1, Majavah, aborrero, GFontenelle_WMF, Sj, FRomeo_WMF, Fuzheado, Dominicbm, HenkvD, Alicia_Fagerving_WMSE, EBernhardson, Aklapper, Jarekt, Invadibot, MPhamWMF, maantietaja, CBogen, Nintendofan885, Akuckartz, Nandana, JKSTNK, Namenlos314, Lahi, Gq86, E1presidente, Ramsey-WMF, Cparle, Anoop, SandraF_WMF, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, Tramullas, Acer, merbst, LawExplorer, Salgo60, Silverfish, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, Susannaanas, Jane023, jkroll, Wikidata-bugs, Jdouglas, matthiasmullie, aude, Tobias1984, Manybubbles, Ricordisamoa, Wesalius, Lydia_Pintscher, Raymond, Steinsplitter, Mbch331 ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T300240: TaskInstance: import_wikidata_ttl.wait_for_all_ttl_dump 2022-01-14T03:00:00+00:00
dcausse created this task.
dcausse added a project: Wikidata-Query-Service.
Restricted Application added a subscriber: Aklapper.
TASK DESCRIPTION
As reported by airflow, a sensor is timing out on:
[2022-01-27 04:41:24,843] {hdfs_cli.py:71} INFO - Checking marker at
hdfs://analytics-hadoop/wmf/data/raw/wikidata/dumps/all_ttl/20220117/_IMPORTED
At the time of the error HDFS only reports this content:
hdfs dfs -ls hdfs://analytics-hadoop/wmf/data/raw/wikidata/dumps/all_ttl/
drwxr-x--- - analytics analytics-privatedata-users 0 2022-01-27
01:30 hdfs://analytics-hadoop/wmf/data/raw/wikidata/dumps/all_ttl/20211213
drwxr-x--- - analytics analytics-privatedata-users 0 2022-01-27
01:30 hdfs://analytics-hadoop/wmf/data/raw/wikidata/dumps/all_ttl/20211220
drwxr-x--- - analytics analytics-privatedata-users 0 2022-01-27
01:30 hdfs://analytics-hadoop/wmf/data/raw/wikidata/dumps/all_ttl/20211227
drwxr-x--- - analytics analytics-privatedata-users 0 2022-01-27
01:30 hdfs://analytics-hadoop/wmf/data/raw/wikidata/dumps/all_ttl/20220103
drwxr-x--- - analytics analytics-privatedata-users 0 2022-01-27
01:30 hdfs://analytics-hadoop/wmf/data/raw/wikidata/dumps/all_ttl/20220110
TASK DETAIL
https://phabricator.wikimedia.org/T300240
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: dcausse
Cc: Aklapper, JAllemandou, AKhatun_WMF, dcausse, MPhamWMF, CBogen, Namenlos314,
Gq86, Lucas_Werkmeister_WMDE, EBjune, merbst, Jonas, Xmlizer, jkroll,
Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles
___
Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org
To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T299460: Evaluate Apache Jena
dcausse added a comment. Sorry for the confusion that the rename I did of this task caused. Just to bring clarity on my reasoning as a maintainer of the wikidata query service stack as to why being specific on TDB2 might be helpful: - Some components of Jena are already being used (i.e. the sparql parser for query analysis) - Jena has been considered in 2015 but declined ref: T90112 <https://phabricator.wikimedia.org/T90112> (sadly no reasons were given) This task is I think about evaluating Jena and its storage component as a storage/query engine for Wikidata Query Service but it does not mean that all of what Jena offers will be discarded if this task is declined. TASK DETAIL https://phabricator.wikimedia.org/T299460 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: Osmasuominen, dcausse, Smalyshev, Aklapper, Lucas_Werkmeister_WMDE, Gehel, Andrawaag, Addshore, Susannaanas, Akuckartz, TomT0m, Jecummings4, Krabina, So9q, Salgo60, WMDE-leszek, GreenReaper, Ostrzyciel, Samantha_Alipio_WMDE, Tagishsimon, Lydia_Pintscher, DanBri, Jneubert, Ivanhercaz, TheKtk, Jerven, Justin0x2004, Afandian, Sj, TallTed, Tpt, Thadguidry, danshick-wmde, Hjfocs, Mohammed_Sadat_WMDE, MarioGom, karapayneWMDE, Daniel_Mietchen, KingsleyIdehen, Izno, RShigapov, Hannah_Bast, Kjauslin, toan, Michael, DD063520, AndreasKuczera, Versant.2612, namedgraph, Iamamz3, YULdigitalpreservation, BenAtOlive, nguyenm9, Fnielsen, accounting_data_logger, JohannesKalmbach, Dr.uesenfieber, Bovlb, AndySeaborne, BeautifulBold, Suran38, Invadibot, MPhamWMF, Jtm-lis, maantietaja, Peteosx1x, NavinRizwi, CBogen, Isaacandy, Demian, Olson.jared.m, Nandana, Namenlos314, Lahi, Gq86, Bryandamon, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, Steko, Samwilson, PhotographerTom, suriyaa, Psychoslave, tosfos, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Darenwelsh, Dinoguy1000, Manybubbles, brion, Mbch331, MarkAHershberger ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T299460: Evaluate Apache Jena TDB2
dcausse renamed this task from "Evaluate Apache Jena" to "Evaluate Apache Jena TDB2". dcausse updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T299460 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: Smalyshev, Aklapper, Lucas_Werkmeister_WMDE, Gehel, Andrawaag, Addshore, Susannaanas, Akuckartz, TomT0m, Jecummings4, Krabina, So9q, Salgo60, WMDE-leszek, GreenReaper, Ostrzyciel, Samantha_Alipio_WMDE, Tagishsimon, Lydia_Pintscher, DanBri, Jneubert, Ivanhercaz, TheKtk, Jerven, Justin0x2004, Afandian, Sj, TallTed, Tpt, Thadguidry, danshick-wmde, Hjfocs, Mohammed_Sadat_WMDE, MarioGom, karapayneWMDE, Daniel_Mietchen, KingsleyIdehen, Izno, RShigapov, Hannah_Bast, Kjauslin, toan, Michael, DD063520, AndreasKuczera, Versant.2612, namedgraph, Iamamz3, YULdigitalpreservation, BenAtOlive, nguyenm9, Fnielsen, accounting_data_logger, JohannesKalmbach, Dr.uesenfieber, Bovlb, AndySeaborne, BeautifulBold, Suran38, Invadibot, MPhamWMF, Jtm-lis, maantietaja, Peteosx1x, NavinRizwi, CBogen, Isaacandy, Demian, Olson.jared.m, Nandana, Namenlos314, Lahi, Gq86, Bryandamon, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, Steko, Samwilson, PhotographerTom, suriyaa, Psychoslave, tosfos, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Darenwelsh, Dinoguy1000, Manybubbles, brion, Mbch331, MarkAHershberger ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T299290: Unexpected behavior in federated queries in WDQS
dcausse added a comment.
WDQS receives `Status Code=502, Status Line=Bad Gateway, Response=`
from lingualibre servers. I'm not totally sure to understand why it's failing
esp. why Shopox is generating a query that is accepted there and why it may
sometimes succeed from wdqs when varying the query.
Few simpler examples that bug me:
SELECT * {
SERVICE <https://lingualibre.org/sparql> {
select ?item {
?item <https://lingualibre.org/prop/direct/P2>
<https://lingualibre.org/entity/Q5> .
}
}
}
is NOT OK (OK via shophox)
SELECT * {
SERVICE <https://lingualibre.org/sparql> {
?item <https://lingualibre.org/prop/direct/P2>
<https://lingualibre.org/entity/Q5> .
}
}
is OK
TASK DETAIL
https://phabricator.wikimedia.org/T299290
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: dcausse
Cc: dcausse, Rdrg109, Invadibot, MPhamWMF, maantietaja, CBogen, Akuckartz,
Eihel, Nandana, Namenlos314, Poslovitch, Lahi, Gq86, Lucas_Werkmeister_WMDE,
GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper,
Pamputt, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, Base,
aude, Tobias1984, Manybubbles, Mbch331, Ltrlg
___
Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org
To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T281468: Automatic SI unit conversion not working on Commons SPARQL engine
dcausse assigned this task to Ladsgroup. dcausse moved this task from Ready for Development to Needs Reporting on the Discovery-Search (Current work) board. dcausse added a comment. I think it's working properly now TASK DETAIL https://phabricator.wikimedia.org/T281468 WORKBOARD https://phabricator.wikimedia.org/project/board/1227/ EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Ladsgroup, dcausse Cc: dcausse, Gehel, Ladsgroup, WMDE-leszek, Addshore, Aklapper, CBogen, Lucas_Werkmeister_WMDE, Multichill, Invadibot, GFontenelle_WMF, MPhamWMF, maantietaja, Y.ssk, FRomeo_WMF, Muchiri124, Hazizibinmahdi, Nintendofan885, Akuckartz, Nandana, JKSTNK, Namenlos314, Lahi, Gq86, E1presidente, Ramsey-WMF, Cparle, Anoop, SandraF_WMF, GoranSMilovanovic, QZanden, EBjune, Tramullas, Acer, merbst, LawExplorer, Salgo60, Silverfish, Poyekhali, _jensen, rosalieper, 4nn1l2, Taiwania_Justo, Scott_WUaS, Jonas, Xmlizer, Susannaanas, Ixocactus, Wong128hk, Jane023, jkroll, Wikidata-bugs, Jdouglas, Base, matthiasmullie, aude, Tobias1984, El_Grafo, Dinoguy1000, Manybubbles, Ricordisamoa, Wesalius, Lydia_Pintscher, Raymond, Steinsplitter, Mbch331 ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T281468: Automatic SI unit conversion not working on Commons SPARQL engine
dcausse updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T281468 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: Gehel, Ladsgroup, WMDE-leszek, Addshore, Aklapper, CBogen, Lucas_Werkmeister_WMDE, Multichill, Invadibot, GFontenelle_WMF, MPhamWMF, maantietaja, Y.ssk, FRomeo_WMF, Muchiri124, Hazizibinmahdi, Nintendofan885, Akuckartz, Nandana, JKSTNK, Namenlos314, Lahi, Gq86, E1presidente, Ramsey-WMF, Cparle, Anoop, SandraF_WMF, GoranSMilovanovic, QZanden, EBjune, Tramullas, Acer, merbst, LawExplorer, Salgo60, Silverfish, Poyekhali, _jensen, rosalieper, 4nn1l2, Taiwania_Justo, Scott_WUaS, Jonas, Xmlizer, Susannaanas, Ixocactus, Wong128hk, Jane023, jkroll, Wikidata-bugs, Jdouglas, Base, matthiasmullie, aude, Tobias1984, El_Grafo, Dinoguy1000, Manybubbles, Ricordisamoa, Wesalius, Lydia_Pintscher, Raymond, Steinsplitter, Mbch331 ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T262265: Provide real-time updates for WCQS
dcausse added a subtask: T298622: Adapt EntityRevisionMapGenerator for wcqs. TASK DETAIL https://phabricator.wikimedia.org/T262265 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: EBernhardson, dcausse Cc: Back_ache, So9q, Salgo60, Gehel, Aklapper, Zbyszko, Invadibot, MPhamWMF, maantietaja, CBogen, Akuckartz, Nandana, Namenlos314, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T298622: Adapt EntityRevisionMapGenerator for wcqs
dcausse added a parent task: T262265: Provide real-time updates for WCQS. TASK DETAIL https://phabricator.wikimedia.org/T298622 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: Aklapper, dcausse, MPhamWMF, CBogen, Namenlos314, Gq86, Lucas_Werkmeister_WMDE, EBjune, merbst, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T298622: Adapt EntityRevisionMapGenerator for wcqs
dcausse created this task. dcausse added a project: Wikidata-Query-Service. Restricted Application added a subscriber: Aklapper. TASK DESCRIPTION This spark job is required for building the initial state of the streaming updater, it must be adapted for commons. AC: - add new arguments to build the proper UriScheme for commons - adapt the import_commons_ttl dag to run it after every import TASK DETAIL https://phabricator.wikimedia.org/T298622 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: Aklapper, dcausse, MPhamWMF, CBogen, Namenlos314, Gq86, Lucas_Werkmeister_WMDE, EBjune, merbst, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T240334: Evaluate adding all/some textual properties to the text field
dcausse added a comment. Forwarding a suggestion made on https://www.wikidata.org/wiki/Wikidata:Report_a_technical_problem/WDQS_and_Search: > It would be interesting to be able to search for street address (P6375)-values, e.g. Special:Search/Getreidegasse Salzburg should find Q37970995. --- Jura 19:11, 28 December 2021 (UTC) TASK DETAIL https://phabricator.wikimedia.org/T240334 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: Lea_Lacroix_WMDE, Aklapper, dcausse, Invadibot, MPhamWMF, maantietaja, Wilmanbeno, CBogen, Akuckartz, Nandana, Lahi, Gq86, GoranSMilovanovic, QZanden, EBjune, LawExplorer, _jensen, rosalieper, Scott_WUaS, Wikidata-bugs, aude, jayvdb, Mbch331, jeremyb ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T297870: WDQS Streaming Updater fails with Timeout expired after 60000milliseconds while awaiting InitProducerId
dcausse created this task. dcausse added a project: Wikidata-Query-Service. Restricted Application added a subscriber: Aklapper. TASK DESCRIPTION This error causes the pipeline to restart and might trigger the latency alert //WdqsStreamingUpdaterFlinkProcessingLatencyIsHigh//. It was seen on the pipeline running in codfw right after one kafka node was removed from the cluster. It was not a single instance of the error and it occurred several times after, timeline: - 2021-12-15T16:12 kafka-main2003 is removed from the cluster - 2021-12-15T16:17 flink fails - 2021-12-15T16:19 flink fails - 2021-12-15T16:29 flink fails - 2021-12-15T16:37 flink fails - 2021-12-15T16:42 flink fails - 2021-12-16T10:46 flink fails The pipeline restarting after a kafka broker is removed is something we should expect but the subsequent failures seem to suggest that this setup flink + kafka-main minus one broker is less stable than usual. Flink is properly resuming without user-facing issues, it's noticeable only because the WdqsStreamingUpdaterFlinkProcessingLatencyIsHigh is being triggered. The flink error stack is: org.apache.flink.runtime.checkpoint.CheckpointException: Could not complete snapshot 216620 for operator RDFPatchChunkOperation -> MeasureEventProcessingLatencyOperation -> Sink: codfw.rdf-streaming-updater.mutation:0 (1/1)#1. Failure reason: Checkpoint was declined. at org.apache.flink.streaming.api.operators.StreamOperatorStateHandler.snapshotState(StreamOperatorStateHandler.java:241) at org.apache.flink.streaming.api.operators.StreamOperatorStateHandler.snapshotState(StreamOperatorStateHandler.java:162) at org.apache.flink.streaming.api.operators.AbstractStreamOperator.snapshotState(AbstractStreamOperator.java:371) at org.apache.flink.streaming.runtime.tasks.SubtaskCheckpointCoordinatorImpl.checkpointStreamOperator(SubtaskCheckpointCoordinatorImpl.java:685) at org.apache.flink.streaming.runtime.tasks.SubtaskCheckpointCoordinatorImpl.buildOperatorSnapshotFutures(SubtaskCheckpointCoordinatorImpl.java:606) at org.apache.flink.streaming.runtime.tasks.SubtaskCheckpointCoordinatorImpl.takeSnapshotSync(SubtaskCheckpointCoordinatorImpl.java:571) at org.apache.flink.streaming.runtime.tasks.SubtaskCheckpointCoordinatorImpl.checkpointState(SubtaskCheckpointCoordinatorImpl.java:298) at org.apache.flink.streaming.runtime.tasks.StreamTask.lambda$performCheckpoint$9(StreamTask.java:1003) at org.apache.flink.streaming.runtime.tasks.StreamTaskActionExecutor$1.runThrowing(StreamTaskActionExecutor.java:50) at org.apache.flink.streaming.runtime.tasks.StreamTask.performCheckpoint(StreamTask.java:993) at org.apache.flink.streaming.runtime.tasks.StreamTask.triggerCheckpointOnBarrier(StreamTask.java:951) at org.apache.flink.streaming.runtime.io.CheckpointBarrierHandler.notifyCheckpoint(CheckpointBarrierHandler.java:115) at org.apache.flink.streaming.runtime.io.SingleCheckpointBarrierHandler.processBarrier(SingleCheckpointBarrierHandler.java:156) at org.apache.flink.streaming.runtime.io.CheckpointedInputGate.handleEvent(CheckpointedInputGate.java:178) at org.apache.flink.streaming.runtime.io.CheckpointedInputGate.pollNext(CheckpointedInputGate.java:155) at org.apache.flink.streaming.runtime.io.StreamTaskNetworkInput.emitNext(StreamTaskNetworkInput.java:179) at org.apache.flink.streaming.runtime.io.StreamOneInputProcessor.processInput(StreamOneInputProcessor.java:65) at org.apache.flink.streaming.runtime.tasks.StreamTask.processInput(StreamTask.java:395) at org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.runMailboxLoop(MailboxProcessor.java:191) at org.apache.flink.streaming.runtime.tasks.StreamTask.runMailboxLoop(StreamTask.java:609) at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:573) at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:755) at org.apache.flink.runtime.taskmanager.Task.run(Task.java:570) at java.base/java.lang.Thread.run(Thread.java:834) Caused by: org.apache.kafka.common.errors.TimeoutException: Timeout expired after 6milliseconds while awaiting InitProducerId error_type org.apache.flink.runtime.checkpoint.CheckpointException TASK DETAIL https://phabricator.wikimedia.org/T297870 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: BTullis, elukey, dcausse, Aklapper, MPhamWMF, CBogen, Namenlos314, Gq86, Lucas_Werkmeister_WMDE, EBjune, merbst, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T294076: Blazegraph and MariaDB contain different sitelinks at Wikidata
dcausse merged a task: T295941: WDQS Data drift. dcausse added subscribers: William_Avery, dcausse. TASK DETAIL https://phabricator.wikimedia.org/T294076 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: dcausse, William_Avery, RShigapov, Aklapper, Invadibot, MPhamWMF, maantietaja, CBogen, Akuckartz, Nandana, Namenlos314, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T295941: WDQS Data drift
dcausse closed this task as a duplicate of T294076: Blazegraph and MariaDB contain different sitelinks at Wikidata. TASK DETAIL https://phabricator.wikimedia.org/T295941 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: dcausse, William_Avery, Aklapper, Invadibot, MPhamWMF, maantietaja, CBogen, Akuckartz, Nandana, Namenlos314, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T295941: WDQS Data drift
dcausse added a comment. @William_Avery thanks for the report this is very helpful. I found almost all listed items in the "fetch failures" data set and these will be corrected once we have T279541 <https://phabricator.wikimedia.org/T279541> in place. Remaining ones are: - Q3130156: still has the statement //Hemiergis quadrilineatum// using P31 <https://phabricator.wikimedia.org/P31> - Q5705943: is no longer returned by your query - Q14828500: still has the statement //Acalolepta pseudotincturata// using P31 <https://phabricator.wikimedia.org/P31> - Q21191097: still has the statement //Forpus spengelli// using P31 <https://phabricator.wikimedia.org/P31> I'm closing as a duplicate of T294076 <https://phabricator.wikimedia.org/T294076> because the root cause is identical and will be solved by T279541 <https://phabricator.wikimedia.org/T279541>. TASK DETAIL https://phabricator.wikimedia.org/T295941 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: dcausse, William_Avery, Aklapper, Invadibot, MPhamWMF, maantietaja, CBogen, Akuckartz, Nandana, Namenlos314, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T280485: Additional capacity on the k8s Flink cluster for WCQS updater
dcausse added a comment. small precision: If we reuse the same cluster (same k8s namescape): - it's 3 more pods at 2.1G ram, cpu: 1000m each If we reuse a separate cluster (new k8s namescape): - add a pod at 1.6G, cpu: 500m to the 3 pods mentioned above TASK DETAIL https://phabricator.wikimedia.org/T280485 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Gehel, dcausse Cc: dcausse, akosiaris, Zbyszko, Aklapper, RKemper, Gehel, MPhamWMF, wkandek, JMeybohm, CBogen, Namenlos314, jijiki, Gq86, Lucas_Werkmeister_WMDE, EBjune, merbst, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Dzahn ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T293063: Write and adapt Runbooks and cookbooks related to the WDQS Streaming Updater and kubernetes
dcausse added a comment. In T293063#7491903 <https://phabricator.wikimedia.org/T293063#7491903>, @JMeybohm wrote: > @dcausse IIRC we said that "something in the areas of hours" would be considered a "short maintenance" and thus would not need any additional actions to be carried out, right? We are targeting a SLO with an update lag below 10minutes for 99% of the time, we are still learning what is the operational cost of this and are happy to discuss/re-adjust all this depending on your constraints. > As part of T251305 <https://phabricator.wikimedia.org/T251305> we will re-create the helm release of flink in both datacenters (one after the other ofc.) and that would mean flink will be down for a couple of minutes. If my memory and understanding is still intact, the checkpoint/tombstone metadata is not part of the helm release itself (it's in those flink managed configmaps). So it should survive purging and recreating the helm release. Yes if the configmaps are kept flink will just autorestart on its own, regarding lag I'm not worried as already flink restarts on its own from time to time without affecting the 10min lag SLO. > @Jelto has alredy done that for the staging flink release. If you have the chance it would be nice if you could double check that is still working as expected. Checking the logs I see 2 restarts in the last 7 days and both restarts properly restored the job: Nov 3, 2021 @ 15:44:33.739 syslog kubestage1002 Restoring job 095b671d83457ebf4c59166fda7a7055 from Checkpoint 106609 @ 1635954210959 for 095b671d83457ebf4c59166fda7a7055 located at swift://rdf-streaming-updater-staging.thanos-swift/wikidata/checkpoints/095b671d83457ebf4c59166fda7a7055/chk-106609. Nov 4, 2021 @ 13:36:35.097 syslog kubestage1002 Restoring job 095b671d83457ebf4c59166fda7a7055 from Checkpoint 109216 @ 1636032918483 for 095b671d83457ebf4c59166fda7a7055 located at swift://rdf-streaming-updater-staging.thanos-swift/wikidata/checkpoints/095b671d83457ebf4c59166fda7a7055/chk-109216. So, if one of these restarts corresponds to the helm 3 upgrade then I can confirm that it will work properly the production clusters. > Besides that I tried to understand what would be needed to do for a "longer downtime" of k8s and it's not exactly clear to me. Could we have a dedicated section for that on whe wikitech page? IIRC that also needed a change to WQDS itself. Certainly, this task is all about clarifying all this. TASK DETAIL https://phabricator.wikimedia.org/T293063 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: JMeybohm, Jelto, Aklapper, jijiki, dcausse, Invadibot, MPhamWMF, GeminiAgaloos, maantietaja, wkandek, CBogen, Akuckartz, Nandana, Namenlos314, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Addshore, Mbch331, Dzahn ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T279541: Add a reconciliation strategy to the wdqs streaming updater
dcausse claimed this task. dcausse moved this task from Incoming to In Progress on the Discovery-Search (Current work) board. TASK DETAIL https://phabricator.wikimedia.org/T279541 WORKBOARD https://phabricator.wikimedia.org/project/board/1227/ EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: RShigapov, dcausse, Aklapper, Invadibot, MPhamWMF, maantietaja, CBogen, Akuckartz, Nandana, Namenlos314, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T293195: Add MCR slot information to revision-create events
dcausse added a comment. In T293195#7459268 <https://phabricator.wikimedia.org/T293195#7459268>, @Ottomata wrote: > I was about to merge that today but then thought that your suggestion to ensure that properties validate with the additionalProperties stuff would be good to add first. So you could implement that :D :D Sure! I took a look and will amend your patch :) TASK DETAIL https://phabricator.wikimedia.org/T293195 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: odimitrijevic, Cparle, JAllemandou, Milimetric, Aklapper, Ottomata, Pchelolo, dcausse, EChetty, Suran38, Biggs657, Invadibot, Lalamarie69, MPhamWMF, maantietaja, Juan90264, Alter-paule, Beast1978, CBogen, Un1tY, Akuckartz, 4748kitoko, Hook696, Kent7301, holger.knust, joker88john, CucyNoiD, Nandana, Namenlos314, Akovalyov, Gaboe420, Giuliamocci, Cpaulf30, Lahi, Gq86, Af420, Bsandipan, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, Lewizho99, Maathavan, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, terrrydactyl, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, GWicke, Manybubbles, Mbch331, jeremyb ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T293195: Add MCR slot information to revision-create events
dcausse added a comment. This is blocked on https://gerrit.wikimedia.org/r/c/analytics/refinery/source/+/629406 which is required to support the new pattern (additionalProperties + properties). @Ottomata is there anything we could do help unblock the work on your refinery patch? TASK DETAIL https://phabricator.wikimedia.org/T293195 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: odimitrijevic, Cparle, JAllemandou, Milimetric, Aklapper, Ottomata, Pchelolo, dcausse, EChetty, Suran38, Biggs657, Invadibot, Lalamarie69, MPhamWMF, maantietaja, Juan90264, Alter-paule, Beast1978, CBogen, Un1tY, Akuckartz, 4748kitoko, Hook696, Kent7301, holger.knust, joker88john, CucyNoiD, Nandana, Namenlos314, Akovalyov, Gaboe420, Giuliamocci, Cpaulf30, Lahi, Gq86, Af420, Bsandipan, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, Lewizho99, Maathavan, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, terrrydactyl, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, GWicke, Manybubbles, Mbch331, jeremyb ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T294076: Blazegraph and MariaDB contain different sitelinks at Wikidata
dcausse moved this task from In Progress to Waiting on the Discovery-Search (Current work) board. dcausse added a comment. Thanks for the report this is very helpful. In the two updates you mention here were missed by the new updater but both of these were properly identified as problematic and will be resolved once we have the reconciliation strategy (work tracked in T279541 <https://phabricator.wikimedia.org/T279541>) For the record here are the notes regarding these two missed updates: | edit time| item | wikibase truth | old updater wdqs1010 | wdqs eqiad wdqs1009 | wdqs codfw wdqs2008 | in revision-create topic | in mutation topic | in fetch-failure | | 2021-10-11T17:11:00 | Q6766777| 1510811138 | 1510811138 | 1392684585 | 1510811138 | yes | codfw only| none (only in raw for eqiad T294361 <https://phabricator.wikimedia.org/T294361>) | | 2021-10-17T6:19:36 | Q19929406| 1512982605 | deleted | 1512453868 | 1512453868 | yes | no| eqiad| | I'm moving this to waiting while T279541 <https://phabricator.wikimedia.org/T279541> is being worked out so that we have a place future inconsistencies. TASK DETAIL https://phabricator.wikimedia.org/T294076 WORKBOARD https://phabricator.wikimedia.org/project/board/1227/ EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: RShigapov, Aklapper, Invadibot, MPhamWMF, maantietaja, CBogen, Akuckartz, Nandana, Namenlos314, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T279541: Add a reconciliation strategy to the wdqs streaming updater
dcausse added a subtask: T294361: Events missing from event.rdf_streaming_updater_fetch_failure but present in /wmf/data/raw/event/eqiad.rdf-streaming-updater.fetch-failure. TASK DETAIL https://phabricator.wikimedia.org/T279541 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: dcausse, Aklapper, Invadibot, MPhamWMF, maantietaja, CBogen, Akuckartz, Nandana, Namenlos314, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T294361: Events missing from event.rdf_streaming_updater_fetch_failure but present in /wmf/data/raw/event/eqiad.rdf-streaming-updater.fetch-failure
dcausse added a parent task: T279541: Add a reconciliation strategy to the wdqs streaming updater. TASK DETAIL https://phabricator.wikimedia.org/T294361 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: dcausse, Aklapper, EChetty, MPhamWMF, CBogen, 4748kitoko, Namenlos314, Akovalyov, Gq86, Lucas_Werkmeister_WMDE, EBjune, merbst, Jonas, Xmlizer, JAllemandou, terrrydactyl, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, jeremyb ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T294361: Events missing from event.rdf_streaming_updater_fetch_failure but present in /wmf/data/raw/event/eqiad.rdf-streaming-updater.fetch-failure
dcausse created this task.
dcausse added projects: Analytics, Wikidata-Query-Service.
Restricted Application added a subscriber: Aklapper.
TASK DESCRIPTION
While investigating missed updates on the WDQS streaming updater I looked at
our flink side-outputs that record all the failures happening while fetching
the data out of MW.
For instance I found that we missed the revision 1510811138 for the wikidata
entity Q6766777 (edit that occurred at `2021-10-11T17:11:00Z`).
The fetch for this entity had failed because of a 404 hitting MW (due to the
race between the events & mysql replication). This type of failure is recorded
in the topics [eqiad|codfw].rdf-streaming-updater.fetch-failure.
Looking at the raw data I can find this record:
hdfs dfs -text
/wmf/data/raw/event/eqiad.rdf-streaming-updater.fetch-failure/year=2021/month=10/day=11/hour=17/part.task_event_default_1633975518391_41_0.txt.gz
| grep Q6766777 | jq .
{
"meta": {
"domain": "www.wikidata.org",
"dt": "2021-10-11T17:11:00.853232Z",
"stream": "rdf-streaming-updater.fetch-failure"
},
"item": "Q6766777",
"original_ingestion_dt": "2021-10-11T17:11:00.732866Z",
"revision_id": 1510811138,
"original_event_info": {
"dt": "2021-10-11T17:11:00Z",
"$schema": "/mediawiki/revision/create/1.1.0",
"meta": {
"id": "b02f7b80-05d0-4060-9808-bcaef6a2304e",
"dt": "2021-10-11T17:11:00Z",
"stream": "mediawiki.revision-create",
"request_id": "d1297274-7e9e-4de9-a9f4-63d86d3b566a",
"domain": "www.wikidata.org"
}
},
"op_type": "diff",
"from_revision_id": 1392684585,
"exception_type":
"org.wikidata.query.rdf.tool.wikibase.WikibaseEntityFetchException",
"exception_msg": "Cannot fetch entity at
https://www.wikidata.org/wiki/Special:EntityData/Q6766777.ttl?flavor=dump=1510811138:
ENTITY_NOT_FOUND",
"fetch_error_type": "ENTITY_NOT_FOUND",
"dt": "2021-10-11T17:11:00.853125Z",
"$schema": "/rdf_streaming_updater/fetch_failure/1.0.0"
}
But looking at the refined table I can't find this same record:
select item, meta.dt from event.rdf_streaming_updater_fetch_failure where
year = 2021 and month = 10 and day = 11 and item = "Q6766777";
finds nothing.
TASK DETAIL
https://phabricator.wikimedia.org/T294361
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: dcausse
Cc: dcausse, Aklapper, EChetty, MPhamWMF, CBogen, 4748kitoko, Namenlos314,
Akovalyov, Gq86, Lucas_Werkmeister_WMDE, EBjune, merbst, Jonas, Xmlizer,
JAllemandou, terrrydactyl, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984,
Manybubbles, jeremyb
___
Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org
To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T294076: Blazegraph and MariaDB contain different sitelinks at Wikidata
dcausse claimed this task. dcausse moved this task from Ready for Development to In Progress on the Discovery-Search (Current work) board. TASK DETAIL https://phabricator.wikimedia.org/T294076 WORKBOARD https://phabricator.wikimedia.org/project/board/1227/ EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: RShigapov, Aklapper, Invadibot, MPhamWMF, maantietaja, CBogen, Akuckartz, Nandana, Namenlos314, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T294133: Expose rdf-streaming-updater.mutation content through EventStreams
dcausse added a subscriber: Ottomata. dcausse added a project: EventStreams. dcausse updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T294133 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: Ottomata, Aklapper, dcausse, MPhamWMF, RBrounley_WMF, CBogen, Namenlos314, Gq86, Xinbenlv, Vacio, Lucas_Werkmeister_WMDE, EBjune, merbst, Jonas, Xmlizer, Nirmos, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Krenair ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T294133: Expose rdf-streaming-updater.mutation content through EventStreams
dcausse updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T294133 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: Aklapper, dcausse, MPhamWMF, CBogen, Namenlos314, Gq86, Lucas_Werkmeister_WMDE, EBjune, merbst, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T294133: Expose rdf-streaming-updater.mutation content through EventStreams
dcausse created this task. dcausse added a project: Wikidata-Query-Service. Restricted Application added a subscriber: Aklapper. TASK DESCRIPTION As a consumer of the wikidata content I want to be able to have access the same RDF data the WMF WDQS servers use to perform their live updates so that I can keep my own replica of the wikidata query service (or another RDF store) up to date more easily. A solution might be to use the EventStreams <https://wikitech.wikimedia.org/wiki/Event_Platform/EventStreams> service. Note on the stream: It was decided to go fully active/active for the flink application powering the WDQS updater. Which means the complete stream of changes is available in both topics: - eqiad.rdf-streaming-updater.mutation - codfw.rdf-streaming-updater.mutation It is sligthly different to what we currently see in our topic topology where if you want to have a complete view of the data you need to consume both eqiad.topic and codfw.topic. Here you must consume only one. AC: - RDF data is exposed through EventStreams - A java client is offered for third parties to use with store comptatible with the SPARQL 1.1 Update operations <https://www.w3.org/TR/2013/REC-sparql11-protocol-20130321/#update-operation> TASK DETAIL https://phabricator.wikimedia.org/T294133 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: Aklapper, dcausse, MPhamWMF, CBogen, Namenlos314, Gq86, Lucas_Werkmeister_WMDE, EBjune, merbst, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T244590: [Epic] Rework the WDQS updater as an event driven application
dcausse closed subtask T266321: Determine flink metrics configuration and backend when running from k8s as Resolved. TASK DETAIL https://phabricator.wikimedia.org/T244590 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: Mohammed_Sadat_WMDE, So9q, Lydia_Pintscher, VladimirAlexiev, karapayneWMDE, MPhamWMF, Daniel_Mietchen, Thadguidry, tfmorris, revi, Ladsgroup, Multichill, darthmon_wmde, Iamamz3, Smalyshev, Ottomata, JAllemandou, Aklapper, Zbyszko, Gehel, dcausse, Suran38, Invadibot, maantietaja, Peteosx1x, NavinRizwi, CBogen, Akuckartz, Nandana, Namenlos314, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Dinoguy1000, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T266321: Determine flink metrics configuration and backend when running from k8s
dcausse closed this task as "Resolved". dcausse claimed this task. dcausse added a comment. updater specific metrics are available here: https://grafana-rw.wikimedia.org/d/fdU5Zx-Mk/wdqs-streaming-updater?orgId=1 flink specific metrics are available here: https://grafana-rw.wikimedia.org/d/gCFgfpG7k/flink-session-cluster?orgId=1 TASK DETAIL https://phabricator.wikimedia.org/T266321 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: dcausse, Aklapper, Invadibot, MPhamWMF, maantietaja, CBogen, Akuckartz, Nandana, Namenlos314, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
