[Wikidata-bugs] [Maniphest] T357101: CVE-2024-34502: Special:MergeLexemes makes edits on GET requests without edit tokens
Mstyles renamed this task from "Special:MergeLexemes makes edits on GET requests without edit tokens" to "CVE-2024-34502: Special:MergeLexemes makes edits on GET requests without edit tokens". TASK DETAIL https://phabricator.wikimedia.org/T357101 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Lucas_Werkmeister_WMDE, Mstyles Cc: Lydia_Pintscher, gerritbot, jnuche, Mstyles, sbassett, mmartorana, Aklapper, Lucas_Werkmeister_WMDE, Danny_Benjafield_WMDE, S8321414, Cleo_Lemoisson, Astuthiodit_1, karapayneWMDE, Invadibot, Dylsss, Devnull, maantietaja, ItamarWMDE, Akuckartz, Dringsim, DannyS712, Nandana, Lahi, Gq86, GoranSMilovanovic, Mahir256, QZanden, KimKelting, LawExplorer, JJMC89, _jensen, rosalieper, Scott_WUaS, Wong128hk, Luke081515, Wikidata-bugs, aude, Bawolff, Grunny, csteipp, Mbch331, Jay8g, Krenair, Legoktm ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T345064: CVE-2023-45371: Merging items is not rate limited and only partially protected by AbuseFilter
Mstyles changed the visibility from "Custom Policy" to "Public (No Login Required)". Mstyles changed the edit policy from "Custom Policy" to "All Users". TASK DETAIL https://phabricator.wikimedia.org/T345064 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Lucas_Werkmeister_WMDE, Mstyles Cc: roti_WMDE, gerritbot, Ifrahkhanyaree_WMDE, WMDE-leszek, Lydia_Pintscher, Michael, Aklapper, Lucas_Werkmeister_WMDE, Danny_Benjafield_WMDE, Isabelladantes1983, Themindcoder, Adamm71, Jersione, Hellket777, Cleo_Lemoisson, LisafBia6531, Astuthiodit_1, 786, Biggs657, karapayneWMDE, Invadibot, Dylsss, Devnull, maantietaja, Juan90264, Alter-paule, Beast1978, ItamarWMDE, Un1tY, Akuckartz, Hook696, Kent7301, joker88john, DannyS712, CucyNoiD, Nandana, sbassett, Gaboe420, lucamauri, Giuliamocci, Cpaulf30, Lahi, Gq86, Af420, Bsandipan, GoranSMilovanovic, QZanden, LawExplorer, Lewizho99, Maathavan, _jensen, rosalieper, Neuronton, Scott_WUaS, Wong128hk, Luke081515, Wikidata-bugs, aude, Bawolff, Grunny, csteipp, Mbch331, Jay8g, Krenair, Legoktm ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T339111: CVE-2023-37302: Style injection into badges on Wikidata due to unescaped quotes
Mstyles closed this task as "Resolved". Mstyles claimed this task. TASK DETAIL https://phabricator.wikimedia.org/T339111 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Mstyles Cc: Mstyles, gerritbot, Tarrow, lojo_wmde, adee_wmde, roti_WMDE, Tobi_WMDE_SW, daniel, Krinkle, thiemowmde, sbassett, Lucas_Werkmeister_WMDE, Lydia_Pintscher, Arian_Bozorg, Aklapper, Michael, Cleo_Lemoisson, Astuthiodit_1, karapayneWMDE, Invadibot, Dylsss, Devnull, maantietaja, Y.ssk, ItamarWMDE, Akuckartz, DannyS712, Nandana, Lahi, Gq86, GoranSMilovanovic, QZanden, LawExplorer, _jensen, rosalieper, Scott_WUaS, Wong128hk, Luke081515, Wikidata-bugs, aude, Bawolff, Grunny, He7d3r, csteipp, Mbch331, Jay8g, Krenair, Legoktm ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T250720: CVE-2023-37301: Wikidata edit filter does not fire when test tool says it should
Mstyles changed the visibility from "All Users" to "Public (No Login Required)". TASK DETAIL https://phabricator.wikimedia.org/T250720 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Lucas_Werkmeister_WMDE, Mstyles Cc: darthmon_wmde, gerritbot, brennen, hashar, WMDE-leszek, roti_WMDE, lojo_wmde, adee_wmde, Fring, hoo, sbassett, noarave, Michael, Arian_Bozorg, ItamarWMDE, karapayneWMDE, Lucas_Werkmeister_WMDE, DannyS712, matej_suchanek, Ahmad252, Daimona, Aklapper, Bovlb, Isabelladantes1983, Themindcoder, Adamm71, Jersione, jdfraine, Hellket777, Cleo_Lemoisson, LisafBia6531, Astuthiodit_1, 786, Biggs657, Invadibot, Dylsss, Devnull, maantietaja, Juan90264, Alter-paule, Beast1978, Un1tY, Akuckartz, Hook696, Kent7301, joker88john, 94rain, CucyNoiD, Nandana, Gaboe420, Giuliamocci, Cpaulf30, Lahi, Gq86, Af420, Bsandipan, GoranSMilovanovic, ihojose, QZanden, LawExplorer, Litanwei, Lewizho99, JJMC89, Maathavan, _jensen, rosalieper, Neuronton, Scott_WUaS, Wong128hk, Luke081515, Nirmos, Wikidata-bugs, aude, Nullzero, Huji, Bawolff, Grunny, Billinghurst, Jackmcbarn, csteipp, Mbch331, Jay8g, Krenair, Legoktm ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T339111: CVE-2023-37302: Style injection into badges on Wikidata due to unescaped quotes
Mstyles renamed this task from "Style injection into badges on Wikidata due to unescaped quotes" to "CVE-2023-37302: Style injection into badges on Wikidata due to unescaped quotes". Mstyles changed the visibility from "Custom Policy" to "Public (No Login Required)". Mstyles changed the edit policy from "Custom Policy" to "All Users". TASK DETAIL https://phabricator.wikimedia.org/T339111 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Mstyles Cc: Mstyles, gerritbot, Tarrow, lojo_wmde, adee_wmde, roti_WMDE, Tobi_WMDE_SW, daniel, Krinkle, thiemowmde, sbassett, Lucas_Werkmeister_WMDE, Lydia_Pintscher, Arian_Bozorg, Aklapper, Michael, Cleo_Lemoisson, Astuthiodit_1, karapayneWMDE, Invadibot, Dylsss, Devnull, maantietaja, Y.ssk, ItamarWMDE, Akuckartz, DannyS712, Nandana, Lahi, Gq86, GoranSMilovanovic, QZanden, LawExplorer, _jensen, rosalieper, Scott_WUaS, Wong128hk, Luke081515, Wikidata-bugs, aude, Bawolff, Grunny, He7d3r, csteipp, Mbch331, Jay8g, Krenair, Legoktm, RhinosF1 ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T250720: CVE-2023-37301: Wikidata edit filter does not fire when test tool says it should
Mstyles renamed this task from "Wikidata edit filter does not fire when test tool says it should" to "CVE-2023-37301: Wikidata edit filter does not fire when test tool says it should". Mstyles changed the visibility from "Custom Policy" to "All Users". TASK DETAIL https://phabricator.wikimedia.org/T250720 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Lucas_Werkmeister_WMDE, Mstyles Cc: darthmon_wmde, gerritbot, brennen, hashar, WMDE-leszek, roti_WMDE, lojo_wmde, adee_wmde, Fring, hoo, sbassett, noarave, Michael, Arian_Bozorg, ItamarWMDE, karapayneWMDE, Lucas_Werkmeister_WMDE, DannyS712, matej_suchanek, Ahmad252, Daimona, Aklapper, Bovlb, Isabelladantes1983, Themindcoder, Adamm71, Jersione, jdfraine, Hellket777, Cleo_Lemoisson, LisafBia6531, Astuthiodit_1, 786, Biggs657, Invadibot, Dylsss, Devnull, maantietaja, Juan90264, Alter-paule, Beast1978, Un1tY, Akuckartz, Hook696, Kent7301, joker88john, 94rain, CucyNoiD, Nandana, Gaboe420, Giuliamocci, Cpaulf30, Lahi, Gq86, Af420, Bsandipan, GoranSMilovanovic, ihojose, QZanden, LawExplorer, Litanwei, Lewizho99, JJMC89, Maathavan, _jensen, rosalieper, Neuronton, Scott_WUaS, Wong128hk, Luke081515, Nirmos, Wikidata-bugs, aude, Nullzero, Huji, Bawolff, Grunny, Billinghurst, Jackmcbarn, csteipp, Mbch331, Jay8g, Krenair, Legoktm, RhinosF1 ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T323592: XSS in Wikibase date formatting
Mstyles closed this task as "Resolved". Mstyles claimed this task. Mstyles added a comment. I'm closing it since everything has been merged. Thanks so much! TASK DETAIL https://phabricator.wikimedia.org/T323592 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Mstyles Cc: Mstyles, sbassett, Jdforrester-WMF, WMDE-leszek, Michael, Aklapper, Lucas_Werkmeister_WMDE, Adamm71, Jersione, Hellket777, LisafBia6531, Astuthiodit_1, 786, Biggs657, karapayneWMDE, Invadibot, Devnull, maantietaja, Y.ssk, Juan90264, Alter-paule, Beast1978, ItamarWMDE, Un1tY, Akuckartz, Hook696, Kent7301, joker88john, DannyS712, CucyNoiD, Nandana, Gaboe420, Giuliamocci, Cpaulf30, Lahi, Gq86, Af420, Bsandipan, GoranSMilovanovic, QZanden, LawExplorer, Lewizho99, Maathavan, _jensen, rosalieper, Neuronton, Scott_WUaS, Wong128hk, Luke081515, Wikidata-bugs, aude, Bawolff, Grunny, He7d3r, csteipp, Mbch331, Jay8g, Krenair, Legoktm ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T323592: XSS in Wikibase date formatting
Mstyles changed the visibility from "Custom Policy" to "Public (No Login Required)". Mstyles changed the edit policy from "Custom Policy" to "All Users". TASK DETAIL https://phabricator.wikimedia.org/T323592 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Mstyles Cc: sbassett, Jdforrester-WMF, dancy, WMDE-leszek, Michael, Aklapper, Lucas_Werkmeister_WMDE, Adamm71, Jersione, Hellket777, LisafBia6531, Astuthiodit_1, 786, Biggs657, karapayneWMDE, Invadibot, Devnull, maantietaja, Y.ssk, Juan90264, Alter-paule, Beast1978, ItamarWMDE, Un1tY, Akuckartz, Hook696, Kent7301, joker88john, DannyS712, CucyNoiD, Nandana, Gaboe420, Giuliamocci, Cpaulf30, Lahi, Gq86, Af420, Bsandipan, GoranSMilovanovic, QZanden, LawExplorer, Lewizho99, Maathavan, _jensen, rosalieper, Neuronton, Scott_WUaS, Wong128hk, Luke081515, Wikidata-bugs, aude, Bawolff, Grunny, He7d3r, csteipp, Mbch331, Jay8g, Krenair, Legoktm, RhinosF1 ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T273098: High Availability Flink
Mstyles added a comment. So one problem with implementing HA without using the session cluster (T280166 <https://phabricator.wikimedia.org/T280166>) is that Flink can't properly start the Streaming Updater job locally, since the jar needs to be stored in the storage directory. I'm not sure of a workaround if we don't end up using the session cluster for local/ci use cases. TASK DETAIL https://phabricator.wikimedia.org/T273098 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Mstyles Cc: Mstyles, dcausse, JMeybohm, jijiki, Aklapper, Gehel, akosiaris, Invadibot, MPhamWMF, maantietaja, Alter-paule, Beast1978, CBogen, Un1tY, Akuckartz, Hook696, Kent7301, joker88john, CucyNoiD, Nandana, Namenlos314, Gaboe420, Giuliamocci, Cpaulf30, Lahi, Gq86, Af420, Bsandipan, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, Lewizho99, Maathavan, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T273098: High Availability Flink
Mstyles added a comment. 3 config maps are created by the pods, `dispatcher-leader`, `resourcemanager-leader`, and `restserver-leader`. There aren't any references from helm to these configmaps, so when helm is updated, it shouldn't affect the configmaps. TASK DETAIL https://phabricator.wikimedia.org/T273098 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Mstyles Cc: Mstyles, dcausse, JMeybohm, jijiki, Aklapper, Gehel, akosiaris, Invadibot, MPhamWMF, maantietaja, CBogen, Akuckartz, Nandana, Namenlos314, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T280166: Investigate using session cluster for Flink
Mstyles created this task. Mstyles added projects: Wikidata, Wikidata-Query-Service, Discovery-Search (Current work), Patch-For-Review. Restricted Application removed a project: Patch-For-Review. TASK DESCRIPTION Different Kubernetes Deployment Modes Docs -> https://ci.apache.org/projects/flink/flink-docs-release-1.12/deployment/resource-providers/standalone/kubernetes.html#deployment-modes Currently the plan is deploy Flink using the Deploy Application Cluster which runs a single application. This means the streaming updater jar is bundled in the Flink image and that stopping the streaming updater job in Flink stops the entire Flink application. This makes it difficult for the Flink job to be stopped with a savepoint, as the Flink application cannot stay alive without the running Streaming Updater job. It was suggested to use the Session Cluster, which would allow us to stop the job cleanly without stopping the entire Flink Application. This would allow any jobs to be stopped with savepoints (via the API or UI) and also started again from savepoints (again via API or UI). This would also allow multiple jobs to uploaded to one session cluster so that the WCQS (commons query service) updater could be run on the same Flink session cluster. Jars are uploaded to the session cluster, and in Flink HA mode, they are saved in Swift. In the event that the Session cluster is shut down by SRE, any running jobs will be still resumed using the state stored in the HA configmaps. TASK DETAIL https://phabricator.wikimedia.org/T280166 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Mstyles Cc: dcausse, JMeybohm, jijiki, Aklapper, Gehel, akosiaris, Mstyles, Invadibot, MPhamWMF, maantietaja, CBogen, Akuckartz, Nandana, Namenlos314, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T278385: Streaming Updater must make all requests to proxy endpoints
Mstyles added a comment. Thanks! TASK DETAIL https://phabricator.wikimedia.org/T278385 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Mstyles Cc: dcausse, JMeybohm, jijiki, Aklapper, Gehel, akosiaris, Mstyles, Invadibot, MPhamWMF, maantietaja, Alter-paule, Beast1978, CBogen, Un1tY, Akuckartz, Hook696, Kent7301, joker88john, CucyNoiD, Nandana, Namenlos314, Gaboe420, Giuliamocci, Cpaulf30, Lahi, Gq86, Af420, Bsandipan, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, Lewizho99, Maathavan, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T278385: Streaming Updater must make all requests to proxy endpoints
Mstyles added a subscriber: dcausse. Mstyles added a comment. @akosiaris the swift proxies in there seem are pointing to the swift cluster that we use (thano-swift). We'll need additional proxies set up for that as well. Would it be more helpful to have a separate ticket for the proxies that need to be created? (meta.wikimedia.org and thanos-swift). Additionally, looking at the list of proxies from envoy, I didn't see anything for wikidata. Am I missing something? @dcausse also wanted to know if there can be a `api-ro.discovery.wmnet` proxy in addition to `api-rw.discovery.wmnet`, since we only read from Mediawiki. TASK DETAIL https://phabricator.wikimedia.org/T278385 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Mstyles Cc: dcausse, JMeybohm, jijiki, Aklapper, Gehel, akosiaris, Mstyles, Invadibot, MPhamWMF, maantietaja, Alter-paule, Beast1978, CBogen, Un1tY, Akuckartz, Hook696, Kent7301, joker88john, CucyNoiD, Nandana, Namenlos314, Gaboe420, Giuliamocci, Cpaulf30, Lahi, Gq86, Af420, Bsandipan, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, Lewizho99, Maathavan, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T278385: Streaming Updater must make all requests to proxy endpoints
Mstyles added a comment. I wanted to clarify for Swift as well, will there be proxies to connect to the swift auth url and connecting the the thanos swift cluster? TASK DETAIL https://phabricator.wikimedia.org/T278385 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Mstyles Cc: JMeybohm, jijiki, Aklapper, Gehel, akosiaris, Mstyles, Invadibot, MPhamWMF, maantietaja, Alter-paule, Beast1978, CBogen, Un1tY, Akuckartz, Hook696, Kent7301, joker88john, CucyNoiD, Nandana, Namenlos314, Gaboe420, Giuliamocci, Cpaulf30, Lahi, Gq86, Af420, Bsandipan, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, Lewizho99, Maathavan, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T278385: Streaming Updater must make all requests to proxy endpoints
Mstyles created this task. Mstyles added projects: Wikidata, Wikidata-Query-Service, Discovery-Search (Current work), Patch-For-Review. Restricted Application removed a project: Patch-For-Review. TASK DESCRIPTION All applications deployed in the Kubernetes cluster must make their requests through proxies. Currently the streaming updater makes requests to the Mediawiki API (eventgate), Metawiki (schemas), and Wikibase. It seems that event utilities library we use also makes some requests to the Mediawiki API and it's not clear how to change those URLs. TASK DETAIL https://phabricator.wikimedia.org/T278385 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Mstyles Cc: JMeybohm, jijiki, Aklapper, Gehel, akosiaris, Mstyles, Invadibot, MPhamWMF, maantietaja, CBogen, Akuckartz, Nandana, Namenlos314, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T273098: High Availability Flink
Mstyles edited subscribers, added: jijiki, JMeybohm; removed: Mstyles. Mstyles added a comment. The changes for HA are mostly changing a few lines of the config file. The part that is unclear is whether we can get a service account for pods that allow them to do CRUD operations for configmaps. Pods will need to be able to update the configmap. Details here <https://ci.apache.org/projects/flink/flink-docs-release-1.12/deployment/ha/kubernetes_ha.html#prerequisites>. Another thing that would have to change is that JobManager pods should be started with their IP address instead of a Kubernetes service as its jobmanager.rpc.address. I'm not sure how that would work. The alternative to updating configmaps is to run a separate zookeeper cluster, more on that here <https://ci.apache.org/projects/flink/flink-docs-release-1.12/deployment/ha/zookeeper_ha.html> TASK DETAIL https://phabricator.wikimedia.org/T273098 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Mstyles Cc: JMeybohm, jijiki, Aklapper, Gehel, akosiaris, Invadibot, MPhamWMF, maantietaja, CBogen, Akuckartz, Nandana, Namenlos314, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T264006: Deploy Flink to kubernetes (k8s)
Mstyles updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T264006 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Mstyles Cc: akosiaris, Mstyles, Gehel, Aklapper, MPhamWMF, maantietaja, Alter-paule, Beast1978, CBogen, Un1tY, Akuckartz, Hook696, Kent7301, joker88john, CucyNoiD, Nandana, Namenlos314, Gaboe420, Giuliamocci, Cpaulf30, Lahi, Gq86, Af420, Bsandipan, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, Lewizho99, Maathavan, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, abian, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T264006: Deploy Flink to kubernetes (k8s)
Mstyles updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T264006 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Mstyles Cc: akosiaris, Mstyles, Gehel, Aklapper, MPhamWMF, maantietaja, Alter-paule, Beast1978, CBogen, Un1tY, Akuckartz, Hook696, Kent7301, joker88john, CucyNoiD, Nandana, Namenlos314, Gaboe420, Giuliamocci, Cpaulf30, Lahi, Gq86, Af420, Bsandipan, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, Lewizho99, Maathavan, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, abian, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T273098: High Availability Flink
Mstyles claimed this task. TASK DETAIL https://phabricator.wikimedia.org/T273098 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Mstyles Cc: Aklapper, Gehel, akosiaris, Mstyles, MPhamWMF, maantietaja, CBogen, Akuckartz, Nandana, Namenlos314, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, abian, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T264006: Deploy Flink to kubernetes (k8s)
Mstyles added a subtask: Restricted Task. TASK DETAIL https://phabricator.wikimedia.org/T264006 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Mstyles Cc: akosiaris, Mstyles, Gehel, Aklapper, MPhamWMF, maantietaja, Alter-paule, Beast1978, CBogen, Un1tY, Akuckartz, Hook696, Kent7301, joker88john, CucyNoiD, Nandana, Namenlos314, Gaboe420, Giuliamocci, Cpaulf30, Lahi, Gq86, Af420, Bsandipan, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, Lewizho99, Maathavan, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, abian, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T273095: Deploy Helm Chart
Mstyles added a comment. currently blocked by a helm test issue described in https://phabricator.wikimedia.org/T277252 TASK DETAIL https://phabricator.wikimedia.org/T273095 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Mstyles Cc: Aklapper, Gehel, akosiaris, Mstyles, MPhamWMF, maantietaja, Alter-paule, Beast1978, CBogen, Un1tY, Akuckartz, Hook696, Kent7301, joker88john, CucyNoiD, Nandana, Namenlos314, Gaboe420, Giuliamocci, Cpaulf30, Lahi, Gq86, Af420, Bsandipan, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, Lewizho99, Maathavan, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, abian, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T276131: Fix Serialization Bug
Mstyles created this task. Mstyles added projects: Wikidata, Wikidata-Query-Service, Discovery-Search (Current work). TASK DESCRIPTION The latest version of the Streaming Updater Producer has a bug that prevents Flink from starting. Relevant part of stack trace at org.apache.flink.streaming.connectors.kafka.FlinkKafkaProducer.(FlinkKafkaProducer.java:571) at org.wikidata.query.rdf.updater.OutputStreams.prepareSideOutputStream(OutputStreams.scala:63) at org.wikidata.query.rdf.updater.OutputStreams.lateEventsOutput(OutputStreams.scala:37) at org.wikidata.query.rdf.updater.OutputStreams.pipelineSinks(OutputStreams.scala:56) at org.wikidata.query.rdf.updater.UpdaterJob$.main(UpdaterJob.scala:27) at org.wikidata.query.rdf.updater.UpdaterJob.main(UpdaterJob.scala) This is a blocker to deployment of the streaming updater TASK DETAIL https://phabricator.wikimedia.org/T276131 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Mstyles Cc: Aklapper, Gehel, akosiaris, Mstyles, MPhamWMF, CBogen, Akuckartz, Nandana, Namenlos314, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, abian, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T273095: Deploy Helm Chart
Mstyles claimed this task. TASK DETAIL https://phabricator.wikimedia.org/T273095 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Mstyles Cc: Aklapper, Gehel, akosiaris, Mstyles, MPhamWMF, CBogen, Akuckartz, Nandana, Namenlos314, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T275196: Add the IDSM/Sachem Wikidata endpoint to the list of federated endpoints
Mstyles claimed this task. Mstyles moved this task from Ready for Development to Needs review on the Discovery-Search (Current work) board. TASK DETAIL https://phabricator.wikimedia.org/T275196 WORKBOARD https://phabricator.wikimedia.org/project/board/1227/ EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Mstyles Cc: Aklapper, Galgonek, MPhamWMF, Alter-paule, Beast1978, CBogen, Un1tY, Akuckartz, Hook696, Kent7301, joker88john, CucyNoiD, Nandana, Namenlos314, Gaboe420, Giuliamocci, Cpaulf30, Lahi, Gq86, Af420, Bsandipan, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, Lewizho99, Maathavan, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T272265: Resolve WDQS changelog situation
Mstyles claimed this task. Mstyles moved this task from Ready for Development to In Progress on the Discovery-Search (Current work) board. TASK DETAIL https://phabricator.wikimedia.org/T272265 WORKBOARD https://phabricator.wikimedia.org/project/board/1227/ EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Mstyles Cc: Gehel, Addshore, Aklapper, Zbyszko, MPhamWMF, CBogen, Akuckartz, Nandana, Namenlos314, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T264006: Deploy Flink to kubernetes (k8s)
Mstyles closed subtask T266495: Create Debian Package for Flink as "Declined". TASK DETAIL https://phabricator.wikimedia.org/T264006 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Mstyles Cc: akosiaris, Mstyles, Gehel, Aklapper, MPhamWMF, Alter-paule, Beast1978, CBogen, Un1tY, Akuckartz, Hook696, Kent7301, joker88john, CucyNoiD, Nandana, Namenlos314, Gaboe420, Giuliamocci, Cpaulf30, Lahi, Gq86, Af420, Bsandipan, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, Lewizho99, Maathavan, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T266495: Create Debian Package for Flink
Mstyles closed this task as "Declined". TASK DETAIL https://phabricator.wikimedia.org/T266495 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Mstyles Cc: dcausse, elukey, EBernhardson, Aklapper, Gehel, Mstyles, Ottomata, MPhamWMF, CBogen, Akuckartz, Nandana, Namenlos314, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T273097: Create Flink Base Image
Mstyles moved this task from Ready for Development to Needs Reporting on the Discovery-Search (Current work) board. Mstyles added a comment. Not being done since no longer needed TASK DETAIL https://phabricator.wikimedia.org/T273097 WORKBOARD https://phabricator.wikimedia.org/project/board/1227/ EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Mstyles Cc: RKemper, dcausse, Aklapper, Gehel, akosiaris, Mstyles, MPhamWMF, CBogen, Akuckartz, Nandana, Namenlos314, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T273097: Create Flink Base Image
Mstyles added a comment. @akosiaris I just learned that there are archive links that have all of the Flink packages. I'm proposing that we close both this ticket and https://phabricator.wikimedia.org/T266495 and just use the Flink archive links where we won't have to worry about the packages no longer being available. TASK DETAIL https://phabricator.wikimedia.org/T273097 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Mstyles Cc: RKemper, dcausse, Aklapper, Gehel, akosiaris, Mstyles, MPhamWMF, CBogen, Akuckartz, Nandana, Namenlos314, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T266495: Create Debian Package for Flink
Mstyles updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T266495 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Mstyles Cc: dcausse, elukey, EBernhardson, Aklapper, Gehel, Mstyles, Ottomata, MPhamWMF, CBogen, Akuckartz, Nandana, Namenlos314, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T273097: Create Flink Base Image
Mstyles added a subscriber: dcausse. Mstyles added a comment. The problem is that we need more than just the jars. We need the entire tar file. @dcausse suggested people.wikimedia.org but that was considered not a good place for CI to hit TASK DETAIL https://phabricator.wikimedia.org/T273097 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Mstyles Cc: dcausse, Aklapper, Gehel, akosiaris, Mstyles, MPhamWMF, CBogen, Akuckartz, Nandana, Namenlos314, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T273097: Create Flink Base Image
Mstyles renamed this task from "Create Flink BaseImage" to "Create Flink Base Image". TASK DETAIL https://phabricator.wikimedia.org/T273097 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Mstyles Cc: Aklapper, Gehel, akosiaris, Mstyles, MPhamWMF, CBogen, Akuckartz, Nandana, Namenlos314, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T273097: Create Flink BaseImage
Mstyles renamed this task from "Create Base Flink Image" to "Create Flink BaseImage". TASK DETAIL https://phabricator.wikimedia.org/T273097 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Mstyles Cc: Aklapper, Gehel, akosiaris, Mstyles, MPhamWMF, CBogen, Akuckartz, Nandana, Namenlos314, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T273095: Deploy Helm Chart
Mstyles updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T273095 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Mstyles Cc: Aklapper, Gehel, akosiaris, Mstyles, MPhamWMF, CBogen, Akuckartz, Nandana, Namenlos314, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T273098: High Availability Flink
Mstyles created this task. Mstyles added projects: Wikidata, Wikidata-Query-Service, Discovery-Search (Current work), Patch-For-Review. Restricted Application removed a project: Patch-For-Review. TASK DESCRIPTION With the latest updated of Flink (1.12), High Availability is now supported for Flink running in Kubernetes. The documentation <https://ci.apache.org/projects/flink/flink-docs-release-1.12/deployment/resource-providers/standalone/kubernetes.html#high-availability-with-standalone-kubernetes> looks very straightforward. AC Upgrade our Flink setup to use HA mode TASK DETAIL https://phabricator.wikimedia.org/T273098 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Mstyles Cc: Aklapper, Gehel, akosiaris, Mstyles, MPhamWMF, CBogen, Akuckartz, Nandana, Namenlos314, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T273097: Create Base Flink Image
Mstyles created this task. Mstyles added projects: Wikidata, Wikidata-Query-Service, Discovery-Search (Current work), Patch-For-Review. Restricted Application removed a project: Patch-For-Review. TASK DESCRIPTION This is an alternative to https://phabricator.wikimedia.org/T266495. Flink release updates quite frequently and every time a new update is released, the previous versions are removed from the download servers. This means that when we do unrelated changes to our streaming updater, that also results in a Flink version upgrade that we might or might not want. Having a base image (or a debian package) allows us to control what version of Flink we use. Base images are controlled by SRE but anyone can make patches (I think). The repo is here <https://gerrit.wikimedia.org/r/admin/repos/operations/docker-images/production-images>. The flink-rdf-streaming-updater blubberfile would then need to be updated to reference the base image once it's merged. Swift will remain in in the flink-rdf-streaming-updater project as it is specific to that use case. The downside of having a base image owned by SRE means that we are reliant on them to merge any Flink version updates. AC flink-rdf-streaming-updater project uses Flink base image from production images TASK DETAIL https://phabricator.wikimedia.org/T273097 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Mstyles Cc: Aklapper, Gehel, akosiaris, Mstyles, MPhamWMF, CBogen, Akuckartz, Nandana, Namenlos314, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T273095: Deploy Helm Chart
Mstyles updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T273095 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Mstyles Cc: Aklapper, Gehel, akosiaris, Mstyles, MPhamWMF, CBogen, Akuckartz, Nandana, Namenlos314, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T273095: Deploy Helm Chart
Mstyles created this task. Mstyles added projects: Wikidata, Wikidata-Query-Service, Discovery-Search (Current work), Patch-For-Review. Restricted Application removed a project: Patch-For-Review. TASK DESCRIPTION In order for the helm chart to be deployed, the helmfile.d structure needs to be created. Deploy is then done manually on the deploy servers. That process is described here <https://wikitech.wikimedia.org/wiki/Deployments_on_kubernetes#Code_deployment/configuration_changes>. The k8s tokens and namespaces will also need to be created by SRE ServiceOps. This ticket is waiting on https://phabricator.wikimedia.org/T269619 to move forward AC Flink is deployed to production TASK DETAIL https://phabricator.wikimedia.org/T273095 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Mstyles Cc: Aklapper, Gehel, akosiaris, Mstyles, MPhamWMF, CBogen, Akuckartz, Nandana, Namenlos314, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T265526: Create Helm Chart
Mstyles updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T265526 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Mstyles Cc: Mirahamira, Aklapper, Gehel, Mstyles, MPhamWMF, CBogen, Akuckartz, Nandana, Namenlos314, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T265526: Create Helm Chart
Mstyles added a comment. Updating this task to remove the helmfile requirement. That is more of a final step that doesn't include the helm chart. The benchmarking that's been highlighted is not possible on a local machine, we will need to do proper load testing in a staging environment. Logs and monitoring are addressed in the current helm chart, so I'm going to go ahead and mark this ticket as done. TASK DETAIL https://phabricator.wikimedia.org/T265526 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Mstyles Cc: Mirahamira, Aklapper, Gehel, Mstyles, MPhamWMF, CBogen, Akuckartz, Nandana, Namenlos314, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T269876: Add swift plugin to Flink k8s
Mstyles added a comment. The swift plugin has been added to the docker image and is properly configured in the helm chart. TASK DETAIL https://phabricator.wikimedia.org/T269876 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Mstyles Cc: Aklapper, Gehel, Mstyles, MPhamWMF, CBogen, Akuckartz, Nandana, Namenlos314, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T266495: Create Debian Package for Flink
Mstyles added a comment. In the interim (until we have the deb package) @dcausse suggested putting the Flink packages on people.wikimedia.org since packages on the official Flink download sites get updated and removed pretty frequently. TASK DETAIL https://phabricator.wikimedia.org/T266495 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Mstyles Cc: dcausse, elukey, EBernhardson, Aklapper, Gehel, Mstyles, Ottomata, MPhamWMF, CBogen, Akuckartz, Nandana, Namenlos314, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T272447: Extract a list of the 200 most viewed black historical figures from WDQS
Mstyles added a comment. also @Miriam both queries still return wikipedia pages that do have images TASK DETAIL https://phabricator.wikimedia.org/T272447 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Mstyles Cc: MPhamWMF, CBogen, Miriam, dcausse, Mstyles, Gehel, Aklapper, Akuckartz, Nandana, Namenlos314, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T272447: Extract a list of the 200 most viewed black historical figures from WDQS
Mstyles added a comment. @dcausse and I discussed getting the page view counts from https://pageviews.toolforge.org/?project=en.wikipedia.org&platform=all-access&agent=user&redirects=0&range=latest-20&pages=Cat|Dog unless you have another suggestion/idea @Miriam TASK DETAIL https://phabricator.wikimedia.org/T272447 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Mstyles Cc: MPhamWMF, CBogen, Miriam, dcausse, Mstyles, Gehel, Aklapper, Akuckartz, Nandana, Namenlos314, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T270371: wikimedia-event-utilities should provide tools for JVM based apps producing directly to kafka
Mstyles assigned this task to dcausse. TASK DETAIL https://phabricator.wikimedia.org/T270371 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse, Mstyles Cc: Ottomata, dcausse, Aklapper, MPhamWMF, CBogen, Akuckartz, 4748kitoko, holger.knust, Nandana, Namenlos314, Akovalyov, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, JAllemandou, terrrydactyl, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, GWicke, Manybubbles, Mbch331, jeremyb ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T267825: Outdated data still present in WCQS a month after statement update
Mstyles added a comment. On the wcqs beta host, `curl -d "query=select * { sdc:M8979671 wdt:P571 ?o . }" localhost/bigdata/namespace/wcq/sparql` returns an item with the correct data and `curl -d "query=select * { sdc:M8979671 wdt:P571 ?o . }" localhost:/bigdata/namespace/wcq/sparql` returns the 1010 date TASK DETAIL https://phabricator.wikimedia.org/T267825 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Mstyles Cc: Mstyles, CBogen, JeanFred, Aklapper, GFontenelle_WMF, MPhamWMF, Nintendofan885, Akuckartz, Nandana, JKSTNK, Namenlos314, Lahi, PDrouin-WMF, Gq86, E1presidente, Ramsey-WMF, Cparle, Anooprao, SandraF_WMF, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, Tramullas, Acer, merbst, LawExplorer, Salgo60, Silverfish, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, Susannaanas, Jane023, jkroll, Wikidata-bugs, Jdouglas, Base, matthiasmullie, aude, Tobias1984, Manybubbles, Ricordisamoa, Wesalius, Lydia_Pintscher, Fabrice_Florin, Raymond, Steinsplitter, Mbch331 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T267825: Outdated data still present in WCQS a month after statement update
Mstyles added a comment. The dumps have been investigated and the data is correct there, the problem is definitely on the search team's side. It could be with the reload scripts or nginx. It was possible to see the incorrect data on the wcqs server, so the problem is most likely with nginx. TASK DETAIL https://phabricator.wikimedia.org/T267825 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Mstyles Cc: Mstyles, CBogen, JeanFred, Aklapper, GFontenelle_WMF, MPhamWMF, Nintendofan885, Akuckartz, Nandana, JKSTNK, Namenlos314, Lahi, PDrouin-WMF, Gq86, E1presidente, Ramsey-WMF, Cparle, Anooprao, SandraF_WMF, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, Tramullas, Acer, merbst, LawExplorer, Salgo60, Silverfish, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, Susannaanas, Jane023, jkroll, Wikidata-bugs, Jdouglas, Base, matthiasmullie, aude, Tobias1984, Manybubbles, Ricordisamoa, Wesalius, Lydia_Pintscher, Fabrice_Florin, Raymond, Steinsplitter, Mbch331 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T269876: Add swift plugin to Flink k8s
Mstyles created this task. Mstyles added projects: Wikidata, Wikidata-Query-Service, Discovery-Search (Current work). TASK DESCRIPTION Flink needs somewhere to store checkpoint data. We've decided to go with Swift as discussed here: https://phabricator.wikimedia.org/T246004. Now that the plugin has been released to archiva, we need to include it in the Flink k8 setup Acceptance Criteria Blubberfile downloads plugin from Archiva and installs in plugin directory Helm chart has proper config added for Swift TASK DETAIL https://phabricator.wikimedia.org/T269876 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Mstyles Cc: Aklapper, Gehel, Mstyles, CBogen, Akuckartz, Nandana, Namenlos314, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T265526: Create Helm Chart
Mstyles updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T265526 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Mstyles Cc: Aklapper, Gehel, Mstyles, Alter-paule, Beast1978, CBogen, Un1tY, Akuckartz, Hook696, Kent7301, joker88john, CucyNoiD, Nandana, Namenlos314, Gaboe420, Giuliamocci, Cpaulf30, Lahi, Gq86, Af420, Bsandipan, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, Lewizho99, Maathavan, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T265512: Set up Pipeline Configuration in WDQS repo
Mstyles moved this task from Needs review to Needs Reporting on the Discovery-Search (Current work) board. Mstyles added a comment. I definitely agree that pipeline lib could be used for Java projects, but adding that current functionality with this project is out of scope. We are moving forward with downloading the jar from archiva as discussed in the meeting on 11/2/2020, but if a Java process for pipeline library is created, we would open to moving towards that in the future. TASK DETAIL https://phabricator.wikimedia.org/T265512 WORKBOARD https://phabricator.wikimedia.org/project/board/1227/ EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Mstyles Cc: dcausse, dduvall, jeena, akosiaris, Aklapper, Gehel, Mstyles, Alter-paule, Beast1978, CBogen, Un1tY, Akuckartz, Hook696, Kent7301, joker88john, CucyNoiD, Nandana, Namenlos314, Gaboe420, Giuliamocci, Cpaulf30, Lahi, Gq86, Af420, Bsandipan, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, Lewizho99, Maathavan, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T269619: Create pipelines for late/spurious/failed events
Mstyles added a subscriber: Ottomata. Mstyles added a comment. @Ottomata also suggested via IRC to consider using the event platform instead of kafka TASK DETAIL https://phabricator.wikimedia.org/T269619 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse, Mstyles Cc: Ottomata, Aklapper, Gehel, Mstyles, CBogen, Akuckartz, Nandana, Namenlos314, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T269619: Create pipelines for late/spurious/failed events
Mstyles created this task. Mstyles added projects: Wikidata, Wikidata-Query-Service, Discovery-Search (Current work). TASK DESCRIPTION As a production service, the flink streaming updater job needs somewhere to send events that are late/failed/spurious. Currently these go to HDFS but apps in the Kubernetes cluster can't talk to the Analytics cluster. There needs to be a Kafka bridge that the rdf-streaming-updater can send events to, and those events will eventually end up in HDFS. Acceptance Criteria: Kafka topics are set up for late/failed/spurious events Those same events end up in HDFS TASK DETAIL https://phabricator.wikimedia.org/T269619 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse, Mstyles Cc: Aklapper, Gehel, Mstyles, CBogen, Akuckartz, Nandana, Namenlos314, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T265504: Create Blubberfile in WDQS repo
Mstyles moved this task from Needs review to Needs Reporting on the Discovery-Search (Current work) board. Mstyles added a comment. blubberfile is done and the docker image is present in the wikimedia docker repository TASK DETAIL https://phabricator.wikimedia.org/T265504 WORKBOARD https://phabricator.wikimedia.org/project/board/1227/ EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Mstyles Cc: hashar, Zbyszko, akosiaris, Ottomata, dcausse, Aklapper, Gehel, Mstyles, Alter-paule, Beast1978, CBogen, Un1tY, Akuckartz, Hook696, Kent7301, joker88john, CucyNoiD, Nandana, Namenlos314, Gaboe420, Giuliamocci, Cpaulf30, Lahi, Gq86, Af420, Bsandipan, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, Lewizho99, Maathavan, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T265512: Set up Pipeline Configuration in WDQS repo
Mstyles added a comment. The projects are all related and there are dependencies. Separating the streaming-updater-producer would actually be a significant task. I think that would be a separate project to possibly consider in the future. TASK DETAIL https://phabricator.wikimedia.org/T265512 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Mstyles Cc: dduvall, jeena, akosiaris, Aklapper, Gehel, Mstyles, Alter-paule, Beast1978, CBogen, Un1tY, Akuckartz, Hook696, Kent7301, joker88john, CucyNoiD, Nandana, Namenlos314, Gaboe420, Giuliamocci, Cpaulf30, Lahi, Gq86, Af420, Bsandipan, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, Lewizho99, Maathavan, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T265512: Set up Pipeline Configuration in WDQS repo
Mstyles added a comment. For #1, that's is correct. We are currently downloading Flink from the internet. I don't think that's the best idea long term, but it seems fine for now. Currently for #2, "Build and verify streaming-updater-producer from source, tracking changes to your project repo." That already happens in a different process, via a weekly deploy to Archiva. I don't think we need to replicate that process in the pipeline. We can include it here, but that would mean having to separate the particular streaming-updater-producer project from the rest of the rdf projects. I think that's possible, but not necessary at this time. In regards to #3, "Build and publish a production deployable image that has Flink installed via the base image and has streaming-updater-producer installed into /opt/flink/usrlib.", that's correct, this is the image we want to use in the helm chart. I think the separate repository approach should be fine. We would still need to update the streaming-updater-producer jar version number in the blubber file, and that alone should be enough to trigger a change and produce a new image. Ideally, we wouldn't change any of the current build processes, unless we really need to. I don't think we need to. It seems that the solutions that you are recommending, hinge on having a Flink base image, either via a separate repository that we (the search team) create or via the official SRE base images. The part that is really important to our team is getting our docker image into the WMF docker registry so that we can use it to deploy our Flink cluster to the WMF Kubernetes cluster. Including the Java development process is a nice to have, but there are many Java projects in the same RDF repository that will still be using our established Java build and deploy process. Changing our Java process just for this particular Java project (the streaming-updater-producer) could lead to more complications down the line. TASK DETAIL https://phabricator.wikimedia.org/T265512 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Mstyles Cc: dduvall, jeena, akosiaris, Aklapper, Gehel, Mstyles, Alter-paule, Beast1978, CBogen, Un1tY, Akuckartz, Hook696, Kent7301, joker88john, CucyNoiD, Nandana, Namenlos314, Gaboe420, Giuliamocci, Cpaulf30, Lahi, Gq86, Af420, Bsandipan, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, Lewizho99, Maathavan, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T265512: Set up Pipeline Configuration in WDQS repo
Mstyles added a comment. Instead of trying to skip CI for every commit, the easiest thing to do would be to move the pipeline directory into its own repo. It's not using any of the code in the current repo anyways. TASK DETAIL https://phabricator.wikimedia.org/T265512 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Mstyles Cc: dduvall, jeena, akosiaris, Aklapper, Gehel, Mstyles, Alter-paule, Beast1978, CBogen, Un1tY, Akuckartz, Hook696, Kent7301, joker88john, CucyNoiD, Nandana, Namenlos314, Gaboe420, Giuliamocci, Cpaulf30, Lahi, Gq86, Af420, Bsandipan, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, Lewizho99, Maathavan, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T265512: Set up Pipeline Configuration in WDQS repo
Mstyles claimed this task. TASK DETAIL https://phabricator.wikimedia.org/T265512 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Mstyles Cc: dduvall, jeena, akosiaris, Aklapper, Gehel, Mstyles, Alter-paule, Beast1978, CBogen, Un1tY, Akuckartz, Hook696, Kent7301, joker88john, CucyNoiD, Nandana, Namenlos314, Gaboe420, Giuliamocci, Cpaulf30, Lahi, Gq86, Af420, Bsandipan, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, Lewizho99, Maathavan, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T265512: Set up Pipeline Configuration in WDQS repo
Mstyles moved this task from Ready for Development to Needs review on the Discovery-Search (Current work) board. Mstyles added a subscriber: akosiaris. Mstyles added a comment. @akosiaris it was unclear to me whether we need the promote section in the pipeline config. I'm referring to this: https://wikitech.wikimedia.org/wiki/PipelineLib/Reference#Promote and I saw it in a couple of configs here: https://gerrit.wikimedia.org/r/plugins/gitiles/mediawiki/services/mathoid/+/refs/heads/master/.pipeline/config.yaml#34. Additionally, just so I'm clear, we don't need to do the Jenkins configuration unless we want this to run on every commit (we do not want that). I'm referring to what I saw in the docs: https://wikitech.wikimedia.org/wiki/PipelineLib/Guides/How_to_configure_CI_for_your_project. We just want to be able to rebuild the image whenever we have a new release of the service (on average, once a week) TASK DETAIL https://phabricator.wikimedia.org/T265512 WORKBOARD https://phabricator.wikimedia.org/project/board/1227/ EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Mstyles Cc: akosiaris, Aklapper, Gehel, Mstyles, Alter-paule, Beast1978, CBogen, Un1tY, Akuckartz, Hook696, Kent7301, joker88john, CucyNoiD, Nandana, Namenlos314, Gaboe420, Giuliamocci, Cpaulf30, Lahi, Gq86, Af420, Bsandipan, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, Lewizho99, Maathavan, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T265504: Create Blubberfile in WDQS repo
Mstyles added a comment. @akosiaris I started using the new Java images that you uploaded. I wasn't able to install gpg in the build process. There are some conflicts. We can skip gpg verification of the Flink tar, but I don't think that's a good idea. I will continue to do some debugging. Error message: The following packages have unmet dependencies: gpg : Depends: gpgconf (= 2.2.12-1+deb10u1~bpo9+1) but it is not going to be installed Depends: libassuan0 (>= 2.5.0) but 2.4.3-2 is to be installed Depends: libgpg-error0 (>= 1.35) but 1.26-2 is to be installed E: Unable to correct problems, you have held broken packages. TASK DETAIL https://phabricator.wikimedia.org/T265504 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Mstyles Cc: Zbyszko, akosiaris, Ottomata, dcausse, Aklapper, Gehel, Mstyles, Alter-paule, Beast1978, CBogen, Un1tY, Akuckartz, Hook696, Kent7301, joker88john, CucyNoiD, Nandana, Namenlos314, Gaboe420, Giuliamocci, Cpaulf30, Lahi, Gq86, Af420, Bsandipan, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, Lewizho99, Maathavan, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T265504: Create Blubberfile in WDQS repo
Mstyles added a comment. @akosiaris when you get some time, can you please take another look at https://gerrit.wikimedia.org/r/c/wikidata/query/rdf/+/635074 TASK DETAIL https://phabricator.wikimedia.org/T265504 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Mstyles Cc: Zbyszko, akosiaris, Ottomata, dcausse, Aklapper, Gehel, Mstyles, Alter-paule, Beast1978, CBogen, Un1tY, Akuckartz, Hook696, Kent7301, joker88john, CucyNoiD, Nandana, Namenlos314, Gaboe420, Giuliamocci, Cpaulf30, Lahi, Gq86, Af420, Bsandipan, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, Lewizho99, Maathavan, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T265526: Create Helm Chart
Mstyles claimed this task. TASK DETAIL https://phabricator.wikimedia.org/T265526 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Mstyles Cc: Aklapper, Gehel, Mstyles, CBogen, Akuckartz, Nandana, Namenlos314, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T266495: Create Debian Package for Flink
Mstyles created this task. Mstyles added projects: Wikidata, Wikidata-Query-Service, Discovery-Search (Current work). TASK DESCRIPTION Currently we are downloading Flink from Apache via wget. Sometimes the gpg key check fails, and the blubber build has to be restarted. Additionally, if we want to create our own base Flink image, we need to have a Debian package. @Ottomata shared a way his team created Deb packages for Presto: https://gerrit.wikimedia.org/r/plugins/gitiles/operations/debs/presto/+/refs/heads/debian/debian/README.Debian. We also have our own example for the Elasticsearch plugins. AC Flink can be installed via a Debian package TASK DETAIL https://phabricator.wikimedia.org/T266495 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Mstyles Cc: Aklapper, Gehel, Mstyles, Ottomata, CBogen, Akuckartz, Nandana, Namenlos314, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T265526: Create Helm Chart
Mstyles updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T265526 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Mstyles Cc: Aklapper, Gehel, Mstyles, CBogen, Akuckartz, Nandana, Namenlos314, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T265504: Create Blubberfile in WDQS repo
Mstyles added a comment. The original Flink dockerfile <https://github.com/apache/flink-docker/blob/149ebb09d9146ec46c6df0ab73d638c4d7549c10/1.11/scala_2.11-java8-debian/Dockerfile> exposes ports, which is not an option in blubber. I'm hoping that any networking/ports issues can be resolved in the helm chart TASK DETAIL https://phabricator.wikimedia.org/T265504 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Mstyles Cc: Aklapper, Gehel, Mstyles, Alter-paule, Beast1978, CBogen, Un1tY, Akuckartz, Hook696, darthmon_wmde, Kent7301, joker88john, CucyNoiD, Nandana, Namenlos314, Gaboe420, Giuliamocci, Cpaulf30, Lahi, Gq86, Af420, Bsandipan, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, Lewizho99, Maathavan, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T265504: Create Blubberfile in WDQS repo
Mstyles added a comment. In order to test the image created by the blubberfile, do the following in the rdf repo `cd .pipeline` `blubber blubber.yaml production | docker build --tag blubber-flink-test- --file - .` -> this builds the image and tags it `docker run blubber-flink-test-` TASK DETAIL https://phabricator.wikimedia.org/T265504 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Mstyles Cc: Aklapper, Gehel, Mstyles, Alter-paule, Beast1978, CBogen, Un1tY, Akuckartz, Hook696, darthmon_wmde, Kent7301, joker88john, CucyNoiD, Nandana, Namenlos314, Gaboe420, Giuliamocci, Cpaulf30, Lahi, Gq86, Af420, Bsandipan, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, Lewizho99, Maathavan, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T265512: Set up Pipeline Configuration in WDQS repo
Mstyles moved this task from All WDQS-related tasks to Current work on the Wikidata-Query-Service board. Mstyles added a project: Discovery-Search (Current work). TASK DETAIL https://phabricator.wikimedia.org/T265512 WORKBOARD https://phabricator.wikimedia.org/project/board/891/ EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Mstyles Cc: Aklapper, Gehel, Mstyles, CBogen, Akuckartz, darthmon_wmde, Nandana, Namenlos314, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T265526: Create Helm Chart
Mstyles moved this task from All WDQS-related tasks to Current work on the Wikidata-Query-Service board. Mstyles added a project: Discovery-Search (Current work). TASK DETAIL https://phabricator.wikimedia.org/T265526 WORKBOARD https://phabricator.wikimedia.org/project/board/891/ EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Mstyles Cc: Aklapper, Gehel, Mstyles, CBogen, Akuckartz, darthmon_wmde, Nandana, Namenlos314, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T256949: The streaming updater should support suppressed deletes
Mstyles claimed this task. TASK DETAIL https://phabricator.wikimedia.org/T256949 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Mstyles Cc: Mstyles, Bugreporter, dcausse, Aklapper, Alter-paule, Beast1978, CBogen, Un1tY, Akuckartz, Hook696, darthmon_wmde, Kent7301, joker88john, CucyNoiD, Nandana, Namenlos314, Gaboe420, Giuliamocci, Cpaulf30, Lahi, Gq86, Af420, Bsandipan, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, Lewizho99, Maathavan, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T265504: Create Blubberfile in WDQS repo
Mstyles claimed this task. TASK DETAIL https://phabricator.wikimedia.org/T265504 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Mstyles Cc: Aklapper, Gehel, Mstyles, CBogen, Akuckartz, darthmon_wmde, Nandana, Namenlos314, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T265504: Create Blubberfile in WDQS repo
Mstyles added a project: Discovery-Search (Current work). TASK DETAIL https://phabricator.wikimedia.org/T265504 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Mstyles Cc: Aklapper, Gehel, Mstyles, CBogen, Akuckartz, darthmon_wmde, Nandana, Namenlos314, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T265526: Create Helm Chart
Mstyles created this task. Mstyles added projects: Wikidata, Wikidata-Query-Service. TASK DESCRIPTION Helm Charts are needed to configure the Kubernetes Deployment. This include a Helmfile which is needed for deployment to prodcution. Helm charts are stored in this gerrit repo <https://gerrit.wikimedia.org/r/plugins/gitiles/operations/deployment-charts/+/refs/heads/master/README.md>. Acceptance Criteria Helm chart should have prometheus endpoint information to enable monitoring Helm chart has been benchmarked <https://wikitech.wikimedia.org/wiki/User:Alexandros_Kosiaris/Benchmarking_kubernetes_apps> Helm chart successfully allows Flink to deploy to production TASK DETAIL https://phabricator.wikimedia.org/T265526 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Mstyles Cc: Aklapper, Gehel, Mstyles, CBogen, Akuckartz, darthmon_wmde, Nandana, Namenlos314, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T265512: Set up Pipeline Configuration in WDQS repo
Mstyles created this task. Mstyles added projects: Wikidata, Wikidata-Query-Service. TASK DESCRIPTION We need to define the pipeline configuration <https://wikitech.wikimedia.org/wiki/PipelineLib/Guides/How_to_configure_CI_for_your_project> for WDQS. This is done by creating a config file that tells the pipeline what blubber file to use and what tests to run and what container image to use in production. The pipeline library does not explicitly support Java, so it might be necessary to reach out to Release Engineering to get things working with Java. Acceptance Criteria: The WDQS repo has a config.yaml file that defines the pipeline TASK DETAIL https://phabricator.wikimedia.org/T265512 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Mstyles Cc: Aklapper, Gehel, Mstyles, CBogen, Akuckartz, darthmon_wmde, Nandana, Namenlos314, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T265504: Create Blubberfile in WDQS repo
Mstyles created this task. Mstyles added projects: Wikidata, Wikidata-Query-Service. TASK DESCRIPTION We need to create a blubberfile <https://wikitech.wikimedia.org/wiki/Blubber/User_Guide#Blubberfiles> in order to use the WMF pipeline to deploy to Kubernetes. Blubberfiles are higher level yaml files that can be used to generate Dockerfiles. The output Dockerfile will be based on the Flink Dockerfile found here <https://github.com/apache/flink-docker/blob/149ebb09d9146ec46c6df0ab73d638c4d7549c10/1.11/scala_2.11-java8-debian/Dockerfile> Acceptance Criteria: Base image is from the list of WMF base images (found here <https://dockerregistry.toolforge.org/>) The Blubberfile generates a Dockerfile that can run the Flink Job manager with the streaming updater jar TASK DETAIL https://phabricator.wikimedia.org/T265504 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Mstyles Cc: Aklapper, Gehel, Mstyles, CBogen, Akuckartz, darthmon_wmde, Nandana, Namenlos314, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T244590: [Epic] Rework the WDQS updater as an event driven application
Mstyles reopened subtask T256875: The streaming updater should support page deletions as "Open". TASK DETAIL https://phabricator.wikimedia.org/T244590 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Mstyles Cc: Thadguidry, tfmorris, revi, Ladsgroup, Multichill, darthmon_wmde, Iamamz3, Smalyshev, Ottomata, JAllemandou, Aklapper, Zbyszko, Gehel, dcausse, NavinRizwi, CBogen, Akuckartz, DannyS712, Nandana, Namenlos314, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Dinoguy1000, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T256875: The streaming updater should support page deletions
Mstyles reopened this task as "Open". TASK DETAIL https://phabricator.wikimedia.org/T256875 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Mstyles Cc: Aklapper, dcausse, Alter-paule, Beast1978, CBogen, Un1tY, Akuckartz, Hook696, darthmon_wmde, Kent7301, joker88john, CucyNoiD, Nandana, Namenlos314, Gaboe420, Giuliamocci, Cpaulf30, Lahi, Gq86, Af420, Bsandipan, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, Lewizho99, Maathavan, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T244590: [Epic] Rework the WDQS updater as an event driven application
Mstyles closed subtask T256875: The streaming updater should support page deletions as "Resolved". TASK DETAIL https://phabricator.wikimedia.org/T244590 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Mstyles Cc: Thadguidry, tfmorris, revi, Ladsgroup, Multichill, darthmon_wmde, Iamamz3, Smalyshev, Ottomata, JAllemandou, Aklapper, Zbyszko, Gehel, dcausse, NavinRizwi, CBogen, Akuckartz, DannyS712, Nandana, Namenlos314, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Dinoguy1000, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T256875: The streaming updater should support page deletions
Mstyles moved this task from In Progress to Needs Reporting on the Discovery-Search (Current work) board. Mstyles closed this task as "Resolved". Mstyles added a comment. All patches are merged and items can be deleted successfully TASK DETAIL https://phabricator.wikimedia.org/T256875 WORKBOARD https://phabricator.wikimedia.org/project/board/1227/ EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Mstyles Cc: Aklapper, dcausse, Alter-paule, Beast1978, CBogen, Un1tY, Akuckartz, Hook696, darthmon_wmde, Kent7301, joker88john, CucyNoiD, Nandana, Namenlos314, Gaboe420, Giuliamocci, Cpaulf30, Lahi, Gq86, Af420, Bsandipan, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, Lewizho99, Maathavan, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T264006: Deploy Flink to kubernetes (k8s)
Mstyles added a comment. We're having an in person meeting soon, but I just wanted to outline some things I think will need to happen from a high level. This is based on reading https://wikitech.wikimedia.org/wiki/Deployment_pipeline/Components. 1. Create blubberfiles for both flink and the streaming updater 2. Set up deployment pipeline (or find alternative since it doesn't seem to support Java projects) 3. Create Helm charts and Helm files for flink and the streaming updater Other concerns include logging and where to store flink checkpoint data TASK DETAIL https://phabricator.wikimedia.org/T264006 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Mstyles Cc: Mstyles, Gehel, Aklapper, CBogen, Akuckartz, darthmon_wmde, Nandana, Namenlos314, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T256949: The streaming updater should support suppressed deletes
Mstyles added a comment. I don't see a schema for suppressed deletes in the list of page schemas (https://schema.wikimedia.org/repositories/primary/jsonschema/mediawiki/page/). Is that information intentionally private? Also, I can't see how having the page namespace and other information in the event meta compromises the event. Do we want to default the page namespace to 0? I think we could just listen to the suppressed deletes topic and pass them as regular deletes. Please let me know why we cannot do that TASK DETAIL https://phabricator.wikimedia.org/T256949 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Mstyles Cc: Mstyles, Bugreporter, dcausse, Aklapper, CBogen, Akuckartz, darthmon_wmde, Nandana, Namenlos314, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T256875: The streaming updater should support page deletions
Mstyles added a comment. Delete functionality is complete, optimization work in progress to ensure that patches created on the streaming updater consumer that get sent to blazegraph do not contain statements from entities that will be deleted TASK DETAIL https://phabricator.wikimedia.org/T256875 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Mstyles Cc: Aklapper, dcausse, Alter-paule, Beast1978, CBogen, Un1tY, Akuckartz, Hook696, darthmon_wmde, Kent7301, joker88john, CucyNoiD, Nandana, Namenlos314, Gaboe420, Giuliamocci, Cpaulf30, Lahi, Gq86, Af420, Bsandipan, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, Lewizho99, Maathavan, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T256875: The streaming updater should support page deletions
Mstyles added a comment. Wrapping up the last phase of deleting an entity from blazegraph. I have verified that items to be deleted are present in the streaming updater output. TASK DETAIL https://phabricator.wikimedia.org/T256875 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Mstyles Cc: Aklapper, dcausse, Alter-paule, Beast1978, CBogen, Un1tY, Akuckartz, Hook696, darthmon_wmde, Kent7301, joker88john, CucyNoiD, Nandana, Namenlos314, Gaboe420, Giuliamocci, Cpaulf30, Lahi, Gq86, Af420, Bsandipan, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, Lewizho99, Maathavan, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T262907: EXISTS inside BIND always false when hint included
Mstyles triaged this task as "Low" priority. TASK DETAIL https://phabricator.wikimedia.org/T262907 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Mstyles Cc: Aklapper, Svizel_pritula, CBogen, Akuckartz, darthmon_wmde, Nandana, Namenlos314, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T256875: The streaming updater should support page deletions
Mstyles added a comment. Wrapping up the flink pipeline work, a patch is out for the integration tests and there's some more test cases to be added for clarity on the Decide Mutation Operation to ensure that all delete cases are covered. Pipeline consumer work is up next TASK DETAIL https://phabricator.wikimedia.org/T256875 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Mstyles Cc: Aklapper, dcausse, Alter-paule, Beast1978, CBogen, Un1tY, Akuckartz, Hook696, darthmon_wmde, Kent7301, joker88john, CucyNoiD, Nandana, Namenlos314, Gaboe420, Giuliamocci, Cpaulf30, Lahi, Gq86, Af420, Bsandipan, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, Lewizho99, Maathavan, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T256875: The streaming updater should support page deletions
Mstyles added a comment. Clearing the state is out of the question, I think. Right now, we mark the revision we see a delete at and then we don't allow a Rev Create Event. However, this means that any regular RevCreate events don't allow the item to be reimported ever again. I think it should be something like, if RevCreateEvent revision > DelRev, then go ahead and do a full import. TASK DETAIL https://phabricator.wikimedia.org/T256875 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Mstyles Cc: Aklapper, dcausse, Alter-paule, Beast1978, CBogen, Un1tY, Akuckartz, Hook696, darthmon_wmde, Kent7301, joker88john, CucyNoiD, Nandana, Namenlos314, Gaboe420, Giuliamocci, Cpaulf30, Lahi, Gq86, Af420, Bsandipan, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, Lewizho99, Maathavan, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T256875: The streaming updater should support page deletions
Mstyles added a comment. What do we want to do if we get a PageDelete event for the same item multiple times? Right now we're not tracking deletes, so I'm assuming multiple deletes for the same item probably can't do that much harm. TASK DETAIL https://phabricator.wikimedia.org/T256875 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Mstyles Cc: Aklapper, dcausse, Alter-paule, Beast1978, CBogen, Un1tY, Akuckartz, Hook696, darthmon_wmde, Kent7301, joker88john, CucyNoiD, Nandana, Namenlos314, Gaboe420, Giuliamocci, Cpaulf30, Lahi, Gq86, Af420, Bsandipan, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, Lewizho99, Maathavan, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T256875: The streaming updater should support page deletions
Mstyles added a comment. Shared model delete functionality is complete, currently in the process of adding delete functionality to the flink pipeline TASK DETAIL https://phabricator.wikimedia.org/T256875 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Mstyles Cc: Aklapper, dcausse, Alter-paule, Beast1978, CBogen, Un1tY, Akuckartz, Hook696, darthmon_wmde, Kent7301, joker88john, CucyNoiD, Nandana, Namenlos314, Gaboe420, Giuliamocci, Cpaulf30, Lahi, Gq86, Af420, Bsandipan, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, Lewizho99, Maathavan, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T251515: Automate data reload for SPARQL Endpoint for Commons
Mstyles removed Mstyles as the assignee of this task. TASK DETAIL https://phabricator.wikimedia.org/T251515 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Mstyles Cc: Mstyles, dcausse, Zbyszko, Aklapper, Lea_Lacroix_WMDE, Gehel, CBogen, Akuckartz, darthmon_wmde, Nandana, Namenlos314, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T256875: The streaming updater should support page deletions
Mstyles added a project: Discovery-Search (Current work). TASK DETAIL https://phabricator.wikimedia.org/T256875 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Mstyles Cc: Aklapper, dcausse, CBogen, Akuckartz, darthmon_wmde, Nandana, Namenlos314, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T256875: The streaming updater should support page deletions
Mstyles claimed this task. TASK DETAIL https://phabricator.wikimedia.org/T256875 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Mstyles Cc: Aklapper, dcausse, CBogen, Akuckartz, darthmon_wmde, Nandana, Namenlos314, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T251488: [epic] Create minimal SPARQL Endpoint for Commons on WMCS
Mstyles closed subtask T251515: Automate data reload for SPARQL Endpoint for Commons as "Resolved". TASK DETAIL https://phabricator.wikimedia.org/T251488 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Mstyles Cc: Nintendofan885, Zbyszko, Smalyshev, Ramsey-WMF, Cparle, MarkTraceur, Abit, Jheald, EBernhardson, dcausse, Lucas_Werkmeister_WMDE, Tpt, Marsupium, Abbe98, Tnegrin, Multichill, Tagishsimon, Husky, Jarekt, Fuzheado, Spinster, Jane023, Keegan, Gamaliel, CBogen, David_Haskiya_WMSE, Nuria, Lea_Lacroix_WMDE, Aklapper, Gehel, NavinRizwi, Akuckartz, darthmon_wmde, DannyS712, Nandana, Namenlos314, Lahi, Gq86, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Dinoguy1000, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T251515: Automate data reload for SPARQL Endpoint for Commons
Mstyles closed this task as "Resolved". TASK DETAIL https://phabricator.wikimedia.org/T251515 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Mstyles Cc: Zbyszko, Aklapper, Lea_Lacroix_WMDE, Gehel, CBogen, Akuckartz, darthmon_wmde, Nandana, Namenlos314, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] T257314: Add a Logout link to Wikimedia Commons Query Service
Mstyles claimed this task. TASK DETAIL https://phabricator.wikimedia.org/T257314 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Mstyles Cc: CBogen, Lucas_Werkmeister_WMDE, Aklapper, Zbyszko, Akuckartz, darthmon_wmde, Nandana, Namenlos314, Lahi, Gq86, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T251514: UI for SPARQL Endpoint for Commons
Mstyles added a comment. Since the gui directory is overwritten by scap via symlinks, Guillaume proposed that the config files live in /etc/config and that they be symlinked to the gui directory. See more discussion here:https://gerrit.wikimedia.org/r/c/operations/puppet/+/606297. I want to remove the custom config file that lives in the gui-deploy repo to avoid any confusion of config files: https://gerrit.wikimedia.org/r/c/wikidata/query/gui-deploy/+/606545 TASK DETAIL https://phabricator.wikimedia.org/T251514 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Mstyles Cc: Mstyles, CBogen, Lucas_Werkmeister_WMDE, Zbyszko, WMDE-leszek, Addshore, Aklapper, Lea_Lacroix_WMDE, Gehel, Alter-paule, Beast1978, Un1tY, Hook696, darthmon_wmde, Kent7301, joker88john, CucyNoiD, Nandana, Namenlos314, Gaboe420, Giuliamocci, Cpaulf30, Lahi, Gq86, Af420, Bsandipan, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, Lewizho99, Maathavan, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Updated] T251514: UI for SPARQL Endpoint for Commons
Mstyles added a project: Discovery-Search (Current work). TASK DETAIL https://phabricator.wikimedia.org/T251514 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Mstyles Cc: CBogen, Lucas_Werkmeister_WMDE, Zbyszko, WMDE-leszek, Addshore, Aklapper, Lea_Lacroix_WMDE, Gehel, darthmon_wmde, Nandana, Namenlos314, Lahi, Gq86, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T251500: oAuth authentication for SPARQL Endpoint for Commons
Mstyles added a comment. Here is the nginx example -> https://github.com/nginxinc/NGINX-Demos/tree/master/oauth2-token-introspection-oss TASK DETAIL https://phabricator.wikimedia.org/T251500 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Mstyles Cc: Mstyles, Aklapper, Gehel, Blissjay007, Oblanco79, Alter-paule, Beast1978, CBogen, Un1tY, Hook696, Daryl-TTMG, RomaAmorRoma, E.S.A-Sheild, darthmon_wmde, Kent7301, Meekrab2012, joker88john, CucyNoiD, Nandana, NebulousIris, Namenlos314, Gaboe420, Versusxo, Majesticalreaper22, Giuliamocci, Adrian1985, Cpaulf30, Lahi, Gq86, Af420, Darkminds3113, Bsandipan, Lordiis, Lucas_Werkmeister_WMDE, GoranSMilovanovic, Adik2382, Th3d3v1ls, Ramalepe, Liugev6, QZanden, EBjune, merbst, LawExplorer, WSH1906, Lewizho99, Maathavan, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T251500: oAuth authentication for SPARQL Endpoint for Commons
Mstyles added a comment. Some notes for when/if we do oauth Docs -> https://www.mediawiki.org/wiki/OAuth/For_Developers we might be able to use this npm package <https://www.npmjs.com/package/passport-mediawiki-oauth> for auth with mediawiki This <https://github.com/SQL-enwiki/ipcheck/blob/master/public_html/oauth.php> is an example of a tools project that uses auth and restricts who can log into the tool TASK DETAIL https://phabricator.wikimedia.org/T251500 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Mstyles Cc: Mstyles, Aklapper, Gehel, CBogen, darthmon_wmde, Nandana, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Changed Subscribers] T251515: Automate data reload for SPARQL Endpoint for Commons
Mstyles added a subscriber: Zbyszko. Mstyles added a comment. As discussed in email with @Zbyszko the script should do the following 1. Download the newest dump (maybe - we can provide it manually, but where's the fun in that?) 2. Munge the data (running a script with appropriate data) 3. Stop Blazegraph 4. Delete Blazegraph journal 5. Start (now empty) Blazegraph 6. Execute loadData.sh script with sdc namespace and pointing to the munged data (see Igor's comment <https://phabricator.wikimedia.org/T237089#5761350>). TASK DETAIL https://phabricator.wikimedia.org/T251515 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Mstyles Cc: Zbyszko, Aklapper, Lea_Lacroix_WMDE, Gehel, CBogen, darthmon_wmde, Nandana, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Claimed] T251515: Automate data reload for SPARQL Endpoint for Commons
Mstyles claimed this task. TASK DETAIL https://phabricator.wikimedia.org/T251515 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Mstyles Cc: Aklapper, Lea_Lacroix_WMDE, Gehel, CBogen, darthmon_wmde, Nandana, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Closed] T247123: Migrate wikidata-query-rdf-release-silent release job to Docker
Mstyles closed this task as "Resolved". TASK DETAIL https://phabricator.wikimedia.org/T247123 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Mstyles Cc: Gehel, Ottomata, Addshore, hashar, Aklapper, Jdforrester-WMF, Blissjay007, Oblanco79, Alter-paule, Beast1978, CBogen, Un1tY, Hook696, Daryl-TTMG, RomaAmorRoma, E.S.A-Sheild, darthmon_wmde, Kent7301, Meekrab2012, joker88john, CucyNoiD, Nandana, NebulousIris, Gaboe420, Versusxo, Majesticalreaper22, Giuliamocci, Adrian1985, Cpaulf30, Lahi, Gq86, Af420, Darkminds3113, Bsandipan, Lordiis, Lucas_Werkmeister_WMDE, GoranSMilovanovic, Adik2382, Th3d3v1ls, Ramalepe, Liugev6, QZanden, EBjune, merbst, LawExplorer, WSH1906, Lewizho99, Maathavan, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, thcipriani, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Lydia_Pintscher, Mbch331, Jay8g ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T247123: Migrate wikidata-query-rdf-release-silent release job to Docker
Mstyles added a comment. Docs are updated -> https://wikitech.wikimedia.org/wiki/Wikidata_query_service#Production_Deployment and verified that the zip file from the latest version can be downloaded from Archiva TASK DETAIL https://phabricator.wikimedia.org/T247123 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Mstyles Cc: Gehel, Ottomata, Addshore, hashar, Aklapper, Jdforrester-WMF, Blissjay007, Oblanco79, Alter-paule, Beast1978, CBogen, Un1tY, Hook696, Daryl-TTMG, RomaAmorRoma, E.S.A-Sheild, darthmon_wmde, Kent7301, Meekrab2012, joker88john, CucyNoiD, Nandana, NebulousIris, Gaboe420, Versusxo, Majesticalreaper22, Giuliamocci, Adrian1985, Cpaulf30, Lahi, Gq86, Af420, Darkminds3113, Bsandipan, Lordiis, Lucas_Werkmeister_WMDE, GoranSMilovanovic, Adik2382, Th3d3v1ls, Ramalepe, Liugev6, QZanden, EBjune, merbst, LawExplorer, WSH1906, Lewizho99, Maathavan, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, thcipriani, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Lydia_Pintscher, Mbch331, Jay8g ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs