[Wikidata-bugs] [Maniphest] T342593: Five deleted Wikidata items pertaining to Wikimedia category pages still present in the Query Service

2023-10-13 Thread Gehel 
Gehel closed this task as "Resolved".

TASK DETAIL
  https://phabricator.wikimedia.org/T342593

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: dcausse, Gehel
Cc: Ottomata, EBernhardson, ArielGlenn, Milimetric, dcausse, Nikki, Aklapper, 
Mahir256, Danny_Benjafield_WMDE, Mohamed-Awnallah, Astuthiodit_1, AWesterinen, 
lbowmaker, BTullis, karapayneWMDE, Invadibot, Ywats0ns, maantietaja, 
ItamarWMDE, Akuckartz, holger.knust, Nandana, Namenlos314, Lahi, Gq86, 
Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, 
LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, 
Wikidata-bugs, Jdouglas, aude, Tobias1984, GWicke, Manybubbles, Mbch331
___
Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org
To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org

[Wikidata-bugs] [Maniphest] T342593: Five deleted Wikidata items pertaining to Wikimedia category pages still present in the Query Service

2023-10-10 Thread dcausse 
dcausse moved this task from To Be Deployed to Needs Reporting on the 
Discovery-Search (Current work) board.
dcausse added a comment.


  Reconciled these items manually, improving the reliability of the event 
system will be tracked in other tasks such as T345195 


TASK DETAIL
  https://phabricator.wikimedia.org/T342593

WORKBOARD
  https://phabricator.wikimedia.org/project/board/1227/

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: dcausse
Cc: Ottomata, EBernhardson, ArielGlenn, Milimetric, dcausse, Nikki, Aklapper, 
Mahir256, Danny_Benjafield_WMDE, Mohamed-Awnallah, Astuthiodit_1, AWesterinen, 
lbowmaker, BTullis, karapayneWMDE, Invadibot, Ywats0ns, maantietaja, 
ItamarWMDE, Akuckartz, holger.knust, Nandana, Namenlos314, Lahi, Gq86, 
Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, 
LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, 
Wikidata-bugs, Jdouglas, aude, Tobias1984, GWicke, Manybubbles, Mbch331
___
Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org
To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org

[Wikidata-bugs] [Maniphest] T342593: Five deleted Wikidata items pertaining to Wikimedia category pages still present in the Query Service

2023-10-03 Thread Ottomata 
Ottomata added a comment.


  > I agree with @Milimetric here and we need to get a better sense of the 
quality of the EventBus/EventGate system
  
  T345195: [SPIKE] Can we identify indicators to inform an SLO for event 
emission and intake? 

TASK DETAIL
  https://phabricator.wikimedia.org/T342593

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: dcausse, Ottomata
Cc: Ottomata, EBernhardson, ArielGlenn, Milimetric, dcausse, Nikki, Aklapper, 
Mahir256, Danny_Benjafield_WMDE, Mohamed-Awnallah, Astuthiodit_1, AWesterinen, 
lbowmaker, BTullis, karapayneWMDE, Invadibot, Ywats0ns, maantietaja, 
ItamarWMDE, Akuckartz, holger.knust, Nandana, Namenlos314, Lahi, Gq86, 
Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, 
LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, 
Wikidata-bugs, Jdouglas, aude, Tobias1984, GWicke, Manybubbles, Mbch331
___
Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org
To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org

[Wikidata-bugs] [Maniphest] T342593: Five deleted Wikidata items pertaining to Wikimedia category pages still present in the Query Service

2023-09-12 Thread Maintenance_bot 
Maintenance_bot removed a project: Patch-For-Review.

TASK DETAIL
  https://phabricator.wikimedia.org/T342593

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: dcausse, Maintenance_bot
Cc: EBernhardson, ArielGlenn, Milimetric, dcausse, Nikki, Aklapper, Mahir256, 
Danny_Benjafield_WMDE, Mohamed-Awnallah, Astuthiodit_1, AWesterinen, lbowmaker, 
BTullis, karapayneWMDE, Invadibot, Ywats0ns, maantietaja, ItamarWMDE, 
Akuckartz, holger.knust, Nandana, Namenlos314, Lahi, Gq86, 
Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, 
LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, 
Wikidata-bugs, Jdouglas, aude, Tobias1984, GWicke, Manybubbles, Mbch331, 
Isabelladantes1983, Themindcoder, Adamm71, Jersione, Hellket777, LisafBia6531, 
786, Biggs657, Juan90264, Alter-paule, Beast1978, Un1tY, Hook696, Kent7301, 
joker88john, CucyNoiD, Gaboe420, Giuliamocci, Cpaulf30, Af420, Bsandipan, 
Lewizho99, Maathavan, Neuronton
___
Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org
To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org

[Wikidata-bugs] [Maniphest] T342593: Five deleted Wikidata items pertaining to Wikimedia category pages still present in the Query Service

2023-09-12 Thread gerritbot 
gerritbot added a comment.


  Change 955783 **merged** by jenkins-bot:
  
  [wikidata/query/rdf@master] Allow reconciling a deletion without knowing its 
revision
  
  https://gerrit.wikimedia.org/r/955783

TASK DETAIL
  https://phabricator.wikimedia.org/T342593

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: dcausse, gerritbot
Cc: EBernhardson, ArielGlenn, Milimetric, dcausse, Nikki, Aklapper, Mahir256, 
Danny_Benjafield_WMDE, Isabelladantes1983, Themindcoder, Mohamed-Awnallah, 
Adamm71, Jersione, Hellket777, LisafBia6531, Astuthiodit_1, AWesterinen, 786, 
lbowmaker, BTullis, Biggs657, karapayneWMDE, Invadibot, Ywats0ns, maantietaja, 
Juan90264, Alter-paule, Beast1978, ItamarWMDE, Un1tY, Akuckartz, Hook696, 
Kent7301, holger.knust, joker88john, CucyNoiD, Nandana, Namenlos314, Gaboe420, 
Giuliamocci, Cpaulf30, Lahi, Gq86, Af420, Bsandipan, Lucas_Werkmeister_WMDE, 
GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, Lewizho99, Maathavan, 
_jensen, rosalieper, Neuronton, Scott_WUaS, Jonas, Xmlizer, jkroll, 
Wikidata-bugs, Jdouglas, aude, Tobias1984, GWicke, Manybubbles, Mbch331
___
Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org
To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org

[Wikidata-bugs] [Maniphest] T342593: Five deleted Wikidata items pertaining to Wikimedia category pages still present in the Query Service

2023-09-07 Thread gerritbot 
gerritbot added a project: Patch-For-Review.

TASK DETAIL
  https://phabricator.wikimedia.org/T342593

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: dcausse, gerritbot
Cc: EBernhardson, ArielGlenn, Milimetric, dcausse, Nikki, Aklapper, Mahir256, 
Danny_Benjafield_WMDE, Isabelladantes1983, Themindcoder, Mohamed-Awnallah, 
Adamm71, Jersione, Hellket777, LisafBia6531, Astuthiodit_1, AWesterinen, 786, 
lbowmaker, BTullis, Biggs657, karapayneWMDE, Invadibot, Ywats0ns, maantietaja, 
Juan90264, Alter-paule, Beast1978, ItamarWMDE, Un1tY, Akuckartz, Hook696, 
Kent7301, holger.knust, joker88john, CucyNoiD, Nandana, Namenlos314, Gaboe420, 
Giuliamocci, Cpaulf30, Lahi, Gq86, Af420, Bsandipan, Lucas_Werkmeister_WMDE, 
GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, Lewizho99, Maathavan, 
_jensen, rosalieper, Neuronton, Scott_WUaS, Jonas, Xmlizer, jkroll, 
Wikidata-bugs, Jdouglas, aude, Tobias1984, GWicke, Manybubbles, Mbch331
___
Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org
To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org

[Wikidata-bugs] [Maniphest] T342593: Five deleted Wikidata items pertaining to Wikimedia category pages still present in the Query Service

2023-09-07 Thread gerritbot 
gerritbot added a comment.


  Change 955783 had a related patch set uploaded (by DCausse; author: DCausse):
  
  [wikidata/query/rdf@master] Allow reconciling a deletion without knowing its 
revision
  
  https://gerrit.wikimedia.org/r/955783

TASK DETAIL
  https://phabricator.wikimedia.org/T342593

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: dcausse, gerritbot
Cc: EBernhardson, ArielGlenn, Milimetric, dcausse, Nikki, Aklapper, Mahir256, 
Danny_Benjafield_WMDE, Mohamed-Awnallah, Astuthiodit_1, AWesterinen, lbowmaker, 
BTullis, karapayneWMDE, Invadibot, Ywats0ns, maantietaja, ItamarWMDE, 
Akuckartz, holger.knust, Nandana, Namenlos314, Lahi, Gq86, 
Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, 
LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, 
Wikidata-bugs, Jdouglas, aude, Tobias1984, GWicke, Manybubbles, Mbch331
___
Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org
To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org

[Wikidata-bugs] [Maniphest] T342593: Five deleted Wikidata items pertaining to Wikimedia category pages still present in the Query Service

2023-09-06 Thread dcausse 
dcausse claimed this task.
dcausse moved this task from Ready for Dev -- SWE to In Progress on the 
Discovery-Search (Current work) board.
dcausse added a comment.


  Going to work on improving the tooling regarding reconciliations of missed 
deletes but I won't be working on the root cause. I agree with @Milimetric here 
and we need to get a better sense of the qualitity the EventBus/EventGate 
system, 5 (*identified*) missed events on a stream that is relatively low 
volume over 1 month seems concerning.

TASK DETAIL
  https://phabricator.wikimedia.org/T342593

WORKBOARD
  https://phabricator.wikimedia.org/project/board/1227/

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: dcausse
Cc: EBernhardson, ArielGlenn, Milimetric, dcausse, Nikki, Aklapper, Mahir256, 
Danny_Benjafield_WMDE, Mohamed-Awnallah, Astuthiodit_1, AWesterinen, lbowmaker, 
BTullis, karapayneWMDE, Invadibot, Ywats0ns, maantietaja, ItamarWMDE, 
Akuckartz, holger.knust, Nandana, Namenlos314, Lahi, Gq86, 
Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, 
LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, 
Wikidata-bugs, Jdouglas, aude, Tobias1984, GWicke, Manybubbles, Mbch331
___
Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org
To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org

[Wikidata-bugs] [Maniphest] T342593: Five deleted Wikidata items pertaining to Wikimedia category pages still present in the Query Service

2023-08-08 Thread dcausse 
dcausse added projects: Data Engineering and Event Platform Team, 
Data-Engineering, Event-Platform.

TASK DETAIL
  https://phabricator.wikimedia.org/T342593

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: dcausse
Cc: ArielGlenn, Milimetric, dcausse, Nikki, Aklapper, Mahir256, 
Danny_Benjafield_WMDE, Mohamed-Awnallah, Astuthiodit_1, AWesterinen, lbowmaker, 
BTullis, karapayneWMDE, Invadibot, Ywats0ns, maantietaja, ItamarWMDE, 
Akuckartz, holger.knust, Nandana, Namenlos314, Lahi, Gq86, 
Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, 
LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, 
Wikidata-bugs, Jdouglas, aude, Tobias1984, GWicke, Manybubbles, Mbch331
___
Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org
To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org

[Wikidata-bugs] [Maniphest] T342593: Five deleted Wikidata items pertaining to Wikimedia category pages still present in the Query Service

2023-08-07 Thread Milimetric 
Milimetric added a comment.


  Just a random drive-by note, since I'm not the one playing with this, but it 
might be interesting to instrument EventBus a little bit.  For example, from 
the deferred job that publishes to Kafka, we could log a basic key for each 
event that we publish.  It should be possible to aggregate these logs and 
compare them against what we see in Kafka to figure out what we missed, perhaps 
even facilitate retries.

TASK DETAIL
  https://phabricator.wikimedia.org/T342593

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Milimetric
Cc: Milimetric, dcausse, Nikki, Aklapper, Mahir256, Danny_Benjafield_WMDE, 
Astuthiodit_1, AWesterinen, karapayneWMDE, Invadibot, maantietaja, ItamarWMDE, 
Akuckartz, Nandana, Namenlos314, Lahi, Gq86, Lucas_Werkmeister_WMDE, 
GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, 
Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, 
Manybubbles, Mbch331
___
Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org
To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org

[Wikidata-bugs] [Maniphest] T342593: Five deleted Wikidata items pertaining to Wikimedia category pages still present in the Query Service

2023-08-07 Thread dcausse 
dcausse added a comment.


  | item | deletion date   |
  | Q10813441| 2023-06-06T14:21:49 |
  | Q32994683| 2023-05-03T16:04:27 |
  | Q55929561| 2023-05-31T20:24:58 |
  | Q109548562| 2023-06-06T14:22:29 |
  | Q111436860| 2023-05-04T06:54:56 |
  |
  
  None of these deletions appear to have been treated by the WDQS updater.
  Looking further I can't find any traces of the corresponding delete events 
searching in `event.mediawiki_page_delete` and  `event.mediawiki_page_suppress`:
  
select * from mediawiki_page_delete where year=2023 and month in (5,6,7,8) 
and page_title in ('Q55929561', 'Q10813441', 'Q32994683', 'Q55929561', 
'Q109548562', 'Q111436860');
  
OK
comment databasemetapage_id page_is_redirect
page_namespace  page_title  parsedcomment   performer   rev_count   
rev_id  chronology_id   _schema is_wmf_domain   normalized_host datacenter  
yearmonth   day hour
Time taken: 1.557 seconds
  
select * from mediawiki_page_suppress where year=2023 and month in 
(5,6,7,8) and page_title in ('Q55929561', 'Q10813441', 'Q32994683', 
'Q55929561', 'Q109548562', 'Q111436860');
  
OK
_schema metadatabaseperformer   page_id page_title  
page_namespace  page_is_redirectrev_id  chronology_id   comment 
parsedcomment   rev_count   is_wmf_domain   normalized_host datacenter  
yearmonth   day hour
Time taken: 0.311 seconds
  
  This possibly means that mediawiki never told the WDQS updater that these 
items were deleted.
  
  Searching for these items in the `event.mediawiki_page_change_v1` did not 
yield any results, it seems to me that the root cause is a problem somewhere 
between MW and kafka-main (EventBus and/or eventgate?).

TASK DETAIL
  https://phabricator.wikimedia.org/T342593

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: dcausse
Cc: dcausse, Nikki, Aklapper, Mahir256, Danny_Benjafield_WMDE, Astuthiodit_1, 
AWesterinen, karapayneWMDE, Invadibot, maantietaja, ItamarWMDE, Akuckartz, 
Nandana, Namenlos314, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, 
QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, 
Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331
___
Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org
To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org

[Wikidata-bugs] [Maniphest] T342593: Five deleted Wikidata items pertaining to Wikimedia category pages still present in the Query Service

2023-08-07 Thread Gehel 
Gehel set the point value for this task to "8".

TASK DETAIL
  https://phabricator.wikimedia.org/T342593

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Gehel
Cc: Nikki, Aklapper, Mahir256, Danny_Benjafield_WMDE, Astuthiodit_1, 
AWesterinen, karapayneWMDE, Invadibot, maantietaja, ItamarWMDE, Akuckartz, 
Nandana, Namenlos314, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, 
QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, 
Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331
___
Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org
To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org

[Wikidata-bugs] [Maniphest] T342593: Five deleted Wikidata items pertaining to Wikimedia category pages still present in the Query Service

2023-08-07 Thread Gehel 
Gehel moved this task from Incoming to Current work on the 
Wikidata-Query-Service board.
Gehel added a project: Discovery-Search (Current work).

TASK DETAIL
  https://phabricator.wikimedia.org/T342593

WORKBOARD
  https://phabricator.wikimedia.org/project/board/891/

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Gehel
Cc: Nikki, Aklapper, Mahir256, Danny_Benjafield_WMDE, Astuthiodit_1, 
AWesterinen, karapayneWMDE, Invadibot, maantietaja, ItamarWMDE, Akuckartz, 
Nandana, Namenlos314, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, 
QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, 
Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331
___
Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org
To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org

[Wikidata-bugs] [Maniphest] T342593: Five deleted Wikidata items pertaining to Wikimedia category pages still present in the Query Service

2023-07-24 Thread Maintenance_bot 
Maintenance_bot added a project: Wikidata.

TASK DETAIL
  https://phabricator.wikimedia.org/T342593

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Maintenance_bot
Cc: Aklapper, Mahir256, Astuthiodit_1, AWesterinen, karapayneWMDE, Invadibot, 
maantietaja, ItamarWMDE, Akuckartz, Nandana, Namenlos314, Lahi, Gq86, 
Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, 
LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, 
Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331
___
Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org
To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org

[Wikidata-bugs] [Maniphest] T342593: Five deleted Wikidata items pertaining to Wikimedia category pages still present in the Query Service

2023-07-24 Thread Mahir256 
Mahir256 created this task.
Mahir256 added a project: Wikidata-Query-Service.
Restricted Application added a subscriber: Aklapper.

TASK DESCRIPTION
  **Steps to replicate the issue** (include links if applicable):
  
  - Run the query https://w.wiki/77P$
  
  **What happens?**:
  
  682 RDF triples where the subject is one of the five Qids Q10813441, 
Q32994683, Q55929561, Q109548562, and Q111436860 are returned.
  
  **What should have happened instead?**:
  
  No triples should be returned (Q32994683 was deleted on 3 May, Q109548562 on 
6 June, and the other items named above were deleted in-between).

TASK DETAIL
  https://phabricator.wikimedia.org/T342593

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Mahir256
Cc: Aklapper, Mahir256, AWesterinen, Namenlos314, Gq86, Lucas_Werkmeister_WMDE, 
EBjune, merbst, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, 
Tobias1984, Manybubbles
___
Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org
To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org