[Wikidata-bugs] [Maniphest] [Commented On] T44325: Prevent creation of items having the same sitelinks (duplicates)

[Robby]

Robby added a comment.


  A few more examples of double creations in wikidata which where done without 
that I intended to do so:
  
  2019-06-28T23:26:49 diff hist +469‎ N Adolph Schreiber House (Q64864961) ‎ 
‎Created a new Item current
  2019-06-28T23:26:49 diff hist +469‎ N Adolph Schreiber House (Q64864962) ‎ 
‎Created a new Item current
  
  2019-06-26T23:47:59 diff hist +436‎ N Klehm House (Q64833963) ‎ ‎Created a 
new Item current
  2019-06-26T23:47:58 diff hist +436‎ N Klehm House (Q64833962) ‎ ‎Created a 
new Item current
  
  2019-06-26T23:42:05 diff hist +466‎ N Kieldson Double House (Q64833830) ‎ 
‎Created a new Item current
  2019-06-26T23:42:04 diff hist +466‎ N Kieldson Double House (Q64833829) ‎ 
‎Created a new Item current
  
  2019-06-05T22:35:56 diff hist +469‎ N Category:Earls of Balfour (Q64409304) ‎ 
‎Created a new Item current Tag: PHP7
  2019-06-05T22:35:55 diff hist +469‎ N (Q64409303) ‎ ‎Created a new Item Tag: 
PHP7
  
  2019-05-22T20:07:25 diff hist +466‎ N Category:1886 in Judaism (Q63985592) ‎ 
‎Created a new Item
  2019-05-22T20:07:25 diff hist +466‎ N (Q63985594) ‎ ‎Created a new Item
  
  2019-05-19T19:26:35 diff hist +478‎ N Category:Fort Meade, Florida 
(Q63955820) ‎ ‎Created a new Item current
  2019-05-19T19:26:34 diff hist +478‎ N Category:Fort Meade, Florida 
(Q63955819) ‎ ‎Created a new Item
  
  as I create most of the items I create in wikidata the same way ans as this 
does not occur systematically I could imagine that this just happens if certain 
parameters are apllicable to the database.
  
  Unfortunately I am not able to reproduce the phenomena I can just locate them 
in my contributions list.

TASK DETAIL
  https://phabricator.wikimedia.org/T44325

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Robby
Cc: Mike_Peel, Lea_Lacroix_WMDE, Takasugi_Shinji, VIGNERON, alaa_wmde, Robby, 
Stashbot, Addshore, Nikki, Liuxinyu970226, aude, Aklapper, AnjaJentzsch, 
Abraham, jeblad, Legoktm, He7d3r, Merl, jayvdb, Denny, revi, matej_suchanek, 
Lydia_Pintscher, Beta16, daniel, hoo, darthmon_wmde, DannyS712, Nandana, Lahi, 
Gq86, GoranSMilovanovic, QZanden, LawExplorer, Poyekhali, _jensen, rosalieper, 
Taiwania_Justo, Wong128hk, Wikidata-bugs, El_Grafo, Dinoguy1000, Steinsplitter, 
Mbch331, Keegan
___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Commented On] T44325: Prevent creation of items having the same sitelinks (duplicates)

[Robby]

Robby added a comment.


  Today  there was again twice a creation of duplicate items in wikidata:
  
  2019-04-25T06:23:34 diff hist +466‎ N Category:2012 in Judaism (Q63323044) ‎ 
‎Created a new Item Tag: PHP7
  2019-04-25T06:23:34 diff hist +466‎ N Category:2012 in Judaism (Q63323045) ‎ 
‎Created a new Item current Tag: PHP7
  
  2019-04-25T06:18:31 diff hist +463‎ N Category:1896 in Poland (Q63323035) ‎ 
‎Created a new Item Tag: PHP7
  2019-04-25T06:18:31 diff hist +463‎ N Category:1896 in Poland (Q63323036) ‎ 
‎Created a new Item current Tag: PHP7

TASK DETAIL
  https://phabricator.wikimedia.org/T44325

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Robby
Cc: Robby, Stashbot, Addshore, Nikki, Liuxinyu970226, aude, Aklapper, 
AnjaJentzsch, Abraham, jeblad, Legoktm, He7d3r, Merl, jayvdb, Denny, revi, 
matej_suchanek, Lydia_Pintscher, Beta16, daniel, hoo, alaa_wmde, Nandana, Lahi, 
Gq86, GoranSMilovanovic, QZanden, LawExplorer, _jensen, rosalieper, 
Wikidata-bugs, Mbch331
___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Commented On] T44325: Prevent creation of items having the same sitelinks (duplicates)

[hoo]

hoo added a comment.


  In T44325#412 , 
@Addshore wrote:
  
  > @hoo did we ever get a result from that script run?
  
  
  I forgot about this… most of this seems to still be unresolved, so I put it 
up on 
https://www.wikidata.org/wiki/Wikidata:True_duplicates#Items_with_conflicting_sitelinks.

TASK DETAIL
  https://phabricator.wikimedia.org/T44325

EMAIL PREFERENCES
  https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: hoo
Cc: Robby, Stashbot, Addshore, Nikki, Liuxinyu970226, TerraCodes, aude, 
Aklapper, AnjaJentzsch, Abraham, jeblad, Legoktm, He7d3r, Merl, jayvdb, Denny, 
revi, matej_suchanek, Lydia_Pintscher, Beta16, daniel, hoo, alaa_wmde, Nandana, 
Lahi, Gq86, GoranSMilovanovic, QZanden, LawExplorer, _jensen, rosalieper, 
Wikidata-bugs, Mbch331
___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Commented On] T44325: Prevent creation of items having the same sitelinks (duplicates)

[Robby]

Robby added a comment.
The list of more examples and a description of how such duplicates were generated unintentionally is now availyble on: https://www.wikidata.org/wiki/Wikidata:Project_chat/Archive/2018/10#Items_created_unintentionally_twice_on_several_occasionsTASK DETAILhttps://phabricator.wikimedia.org/T44325EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: RobbyCc: Robby, Stashbot, Addshore, Nikki, Liuxinyu970226, TerraCodes, aude, Aklapper, AnjaJentzsch, Abraham, jeblad, Legoktm, He7d3r, Merl, jayvdb, Denny, revi, matej_suchanek, Lydia_Pintscher, Beta16, daniel, hoo, Nandana, Lahi, Gq86, GoranSMilovanovic, QZanden, LawExplorer, _jensen, Wikidata-bugs, Mbch331___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Commented On] T44325: Prevent creation of items having the same sitelinks (duplicates)

[Addshore]

Addshore added a comment.
@hoo did we ever get a result from that script run?TASK DETAILhttps://phabricator.wikimedia.org/T44325EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: AddshoreCc: Robby, Stashbot, Addshore, Nikki, Liuxinyu970226, TerraCodes, aude, Aklapper, AnjaJentzsch, Abraham, jeblad, Legoktm, He7d3r, Merl, jayvdb, Denny, revi, matej_suchanek, Lydia_Pintscher, Beta16, daniel, hoo, Nandana, Lahi, Gq86, GoranSMilovanovic, QZanden, LawExplorer, _jensen, Wikidata-bugs, Mbch331___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Commented On] T44325: Prevent creation of items having the same sitelinks (duplicates)

[Robby]

Robby added a comment.
There are more examples and a description of how such duplicates were generated unintentionally https://www.wikidata.org/wiki/Wikidata:Project_chat#Items_created_unintentionally_twice_on_several_occasionsTASK DETAILhttps://phabricator.wikimedia.org/T44325EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: RobbyCc: Robby, Stashbot, Addshore, Nikki, Liuxinyu970226, TerraCodes, aude, Aklapper, AnjaJentzsch, Abraham, jeblad, Legoktm, He7d3r, Merl, jayvdb, Denny, revi, matej_suchanek, Lydia_Pintscher, Beta16, daniel, hoo, Nandana, Lahi, Gq86, GoranSMilovanovic, QZanden, LawExplorer, Wikidata-bugs, Mbch331___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Commented On] T44325: Prevent creation of items having the same sitelinks (duplicates)

[Stashbot]

Stashbot added a comment.
Mentioned in SAL (#wikimedia-operations) [2018-10-15T11:29:24Z]  Started rebuildItemsPerSite on mwmaint1002 (T44325). Can be killed at any time, if necessary.TASK DETAILhttps://phabricator.wikimedia.org/T44325EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: StashbotCc: Stashbot, Addshore, Nikki, Liuxinyu970226, TerraCodes, aude, Aklapper, AnjaJentzsch, Abraham, jeblad, Legoktm, He7d3r, Merl, jayvdb, Denny, revi, matej_suchanek, Lydia_Pintscher, Beta16, daniel, hoo, Nandana, Lahi, Gq86, GoranSMilovanovic, QZanden, LawExplorer, Wikidata-bugs, Mbch331___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Commented On] T44325: Prevent creation of items having the same sitelinks (duplicates)

[hoo]

hoo added a comment.

In T44325#4650472, @Addshore wrote:
@hoo

related to https://www.wikidata.org/wiki/Wikidata:Project_chat/Archive/2014/05#Items_with_same_sitelink and https://www.wikidata.org/wiki/Wikidata:Contact_the_development_team#True_duplicates_clean_up?

What script did you run to generate the list?


I think that was repo/maintenance/rebuildItemsPerSite.php which reports problematic items.

We can re-run it, but should probably only do so after the DC switchback (given the script runs for quite some time). I'll kick it off on Monday (in case everything is fine).TASK DETAILhttps://phabricator.wikimedia.org/T44325EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: hooCc: Addshore, Nikki, Liuxinyu970226, TerraCodes, aude, Aklapper, AnjaJentzsch, Abraham, jeblad, Legoktm, He7d3r, Merl, jayvdb, Denny, revi, matej_suchanek, Lydia_Pintscher, Beta16, daniel, hoo, Nandana, Lahi, Gq86, GoranSMilovanovic, QZanden, LawExplorer, Wikidata-bugs, Mbch331___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Commented On] T44325: Prevent creation of items having the same sitelinks (duplicates)

[hoo]

hoo added a comment.
The problem here is that we do these constraints checks based on the wb_items_per_site table. This is (should be) updated immediately after an edit happened (thus the checks worked correctly after the change got picked up by all database replicas).TASK DETAILhttps://phabricator.wikimedia.org/T44325EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: hooCc: Nikki, Liuxinyu970226, TerraCodes, aude, Aklapper, AnjaJentzsch, Abraham, jeblad, Legoktm, He7d3r, Merl, jayvdb, Denny, revi, matej_suchanek, Lydia_Pintscher, Beta16, daniel, hoo, D3r1ck01, Izno, Luke081515, Wikidata-bugs, Mbch331___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

[Wikidata-bugs] [Maniphest] [Commented On] T44325: Prevent creation of items having the same sitelinks (duplicates)

[Nikki]

Nikki added a comment.
We're still getting quite a few duplicates. In the 2016-02-15 dump I found 951 sitelinks that appear more than once, in the 2016-10-10 dump there are 3914. I haven't checked all of them, but I've already come across a bunch of examples with quite a bit of time between the creations, e.g.:

https://www.wikidata.org/wiki/Q23889992
https://www.wikidata.org/wiki/Q23890002
https://www.wikidata.org/wiki/Q23890013
https://www.wikidata.org/wiki/Q23890309

These are from April 2016. There's nearly half an hour between the first and last one.

https://www.wikidata.org/wiki/Q21066942
https://www.wikidata.org/wiki/Q23760872

The sitelink on the first item was updated in March 2016, the second item was created almost a month later.

https://www.wikidata.org/wiki/Q19656655
https://www.wikidata.org/wiki/Q19972345

The first item was created in March 2015, the second was created over two months later.

Also:

https://www.wikidata.org/wiki/Q3778334
https://www.wikidata.org/wiki/Q3778170
https://www.wikidata.org/wiki/Q3778109
https://www.wikidata.org/wiki/Q3777949
https://www.wikidata.org/wiki/Q3778306
https://www.wikidata.org/wiki/Q3777922

It seems that in May 2015 the pages were combined together and the histories merged. Those didn't involve creating a new item... should there be a different ticket for that?TASK DETAILhttps://phabricator.wikimedia.org/T44325EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: NikkiCc: Nikki, Liuxinyu970226, TerraCodes, aude, Aklapper, AnjaJentzsch, Abraham, jeblad, Legoktm, He7d3r, Merl, jayvdb, Denny, revi, matej_suchanek, Lydia_Pintscher, Beta16, daniel, hoo, D3r1ck01, Izno, Luke081515, Wikidata-bugs, Mbch331___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs