| eranroz added a comment. |
wbc_entity_usage tracking for statement group - PASS
As hoo showed in the previous comments, we get C records, and the table doesn't blowup too much. (currently 2037539 records in that table).
only purge the page when that statement group changes - FAIL
Keep in mind that the original goal of this ticket is:
Similarly, we should record which statement group is used (by property ID), so we only purge the page when that statement group changes.
Here I'm going to use recentchanges table with rc_source=wb as proxy to purges.
While this works in the micro level, it doesn't yet look nice enough in the macro level.
Micro level:
- Selecting a page for which we track on specific claim: select * from wbc_entity_usage wbc_a where wbc_a.eu_aspect like 'C%' and not exists(select * from wbc_entity_usage wbc_b where wbc_b.eu_page_id=wbc_a.eu_page_id and wbc_b.eu_aspect='X') limit 5;
- Doing some change in wikidata that doesn't affect that property indeed doesn't create rc record: select * from recentchanges where rc_source ='wb' and rc_title = 'Γιάννης_Μπουτάρης' and rc_namespace=0 limit 5;/*empty results */
Macro level - there are still many records from WB
select left(rc_timestamp, 8), count(*) from recentchanges where rc_source ='wb' and rc_timestamp>20170918000000 group by left(rc_timestamp, 8); +-----------------------+----------+ | left(rc_timestamp, 8) | count(*) | +-----------------------+----------+ | 20170918 | 93493 | | 20170919 | 31978 | | 20170920 | 39313 | | 20170921 | 5753 | +-----------------------+----------+
Or in hour resolution (since not much time passed since deployment):
select left(rc_timestamp, 10), count(*) from recentchanges where rc_source ='wb' and rc_timestamp>20170918000000 group by left(rc_timestamp, 10); | 2017091800 | 120 | | 2017091801 | 363 | | 2017091802 | 2098 | | 2017091803 | 284 | ... | 2017092000 | 336 | | 2017092001 | 389 | | 2017092002 | 169 | | 2017092003 | 188 |This is probably because there are still as many as X usages as there were previously. I guess the next step to get it fixed is to go on with T172914 ?
Cc: Bawolff, eranroz, Ottomata, PokestarFan, Ladsgroup, Stashbot, gerritbot, Halfak, jcrespo, TomT0m, Hall1467, hoo, zhuyifei1999, Eloquence, Lydia_Pintscher, Sannita, Ainali, Liuxinyu970226, MZMcBride, Ricordisamoa, Micru, jayvdb, Daniel_Mietchen, Tobi_WMDE_SW, Legoktm, Abraham, Wikidata-bugs, liangent, jeremyb, aude, Bianjiang, Aklapper, DixonD, daniel, Lordiis, GoranSMilovanovic, Adik2382, Th3d3v1ls, Ramalepe, Liugev6, QZanden, Marostegui, Lewizho99, Minhnv-2809, Maathavan, Izno, Luke081515, Mbch331, Jay8g, Krenair
_______________________________________________ Wikidata-bugs mailing list [email protected] https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
