Ladsgroup added a comment.
Well, enabling it on all wikis is one thing that we need to do and with T185693: Implement a (more liberal) usage aspect deduplicater (days: 3) I think it's fine to move forward in everywhere. Give me a detailed enabling plan and consider it done.TASK
Lydia_Pintscher removed a project: Wikidata-Former-Sprint-Board.
TASK DETAILhttps://phabricator.wikimedia.org/T151717EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: hoo, Lydia_PintscherCc: Marostegui, Bawolff, eranroz, Ottomata, PokestarFan, Ladsgroup,
matej_suchanek added a parent task: T175376: Parser and Lua functions should track changes to 'formatter URL' property.
TASK DETAILhttps://phabricator.wikimedia.org/T151717EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: hoo, matej_suchanekCc: Marostegui,
hoo removed a project: Patch-For-Review.hoo added a comment.
We're already at 6,213,948 statement usages on cawiki (this is a lot).
The number of usages is still growing strong there, but it seems to be within sane bounds.TASK DETAILhttps://phabricator.wikimedia.org/T151717EMAIL
hoo added a subscriber: Marostegui.hoo added a comment.
Moved the deploy to 8 UTC tomorrow (October 18), per @Marostegui. Will be deployed together with T177155: Find test wiki(s) for new description usage and enable there.TASK DETAILhttps://phabricator.wikimedia.org/T151717EMAIL
hoo added a comment.
The cawiki module has been fixed now, but many other wikis still have the problematic code T178114: Many Wikipedia's Wikidata module iterate over all entity claims if a Statement is searched for by property label.
I suggest to re-enable the usage tracking on cawiki next week
hoo added a comment.
In T151717#3679310, @eranroz wrote:
1, @hoo so how do we move forward with deployment to larger wikis/wikis with more wikidata usage such as cawiki? Do we have an issue with DB size? (e.g requires more thought on wbc_entity_usage design) or is it disabled in cawiki because
hoo added a comment.
I am a bit lost with the estimation- is that realistic, is the number of usages more or less right with your estimation, or is it going to be a quadratic _expression_ because we do not know accurately the number of statement usages per statement?
I guess we can't know for
hoo added a comment.
We still track Other ("O") and All usages ("X") on elwiki like we did before. If we conclude the test there ok, we can stop doing this and move forward here and with T172914.
Please note that no longer tracking O or X usages along also requires T176413: Make
eranroz added a comment.
wbc_entity_usage tracking for statement group - PASS
As hoo showed in the previous comments, we get C records, and the table doesn't blowup too much. (currently 2037539 records in that table).
only purge the page when that statement group changes - FAIL
Keep in mind that
hoo added a comment.
This is in effect now and the first statement usages are coming in on elwiki:
++---+
| No | eu_aspect |
++---+
| 270848 | X |
| 192094 | S |
| 150218 | T |
| 99255 | L.el |
| 79981 | O |
| 6461 |
Bawolff added a comment.
I suspect that fixing this bug will significantly help with T171027 (watchlists being too slow)TASK DETAILhttps://phabricator.wikimedia.org/T151717EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: BawolffCc: Bawolff, eranroz, Ottomata,
Halfak added a subtask: T172905: Lua: Make it possible to load only specific Statements using mw.wikibase.
TASK DETAILhttps://phabricator.wikimedia.org/T151717EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: HalfakCc: Ottomata, PokestarFan, Ladsgroup,
Halfak added a subtask: T172914: mw.wikibase.entity: Use __index to lazy register entity usages.
TASK DETAILhttps://phabricator.wikimedia.org/T151717EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: HalfakCc: Ottomata, PokestarFan, Ladsgroup, Stashbot,
Halfak added a parent task: T173124: Improve wikidata usage tracking via Lua .
TASK DETAILhttps://phabricator.wikimedia.org/T151717EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: HalfakCc: Ottomata, PokestarFan, Ladsgroup, Stashbot, gerritbot, Halfak,
Lydia_Pintscher added a parent task: T169708: Wikidata 2017 Q3 goals.
TASK DETAILhttps://phabricator.wikimedia.org/T151717EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Lydia_PintscherCc: Ladsgroup, Stashbot, gerritbot, Halfak, jcrespo, TomT0m, Hall1467,
hoo added a project: Wikidata-Sprint.
TASK DETAILhttps://phabricator.wikimedia.org/T151717EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: hooCc: gerritbot, Halfak, jcrespo, TomT0m, Hall1467, hoo, zhuyifei1999, Eloquence, Lydia_Pintscher, Sannita, Ainali,
gerritbot added a project: Patch-For-Review.
TASK DETAILhttps://phabricator.wikimedia.org/T151717EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: gerritbotCc: gerritbot, Halfak, jcrespo, TomT0m, Hall1467, hoo, zhuyifei1999, Eloquence, Lydia_Pintscher, Sannita,
matej_suchanek added a parent task: T90436: Improve usage tracking granularity to avoid irrelevant changes showing in the watchlist..
TASK DETAILhttps://phabricator.wikimedia.org/T151717EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: matej_suchanekCc: Halfak,
Lydia_Pintscher added a parent task: T150178: Wikidata 2017 Q2 goals.
TASK DETAILhttps://phabricator.wikimedia.org/T151717EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Lydia_PintscherCc: Halfak, jcrespo, TomT0m, Hall1467, hoo, zhuyifei1999, Eloquence,
Hall1467 added a project: DBA.
TASK DETAILhttps://phabricator.wikimedia.org/T151717EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Hall1467Cc: Halfak, jcrespo, TomT0m, Hall1467, hoo, zhuyifei1999, Eloquence, Lydia_Pintscher, Sannita, Ainali, Liuxinyu970226,
jcrespo added a comment.
how would we generate the kind of estimates you would need in order to sign off on this type of change?
Measure the QPS writes/rows written/percentage of write IOPS you have now, evaluate what is the increase with the new method, and scale with a worst-case-scenario
matej_suchanek added a parent task: T150690: Special:Pages using property/P.
TASK DETAILhttps://phabricator.wikimedia.org/T151717EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: matej_suchanekCc: Halfak, jcrespo, TomT0m, Hall1467, hoo, zhuyifei1999, Eloquence,
daniel added a comment.
@Halfak Using numeric property IDs has the problem that this will not work with federation. We are phasing out numeric IDs wherever we can for that reason. The idea is that you may be using properties from multiple different repositories (typically, wikidata and your own,
Halfak added a comment.
I forgot that we could also use EventLogging or some other high-capacity endpoint for this.
For a good reference (IMO) on doing a data storage requirement analysis, see T146718#2677286TASK DETAILhttps://phabricator.wikimedia.org/T151717EMAIL
daniel added a project: User-Daniel.
TASK DETAILhttps://phabricator.wikimedia.org/T151717EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: danielCc: TomT0m, Hall1467, hoo, zhuyifei1999, Eloquence, Lydia_Pintscher, Sannita, Ainali, Liuxinyu970226, MZMcBride,
matej_suchanek removed projects: Epic, Notice, Tracking, Roadmap.
TASK DETAILhttps://phabricator.wikimedia.org/T151717EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: matej_suchanekCc: Hall1467, hoo, zhuyifei1999, Eloquence, Lydia_Pintscher, Sannita, Ainali,
27 matches
Mail list logo
