[Wikidata-bugs] [Maniphest] [Closed] T194037: Track dependencies for multiple Content objects per page

2018-09-17 Thread daniel
daniel closed this task as "Resolved".daniel moved this task from Backlog to Done on the Multi-Content-Revisions (MCR-SDC Statement Support - phase 3) board. TASK DETAILhttps://phabricator.wikimedia.org/T194037WORKBOARDhttps://phabricator.wikimedia.org/project/board/3464/EMAIL PREFER

[Wikidata-bugs] [Maniphest] [Unblock] T196087: Refactored implementation of MCR page update interface

2018-09-17 Thread daniel
daniel closed subtask T194037: Track dependencies for multiple Content objects per page as "Resolved". TASK DETAILhttps://phabricator.wikimedia.org/T196087EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: danielCc: gerritbot, Aklapper, daniel, CCi

[Wikidata-bugs] [Maniphest] [Unblock] T199352: Deploy Structured Data on Commons with arbitrary Statements

2018-09-17 Thread daniel
daniel closed subtask T194037: Track dependencies for multiple Content objects per page as "Resolved". TASK DETAILhttps://phabricator.wikimedia.org/T199352EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: danielCc: Cparle, Aklapper, daniel, Lahi, P

[Wikidata-bugs] [Maniphest] [Updated] T204582: Verify that AbuseFilter and SpamBlacklist work with Wikibase MediaInfo

2018-09-17 Thread daniel
daniel added a parent task: T199352: Deploy Structured Data on Commons with arbitrary Statements. TASK DETAILhttps://phabricator.wikimedia.org/T204582EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: danielCc: Aklapper, CCicalese_WMF, Ramsey-WMF, daniel, Lahi

[Wikidata-bugs] [Maniphest] [Updated] T204582: Verify that AbuseFilter and SpamBlacklist work with Wikibase MediaInfo

2018-09-17 Thread daniel
daniel added a project: MediaWiki-extensions-WikibaseMediaInfo. TASK DETAILhttps://phabricator.wikimedia.org/T204582EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: danielCc: Aklapper, CCicalese_WMF, Ramsey-WMF, daniel, Lahi, PDrouin-WMF, Gq86, E1presidente

[Wikidata-bugs] [Maniphest] [Updated] T199352: Deploy Structured Data on Commons with arbitrary Statements

2018-09-17 Thread daniel
daniel added a subtask: T204582: Verify that AbuseFilter and SpamBlacklist work with Wikibase MediaInfo. TASK DETAILhttps://phabricator.wikimedia.org/T199352EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: danielCc: Cparle, Aklapper, daniel, Lahi, PDrouin-WMF

[Wikidata-bugs] [Maniphest] [Unblock] T194037: Track dependencies for multiple Content objects per page

2018-09-17 Thread daniel
daniel closed subtask T194038: Introduce ContentHandler::getSecondaryDataUpdates to replace Content::getSecondaryDataUpdates as "Resolved". TASK DETAILhttps://phabricator.wikimedia.org/T194037EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: danielCc

[Wikidata-bugs] [Maniphest] [Closed] T194038: Introduce ContentHandler::getSecondaryDataUpdates to replace Content::getSecondaryDataUpdates

2018-09-17 Thread daniel
daniel closed this task as "Resolved".daniel moved this task from In Progress to Done on the Multi-Content-Revisions (MCR-SDC Statement Support - phase 3) board. TASK DETAILhttps://phabricator.wikimedia.org/T194038WORKBOARDhttps://phabricator.wikimedia.org/project/board/3464/EMAIL PREFER

[Wikidata-bugs] [Maniphest] [Created] T204582: Verify that AbuseFilter and SpamBlacklist work with Wikibase MediaInfo

2018-09-17 Thread daniel
daniel created this task.daniel added a project: Structured-Data-Commons.Restricted Application added a subscriber: Aklapper.Restricted Application added a project: Wikidata. TASK DESCRIPTIONBefore deploying the SDC baseline, we need to be sure that vandalism and spam is caught. Two important

[Wikidata-bugs] [Maniphest] [Edited] T198308: Enable MCR migration stage "write both, read new" on live systems

2018-09-17 Thread daniel
daniel updated the task description. (Show Details) CHANGES TO TASK DESCRIPTION...WARNING: Because of cross-wiki revision access, this can only be done an ANY system on the cluster once ALL systems on the cluster write the new schema.This is blocked on operations being switched back to eqiad per

[Wikidata-bugs] [Maniphest] [Updated] T183487: MCR schema migration stage 3: drop support for legacy fields (wmf production)

2018-09-14 Thread daniel
daniel added a comment. @awight It is, via T183490. https://wikifarm.wmflabs.org/mcr/index.php/Deployment_Pert_ChartTASK DETAILhttps://phabricator.wikimedia.org/T183487EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: danielCc: awight, gerritbot, Tgr

[Wikidata-bugs] [Maniphest] [Unblock] T174044: Deploy MCR storage layer

2018-09-12 Thread daniel
daniel closed subtask T198561: Make "write both, read new" the default MCR migration stage for fresh MediaWiki installs / for CI as "Resolved". TASK DETAILhttps://phabricator.wikimedia.org/T174044EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpref

[Wikidata-bugs] [Maniphest] [Closed] T204065: WikibaseLexeme CI broken (database errors)

2018-09-12 Thread daniel
daniel closed this task as "Resolved". TASK DETAILhttps://phabricator.wikimedia.org/T204065EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: danielCc: Jdforrester-WMF, greg, gerritbot, WMDE-leszek, Pablo-WMDE, hashar, TerraCodes, Liuxinyu9702

[Wikidata-bugs] [Maniphest] [Closed] T198561: Make "write both, read new" the default MCR migration stage for fresh MediaWiki installs / for CI

2018-09-12 Thread daniel
daniel closed this task as "Resolved".daniel moved this task from In Progress to Done on the Multi-Content-Revisions (MCR Deployment) board.daniel added a comment. let's hope it stays closed this time :)TASK DETAILhttps://phabricator.wikimedia.org/T198561WORKBOARDhttps://phabricato

[Wikidata-bugs] [Maniphest] [Unblock] T198308: Enable MCR migration stage "write both, read new" on live systems

2018-09-12 Thread daniel
daniel closed subtask T198561: Make "write both, read new" the default MCR migration stage for fresh MediaWiki installs / for CI as "Resolved". TASK DETAILhttps://phabricator.wikimedia.org/T198308EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpref

[Wikidata-bugs] [Maniphest] [Unblock] T194750: Deploy Structured Data on Commons baseline

2018-09-12 Thread daniel
daniel closed subtask T174036: Diffs page should show diffs and content from multiple slots [MCR] as "Resolved". TASK DETAILhttps://phabricator.wikimedia.org/T194750EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: danielCc: Cparle, Aklapper, Abit,

[Wikidata-bugs] [Maniphest] [Closed] T174036: Diffs page should show diffs and content from multiple slots [MCR]

2018-09-12 Thread daniel
daniel closed this task as "Resolved".daniel moved this task from Epic to Done on the Multi-Content-Revisions (MCR-SDC File Caption Support - phase 2) board. TASK DETAILhttps://phabricator.wikimedia.org/T174036WORKBOARDhttps://phabricator.wikimedia.org/project/board/3374/EMAIL PREFER

[Wikidata-bugs] [Maniphest] [Commented On] T204024: Store WikibaseQualityConstraint check data in an SQL table instead of in the cache

2018-09-12 Thread daniel
daniel added a comment. This use case seems similar to caching parsoid HTML, which is done in RESTbase and backed by Cassandra. It's similar, because it's re-generated upon edit, and accessed from clients upon view, via an API.TASK DETAILhttps://phabricator.wikimedia.org/T2

[Wikidata-bugs] [Maniphest] [Unblock] T174036: Diffs page should show diffs and content from multiple slots [MCR]

2018-09-12 Thread daniel
daniel closed subtask T174035: Allow the view action to show multiple slots [MCR] as "Resolved". TASK DETAILhttps://phabricator.wikimedia.org/T174036EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: danielCc: CCicalese_WMF, gerritbot, T

[Wikidata-bugs] [Maniphest] [Closed] T174035: Allow the view action to show multiple slots [MCR]

2018-09-12 Thread daniel
daniel closed this task as "Resolved".daniel moved this task from Needs Review to Done on the Multi-Content-Revisions (MCR-SDC File Caption Support - phase 2) board. TASK DETAILhttps://phabricator.wikimedia.org/T174035WORKBOARDhttps://phabricator.wikimedia.org/project/board/

[Wikidata-bugs] [Maniphest] [Commented On] T97368: Investigate more efficient memcached solution for CacheAwarePropertyInfoStore

2018-09-12 Thread daniel
daniel added a comment. Another option would be to force this to go into APC instead of memcached.TASK DETAILhttps://phabricator.wikimedia.org/T97368EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: danielCc: Stashbot, gerritbot, Jdforrester-WMF, Joe, mark

[Wikidata-bugs] [Maniphest] [Block] T198309: Enable MCR migration stage "write both, read new" on testwiki

2018-09-11 Thread daniel
daniel reopened subtask T198561: Make "write both, read new" the default MCR migration stage for fresh MediaWiki installs / for CI as "Open". TASK DETAILhttps://phabricator.wikimedia.org/T198309EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpref

[Wikidata-bugs] [Maniphest] [Reopened] T198561: Make "write both, read new" the default MCR migration stage for fresh MediaWiki installs / for CI

2018-09-11 Thread daniel
daniel reopened this task as "Open".daniel moved this task from Done to In Progress on the Multi-Content-Revisions (MCR Deployment) board.daniel added a comment. Reverted by https://gerrit.wikimedia.org/r/c/mediawiki/core/+/459812 to fix T204065: Wikibase CI broken (database er

[Wikidata-bugs] [Maniphest] [Block] T174044: Deploy MCR storage layer

2018-09-11 Thread daniel
daniel reopened subtask T198561: Make "write both, read new" the default MCR migration stage for fresh MediaWiki installs / for CI as "Open". TASK DETAILhttps://phabricator.wikimedia.org/T174044EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpref

[Wikidata-bugs] [Maniphest] [Block] T198308: Enable MCR migration stage "write both, read new" on live systems

2018-09-11 Thread daniel
daniel reopened subtask T198561: Make "write both, read new" the default MCR migration stage for fresh MediaWiki installs / for CI as "Open". TASK DETAILhttps://phabricator.wikimedia.org/T198308EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/

[Wikidata-bugs] [Maniphest] [Commented On] T174036: Diffs page should show diffs and content from multiple slots [MCR]

2018-09-11 Thread daniel
daniel added a comment. I don't think "make MCR feature-complete" can happen in time for 1.32 Yea, i meant "able to properly function", not "used in all places where we want to use it in core". Having MCR without the ability of atomic edits is still ok. Ha

[Wikidata-bugs] [Maniphest] [Unblock] T174044: Deploy MCR storage layer

2018-09-11 Thread daniel
daniel closed subtask T198561: Make "write both, read new" the default MCR migration stage for fresh MediaWiki installs / for CI as "Resolved". TASK DETAILhttps://phabricator.wikimedia.org/T174044EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpref

[Wikidata-bugs] [Maniphest] [Closed] T198561: Make "write both, read new" the default MCR migration stage for fresh MediaWiki installs / for CI

2018-09-11 Thread daniel
daniel closed this task as "Resolved". TASK DETAILhttps://phabricator.wikimedia.org/T198561EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: danielCc: gerritbot, greg, Aklapper, CCicalese_WMF, Fjalapeno, daniel, Gaboe420, Versusxo, Majestic

[Wikidata-bugs] [Maniphest] [Unblock] T198308: Enable MCR migration stage "write both, read new" on live systems

2018-09-11 Thread daniel
daniel closed subtask T198561: Make "write both, read new" the default MCR migration stage for fresh MediaWiki installs / for CI as "Resolved". TASK DETAILhttps://phabricator.wikimedia.org/T198308EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpref

[Wikidata-bugs] [Maniphest] [Commented On] T174047: Hide deprecated/unused fields on toolforge replica [MCR]

2018-09-11 Thread daniel
daniel added a comment. @jcrespo sounds good to me, yet. Compare also https://www.wikidata.org/wiki/Wikidata:Stable_Interface_Policy, which states that changes to the database schema are announced like changes to a public API, but less effort is made to provide backwards compatibility as would be

[Wikidata-bugs] [Maniphest] [Edited] T174047: Hide deprecated/unused fields on toolforge replica [MCR]

2018-09-11 Thread daniel
daniel updated the task description. (Show Details) CHANGES TO TASK DESCRIPTION...Tools that need to know the content model of a revision's content can look it up using the new `slots` and `content` tables, which are already available on labs, but not yet populated. Populating these tables is

[Wikidata-bugs] [Maniphest] [Updated] T174047: Hide deprecated/unused fields on toolforge replica [MCR]

2018-09-11 Thread daniel
daniel added a comment. I don't think because fields have been added but not yet removed, but I cannot promise that won't happen soon (it is not in my hand, it is the mw core maintainers who handle that) or it is happening now (we should test on deploy on a smaller db first). No fields

[Wikidata-bugs] [Maniphest] [Commented On] T198308: Enable MCR migration stage "write both, read new" on live systems

2018-09-11 Thread daniel
daniel added a comment. We should probably not do all wikis immediately. Perhaps we can do something like this: testwiki. watch for bugs. mediawikiwiki and metawiki. watch for bugs. commonswiki and dewiki. watch for performance issues. all the rest. TASK DETAILhttps

[Wikidata-bugs] [Maniphest] [Updated] T174036: Diffs page should show diffs and content from multiple slots [MCR]

2018-09-11 Thread daniel
daniel added a project: MW-1.32-release.daniel added a comment. Tagging for 1.32 release, since this is needed to make MCR feature-complete.TASK DETAILhttps://phabricator.wikimedia.org/T174036EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: danielCc

[Wikidata-bugs] [Maniphest] [Updated] T194037: Track dependencies for multiple Content objects per page

2018-09-11 Thread daniel
daniel added a project: MW-1.32-release.daniel added a comment. Tagging for 1.32 release, since this is needed to make MCR feature-complete.TASK DETAILhttps://phabricator.wikimedia.org/T194037EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: danielCc: Aklapper

[Wikidata-bugs] [Maniphest] [Updated] T198308: Enable MCR migration stage "write both, read new" on live systems

2018-09-10 Thread daniel
daniel added a project: MW-1.32-release. TASK DETAILhttps://phabricator.wikimedia.org/T198308EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: danielCc: Keegan, Fjalapeno, CCicalese_WMF, Aklapper, Tgr, Anomie, Abit, daniel, Lahi, PDrouin-WMF, Gq86, E1presidente

[Wikidata-bugs] [Maniphest] [Edited] T198308: Enable MCR migration stage "write both, read new" on live systems

2018-09-10 Thread daniel
daniel updated the task description. (Show Details) CHANGES TO TASK DESCRIPTION...NOTE: we'll want to have this enabled on the live systems for a while before releasing it as the default for 3rd party installs, see T198561 with #MW-1.32-release. But we'll want to have this as the default

[Wikidata-bugs] [Maniphest] [Updated] T194037: Track dependencies for multiple Content objects per page

2018-09-10 Thread daniel
daniel removed a project: Patch-For-Review. TASK DETAILhttps://phabricator.wikimedia.org/T194037EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: danielCc: Aklapper, gerritbot, daniel, Lahi, PDrouin-WMF, Gq86, E1presidente, Ramsey-WMF, Cparle, Anooprao

[Wikidata-bugs] [Maniphest] [Updated] T198706: Make BackupDumper, WikiExporter and XmlDumpWriter compliant with the MCR revision retrival mechanism (main slot only)

2018-09-07 Thread daniel
daniel added a subtask: T203424: Replace the WikiExporter backup dump streaming mode with batched queries. TASK DETAILhttps://phabricator.wikimedia.org/T198706EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: danielCc: ArielGlenn, gerritbot, CCicalese_WMF

[Wikidata-bugs] [Maniphest] [Unblock] T194037: Track dependencies for multiple Content objects per page

2018-09-05 Thread daniel
daniel closed subtask T194043: Replace usages of Content::getSecondaryDataUpdates as "Resolved". TASK DETAILhttps://phabricator.wikimedia.org/T194037EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: danielCc: Aklapper, gerritbot, daniel, Gaboe420

[Wikidata-bugs] [Maniphest] [Closed] T194043: Replace usages of Content::getSecondaryDataUpdates

2018-09-05 Thread daniel
daniel closed this task as "Resolved".daniel moved this task from In Progress to Done on the Multi-Content-Revisions (MCR-SDC Statement Support - phase 3) board. TASK DETAILhttps://phabricator.wikimedia.org/T194043WORKBOARDhttps://phabricator.wikimedia.org/project/board/3464/EMAIL PREFER

[Wikidata-bugs] [Maniphest] [Unblock] T198561: Make "write both, read new" the default MCR migration stage for fresh MediaWiki installs / for CI

2018-08-29 Thread daniel
daniel closed subtask T201164: Temporarily disable deprecation warnings for code that accesses rev_text_id or the text table directly as "Resolved". TASK DETAILhttps://phabricator.wikimedia.org/T198561EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To

[Wikidata-bugs] [Maniphest] [Closed] T201164: Temporarily disable deprecation warnings for code that accesses rev_text_id or the text table directly

2018-08-29 Thread daniel
daniel closed this task as "Resolved".daniel claimed this task. TASK DETAILhttps://phabricator.wikimedia.org/T201164EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: danielCc: Fjalapeno, CCicalese_WMF, Aklapper, greg, gerritbot, daniel, Gaboe420

[Wikidata-bugs] [Maniphest] [Commented On] T183488: MCR schema migration stage 2: populate new fields

2018-08-27 Thread daniel
daniel added a comment. It looks like the "fake" SlotRecord created by RevisionStore::newRevisionFromArchiveRow() passes the $slot->hasRevision() check in RevisionStore::insertRevisionInternal() so it doesn't actually insert it. That's as designed (well, partially): when un

[Wikidata-bugs] [Maniphest] [Commented On] T199121: RFC: Spec for representing multiple content objects per revision (MCR) in XML dumps

2018-08-27 Thread daniel
daniel added a comment. we'd have to have someplace for third-party installations to store their revision text, and whatever that mechanism is (not necessarily an external store), would be supported along with the external store. We also need to support installations that do not enable MCR.

[Wikidata-bugs] [Maniphest] [Commented On] T183488: MCR schema migration stage 2: populate new fields

2018-08-27 Thread daniel
daniel added a comment. In T183488#4535083, @Anomie wrote: I'm running queries on all wikis to see if there are any revision or archive rows lacking a slots row. That found T202904 so far. Wow, nice find!TASK DETAILhttps://phabricator.wikimedia.org/T183488EMAIL PREFERENCES

[Wikidata-bugs] [Maniphest] [Commented On] T183488: MCR schema migration stage 2: populate new fields

2018-08-27 Thread daniel
daniel added a comment. @Anomie What do you think of re-running the script on a few more wikis, as a spot check to verify that it finds nothing to do? If otoh the script finds stuff to migrate, something is wrong...TASK DETAILhttps://phabricator.wikimedia.org/T183488EMAIL PREFERENCEShttps

[Wikidata-bugs] [Maniphest] [Commented On] T202676: Support non-Q and non-P items in custom Wikibase installations

2018-08-27 Thread daniel
daniel added a comment. Note from the past: we initially intended the prefixes to be configurable, but this turned out to be rather painful to implement consistently. So we dropped this possibility in favor of a guarantee that the prefix identifies the entity type. Breaking that guarantee is

[Wikidata-bugs] [Maniphest] [Commented On] T189808: Make undo work with multiple slots.

2018-08-25 Thread daniel
daniel added a comment. In T189808#4532284, @CCicalese_WMF wrote: Thanks for catching that! It should not block T194750: Deploy Structured Data on Commons baseline . I think we went back and forth on this a few times, and ended up pulling it into phase 2. We can drop it again, but it'S n

[Wikidata-bugs] [Maniphest] [Commented On] T199121: RFC: Spec for representing multiple content objects per revision (MCR) in XML dumps

2018-08-24 Thread daniel
daniel added a comment. Question: what is address="tt:12345" in the alternative proposal here https://www.mediawiki.org/wiki/Multi-Content_Revisions/Dumps ? That's the replacement for the text id, for use with stub dumps. We now have a blob store that resolves url-style addresses.

[Wikidata-bugs] [Maniphest] [Commented On] T202706: Merge can not create redirect

2018-08-24 Thread daniel
daniel added a comment. @Bugreporter which entities did you try to merge?TASK DETAILhttps://phabricator.wikimedia.org/T202706EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Addshore, danielCc: daniel, Addshore, Aklapper, Bugreporter, stebsco, Lahi, Gq86

[Wikidata-bugs] [Maniphest] [Closed] T197816: Enable MCR migration stage "write both, read old" on live systems

2018-08-22 Thread daniel
daniel closed this task as "Resolved". TASK DETAILhttps://phabricator.wikimedia.org/T197816EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: tstarling, danielCc: Stashbot, Aklapper, gerritbot, aude, Addshore, Anomie, Jdforrester-WMF, Abit, daniel

[Wikidata-bugs] [Maniphest] [Unblock] T183488: MCR schema migration stage 2: populate new fields

2018-08-22 Thread daniel
daniel closed subtask T197816: Enable MCR migration stage "write both, read old" on live systems as "Resolved". TASK DETAILhttps://phabricator.wikimedia.org/T183488EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: tstarling, danielCc: jcre

[Wikidata-bugs] [Maniphest] [Updated] T198309: Enable MCR migration stage "write both, read new" on testwiki

2018-08-22 Thread daniel
daniel added a subtask: T198561: Make "write both, read new" the default MCR migration stage for fresh MediaWiki installs / for CI. TASK DETAILhttps://phabricator.wikimedia.org/T198309EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: danielCc: Abit, A

[Wikidata-bugs] [Maniphest] [Updated] T198561: Make "write both, read new" the default MCR migration stage for fresh MediaWiki installs / for CI

2018-08-22 Thread daniel
daniel added a parent task: T198309: Enable MCR migration stage "write both, read new" on testwiki. TASK DETAILhttps://phabricator.wikimedia.org/T198561EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: danielCc: gerritbot, greg, Aklapper, CCi

[Wikidata-bugs] [Maniphest] [Block] T194750: Deploy Structured Data on Commons baseline

2018-08-22 Thread daniel
daniel reopened subtask T197816: Enable MCR migration stage "write both, read old" on live systems as "Open". TASK DETAILhttps://phabricator.wikimedia.org/T194750EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: danielCc: Cparle, Aklapp

[Wikidata-bugs] [Maniphest] [Block] T183488: MCR schema migration stage 2: populate new fields

2018-08-22 Thread daniel
daniel reopened subtask T197816: Enable MCR migration stage "write both, read old" on live systems as "Open". TASK DETAILhttps://phabricator.wikimedia.org/T183488EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: tstarling, danielCc: jcre

[Wikidata-bugs] [Maniphest] [Reopened] T197816: Enable MCR migration stage "write both, read old" on live systems

2018-08-22 Thread daniel
daniel reopened this task as "Open".daniel added a comment. Reopening, since this is not quite done: we need the current default to be ''explicit'' in the live config. Otherwise, we'll change the config of the live cluster when we change the default for master an

[Wikidata-bugs] [Maniphest] [Commented On] T202483: www.mediawiki.org showing: error:Unknown database 'wikidatawiki' on shard: s3

2018-08-22 Thread daniel
daniel added a comment. In T202483#4522033, @Addshore wrote: In T202483#4522020, @daniel wrote: I wonder why this wasn't caught on beta. We really need to do more automated testing there. beta has a single shard, so there was no chance it will catch issues like this https://githu

[Wikidata-bugs] [Maniphest] [Commented On] T202483: www.mediawiki.org showing: error:Unknown database 'wikidatawiki' on shard: s3

2018-08-22 Thread daniel
daniel added a comment. I wonder why this wasn't caught on beta. We really need to do more automated testing there.TASK DETAILhttps://phabricator.wikimedia.org/T202483EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: danielCc: gerritbot, Addshore, A

[Wikidata-bugs] [Maniphest] [Changed Project Column] T174036: Diffs page should show diffs and content from multiple slots [MCR]

2018-08-21 Thread daniel
daniel moved this task from In Progress to Epic on the Multi-Content-Revisions (MCR-SDC File Caption Support - phase 2) board.daniel added a comment. Moving this to the "eipc" column, since this is purely a tracking task. There is nothing to do here.TASK DETAILhttps://phabricator.wik

[Wikidata-bugs] [Maniphest] [Closed] T194731: Show diffs for all slots [MCR]

2018-08-21 Thread daniel
daniel closed this task as "Resolved".daniel moved this task from Needs Review to Done on the Multi-Content-Revisions (MCR-SDC File Caption Support - phase 2) board. TASK DETAILhttps://phabricator.wikimedia.org/T194731WORKBOARDhttps://phabricator.wikimedia.org/project/board/

[Wikidata-bugs] [Maniphest] [Unblock] T200216: Make undo work with SDC by showing a UI to confirm undo without allowing an edit

2018-08-21 Thread daniel
daniel closed subtask T194731: Show diffs for all slots [MCR] as "Resolved". TASK DETAILhttps://phabricator.wikimedia.org/T200216EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Anomie, danielCc: daniel, Liuxinyu970226, TomT0m, Smalyshev, Lokal_Pr

[Wikidata-bugs] [Maniphest] [Unblock] T174036: Diffs page should show diffs and content from multiple slots [MCR]

2018-08-21 Thread daniel
daniel closed subtask T194731: Show diffs for all slots [MCR] as "Resolved". TASK DETAILhttps://phabricator.wikimedia.org/T174036EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: danielCc: gerritbot, Tgr, Anomie, Liuxinyu970226, TomT0m, Smalyshev, Lo

[Wikidata-bugs] [Maniphest] [Unblock] T194731: Show diffs for all slots [MCR]

2018-08-21 Thread daniel
daniel closed subtask T201842: Use ContentHandler to obtain DifferenceEngine in MobileFrontend as "Resolved". TASK DETAILhttps://phabricator.wikimedia.org/T194731EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Tgr, danielCc: Jdlrobson, pmiazga

[Wikidata-bugs] [Maniphest] [Commented On] T202032: Duplicate ar_rev_id values in several wikis

2018-08-21 Thread daniel
daniel added a comment. @Anomie I outlined an idea on the patch, can you tell me if that would work? I'm not sure I got the locking behavior right.TASK DETAILhttps://phabricator.wikimedia.org/T202032EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: A

[Wikidata-bugs] [Maniphest] [Commented On] T202032: Duplicate ar_rev_id values in several wikis

2018-08-21 Thread daniel
daniel added a comment. I just got hit by `Error: 1062 Duplicate entry '2871-1' for key 'PRIMARY' (localhost)` on my local machine, very likely caused by me deleting some pages, and then rebooting, causing mysql to be re-started. While this situation may be rare enough on a pro

[Wikidata-bugs] [Maniphest] [Unblock] T159708: Deploy WikibaseMediaInfo extension to production

2018-08-20 Thread daniel
daniel closed subtask T194729: Allow Wikibase Entities to be stored in alternative slots [MCR] as "Resolved". TASK DETAILhttps://phabricator.wikimedia.org/T159708EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: danielCc: Jdforrester-WMF

[Wikidata-bugs] [Maniphest] [Unblock] T180981: Deploy WikibaseMediaInfo extension to beta

2018-08-20 Thread daniel
daniel closed subtask T194729: Allow Wikibase Entities to be stored in alternative slots [MCR] as "Resolved". TASK DETAILhttps://phabricator.wikimedia.org/T180981EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: danielCc: gerritbot, Lydia_Pintscher

[Wikidata-bugs] [Maniphest] [Closed] T194729: Allow Wikibase Entities to be stored in alternative slots [MCR]

2018-08-20 Thread daniel
daniel closed this task as "Resolved". TASK DETAILhttps://phabricator.wikimedia.org/T194729EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: danielCc: Jdforrester-WMF, Addshore, Cparle, daniel, Gaboe420, Versusxo, Majesticalreaper22, Giuliamocci,

[Wikidata-bugs] [Maniphest] [Commented On] T202032: Duplicate ar_rev_id values in several wikis

2018-08-16 Thread daniel
daniel added a comment. We'd then want to check in case any of the reassigned rows are the ones that populateContentTables.php had populated (versus being the rows that it errored out on). Or we could just blank the slots and content tables. populateContentTables.php failed when hitting the

[Wikidata-bugs] [Maniphest] [Commented On] T183488: MCR schema migration stage 2: populate new fields

2018-08-16 Thread daniel
daniel added a comment. Duplicates are prevented by the unique index on slot_revision_id/slot_role_id. Orphan slot rows are possible, just as orphan revision rows have been possible before. We can run a query to detect those on some wikis as a spot check. If we find nothing, I don't think

[Wikidata-bugs] [Maniphest] [Commented On] T183488: MCR schema migration stage 2: populate new fields

2018-08-16 Thread daniel
daniel added a comment. @jcrespo Ooops, I updated my comment after you already answered, sorry.TASK DETAILhttps://phabricator.wikimedia.org/T183488EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: tstarling, danielCc: jcrespo, greg, tstarling, Stashbot, Abit

[Wikidata-bugs] [Maniphest] [Commented On] T183488: MCR schema migration stage 2: populate new fields

2018-08-16 Thread daniel
daniel added a comment. it would be nice to run a read-only process to check the consistency of the migration We did pretty extensive testing of this on the test copies (db and db1112). We could have a script that does basic sanity checks, but I'm not sure what to test, beyond the tr

[Wikidata-bugs] [Maniphest] [Updated] T202032: Duplicate ar_rev_id values in several wikis

2018-08-16 Thread daniel
daniel added a comment. I just realized, we already fixed duplicate ar_rev_id entries before... And then the problem re-appeared? That doesn't sound good. I re-opened T193180: Clean up archive rows with duplicate revision IDs and made it a blocker for this task. @Anomie, can you look into

[Wikidata-bugs] [Maniphest] [Raised Priority] T193180: Clean up archive rows with duplicate revision IDs

2018-08-16 Thread daniel
daniel raised the priority of this task from "Normal" to "High".daniel added a comment. Bumping to high, since this now blocks the completion of T183488.TASK DETAILhttps://phabricator.wikimedia.org/T193180EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpr

[Wikidata-bugs] [Maniphest] [Reopened] T193180: Clean up archive rows with duplicate revision IDs

2018-08-16 Thread daniel
daniel reopened this task as "Open".daniel added a comment. Re-opening, since we again have multiple archive rows with the same ar_rev_id on several wikis, see T202032.TASK DETAILhttps://phabricator.wikimedia.org/T193180EMAIL PREFERENCEShttps://phabricator.wikimedia.org/sett

[Wikidata-bugs] [Maniphest] [Block] T183489: MCR schema migration stage 1: Fix Legacy Archive Rows

2018-08-16 Thread daniel
daniel reopened subtask T193180: Clean up archive rows with duplicate revision IDs as "Open". TASK DETAILhttps://phabricator.wikimedia.org/T183489EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Anomie, danielCc: Abit, Jdforrester-WMF, Aklapper, aude

[Wikidata-bugs] [Maniphest] [Updated] T202032: Duplicate ar_rev_id values in several wikis

2018-08-16 Thread daniel
daniel added a subtask: T193180: Clean up archive rows with duplicate revision IDs. TASK DETAILhttps://phabricator.wikimedia.org/T202032EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: tstarling, danielCc: gerritbot, Aklapper, daniel, aude, Addshore, Anomie

[Wikidata-bugs] [Maniphest] [Updated] T193180: Clean up archive rows with duplicate revision IDs

2018-08-16 Thread daniel
daniel added a parent task: T202032: Duplicate ar_rev_id values in several wikis. TASK DETAILhttps://phabricator.wikimedia.org/T193180EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Anomie, danielCc: Aklapper, gerritbot, greg, Stashbot, CCicalese_WMF, daniel

[Wikidata-bugs] [Maniphest] [Commented On] T202032: Duplicate ar_rev_id values in several wikis

2018-08-16 Thread daniel
daniel added a comment. The patch I made should protect against the edge case that @jcrespo mentioned. Not sure that it's worth the overhead given how unlikely we are to hit that situation, but that can be discussed on the ticket.TASK DETAILhttps://phabricator.wikimedia.org/T202032

[Wikidata-bugs] [Maniphest] [Commented On] T202032: Duplicate ar_rev_id values in several wikis

2018-08-16 Thread daniel
daniel added a comment. The MCR write-both mode, with its unique index on slot_revision_id, should at least prevent conflicting rows from being inserted, although this potentially comes at the cost of throwing an exception on every edit until someone manually advances the autoincrement value for

[Wikidata-bugs] [Maniphest] [Commented On] T174035: Allow the view action to show multiple slots [MCR]

2018-08-14 Thread daniel
daniel added a comment. This is work in progress, but here's a first screenshot! F24939656: Bildschirmfoto vom 2018-08-14 18-40-25.pngTASK DETAILhttps://phabricator.wikimedia.org/T174035EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: danielCc: Tgr, gerr

[Wikidata-bugs] [Maniphest] [Claimed] T174036: Diffs page should show diffs and content from multiple slots [MCR]

2018-08-14 Thread daniel
daniel claimed this task.daniel added a subscriber: Tgr. TASK DETAILhttps://phabricator.wikimedia.org/T174036EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: danielCc: Tgr, Anomie, Liuxinyu970226, TomT0m, Smalyshev, Lokal_Profil, -jem-, Aklapper, daniel, Lahi

[Wikidata-bugs] [Maniphest] [Commented On] T199121: RFC: Spec for representing multiple content objects per revision (MCR) in XML dumps

2018-08-14 Thread daniel
daniel added a comment. question about possible changes to slot role names Slot role names are public identifiers, they cannot change, ever. Same for model names.TASK DETAILhttps://phabricator.wikimedia.org/T199121EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences

[Wikidata-bugs] [Maniphest] [Claimed] T174035: Allow the view action to show multiple slots [MCR]

2018-08-14 Thread daniel
daniel claimed this task.daniel added a subscriber: Tgr. TASK DETAILhttps://phabricator.wikimedia.org/T174035EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: danielCc: Tgr, gerritbot, Aklapper, daniel, Gaboe420, Versusxo, Majesticalreaper22, Giuliamocci

[Wikidata-bugs] [Maniphest] [Commented On] T183488: MCR schema migration stage 2: populate new fields

2018-08-14 Thread daniel
daniel added a comment. aawikibooks failed with "Error: 1062 Duplicate entry '3003-1' for key 'PRIMARY' (10.64.0.205)" It's annoying that this error message doesn't say which *table* this was on. My guess is that it'S the slots table, which has

[Wikidata-bugs] [Maniphest] [Commented On] T194731: Show diffs for all slots [MCR]

2018-08-13 Thread daniel
daniel added a comment. In T194731#4498498, @Tgr wrote: Went through the high and medium priority steps of the test plan. MobileFrontend diffs break (use TextSlotDiffRenderer instead of InlineDifferenceEngine). MobileFrontend hand-creates the InlineDiffRenderer so it does not get used for the

[Wikidata-bugs] [Maniphest] [Commented On] T199121: RFC: Spec for representing multiple content objects per revision (MCR) in XML dumps

2018-08-10 Thread daniel
daniel added a comment. SAX parsers would typically take the last and get confused True, that's a risk - they would then be using the wrong slot.TASK DETAILhttps://phabricator.wikimedia.org/T199121EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Ariel

[Wikidata-bugs] [Maniphest] [Commented On] T199121: RFC: Spec for representing multiple content objects per revision (MCR) in XML dumps

2018-08-10 Thread daniel
daniel added a comment. CR-aware clients will maybe need very slightly less elegant parsing code, which does not seem like a bad trade-off. It's an option, but it feels like we'd be shifting the paint to consumers, who have to maintain that duplicate logic forever. And I don't see

[Wikidata-bugs] [Maniphest] [Commented On] T199121: RFC: Spec for representing multiple content objects per revision (MCR) in XML dumps

2018-08-10 Thread daniel
daniel added a comment. Introducing a tag that wraps a tag and additional tags for meta-data looks nice and clean, but it's also completely incompatible with existing consumers. Just generating multiple tags, and addition any new info as attributes on them, would allow existing consumer

[Wikidata-bugs] [Maniphest] [Commented On] T199121: RFC: Spec for representing multiple content objects per revision (MCR) in XML dumps

2018-08-10 Thread daniel
daniel added a comment. I'm not sure the transitional format is useful. What client would use it, and what for? A client that wants to process the transitional format would need to implement handling for two different XML structures, and actually use both in parallel. A client that still reli

[Wikidata-bugs] [Maniphest] [Changed Subscribers] T199121: RFC: Spec for representing multiple content objects per revision (MCR) in XML dumps

2018-08-10 Thread daniel
daniel added a subscriber: tstarling.daniel added a comment. @ArielGlenn Thanks for setting this up! First thing I noticed: I think it would be a bad idea to expose the numeric IDs of content models and slot roles. These numeric IDs are strictly internal to the storage layer. They are not even to

[Wikidata-bugs] [Maniphest] [Commented On] T199121: RFC: Spec for representing multiple content objects per revision (MCR) in XML dumps

2018-08-08 Thread daniel
daniel added a comment. @ArielGlenn I'm happy to be a co-author, I just can't invest much time into this right now. Thanks for taking care of this!TASK DETAILhttps://phabricator.wikimedia.org/T199121EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To:

[Wikidata-bugs] [Maniphest] [Commented On] T199096: Add support for wikidata summaries in the /page/summary/ endpoint

2018-08-07 Thread daniel
daniel added a comment. A quick thought on how events for purging the cached preview could be triggered by Wikibase: Any Wikibase client (including wikidata.org itself) receives "change notifications", which are picked up by the ChangeHandler, and can be intercepted by the WikibaseHa

[Wikidata-bugs] [Maniphest] [Updated] T199121: RFC: Spec for representing multiple content objects per revision (MCR) in XML dumps

2018-08-06 Thread daniel
daniel added a comment. What happens to the text table? Does it become unused as all content is migrated into the MCR tables? For now, the text table stays exactly as it is, but some time in the foreseeable future rev_text_id will no longer be used, and eventually dropped. At some point, text

[Wikidata-bugs] [Maniphest] [Claimed] T194729: Allow Wikibase Entities to be stored in alternative slots [MCR]

2018-08-06 Thread daniel
daniel claimed this task. TASK DETAILhttps://phabricator.wikimedia.org/T194729EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: danielCc: Jdforrester-WMF, Addshore, Cparle, daniel, Gaboe420, Versusxo, Majesticalreaper22, Giuliamocci, Adrian1985, Cpaulf30, Lahi

[Wikidata-bugs] [Maniphest] [Closed] T201194: Beta cluster: Call to undefined method WikitextContent::getEntity() in /srv/mediawiki/php-master/extensions/Wikibase/lib/includes/Store/Sql/WikiPageEntity

2018-08-06 Thread daniel
daniel closed this task as "Resolved".daniel claimed this task.daniel added a comment. Closing, since there is nothing to be done here beyond the revert. An alternative patch for T194729 is now up at https://gerrit.wikimedia.org/r/c/mediawiki/extensions/Wikibase/+/450541.TASK D

[Wikidata-bugs] [Maniphest] [Updated] T194729: Allow Wikibase Entities to be stored in alternative slots [MCR]

2018-08-05 Thread daniel
daniel added a comment. The change to WikiPageEntityRevisionLookup has been reverted, see T201194#4479699.TASK DETAILhttps://phabricator.wikimedia.org/T194729EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Addshore, danielCc: Jdforrester-WMF, Addshore, Cparle

[Wikidata-bugs] [Maniphest] [Commented On] T201194: Beta cluster: Call to undefined method WikitextContent::getEntity() in /srv/mediawiki/php-master/extensions/Wikibase/lib/includes/Store/Sql/WikiPage

2018-08-05 Thread daniel
daniel added a comment. Ok, confirmed locally: this is caused by the new WikiPageEntityRevisionLookup code. Analysis: RevisionStore loads the correct revision content from the correct database (wikidatawiki), but since rev_content_model is NULL in the database, it falls back to using the default

<    1   2   3   4   5   6   7   8   9   10   >