gerritbot added a comment.
Change 455630 merged by jenkins-bot:
[mediawiki/core@master] Fix undeletion write-both/read-old mode.
https://gerrit.wikimedia.org/r/455630TASK DETAILhttps://phabricator.wikimedia.org/T183488EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferenc
jcrespo added a comment.
Thank you anomie for running those queries, which I had suggested or even offered to do to check things look consistent.TASK DETAILhttps://phabricator.wikimedia.org/T183488EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: tstarling, jcre
gerritbot added a comment.
Change 455630 had a related patch set uploaded (by Daniel Kinzler; owner: Daniel Kinzler):
[mediawiki/core@master] Fix undeletion write-both/read-old mode.
https://gerrit.wikimedia.org/r/455630TASK DETAILhttps://phabricator.wikimedia.org/T183488EMAIL PREFERENCEShttps://p
Anomie added a comment.
We should get it fixed before 1.32 is released (or did MCR undeletion make it into 1.31? I forget). For Wikimedia sites I can just make sure everything got migrated as I'm already doing.TASK DETAILhttps://phabricator.wikimedia.org/T183488EMAIL PREFERENCEShttps://phabricator.
daniel added a comment.
It looks like the "fake" SlotRecord created by RevisionStore::newRevisionFromArchiveRow() passes the $slot->hasRevision() check in RevisionStore::insertRevisionInternal() so it doesn't actually insert it.
That's as designed (well, partially): when undeleting, it should not
Anomie added a comment.
Also it looks like undeletions can somehow manage to escape the script, if the undeletion happens between when the rev_id is reached in revision and when the ar_rev_id is reached in archive. Found a few on commonswiki so far, and I won't be too surprised if there are more on
Stashbot added a comment.
Mentioned in SAL (#wikimedia-operations) [2018-08-27T16:31:53Z] Running populateContentTables.php on advisorswiki for T183488 and T202904TASK DETAILhttps://phabricator.wikimedia.org/T183488EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/
daniel added a comment.
In T183488#4535083, @Anomie wrote:
I'm running queries on all wikis to see if there are any revision or archive rows lacking a slots row. That found T202904 so far.
Wow, nice find!TASK DETAILhttps://phabricator.wikimedia.org/T183488EMAIL PREFERENCEShttps://phabricator.wik
daniel added a comment.
@Anomie What do you think of re-running the script on a few more wikis, as a spot check to verify that it finds nothing to do? If otoh the script finds stuff to migrate, something is wrong...TASK DETAILhttps://phabricator.wikimedia.org/T183488EMAIL PREFERENCEShttps://phabric
Stashbot added a comment.
Mentioned in SAL (#wikimedia-operations) [2018-08-27T15:49:02Z] Re-running populateContentTables.php on aawikibooks, gotwikibooks, kswikiquote, lvwikibooks, nostalgiawiki, wawikibooks and wikimania2005wiki for T183488TASK DETAILhttps://phabricator.wikimedia.org/T183488EMA
Stashbot added a comment.
Mentioned in SAL (#wikimedia-operations) [2018-08-22T19:01:36Z] re-running populateContentTables.php on cawiki for T183488TASK DETAILhttps://phabricator.wikimedia.org/T183488EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: tstarling,
tstarling added a comment.
enwiki is complete now, so only the T202032 wikis remain.TASK DETAILhttps://phabricator.wikimedia.org/T183488EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: tstarlingCc: jcrespo, greg, tstarling, Stashbot, Abit, gerritbot, Jdforreste
Anomie added a comment.
In T183488#4504507, @tstarling wrote:
So how do we end up trying to insert a row for revision 3003 twice?
max(rev_id) on this wiki is 3002, so the ar_rev_id certainly came from populateArchiveRevId.php, which allocated ar_rev_id values by inserting dummy rows into revision
daniel added a comment.
Duplicates are prevented by the unique index on slot_revision_id/slot_role_id. Orphan slot rows are possible, just as orphan revision rows have been possible before.
We can run a query to detect those on some wikis as a spot check. If we find nothing, I don't think full val
jcrespo added a comment.
So the script will make sure to not miss any content, but doesn't assure it introduces extra one, or duplicates some, that is why I mention the usage of extra checks. Takes very little time (1 long running query per check) and validates the whole process. The same thing tha
daniel added a comment.
@jcrespo Ooops, I updated my comment after you already answered, sorry.TASK DETAILhttps://phabricator.wikimedia.org/T183488EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: tstarling, danielCc: jcrespo, greg, tstarling, Stashbot, Abit, g
jcrespo added a comment.
In T183488#4506824, @daniel wrote:
We did pretty extensive testing of this on the test copies (db and db1112). We could have a script that does basic sanity checks, but I'm not sure what to test, beyond the trivial. All I can think of is:
all revision rows have an as
daniel added a comment.
it would be nice to run a read-only process to check the consistency of the migration
We did pretty extensive testing of this on the test copies (db and db1112). We could have a script that does basic sanity checks, but I'm not sure what to test, beyond the trivial. All
jcrespo added a comment.
I mentioned this to Tim, and he said to better mention it to Brad (and Daniel?), that it would be nice to run a read-only process to check the consistency of the migration- it should take much less time than the writes and would avoid headaches given the issues with the exi
tstarling added a comment.
So how do we end up trying to insert a row for revision 3003 twice?
max(rev_id) on this wiki is 3002, so the ar_rev_id certainly came from populateArchiveRevId.php, which allocated ar_rev_id values by inserting dummy rows into revision and deleting them in the same trans
Stashbot added a comment.
Mentioned in SAL (#wikimedia-operations) [2018-08-14T11:40:41Z] restarted populateContentTables.php on s2 (T183488)TASK DETAILhttps://phabricator.wikimedia.org/T183488EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: tstarling, Stashbo
tstarling added a comment.
You can see the full logs at mwmaint1001:/var/log/mediawiki/populateContentTables/ . On both aawikibooks and gotwikibooks, the error occurred on the second batch of the archive table, starting at ar_rev_id 2001. In both cases it was also the last batch, with the maximum a
daniel added a comment.
aawikibooks failed with "Error: 1062 Duplicate entry '3003-1' for key 'PRIMARY' (10.64.0.205)"
It's annoying that this error message doesn't say which *table* this was on. My guess is that it'S the slots table, which has primary key ( slot_revision_id, slot_role_id ). slot_
tstarling added a comment.
Log summary:
aawikibooks failed with "Error: 1062 Duplicate entry '3003-1' for key 'PRIMARY' (10.64.0.205)"
cawiki failed with "Replication wait failed: Server shutdown in progress" and will need to be restarted.
gotwikibooks also failed with a duplicate key error
s5 (d
tstarling added a comment.
In T183488#4499288, @jcrespo wrote:
@tstarling Please stop writes going to *s2* unless they have already finished
Done. s2 was up to itwiki rev_id 3012040.
enwiki should take about another 6 days at the current rate.TASK DETAILhttps://phabricator.wikimedia.org/T183488
jcrespo added a comment.
Let's keep an eye on performance metrics, specially https://grafana.wikimedia.org/dashboard/db/save-timing Although with the estimations given T183488#4480579, it may be desirable to make things faster rather and short than extended for a long period of time.
Meanwhile, I
Stashbot added a comment.
Mentioned in SAL (#wikimedia-operations) [2018-08-13T01:45:10Z] on mwmaint1001 running populateContentTables.php on all wikis using ~tstarling/pct-list T183488TASK DETAILhttps://phabricator.wikimedia.org/T183488EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/p
Stashbot added a comment.
Mentioned in SAL (#wikimedia-operations) [2018-08-07T03:46:00Z] on mwmaint1001 running populateContentTables.php concurrently on wikidatawiki and commonswiki (T183488)TASK DETAILhttps://phabricator.wikimedia.org/T183488EMAIL PREFERENCEShttps://phabricator.wikimedia.org/se
greg added a comment.
ack, thanks!TASK DETAILhttps://phabricator.wikimedia.org/T183488EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Anomie, gregCc: greg, tstarling, Stashbot, Abit, gerritbot, Jdforrester-WMF, Anomie, Addshore, aude, Aklapper, daniel, Gaboe42
tstarling added a comment.
@greg The WN31 things are done now, only 1081 seconds for mediawikiwiki and 9252 seconds for metawiki. For metawiki the rate was about the same as anomie got for testwiki, 2000 rows per second for the revision table and 600 rows per second for the archive table. At that r
Stashbot added a comment.
Mentioned in SAL (#wikimedia-operations) [2018-08-06T04:29:45Z] on mwmaint1001 running populateContentTables.php on metawiki T183488TASK DETAILhttps://phabricator.wikimedia.org/T183488EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: A
Stashbot added a comment.
Mentioned in SAL (#wikimedia-operations) [2018-08-06T04:09:55Z] on mwmaint1001 running populateContentTables.php on mediawikiwiki T183488TASK DETAILhttps://phabricator.wikimedia.org/T183488EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/
greg added a comment.
In T183488#4471137, @tstarling wrote:
Daniel proposed the following schedule:
Today: test2wiki, testwikidatawiki
This week (WN31): mediawikiwiki, metawiki
Next week (WN32): wikidatawiki, commonswiki
Week after (WN33): everything
Given it's Friday I don't think the rest o
tstarling added a comment.
test2wiki and testwikidatawiki are complete.TASK DETAILhttps://phabricator.wikimedia.org/T183488EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Anomie, tstarlingCc: tstarling, Stashbot, Abit, gerritbot, Jdforrester-WMF, Anomie, Addsh
Stashbot added a comment.
Mentioned in SAL (#wikimedia-operations) [2018-08-02T05:30:43Z] on mwmaint1001 running populateContentTables.php as described in T183488TASK DETAILhttps://phabricator.wikimedia.org/T183488EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/T
tstarling added a comment.
Daniel proposed the following schedule:
Today: test2wiki, testwikidatawiki
This week (WN31): mediawikiwiki, metawiki
Next week (WN32): wikidatawiki, commonswiki
Week after (WN33): everything
I'm going to start on this now.TASK DETAILhttps://phabricator.wikimedia.org/T
Stashbot added a comment.
Mentioned in SAL (#wikimedia-operations) [2018-07-25T15:31:45Z] running populateContentTables.php on testwiki for T183488TASK DETAILhttps://phabricator.wikimedia.org/T183488EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: Anomie, Stas
gerritbot added a comment.
Change 406595 merged by jenkins-bot:
[mediawiki/core@master] [MCR] RevisionStore, enable insertions for new schema
https://gerrit.wikimedia.org/r/406595TASK DETAILhttps://phabricator.wikimedia.org/T183488EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/e
Abit added a comment.
In T183488#4027864, @Addshore wrote:
In T183488#4027858, @Abit wrote:
@Addshore, the MCR pert says you and Katie are working on this, and I see you wrote a related gerrit patch. Should this be assigned to you?
So I am writing some of the underlying code, but that is proba
Abit added a comment.
@Addshore, the MCR pert says you and Katie are working on this, and I see you wrote a related gerrit patch. Should this be assigned to you?TASK DETAILhttps://phabricator.wikimedia.org/T183488EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To
gerritbot added a comment.
Change 406595 had a related patch set uploaded (by Addshore; owner: Addshore):
[mediawiki/core@master] [MCR] RevisionStore, enable insertions for new schema
https://gerrit.wikimedia.org/r/406595TASK DETAILhttps://phabricator.wikimedia.org/T183488EMAIL PREFERENCEShttps://
41 matches
Mail list logo
