gerritbot added a comment.
Change 233743 merged by jenkins-bot:
EntityUsageTable: Wrap batches in transactions
https://gerrit.wikimedia.org/r/233743
TASK DETAIL
https://phabricator.wikimedia.org/T107319
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences/
daniel added a comment.
Patches merged. Let's keep this open for a while and observe whether the
problem is indeed gone.
TASK DETAIL
https://phabricator.wikimedia.org/T107319
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: daniel
Cc: gerritbot,
gerritbot added a comment.
Change 233743 had a related patch set uploaded (by JanZerebecki):
EntityUsageTable: Wrap batches in transactions
https://gerrit.wikimedia.org/r/233743
TASK DETAIL
https://phabricator.wikimedia.org/T107319
EMAIL PREFERENCES
daniel added a comment.
@jcrespo: As far as I know, the DBO_TRX flag is set on the cluster, causing
MediaWiki to automatically start a transaction on the first
insert/update/delete. Such automatic transactions are committed when an
explicit transaction is started, or the request terminates.
I
gerritbot added a comment.
Change 233724 had a related patch set uploaded (by Daniel Kinzler):
EntityUsageTable: Wrap insert batches in transactions
https://gerrit.wikimedia.org/r/233724
TASK DETAIL
https://phabricator.wikimedia.org/T107319
EMAIL PREFERENCES
gerritbot added a comment.
Change 233724 merged by jenkins-bot:
EntityUsageTable: Wrap batches in transactions
https://gerrit.wikimedia.org/r/233724
TASK DETAIL
https://phabricator.wikimedia.org/T107319
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences/
JanZerebecki added a comment.
We are now doing the update in batches of 100. But I think Mediawiki wraps all
of them in an implicit transaction. To fix this bug, do we need to split that
up into individual transactions, one for each update?
TASK DETAIL
jcrespo added a comment.
@JanZerebecki not sure if you are asking me: Yes, IF that is the case (I cannot
say)- batching will actually be worse than no batching at all- as it will be
slower than a single query and more open to clashing with other queries without
any of the advantages (reduced
gerritbot added a comment.
Change 232917 merged by jenkins-bot:
Use eu_row_id when touching usage entries.
https://gerrit.wikimedia.org/r/232917
TASK DETAIL
https://phabricator.wikimedia.org/T107319
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To:
jcrespo added a comment.
Could this job have caused:
/* JobRunner::commitMasterChanges 127.0.0.1 */
GET_LOCK('jobrunner-serial-commit', 30) AS lockstatus /*
31bdf25cc42f95f4f4a9b493cafe0299 db1024 plwiki 11s */ /* localhost */
we had another large lag spike around the same time than other
jcrespo added a comment.
I am almost sure that the first part of my previous comment was not caused by
this job, but a very large LinksUpdate::incrTableUpdate, sorry about that.
TASK DETAIL
https://phabricator.wikimedia.org/T107319
EMAIL PREFERENCES
gerritbot added a subscriber: gerritbot.
gerritbot added a comment.
Change 232917 had a related patch set uploaded (by Daniel Kinzler):
Use eu_row_id when touching usage entries.
https://gerrit.wikimedia.org/r/232917
TASK DETAIL
https://phabricator.wikimedia.org/T107319
EMAIL PREFERENCES
daniel added a comment.
Proposal for a solution that does not need knowledge about how the parser cache
is split:
Split the update into two steps:
1. a SELECT query using the WHERE clause we currently see in the UPDATE query,
collecting all the matching eu_row_ids.
2. an UPDATE query using
aude added a comment.
I looked into this some and some ways I think we can improve:
only do selective updates (for specific aspects) for wikis with the
allowDataAccessInUserLanguage setting = true (e.g. multilingual wikis where we
split parser cache by language, such as commons). As far as I
aude added a comment.
another idea...
before we execute the job, we could check the page touched timestamp. if the
page has been touched more recently than the entity usage timestamp (e.g. has
since been edited), then don't run the job.
there should be another job in the queue with a later
jcrespo added a comment.
And here it is actual data of an example spike:
Host UserSchema Client Source Thread Transaction Runtime
Stamp
db1038wikiusersimplewiki mw1001 - 5096510924
30A18B0D6 15s 2015-08-15 10:04:05
UPDATE /*
aude added a comment.
@jcrespo thanks for helping look into this.
TASK DETAIL
https://phabricator.wikimedia.org/T107319
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: aude
Cc: jcrespo, Aklapper, aude, Wikidata-bugs, Malyacko
jcrespo added a comment.
@aude If you are increasing the priority based on my comment, don't. Regular
databases can handle the extra load (for now), the main issue is in research
databases or labs, where they contain all dbs, or when there is maintenance and
we are in a degraded state. This
18 matches
Mail list logo
