| daniel created this task. daniel added projects: Wikidata, DBA, Performance, netops. Herald added a project: Operations. |
TASK DESCRIPTION
Status Quo:
- we have several (about 5 to 10) instances of dispatchChanges.php running as cron jobs concurrently.
- dispatchChanges.php picks the most lagged client by looking at wbchanges_dispatch, grabs a lock representing that client, and pushes updates to it. On success, wbchanges_dispatch is updated and the lock released.
- Locking is done using the Database::lock method, which translates to MySQLs GET_LOCK. The lock is bound to the DB connedction.
- Each run of dispatchChanges.php may make several dispatch passes, picking and locking a client wiki each time.
The Problem:
- Each dispatchChanges.php instance (about 10) holds a connection to the master DB for several minutes. (How much of a problem is this, really?)
- We cannot close the connection and re-connect, since we would lose the lock.
Ideas:
- Rely on information written to wbchanges_dispatch. But how to prevent starvation due to stale locks, left by a failed dispatch pass or dead process? High timeout values cause starvation, low timeout values cause false negatives.
- Use some other locking mechanism. We have the LockManager base class with several implementations, including Memc, Redis, file system, and MySQL.
- Put wbchanges on a separate DB system. But running a separate server for a singe DB table with 1000 rows and 10 concurrent connections seems overkill.
Caveat:
Switching from one locking mechanism to the other requires careful coordination. Execution of dispatchChanges.php must be halted and then re-started with the new configuration, without causing more than a few minutes lag for client wikis.
TASK DETAIL
EMAIL PREFERENCES
To: daniel
Cc: jcrespo, Aklapper, Jonas, Lydia_Pintscher, hoo, daniel, Marostegui, Vali.matei, Minhnv-2809, Zppix, D3r1ck01, Izno, Luke081515, Cmjohnson, Wikidata-bugs, aude, GWicke, mark, faidon, Mbch331, Jay8g, Krenair, akosiaris, fgiunchedi
Cc: jcrespo, Aklapper, Jonas, Lydia_Pintscher, hoo, daniel, Marostegui, Vali.matei, Minhnv-2809, Zppix, D3r1ck01, Izno, Luke081515, Cmjohnson, Wikidata-bugs, aude, GWicke, mark, faidon, Mbch331, Jay8g, Krenair, akosiaris, fgiunchedi
_______________________________________________ Wikidata-bugs mailing list [email protected] https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
