today, while performing maintenance to the Eventlogging Master database, we
ended up in https://phabricator.wikimedia.org/T188991 (TL;DR: two hours of
data inserted to the slave database and not the master one). We are working
to find a feasible solution to avoid loosing data and getting out this
inconsistent state, so as precautionary measure the Eventlogging mysql
consumers have been stopped.
A couple of notes:
- The Eventlogging machinery is working as expected, except mysql insertion
- The HDFS data has not been affected by this issue.
Please check the task for more updates, or follow up with the Analytics
team on IRC (#wikimedia-analytics on freenode).
Thanks and sorry for the trouble!
Luca (on behalf of the Analytics team)
Analytics mailing list