Sean, thanks for the quick response:
> We have a binary log on the EL master that holds the last week of > INSERT statements. It can be dumped and grepped, eg looking at > 10-minute blocks around 2015-04-13 16:30: > Good to know! Zero(!) during 10min after 16:30 doesn't look good. This means the > database master did not see any INSERTs with 20150413163* timestamps > within the last week. Ok, makes more sense. > Can you describe how you know that events were > written normally? Simply a lack of errors from mysql consumer? > MySQL consumer log not only lacks errors, but has records of successful writes to the db at the time of the problems. Also, the processor logs indicate homogeneous throughput of valid events during all times. *However*, the consumer logs indicate the insert timestamp only, not the event timestamp (which goes to the table). So it could be that there's some data loss inside the consumer code (or in zmq?) that wouldn't stop the write flow, but would skip a segment of events. I'll look deeper into this. > Can you supply some specific records from the EL logs with timestamps > that should definitely be in the database, so we can scan the database > binlog for specific UUIDs or suchlike? > I'll try to get those. -- can you give me some idea of how long your "at other moments" delay > is? > I'll observe the db during the day and give you an estimate. Thanks!
_______________________________________________ Analytics mailing list [email protected] https://lists.wikimedia.org/mailman/listinfo/analytics
