https://bugzilla.wikimedia.org/show_bug.cgi?id=69182
--- Comment #3 from Sean Pringle <[email protected]> --- Setting tokudb_empty_scan=disabled looks to have improved the speed of TokuDB /opening/ tables. Stack traces indicate the eventual lockup is due to TokuDB /closing/ tables slowly, which starts to occur en masse once the MariaDB table cache becomes full and one of these INFORMATION_SCHEMA queries is causing thousands of previously opened tables to be flushed to make room. This contention affects all threads because the table cache is global. We could potentially increase the table cache further -- presently 10000 -- however there are issues with that approach, such as hundreds of thousands more file handles (multiple per table) and poor scalability for misses [1]. We could go the other way and decrease the table cache which can sometimes, counter intuitively, improve performance. This would need to be attempted carefully. In the meantime: a) Have you tried the above loop with a specific delay between queries? There may be a sweet spot at which things can keep up. b) Try only querying table_schema on the proper <schema>.labsdb host names. Those are more likely to already have a stable table cache for their schemas, and/or reduce the overall impact if in combination with (a). c) A static version of INFORMATION_SCHEMA.TABLES would be better, say re-materialized daily as information_schema_p.tables. Will look at generating this upstream in the prod>labs replicas where table cache contention is much less. [1] http://www.mysqlperformanceblog.com/2009/11/16/table_cache-negative-scalability/ (though we should retest this with MariaDB 10) -- You are receiving this mail because: You are on the CC list for the bug. _______________________________________________ Wikibugs-l mailing list [email protected] https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
