nickva commented on issue #5879: URL: https://github.com/apache/couchdb/issues/5879#issuecomment-3863898330
Thank you @oisheeaa for your patience and for the additional info. I think what's causing the increased memory usage is the rather high `max_dbs_open = 60000` value. That's entirely too high for an 8GB instance. Even on 128GB instances we keep it at our default of 5000 and only on some clusters raise to 15000 or 20000. In 3.4.2 there was an additional `close_on_idle` mechanism to periodically close idle db handles (see https://github.com/apache/couchdb/blob/3.4.2/src/couch/src/couch_db_updater.erl#L235 for details), however that mechanism had race conditions and a few other issues with it. The way the db handles cache works is once the handle is opened, it will stay opened until used again. If at some point there is not more room in the cache, the oldest unused handle is closed and replaced with the new one. So size it accordingly to how much memory you can allocate to it. For an 8GB maybe start with 1000-1500 and see if you get any `all_dbs_active` errors. (Try on a staging cluster first if you can generate similar load as the production cluster). There is a balance to how much memory should be used for the db handle cache vs having more for the page cache. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
