Hi Marc-Aurele,
Thanks for starting a discussion thread. I think the GenericDaoBase uses ehcache to build an internal cache and the rate-limiting plugin uses ehcache, however I'm not sure to what extent they are used and the impact of removing them on a production management server. Let me evaluate this soon and get back on this thread (I'll be away next week on holiday). Thank you for your contribution, it has been a pleasure working with you in the community. I wish you good luck in your next endeavor. - Rohit <https://cloudstack.apache.org> ________________________________ From: Marc-Aurèle Brothier <m...@brothier.org> Sent: Sunday, October 28, 2018 1:39:53 AM To: dev@cloudstack.apache.org Subject: Caching / Ehcache Hi everyone, (Again as the email formatting has been removed and was hard to read - I hope it will be better this time). While trying to lower the DB load for CloudStack I did some long testing and here are my outcomes for the current cache mechanism in CloudStack. I would be interested to hear from people who try to customize the ehcache configuration in CS. A PR ( https://github.com/apache/cloudstack/pull/2913) is also open to desactivate (before deleting) ehcache in CS, read below to understand why. # Problems The code in CS does not seem to fit any caching mechanism especially due to the homemade DAO code. The main 3 flaws are the following: ## Entities are not expected to be shared There is quite a lot of code with method calls passing entity IDs value as long, which does some object fetching. Without caching, this behavior will create distinct objects each time an entity with the same ID is fetched. With the cache enabled, the same object will be shared among those methods. It has been seen that it does generate some side effects where code still expected unchanged entity attributes after calling different methods thus generating exception/bugs. ## DAO update operations are using search queries Some part of the code are updating entities based on a search query, therefore the whole cache must be invalidated (see GenericDaoBase: public int update(UpdateBuilder ub, final SearchCriteria<?> sc, Integer rows);). ## Entities based on views joining multiple tables There are quite a lot of entities based on SQL views joining multiple entities in a same object. Enabling caching on those would require a mechanism to link and cross-remove related objects whenever one of the sub-entity is changed. # Final word Based on the previously discussed points, the best approach IMHO would be to move out of the custom DAO framework in CS and use a well known one. It will handle caching well and the joins made by the views in the code. It's not an easy change, but it will fix along a lot of issues and add a proven / robust framework to an important part of the code. The work to change the DAO layer is a huge task, I don't know how / who will perform it. What are the proposals for a new DAO framework ? FYI I will stop working for Exoscale at the end of the month, so I won't be able to tackle such challenge as I won't be working with CS anymore. I'll try my best to continue looking at the project to give my insights and share the experienced I have with CS. Marc-Aurèle rohit.ya...@shapeblue.com www.shapeblue.com Amadeus House, Floral Street, London WC2E 9DPUK @shapeblue
