Hello, The OAK-2066 contains a number of patches, which finally will lead to use batch insert/update operations available in RDB and Mongo. It’ll increase the performance of applying a commit, especially when we have many small updates of different documents.
There are some documents that shouldn’t be included in the batch update, because they are changing too often (like root). Otherwise, they’ll cause a conflict and we need to send another bulk update, containing only failing documents, etc. (detailed description can be found in OAK-3748). It would be good to find such documents, extract them from the bulk operation and update them sequentially, one after another. I prepared OAK-3748, which uses following way to find the hotspots: if the document was included in at least 20 bulk operations during the last 1h and it conflicted in more than 50% cases, it should be extracted from the future bulk updates. The first two constraints makes it self refreshing - after a while the number of bulk operations in which the “blocked" document was included during the last hour will be less than 20 (all constants are configurable). I’d appreciate a feedback, both on the “algorithm” and on the implementation in OAK-3748. Best regards, Tomek -- Tomek Rękawek | Adobe Research | www.adobe.com [email protected]
