Aleksey Plekhanov created IGNITE-14341: ------------------------------------------
Summary: Significant performance drop when entries expiring concurrently Key: IGNITE-14341 URL: https://issues.apache.org/jira/browse/IGNITE-14341 Project: Ignite Issue Type: Bug Reporter: Aleksey Plekhanov Currently, there is a significant performance drop when expired entries concurrently evicted by threads that perform some actions with cache (see attached reproducer): {noformat} Benchmark Mode Cnt Score Error Units JmhCacheExpireBenchmark.putWithExpire thrpt 3 100,132 ± 21,025 ops/ms JmhCacheExpireBenchmark.putWithoutExpire thrpt 3 2133,122 ± 559,694 ops/ms{noformat} Root cause: pending entries tree (offheap BPlusTree) is used to track expired entries, after each cache operation (and by timeout thread) there is an attempt to evict some amount of expired entries. these entries looked up from the start of the pending entries tree and there is a contention on the first leaf page of that tree. All threads waiting for the same page lock: {noformat} at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at org.apache.ignite.internal.util.OffheapReadWriteLock.waitAcquireWriteLock(OffheapReadWriteLock.java:503) at org.apache.ignite.internal.util.OffheapReadWriteLock.writeLock(OffheapReadWriteLock.java:244) at org.apache.ignite.internal.pagemem.impl.PageMemoryNoStoreImpl.writeLock(PageMemoryNoStoreImpl.java:528) at org.apache.ignite.internal.processors.cache.persistence.tree.util.PageHandler.writeLock(PageHandler.java:422) at org.apache.ignite.internal.processors.cache.persistence.tree.util.PageHandler.writePage(PageHandler.java:350) at org.apache.ignite.internal.processors.cache.persistence.DataStructure.write(DataStructure.java:325) at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.access$13200(BPlusTree.java:100) at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$Remove.doRemoveFromLeaf(BPlusTree.java:4588) at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$Remove.removeFromLeaf(BPlusTree.java:4567) at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$Remove.tryRemoveFromLeaf(BPlusTree.java:5196) at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$Remove.access$6800(BPlusTree.java:4209) at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.removeDown(BPlusTree.java:2189) at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.removeDown(BPlusTree.java:2165) at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.removeDown(BPlusTree.java:2165) at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.doRemove(BPlusTree.java:2076) at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.removex(BPlusTree.java:1905) at org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl.expireInternal(IgniteCacheOffheapManagerImpl.java:1426) at org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl.expire(IgniteCacheOffheapManagerImpl.java:1375) at org.apache.ignite.internal.processors.cache.GridCacheTtlManager.expire(GridCacheTtlManager.java:246) at org.apache.ignite.internal.processors.cache.GridCacheUtils.unwindEvicts(GridCacheUtils.java:882){noformat} -- This message was sent by Atlassian Jira (v8.3.4#803005)