Joerg Hoh created OAK-12290:
-------------------------------
Summary: Lock Contention in SegmentCache
Key: OAK-12290
URL: https://issues.apache.org/jira/browse/OAK-12290
Project: Jackrabbit Oak
Issue Type: Improvement
Components: segment-tar
Affects Versions: 2.2.0
Reporter: Joerg Hoh
I investigated a performance bottleneck on an environment running with Oak
2.2.0, and I have come across a series of instances where freshly introduced
CaffeineCache is involved.
I see a massive series of calls like this:
{noformat}
java.lang.Thread.State: RUNNABLE
at
com.github.benmanes.caffeine.cache.BoundedLocalCache.afterRead(BoundedLocalCache.java:1286)
at
com.github.benmanes.caffeine.cache.BoundedLocalCache.getIfPresent(BoundedLocalCache.java:2206)
at
com.github.benmanes.caffeine.cache.LocalManualCache.getIfPresent(LocalManualCache.java:56)
at
org.apache.jackrabbit.oak.cache.impl.caffeine.CaffeineCacheAdapter.getIfPresent(CaffeineCacheAdapter.java:43)
at
org.apache.jackrabbit.oak.segment.SegmentCache$NonEmptyCache.recordHit(SegmentCache.java:247)
at
org.apache.jackrabbit.oak.segment.file.AbstractFileStore$2$$Lambda/0x0000000800caa900.accept(Unknown
Source)
at
org.apache.jackrabbit.oak.segment.SegmentId.getSegment(SegmentId.java:164)
at org.apache.jackrabbit.oak.segment.Record.getSegment(Record.java:71)
at
org.apache.jackrabbit.oak.segment.MapRecord.getEntry(MapRecord.java:182)
at
org.apache.jackrabbit.oak.segment.SegmentNodeState.getChildNode(SegmentNodeState.java:450)
at
org.apache.jackrabbit.oak.plugins.memory.MemoryNodeBuilder.<init>(MemoryNodeBuilder.java:141)
{noformat}
and
{noformat}
java.lang.Thread.State: RUNNABLE
at com.github.benmanes.caffeine.cache.Node.inWindow(Node.java:162)
at
com.github.benmanes.caffeine.cache.BoundedLocalCache.onAccess(BoundedLocalCache.java:1782)
at
com.github.benmanes.caffeine.cache.BoundedLocalCache$$Lambda/0x0000000800a4c290.accept(Unknown
Source)
at
com.github.benmanes.caffeine.cache.BoundedBuffer$RingBuffer.drainTo(BoundedBuffer.java:101)
at
com.github.benmanes.caffeine.cache.StripedBuffer.drainTo(StripedBuffer.java:228)
at
com.github.benmanes.caffeine.cache.BoundedLocalCache.drainReadBuffer(BoundedLocalCache.java:1769)
at
com.github.benmanes.caffeine.cache.BoundedLocalCache.maintenance(BoundedLocalCache.java:1711)
at
com.github.benmanes.caffeine.cache.BoundedLocalCache.performCleanUp(BoundedLocalCache.java:1660)
at
com.github.benmanes.caffeine.cache.BoundedLocalCache$PerformCleanupTask.run(BoundedLocalCache.java:3886)
at
org.apache.jackrabbit.oak.cache.api.CacheBuilder$$Lambda/0x0000000800a115c8.execute(Unknown
Source)
at
com.github.benmanes.caffeine.cache.BoundedLocalCache.scheduleDrainBuffers(BoundedLocalCache.java:1633)
at
com.github.benmanes.caffeine.cache.BoundedLocalCache.afterRead(BoundedLocalCache.java:1288)
at
com.github.benmanes.caffeine.cache.BoundedLocalCache.getIfPresent(BoundedLocalCache.java:2206)
at
com.github.benmanes.caffeine.cache.LocalManualCache.getIfPresent(LocalManualCache.java:56)
at
org.apache.jackrabbit.oak.cache.impl.caffeine.CaffeineCacheAdapter.getIfPresent(CaffeineCacheAdapter.java:43)
at
org.apache.jackrabbit.oak.segment.SegmentCache$NonEmptyCache.recordHit(SegmentCache.java:247)
at
org.apache.jackrabbit.oak.segment.file.AbstractFileStore$2$$Lambda/0x0000000800caa900.accept(Unknown
Source)
at
org.apache.jackrabbit.oak.segment.SegmentId.getSegment(SegmentId.java:164)
at org.apache.jackrabbit.oak.segment.Record.getSegment(Record.java:71)
{noformat}
{noformat:title=SegmentCache.java}
@Override
public void recordHit(@NotNull SegmentId id) {
stats.hitCount.incrementAndGet();
if (id.isDataSegmentId() &&
FT_OAK_12214_PROPAGATE_L1_HITS_TO_L2_ENABLED.get()) {
cache.getIfPresent(id);
}
}
{noformat}
I think that the FT {{FT_OAK_12214_PROPAGATE_L1_HITS_TO_L2_ENABLED}} is
responsible for that, and it creates unnecessary latency (see OAK-12214); while
I think that the idea is good, there should be a different way to increment the
counter.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
