[
https://issues.apache.org/jira/browse/CAMEL-21888?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17937558#comment-17937558
]
Nicolas Filotto commented on CAMEL-21888:
-----------------------------------------
[~ben.manes] Hi, thanks for sharing. It is always priceless to have your
feedback on the subject. I proposed a new PR to manage the second chance by
providing a unique ID to each value to avoid evicting the same key/value pair
that has been put multiple times in the cache, it allows me to avoid
compressing the changes too often which seems to be the root cause of the
problem in this case.
If this new PR doesn't allow fixing the problem, I will definitively drop the
queue of changes for something similar to your LRU algorithm based on a global
lock to manage the update of the two links, which I initially wanted to avoid
to keep a globally "lock-free" approach.
Regarding your algorithm Second Chance, it could have been a good trade-off but
it has some drawbacks as it could evict more recent key/value pairs in some
situations like the following, and it could be seen as a bug by end-users:
{code:java}
ProductionMap<String, String> map = new ProductionMap<>(16, 3,
EvictionPolicy.SECOND_CHANCE);
map.put("A", "1");
map.put("A", "2"); // Marked
map.put("B", "1");
map.put("B", "2"); // Marked
map.put("C", "1");
map.put("C", "2"); // Marked
map.put("D", "1"); // last entry is evicted as it is the first
non-marked entry
{code}
> High CPU usage at startup due to Deque.size() iteration in SimpleLRUCache
> -------------------------------------------------------------------------
>
> Key: CAMEL-21888
> URL: https://issues.apache.org/jira/browse/CAMEL-21888
> Project: Camel
> Issue Type: Bug
> Components: came-core
> Affects Versions: 4.7.0
> Reporter: Nadina Florea
> Assignee: Nicolas Filotto
> Priority: Major
>
> Hello,
> Starting with *Camel 4.7.0,* a change was introduced in *SimpleLRUCache,*
> where a *Deque* is now used to track *lastChanges.*
> Please see:
> [https://github.com/apache/camel/commit/a7e696927dea30795a49a4c0ac4a36ee700131ff]
> and CAMEL-20850
> This change causes *high CPU usage at startup* when the cache is populated,
> as *Deque.size() iterates over all cache entries.* This results in 100% CPU
> usage, and the *application failed to start.*
> This issue was observed only in production, after migrating to Camel 4.7.0,
> where the cache size is significantly large (1,200,000 entries at that time).
> Current stable version is Camel 4.6.0. The behavior was reproduced with Camel
> 4.10.0 as well.
>
> We are using {*}KafkaIdempotentRepository{*}, which relies on SimpleLRUCache.
> In our app, we expose a metric that contains the size of the cache at a
> certain point.
> {code:java}
> @Bean
> @ConditionalOnBean(KafkaIdempotentRepository.class)
> public MeterBinder cacheSize(final KafkaIdempotentRepository
> idempotentRepository) {
> return registry ->
> Gauge.builder("cacheSize",idempotentRepository::getCacheSize)
> .register(registry);
> }{code}
> Would appreciate any insights on this issue.
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
