[
https://issues.apache.org/jira/browse/METRON-1467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16389665#comment-16389665
]
ASF GitHub Bot commented on METRON-1467:
----------------------------------------
Github user cestella commented on a diff in the pull request:
https://github.com/apache/metron/pull/947#discussion_r172870992
--- Diff:
metron-platform/metron-enrichment/src/main/java/org/apache/metron/enrichment/bolt/JoinBolt.java
---
@@ -89,29 +91,25 @@ public void prepare(Map map, TopologyContext
topologyContext, OutputCollector ou
if (this.maxTimeRetain == null) {
throw new IllegalStateException("maxTimeRetain must be specified");
}
- loader = new CacheLoader<String, Map<String, Tuple>>() {
- @Override
- public Map<String, Tuple> load(String key) throws Exception {
- return new HashMap<>();
- }
- };
- cache = CacheBuilder.newBuilder().maximumSize(maxCacheSize)
- .expireAfterWrite(maxTimeRetain,
TimeUnit.MINUTES).removalListener(new JoinRemoveListener())
- .build(loader);
+ loader = s -> new HashMap<>();
+ cache = Caffeine.newBuilder().maximumSize(maxCacheSize)
+ .expireAfterWrite(maxTimeRetain, TimeUnit.MINUTES)
+ .removalListener(new JoinRemoveListener())
--- End diff --
So, I believe this was intentionally done before this PR (I migrated this
to the new caching strategy) and the idea is that if a removal is happening
from the join cache under specific circumstances, we want to know about it
because a message could be being dropped because the cache is being
overwhelmed. @merrimanr Can you chime in here on the rationale?
> Replace guava caches in places where the keyspace might be large
> ----------------------------------------------------------------
>
> Key: METRON-1467
> URL: https://issues.apache.org/jira/browse/METRON-1467
> Project: Metron
> Issue Type: Improvement
> Reporter: Casey Stella
> Priority: Major
>
> Based on the performance tuning exercise as part of METRON-1460, guava has
> difficulties with cache sizes over 10k. We, unfortunately, are quite
> demanding of guava in this regard so we should transition a few uses of guava
> to Caffeine:
> * Stellar processor cache
> * The JoinBolt cache
> * The Enrichment Bolt Cache
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
