[
https://issues.apache.org/jira/browse/METRON-1520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16434529#comment-16434529
]
ASF GitHub Bot commented on METRON-1520:
----------------------------------------
Github user cestella commented on the issue:
https://github.com/apache/metron/pull/990
This PR provides an expression cache for stellar when using the stellar
field transformations in parsers. Specifically, we found that without caching
we were substantially stymied when performance tuning enrichments. Thus, other
places where stellar is in the critical path might derive benefit from similar
caching in Enrichments.
A couple of points about the implementation:
* We use the same underlying caching strategy as in enrichments
(caffeine-based least frequently used cache).
* This caching is implemented in stellar core as an alternative to the
StellarProcessor. This way it is reusable elsewhere that stellar is used
eventually.
* This caching is slightly different than the caching in enrichments in
that:
* Enrichment caching is done regardless of adapter (stellar enrichments
uses the same cache as the hbase enrichments)
* Enrichment caching for stellar is at the group level (e.g. the key is
the group of stellar expressions and the input variables used in the group adn
the value is the set of results for that group given the input). This
implementation is at the individual expression level.
> Add caching for stellar field transformations
> ---------------------------------------------
>
> Key: METRON-1520
> URL: https://issues.apache.org/jira/browse/METRON-1520
> Project: Metron
> Issue Type: Improvement
> Reporter: Casey Stella
> Priority: Major
>
> Given how important caching is in the enrichment topology, we should have
> caching for stellar field transformations in the parsers as well.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
