The stellar enrichments do cache results. It caches at the bolt level, so it will associate an input message with computed output.
On Mon, Jun 19, 2017 at 6:28 AM, Simon Elliston Ball < [email protected]> wrote: > Surely the caching should make this effectively an in memory lookup. Does > the stellar enrichment function not use the same clientside caching as the > Hbase bolt? > > Simon > > On 19 Jun 2017, at 06:21, Casey Stella <[email protected]> wrote: > > In order to do that, the easiest thing to do is to create a stellar > function to load and do in-memory lookups. > > On Sun, Jun 18, 2017 at 11:48 PM, Ali Nazemian <[email protected]> > wrote: > >> Hi all, >> >> We are using Metron HBase enrichment for a few use cases, but we have >> noticed the achievable throughput is not very great. I was wondering >> whether there is a way to load the external enrichment data in-memory and >> use it with normal Stellar enrichments. In our use cases, the number of >> rows in the external enrichments that we are dealing with is less than a >> 100k and it is a static list, so it is feasible to load them in-memory and >> use that for the enrichment. However, I am not sure how that would be >> achievable from the Metron capabilities. >> >> Regards, >> Ali >> > >
