Github user cestella commented on the issue:

    https://github.com/apache/metron/pull/940
  
    The current architecture is described ![Image of 
Yaktocat](https://github.com/apache/metron/raw/master/metron-platform/metron-enrichment/enrichment_arch.png)
    
    In short, for each message each splitter will
    * inspect the configs for the sensor 
    * For each sensor, extract the fields required for enrichment and send them 
to the appropriate enrichment bolt (e.g. hbase, geo, stellar)
      * If one enrichment enriches k fields, then k messages will be sent to 
the enrichment bolt
      * In the case of stellar, each stellar subgroup will be a separate message
      * the original message is sent directly to the join bolt
    * The enrichment bolts do the enrichment and send the additional fields and 
values to the original message
    * The join bolt will asynchronously collect the subresults and join them 
with the original message
      * The join bolt has a LRU cache to hold subresults until all results 
arrive
      * Tuning performance involves tuning this cache (max size and time until 
eviction)
      * Tuning this can be complex because it has to be large enough to handle 
spikes in traffic
    



---

Reply via email to