Great. Thanks, Casey. Neither Metron passes list attribute, nor we have a clear understanding of the exact enrichment, so we need to wait a little more...
On Fri, Apr 7, 2017 at 2:16 PM, Casey Stella <ceste...@gmail.com> wrote: > METRON-831, PR @ https://github.com/apache/incubator-metron/pull/517 > > Just so we're clear, let's assume the following: > > - Enrichment table called 'enrichments' > - Enrichment CF called 't' > - A message field called user_ids that is a list of user IDs > - The enrichment type for this HBase enrichment is 'et' > - The indicator in the HBase enrichment is a user ID > - You want a certain field out of the HBase enrichment data per user > ID. Let's call that field 'login_time' > > In order do that with METRON-831, you'd do the following to get the login > time fields for the list of users. For the sake of simplicity, I'll break > it into temporary variables: > > - enriched_users := MAP( user_ids, &( user_id : ENRICHMENT_GET('et', > user_id, 'enrichments', 't') ) ) > - login_times := MAP(enriched_users, &( enrichment : > MAP_GET(enrichment, 'login_time') ) ) > - MAP_GET here retrieves the value associated with the key > 'login_time', which is the name. > > Since you probably don't want intermediate values out there, you might > want to smash that into one big statement (we need a way to remove > temporary variables in stellar enrichments, btw): > > - MAP(MAP( user_ids, &( user_id : ENRICHMENT_GET('et', user_id, > 'enrichments', 't') ) ), &( enrichment : MAP_GET(enrichment, 'login_time') > ) ) > > On a side-note, it might be nice to have an optional arg to ENRICHMENT_GET > that lets you specify just the fields to return. That would simplify the > call to: > > - MAP( user_ids, &( user_id : ENRICHMENT_GET('et', user_id, > 'enrichments', 't', ['login_time']) ) ) > > > On Thu, Apr 6, 2017 at 8:10 PM, Casey Stella <ceste...@gmail.com> wrote: > >> There'll be a JIRA and a PR tonight ;) It sprung from the keyboard. I've >> been waiting for a good reason for some time. heh >> >> On Thu, Apr 6, 2017 at 8:08 PM, Otto Fowler <ottobackwa...@gmail.com> >> wrote: >> >>> Is there a Jira for the MAP Casey? >>> >>> >>> On April 6, 2017 at 14:07:15, Casey Stella (ceste...@gmail.com) wrote: >>> >>> Ok, so yeah, you've hit upon a limitation currently. Right now, via >>> Stellar you can use ENRICHMENT_GET which takes the following parameters: >>> >>> - enrichment_type - The enrichment type >>> - indicator - The string indicator to look up >>> - hbase_table - The HBase Table to use >>> - column_family - The Column Family to use >>> >>> Right now we only accept a string for the indicator (which likely would >>> be your user_id). You'd probably like to call ENRICHMENT_GET for each id >>> in the user_id variable. We can't quite do that yet. There has been some >>> talk about a MAP function created where you can apply a stellar function >>> across a list of values. i.e. MAP( user_id, @ENRICHMENT_GET('et', $, >>> 'enrichments', 't')) which would return a list containing the output of >>> ENRICHMENT_GET for each call. >>> >>> There is another, more immediate change that could be made for this >>> specific case. We could enable ENRICHMENT_GET to take a list of indicators >>> as the second argument. >>> >>> Sorry, that doesn't exactly solve your problem in the immediate-case, >>> but it provides some context for future fixes. ;) I don't suppose you know >>> the length of the list beforehand, right? Even the maximum size? >>> >>> Casey >>> >>> >>> On Sun, Apr 2, 2017 at 10:26 AM, Ali Nazemian <alinazem...@gmail.com> >>> wrote: >>> >>>> Hi all, >>>> >>>> >>>> I was wondering how I can achieve the following use case in the current >>>> version of Metron? >>>> >>>> >>>> >>>> I want to have attributes in the Metron JSON object that are an array. >>>> For example, if a threat is impacting multiple users, they are all >>>> contained in an attribute (e.g. user_id:[id1, id2, id3]). Now if I want >>>> to enrich the event with data that requires the user_id as a key in >>>> enrichment stored in HBASE, how would I do this? >>>> >>>> >>>> Cheers, >>>> Ali >>>> >>> >>> >> > -- A.Nazemian
