Hi Ali, When writing Stellar statements, it is convenient to test them out in the REPL, which can be invoked via some variant of the commands in https://github.com/apache/metron/blob/master/metron-platform/metron-common/src/main/scripts/stellar , depending on the particular environment you’re working in . For instance, from the root of a git clone (that has already been successfully compiled), you can run:
java -cp metron-platform/metron-enrichment/target/metron-enrichment-0.3.1-uber.jar:metron-platform/metron-data-management/target/metron-data-management-0.3.1.jar:metron-platform/metron-common/target/metron-common-0.3.1.jar:metron-platform/metron-management/target/metron-management-0.3.1.jar:metron-platform/metron-parsers/target/metron-parsers-0.3.1-uber.jar org.apache.metron.common.stellar.shell.StellarShell As you probably infer from the above, any jar that has @Stellar annotated classes has to be included in the cp, or those stellar operators won’t be available. The main jar is metron-common, which will give you the REPL and the “base” operator set, but many interesting ops are in those other jars. (If I missed one, just add it to the cp.) Once in the REPL you can type `%functions` to get a list of all available stellar operators. Basically, any class annotated with @Stellar will automatically be loaded, both in the REPL and in the installed runtime environment. Cheers, --Matt From: Otto Fowler <[email protected]> Reply-To: "[email protected]" <[email protected]> Date: Thursday, May 25, 2017 at 5:23 AM To: Nick Allen <[email protected]>, "[email protected]" <[email protected]> Subject: Re: Metron HBase conditional enrichment I think most of those restricted functions are in the metron-managment section. On May 25, 2017 at 07:27:24, Nick Allen ([email protected]<mailto:[email protected]>) wrote: > everywhere I can use Stellar DSL, all of the functions have been implemented > and ready to use? Generally, yes, you are right. I vaguely remember a couple instances of functions that are useful in the REPL only, but I cannot remember what those are right now. Hopefully we have those doc'd appropriately. On Wed, May 24, 2017 at 10:38 PM, Ali Nazemian <[email protected]<mailto:[email protected]>> wrote: Hi Nick, I was not sure about the implementation, so does it generally mean everywhere I can use Stellar DSL, all of the functions have been implemented and ready to use? Cheers, Ali On Thu, May 25, 2017 at 2:52 AM, Nick Allen <[email protected]<mailto:[email protected]>> wrote: > can I do the concatenation on the fly at the enrichment level, so I don't > need to store this temp field in Elasticsearch/HDFS. Sure, absolutely. > Moreover, I need to have a conditional enrichment to say if you couldn't find > any match for "tenant_name+device_type+device_name" lookup for > "tenant_name+device_type+default_device". Yes, you can. You've got if/else, JOIN, IS_EMPTY, and others that should make implementing this logic pretty easy. On Tue, May 23, 2017 at 10:34 PM, Ali Nazemian <[email protected]<mailto:[email protected]>> wrote: Hi, I was wondering how I can manage Stellar syntax to be aligned with the following structure for the HBase enrichment: HBase_row_key: tenant_name+device_type+device_name At the high-level, I need to create a separate field via a post-parse Stellar function to be a concatenation of tenan_name, device_type and device_name. Let's call this field "key". Basically, I need to do the enrichment on the "key" which would be corresponding to the HBase row key. My first question is can I do the concatenation on the fly at the enrichment level, so I don't need to store this temp field in Elasticsearch/HDFS. Moreover, I need to have a conditional enrichment to say if you couldn't find any match for "tenant_name+device_type+device_name" lookup for "tenant_name+device_type+default_device". The second question would be how can I manage conditional enrichment like this one. I would be really grateful if you can provide some example. Regards, Ali -- A.Nazemian
