Re: Define a function that can be used in Stellar

[Ali Nazemian]

What we need to have is a function to produce entity row_key based on
device format and tenant. We can put all the configurations in enrichment
config, but it makes it very complex because we will have different logic
for different customers.

I was thinking of storing that logic somewhere and create a function which
gets customer identifier, device name and a map of important event
properties. Although we need to modify logic, it won't be very frequently.
Putting that in the global config will not make any difference from the
operational complexity and it probably even makes it worse! However, an
extra latency per every event to load config would be a deal breaker for
us. Technically, it should be safe to use OBJECT_GET, but for the
configuration, it needs to parse it per event, right?



On Sat, Feb 3, 2018 at 1:49 AM, Otto Fowler  wrote:

> I think if we understand the use case, we may be able to think of a more
> general set of functionality for stellar to meet this and other cases.
>
> Will this configuration change?  Do you need to track that change without
> reloading?  How *much* is in the configuration?  Do we want
> people putting their own keys in the global and have to manage that
> upgrading ( I think it will get blown away )?
>
>
>
>
>
>
> On February 2, 2018 at 09:38:28, Casey Stella (ceste...@gmail.com) wrote:
>
> We use a guava cache to cache the data for 24 hours.  You can see how it's
> done here: https://github.com/apache/metron/blob/master/metron-
> platform/metron-enrichment/src/main/java/org/apache/
> metron/enrichment/stellar/ObjectGet.java
>
> We also do something like this in GEO_GET as well, but it's a bit more
> complex.
>
> On Fri, Feb 2, 2018 at 9:35 AM, Simon Elliston Ball <
> si...@simonellistonball.com> wrote:
>
>> I forgot we added OBJECT_GET. How does the caching work on that?
>>
>> Simn
>>
>>
>> On 2 Feb 2018, at 14:33, Nick Allen  wrote:
>>
>> There are many functions that use the global configuration.  For example,
>> GET_GEO in org.apache.metron.enrichment.stellar.GeoEnrichmentFunctions.
>> There might be a better example, but that is one is staring at me at the
>> moment.
>>
>> There is an OBJECT_GET function defined in 
>> org.apache.metron.enrichment.stellar.ObjectGet
>> that was purpose-built to retrieve files from HDFS.  If you wanted to
>> retrieve a configuration from HDFS that would be a good example (if you
>> can't just use that functions directly).
>>
>> On Fri, Feb 2, 2018 at 8:50 AM Ali Nazemian 
>> wrote:
>>
>>> Is there any Stellar function already been implemented in Metron that
>>> has a config file associated with it? I am trying to get an idea of how it
>>> works.
>>>
>>> On 3 Feb. 2018 00:44, "Simon Elliston Ball" 
>>> wrote:
>>>
 Depends how you write the function class, but most likely, yes. Hence
 global config option.

 Simon

 On 2 Feb 2018, at 13:42, Ali Nazemian  wrote:

 Does it mean every time the function gets called it will load the
 config, but if I use the global one it will only read it one time and it
 will be available in memory?

 On 2 Feb. 2018 21:53, "Simon Elliston Ball" <
 si...@simonellistonball.com> wrote:

> Shouldn’t be. The one this I would point out though is that you don’t
> necessarily know which supervisor you will be running from, so pulling 
> from
> HDFS would make sense. That said, the performance implications are 
> probably
> not great. A good option here would be to have the config available in the
> global config for example and refer to that, since most instances of
> stellar apply global config to their context.
>
> Simon
>
>
> On 2 Feb 2018, at 07:14, Ali Nazemian  wrote:
>
> Will be any problem if the Stellar function we want to implement need
> to load an external config file?
>
> Cheers,
> Ali
>
> On Thu, Jan 18, 2018 at 4:58 PM, Ali Nazemian 
> wrote:
>
>> Thanks, All.
>>
>> Yes, Nick. It is highly related to our use case and the way that we
>> are going to enrich events with assets and vulnerability properties. It 
>> is
>> not a general case at all.
>>
>> Cheers,
>> Ali
>>
>> On Thu, Jan 18, 2018 at 5:43 AM, Matt Foley  wrote:
>>
>>> Besides the example code Simon mentioned at
>>> https://github.com/apache/metron/tree/master/metron-stellar/
>>> stellar-3rd-party-example ,
>>> there is some documentation at http://metron.apache.org/curre
>>> nt-book/metron-stellar/stellar-common/3rdPartyStellar.html
>>>
>>>
>>>
>>> *From:* Nick Allen 
>>> *Reply-To:* "user@metron.apache.org" 
>>> *Date:* Wednesday, January 17, 2018 at 4:46 AM
>>> *To:* 

Re: Define a function that can be used in Stellar

[Otto Fowler]

I think if we understand the use case, we may be able to think of a more
general set of functionality for stellar to meet this and other cases.

Will this configuration change?  Do you need to track that change without
reloading?  How *much* is in the configuration?  Do we want
people putting their own keys in the global and have to manage that
upgrading ( I think it will get blown away )?






On February 2, 2018 at 09:38:28, Casey Stella (ceste...@gmail.com) wrote:

We use a guava cache to cache the data for 24 hours.  You can see how it's
done here:
https://github.com/apache/metron/blob/master/metron-platform/metron-enrichment/src/main/java/org/apache/metron/enrichment/stellar/ObjectGet.java

We also do something like this in GEO_GET as well, but it's a bit more
complex.

On Fri, Feb 2, 2018 at 9:35 AM, Simon Elliston Ball <
si...@simonellistonball.com> wrote:

> I forgot we added OBJECT_GET. How does the caching work on that?
>
> Simn
>
>
> On 2 Feb 2018, at 14:33, Nick Allen  wrote:
>
> There are many functions that use the global configuration.  For example,
> GET_GEO in org.apache.metron.enrichment.stellar.GeoEnrichmentFunctions.
> There might be a better example, but that is one is staring at me at the
> moment.
>
> There is an OBJECT_GET function defined in 
> org.apache.metron.enrichment.stellar.ObjectGet
> that was purpose-built to retrieve files from HDFS.  If you wanted to
> retrieve a configuration from HDFS that would be a good example (if you
> can't just use that functions directly).
>
> On Fri, Feb 2, 2018 at 8:50 AM Ali Nazemian  wrote:
>
>> Is there any Stellar function already been implemented in Metron that has
>> a config file associated with it? I am trying to get an idea of how it
>> works.
>>
>> On 3 Feb. 2018 00:44, "Simon Elliston Ball" 
>> wrote:
>>
>>> Depends how you write the function class, but most likely, yes. Hence
>>> global config option.
>>>
>>> Simon
>>>
>>> On 2 Feb 2018, at 13:42, Ali Nazemian  wrote:
>>>
>>> Does it mean every time the function gets called it will load the
>>> config, but if I use the global one it will only read it one time and it
>>> will be available in memory?
>>>
>>> On 2 Feb. 2018 21:53, "Simon Elliston Ball" 
>>> wrote:
>>>
 Shouldn’t be. The one this I would point out though is that you don’t
 necessarily know which supervisor you will be running from, so pulling from
 HDFS would make sense. That said, the performance implications are probably
 not great. A good option here would be to have the config available in the
 global config for example and refer to that, since most instances of
 stellar apply global config to their context.

 Simon


 On 2 Feb 2018, at 07:14, Ali Nazemian  wrote:

 Will be any problem if the Stellar function we want to implement need
 to load an external config file?

 Cheers,
 Ali

 On Thu, Jan 18, 2018 at 4:58 PM, Ali Nazemian 
 wrote:

> Thanks, All.
>
> Yes, Nick. It is highly related to our use case and the way that we
> are going to enrich events with assets and vulnerability properties. It is
> not a general case at all.
>
> Cheers,
> Ali
>
> On Thu, Jan 18, 2018 at 5:43 AM, Matt Foley  wrote:
>
>> Besides the example code Simon mentioned at
>> https://github.com/apache/metron/tree/master/metron-
>> stellar/stellar-3rd-party-example ,
>> there is some documentation at http://metron.apache.org/
>> current-book/metron-stellar/stellar-common/3rdPartyStellar.html
>>
>>
>>
>> *From:* Nick Allen 
>> *Reply-To:* "user@metron.apache.org" 
>> *Date:* Wednesday, January 17, 2018 at 4:46 AM
>> *To:* "user@metron.apache.org" 
>> *Subject:* Re: Define a function that can be used in Stellar
>>
>>
>>
>>
>>
>>
>>
>> If something we have already does not fit the bill, I would recommend
>> creating that function in Java.   Since you described it as "a bit 
>> complex"
>> and "the logic would be complicated" I don't see any value in defining
>> something like this in Stellar with named functions.
>>
>>
>>
>> Best
>>
>>
>>
>>
>>
>>
>>
>>
>>
>> On Wed, Jan 17, 2018 at 7:38 AM Simon Elliston Ball <
>> si...@simonellistonball.com> wrote:
>>
>> Have you looked at the recent TLSH functions in Stellar? We already
>> have that for similarity preserving hashes.
>>
>>
>>
>> Simon
>>
>>
>>
>>
>> On 17 Jan 2018, at 12:35, Ali Nazemian  wrote:
>>
>> It is a bit complex. We want to create a function that accepts a list
>> 

Re: Define a function that can be used in Stellar

[Casey Stella]

We use a guava cache to cache the data for 24 hours.  You can see how it's
done here:
https://github.com/apache/metron/blob/master/metron-platform/metron-enrichment/src/main/java/org/apache/metron/enrichment/stellar/ObjectGet.java

We also do something like this in GEO_GET as well, but it's a bit more
complex.

On Fri, Feb 2, 2018 at 9:35 AM, Simon Elliston Ball <
si...@simonellistonball.com> wrote:

> I forgot we added OBJECT_GET. How does the caching work on that?
>
> Simn
>
>
> On 2 Feb 2018, at 14:33, Nick Allen  wrote:
>
> There are many functions that use the global configuration.  For example,
> GET_GEO in org.apache.metron.enrichment.stellar.GeoEnrichmentFunctions.
> There might be a better example, but that is one is staring at me at the
> moment.
>
> There is an OBJECT_GET function defined in 
> org.apache.metron.enrichment.stellar.ObjectGet
> that was purpose-built to retrieve files from HDFS.  If you wanted to
> retrieve a configuration from HDFS that would be a good example (if you
> can't just use that functions directly).
>
> On Fri, Feb 2, 2018 at 8:50 AM Ali Nazemian  wrote:
>
>> Is there any Stellar function already been implemented in Metron that has
>> a config file associated with it? I am trying to get an idea of how it
>> works.
>>
>> On 3 Feb. 2018 00:44, "Simon Elliston Ball" 
>> wrote:
>>
>>> Depends how you write the function class, but most likely, yes. Hence
>>> global config option.
>>>
>>> Simon
>>>
>>> On 2 Feb 2018, at 13:42, Ali Nazemian  wrote:
>>>
>>> Does it mean every time the function gets called it will load the
>>> config, but if I use the global one it will only read it one time and it
>>> will be available in memory?
>>>
>>> On 2 Feb. 2018 21:53, "Simon Elliston Ball" 
>>> wrote:
>>>
 Shouldn’t be. The one this I would point out though is that you don’t
 necessarily know which supervisor you will be running from, so pulling from
 HDFS would make sense. That said, the performance implications are probably
 not great. A good option here would be to have the config available in the
 global config for example and refer to that, since most instances of
 stellar apply global config to their context.

 Simon


 On 2 Feb 2018, at 07:14, Ali Nazemian  wrote:

 Will be any problem if the Stellar function we want to implement need
 to load an external config file?

 Cheers,
 Ali

 On Thu, Jan 18, 2018 at 4:58 PM, Ali Nazemian 
 wrote:

> Thanks, All.
>
> Yes, Nick. It is highly related to our use case and the way that we
> are going to enrich events with assets and vulnerability properties. It is
> not a general case at all.
>
> Cheers,
> Ali
>
> On Thu, Jan 18, 2018 at 5:43 AM, Matt Foley  wrote:
>
>> Besides the example code Simon mentioned at
>> https://github.com/apache/metron/tree/master/metron-
>> stellar/stellar-3rd-party-example ,
>> there is some documentation at http://metron.apache.org/
>> current-book/metron-stellar/stellar-common/3rdPartyStellar.html
>>
>>
>>
>> *From: *Nick Allen 
>> *Reply-To: *"user@metron.apache.org" 
>> *Date: *Wednesday, January 17, 2018 at 4:46 AM
>> *To: *"user@metron.apache.org" 
>> *Subject: *Re: Define a function that can be used in Stellar
>>
>>
>>
>>
>>
>>
>>
>> If something we have already does not fit the bill, I would recommend
>> creating that function in Java.   Since you described it as "a bit 
>> complex"
>> and "the logic would be complicated" I don't see any value in defining
>> something like this in Stellar with named functions.
>>
>>
>>
>> Best
>>
>>
>>
>>
>>
>>
>>
>>
>>
>> On Wed, Jan 17, 2018 at 7:38 AM Simon Elliston Ball <
>> si...@simonellistonball.com> wrote:
>>
>> Have you looked at the recent TLSH functions in Stellar? We already
>> have that for similarity preserving hashes.
>>
>>
>>
>> Simon
>>
>>
>>
>>
>> On 17 Jan 2018, at 12:35, Ali Nazemian  wrote:
>>
>> It is a bit complex. We want to create a function that accepts a list
>> of arguments for an asset and generate an asset identifier that can be 
>> used
>> as a row_key for the enrichment store. The logic would be complicated,
>> though. We may need to include some sort of similarity aware hash 
>> function
>> as a part of this custom function.
>>
>>
>>
>> On Wed, Jan 17, 2018 at 10:32 PM, Nick Allen 
>> wrote:
>>
>> Ali - Can you describe the logic that you are trying to 

Re: Define a function that can be used in Stellar

[Simon Elliston Ball]

I forgot we added OBJECT_GET. How does the caching work on that? 

Simn

> On 2 Feb 2018, at 14:33, Nick Allen  wrote:
> 
> There are many functions that use the global configuration.  For example, 
> GET_GEO in org.apache.metron.enrichment.stellar.GeoEnrichmentFunctions.  
> There might be a better example, but that is one is staring at me at the 
> moment.
>   
> There is an OBJECT_GET function defined in 
> org.apache.metron.enrichment.stellar.ObjectGet that was purpose-built to 
> retrieve files from HDFS.  If you wanted to retrieve a configuration from 
> HDFS that would be a good example (if you can't just use that functions 
> directly).
> 
> On Fri, Feb 2, 2018 at 8:50 AM Ali Nazemian  > wrote:
> Is there any Stellar function already been implemented in Metron that has a 
> config file associated with it? I am trying to get an idea of how it works.
> 
> On 3 Feb. 2018 00:44, "Simon Elliston Ball"  > wrote:
> Depends how you write the function class, but most likely, yes. Hence global 
> config option. 
> 
> Simon
> 
>> On 2 Feb 2018, at 13:42, Ali Nazemian > > wrote:
>> 
>> Does it mean every time the function gets called it will load the config, 
>> but if I use the global one it will only read it one time and it will be 
>> available in memory?
>> 
>> On 2 Feb. 2018 21:53, "Simon Elliston Ball" > > wrote:
>> Shouldn’t be. The one this I would point out though is that you don’t 
>> necessarily know which supervisor you will be running from, so pulling from 
>> HDFS would make sense. That said, the performance implications are probably 
>> not great. A good option here would be to have the config available in the 
>> global config for example and refer to that, since most instances of stellar 
>> apply global config to their context. 
>> 
>> Simon
>> 
>> 
>>> On 2 Feb 2018, at 07:14, Ali Nazemian >> > wrote:
>>> 
>>> Will be any problem if the Stellar function we want to implement need to 
>>> load an external config file?
>>> 
>>> Cheers,
>>> Ali
>>> 
>>> On Thu, Jan 18, 2018 at 4:58 PM, Ali Nazemian >> > wrote:
>>> Thanks, All.
>>> 
>>> Yes, Nick. It is highly related to our use case and the way that we are 
>>> going to enrich events with assets and vulnerability properties. It is not 
>>> a general case at all.
>>> 
>>> Cheers,
>>> Ali
>>> 
>>> On Thu, Jan 18, 2018 at 5:43 AM, Matt Foley >> > wrote:
>>> Besides the example code Simon mentioned at 
>>> https://github.com/apache/metron/tree/master/metron-stellar/stellar-3rd-party-example
>>>  
>>> 
>>>  ,
>>> there is some documentation at 
>>> http://metron.apache.org/current-book/metron-stellar/stellar-common/3rdPartyStellar.html
>>>  
>>> 
>>>  
>>> 
>>> From: Nick Allen >
>>> Reply-To: "user@metron.apache.org " 
>>> >
>>> Date: Wednesday, January 17, 2018 at 4:46 AM
>>> To: "user@metron.apache.org " 
>>> >
>>> Subject: Re: Define a function that can be used in Stellar
>>> 
>>>  
>>> 
>>>  
>>> 
>>>  
>>> 
>>> If something we have already does not fit the bill, I would recommend 
>>> creating that function in Java.   Since you described it as "a bit complex" 
>>> and "the logic would be complicated" I don't see any value in defining 
>>> something like this in Stellar with named functions.
>>> 
>>>  
>>> 
>>> Best
>>> 
>>>  
>>> 
>>>  
>>> 
>>>  
>>> 
>>>  
>>> 
>>> On Wed, Jan 17, 2018 at 7:38 AM Simon Elliston Ball 
>>> > wrote:
>>> 
>>> Have you looked at the recent TLSH functions in Stellar? We already have 
>>> that for similarity preserving hashes.
>>> 
>>>  
>>> 
>>> Simon
>>> 
>>>  
>>> 
>>> 
>>> On 17 Jan 2018, at 12:35, Ali Nazemian >> > wrote:
>>> 
>>> It is a bit complex. We want to create a function that accepts a list of 
>>> arguments for an asset and generate an asset identifier that can be used as 
>>> a row_key for the enrichment store. The logic would be complicated, though. 
>>> We may need to include some sort of similarity aware hash function as a 
>>> part of this custom function.
>>> 
>>>  
>>> 
>>> On Wed, Jan 17, 2018 at 10:32 PM, Nick Allen >> > 

Re: Define a function that can be used in Stellar

[Ali Nazemian]

Is there any Stellar function already been implemented in Metron that has a
config file associated with it? I am trying to get an idea of how it works.

On 3 Feb. 2018 00:44, "Simon Elliston Ball" 
wrote:

> Depends how you write the function class, but most likely, yes. Hence
> global config option.
>
> Simon
>
> On 2 Feb 2018, at 13:42, Ali Nazemian  wrote:
>
> Does it mean every time the function gets called it will load the config,
> but if I use the global one it will only read it one time and it will be
> available in memory?
>
> On 2 Feb. 2018 21:53, "Simon Elliston Ball" 
> wrote:
>
>> Shouldn’t be. The one this I would point out though is that you don’t
>> necessarily know which supervisor you will be running from, so pulling from
>> HDFS would make sense. That said, the performance implications are probably
>> not great. A good option here would be to have the config available in the
>> global config for example and refer to that, since most instances of
>> stellar apply global config to their context.
>>
>> Simon
>>
>>
>> On 2 Feb 2018, at 07:14, Ali Nazemian  wrote:
>>
>> Will be any problem if the Stellar function we want to implement need to
>> load an external config file?
>>
>> Cheers,
>> Ali
>>
>> On Thu, Jan 18, 2018 at 4:58 PM, Ali Nazemian 
>> wrote:
>>
>>> Thanks, All.
>>>
>>> Yes, Nick. It is highly related to our use case and the way that we are
>>> going to enrich events with assets and vulnerability properties. It is not
>>> a general case at all.
>>>
>>> Cheers,
>>> Ali
>>>
>>> On Thu, Jan 18, 2018 at 5:43 AM, Matt Foley  wrote:
>>>
 Besides the example code Simon mentioned at
 https://github.com/apache/metron/tree/master/metron-stellar/
 stellar-3rd-party-example ,
 there is some documentation at http://metron.apache.org/curre
 nt-book/metron-stellar/stellar-common/3rdPartyStellar.html



 *From: *Nick Allen 
 *Reply-To: *"user@metron.apache.org" 
 *Date: *Wednesday, January 17, 2018 at 4:46 AM
 *To: *"user@metron.apache.org" 
 *Subject: *Re: Define a function that can be used in Stellar







 If something we have already does not fit the bill, I would recommend
 creating that function in Java.   Since you described it as "a bit complex"
 and "the logic would be complicated" I don't see any value in defining
 something like this in Stellar with named functions.



 Best









 On Wed, Jan 17, 2018 at 7:38 AM Simon Elliston Ball <
 si...@simonellistonball.com> wrote:

 Have you looked at the recent TLSH functions in Stellar? We already
 have that for similarity preserving hashes.



 Simon




 On 17 Jan 2018, at 12:35, Ali Nazemian  wrote:

 It is a bit complex. We want to create a function that accepts a list
 of arguments for an asset and generate an asset identifier that can be used
 as a row_key for the enrichment store. The logic would be complicated,
 though. We may need to include some sort of similarity aware hash function
 as a part of this custom function.



 On Wed, Jan 17, 2018 at 10:32 PM, Nick Allen 
 wrote:

 Ali - Can you describe the logic that you are trying to perform? That
 would be useful as a use case to help drive a discussion around creating
 named functions in Stellar.









 On Wed, Jan 17, 2018 at 6:29 AM Ali Nazemian 
 wrote:

 Thanks, Simon. We have already got a script to deal with classpath
 management for the parsers. We should be able to use it for this extension
 as well.



 Yeah, I agree. It will be much easier to define functions on the fly
 and use them afterwards. It could be defined as Lambda or custom function.



 Regards,

 Ali







 On Wed, Jan 17, 2018 at 9:42 PM, Simon Elliston Ball <
 si...@simonellistonball.com> wrote:

 https://github.com/apache/metron/tree/master/metron-stellar/
 stellar-3rd-party-example gives good details on how to add a stellar
 function.



 Stellar will pick up an annotated function on its class path, so to add
 function there is no need to rebuild metron module, but you do need your
 modules on the classpath, and, pending 777, to deal with things like class
 path clash in your dependencies.



 Another idea worth discussion on the dev list is probably the notion of
 defining stellar functions in stellar, which would be a much simpler
 solution than custom java 

Re: Define a function that can be used in Stellar

[Ali Nazemian]

Does it mean every time the function gets called it will load the config,
but if I use the global one it will only read it one time and it will be
available in memory?

On 2 Feb. 2018 21:53, "Simon Elliston Ball" 
wrote:

> Shouldn’t be. The one this I would point out though is that you don’t
> necessarily know which supervisor you will be running from, so pulling from
> HDFS would make sense. That said, the performance implications are probably
> not great. A good option here would be to have the config available in the
> global config for example and refer to that, since most instances of
> stellar apply global config to their context.
>
> Simon
>
>
> On 2 Feb 2018, at 07:14, Ali Nazemian  wrote:
>
> Will be any problem if the Stellar function we want to implement need to
> load an external config file?
>
> Cheers,
> Ali
>
> On Thu, Jan 18, 2018 at 4:58 PM, Ali Nazemian 
> wrote:
>
>> Thanks, All.
>>
>> Yes, Nick. It is highly related to our use case and the way that we are
>> going to enrich events with assets and vulnerability properties. It is not
>> a general case at all.
>>
>> Cheers,
>> Ali
>>
>> On Thu, Jan 18, 2018 at 5:43 AM, Matt Foley  wrote:
>>
>>> Besides the example code Simon mentioned at
>>> https://github.com/apache/metron/tree/master/metron-stellar/
>>> stellar-3rd-party-example ,
>>> there is some documentation at http://metron.apache.org/curre
>>> nt-book/metron-stellar/stellar-common/3rdPartyStellar.html
>>>
>>>
>>>
>>> *From: *Nick Allen 
>>> *Reply-To: *"user@metron.apache.org" 
>>> *Date: *Wednesday, January 17, 2018 at 4:46 AM
>>> *To: *"user@metron.apache.org" 
>>> *Subject: *Re: Define a function that can be used in Stellar
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> If something we have already does not fit the bill, I would recommend
>>> creating that function in Java.   Since you described it as "a bit complex"
>>> and "the logic would be complicated" I don't see any value in defining
>>> something like this in Stellar with named functions.
>>>
>>>
>>>
>>> Best
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> On Wed, Jan 17, 2018 at 7:38 AM Simon Elliston Ball <
>>> si...@simonellistonball.com> wrote:
>>>
>>> Have you looked at the recent TLSH functions in Stellar? We already have
>>> that for similarity preserving hashes.
>>>
>>>
>>>
>>> Simon
>>>
>>>
>>>
>>>
>>> On 17 Jan 2018, at 12:35, Ali Nazemian  wrote:
>>>
>>> It is a bit complex. We want to create a function that accepts a list of
>>> arguments for an asset and generate an asset identifier that can be used as
>>> a row_key for the enrichment store. The logic would be complicated, though.
>>> We may need to include some sort of similarity aware hash function as a
>>> part of this custom function.
>>>
>>>
>>>
>>> On Wed, Jan 17, 2018 at 10:32 PM, Nick Allen  wrote:
>>>
>>> Ali - Can you describe the logic that you are trying to perform? That
>>> would be useful as a use case to help drive a discussion around creating
>>> named functions in Stellar.
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> On Wed, Jan 17, 2018 at 6:29 AM Ali Nazemian 
>>> wrote:
>>>
>>> Thanks, Simon. We have already got a script to deal with classpath
>>> management for the parsers. We should be able to use it for this extension
>>> as well.
>>>
>>>
>>>
>>> Yeah, I agree. It will be much easier to define functions on the fly and
>>> use them afterwards. It could be defined as Lambda or custom function.
>>>
>>>
>>>
>>> Regards,
>>>
>>> Ali
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> On Wed, Jan 17, 2018 at 9:42 PM, Simon Elliston Ball <
>>> si...@simonellistonball.com> wrote:
>>>
>>> https://github.com/apache/metron/tree/master/metron-stellar/
>>> stellar-3rd-party-example gives good details on how to add a stellar
>>> function.
>>>
>>>
>>>
>>> Stellar will pick up an annotated function on its class path, so to add
>>> function there is no need to rebuild metron module, but you do need your
>>> modules on the classpath, and, pending 777, to deal with things like class
>>> path clash in your dependencies.
>>>
>>>
>>>
>>> Another idea worth discussion on the dev list is probably the notion of
>>> defining stellar functions in stellar, which would be a much simpler
>>> solution than custom java functions if you can already express you logic in
>>> stellar.
>>>
>>>
>>>
>>> Simon
>>>
>>>
>>>
>>>
>>>
>>> On 17 Jan 2018, at 10:37, Ali Nazemian  wrote:
>>>
>>>
>>>
>>> Hi Simon,
>>>
>>>
>>>
>>> Yes, that is exactly what we are looking for. Is there any example
>>> regarding adding a Stellar function in Java? Hopefully, we don't need to
>>> rebuild the corresponding modules for this?
>>>
>>>
>>>
>>> Cheers,
>>>
>>> Ali
>>>
>>>
>>>
>>> On Wed, Jan 17, 2018 at 8:40 PM, Simon Elliston Ball <
>>> si...@simonellistonball.com> wrote:
>>>
>>> At present 

Re: Define a function that can be used in Stellar

[Simon Elliston Ball]

Shouldn’t be. The one this I would point out though is that you don’t 
necessarily know which supervisor you will be running from, so pulling from 
HDFS would make sense. That said, the performance implications are probably not 
great. A good option here would be to have the config available in the global 
config for example and refer to that, since most instances of stellar apply 
global config to their context. 

Simon


> On 2 Feb 2018, at 07:14, Ali Nazemian  wrote:
> 
> Will be any problem if the Stellar function we want to implement need to load 
> an external config file?
> 
> Cheers,
> Ali
> 
> On Thu, Jan 18, 2018 at 4:58 PM, Ali Nazemian  > wrote:
> Thanks, All.
> 
> Yes, Nick. It is highly related to our use case and the way that we are going 
> to enrich events with assets and vulnerability properties. It is not a 
> general case at all.
> 
> Cheers,
> Ali
> 
> On Thu, Jan 18, 2018 at 5:43 AM, Matt Foley  > wrote:
> Besides the example code Simon mentioned at 
> https://github.com/apache/metron/tree/master/metron-stellar/stellar-3rd-party-example
>  
> 
>  ,
> there is some documentation at 
> http://metron.apache.org/current-book/metron-stellar/stellar-common/3rdPartyStellar.html
>  
> 
>  
> 
> From: Nick Allen >
> Reply-To: "user@metron.apache.org " 
> >
> Date: Wednesday, January 17, 2018 at 4:46 AM
> To: "user@metron.apache.org " 
> >
> Subject: Re: Define a function that can be used in Stellar
> 
>  
> 
>  
> 
>  
> 
> If something we have already does not fit the bill, I would recommend 
> creating that function in Java.   Since you described it as "a bit complex" 
> and "the logic would be complicated" I don't see any value in defining 
> something like this in Stellar with named functions.
> 
>  
> 
> Best
> 
>  
> 
>  
> 
>  
> 
>  
> 
> On Wed, Jan 17, 2018 at 7:38 AM Simon Elliston Ball 
> > wrote:
> 
> Have you looked at the recent TLSH functions in Stellar? We already have that 
> for similarity preserving hashes.
> 
>  
> 
> Simon
> 
>  
> 
> 
> On 17 Jan 2018, at 12:35, Ali Nazemian  > wrote:
> 
> It is a bit complex. We want to create a function that accepts a list of 
> arguments for an asset and generate an asset identifier that can be used as a 
> row_key for the enrichment store. The logic would be complicated, though. We 
> may need to include some sort of similarity aware hash function as a part of 
> this custom function.
> 
>  
> 
> On Wed, Jan 17, 2018 at 10:32 PM, Nick Allen  > wrote:
> 
> Ali - Can you describe the logic that you are trying to perform? That would 
> be useful as a use case to help drive a discussion around creating named 
> functions in Stellar.
> 
>  
> 
>  
> 
>  
> 
>  
> 
> On Wed, Jan 17, 2018 at 6:29 AM Ali Nazemian  > wrote:
> 
> Thanks, Simon. We have already got a script to deal with classpath management 
> for the parsers. We should be able to use it for this extension as well.
> 
>  
> 
> Yeah, I agree. It will be much easier to define functions on the fly and use 
> them afterwards. It could be defined as Lambda or custom function. 
> 
>  
> 
> Regards,
> 
> Ali
> 
>  
> 
>  
> 
>  
> 
> On Wed, Jan 17, 2018 at 9:42 PM, Simon Elliston Ball 
> > wrote:
> 
> https://github.com/apache/metron/tree/master/metron-stellar/stellar-3rd-party-example
>  
> 
>  gives good details on how to add a stellar function.
> 
>  
> 
> Stellar will pick up an annotated function on its class path, so to add 
> function there is no need to rebuild metron module, but you do need your 
> modules on the classpath, and, pending 777, to deal with things like class 
> path clash in your dependencies. 
> 
>  
> 
> Another idea worth discussion on the dev list is probably the notion of 
> defining stellar functions in stellar, which would be a much simpler solution 
> than custom java functions if you can already express you logic in stellar. 
> 
>  
> 
> Simon
> 
>  
> 
> 
> 
> 
> On 17 Jan 2018, at 10:37, Ali Nazemian  > wrote:
> 
>  
> 
> Hi Simon,
> 
>  
> 
> Yes, that is exactly what we are looking for. Is there any example 

Re: Define a function that can be used in Stellar

[Ali Nazemian]

Will be any problem if the Stellar function we want to implement need to
load an external config file?

Cheers,
Ali

On Thu, Jan 18, 2018 at 4:58 PM, Ali Nazemian  wrote:

> Thanks, All.
>
> Yes, Nick. It is highly related to our use case and the way that we are
> going to enrich events with assets and vulnerability properties. It is not
> a general case at all.
>
> Cheers,
> Ali
>
> On Thu, Jan 18, 2018 at 5:43 AM, Matt Foley  wrote:
>
>> Besides the example code Simon mentioned at
>> https://github.com/apache/metron/tree/master/metron-stellar/
>> stellar-3rd-party-example ,
>> there is some documentation at http://metron.apache.org/curre
>> nt-book/metron-stellar/stellar-common/3rdPartyStellar.html
>>
>>
>>
>> *From: *Nick Allen 
>> *Reply-To: *"user@metron.apache.org" 
>> *Date: *Wednesday, January 17, 2018 at 4:46 AM
>> *To: *"user@metron.apache.org" 
>> *Subject: *Re: Define a function that can be used in Stellar
>>
>>
>>
>>
>>
>>
>>
>> If something we have already does not fit the bill, I would recommend
>> creating that function in Java.   Since you described it as "a bit complex"
>> and "the logic would be complicated" I don't see any value in defining
>> something like this in Stellar with named functions.
>>
>>
>>
>> Best
>>
>>
>>
>>
>>
>>
>>
>>
>>
>> On Wed, Jan 17, 2018 at 7:38 AM Simon Elliston Ball <
>> si...@simonellistonball.com> wrote:
>>
>> Have you looked at the recent TLSH functions in Stellar? We already have
>> that for similarity preserving hashes.
>>
>>
>>
>> Simon
>>
>>
>>
>>
>> On 17 Jan 2018, at 12:35, Ali Nazemian  wrote:
>>
>> It is a bit complex. We want to create a function that accepts a list of
>> arguments for an asset and generate an asset identifier that can be used as
>> a row_key for the enrichment store. The logic would be complicated, though.
>> We may need to include some sort of similarity aware hash function as a
>> part of this custom function.
>>
>>
>>
>> On Wed, Jan 17, 2018 at 10:32 PM, Nick Allen  wrote:
>>
>> Ali - Can you describe the logic that you are trying to perform? That
>> would be useful as a use case to help drive a discussion around creating
>> named functions in Stellar.
>>
>>
>>
>>
>>
>>
>>
>>
>>
>> On Wed, Jan 17, 2018 at 6:29 AM Ali Nazemian 
>> wrote:
>>
>> Thanks, Simon. We have already got a script to deal with classpath
>> management for the parsers. We should be able to use it for this extension
>> as well.
>>
>>
>>
>> Yeah, I agree. It will be much easier to define functions on the fly and
>> use them afterwards. It could be defined as Lambda or custom function.
>>
>>
>>
>> Regards,
>>
>> Ali
>>
>>
>>
>>
>>
>>
>>
>> On Wed, Jan 17, 2018 at 9:42 PM, Simon Elliston Ball <
>> si...@simonellistonball.com> wrote:
>>
>> https://github.com/apache/metron/tree/master/metron-stellar/
>> stellar-3rd-party-example gives good details on how to add a stellar
>> function.
>>
>>
>>
>> Stellar will pick up an annotated function on its class path, so to add
>> function there is no need to rebuild metron module, but you do need your
>> modules on the classpath, and, pending 777, to deal with things like class
>> path clash in your dependencies.
>>
>>
>>
>> Another idea worth discussion on the dev list is probably the notion of
>> defining stellar functions in stellar, which would be a much simpler
>> solution than custom java functions if you can already express you logic in
>> stellar.
>>
>>
>>
>> Simon
>>
>>
>>
>>
>>
>> On 17 Jan 2018, at 10:37, Ali Nazemian  wrote:
>>
>>
>>
>> Hi Simon,
>>
>>
>>
>> Yes, that is exactly what we are looking for. Is there any example
>> regarding adding a Stellar function in Java? Hopefully, we don't need to
>> rebuild the corresponding modules for this?
>>
>>
>>
>> Cheers,
>>
>> Ali
>>
>>
>>
>> On Wed, Jan 17, 2018 at 8:40 PM, Simon Elliston Ball <
>> si...@simonellistonball.com> wrote:
>>
>> At present you can certainly create custom stellar functions in Java. I’m
>> guessing however that what you’re looking to do is create a kind of
>> function that combines a number of stellar functions to avoid repetition,
>> or to ensure consistency of certain parameters for example. Is that what
>> you’re looking for? Maybe some sort of syntax to create a named stellar
>> function similar to the way we create lambdas?
>>
>> Simon
>>
>>
>> > On 17 Jan 2018, at 07:25, Ali Nazemian  wrote:
>> >
>> > Hi all,
>> >
>> > Is there any way that we can define a function that can be used rather
>> than duplicating a logic multiple times?
>> >
>> > Cheers,
>> > Ali
>>
>>
>>
>>
>>
>> --
>>
>> A.Nazemian
>>
>>
>>
>>
>>
>>
>>
>> --
>>
>> A.Nazemian
>>
>>
>>
>>
>>
>> --
>>
>> A.Nazemian
>>
>>
>
>
> --
> A.Nazemian
>



-- 
A.Nazemian

Re: Define a function that can be used in Stellar

[Ali Nazemian]

Thanks, All.

Yes, Nick. It is highly related to our use case and the way that we are
going to enrich events with assets and vulnerability properties. It is not
a general case at all.

Cheers,
Ali

On Thu, Jan 18, 2018 at 5:43 AM, Matt Foley  wrote:

> Besides the example code Simon mentioned at https://github.com/apache/
> metron/tree/master/metron-stellar/stellar-3rd-party-example ,
> there is some documentation at http://metron.apache.org/
> current-book/metron-stellar/stellar-common/3rdPartyStellar.html
>
>
>
> *From: *Nick Allen 
> *Reply-To: *"user@metron.apache.org" 
> *Date: *Wednesday, January 17, 2018 at 4:46 AM
> *To: *"user@metron.apache.org" 
> *Subject: *Re: Define a function that can be used in Stellar
>
>
>
>
>
>
>
> If something we have already does not fit the bill, I would recommend
> creating that function in Java.   Since you described it as "a bit complex"
> and "the logic would be complicated" I don't see any value in defining
> something like this in Stellar with named functions.
>
>
>
> Best
>
>
>
>
>
>
>
>
>
> On Wed, Jan 17, 2018 at 7:38 AM Simon Elliston Ball <
> si...@simonellistonball.com> wrote:
>
> Have you looked at the recent TLSH functions in Stellar? We already have
> that for similarity preserving hashes.
>
>
>
> Simon
>
>
>
>
> On 17 Jan 2018, at 12:35, Ali Nazemian  wrote:
>
> It is a bit complex. We want to create a function that accepts a list of
> arguments for an asset and generate an asset identifier that can be used as
> a row_key for the enrichment store. The logic would be complicated, though.
> We may need to include some sort of similarity aware hash function as a
> part of this custom function.
>
>
>
> On Wed, Jan 17, 2018 at 10:32 PM, Nick Allen  wrote:
>
> Ali - Can you describe the logic that you are trying to perform? That
> would be useful as a use case to help drive a discussion around creating
> named functions in Stellar.
>
>
>
>
>
>
>
>
>
> On Wed, Jan 17, 2018 at 6:29 AM Ali Nazemian 
> wrote:
>
> Thanks, Simon. We have already got a script to deal with classpath
> management for the parsers. We should be able to use it for this extension
> as well.
>
>
>
> Yeah, I agree. It will be much easier to define functions on the fly and
> use them afterwards. It could be defined as Lambda or custom function.
>
>
>
> Regards,
>
> Ali
>
>
>
>
>
>
>
> On Wed, Jan 17, 2018 at 9:42 PM, Simon Elliston Ball <
> si...@simonellistonball.com> wrote:
>
> https://github.com/apache/metron/tree/master/metron-
> stellar/stellar-3rd-party-example gives good details on how to add a
> stellar function.
>
>
>
> Stellar will pick up an annotated function on its class path, so to add
> function there is no need to rebuild metron module, but you do need your
> modules on the classpath, and, pending 777, to deal with things like class
> path clash in your dependencies.
>
>
>
> Another idea worth discussion on the dev list is probably the notion of
> defining stellar functions in stellar, which would be a much simpler
> solution than custom java functions if you can already express you logic in
> stellar.
>
>
>
> Simon
>
>
>
>
>
> On 17 Jan 2018, at 10:37, Ali Nazemian  wrote:
>
>
>
> Hi Simon,
>
>
>
> Yes, that is exactly what we are looking for. Is there any example
> regarding adding a Stellar function in Java? Hopefully, we don't need to
> rebuild the corresponding modules for this?
>
>
>
> Cheers,
>
> Ali
>
>
>
> On Wed, Jan 17, 2018 at 8:40 PM, Simon Elliston Ball <
> si...@simonellistonball.com> wrote:
>
> At present you can certainly create custom stellar functions in Java. I’m
> guessing however that what you’re looking to do is create a kind of
> function that combines a number of stellar functions to avoid repetition,
> or to ensure consistency of certain parameters for example. Is that what
> you’re looking for? Maybe some sort of syntax to create a named stellar
> function similar to the way we create lambdas?
>
> Simon
>
>
> > On 17 Jan 2018, at 07:25, Ali Nazemian  wrote:
> >
> > Hi all,
> >
> > Is there any way that we can define a function that can be used rather
> than duplicating a logic multiple times?
> >
> > Cheers,
> > Ali
>
>
>
>
>
> --
>
> A.Nazemian
>
>
>
>
>
>
>
> --
>
> A.Nazemian
>
>
>
>
>
> --
>
> A.Nazemian
>
>


-- 
A.Nazemian

Re: Define a function that can be used in Stellar

[Matt Foley]

Besides the example code Simon mentioned at 
https://github.com/apache/metron/tree/master/metron-stellar/stellar-3rd-party-example
 ,
there is some documentation at 
http://metron.apache.org/current-book/metron-stellar/stellar-common/3rdPartyStellar.html
 

 

From: Nick Allen 
Reply-To: "user@metron.apache.org" 
Date: Wednesday, January 17, 2018 at 4:46 AM
To: "user@metron.apache.org" 
Subject: Re: Define a function that can be used in Stellar

 

 

 

If something we have already does not fit the bill, I would recommend creating 
that function in Java.   Since you described it as "a bit complex" and "the 
logic would be complicated" I don't see any value in defining something like 
this in Stellar with named functions. 

 

Best

 

 

 

 

On Wed, Jan 17, 2018 at 7:38 AM Simon Elliston Ball 
 wrote:

Have you looked at the recent TLSH functions in Stellar? We already have that 
for similarity preserving hashes. 

 

Simon

 


On 17 Jan 2018, at 12:35, Ali Nazemian  wrote:

It is a bit complex. We want to create a function that accepts a list of 
arguments for an asset and generate an asset identifier that can be used as a 
row_key for the enrichment store. The logic would be complicated, though. We 
may need to include some sort of similarity aware hash function as a part of 
this custom function.

 

On Wed, Jan 17, 2018 at 10:32 PM, Nick Allen  wrote:

Ali - Can you describe the logic that you are trying to perform? That would be 
useful as a use case to help drive a discussion around creating named functions 
in Stellar. 

 

 

 

 

On Wed, Jan 17, 2018 at 6:29 AM Ali Nazemian  wrote:

Thanks, Simon. We have already got a script to deal with classpath management 
for the parsers. We should be able to use it for this extension as well. 

 

Yeah, I agree. It will be much easier to define functions on the fly and use 
them afterwards. It could be defined as Lambda or custom function. 

 

Regards,

Ali

 

 

 

On Wed, Jan 17, 2018 at 9:42 PM, Simon Elliston Ball 
 wrote:

https://github.com/apache/metron/tree/master/metron-stellar/stellar-3rd-party-example
 gives good details on how to add a stellar function. 

 

Stellar will pick up an annotated function on its class path, so to add 
function there is no need to rebuild metron module, but you do need your 
modules on the classpath, and, pending 777, to deal with things like class path 
clash in your dependencies. 

 

Another idea worth discussion on the dev list is probably the notion of 
defining stellar functions in stellar, which would be a much simpler solution 
than custom java functions if you can already express you logic in stellar. 

 

Simon 

 



On 17 Jan 2018, at 10:37, Ali Nazemian  wrote:

 

Hi Simon, 

 

Yes, that is exactly what we are looking for. Is there any example regarding 
adding a Stellar function in Java? Hopefully, we don't need to rebuild the 
corresponding modules for this?

 

Cheers,

Ali

 

On Wed, Jan 17, 2018 at 8:40 PM, Simon Elliston Ball 
 wrote:

At present you can certainly create custom stellar functions in Java. I’m 
guessing however that what you’re looking to do is create a kind of function 
that combines a number of stellar functions to avoid repetition, or to ensure 
consistency of certain parameters for example. Is that what you’re looking for? 
Maybe some sort of syntax to create a named stellar function similar to the way 
we create lambdas?

Simon


> On 17 Jan 2018, at 07:25, Ali Nazemian  wrote:
>
> Hi all,
>
> Is there any way that we can define a function that can be used rather than 
> duplicating a logic multiple times?
>
> Cheers,
> Ali



 

-- 

A.Nazemian

 



 

-- 

A.Nazemian



 

-- 

A.Nazemian

Re: Define a function that can be used in Stellar

[Simon Elliston Ball]

Have you looked at the recent TLSH functions in Stellar? We already have that 
for similarity preserving hashes.

Simon

> On 17 Jan 2018, at 12:35, Ali Nazemian  wrote:
> 
> It is a bit complex. We want to create a function that accepts a list of 
> arguments for an asset and generate an asset identifier that can be used as a 
> row_key for the enrichment store. The logic would be complicated, though. We 
> may need to include some sort of similarity aware hash function as a part of 
> this custom function.
> 
>> On Wed, Jan 17, 2018 at 10:32 PM, Nick Allen  wrote:
>> Ali - Can you describe the logic that you are trying to perform? That would 
>> be useful as a use case to help drive a discussion around creating named 
>> functions in Stellar.
>> 
>> 
>> 
>> 
>>> On Wed, Jan 17, 2018 at 6:29 AM Ali Nazemian  wrote:
>>> Thanks, Simon. We have already got a script to deal with classpath 
>>> management for the parsers. We should be able to use it for this extension 
>>> as well.
>>> 
>>> Yeah, I agree. It will be much easier to define functions on the fly and 
>>> use them afterwards. It could be defined as Lambda or custom function. 
>>> 
>>> Regards,
>>> Ali
>>> 
>>> 
>>> 
 On Wed, Jan 17, 2018 at 9:42 PM, Simon Elliston Ball 
  wrote:
 https://github.com/apache/metron/tree/master/metron-stellar/stellar-3rd-party-example
  gives good details on how to add a stellar function.
 
 Stellar will pick up an annotated function on its class path, so to add 
 function there is no need to rebuild metron module, but you do need your 
 modules on the classpath, and, pending 777, to deal with things like class 
 path clash in your dependencies. 
 
 Another idea worth discussion on the dev list is probably the notion of 
 defining stellar functions in stellar, which would be a much simpler 
 solution than custom java functions if you can already express you logic 
 in stellar. 
 
 Simon
 
 
> On 17 Jan 2018, at 10:37, Ali Nazemian  wrote:
> 
> Hi Simon,
> 
> Yes, that is exactly what we are looking for. Is there any example 
> regarding adding a Stellar function in Java? Hopefully, we don't need to 
> rebuild the corresponding modules for this?
> 
> Cheers,
> Ali
> 
>> On Wed, Jan 17, 2018 at 8:40 PM, Simon Elliston Ball 
>>  wrote:
>> At present you can certainly create custom stellar functions in Java. 
>> I’m guessing however that what you’re looking to do is create a kind of 
>> function that combines a number of stellar functions to avoid 
>> repetition, or to ensure consistency of certain parameters for example. 
>> Is that what you’re looking for? Maybe some sort of syntax to create a 
>> named stellar function similar to the way we create lambdas?
>> 
>> Simon
>> 
>> > On 17 Jan 2018, at 07:25, Ali Nazemian  wrote:
>> >
>> > Hi all,
>> >
>> > Is there any way that we can define a function that can be used rather 
>> > than duplicating a logic multiple times?
>> >
>> > Cheers,
>> > Ali
>> 
> 
> 
> 
> -- 
> A.Nazemian
 
>>> 
>>> 
>>> 
>>> -- 
>>> A.Nazemian
> 
> 
> 
> -- 
> A.Nazemian

Re: Define a function that can be used in Stellar

[Ali Nazemian]

It is a bit complex. We want to create a function that accepts a list of
arguments for an asset and generate an asset identifier that can be used as
a row_key for the enrichment store. The logic would be complicated, though.
We may need to include some sort of similarity aware hash function as a
part of this custom function.

On Wed, Jan 17, 2018 at 10:32 PM, Nick Allen  wrote:

> Ali - Can you describe the logic that you are trying to perform? That
> would be useful as a use case to help drive a discussion around creating
> named functions in Stellar.
>
>
>
>
> On Wed, Jan 17, 2018 at 6:29 AM Ali Nazemian 
> wrote:
>
>> Thanks, Simon. We have already got a script to deal with classpath
>> management for the parsers. We should be able to use it for this extension
>> as well.
>>
>> Yeah, I agree. It will be much easier to define functions on the fly and
>> use them afterwards. It could be defined as Lambda or custom function.
>>
>> Regards,
>> Ali
>>
>>
>>
>> On Wed, Jan 17, 2018 at 9:42 PM, Simon Elliston Ball <
>> si...@simonellistonball.com> wrote:
>>
>>> https://github.com/apache/metron/tree/master/metron-
>>> stellar/stellar-3rd-party-example gives good details on how to add a
>>> stellar function.
>>>
>>> Stellar will pick up an annotated function on its class path, so to add
>>> function there is no need to rebuild metron module, but you do need your
>>> modules on the classpath, and, pending 777, to deal with things like class
>>> path clash in your dependencies.
>>>
>>> Another idea worth discussion on the dev list is probably the notion of
>>> defining stellar functions in stellar, which would be a much simpler
>>> solution than custom java functions if you can already express you logic in
>>> stellar.
>>>
>>> Simon
>>>
>>>
>>> On 17 Jan 2018, at 10:37, Ali Nazemian  wrote:
>>>
>>> Hi Simon,
>>>
>>> Yes, that is exactly what we are looking for. Is there any example
>>> regarding adding a Stellar function in Java? Hopefully, we don't need to
>>> rebuild the corresponding modules for this?
>>>
>>> Cheers,
>>> Ali
>>>
>>> On Wed, Jan 17, 2018 at 8:40 PM, Simon Elliston Ball <
>>> si...@simonellistonball.com> wrote:
>>>
 At present you can certainly create custom stellar functions in Java.
 I’m guessing however that what you’re looking to do is create a kind of
 function that combines a number of stellar functions to avoid repetition,
 or to ensure consistency of certain parameters for example. Is that what
 you’re looking for? Maybe some sort of syntax to create a named stellar
 function similar to the way we create lambdas?

 Simon

 > On 17 Jan 2018, at 07:25, Ali Nazemian  wrote:
 >
 > Hi all,
 >
 > Is there any way that we can define a function that can be used
 rather than duplicating a logic multiple times?
 >
 > Cheers,
 > Ali


>>>
>>>
>>> --
>>> A.Nazemian
>>>
>>>
>>>
>>
>>
>> --
>> A.Nazemian
>>
>


-- 
A.Nazemian

Re: Define a function that can be used in Stellar

[Nick Allen]

Ali - Can you describe the logic that you are trying to perform? That would
be useful as a use case to help drive a discussion around creating named
functions in Stellar.




On Wed, Jan 17, 2018 at 6:29 AM Ali Nazemian  wrote:

> Thanks, Simon. We have already got a script to deal with classpath
> management for the parsers. We should be able to use it for this extension
> as well.
>
> Yeah, I agree. It will be much easier to define functions on the fly and
> use them afterwards. It could be defined as Lambda or custom function.
>
> Regards,
> Ali
>
>
>
> On Wed, Jan 17, 2018 at 9:42 PM, Simon Elliston Ball <
> si...@simonellistonball.com> wrote:
>
>>
>> https://github.com/apache/metron/tree/master/metron-stellar/stellar-3rd-party-example
>>  gives
>> good details on how to add a stellar function.
>>
>> Stellar will pick up an annotated function on its class path, so to add
>> function there is no need to rebuild metron module, but you do need your
>> modules on the classpath, and, pending 777, to deal with things like class
>> path clash in your dependencies.
>>
>> Another idea worth discussion on the dev list is probably the notion of
>> defining stellar functions in stellar, which would be a much simpler
>> solution than custom java functions if you can already express you logic in
>> stellar.
>>
>> Simon
>>
>>
>> On 17 Jan 2018, at 10:37, Ali Nazemian  wrote:
>>
>> Hi Simon,
>>
>> Yes, that is exactly what we are looking for. Is there any example
>> regarding adding a Stellar function in Java? Hopefully, we don't need to
>> rebuild the corresponding modules for this?
>>
>> Cheers,
>> Ali
>>
>> On Wed, Jan 17, 2018 at 8:40 PM, Simon Elliston Ball <
>> si...@simonellistonball.com> wrote:
>>
>>> At present you can certainly create custom stellar functions in Java.
>>> I’m guessing however that what you’re looking to do is create a kind of
>>> function that combines a number of stellar functions to avoid repetition,
>>> or to ensure consistency of certain parameters for example. Is that what
>>> you’re looking for? Maybe some sort of syntax to create a named stellar
>>> function similar to the way we create lambdas?
>>>
>>> Simon
>>>
>>> > On 17 Jan 2018, at 07:25, Ali Nazemian  wrote:
>>> >
>>> > Hi all,
>>> >
>>> > Is there any way that we can define a function that can be used rather
>>> than duplicating a logic multiple times?
>>> >
>>> > Cheers,
>>> > Ali
>>>
>>>
>>
>>
>> --
>> A.Nazemian
>>
>>
>>
>
>
> --
> A.Nazemian
>

Re: Define a function that can be used in Stellar

[Simon Elliston Ball]

https://github.com/apache/metron/tree/master/metron-stellar/stellar-3rd-party-example
 

 gives good details on how to add a stellar function.

Stellar will pick up an annotated function on its class path, so to add 
function there is no need to rebuild metron module, but you do need your 
modules on the classpath, and, pending 777, to deal with things like class path 
clash in your dependencies. 

Another idea worth discussion on the dev list is probably the notion of 
defining stellar functions in stellar, which would be a much simpler solution 
than custom java functions if you can already express you logic in stellar. 

Simon

> On 17 Jan 2018, at 10:37, Ali Nazemian  wrote:
> 
> Hi Simon,
> 
> Yes, that is exactly what we are looking for. Is there any example regarding 
> adding a Stellar function in Java? Hopefully, we don't need to rebuild the 
> corresponding modules for this?
> 
> Cheers,
> Ali
> 
> On Wed, Jan 17, 2018 at 8:40 PM, Simon Elliston Ball 
> > wrote:
> At present you can certainly create custom stellar functions in Java. I’m 
> guessing however that what you’re looking to do is create a kind of function 
> that combines a number of stellar functions to avoid repetition, or to ensure 
> consistency of certain parameters for example. Is that what you’re looking 
> for? Maybe some sort of syntax to create a named stellar function similar to 
> the way we create lambdas?
> 
> Simon
> 
> > On 17 Jan 2018, at 07:25, Ali Nazemian  > > wrote:
> >
> > Hi all,
> >
> > Is there any way that we can define a function that can be used rather than 
> > duplicating a logic multiple times?
> >
> > Cheers,
> > Ali
> 
> 
> 
> 
> -- 
> A.Nazemian

Re: Define a function that can be used in Stellar

[Ali Nazemian]

Hi Simon,

Yes, that is exactly what we are looking for. Is there any example
regarding adding a Stellar function in Java? Hopefully, we don't need to
rebuild the corresponding modules for this?

Cheers,
Ali

On Wed, Jan 17, 2018 at 8:40 PM, Simon Elliston Ball <
si...@simonellistonball.com> wrote:

> At present you can certainly create custom stellar functions in Java. I’m
> guessing however that what you’re looking to do is create a kind of
> function that combines a number of stellar functions to avoid repetition,
> or to ensure consistency of certain parameters for example. Is that what
> you’re looking for? Maybe some sort of syntax to create a named stellar
> function similar to the way we create lambdas?
>
> Simon
>
> > On 17 Jan 2018, at 07:25, Ali Nazemian  wrote:
> >
> > Hi all,
> >
> > Is there any way that we can define a function that can be used rather
> than duplicating a logic multiple times?
> >
> > Cheers,
> > Ali
>
>


-- 
A.Nazemian

Define a function that can be used in Stellar

[Ali Nazemian]

Hi all,

Is there any way that we can define a function that can be used rather than
duplicating a logic multiple times?

Cheers,
Ali