Do you have any thoughts on what these other operations might be?
What I’m imagining is something that basically specifies a policy on how to
handle things that the transformation block does not explicitly handle. Right
now, we just leave them along and they flow through.
Would “policy”: “explicit”, or “policy”: “onlyExplict” make sense and give the
flex?
To my mind “operation” implies further transformation, which would just be
another block, no?
Maybe it’s just semantic pedantry on my part… would we see this sort of policy
logic applying to other transformations? It doesn’t really make sense for
“remove”, and well… who cares about any of the other legacy transforms now we
have Stellar :)
Simon
> On 1 Dec 2017, at 01:14, Otto Fowler <ottobackwa...@gmail.com> wrote:
>
> I would suggest that instead of explicitly having “complete”, we have
> “operation”:”complete”
>
> Such that we can have multiple transformations, each with a different
> “operation”.
> No operation would be the status quo ante, if we can do it so that we don’t
> get errors with old configs and the keep same behavior.
>
> {
> "fieldTransformations": [
> {
> "transformation": "STELLAR",
> “operation": “complete",
> "output": ["ip_src_addr", "ip_dst_addr"],
> "config": {
> "ip_src_addr": "ipSrc",
> "ip_dest_addr": "ipDst"
> } ,
> {
> "transformation": "STELLAR",
> “operation": “SomeOtherThing",
> "output": [“foo", “bar"],
> "config": {
> “foo": “TO_UPPER(foo)",
> “bar": “TO_LOWER(bar)"
> }
> }
> ]
> }
>
>
> Sorry for the junk examples, but hopefully it makes sense.
>
>
>
>
>
> On November 30, 2017 at 20:00:06, Simon Elliston Ball
> (si...@simonellistonball.com <mailto:si...@simonellistonball.com>) wrote:
>
>> I’m looking at the way parser config works, and transformation of field from
>> their native names in, for example the ASA or CEF parsers, into a standard
>> data model.
>>
>> At the moment I would do something like this:
>>
>> assuming I have fields [ipSrc, ipDst, pointlessExtraStuff, message] I might
>> have:
>>
>> {
>> "fieldTransformations": [
>> {
>> "transformation": "STELLAR",
>> "output": ["ip_src_addr", "ip_dst_addr", "message"],
>> "config": {
>> "ip_src_addr": "ipSrc",
>> "ip_dest_addr": "ipDst"
>> }
>> }
>> ]
>> }
>>
>> which leave me with the field set:
>> [ipSrc, ipDst, pointlessExtraStuff, message, ip_src_addr, ip_dest_addr]
>>
>> unless I go with:-
>>
>> {
>> "fieldTransformations": [
>> {
>> "transformation": "STELLAR",
>> "output": ["ip_src_addr", "ip_dst_addr", "message"],
>> "config": {
>> "ip_src_addr": "ipSrc",
>> "ip_dest_addr": "ipDst",
>> "pointlessExtraStuff": null,
>> "ipSrc": null,
>> "ipDst": null
>> }
>> }
>> ]
>> }
>>
>> which seems a little over verbose.
>>
>> Do you think it would be valuable to add a switch of some sort on the
>> transformation to make it “complete”, i.e. to only preserve fields which are
>> explicitly set.
>>
>> To my mind, this breaks a principal of mutability, but gives us much much
>> cleaner mapping of data.
>>
>> I would propose something like:
>>
>> {
>> "fieldTransformations": [
>> {
>> "transformation": "STELLAR",
>> "complete": true,
>> "output": ["ip_src_addr", "ip_dst_addr", "message"],
>> "config": {
>> "ip_src_addr": "ipSrc",
>> "ip_dest_addr": "ipDst"
>> }
>> }
>> ]
>> }
>>
>> which would give me the set ["ip_src_addr", "ip_dst_addr", "message”]
>> effectively making the nulling in my previous example implicit.
>>
>> Thoughts?
>>
>> Also, in the second scenario, if ‘output' were to be empty would we assume
>> that the output field set should be ["ip_src_addr", “ip_dst_addr”]?
>>
>> Simon
