Hmmm… Actually, I kinda like that.
May want a little refactoring in the back for clarity.
My question about whether we could ever imagine this ‘cleanup policy’ applying
to other transforms would sway me to the field rather than transformation name
approach though.
Simon
> On 1 Dec 2017,
Do you have any thoughts on what these other operations might be?
What I’m imagining is something that basically specifies a policy on how to
handle things that the transformation block does not explicitly handle. Right
now, we just leave them along and they flow through.
Would “policy”:
Or, we can create new transformation types
STELLAR_COMPLETE, which may be more in line with the original design.
On November 30, 2017 at 20:14:46, Otto Fowler (ottobackwa...@gmail.com)
wrote:
I would suggest that instead of explicitly having “complete”, we have
“operation”:”complete”
Such
I would suggest that instead of explicitly having “complete”, we have
“operation”:”complete”
Such that we can have multiple transformations, each with a different
“operation”.
No operation would be the status quo ante, if we can do it so that we don’t
get errors with old configs and the keep same
Github user ottobackwards commented on the issue:
https://github.com/apache/metron/pull/856
Per the conversation above, i'm going to take a stab at the attributed
approach.
I think the Stellar Functions should be a separate Jira.
---
Github user ottobackwards commented on a diff in the pull request:
https://github.com/apache/metron/pull/856#discussion_r154245631
--- Diff:
metron-platform/metron-common/src/main/java/org/apache/metron/common/configuration/StellarStatementReporter.java
---
@@ -0,0 +1,166 @@
I’m looking at the way parser config works, and transformation of field from
their native names in, for example the ASA or CEF parsers, into a standard data
model.
At the moment I would do something like this:
assuming I have fields [ipSrc, ipDst, pointlessExtraStuff, message] I might
have:
Github user ottobackwards commented on a diff in the pull request:
https://github.com/apache/metron/pull/856#discussion_r154241894
--- Diff:
metron-platform/metron-common/src/main/java/org/apache/metron/common/configuration/StellarStatementReporter.java
---
@@ -0,0 +1,166 @@
Github user mmiklavc commented on a diff in the pull request:
https://github.com/apache/metron/pull/851#discussion_r154231709
--- Diff:
metron-platform/metron-common/src/main/java/org/apache/metron/common/configuration/ConfigurationsUtils.java
---
@@ -343,25 +343,57 @@ public
Github user cestella commented on a diff in the pull request:
https://github.com/apache/metron/pull/851#discussion_r154215790
--- Diff:
metron-platform/metron-common/src/main/java/org/apache/metron/common/configuration/ConfigurationsUtils.java
---
@@ -343,25 +343,57 @@ public
Github user nickwallen commented on a diff in the pull request:
https://github.com/apache/metron/pull/851#discussion_r154209165
--- Diff:
metron-platform/metron-common/src/main/java/org/apache/metron/common/configuration/ConfigurationsUtils.java
---
@@ -343,25 +343,57 @@ public
Github user nickwallen commented on a diff in the pull request:
https://github.com/apache/metron/pull/851#discussion_r154208493
--- Diff:
metron-platform/metron-common/src/main/java/org/apache/metron/common/configuration/ConfigurationsUtils.java
---
@@ -343,25 +343,57 @@ public
Github user cestella commented on a diff in the pull request:
https://github.com/apache/metron/pull/851#discussion_r154208318
--- Diff:
metron-platform/metron-common/src/main/java/org/apache/metron/common/configuration/ConfigurationsUtils.java
---
@@ -156,7 +156,7 @@ public
Github user mmiklavc commented on a diff in the pull request:
https://github.com/apache/metron/pull/851#discussion_r154207865
--- Diff:
metron-platform/metron-common/src/main/java/org/apache/metron/common/configuration/ConfigurationsUtils.java
---
@@ -156,7 +156,7 @@ public
Github user cestella commented on a diff in the pull request:
https://github.com/apache/metron/pull/851#discussion_r154207614
--- Diff:
metron-platform/metron-common/src/main/java/org/apache/metron/common/configuration/ConfigurationsUtils.java
---
@@ -156,7 +156,7 @@ public
Github user nickwallen commented on a diff in the pull request:
https://github.com/apache/metron/pull/851#discussion_r154207266
--- Diff:
metron-platform/metron-common/src/main/java/org/apache/metron/common/configuration/ConfigurationsUtils.java
---
@@ -156,7 +156,7 @@ public
Github user cestella commented on a diff in the pull request:
https://github.com/apache/metron/pull/851#discussion_r154206090
--- Diff:
metron-platform/metron-common/src/main/java/org/apache/metron/common/configuration/ConfigurationsUtils.java
---
@@ -343,25 +343,57 @@ public
Github user cestella commented on a diff in the pull request:
https://github.com/apache/metron/pull/851#discussion_r154204458
--- Diff:
metron-platform/metron-common/src/main/java/org/apache/metron/common/configuration/ConfigurationsUtils.java
---
@@ -343,25 +343,57 @@ public
Github user cestella commented on a diff in the pull request:
https://github.com/apache/metron/pull/851#discussion_r154202685
--- Diff:
metron-platform/metron-common/src/main/java/org/apache/metron/common/configuration/ConfigurationsUtils.java
---
@@ -343,25 +343,57 @@ public
Github user ottobackwards commented on the issue:
https://github.com/apache/metron/pull/856
I am glad for the interest in this PR, and that it seems to have sparked
some great ideas for continuing on.
What I would like to do is line it up as follows
1. This PR with
Github user cestella commented on the issue:
https://github.com/apache/metron/pull/855
+1 by inspection
---
Github user asfgit closed the pull request at:
https://github.com/apache/metron/pull/814
---
Github user nickwallen commented on a diff in the pull request:
https://github.com/apache/metron/pull/856#discussion_r154189837
--- Diff:
metron-platform/metron-common/src/main/java/org/apache/metron/common/configuration/StellarStatementReporter.java
---
@@ -0,0 +1,166 @@
Github user cestella commented on the issue:
https://github.com/apache/metron/pull/814
Ok, my +1 stands
---
Github user ottobackwards commented on a diff in the pull request:
https://github.com/apache/metron/pull/856#discussion_r154185591
--- Diff:
metron-platform/metron-common/src/main/java/org/apache/metron/common/configuration/StellarStatementReporter.java
---
@@ -0,0 +1,166 @@
Github user nickwallen commented on a diff in the pull request:
https://github.com/apache/metron/pull/856#discussion_r154171805
--- Diff:
metron-platform/metron-common/src/main/java/org/apache/metron/common/configuration/StellarStatementReporter.java
---
@@ -0,0 +1,166 @@
Github user ottobackwards commented on a diff in the pull request:
https://github.com/apache/metron/pull/856#discussion_r154155977
--- Diff:
metron-platform/metron-common/src/main/java/org/apache/metron/common/configuration/StellarStatementReporter.java
---
@@ -0,0 +1,166 @@
Github user ottobackwards commented on a diff in the pull request:
https://github.com/apache/metron/pull/856#discussion_r154155446
--- Diff:
metron-platform/metron-common/src/main/java/org/apache/metron/common/configuration/StellarStatementReporter.java
---
@@ -0,0 +1,166 @@
Github user ottobackwards commented on a diff in the pull request:
https://github.com/apache/metron/pull/856#discussion_r154135697
--- Diff:
metron-platform/metron-common/src/main/java/org/apache/metron/common/configuration/StellarStatementReporter.java
---
@@ -0,0 +1,166 @@
Github user nickwallen commented on a diff in the pull request:
https://github.com/apache/metron/pull/856#discussion_r154133974
--- Diff:
metron-platform/metron-common/src/main/java/org/apache/metron/common/configuration/StellarStatementReporter.java
---
@@ -0,0 +1,166 @@
Github user nickwallen commented on a diff in the pull request:
https://github.com/apache/metron/pull/856#discussion_r154133309
--- Diff:
metron-platform/metron-common/src/main/java/org/apache/metron/common/configuration/StellarStatementReporter.java
---
@@ -0,0 +1,166 @@
Github user ottobackwards commented on a diff in the pull request:
https://github.com/apache/metron/pull/856#discussion_r154124236
--- Diff:
metron-platform/metron-common/src/main/java/org/apache/metron/common/configuration/StellarStatementReporter.java
---
@@ -0,0 +1,166 @@
Github user cestella commented on the issue:
https://github.com/apache/metron/pull/856
@simonellistonball Agree to the namespace idea. My bad :)
---
Github user ottobackwards commented on a diff in the pull request:
https://github.com/apache/metron/pull/856#discussion_r154123254
--- Diff:
metron-platform/metron-common/src/main/java/org/apache/metron/common/configuration/StellarStatementReporter.java
---
@@ -0,0 +1,166 @@
Github user nickwallen commented on a diff in the pull request:
https://github.com/apache/metron/pull/856#discussion_r154118407
--- Diff:
metron-platform/metron-common/src/main/java/org/apache/metron/common/configuration/StellarStatementReporter.java
---
@@ -0,0 +1,166 @@
Github user ottobackwards commented on a diff in the pull request:
https://github.com/apache/metron/pull/856#discussion_r154114714
--- Diff:
metron-platform/metron-common/src/main/java/org/apache/metron/common/configuration/StellarStatementReporter.java
---
@@ -0,0 +1,166 @@
Github user ottobackwards commented on a diff in the pull request:
https://github.com/apache/metron/pull/856#discussion_r15404
--- Diff:
metron-platform/metron-common/src/main/java/org/apache/metron/common/configuration/StellarStatementReporter.java
---
@@ -0,0 +1,166 @@
Github user ottobackwards commented on a diff in the pull request:
https://github.com/apache/metron/pull/856#discussion_r154110765
--- Diff:
metron-platform/metron-common/src/main/java/org/apache/metron/common/configuration/StellarStatementReporter.java
---
@@ -0,0 +1,166 @@
Github user nickwallen commented on a diff in the pull request:
https://github.com/apache/metron/pull/856#discussion_r154110355
--- Diff:
metron-platform/metron-common/src/main/java/org/apache/metron/common/configuration/StellarStatementReporter.java
---
@@ -0,0 +1,166 @@
Github user ottobackwards commented on a diff in the pull request:
https://github.com/apache/metron/pull/856#discussion_r154109552
--- Diff:
metron-platform/metron-common/src/main/java/org/apache/metron/common/configuration/StellarStatementReporter.java
---
@@ -0,0 +1,166 @@
Github user nickwallen commented on a diff in the pull request:
https://github.com/apache/metron/pull/856#discussion_r154105765
--- Diff:
metron-platform/metron-common/src/main/java/org/apache/metron/common/configuration/StellarStatementReporter.java
---
@@ -0,0 +1,166 @@
Github user ottobackwards commented on the issue:
https://github.com/apache/metron/pull/814
Ok, I'll check it out after i do my full dev test on validate
---
Github user cestella commented on the issue:
https://github.com/apache/metron/pull/814
sorry, it's not set, so it's null.
---
Github user ottobackwards commented on the issue:
https://github.com/apache/metron/pull/814
Ok, what was is_alert in this test?
---
Github user cestella commented on the issue:
https://github.com/apache/metron/pull/814
actually, hold on there, If ound one more bug:
```
[Stellar]>>> match { is_alert == null => null, is_alert => 'alert', default
=> 'nah' }
[!] null
java.lang.NullPointerException
Github user cestella commented on the issue:
https://github.com/apache/metron/pull/814
Alright, +1 otto, you did great work here. I'm very, very impressed.
Thanks to @jjmeyer0 for the careful review and assistance. Open source dev at
its finest.
---
Github user ottobackwards commented on the issue:
https://github.com/apache/metron/pull/856
@simonellistonball, yes, the namespace should be part of the jira and
interface design
---
Github user ottobackwards commented on the issue:
https://github.com/apache/metron/pull/856
So, the scenario here is checking things that *were* valid when uploaded,
but have been invalidated by external changes ( language changes ). I would
like to keep the magic specific.
Github user simonellistonball commented on the issue:
https://github.com/apache/metron/pull/856
@cestella I would say that proposed validate function has to be very much
in a namespace. It feels like a name that would be much more useful for a
function replacing our current approach
Github user cestella commented on the issue:
https://github.com/apache/metron/pull/856
Also, it might be useful for `%validate_configured_expressions` to take a
file path so you can validate a set of configs on disk (again, if it gets to
zookeeper, zk_load_utils.sh should fail if
Github user cestella commented on the issue:
https://github.com/apache/metron/pull/856
Any chance we can add a `VALIDATE(str, type)` function to the stellar
management functions where str is the json blob string for the config and the
type is the type of config? Generally the goal
Github user ottobackwards commented on the issue:
https://github.com/apache/metron/pull/814
PR: https://github.com/apache/metron/pull/856 adds capability to use the
stellar shell to validate stellar statements at rest out of ZK
---
Github user nickwallen commented on the issue:
https://github.com/apache/metron/pull/854
Yes, I would just get in the habit of running `./mvn clean install
-DskipTests` rather than what we normally do.
---
Github user cestella commented on the issue:
https://github.com/apache/metron/pull/814
Yeah, let me run this up once more in the REPL and take a final look at the
docs, but I've been monitoring and I like what I see so far.
---
GitHub user ottobackwards opened a pull request:
https://github.com/apache/metron/pull/856
METRON-1339 Stellar Shell functionality to verify stored stellar statements
This will allow users to check their deployed statements, say after
upgrade, when they are at rest ( and would
Github user cestella commented on the issue:
https://github.com/apache/metron/pull/823
I'm all set. +1
---
Github user ottobackwards commented on the issue:
https://github.com/apache/metron/pull/823
@jasper-k thanks for the contribution. Really great job.
+1
@cestella @justinleet are you guys all set?
---
Github user jasper-k closed the pull request at:
https://github.com/apache/metron/pull/823
---
GitHub user jasper-k reopened a pull request:
https://github.com/apache/metron/pull/823
METRON-1286 Add MIN & MAX Stellar functions
## Contributor Comments
Currently Stellar lacks straightforward MAX & MIN functions that take just
a list of values as input.
The functions
Github user ottobackwards commented on the issue:
https://github.com/apache/metron/pull/823
That looks great! Can you close and reopen the PR so we can get a green
build? Looks like there was a failure due to network issues.
---
Github user jasper-k commented on the issue:
https://github.com/apache/metron/pull/823
@ottobackwards
Aligned annotation and README now. Added example and more clear function
description
---
Github user anandsubbu commented on the issue:
https://github.com/apache/metron/pull/852
+1 (non-binding) @nickwallen . This is a much needed fix since it is now
straight-forward to anyone new and wanting to try Metron.
---
62 matches
Mail list logo
