Re: SUM aggregator not working?

It's working now, so I'm happy :) On 2017-10-04 14:03, Casey Stella wrote: Ok, so this is subtle. Your rules are wrong and I totally understand why you thought they were right. When we index into ES, we take . and convert them to :, however PRIOR to indexing (when threat triage is running)

[GitHub] metron pull request #737: METRON-1161: Add ability to edit parser command li...

Github user asfgit closed the pull request at: https://github.com/apache/metron/pull/737 ---

[GitHub] metron issue #768: Metron 1123: Add group by option using faceted search cap...

Github user iraghumitra commented on the issue: https://github.com/apache/metron/pull/768 @ottobackwards Good to hear the build issue is out of the way I was going nowhere in figuring out the build failure. - The ungroup button does nothing when alerts are not grouped. I feel we

[GitHub] metron pull request #768: Metron 1123: Add group by option using faceted sea...

Github user iraghumitra commented on a diff in the pull request: https://github.com/apache/metron/pull/768#discussion_r142671246 --- Diff: metron-interface/metron-alerts/e2e/alerts-list/tree-view/tree-view.e2e-spec.ts --- @@ -0,0 +1,183 @@ +/// +/** + * Licensed to

[GitHub] metron issue #768: Metron 1123: Add group by option using faceted search cap...

Github user ottobackwards commented on the issue: https://github.com/apache/metron/pull/768 I think we should disable the button if there are no groups. I *does* do something, I don't know if it is resorting or something else, but the table refreshes and seems to have different

[GitHub] metron pull request #779: METRON-1218: Metron REST should return better erro...

Github user asfgit closed the pull request at: https://github.com/apache/metron/pull/779 ---

[GitHub] metron issue #768: Metron 1123: Add group by option using faceted search cap...

Github user ottobackwards commented on the issue: https://github.com/apache/metron/pull/768 I am not sure. For multi field names "foo:bar:baz" maybe just "baz". or maybe like exceptions "f:b:baz". But that may not be great either. Just wanted to bring up the point for discussion,

[GitHub] metron issue #737: METRON-1161: Add ability to edit parser command line opti...

Github user cestella commented on the issue: https://github.com/apache/metron/pull/737 +1 by inspection. This looks good to me. Great job! ---

[GitHub] metron issue #768: Metron 1123: Add group by option using faceted search cap...

Github user iraghumitra commented on the issue: https://github.com/apache/metron/pull/768 @merrimanr I will check the escalation actions, I guess I missed something there. ---

[GitHub] metron pull request #779: METRON-1218: Metron REST should return better erro...

Github user cestella commented on a diff in the pull request: https://github.com/apache/metron/pull/779#discussion_r142672128 --- Diff: metron-interface/metron-rest/src/main/java/org/apache/metron/rest/controller/RestExceptionHandler.java --- @@ -35,7 +36,7 @@

[GitHub] metron issue #779: METRON-1218: Metron REST should return better error messa...

Github user cestella commented on the issue: https://github.com/apache/metron/pull/779 +1, great job ---

[GitHub] metron issue #768: Metron 1123: Add group by option using faceted search cap...

Github user merrimanr commented on the issue: https://github.com/apache/metron/pull/768 With respect to displaying field names, I think it's going to be tough coming up with a strategy that solves every case. This will likely depend on a user's preference and is subjective anyways.

[GitHub] metron issue #783: METRON-1228: Configuration Management PUSH immediately do...

Github user cestella commented on the issue: https://github.com/apache/metron/pull/783 +1 this has been pissing me off for days ---

[GitHub] metron issue #768: Metron 1123: Add group by option using faceted search cap...

Github user iraghumitra commented on the issue: https://github.com/apache/metron/pull/768 @ottobackwards & @merrimanr Fixed the following - Disabling the ungroup button when no groups are selected - Fixed issue with select all alerts in tree view ---

Re: [DISCUSS] Dropping support for elastic 2.x

So, how would this work in an upgrade scenario that does not involve losing the existing indexed data? On Wed, Oct 4, 2017 at 12:55 PM, Michael Miklavcic < michael.miklav...@gmail.com> wrote: > The client I'm currently working on moving towards would *not* be backwards > compatible. >

Re: [DISCUSS] Dropping support for elastic 2.x

Forgot the link https://www.elastic.co/guide/en/elasticsearch/reference/current/setup-upgrade.html On Wed, Oct 4, 2017 at 1:07 PM, Simon Elliston Ball < si...@simonellistonball.com> wrote: > The simplest option would probably be to upgrade the ES and then reindex > from the HDFS store.

Re: [DISCUSS] Dropping support for elastic 2.x

The question comes back to the DISCUSS I opened the other day about upgrading ES. I believe we could theoretically maintain backwards compatibility, but we'd have to keep the existing TransportClient. It's not deprecated yet, but it will be. Keeping the ability to manage ES 2.x and 5.x+ via Ambari

Re: [DISCUSS] Dropping support for elastic 2.x

Regarding backwards compatibility at the code level, what are the pros/cons (outside of the obvious con that the transport client will be deprecated)? I guess what I'm trying to get at is what do we get in terms of functionality moving to a new backwards-incompatible transport client? A separate

[GitHub] metron issue #780: METRON-1220: Create documentation around alert nested fie...

Github user justinleet commented on the issue: https://github.com/apache/metron/pull/780 @cestella @nickwallen Hopefully took care of comments that don't involve migrating wiki docs. Let me know if I looked over anything. Sidenote, does anybody know how to actually link to a head

[DISCUSS] Dropping support for elastic 2.x

A number of people are currently working on upgrading the ES support in Metron to 5.x (including the clients, and the mpack managed install). Would anyone have any objections to dropping formal support for 2.x as a result of this work? In theory the clients should be backward compatible against

Re: [DISCUSS] Dropping support for elastic 2.x

The client I'm currently working on moving towards would *not* be backwards compatible. https://www.elastic.co/guide/en/elasticsearch/client/java-rest/current/java-rest-high-compatibility.html " The High Level Client is guaranteed to be able to communicate with any Elasticsearch node running on

[GitHub] metron pull request #784: METRON-1229: Management UI type is part of the dec...

GitHub user merrimanr opened a pull request: https://github.com/apache/metron/pull/784 METRON-1229: Management UI type is part of the declarations of 2 modules ## Contributor Comments The PR fixes an intermittent bug that was recently introduced. This can be verified by

Re: [DISCUSS] Dropping support for elastic 2.x

The simplest option would probably be to upgrade the ES and then reindex from the HDFS store. Alternatively there are means to do inplace upgrades from 2.x to 5.x I believe. Simon > On 4 Oct 2017, at 18:05, Casey Stella wrote: > > So, how would this work in an upgrade

Re: [DISCUSS] Dropping support for elastic 2.x

ES should be upgradeable without wiping. It's the client itself that isn't backwards compatible. It'll require both an upgrade of Metron and an ES cluster. On Wed, Oct 4, 2017 at 1:05 PM, Casey Stella wrote: > So, how would this work in an upgrade scenario that does not

Re: [DISCUSS] Dropping support for elastic 2.x

I should note that there's a difference between supporting INSTALLING multiple versions versus being able to manage them. On Wed, Oct 4, 2017 at 11:20 AM, Michael Miklavcic < michael.miklav...@gmail.com> wrote: > The question comes back to the DISCUSS I opened the other day about > upgrading ES.

Re: [DISCUSS] Dropping support for elastic 2.x

If we break out the indexing from the hdfswriting, we could just have two different topologies to configure couldn’t we? On October 4, 2017 at 13:20:19, Michael Miklavcic ( michael.miklav...@gmail.com) wrote: The question comes back to the DISCUSS I opened the other day about upgrading ES. I

Re: [DISCUSS] Dropping support for elastic 2.x

Ok, so, whoever does this ES work, we should ensure the upgrade path is at least spelled out in the Upgrade doc. This would also probably, IMO, necessitate a major version change in metron. On Wed, Oct 4, 2017 at 1:07 PM, Justin Leet wrote: > Forgot the link >

[GitHub] metron pull request #785: METRON-1230: As a stopgap prior to METRON-777, add...

GitHub user cestella opened a pull request: https://github.com/apache/metron/pull/785 METRON-1230: As a stopgap prior to METRON-777, add more simplistic sideloading of custom Parsers ## Contributor Comments Until we get METRON-777 in, it'd be nice to have a simple ability using

[GitHub] metron issue #785: METRON-1230: As a stopgap prior to METRON-777, add more s...

Github user cestella commented on the issue: https://github.com/apache/metron/pull/785 The only real change here is the addition of a Transformer that hooks into Storm's `storm jar` command and will merge the dependent jars. This was done so that we could avoid touching the Parser

[GitHub] metron pull request #786: METRON-1231: Separate Sensor name and topic in the...

GitHub user merrimanr opened a pull request: https://github.com/apache/metron/pull/786 METRON-1231: Separate Sensor name and topic in the Management UI ## Contributor Comments This PR separates sensor name and sensor Kafka topic in both the REST app as well as the Management

[GitHub] metron pull request #783: METRON-1228: Configuration Management PUSH immedia...

GitHub user mmiklavc opened a pull request: https://github.com/apache/metron/pull/783 METRON-1228: Configuration Management PUSH immediately does DUMP after Addresses https://issues.apache.org/jira/browse/METRON-1228 ## Contributor Comments Missing 'break' is causing

[GitHub] metron issue #783: METRON-1228: Configuration Management PUSH immediately do...

Github user ottobackwards commented on the issue: https://github.com/apache/metron/pull/783 +1 ---

Re: SUM aggregator not working?

You're right, with ES 5 we can use periods directly instead of transforming them in indexing to colons (actually, this feature was reintroduced sin 2.4 ). I outlined this as a benefit in the original JIRA

Re: who is having problems installing?

Extending this to the user list as well. Whoever needs help can you quickly let me know: - What environment are you installing on (a single VM, multiple VMs, bare metal, AWS, etc) - What OS are you using - How many sensors are you going to be consuming I'll throw a meeting together once I

Re: who is having problems installing?

Yes the intent is for everyone that has any type of metron installation issue or question attend the meeting 03.10.2017, 17:35, "Ronirose Caryll De Castro" : > Can those who are planning to install Metron join the meeting? > > *Thank you!* > *Caryll* > > On

Re: [DISCUSS] Build broken due to transitive dependencies

Can you run it with the -X flag and paste the error? What version of the gcc compiler do you have? 02.10.2017, 09:37, "Laurens Vets" : > I might have spoken too soon. This is what I see now on 0.4.1-release: > > ... > [INFO] metron-contrib

SUM aggregator not working?

No idea whether it's a bug yet, I just need a 2nd set of eyes :) This is my event as indexed in ES (Obviously some parts have been obfuscated): { "_index": "cloudtrail_index_2017.10.04.19", "_type": "cloudtrail_doc", "_id": "95617686-bd39-46ff-b5c0-db3aeb5b6bab", "_score": null,

Re: [DISCUSS] Is there a reason for separate Management & Alerts UIs?

At some point in the future we may think about converging them because functions like defining threat rules and setting up profiles may overlap the SOC and ops personnel. But as you said, the initial intent was that the two UIs target two different user personas. 02.10.2017, 11:35, "Nick

Re: SUM aggregator not working?

Ok, so this is subtle. Your rules are wrong and I totally understand why you thought they were right. When we index into ES, we take . and convert them to :, however PRIOR to indexing (when threat triage is running) those fields have .'s not :'s Therefore, your rules should be: