[GitHub] metron issue #850: METRON-1335: Install metron-maas-service RPM as a part of...

2017-12-04 Thread anandsubbu
Github user anandsubbu commented on the issue: https://github.com/apache/metron/pull/850 bump. Any feedback on this pull request? ---

Re: Heterogeneous indexing batch size for different Metron feeds

2017-12-04 Thread Ali Nazemian
No specific error in the logs. I haven't enabled debug/trace, though. On Tue, Dec 5, 2017 at 11:54 AM, Otto Fowler wrote: > My first thought is what are the errors when you get a high error rate? > > > On December 4, 2017 at 19:34:29, Ali Nazemian

Re: [MENTORS][DISCUSS] Release Procedure + 'Kafka Plugin for Bro'

2017-12-04 Thread zeo...@gmail.com
I would prefer a, but I was initially thinking of doing the plugin first and then get in the two PRs out to use this new tag, which are already +1'd and just waiting on this conversation. For reference, https://github.com/apache/metron/pull/847 and

Re: [MENTORS][DISCUSS] Release Procedure + 'Kafka Plugin for Bro'

2017-12-04 Thread Matt Foley
Honestly, I think (a) is sufficient, and less work. But I’m willing to do (b) if the community prefers. On 12/4/17, 5:54 PM, "Otto Fowler" wrote: It seems to me, as I believe I have stated before that a) feels like the proper way to handle this. It is how I

Re: [MENTORS][DISCUSS] Release Procedure + 'Kafka Plugin for Bro'

2017-12-04 Thread Otto Fowler
It seems to me, as I believe I have stated before that a) feels like the proper way to handle this. It is how I have seen other projects like NiFi handle things as well. On December 4, 2017 at 17:14:41, Matt Foley (ma...@apache.org) wrote: Okay, looking at this from the perspective of making

[GitHub] metron issue #856: METRON-1339 Stellar Shell functionality to verify stored ...

2017-12-04 Thread ottobackwards
Github user ottobackwards commented on the issue: https://github.com/apache/metron/pull/856 @nickwallen @mattf-horton I think we can use the annotation approach to resolve [METRON-989](https://issues.apache.org/jira/browse/METRON-989) as well. Thoughts? ---

[GitHub] metron pull request #851: METRON-1336 Patching Can Result in Bad Configurati...

2017-12-04 Thread ottobackwards
Github user ottobackwards commented on a diff in the pull request: https://github.com/apache/metron/pull/851#discussion_r154824244 --- Diff: metron-platform/metron-common/src/main/java/org/apache/metron/common/configuration/ConfigurationsUtils.java --- @@ -343,25 +343,42 @@

[GitHub] metron pull request #851: METRON-1336 Patching Can Result in Bad Configurati...

2017-12-04 Thread ottobackwards
Github user ottobackwards commented on a diff in the pull request: https://github.com/apache/metron/pull/851#discussion_r154824002 --- Diff: metron-platform/metron-common/src/main/java/org/apache/metron/common/configuration/ConfigurationsUtils.java --- @@ -343,25 +343,42 @@

[GitHub] metron pull request #851: METRON-1336 Patching Can Result in Bad Configurati...

2017-12-04 Thread ottobackwards
Github user ottobackwards commented on a diff in the pull request: https://github.com/apache/metron/pull/851#discussion_r154824031 --- Diff: metron-platform/metron-common/src/main/java/org/apache/metron/common/configuration/ConfigurationsUtils.java --- @@ -343,25 +343,42 @@

[GitHub] metron pull request #851: METRON-1336 Patching Can Result in Bad Configurati...

2017-12-04 Thread nickwallen
Github user nickwallen commented on a diff in the pull request: https://github.com/apache/metron/pull/851#discussion_r154821946 --- Diff: metron-platform/metron-common/src/main/java/org/apache/metron/common/configuration/ConfigurationsUtils.java --- @@ -343,25 +343,42 @@ public

Re: Heterogeneous indexing batch size for different Metron feeds

2017-12-04 Thread Otto Fowler
My first thought is what are the errors when you get a high error rate? On December 4, 2017 at 19:34:29, Ali Nazemian (alinazem...@gmail.com) wrote: Any thoughts? On Sun, Dec 3, 2017 at 11:27 PM, Ali Nazemian wrote: > Hi, > > We have noticed recently that no matter

Re: Heterogeneous indexing batch size for different Metron feeds

2017-12-04 Thread Ali Nazemian
Any thoughts? On Sun, Dec 3, 2017 at 11:27 PM, Ali Nazemian wrote: > Hi, > > We have noticed recently that no matter what batch size we use for Metron > indexing feeds, as long as we start using different batch size for > different Metron feeds, indexing topology

[GitHub] metron pull request #851: METRON-1336 Patching Can Result in Bad Configurati...

2017-12-04 Thread ottobackwards
Github user ottobackwards commented on a diff in the pull request: https://github.com/apache/metron/pull/851#discussion_r154804149 --- Diff: metron-platform/metron-common/src/main/java/org/apache/metron/common/configuration/ConfigurationsUtils.java --- @@ -572,15 +590,22 @@

[GitHub] metron pull request #851: METRON-1336 Patching Can Result in Bad Configurati...

2017-12-04 Thread ottobackwards
Github user ottobackwards commented on a diff in the pull request: https://github.com/apache/metron/pull/851#discussion_r154803527 --- Diff: metron-platform/metron-common/src/main/java/org/apache/metron/common/configuration/ConfigurationsUtils.java --- @@ -572,15 +590,22 @@

[GitHub] metron pull request #851: METRON-1336 Patching Can Result in Bad Configurati...

2017-12-04 Thread ottobackwards
Github user ottobackwards commented on a diff in the pull request: https://github.com/apache/metron/pull/851#discussion_r154802693 --- Diff: metron-platform/metron-common/src/main/java/org/apache/metron/common/configuration/ConfigurationsUtils.java --- @@ -343,25 +343,42 @@

[GitHub] metron pull request #851: METRON-1336 Patching Can Result in Bad Configurati...

2017-12-04 Thread nickwallen
Github user nickwallen commented on a diff in the pull request: https://github.com/apache/metron/pull/851#discussion_r154799152 --- Diff: metron-platform/metron-common/src/main/java/org/apache/metron/common/configuration/ConfigurationsUtils.java --- @@ -572,15 +590,22 @@ public

[GitHub] metron pull request #851: METRON-1336 Patching Can Result in Bad Configurati...

2017-12-04 Thread nickwallen
Github user nickwallen commented on a diff in the pull request: https://github.com/apache/metron/pull/851#discussion_r154798328 --- Diff: metron-platform/metron-common/src/main/java/org/apache/metron/common/configuration/ConfigurationsUtils.java --- @@ -343,25 +343,42 @@ public

Re: [MENTORS][DISCUSS] Release Procedure + 'Kafka Plugin for Bro'

2017-12-04 Thread Matt Foley
Okay, looking at this from the perspective of making a release: We have two choices: a) I can simply make a 0.1 (or 1.0 or 0.4.2) release of metron-bro-plugin-kafka, at the same time and using the same process (modulo the necessary) as Metron.  This is dirt simple.  b) I or someone needs

Re: [DISCUSS] Upcoming Release

2017-12-04 Thread zeo...@gmail.com
Can we resolve the conversation regarding the second repo? I was waiting to get more input/preferences from people There's also a documentation update that fixes a few broken Stellar docs that already has aa +1, I just need to merge it. Jon On Mon, Dec 4, 2017, 17:01 Casey Stella

Re: [DISCUSS] Upcoming Release

2017-12-04 Thread Casey Stella
I would be in favor of a release at this point. On Mon, Dec 4, 2017 at 4:57 PM, Matt Foley wrote: > Hey all, > I see METRON-1252 was resolved over the weekend. Shall I go ahead and > start the process with 0.4.2 release? > Does anyone have any commits they feel strongly

Re: [DISCUSS] Upcoming Release

2017-12-04 Thread Matt Foley
Hey all, I see METRON-1252 was resolved over the weekend. Shall I go ahead and start the process with 0.4.2 release? Does anyone have any commits they feel strongly should go in before 0.4.2 is done, or are we ready to call it good? I believe there is consensus the 0.4.2 release should

[GitHub] metron issue #840: METRON-939: Upgrade ElasticSearch and Kibana

2017-12-04 Thread ottobackwards
Github user ottobackwards commented on the issue: https://github.com/apache/metron/pull/840 Yeah, I'm trying to think of the right way to do it, without having to have it be "officially maintained". more like an informal /testing_stuff , with some descriptions, when it was

[GitHub] metron issue #840: METRON-939: Upgrade ElasticSearch and Kibana

2017-12-04 Thread mmiklavc
Github user mmiklavc commented on the issue: https://github.com/apache/metron/pull/840 I don't think that's a bad idea. I definitely like having test scripts associated with the PR's because we can explicitly see what was done and/or recommended at the time the PR was being ushered

[GitHub] metron issue #840: METRON-939: Upgrade ElasticSearch and Kibana

2017-12-04 Thread ottobackwards
Github user ottobackwards commented on the issue: https://github.com/apache/metron/pull/840 So, this has me thinking It is a shame to have these test materials ( and others like @JonZeolla creates ) embedded in these pr's. Maybe we should have someplace to put them in

[GitHub] metron issue #840: METRON-939: Upgrade ElasticSearch and Kibana

2017-12-04 Thread mmiklavc
Github user mmiklavc commented on the issue: https://github.com/apache/metron/pull/840 Ok, here is the remaining bit of the test plan for verifying ES. Thanks @cestella for pulling together some of the public test scripts around this. # Test Script Testing Kibana

[GitHub] metron pull request #851: METRON-1336 Patching Can Result in Bad Configurati...

2017-12-04 Thread ottobackwards
Github user ottobackwards commented on a diff in the pull request: https://github.com/apache/metron/pull/851#discussion_r154761355 --- Diff: metron-platform/metron-common/src/main/java/org/apache/metron/common/configuration/ConfigurationsUtils.java --- @@ -343,25 +343,42 @@

[GitHub] metron pull request #851: METRON-1336 Patching Can Result in Bad Configurati...

2017-12-04 Thread ottobackwards
Github user ottobackwards commented on a diff in the pull request: https://github.com/apache/metron/pull/851#discussion_r154763112 --- Diff: metron-platform/metron-common/src/main/java/org/apache/metron/common/configuration/ConfigurationsUtils.java --- @@ -572,15 +590,22 @@

[GitHub] metron issue #851: METRON-1336 Patching Can Result in Bad Configuration

2017-12-04 Thread nickwallen
Github user nickwallen commented on the issue: https://github.com/apache/metron/pull/851 @mmiklavc @cestella The kick is complete. All PRs merged. Ready for review. ---

[GitHub] metron pull request #855: METRON-1338 Excluding retry files from RAT check

2017-12-04 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/metron/pull/855 ---

Re: DISCUSS: Quick change to parser config

2017-12-04 Thread Otto Fowler
I’m not sure about consensus. I would like to see it summarized. My point about assignment has to do with how many assignment like operators we are going to support. The fact that the assignment is to a variable that is temporary or not doesn’t need to be part of the grammar/language, since all

Re: DISCUSS: Quick change to parser config

2017-12-04 Thread Simon Elliston Ball
Personally I suspect that temporary variable is a different thing as is the assignment PR. Might be useful for intermediate steps in a parser, but then we’re potentially getting more complex than a parser wants to be. I am warming to the idea of temporary variables though. In terms of the

Re: DISCUSS: Quick change to parser config

2017-12-04 Thread Otto Fowler
Would https://github.com/apache/metron/pull/687 play some role in this? Or could it be made to? On December 4, 2017 at 12:21:40, Casey Stella (ceste...@gmail.com) wrote: So, just chiming in here. It seems to me that we have a problem with extraneous fields in a couple of different ways: *

Re: DISCUSS: Quick change to parser config

2017-12-04 Thread Casey Stella
So, just chiming in here. It seems to me that we have a problem with extraneous fields in a couple of different ways: * Temporary Variables I think that the problem of temporary variables is one beyond just the parser. What I'd like to see is the Stellar field transformations operate similar

[GitHub] metron pull request #823: METRON-1286 Add MIN & MAX Stellar functions

2017-12-04 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/metron/pull/823 ---

[GitHub] metron issue #823: METRON-1286 Add MIN & MAX Stellar functions

2017-12-04 Thread justinleet
Github user justinleet commented on the issue: https://github.com/apache/metron/pull/823 +1, I'm set ---

[GitHub] metron pull request #851: METRON-1336 Patching Can Result in Bad Configurati...

2017-12-04 Thread cestella
Github user cestella commented on a diff in the pull request: https://github.com/apache/metron/pull/851#discussion_r154663544 --- Diff: metron-platform/metron-common/src/main/java/org/apache/metron/common/configuration/ConfigurationsUtils.java --- @@ -343,25 +343,57 @@ public

[GitHub] metron issue #857: METRON-1340: Improve e2e tests for metron alerts

2017-12-04 Thread justinleet
Github user justinleet commented on the issue: https://github.com/apache/metron/pull/857 I haven't looked at the code yet, but I was able to pull this down and run the e2e tests repeatedly without failures, so this is definitely great stuff. ---

[GitHub] metron issue #856: METRON-1339 Stellar Shell functionality to verify stored ...

2017-12-04 Thread ottobackwards
Github user ottobackwards commented on the issue: https://github.com/apache/metron/pull/856 @cestella @simonellistonball With the new implementation, doing the blob or file check should be a piece of cake would you prefer it as part of this or as a new issue? ---