[GitHub] incubator-metron pull request #308: Metron-498 Grok patterns are now read fr...

2016-10-13 Thread merrimanr
Github user merrimanr commented on a diff in the pull request:

https://github.com/apache/incubator-metron/pull/308#discussion_r83234039
  
--- Diff: 


[GitHub] incubator-metron pull request #308: Metron-498 Grok patterns are now read fr...

2016-10-13 Thread ottobackwards
Github user ottobackwards commented on a diff in the pull request:

https://github.com/apache/incubator-metron/pull/308#discussion_r83252938
  
--- Diff: 


[GitHub] incubator-metron issue #276: METRON-363 Fix Cisco ASA Parser

2016-10-13 Thread mattf-horton
Github user mattf-horton commented on the issue: https://github.com/apache/incubator-metron/pull/276 I added a comment above, to SyslogUtils.java line 36, which the system did not email to the list, probably because I immediately edited it to fix a format error. @kylerichardson

[GitHub] incubator-metron pull request #308: Metron-498 Grok patterns are now read fr...

2016-10-13 Thread merrimanr
Github user merrimanr commented on a diff in the pull request:

https://github.com/apache/incubator-metron/pull/308#discussion_r83227314
  
--- Diff: 


[GitHub] incubator-metron pull request #308: Metron-498 Grok patterns are now read fr...

2016-10-13 Thread ottobackwards
Github user ottobackwards commented on a diff in the pull request:

https://github.com/apache/incubator-metron/pull/308#discussion_r83238162
  
--- Diff: 


[GitHub] incubator-metron pull request #308: Metron-498 Grok patterns are now read fr...

2016-10-13 Thread merrimanr
Github user merrimanr commented on a diff in the pull request:

https://github.com/apache/incubator-metron/pull/308#discussion_r83243880
  
--- Diff: 


[GitHub] incubator-metron issue #303: METRON-424 ability to validate ip addresses aga...

2016-10-13 Thread ottobackwards
Github user ottobackwards commented on the issue: https://github.com/apache/incubator-metron/pull/303 Are you going to open a jira for that? -- On October 13, 2016 at 10:16:13, Casey Stella (notificati...@github.com) wrote: @nickwallen

[GitHub] incubator-metron issue #303: METRON-424 ability to validate ip addresses aga...

2016-10-13 Thread cestella
Github user cestella commented on the issue: https://github.com/apache/incubator-metron/pull/303 Yep, I will. :) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] incubator-metron pull request #308: Metron-498 Grok patterns are now read fr...

2016-10-13 Thread merrimanr
Github user merrimanr commented on a diff in the pull request:

https://github.com/apache/incubator-metron/pull/308#discussion_r83230626
  
--- Diff: 


Re: [DISCUSS] Improving quick-dev

2016-10-13 Thread Otto Fowler
- create scripts/utilities to easily run a topology locally in an IDE instead of in the VM THIS. On October 13, 2016 at 12:36:45, Ryan Merriman (merrim...@gmail.com) wrote: Working with the quick-dev vagrant VM recently left a lot to be desired. All forthcoming comments are made

[GitHub] incubator-metron issue #303: METRON-424 ability to validate ip addresses aga...

2016-10-13 Thread cestella
Github user cestella commented on the issue: https://github.com/apache/incubator-metron/pull/303 @ottobackwards Sure, why don't you start a DISCUSS thread and lay out how you envision that looking? --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] incubator-metron issue #303: METRON-424 ability to validate ip addresses aga...

2016-10-13 Thread ottobackwards
Github user ottobackwards commented on the issue: https://github.com/apache/incubator-metron/pull/303 I would like to get the discussion on that going, as adding in new models is something near and dear. --- If your project is set up for it, you can reply to this email and have

[GitHub] incubator-metron issue #303: METRON-424 ability to validate ip addresses aga...

2016-10-13 Thread ottobackwards
Github user ottobackwards commented on the issue: https://github.com/apache/incubator-metron/pull/303 I can't be assigned jiras but I'll take a stab at METRON-502, unless you have already completed it ;) --- If your project is set up for it, you can reply to this email and have your

[GitHub] incubator-metron issue #303: METRON-424 ability to validate ip addresses aga...

2016-10-13 Thread cestella
Github user cestella commented on the issue: https://github.com/apache/incubator-metron/pull/303 That JIRA about making the ParserIntegrationTest better is at https://issues.apache.org/jira/browse/METRON-502 --- If your project is set up for it, you can reply to this email and have

Re: [DISCUSS] Improving quick-dev

2016-10-13 Thread zeo...@gmail.com
+1 Ryan and Otto's comments. I also strongly think we need to make a demo environment easier, but that should be different than quick-dev. Jon On Thu, Oct 13, 2016 at 1:15 PM Otto Fowler wrote: > - create scripts/utilities to easily run a topology locally in an IDE >

[GitHub] incubator-metron issue #303: METRON-424 ability to validate ip addresses aga...

2016-10-13 Thread cestella
Github user cestella commented on the issue: https://github.com/apache/incubator-metron/pull/303 Yeah, I agree. I'd like to have a sensor-specific validator as well as a global validator. Right now we have a `MessageFilter`, but it's slightly different in intent. It is intended to

[GitHub] incubator-metron issue #303: METRON-424 ability to validate ip addresses aga...

2016-10-13 Thread cestella
Github user cestella commented on the issue: https://github.com/apache/incubator-metron/pull/303 Go for it, man. It's all yours. :) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] incubator-metron issue #300: METRON-489: RemoveSubdomains Stellar Function b...

2016-10-13 Thread cestella
Github user cestella commented on the issue: https://github.com/apache/incubator-metron/pull/300 well, this is at least as graceful as a baby giraffe, so done and done. ;) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well.

[GitHub] incubator-metron pull request #308: Metron-498 Grok patterns are now read fr...

2016-10-13 Thread mattf-horton
Github user mattf-horton commented on a diff in the pull request: https://github.com/apache/incubator-metron/pull/308#discussion_r83283132 --- Diff: metron-platform/metron-parsers/src/main/java/org/apache/metron/parsers/interfaces/MessageParser.java --- @@ -34,15 +34,15 @@

[GitHub] incubator-metron pull request #308: Metron-498 Grok patterns are now read fr...

2016-10-13 Thread mattf-horton
Github user mattf-horton commented on a diff in the pull request: https://github.com/apache/incubator-metron/pull/308#discussion_r83277025 --- Diff: metron-platform/metron-parsers/src/main/java/org/apache/metron/parsers/GrokParser.java --- @@ -150,8 +149,8 @@ public void init() {

[GitHub] incubator-metron pull request #308: Metron-498 Grok patterns are now read fr...

2016-10-13 Thread mattf-horton
Github user mattf-horton commented on a diff in the pull request: https://github.com/apache/incubator-metron/pull/308#discussion_r83276073 --- Diff: metron-platform/metron-parsers/src/main/java/org/apache/metron/parsers/GrokParser.java --- @@ -132,8 +130,9 @@ public void init() {

[GitHub] incubator-metron pull request #308: Metron-498 Grok patterns are now read fr...

2016-10-13 Thread merrimanr
Github user merrimanr commented on a diff in the pull request: https://github.com/apache/incubator-metron/pull/308#discussion_r83303723 --- Diff: metron-platform/metron-parsers/src/main/java/org/apache/metron/parsers/bolt/ParserBolt.java --- @@ -117,7 +117,7 @@ public void

Re: [DISCUSS] Improving quick-dev

2016-10-13 Thread Casey Stella
You could. This was a consideration when creating the IntegrationTest infrastructure.There are some challenges here, though, with hbase and storm coexisting in the same jvm. I do not know if this is still an issue with storm 1.0+. On Thu, Oct 13, 2016 at 2:26 PM, Otto Fowler

[GitHub] incubator-metron issue #308: Metron-498 Grok patterns are now read from zook...

2016-10-13 Thread merrimanr
Github user merrimanr commented on the issue: https://github.com/apache/incubator-metron/pull/308 @james-sirota You are correct about changes on the Ansible side. There are steps where HDFS directories are created and grok pattern files written that are not necessary anymore. I

[GitHub] incubator-metron pull request #308: Metron-498 Grok patterns are now read fr...

2016-10-13 Thread merrimanr
Github user merrimanr commented on a diff in the pull request: https://github.com/apache/incubator-metron/pull/308#discussion_r83301389 --- Diff: metron-platform/metron-parsers/src/main/java/org/apache/metron/parsers/GrokParser.java --- @@ -132,8 +130,9 @@ public void init() {

[GitHub] incubator-metron pull request #308: Metron-498 Grok patterns are now read fr...

2016-10-13 Thread merrimanr
Github user merrimanr commented on a diff in the pull request: https://github.com/apache/incubator-metron/pull/308#discussion_r83310911 --- Diff: metron-platform/metron-parsers/src/main/java/org/apache/metron/parsers/interfaces/MessageParser.java --- @@ -34,15 +34,15 @@ *

[DISCUSS] Metron REST API Architecture and Design

2016-10-13 Thread Ryan Merriman
I created a Jira to track this new feature at https://issues.apache.org/jira/browse/METRON-503. I also started and attached an architecture doc to that Jira with some of my ideas about how we should implement it. Please feel free to review and comment or add to it. Looking forward to everyone's

[GitHub] incubator-metron issue #304: METRON-496: Field transformations are applied a...

2016-10-13 Thread justinleet
Github user justinleet commented on the issue: https://github.com/apache/incubator-metron/pull/304 I'm +1 by inspection. I'm pretty agnostic on the question of leaving the capability in and would be inclined to do what Casey said and add another jira if we are interesting removing

[GitHub] incubator-metron pull request #308: Metron-498 Grok patterns are now read fr...

2016-10-13 Thread merrimanr
Github user merrimanr commented on a diff in the pull request: https://github.com/apache/incubator-metron/pull/308#discussion_r83315119 --- Diff: metron-platform/metron-parsers/src/main/java/org/apache/metron/parsers/interfaces/MessageParser.java --- @@ -34,15 +34,15 @@ *

Re: [DISCUSS] Metron REST API Architecture and Design

2016-10-13 Thread zeo...@gmail.com
Along the lines of: • Must be deployed to a machine with adequate resources so that resource contention is avoided. • Will need network access to all other services within Metron Has there been any consideration of a "Metron Manager" node? In the old TP2 bare metal install guide

Re: [DISCUSS] Improving quick-dev

2016-10-13 Thread Nick Allen
To Jon's point, I think it would be useful to have a Demo box that uses generators to produce 3 or 4 types of telemetry that shows up in the Metron Dashboard. This box would be different from Quick-Dev in that everything starts automatically, so that a user just has to launch it and the should

Re: [DISCUSS] Improving quick-dev

2016-10-13 Thread Nick Allen
I really like the idea to replace real sensors in Quick-Dev with data generators; aka spouts that spit-out canned data. The pcap replay mechanism is fairly resource intensive, not to mention all of the sensors like Bro, Snort, etc. Removing these should give us significantly more head room. At

Re: [DISCUSS] Improving quick-dev

2016-10-13 Thread Michael Miklavcic
I think this may have come up in another PR already (have to look for it). But maybe we could maintain our flexibility in quick-dev by installing the sensors and not starting them until we need them. I think it's useful to have a quick "genuine" e2e testing environment that doesn't require running

[GitHub] incubator-metron issue #307: METRON-499 Check for Metron Jar Fails During Qu...

2016-10-13 Thread dlyle65535
Github user dlyle65535 commented on the issue: https://github.com/apache/incubator-metron/pull/307 +1 Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] incubator-metron pull request #307: METRON-499 Check for Metron Jar Fails Du...

2016-10-13 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/incubator-metron/pull/307 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] incubator-metron issue #303: METRON-424 ability to validate ip addresses aga...

2016-10-13 Thread cestella
Github user cestella commented on the issue: https://github.com/apache/incubator-metron/pull/303 I figured out what was going on with this PR and it's kinda interesting, so forgive the long-winded explanation. @ottobackwards is indeed right, it is not apparent why his PR

[GitHub] incubator-metron issue #307: METRON-499 Check for Metron Jar Fails During Qu...

2016-10-13 Thread y0no
Github user y0no commented on the issue: https://github.com/apache/incubator-metron/pull/307 Thanks for fixing! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

Re: Build failure - metron_mpack: Assembly is incorrectly configured

2016-10-13 Thread Yohann Lepage
2016-10-13 15:49 GMT+02:00 David Lyle : > Also works for me, but I think your analysis is correct. Could you open up > a bug jira for a fix? Done: https://issues.apache.org/jira/browse/METRON-500 -- Yohann L.

[GitHub] incubator-metron issue #303: METRON-424 ability to validate ip addresses aga...

2016-10-13 Thread cestella
Github user cestella commented on the issue: https://github.com/apache/incubator-metron/pull/303 @nickwallen Funny you mention that; yes, I think we can. We should modify `ParserIntegrationTest`

[GitHub] incubator-metron issue #303: METRON-424 ability to validate ip addresses aga...

2016-10-13 Thread ottobackwards
Github user ottobackwards commented on the issue: https://github.com/apache/incubator-metron/pull/303 Ok - from what I can see things are written to the kafka topic, but the bolt is never called. I don't see an crashes in the field validation code at this time. Can anyone make