[GitHub] incubator-metron issue #400: METRON-636: Capture memory and cpu details as a...

Github user anandsubbu commented on the issue: https://github.com/apache/incubator-metron/pull/400 Hi @nickwallen , could you please merge the PR by retaining @JonZeolla as the co-opt for this commit? --- If your project is set up for it, you can reply to this email and have your

[GitHub] incubator-metron issue #454: METRON-716 site-book README.md

Github user JonZeolla commented on the issue: https://github.com/apache/incubator-metron/pull/454 +1 (non-binding) by inspection. Thanks for putting this together Otto, good catch. On a related note, it looks like Travis is either backed up or having an issue. Another PR

[GitHub] incubator-metron pull request #451: METRON-157: Added CEF Parser

Github user simonellistonball commented on a diff in the pull request: https://github.com/apache/incubator-metron/pull/451#discussion_r100941897 --- Diff: metron-platform/metron-parsers/src/main/java/org/apache/metron/parsers/utils/DateUtils.java --- @@ -0,0 +1,78 @@ +/**

[GitHub] incubator-metron pull request #454: METRON-716 site-book README.md

Github user mattf-horton commented on a diff in the pull request: https://github.com/apache/incubator-metron/pull/454#discussion_r100938289 --- Diff: site-book/README.md --- @@ -0,0 +1,50 @@ +# Metron Site-Book documentation + +Metron's Site Book is an attempt at

Re: [VOTE] Releasing Apache Metron (incubating) 0.3.1-RC4

+1 Compared contents of release tarball https://dist.apache.org/repos/dist/dev/incubator/metron/0.3.1-RC4-incubating/apache-metron-0.3.1-rc4-incubating.tar.gz with contents of git tag apache-metron-0.3.0-rc4-incubating. They match. Confirmed build and full unit test. Build Mpack Build RPMs

[GitHub] incubator-metron pull request #451: METRON-157: Added CEF Parser

Github user simonellistonball commented on a diff in the pull request: https://github.com/apache/incubator-metron/pull/451#discussion_r100937396 --- Diff: metron-platform/metron-parsers/src/main/java/org/apache/metron/parsers/cef/CEFParser.java --- @@ -0,0 +1,274 @@ +/**

[GitHub] incubator-metron pull request #454: METRON-716 site-book README.md

Github user JonZeolla commented on a diff in the pull request: https://github.com/apache/incubator-metron/pull/454#discussion_r100935985 --- Diff: site-book/README.md --- @@ -0,0 +1,50 @@ +# Metron Site-Book documentation + +Metron's Site Book is an attempt at producing

[GitHub] incubator-metron pull request #454: METRON-716 site-book README.md

Github user JonZeolla commented on a diff in the pull request: https://github.com/apache/incubator-metron/pull/454#discussion_r100935773 --- Diff: site-book/README.md --- @@ -0,0 +1,50 @@ +# Metron Site-Book documentation + +Metron's Site Book is an attempt at producing

[GitHub] incubator-metron pull request #454: METRON-716 site-book README.md

Github user JonZeolla commented on a diff in the pull request: https://github.com/apache/incubator-metron/pull/454#discussion_r100936329 --- Diff: site-book/README.md --- @@ -0,0 +1,50 @@ +# Metron Site-Book documentation + +Metron's Site Book is an attempt at producing

[GitHub] incubator-metron pull request #454: METRON-716 site-book README.md

Github user JonZeolla commented on a diff in the pull request: https://github.com/apache/incubator-metron/pull/454#discussion_r100935685 --- Diff: site-book/README.md --- @@ -0,0 +1,50 @@ +# Metron Site-Book documentation + +Metron's Site Book is an attempt at producing

[GitHub] incubator-metron pull request #454: METRON-716 site-book README.md

Github user JonZeolla commented on a diff in the pull request: https://github.com/apache/incubator-metron/pull/454#discussion_r100936703 --- Diff: site-book/README.md --- @@ -0,0 +1,50 @@ +# Metron Site-Book documentation + +Metron's Site Book is an attempt at producing

[GitHub] incubator-metron pull request #454: METRON-716 site-book README.md

Github user JonZeolla commented on a diff in the pull request: https://github.com/apache/incubator-metron/pull/454#discussion_r100936433 --- Diff: site-book/README.md --- @@ -0,0 +1,50 @@ +# Metron Site-Book documentation + +Metron's Site Book is an attempt at producing

[GitHub] incubator-metron pull request #454: METRON-716 site-book README.md

Github user JonZeolla commented on a diff in the pull request: https://github.com/apache/incubator-metron/pull/454#discussion_r100936871 --- Diff: site-book/README.md --- @@ -0,0 +1,50 @@ +# Metron Site-Book documentation + +Metron's Site Book is an attempt at producing

[GitHub] incubator-metron pull request #454: METRON-716 site-book README.md

Github user mattf-horton commented on a diff in the pull request: https://github.com/apache/incubator-metron/pull/454#discussion_r100912944 --- Diff: site-book/README.md --- @@ -0,0 +1,51 @@ +# Site Book Metron documentation + +Metron's Site Book is an attempt at

[GitHub] incubator-metron pull request #454: METRON-716 site-book README.md

Github user mattf-horton commented on a diff in the pull request: https://github.com/apache/incubator-metron/pull/454#discussion_r100910894 --- Diff: site-book/README.md --- @@ -0,0 +1,51 @@ +# Site Book Metron documentation + +Metron's Site Book is an attempt at

[GitHub] incubator-metron pull request #454: METRON-716 site-book README.md

Github user mattf-horton commented on a diff in the pull request: https://github.com/apache/incubator-metron/pull/454#discussion_r100912086 --- Diff: site-book/README.md --- @@ -0,0 +1,51 @@ +# Site Book Metron documentation --- End diff -- Suggest "Metron

[GitHub] incubator-metron pull request #454: METRON-716 site-book README.md

Github user mattf-horton commented on a diff in the pull request: https://github.com/apache/incubator-metron/pull/454#discussion_r100911935 --- Diff: site-book/README.md --- @@ -0,0 +1,51 @@ +# Site Book Metron documentation + +Metron's Site Book is an attempt at

[GitHub] incubator-metron pull request #438: METRON-686 Record Rule Set that Fired Du...

Github user nickwallen commented on a diff in the pull request: https://github.com/apache/incubator-metron/pull/438#discussion_r100901277 --- Diff: metron-platform/metron-enrichment/src/main/java/org/apache/metron/enrichment/bolt/ThreatIntelJoinBolt.java --- @@ -133,14 +136,18 @@

[GitHub] incubator-metron pull request #454: METRON-716 site-book README.md

Github user JonZeolla commented on a diff in the pull request: https://github.com/apache/incubator-metron/pull/454#discussion_r100898592 --- Diff: site-book/README.md --- @@ -0,0 +1,42 @@ +# Site Book Metron documentation + +Metron's Site Book is an attempt at

[GitHub] incubator-metron pull request #454: METRON-716 site-book README.md

Github user JonZeolla commented on a diff in the pull request: https://github.com/apache/incubator-metron/pull/454#discussion_r100899551 --- Diff: site-book/README.md --- @@ -0,0 +1,42 @@ +# Site Book Metron documentation + +Metron's Site Book is an attempt at

[GitHub] incubator-metron pull request #454: METRON-716 site-book README.md

Github user JonZeolla commented on a diff in the pull request: https://github.com/apache/incubator-metron/pull/454#discussion_r100899029 --- Diff: site-book/README.md --- @@ -0,0 +1,42 @@ +# Site Book Metron documentation + +Metron's Site Book is an attempt at

[GitHub] incubator-metron pull request #454: METRON-716 site-book README.md

Github user JonZeolla commented on a diff in the pull request: https://github.com/apache/incubator-metron/pull/454#discussion_r100899087 --- Diff: site-book/README.md --- @@ -0,0 +1,42 @@ +# Site Book Metron documentation + +Metron's Site Book is an attempt at

[GitHub] incubator-metron pull request #454: METRON-716 site-book README.md

Github user JonZeolla commented on a diff in the pull request: https://github.com/apache/incubator-metron/pull/454#discussion_r100899121 --- Diff: site-book/README.md --- @@ -0,0 +1,42 @@ +# Site Book Metron documentation + +Metron's Site Book is an attempt at

[GitHub] incubator-metron pull request #454: METRON-716 site-book README.md

Github user JonZeolla commented on a diff in the pull request: https://github.com/apache/incubator-metron/pull/454#discussion_r100899291 --- Diff: site-book/README.md --- @@ -0,0 +1,42 @@ +# Site Book Metron documentation + +Metron's Site Book is an attempt at

[GitHub] incubator-metron pull request #453: METRON-694: Index Errors from Topologies

Github user merrimanr commented on a diff in the pull request: https://github.com/apache/incubator-metron/pull/453#discussion_r100897656 --- Diff: metron-deployment/packaging/ambari/metron-mpack/src/main/resources/common-services/METRON/CURRENT/configuration/metron-env.xml --- @@

[GitHub] incubator-metron pull request #454: METRON-716 site-book README.md

GitHub user ottobackwards opened a pull request: https://github.com/apache/incubator-metron/pull/454 METRON-716 site-book README.md Initial draft of a readme for site-book documentation. Questions: should we list the tools used more explicitly? You can merge this pull

[GitHub] incubator-metron pull request #453: METRON-694: Index Errors from Topologies

Github user merrimanr commented on a diff in the pull request: https://github.com/apache/incubator-metron/pull/453#discussion_r100894071 --- Diff: metron-deployment/packaging/ambari/metron-mpack/src/main/resources/common-services/METRON/CURRENT/configuration/metron-env.xml --- @@

[GitHub] incubator-metron pull request #453: METRON-694: Index Errors from Topologies

Github user merrimanr commented on a diff in the pull request: https://github.com/apache/incubator-metron/pull/453#discussion_r100892848 --- Diff: metron-platform/metron-writer/src/main/java/org/apache/metron/writer/bolt/BulkMessageWriterBolt.java --- @@ -74,7 +81,11 @@ public

Re: Site-Book

Okay, thanks.  I suggest grabbing the text from the PR#429 introduction. From: Otto Fowler Date: Monday, February 13, 2017 at 11:09 AM To: "dev@metron.incubator.apache.org" , Matt Foley Subject: Re: Site-Book

[GitHub] incubator-metron issue #453: METRON-694: Index Errors from Topologies

Github user ottobackwards commented on the issue: https://github.com/apache/incubator-metron/pull/453 @cestella - how can we utilize the error indexing if we were going to say - output errors or warnings that there were deprecated stellar statements? --- If your project is set up

Re: Site-Book

Actually I was going to take a stab at it, but I was reviewing the error indexing stuff. Sorry to be tardy. I’ll still take a stab if you have not done it. Assign the jira to me On February 13, 2017 at 13:51:55, Matt Foley (ma...@apache.org) wrote: Assuming that I should take that as a

Re: Site-Book

Assuming that I should take that as a request rather than an offer :-) , I’ve opened https://issues.apache.org/jira/browse/METRON-716 Thanks, --Matt On 2/13/17, 7:02 AM, "Casey Stella" wrote: Yes, definitely. On Mon, Feb 13, 2017 at 09:01 Otto Fowler

[GitHub] incubator-metron pull request #451: METRON-157: Added CEF Parser

Github user kylerichardson commented on a diff in the pull request: https://github.com/apache/incubator-metron/pull/451#discussion_r100864330 --- Diff: metron-platform/metron-parsers/src/main/java/org/apache/metron/parsers/utils/DateUtils.java --- @@ -0,0 +1,78 @@ +/** +

[GitHub] incubator-metron pull request #451: METRON-157: Added CEF Parser

Github user kylerichardson commented on a diff in the pull request: https://github.com/apache/incubator-metron/pull/451#discussion_r100862828 --- Diff: metron-platform/metron-parsers/src/main/java/org/apache/metron/parsers/cef/CEFParser.java --- @@ -0,0 +1,274 @@ +/** + *

[GitHub] incubator-metron pull request #453: METRON-694: Index Errors from Topologies

Github user JonZeolla commented on a diff in the pull request: https://github.com/apache/incubator-metron/pull/453#discussion_r100859285 --- Diff: metron-platform/metron-common/src/main/java/org/apache/metron/common/error/MetronError.java --- @@ -0,0 +1,200 @@ +/** + *

[GitHub] incubator-metron issue #453: METRON-694: Index Errors from Topologies

Github user ottobackwards commented on the issue: https://github.com/apache/incubator-metron/pull/453 What comes to mind is the 'source' of an error. Is this error wrong because METRON thinks it is invalid, or it it wrong because of some other configuration specific evaluation. I

[GitHub] incubator-metron issue #453: METRON-694: Index Errors from Topologies

Github user merrimanr commented on the issue: https://github.com/apache/incubator-metron/pull/453 On the topic of invalid messages, they are now treated as error messages. They can still be distinguished as invalid message though. Is there any reason they should be treated

[GitHub] incubator-metron issue #453: METRON-694: Index Errors from Topologies

Github user merrimanr commented on the issue: https://github.com/apache/incubator-metron/pull/453 In response to "Is there any reason we didn't just use the normal indexing topology". Here are the issues I see with doing that. First, I think we should be careful about putting

[GitHub] incubator-metron issue #453: METRON-694: Index Errors from Topologies

Github user ottobackwards commented on the issue: https://github.com/apache/incubator-metron/pull/453 That makes complete sense, we should call that stuff out pre-review. I think what we are seeing is that folks have some really good ideas and are willing to contribute to

[GitHub] incubator-metron pull request #453: METRON-694: Index Errors from Topologies

Github user merrimanr commented on a diff in the pull request: https://github.com/apache/incubator-metron/pull/453#discussion_r100848948 --- Diff: metron-platform/metron-common/src/main/java/org/apache/metron/common/error/MetronError.java --- @@ -0,0 +1,200 @@ +/** + *

[GitHub] incubator-metron pull request #453: METRON-694: Index Errors from Topologies

Github user merrimanr commented on a diff in the pull request: https://github.com/apache/incubator-metron/pull/453#discussion_r100848754 --- Diff: metron-platform/metron-indexing/src/main/config/zookeeper/indexing/error.json --- @@ -0,0 +1,17 @@ +{ + "hdfs" : { ---

[GitHub] incubator-metron issue #453: METRON-694: Index Errors from Topologies

Github user merrimanr commented on the issue: https://github.com/apache/incubator-metron/pull/453 Sorry I should have included this in the original description. I still need to update the various READMEs, that task is outstanding and this should not be merged until that is done.

Re: [VOTE] Releasing Apache Metron (incubating) 0.3.1-RC4

+1 (non-binding) - Built Metron 0.3.1 Ambari Mpack and RPMs. - Setup 12 node cluster on Openstack VMs using Mpack and RPMs. - Ran sample tests for bro, yaf, snort and squid topologies to validate GEO enrichment and indexing are working fine. No issues found. Regards, Anand On 2/11/17, 1:52

[GitHub] incubator-metron pull request #453: METRON-694: Index Errors from Topologies

Github user ottobackwards commented on a diff in the pull request: https://github.com/apache/incubator-metron/pull/453#discussion_r100820671 --- Diff: metron-platform/metron-indexing/src/main/config/zookeeper/indexing/error.json --- @@ -0,0 +1,17 @@ +{ + "hdfs" : {

[GitHub] incubator-metron pull request #453: METRON-694: Index Errors from Topologies

Github user ottobackwards commented on a diff in the pull request: https://github.com/apache/incubator-metron/pull/453#discussion_r100813285 --- Diff: metron-deployment/packaging/ambari/metron-mpack/src/main/resources/common-services/METRON/CURRENT/configuration/metron-env.xml ---

[GitHub] incubator-metron pull request #453: METRON-694: Index Errors from Topologies

Github user ottobackwards commented on a diff in the pull request: https://github.com/apache/incubator-metron/pull/453#discussion_r100817317 --- Diff: metron-platform/metron-elasticsearch/src/main/config/elasticsearch_error.properties --- @@ -0,0 +1,69 @@ +# Licensed to the

[GitHub] incubator-metron pull request #453: METRON-694: Index Errors from Topologies

Github user ottobackwards commented on a diff in the pull request: https://github.com/apache/incubator-metron/pull/453#discussion_r100815478 --- Diff: metron-platform/metron-common/src/main/java/org/apache/metron/common/error/MetronError.java --- @@ -0,0 +1,200 @@ +/** +

[GitHub] incubator-metron pull request #453: METRON-694: Index Errors from Topologies

Github user ottobackwards commented on a diff in the pull request: https://github.com/apache/incubator-metron/pull/453#discussion_r100815725 --- Diff: metron-platform/metron-common/src/main/java/org/apache/metron/common/error/MetronError.java --- @@ -0,0 +1,200 @@ +/** +

Re: Site-Book

Yes, definitely. On Mon, Feb 13, 2017 at 09:01 Otto Fowler wrote: > Should Site-Book have a README.md describing the contents, how to build > etc? >

Site-Book

Should Site-Book have a README.md describing the contents, how to build etc?

Re: Unable to build Ansible 2.0.0.2 on macOS using our instructions

That'll be great, thanks. It'd be great to run it up on your rig. On Mon, Feb 13, 2017 at 09:34 zeo...@gmail.com wrote: > Ok sounds good. I'm traveling next week, but if the stars align I'll > review when I get back. > > Jon > > On Mon, Feb 13, 2017 at 9:08 AM David Lyle

Re: Unable to build Ansible 2.0.0.2 on macOS using our instructions

Ok sounds good. I'm traveling next week, but if the stars align I'll review when I get back. Jon On Mon, Feb 13, 2017 at 9:08 AM David Lyle wrote: > I've got a working branch that needs a bit of testing. I'm traveling this > week, so I won't get it in a submittable state

[GitHub] incubator-metron issue #453: METRON-694: Index Errors from Topologies

Github user ottobackwards commented on the issue: https://github.com/apache/incubator-metron/pull/453 Just real quick before I start to review: - What are the performance implications of this? How can we measure that? - What is the effect of running this on quick dev? is

[GitHub] incubator-metron pull request #453: METRON-694: Index Errors from Topologies

Github user cestella commented on a diff in the pull request: https://github.com/apache/incubator-metron/pull/453#discussion_r100801585 --- Diff: metron-platform/metron-writer/src/main/java/org/apache/metron/writer/bolt/BulkMessageWriterBolt.java --- @@ -74,7 +81,11 @@ public

[GitHub] incubator-metron pull request #453: METRON-694: Index Errors from Topologies

Github user cestella commented on a diff in the pull request: https://github.com/apache/incubator-metron/pull/453#discussion_r100799809 --- Diff: metron-platform/metron-common/src/main/java/org/apache/metron/common/error/MetronError.java --- @@ -0,0 +1,200 @@ +/** + *

[GitHub] incubator-metron pull request #453: METRON-694: Index Errors from Topologies

Github user cestella commented on a diff in the pull request: https://github.com/apache/incubator-metron/pull/453#discussion_r100799375 --- Diff: metron-platform/metron-common/src/main/java/org/apache/metron/common/error/MetronError.java --- @@ -0,0 +1,200 @@ +/** + *

[GitHub] incubator-metron pull request #453: METRON-694: Index Errors from Topologies

Github user cestella commented on a diff in the pull request: https://github.com/apache/incubator-metron/pull/453#discussion_r100796802 --- Diff: metron-platform/metron-indexing/src/main/config/zookeeper/indexing/error.json --- @@ -0,0 +1,17 @@ +{ + "hdfs" : { ---

[GitHub] incubator-metron issue #453: METRON-694: Index Errors from Topologies

Github user cestella commented on the issue: https://github.com/apache/incubator-metron/pull/453 Based on this PR, what I'd probably do is: * Walk through setting up the CSV parser * Set up a field validation * Send valid data through, make sure it works and is in the index

[GitHub] incubator-metron issue #452: Removed MySQL from Enrichment Diagram

Github user JonZeolla commented on the issue: https://github.com/apache/incubator-metron/pull/452 Please rename the PR to "Metron-715: Enrichment Diagram still includes MySQL, which is no longer used". Also +1 (non-binding) by inspection --- If your project is set up for it, you

[GitHub] incubator-metron issue #453: METRON-694: Index Errors from Topologies

Github user merrimanr commented on the issue: https://github.com/apache/incubator-metron/pull/453 No you are correct, we need a more comprehensive test plan. I'm still thinking about it. Triggering errors at each point in the topologies is not straightforward. Sending in a message

[GitHub] incubator-metron issue #453: METRON-694: Index Errors from Topologies

Github user cestella commented on the issue: https://github.com/apache/incubator-metron/pull/453 So from what I see we have a new topology and every error message has a source type of "error", correct? Is there any reason we didn't just use the normal indexing topology and index

[GitHub] incubator-metron issue #453: METRON-694: Index Errors from Topologies

Github user cestella commented on the issue: https://github.com/apache/incubator-metron/pull/453 Whoops sorry I see that you did mention a way to test it out. Sorry, on a phone ;) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub

[GitHub] incubator-metron issue #453: METRON-694: Index Errors from Topologies

Github user cestella commented on the issue: https://github.com/apache/incubator-metron/pull/453 Can you provide an acceptance test plan for validation on vagrant or a cluster? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as

[GitHub] incubator-metron issue #453: METRON-694: Index Errors from Topologies

Github user merrimanr commented on the issue: https://github.com/apache/incubator-metron/pull/453 METRON-694 includes both the topology changes and the Ambari MPack changes. I started on METRON-695 but decided to include both in a single PR, hence the branch being named METRON-695

[GitHub] incubator-metron issue #453: METRON-694: Index Errors from Topologies

Github user cestella commented on the issue: https://github.com/apache/incubator-metron/pull/453 Did you name this PR correctly? Should it be METRON-695? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] incubator-metron pull request #453: METRON-694: Index Errors from Topologies

GitHub user merrimanr opened a pull request: https://github.com/apache/incubator-metron/pull/453 METRON-694: Index Errors from Topologies This PR addresses METRON-695, including updates to the Ambari MPack. A summary of the changes: - Defaulted FieldValidator.input to