Re: [DISCUSS] Synopsis of Community Meeting on 8/22/2017

2017-08-23 Thread zeo...@gmail.com
Was there any discussion about future features of Metron aside from 777/942? In the initial announce thread the agenda mentioned where want to take the project long-term and feature requests and comments on existing features. My thoughts on the topic are that I would like to see a move quickly aft

Re: [DISCUSS] Synopsis of Community Meeting on 8/22/2017

2017-08-23 Thread Simon Elliston Ball
Hi Jon, Good points all. Some of the versions we have been using are certainly aging. There are some PRs and tickets out for ES 5 upgrade, and some fine work has already been done on Centos 7 installation. I think it would be well worth us making some community decisions about bumping up some

Re: [DISCUSS] Synopsis of Community Meeting on 8/22/2017

2017-08-23 Thread zeo...@gmail.com
My assumption with HDP 3 was it would include Kafka 0.11 so we could leverage the new idempotent producer + transactions API. On the bro side, unless you know something I don't the Kafka plugin package is the one I'm working on getting released using the Metron code ( https://github.com/jonzeolla/

[GitHub] metron issue #667: METRON-1061 Add FUZZY_SCORE STELLAR function

2017-08-23 Thread nickwallen
Github user nickwallen commented on the issue: https://github.com/apache/metron/pull/667 Yes, I have some outstanding comments that Otto hasn't responded to yet. It is in the "fold" which is why they were probably missed. https://github.com/apache/metron/pull/667#discussion_r

Re: [DISCUSS] Synopsis of Community Meeting on 8/22/2017

2017-08-23 Thread Nick Allen
That is actually our plugin. Well, ours is theirs. The Bro Project is getting out of the business of maintaining plugins for the most part. This is one side effect of the new Bro packaging mechanism. The plan is to migrate the code to the new Bro packaging mechanism and for it to live here http

[GitHub] metron pull request #667: METRON-1061 Add FUZZY_SCORE STELLAR function

2017-08-23 Thread ottobackwards
Github user ottobackwards commented on a diff in the pull request: https://github.com/apache/metron/pull/667#discussion_r134747025 --- Diff: metron-stellar/stellar-common/src/main/java/org/apache/metron/stellar/dsl/functions/TextFunctions.java --- @@ -0,0 +1,63 @@ +/** + *

[GitHub] metron pull request #711: METRON-1127: Add ability to escalate alerts for ex...

2017-08-23 Thread merrimanr
GitHub user merrimanr opened a pull request: https://github.com/apache/metron/pull/711 METRON-1127: Add ability to escalate alerts for external consumption ## Contributor Comments This PR exposes a way to put alerts (or any message) into a configurable escalation Kafka topic. I

[GitHub] metron issue #707: METRON-1121 Ignore Profile with Bad 'init', 'update' or '...

2017-08-23 Thread cestella
Github user cestella commented on the issue: https://github.com/apache/metron/pull/707 +1 by inspection --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or

[GitHub] metron issue #706: METRON-1119 Bad Out-of-Order Logic for SaltyRowKeyBuilder

2017-08-23 Thread cestella
Github user cestella commented on the issue: https://github.com/apache/metron/pull/706 What's the consequence of this one? How does the error look like in the wild? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If you

[DISCUSS] METRON-777 and the road to perditi... er enlightenment

2017-08-23 Thread Otto Fowler
A feature branch As discussed in the community meeting we are going to create a ‘feature’ branch for METRON–777 and it’s associated PR’s. The purpose behind the feature branch is to more formally stage a large PR or set of PR’s to allow for for better review participation ( of the pr and changes d

[GitHub] metron pull request #707: METRON-1121 Ignore Profile with Bad 'init', 'updat...

2017-08-23 Thread cestella
Github user cestella commented on a diff in the pull request: https://github.com/apache/metron/pull/707#discussion_r134758762 --- Diff: metron-analytics/metron-profiler-common/src/main/java/org/apache/metron/profiler/DefaultProfileBuilder.java --- @@ -140,33 +144,41 @@ public void

[GitHub] metron pull request #707: METRON-1121 Ignore Profile with Bad 'init', 'updat...

2017-08-23 Thread cestella
Github user cestella commented on a diff in the pull request: https://github.com/apache/metron/pull/707#discussion_r134758959 --- Diff: metron-analytics/metron-profiler-common/src/main/java/org/apache/metron/profiler/DefaultProfileBuilder.java --- @@ -253,7 +265,8 @@ private void

[GitHub] metron pull request #707: METRON-1121 Ignore Profile with Bad 'init', 'updat...

2017-08-23 Thread cestella
Github user cestella commented on a diff in the pull request: https://github.com/apache/metron/pull/707#discussion_r134758686 --- Diff: metron-analytics/metron-profiler-common/src/main/java/org/apache/metron/profiler/DefaultProfileBuilder.java --- @@ -140,33 +144,41 @@ public void

[GitHub] metron pull request #705: METRON-1118 Ignore Profile with Bad 'onlyif' or 'f...

2017-08-23 Thread cestella
Github user cestella commented on a diff in the pull request: https://github.com/apache/metron/pull/705#discussion_r134758177 --- Diff: metron-analytics/metron-profiler-common/src/main/java/org/apache/metron/profiler/DefaultMessageRouter.java --- @@ -62,21 +68,45 @@ public Default

[GitHub] metron pull request #705: METRON-1118 Ignore Profile with Bad 'onlyif' or 'f...

2017-08-23 Thread cestella
Github user cestella commented on a diff in the pull request: https://github.com/apache/metron/pull/705#discussion_r134758511 --- Diff: metron-analytics/metron-profiler-common/src/main/java/org/apache/metron/profiler/DefaultMessageRouter.java --- @@ -62,21 +68,45 @@ public Default

[GitHub] metron issue #706: METRON-1119 Bad Out-of-Order Logic for SaltyRowKeyBuilder

2017-08-23 Thread nickwallen
Github user nickwallen commented on the issue: https://github.com/apache/metron/pull/706 This is a bug that would be hard to trigger. That is probably why it has been in the code base for quite some time. It took @mattf-horton 's sharp eyes to uncover it. This bug cannot be

Re: [DISCUSS] METRON-777 and the road to perditi... er enlightenment

2017-08-23 Thread Otto Fowler
WRT : regression fixes, I would also like us to consider putting these the initial 777 to feature branch PR as an option. On August 23, 2017 at 09:56:33, Otto Fowler (ottobackwa...@gmail.com) wrote: A feature branch As discussed in the community meeting we are going to create a ‘feature’ bran

[GitHub] metron issue #707: METRON-1121 Ignore Profile with Bad 'init', 'update' or '...

2017-08-23 Thread nickwallen
Github user nickwallen commented on the issue: https://github.com/apache/metron/pull/707 Good eye, @cestella . Latest commit is catching Throwable and logging e's. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] metron issue #705: METRON-1118 Ignore Profile with Bad 'onlyif' or 'foreach'...

2017-08-23 Thread nickwallen
Github user nickwallen commented on the issue: https://github.com/apache/metron/pull/705 Thanks @cestella for catching those. The latest commit fixes those items. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] metron pull request #667: METRON-1061 Add FUZZY_SCORE STELLAR function

2017-08-23 Thread nickwallen
Github user nickwallen commented on a diff in the pull request: https://github.com/apache/metron/pull/667#discussion_r134782723 --- Diff: metron-stellar/stellar-common/src/main/java/org/apache/metron/stellar/dsl/functions/TextFunctions.java --- @@ -0,0 +1,63 @@ +/** + * Li

[GitHub] metron pull request #667: METRON-1061 Add FUZZY_SCORE STELLAR function

2017-08-23 Thread ottobackwards
Github user ottobackwards commented on a diff in the pull request: https://github.com/apache/metron/pull/667#discussion_r134790594 --- Diff: metron-stellar/stellar-common/src/main/java/org/apache/metron/stellar/dsl/functions/TextFunctions.java --- @@ -0,0 +1,63 @@ +/** + *

[GitHub] metron issue #706: METRON-1119 Bad Out-of-Order Logic for SaltyRowKeyBuilder

2017-08-23 Thread mattf-horton
Github user mattf-horton commented on the issue: https://github.com/apache/metron/pull/706 +1, lgtm --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if t

[GitHub] metron issue #699: METRON-1068: Use Elastic end point in alerts ui

2017-08-23 Thread merrimanr
Github user merrimanr commented on the issue: https://github.com/apache/metron/pull/699 I have tested this in full dev and it works as advertised. I haven't noticed any obvious bugs and it was fairly easy to install and start. I would suggest we add a note in the "Development Setup"

[GitHub] metron pull request #712: Metron 1129

2017-08-23 Thread nickwallen
GitHub user nickwallen opened a pull request: https://github.com/apache/metron/pull/712 Metron 1129 If you install the Management UI using the RPM, it will not install all of the required Node dependencies. You have to manually run `npm install ...` to get those dependencies insta

[GitHub] metron pull request #712: Metron 1129

2017-08-23 Thread nickwallen
Github user nickwallen commented on a diff in the pull request: https://github.com/apache/metron/pull/712#discussion_r134798441 --- Diff: metron-deployment/packaging/ambari/metron-mpack/src/main/resources/common-services/METRON/CURRENT/package/scripts/management_ui_master.py --- @

[GitHub] metron pull request #712: Metron 1129

2017-08-23 Thread nickwallen
Github user nickwallen commented on a diff in the pull request: https://github.com/apache/metron/pull/712#discussion_r134800504 --- Diff: metron-interface/metron-config/README.md --- @@ -49,12 +49,6 @@ This module provides a user interface for management functions in Metron.

[GitHub] metron pull request #712: Metron 1129

2017-08-23 Thread nickwallen
Github user nickwallen commented on a diff in the pull request: https://github.com/apache/metron/pull/712#discussion_r134799112 --- Diff: metron-deployment/packaging/docker/rpm-docker/SPECS/metron.spec --- @@ -30,6 +30,8 @@ %define metron_root %{_prefix}/%{base_name}

[GitHub] metron pull request #712: Metron 1129

2017-08-23 Thread nickwallen
Github user nickwallen commented on a diff in the pull request: https://github.com/apache/metron/pull/712#discussion_r134799272 --- Diff: metron-deployment/packaging/docker/rpm-docker/SPECS/metron.spec --- @@ -91,6 +93,9 @@ tar -xzf %{SOURCE11} -C %{buildroot}%{metron_home} in

[GitHub] metron pull request #712: Metron 1129

2017-08-23 Thread nickwallen
Github user nickwallen commented on a diff in the pull request: https://github.com/apache/metron/pull/712#discussion_r134800405 --- Diff: metron-deployment/packaging/docker/rpm-docker/build.sh --- @@ -36,7 +36,7 @@ if [ $? -ne 0 ] && [ $OWNER_UID -ne 0 ]; then fi rm

[GitHub] metron pull request #712: Metron 1129

2017-08-23 Thread nickwallen
Github user nickwallen commented on a diff in the pull request: https://github.com/apache/metron/pull/712#discussion_r134798609 --- Diff: metron-deployment/packaging/docker/rpm-docker/Dockerfile --- @@ -27,3 +27,7 @@ RUN mv apache-maven-3.2.5 /opt/maven RUN ln -s /opt/maven/bin

[GitHub] metron pull request #646: METRON-1030: Metron JDBC platform REST option shou...

2017-08-23 Thread merrimanr
Github user merrimanr closed the pull request at: https://github.com/apache/metron/pull/646 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is e

[GitHub] metron pull request #712: METRON-1129 Management UI - Package Node Dependenc...

2017-08-23 Thread ottobackwards
Github user ottobackwards commented on a diff in the pull request: https://github.com/apache/metron/pull/712#discussion_r134802794 --- Diff: metron-deployment/packaging/docker/rpm-docker/Dockerfile --- @@ -27,3 +27,7 @@ RUN mv apache-maven-3.2.5 /opt/maven RUN ln -s /opt/maven/

[GitHub] metron pull request #712: METRON-1129 Management UI - Package Node Dependenc...

2017-08-23 Thread nickwallen
Github user nickwallen commented on a diff in the pull request: https://github.com/apache/metron/pull/712#discussion_r134804476 --- Diff: metron-deployment/packaging/docker/rpm-docker/Dockerfile --- @@ -27,3 +27,7 @@ RUN mv apache-maven-3.2.5 /opt/maven RUN ln -s /opt/maven/bin

Re: [DISCUSS] METRON-777 and the road to perditi... er enlightenment

2017-08-23 Thread Nick Allen
+1 I like it all, Otto. You deserve a freakin' medal. On Wed, Aug 23, 2017 at 10:04 AM Otto Fowler wrote: > WRT : regression fixes, I would also like us to consider putting these the > initial 777 to feature branch PR as an option. > > > On August 23, 2017 at 09:56:33, Otto Fowler (ottoback

[GitHub] metron pull request #714: METRON-1128: MAAS_GET_ENDPOINT - Unable to resolve...

2017-08-23 Thread cestella
GitHub user cestella opened a pull request: https://github.com/apache/metron/pull/714 METRON-1128: MAAS_GET_ENDPOINT - Unable to resolve function error seen for squid example ## Contributor Comments This is due to an artifact of the extraction of Stellar. We moved the location

[GitHub] metron pull request #713: METRON-1130 - Updating Amazon EC2 README.md

2017-08-23 Thread as22323
GitHub user as22323 opened a pull request: https://github.com/apache/metron/pull/713 METRON-1130 - Updating Amazon EC2 README.md ## Contributor Comments Hello, This is my first time submitting a pull request to the Apache Metron project. Feel free to point me in the right direct

[GitHub] metron pull request #712: METRON-1129 Management UI - Package Node Dependenc...

2017-08-23 Thread ottobackwards
Github user ottobackwards commented on a diff in the pull request: https://github.com/apache/metron/pull/712#discussion_r134829464 --- Diff: metron-deployment/packaging/docker/rpm-docker/Dockerfile --- @@ -27,3 +27,7 @@ RUN mv apache-maven-3.2.5 /opt/maven RUN ln -s /opt/maven/

Re: [DISCUSS] METRON-777 and the road to perditi... er enlightenment

2017-08-23 Thread zeo...@gmail.com
This is all great stuff. As far as feature branch naming, I would suggest something like feature/$brief_explanation accompanied with a feature branch JIRA that explains the original intent of the branch and its goals/"complete" indicators. Along the lines of the FEATURE.md, I feel like at the ver

Re: [DISCUSS] METRON-777 and the road to perditi... er enlightenment

2017-08-23 Thread Nick Allen
I propose that we go forward with our first feature branch without guideline changes up-front. We can use this as a learning experience out of which guideline changes can be proposed. I'd prefer to have a little more flexibility on our first feature branch so we can determine how best to push it

[GitHub] metron pull request #712: METRON-1129 Management UI - Package Node Dependenc...

2017-08-23 Thread nickwallen
Github user nickwallen commented on a diff in the pull request: https://github.com/apache/metron/pull/712#discussion_r134834099 --- Diff: metron-deployment/packaging/docker/rpm-docker/Dockerfile --- @@ -27,3 +27,7 @@ RUN mv apache-maven-3.2.5 /opt/maven RUN ln -s /opt/maven/bin

[GitHub] metron issue #713: METRON-1130 - Updating Amazon EC2 README.md

2017-08-23 Thread nickwallen
Github user nickwallen commented on the issue: https://github.com/apache/metron/pull/713 Hi @as22323 - Thanks for submitting a PR. You want the "base branch" of this PR to be "master". Right now it is set to "apache:Metron_0.4.0". --- If your project is set up for it, you can repl

[GitHub] metron issue #713: METRON-1130 - Updating Amazon EC2 README.md

2017-08-23 Thread as22323
Github user as22323 commented on the issue: https://github.com/apache/metron/pull/713 Thanks. I have updated to "master". According to the Metron Dev Guidelines I am suppose to assign the METRON JIRA ticket to my self. I'm not sure how to do that. --- If your project is s

[GitHub] metron pull request #677: METRON-1073: Add option to limit fields returned f...

2017-08-23 Thread mmiklavc
Github user mmiklavc commented on a diff in the pull request: https://github.com/apache/metron/pull/677#discussion_r134839995 --- Diff: metron-platform/metron-elasticsearch/src/main/java/org/apache/metron/elasticsearch/dao/ElasticsearchDao.java --- @@ -100,7 +107,16 @@ public Sear

[GitHub] metron pull request #677: METRON-1073: Add option to limit fields returned f...

2017-08-23 Thread merrimanr
Github user merrimanr commented on a diff in the pull request: https://github.com/apache/metron/pull/677#discussion_r134860066 --- Diff: metron-platform/metron-elasticsearch/src/main/java/org/apache/metron/elasticsearch/dao/ElasticsearchDao.java --- @@ -100,7 +107,16 @@ public Sea

Re: [ANNOUNCE] Metron community meeting

2017-08-23 Thread James Sirota
Hi Guys, apologies about this. I couldn't record yesterday, but Casey posted a synopsis of the meeting. 22.08.2017, 10:27, "James Sirota" : > Yes, I will post a recording > > 21.08.2017, 14:57, "Kyle Richardson" : >>  Unfortunately I have a conflict. Will there be a recording posted afterward

[GitHub] metron pull request #694: METRON-1085: Add REST endpoint to save a user prof...

2017-08-23 Thread mmiklavc
Github user mmiklavc commented on a diff in the pull request: https://github.com/apache/metron/pull/694#discussion_r134930645 --- Diff: metron-platform/metron-indexing/src/main/java/org/apache/metron/indexing/dao/search/SearchRequest.java --- @@ -106,4 +107,33 @@ public void setSo

[GitHub] metron pull request #694: METRON-1085: Add REST endpoint to save a user prof...

2017-08-23 Thread mmiklavc
Github user mmiklavc commented on a diff in the pull request: https://github.com/apache/metron/pull/694#discussion_r134926549 --- Diff: metron-interface/metron-rest/src/test/java/org/apache/metron/rest/controller/AlertsProfileControllerIntegrationTest.java --- @@ -0,0 +1,226 @@

[GitHub] metron pull request #694: METRON-1085: Add REST endpoint to save a user prof...

2017-08-23 Thread mmiklavc
Github user mmiklavc commented on a diff in the pull request: https://github.com/apache/metron/pull/694#discussion_r134926541 --- Diff: metron-interface/metron-rest/src/test/java/org/apache/metron/rest/controller/AlertsProfileControllerIntegrationTest.java --- @@ -0,0 +1,226 @@

[GitHub] metron pull request #694: METRON-1085: Add REST endpoint to save a user prof...

2017-08-23 Thread mmiklavc
Github user mmiklavc commented on a diff in the pull request: https://github.com/apache/metron/pull/694#discussion_r134927788 --- Diff: metron-interface/metron-rest/src/test/java/org/apache/metron/rest/controller/AlertsProfileControllerIntegrationTest.java --- @@ -0,0 +1,226 @@

[GitHub] metron pull request #694: METRON-1085: Add REST endpoint to save a user prof...

2017-08-23 Thread mmiklavc
Github user mmiklavc commented on a diff in the pull request: https://github.com/apache/metron/pull/694#discussion_r134927600 --- Diff: metron-interface/metron-rest/src/test/java/org/apache/metron/rest/controller/AlertsProfileControllerIntegrationTest.java --- @@ -0,0 +1,226 @@

[GitHub] metron pull request #694: METRON-1085: Add REST endpoint to save a user prof...

2017-08-23 Thread mmiklavc
Github user mmiklavc commented on a diff in the pull request: https://github.com/apache/metron/pull/694#discussion_r134930540 --- Diff: metron-interface/metron-rest/src/test/java/org/apache/metron/rest/service/impl/AlertsProfileServiceImplTest.java --- @@ -0,0 +1,127 @@ +/*

[GitHub] metron pull request #694: METRON-1085: Add REST endpoint to save a user prof...

2017-08-23 Thread mmiklavc
Github user mmiklavc commented on a diff in the pull request: https://github.com/apache/metron/pull/694#discussion_r134931179 --- Diff: metron-interface/metron-rest-client/src/main/java/org/apache/metron/rest/model/AlertsProfile.java --- @@ -0,0 +1,89 @@ +/** + * Licensed

[GitHub] metron pull request #694: METRON-1085: Add REST endpoint to save a user prof...

2017-08-23 Thread mmiklavc
Github user mmiklavc commented on a diff in the pull request: https://github.com/apache/metron/pull/694#discussion_r134924613 --- Diff: metron-interface/metron-rest-client/src/main/java/org/apache/metron/rest/converter/JsonConverter.java --- @@ -0,0 +1,53 @@ +/** + * Licen

[GitHub] metron pull request #694: METRON-1085: Add REST endpoint to save a user prof...

2017-08-23 Thread mmiklavc
Github user mmiklavc commented on a diff in the pull request: https://github.com/apache/metron/pull/694#discussion_r134924778 --- Diff: metron-interface/metron-rest-client/src/main/java/org/apache/metron/rest/model/AlertsProfile.java --- @@ -0,0 +1,89 @@ +/** + * Licensed