Re: How to clean Metro 0.3.1 quick-dev-platform box ?

Sorry about the brevity, answering on my phone. 1. By default the data comes from the sensor-stubs. For more details see https://github.com/apache/incubator-metron/blob/master/metron-deployment/roles/sensor-stubs/README.md 2. The vagrant details come from these two locations -

Re: [GitHub] incubator-metron pull request #503: METRON-815 sensor-stubs sometimes send m...

Bro timestamps are often out of order depending on the log because some lines are written when the connection ends and others are written when an event within a connection occurs. As such, timestamps can be confusing to look at initially, but it is very normal for them not to be in order. Also,

Podling Report Reminder - April 2017

Dear podling, This email was sent by an automated system on behalf of the Apache Incubator PMC. It is an initial reminder to give you plenty of time to prepare your quarterly board report. The board meeting is scheduled for Wed, 19 April 2017, 10:30 am PDT. The report for your podling will form

[GitHub] incubator-metron pull request #502: METRON-814 minor tweaks in document form...

Github user asfgit closed the pull request at: https://github.com/apache/incubator-metron/pull/502 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] incubator-metron pull request #406: METRON-642 Correct path to ES gc log fil...

Github user asfgit closed the pull request at: https://github.com/apache/incubator-metron/pull/406 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

Re: Kerberos changes affected quick-dev and full-dev

I've pushed a new Vagrant image for Quick Dev. You should be asked to update the box the next time you 'vagrant up' Quick Dev. -D... On Mon, Apr 3, 2017 at 2:33 PM, Casey Stella wrote: > Thanks Justin, the packer build is started, but this is going to take some > time.

[GitHub] incubator-metron issue #406: METRON-642 Correct path to ES gc log file

Github user mattf-horton commented on the issue: https://github.com/apache/incubator-metron/pull/406 @DimDroll - Dima, sorry it took so long to get this in. It's still needed, so I've merged it. Thanks for the contribution! --- If your project is set up for it, you can reply to

[GitHub] incubator-metron issue #499: METRON-196: Deployment Fails Without Ansible 2....

Github user JonZeolla commented on the issue: https://github.com/apache/incubator-metron/pull/499 Now that master is cleaned up I gave this a shot under 2.2.2.0 and 2.0.0.2. Looks good to me - spun up full dev on both, checked kibana, etc. +1 --- If your project is set up

Re: [DISCUSS] The bro kafka plugin

Can someone on the PMC submit a ticket to INFRA? It looks like committers aren't supposed to. Jon On Fri, Mar 31, 2017 at 4:23 PM zeo...@gmail.com wrote: > I would be happy to try it again but I attempted to do that before with >

[GitHub] incubator-metron pull request #489: METRON-623: Management UI

Github user iraghumitra commented on a diff in the pull request: https://github.com/apache/incubator-metron/pull/489#discussion_r109429428 --- Diff: metron-interface/metron-config/src/app/_main.scss --- @@ -0,0 +1,113 @@ +/** + * Licensed to the Apache Software Foundation

[GitHub] incubator-metron issue #502: METRON-814 minor tweaks in document format of K...

Github user mmiklavc commented on the issue: https://github.com/apache/incubator-metron/pull/502 +1, looks good @mattf-horton. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] incubator-metron issue #506: METRON-818: Ambari elasticsearch.properties tem...

Github user mmiklavc commented on the issue: https://github.com/apache/incubator-metron/pull/506 +1 by inspection. Same thing we're doing with enrichment and must have missed in the indexing configs. Do we need to add this for Solr? --- If your project is set up for it, you can

[GitHub] incubator-metron pull request #505: METRON-817: Customise output file path p...

Github user ottobackwards commented on a diff in the pull request: https://github.com/apache/incubator-metron/pull/505#discussion_r109428569 --- Diff: metron-platform/metron-writer/src/main/java/org/apache/metron/writer/hdfs/HdfsWriter.java --- @@ -74,17 +91,43 @@ public

[GitHub] incubator-metron pull request #505: METRON-817: Customise output file path p...

Github user justinleet commented on a diff in the pull request: https://github.com/apache/incubator-metron/pull/505#discussion_r109432502 --- Diff: metron-platform/metron-writer/src/main/java/org/apache/metron/writer/hdfs/HdfsWriter.java --- @@ -74,17 +91,43 @@ public

[GitHub] incubator-metron pull request #505: METRON-817: Customise output file path p...

Github user cestella commented on a diff in the pull request: https://github.com/apache/incubator-metron/pull/505#discussion_r109441372 --- Diff: metron-platform/metron-writer/src/main/java/org/apache/metron/writer/hdfs/HdfsWriter.java --- @@ -74,17 +91,43 @@ public

[GitHub] incubator-metron issue #506: METRON-818: Ambari elasticsearch.properties tem...

Github user justinleet commented on the issue: https://github.com/apache/incubator-metron/pull/506 @mmiklavc We probably should edit the Solr config. That isn't carried through Ambari, so we don't have the same concern as here. However, it does look like `storm.auto.credentials=[]`

[GitHub] incubator-metron issue #506: METRON-818: Ambari elasticsearch.properties tem...

Github user cestella commented on the issue: https://github.com/apache/incubator-metron/pull/506 Also +1 by inspection --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] incubator-metron issue #506: METRON-818: Ambari elasticsearch.properties tem...

Github user cestella commented on the issue: https://github.com/apache/incubator-metron/pull/506 I'd argue for a best effort fix for solr and that you should just drop that config and add topology.worker.childopts. --- If your project is set up for it, you can reply to this email

[GitHub] incubator-metron issue #506: METRON-818: Ambari elasticsearch.properties tem...

Github user JonZeolla commented on the issue: https://github.com/apache/incubator-metron/pull/506 Spinning this up now. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] incubator-metron pull request #505: METRON-817: Customise output file path p...

Github user ottobackwards commented on a diff in the pull request: https://github.com/apache/incubator-metron/pull/505#discussion_r109428017 --- Diff: metron-platform/metron-writer/src/main/java/org/apache/metron/writer/hdfs/HdfsWriter.java --- @@ -74,17 +91,43 @@ public

[GitHub] incubator-metron pull request #489: METRON-623: Management UI

Github user iraghumitra commented on a diff in the pull request: https://github.com/apache/incubator-metron/pull/489#discussion_r109430301 --- Diff: metron-interface/metron-config/src/app/app.component.ts --- @@ -0,0 +1,41 @@ +/** + * Licensed to the Apache Software

[GitHub] incubator-metron issue #506: METRON-818: Ambari elasticsearch.properties tem...

Github user justinleet commented on the issue: https://github.com/apache/incubator-metron/pull/506 Full dev spun up and ran fine, and I see items showing up in ES and HDFS --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well.

Re: Kerberos changes affected quick-dev and full-dev

Btw, here is a workaround for full-dev. In Ambari, add the line "topology.worker.childopts=" (no argument) to the elasticsearch.properties template, then restart indexing through Ambari to propogate the change out. For example, make the Storm section look like: # Storm #

[GitHub] incubator-metron pull request #505: METRON-817: Customise output file path p...

GitHub user justinleet opened a pull request: https://github.com/apache/incubator-metron/pull/505 METRON-817: Customise output file path patterns for HDFS indexing ## Contributor Comments Primarily this affects HdfsWriter by changing the output path from a set path

[GitHub] incubator-metron pull request #506: METRON-818: Ambari elasticsearch.propert...

GitHub user justinleet opened a pull request: https://github.com/apache/incubator-metron/pull/506 METRON-818: Ambari elasticsearch.properties template is missing topology.worker.childopts ## Contributor Comments Adding the empty config to the Ambari elasticsearch.properties

[GitHub] incubator-metron pull request #501: METRON-812: Make the bro-kafka plugin wo...

Github user asfgit closed the pull request at: https://github.com/apache/incubator-metron/pull/501 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] incubator-metron issue #499: METRON-196: Deployment Fails Without Ansible 2....

Github user JonZeolla commented on the issue: https://github.com/apache/incubator-metron/pull/499 Looks like this is fixed in #506 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] incubator-metron pull request #489: METRON-623: Management UI

Github user iraghumitra commented on a diff in the pull request: https://github.com/apache/incubator-metron/pull/489#discussion_r109430089 --- Diff: metron-interface/metron-config/src/app/app.component.ts --- @@ -0,0 +1,41 @@ +/** + * Licensed to the Apache Software

[GitHub] incubator-metron pull request #505: METRON-817: Customise output file path p...

Github user justinleet commented on a diff in the pull request: https://github.com/apache/incubator-metron/pull/505#discussion_r109438625 --- Diff: metron-platform/metron-writer/src/main/java/org/apache/metron/writer/hdfs/HdfsWriter.java --- @@ -74,17 +91,43 @@ public

[GitHub] incubator-metron pull request #503: METRON-815 sensor-stubs sometimes send m...

Github user JonZeolla commented on a diff in the pull request: https://github.com/apache/incubator-metron/pull/503#discussion_r109411418 --- Diff: metron-deployment/roles/sensor-stubs/templates/start-bro-stub --- @@ -47,8 +47,8 @@ TOPIC="bro" while true; do #

[GitHub] incubator-metron pull request #489: METRON-623: Management UI

Github user merrimanr commented on a diff in the pull request: https://github.com/apache/incubator-metron/pull/489#discussion_r109464805 --- Diff: metron-interface/metron-config/README.md --- @@ -0,0 +1,67 @@ +# Metron Management UI + +This module provides a user

[GitHub] incubator-metron pull request #503: METRON-815 sensor-stubs sometimes send m...

Github user mattf-horton commented on a diff in the pull request: https://github.com/apache/incubator-metron/pull/503#discussion_r109444746 --- Diff: metron-deployment/roles/sensor-stubs/templates/start-bro-stub --- @@ -47,8 +47,8 @@ TOPIC="bro" while true; do #

[GitHub] incubator-metron pull request #508: METRON-820: StellarProcessor should have...

GitHub user cestella opened a pull request: https://github.com/apache/incubator-metron/pull/508 METRON-820: StellarProcessor should have a static expression cache ## Contributor Comments We recreate StellarProcessor objects all over the place which do not reuse their expression

[GitHub] incubator-metron pull request #505: METRON-817: Customise output file path p...

Github user justinleet commented on a diff in the pull request: https://github.com/apache/incubator-metron/pull/505#discussion_r109447598 --- Diff: metron-platform/metron-writer/src/main/java/org/apache/metron/writer/hdfs/HdfsWriter.java --- @@ -74,17 +91,43 @@ public

[GitHub] incubator-metron pull request #507: METRON-819: Document kafka console produ...

GitHub user mmiklavc opened a pull request: https://github.com/apache/incubator-metron/pull/507 METRON-819: Document kafka console producer parameter for sensors with kerberos Addresses https://issues.apache.org/jira/browse/METRON-819 Adds a note about adding the security

[GitHub] incubator-metron pull request #505: METRON-817: Customise output file path p...

Github user cestella commented on a diff in the pull request: https://github.com/apache/incubator-metron/pull/505#discussion_r109467330 --- Diff: metron-platform/metron-writer/src/main/java/org/apache/metron/writer/hdfs/HdfsWriter.java --- @@ -74,17 +91,43 @@ public

[GitHub] incubator-metron pull request #489: METRON-623: Management UI

Github user merrimanr commented on a diff in the pull request: https://github.com/apache/incubator-metron/pull/489#discussion_r109464464 --- Diff: metron-interface/metron-config/README.md --- @@ -0,0 +1,67 @@ +# Metron Management UI + +This module provides a user

[GitHub] incubator-metron pull request #503: METRON-815 sensor-stubs sometimes send m...

Github user asfgit closed the pull request at: https://github.com/apache/incubator-metron/pull/503 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] incubator-metron issue #506: METRON-818: Ambari elasticsearch.properties tem...

Github user justinleet commented on the issue: https://github.com/apache/incubator-metron/pull/506 @JonZeolla did you have any issues spinning this up? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] incubator-metron issue #506: METRON-818: Ambari elasticsearch.properties tem...

Github user JonZeolla commented on the issue: https://github.com/apache/incubator-metron/pull/506 I ran into a few issues related to my environment (surprise - if you don't have a network it's hard to do much) but I did end up getting full-dev up and functional once that was sorted

[GitHub] incubator-metron pull request #506: METRON-818: Ambari elasticsearch.propert...

Github user asfgit closed the pull request at: https://github.com/apache/incubator-metron/pull/506 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] incubator-metron issue #506: METRON-818: Ambari elasticsearch.properties tem...

Github user justinleet commented on the issue: https://github.com/apache/incubator-metron/pull/506 Thanks for running it up and verifying it. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] incubator-metron issue #508: METRON-820: StellarProcessor should have a stat...

Github user cestella commented on the issue: https://github.com/apache/incubator-metron/pull/508 @ottobackwards It should not. When we start being able to handle people dropping new jars in HDFS on a *running* topology, then we will need to prune this cache. As of now, though, a

Re: Kerberos changes affected quick-dev and full-dev

Thanks Justin, the packer build is started, but this is going to take some time. Please use full-dev to validate your PRs in the meantime. I will update this thread once it's uploaded. On Mon, Apr 3, 2017 at 2:13 PM, Justin Leet wrote: > The PR to fix full-dev is in

[GitHub] incubator-metron issue #508: METRON-820: StellarProcessor should have a stat...

Github user ottobackwards commented on the issue: https://github.com/apache/incubator-metron/pull/508 1 question -> does this require any relation to the VFS cache? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] incubator-metron issue #508: METRON-820: StellarProcessor should have a stat...

Github user ottobackwards commented on the issue: https://github.com/apache/incubator-metron/pull/508 +1 by inspection, although I'm inclined to think of a reason not to have a static cache, but instead an instance based one, I can't think of a specific case. --- If your project

Re: Kerberos changes affected quick-dev and full-dev

The PR to fix full-dev is in master now. We still need a new packer build before we have quick-dev available again. Justin On Mon, Apr 3, 2017 at 10:53 AM, Justin Leet wrote: > Btw, here is a workaround for full-dev. In Ambari, add the line >

Journey out of the Incubator (update)

Hi All, For those of you who aren't following the discussion and vote, the incubator general, after a vigorous discussion, voted to approve recommending Metron to become a top level project with all +1s. I will next submit the resolution to the apache board. For those following along, the

Re: Journey out of the Incubator (update)

For reference and to save people the ponymail searches: * The vote thread was at https://lists.apache.org/thread.html/0aa75e8ffb0cd5a0446474f82ff4227ddacfbb8f1e84c442934bfabe@%3Cgeneral.incubator.apache.org%3E * The discussion thread was at

Re: Journey out of the Incubator (update)

That's great news. Thanks, Casey! On Mon, Apr 3, 2017 at 6:40 PM, Casey Stella wrote: > For reference and to save people the ponymail searches: > * The vote thread was at https://lists.apache.org/thread.html/ > 0aa75e8ffb0cd5a0446474f82ff4227ddacfbb8f1e84c442934bfabe@% >