Re: metron UI

2017-04-25 Thread Raghu Mitra Kandikonda
Kevin, HTML5/d3 is a great choice for any charting/dashboard kind of requirements but it might be a retro fit for a ui that has predominantly tables and forms. Currently metron-ui is designed only to manage parser configurations. We can still use d3 for our charting needs. Do you see any parti

Re: metron UI

2017-04-25 Thread moshe jarusalem
I may reserve some time to put some effort to your project. But first, need to clarify some points; I have seen alert interface at the following link is it separate project from yours ? http://imgur.com/a/KMTKN If you may tell us work you need support for the project I will consider if I can help

Re: [DISCUSS] Regression introduced in Full Dev

2017-04-25 Thread Otto Fowler
I *think* I have a fix, I have to try to test again tomorrow, my vagrant up worked though. I had to put the rpm copy executions inside the profiles ( so they get merged in order ). On April 25, 2017 at 22:44:13, Otto Fowler (ottobackwa...@gmail.com) wrote: http://stackoverflow.com/questions/221

Re: [DISCUSS] Regression introduced in Full Dev

2017-04-25 Thread Otto Fowler
http://stackoverflow.com/questions/22150209/maven-changes-the-order-of-plugins-of-different-profiles https://issues.apache.org/jira/browse/MNG-2258 On April 25, 2017 at 22:33:58, Otto Fowler (ottobackwa...@gmail.com) wrote: So, the copy resources configuration is still there, and not governed by

Re: [DISCUSS] Regression introduced in Full Dev

2017-04-25 Thread Otto Fowler
I think the copy resources should run in the post-package phase, to explicitly set the order. Does that make sense to anyone else? On April 25, 2017 at 22:33:58, Otto Fowler (ottobackwa...@gmail.com) wrote: So, the copy resources configuration is still there, and not governed by the profile. Is

Re: [DISCUSS] Regression introduced in Full Dev

2017-04-25 Thread Otto Fowler
So, the copy resources configuration is still there, and not governed by the profile. Is it possible that since they both run in package, that they are out of order? On April 25, 2017 at 22:31:27, Otto Fowler (ottobackwa...@gmail.com) wrote: Having said that, i see the problem. I am very sorry

Re: [DISCUSS] Regression introduced in Full Dev

2017-04-25 Thread Otto Fowler
Having said that, i see the problem. I am very sorry How do I go about resubmitting if I can fix it? On April 25, 2017 at 22:29:56, Otto Fowler (ottobackwa...@gmail.com) wrote: OK, I’ll take a look. Sorry - I thought they read out of the RPMS dir and verified that they were produced both way

Re: [DISCUSS] Regression introduced in Full Dev

2017-04-25 Thread Otto Fowler
OK, I’ll take a look. Sorry - I thought they read out of the RPMS dir and verified that they were produced both ways. I am not sure how my change would change the target copy. On April 25, 2017 at 22:25:04, Justin Leet (justinjl...@gmail.com) wrote: The problem seems to be that full/quick dev r

Re: [DISCUSS] Regression introduced in Full Dev

2017-04-25 Thread Justin Leet
The problem seems to be that full/quick dev rely on having the RPMs in the /target folder >From metron-deployment/roles/metron-rpms/defaults: metron_rpm_glob: "{{ playbook_dir }}/../packaging/docker/rpm-docker/target/RPMS/noarch/*.rpm" However, that PR appears to no longer copy to the /target fol

[GitHub] incubator-metron pull request #550: METRON-890: Intermittent unit test error...

2017-04-25 Thread cestella
GitHub user cestella reopened a pull request: https://github.com/apache/incubator-metron/pull/550 METRON-890: Intermittent unit test errors in shutting down Storm in memory component ## Contributor Comments Cross your fingers. This may or may not work. Please don't merge until

[GitHub] incubator-metron pull request #550: METRON-890: Intermittent unit test error...

2017-04-25 Thread cestella
Github user cestella closed the pull request at: https://github.com/apache/incubator-metron/pull/550 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the fea

[GitHub] incubator-metron pull request #551: METRON-892 add docker to platform info

2017-04-25 Thread ottobackwards
GitHub user ottobackwards opened a pull request: https://github.com/apache/incubator-metron/pull/551 METRON-892 add docker to platform info ## Contributor Comments Now that docker is a requirement, we need to include the version in the platform info script Tested on mac

Re: auto-install on bare metal

2017-04-25 Thread Otto Fowler
I failed at this today, but maybe it was the way I tried. An example would be great. On April 25, 2017 at 20:11:26, David Lyle (dlyle65...@gmail.com) wrote: Hi Dima, The same Ansible playbooks that work for EC2 and Vagrant will work for bare metal installations. The only difference is that you

[GitHub] incubator-metron pull request #550: METRON-890: Intermittent unit test error...

2017-04-25 Thread cestella
GitHub user cestella reopened a pull request: https://github.com/apache/incubator-metron/pull/550 METRON-890: Intermittent unit test errors in shutting down Storm in memory component ## Contributor Comments Cross your fingers. This may or may not work. Please don't merge until

[GitHub] incubator-metron pull request #550: METRON-890: Intermittent unit test error...

2017-04-25 Thread cestella
Github user cestella closed the pull request at: https://github.com/apache/incubator-metron/pull/550 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the fea

[GitHub] incubator-metron pull request #542: METRON-873: Stellar string literals do n...

2017-04-25 Thread cestella
Github user cestella commented on a diff in the pull request: https://github.com/apache/incubator-metron/pull/542#discussion_r113348853 --- Diff: metron-platform/metron-common/src/main/java/org/apache/metron/common/stellar/StellarCompiler.java --- @@ -263,7 +263,56 @@ public void

Re: [DISCUSS] Regression introduced in Full Dev

2017-04-25 Thread Otto Fowler
Ryan, did you see why it was not working? On April 25, 2017 at 18:17:29, Ryan Merriman (merrim...@gmail.com) wrote: A regression was introduced recently that breaks full dev. I've narrowed down the commit that introduced it and have submitted a PR to revert that commit: https://github.com/apache

[GitHub] incubator-metron pull request #542: METRON-873: Stellar string literals do n...

2017-04-25 Thread cestella
Github user cestella commented on a diff in the pull request: https://github.com/apache/incubator-metron/pull/542#discussion_r113347347 --- Diff: metron-platform/metron-common/src/main/java/org/apache/metron/common/stellar/StellarCompiler.java --- @@ -263,7 +263,56 @@ public void

Re: auto-install on bare metal

2017-04-25 Thread David Lyle
Hi Dima, The same Ansible playbooks that work for EC2 and Vagrant will work for bare metal installations. The only difference is that you would need to pre-provision your machines and hand-build your inventory file. The AWS playbooks only provision the machines. All deployment of Metron is handled

[GitHub] incubator-metron pull request #550: METRON-890: Intermittent unit test error...

2017-04-25 Thread cestella
Github user cestella closed the pull request at: https://github.com/apache/incubator-metron/pull/550 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the fea

[GitHub] incubator-metron pull request #550: METRON-890: Intermittent unit test error...

2017-04-25 Thread cestella
GitHub user cestella reopened a pull request: https://github.com/apache/incubator-metron/pull/550 METRON-890: Intermittent unit test errors in shutting down Storm in memory component ## Contributor Comments Cross your fingers. This may or may not work. Please don't merge until

[GitHub] incubator-metron issue #550: METRON-890: Intermittent unit test errors in sh...

2017-04-25 Thread cestella
Github user cestella commented on the issue: https://github.com/apache/incubator-metron/pull/550 yep, that's the belief. We're going to test it out here, in front of everyone. :) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub a

Re: [DISCUSS] Regression introduced in Full Dev

2017-04-25 Thread Casey Stella
Ok, I spun up that discussion in another thread. Hopefully we can get some better sense about the various ways to spin up metron and a centralized place to direct people to along with with guidance on when some approach would be better than another. I'll be honest, I've totally lost track and nev

[DISCUSS] The various methods and incantations to spin up Metron

2017-04-25 Thread Casey Stella
On another discuss thread, it was requested that we have some guidance explicitly created to document the various methods and incantations to spin up Metron. I want to start this discussion here and the result, I propose, will be placed in the top level README.md under a heading titled "Ways to St

RE: metron UI

2017-04-25 Thread Waterson, Kevin
Of course, I should disclaim, this is my own project, and not a Telstra related initiative. Kev -Original Message- From: Waterson, Kevin [mailto:kevin.water...@team.telstra.com] Sent: Wednesday, 26 April 2017 8:37 AM To: dev@metron.apache.org Subject: RE: metron UI HTML5/D3 Kev -

[GitHub] incubator-metron pull request #545: METRON-883 Capture Bro Plugin Enhancemen...

2017-04-25 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/incubator-metron/pull/545 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the featu

auto-install on bare metal

2017-04-25 Thread Dima Kovalyov
Hello Metron Team, We have developed a script that performs auto-install of the Metron on bare metal machines, but still working on few issues here and there. I am curios as to what automate solutions we do have for Metron installation right now? The ones I am aware of are in https://github.com/a

Re: [DISCUSS] Regression introduced in Full Dev

2017-04-25 Thread Casey Stella
Yeah, I tend to agree that a rundown of the various methods and when you would use them is in order. I will say that full-dev is especially important to have working since it is required for validating PRs. On Tue, Apr 25, 2017 at 18:56 zeo...@gmail.com wrote: > Can somebody map out all of the cu

Re: [DISCUSS] Regression introduced in Full Dev

2017-04-25 Thread zeo...@gmail.com
Can somebody map out all of the current methods and procedures to spin up Metron components? I swear I find out about new ones every month. Metron-docker, the 4 vagrants, rpm-docker, ansible-docker, any others? Perhaps an agreed upon write up of when to use what would be helpful. Jon On Tue, Apr

RE: metron UI

2017-04-25 Thread Waterson, Kevin
HTML5/D3 Kev -Original Message- From: Ryan Merriman [mailto:merrim...@gmail.com] Sent: Wednesday, 26 April 2017 8:24 AM To: dev@metron.apache.org Subject: Re: metron UI Kevin, Would be interested in learning more about the issues you have with Angular. Which technology would you pref

[GitHub] incubator-metron issue #550: METRON-890: Intermittent unit test errors in sh...

2017-04-25 Thread mmiklavc
Github user mmiklavc commented on the issue: https://github.com/apache/incubator-metron/pull/550 Is the belief here that killing the topologies first might allow full cluster shutdown and cleanup? --- If your project is set up for it, you can reply to this email and have your reply a

[GitHub] incubator-metron issue #548: METRON-888: Update README to show requirement o...

2017-04-25 Thread mmiklavc
Github user mmiklavc commented on the issue: https://github.com/apache/incubator-metron/pull/548 +1 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if t

Re: metron UI

2017-04-25 Thread Ryan Merriman
Kevin, Would be interested in learning more about the issues you have with Angular. Which technology would you prefer? Ryan On Tue, Apr 25, 2017 at 5:16 PM, Waterson, Kevin < kevin.water...@team.telstra.com> wrote: > The use of Angular has me writing my own UI for Metron > > Kev > > -Origi

RE: metron UI

2017-04-25 Thread Waterson, Kevin
The use of Angular has me writing my own UI for Metron Kev -Original Message- From: Justin Leet [mailto:justinjl...@gmail.com] Sent: Tuesday, 25 April 2017 12:19 AM To: dev@metron.apache.org Subject: Re: metron UI To elaborate on Ryan's reply a bit, the UI is a fairly new component in

[DISCUSS] Regression introduced in Full Dev

2017-04-25 Thread Ryan Merriman
A regression was introduced recently that breaks full dev. I've narrowed down the commit that introduced it and have submitted a PR to revert that commit: https://github.com/apache/incubator-metron/pull/549. Given there has been confusion recently over our deployment build process, I think it's

[GitHub] incubator-metron pull request #550: METRON-890: Intermittent unit test error...

2017-04-25 Thread cestella
GitHub user cestella opened a pull request: https://github.com/apache/incubator-metron/pull/550 METRON-890: Intermittent unit test errors in shutting down Storm in memory component ## Contributor Comments Cross your fingers. This may or may not work. Please don't merge until

[GitHub] incubator-metron pull request #549: METRON-889: Regression introduced in Ful...

2017-04-25 Thread merrimanr
GitHub user merrimanr opened a pull request: https://github.com/apache/incubator-metron/pull/549 METRON-889: Regression introduced in Full Dev ## Contributor Comments Full dev is currently broken because the RPM files are not being copied to vagrant. I've determined the regress

Re: [DISCUSS] Storm topology.classpath setting

2017-04-25 Thread Michael Miklavcic
Wait, which restart isn't required? Justin and I just stepped through this and we were getting errors with the Maxmind DB without restarting the enrichment topology. The reason was that the topology had null for the topology.classpath until I restarted it. On Tue, Apr 25, 2017 at 3:21 PM, Michael

Re: [DISCUSS] Storm topology.classpath setting

2017-04-25 Thread Michael Miklavcic
Yeah, I had originally added that feature a while back in https://issues.apache.org/jira/browse/METRON-356. But strictly speaking I think it should be able to be specified at the topology level per the Storm docs, https://github.com/apache/storm/blob/master/conf/defaults.yaml#L224 On Tue, Apr 25,

Re: [DISCUSS] Storm topology.classpath setting

2017-04-25 Thread Justin Leet
topology.classpath is a topology level config, so Mike's potential alternative is to migrate that down from Ambari into the actual topologies we launch (assuming it works as expected). Basically we're trading ease of use in Ambari vs. additional setup for each topology that's run. I'm in favor of

Re: [DISCUSS] Storm topology.classpath setting

2017-04-25 Thread Casey Stella
I don't know that I'm offended by either approach, but the way we have it now isn't offending me overly, honestly. Casey On Tue, Apr 25, 2017 at 5:11 PM, David Lyle wrote: > I think you have one more restart than is strictly required but I get your > point. IIRC, when I did the original impleme

Re: [DISCUSS] Storm topology.classpath setting

2017-04-25 Thread David Lyle
I think you have one more restart than is strictly required but I get your point. IIRC, when I did the original implementation there was a bit of a wrinkle getting the right configs on the right classpath so I punted in favor of topology.classpath. If you've got a more efficient way, I'm all for i

[DISCUSS] Storm topology.classpath setting

2017-04-25 Thread Michael Miklavcic
We currently define topology.classpath at the global level for Storm. The reason for this feature was to enable setting the classpath to include things like hbase-site and core-site without modifying the topology jar files on the cluster. That made things simpler, but this also has the consequence

Re: Ambari Wizard: Repo Tab

2017-04-25 Thread Otto Fowler
Nm. sorry. I fixed it. On April 25, 2017 at 16:42:05, Otto Fowler (ottobackwa...@gmail.com) wrote: Ok, now I see the repos in the ‘pick version’ screen, but it is erring on the f://localrepo even though the folder exists, there is no repodata/repomd.xml. What is the command to create a local

Re: Ambari Wizard: Repo Tab

2017-04-25 Thread Nick Allen
createrepo On Tue, Apr 25, 2017 at 4:42 PM, Otto Fowler wrote: > Ok, now I see the repos in the ‘pick version’ screen, but it is erring on > the f://localrepo > even though the folder exists, there is no repodata/repomd.xml. > > What is the command to create a local repo? > > > > On April 25, 20

Re: Ambari Wizard: Repo Tab

2017-04-25 Thread Otto Fowler
Ok, now I see the repos in the ‘pick version’ screen, but it is erring on the f://localrepo even though the folder exists, there is no repodata/repomd.xml. What is the command to create a local repo? On April 25, 2017 at 16:05:17, Otto Fowler (ottobackwa...@gmail.com) wrote: I was going by the

[GitHub] incubator-metron issue #541: METRON-870: Add filtering by packet payload to ...

2017-04-25 Thread cestella
n a fixed pcap query by executing a command similar to the following: ``` $METRON_HOME/bin/pcap_query.sh fixed --ip_dst_port 8080 -st "20170425" -df "MMdd" ``` * Verify the MR job finishes successfully. Upon completion, you should see multiple files nam

[GitHub] incubator-metron issue #548: METRON-888: Update README to show requirement o...

2017-04-25 Thread ottobackwards
Github user ottobackwards commented on the issue: https://github.com/apache/incubator-metron/pull/548 +1 by inspection --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] incubator-metron issue #524: METRON-836 Use Pycapa with Kerberos

2017-04-25 Thread nickwallen
Github user nickwallen commented on the issue: https://github.com/apache/incubator-metron/pull/524 I updated the validation instructions to setup a Kerberized environment and then validate Pycapa against that environment. --- If your project is set up for it, you can reply to this em

[GitHub] incubator-metron pull request #548: METRON-888: Update README to show requir...

2017-04-25 Thread dlyle65535
GitHub user dlyle65535 opened a pull request: https://github.com/apache/incubator-metron/pull/548 METRON-888: Update README to show requirement of Ambari 2.4.2 ## Contributor Comments Quick fix to reflect Ambari 2.4.2+ requirement. ## Pull Request Checklist Than

Re: Quick Dev - Atlas Images

2017-04-25 Thread Otto Fowler
If it is docker, then it is just the Dockerfiles. I think you can have vagrant use docker as a back end too right? On April 25, 2017 at 14:34:14, Nick Allen (n...@nickallen.org) wrote: >> I hadn't really reasoned about the notion of a "released" Quick Dev image, but I can see a lot of value in

Re: Ambari Wizard: Repo Tab

2017-04-25 Thread Otto Fowler
I was going by the HW community page. Ok, Let me try it On April 25, 2017 at 16:04:07, David Lyle (dlyle65...@gmail.com) wrote: That would do it. It requires 2.4.2+. I would have sworn I put that in the README, but I must have only annotated the PR. :( I'll get that in asap. -D... On Tue,

Re: Ambari Wizard: Repo Tab

2017-04-25 Thread David Lyle
That would do it. It requires 2.4.2+. I would have sworn I put that in the README, but I must have only annotated the PR. :( I'll get that in asap. -D... On Tue, Apr 25, 2017 at 3:44 PM, Otto Fowler wrote: > 2.4.1 as a matter of fact > > > On April 25, 2017 at 15:29:43, David Lyle (dlyle65...

[GitHub] incubator-metron issue #531: METRON-854 create dhcp dump parser

2017-04-25 Thread JonZeolla
Github user JonZeolla commented on the issue: https://github.com/apache/incubator-metron/pull/531 I would love to see Metron have a solution for both approaches - ingesting DHCP server logs, as well as DHCP observations based on network traffic. Like @ottobackwards mentioned, not eve

[GitHub] incubator-metron issue #535: METRON-859: Use REST application with Kerberos

2017-04-25 Thread merrimanr
Github user merrimanr commented on the issue: https://github.com/apache/incubator-metron/pull/535 It would take some work but I think it could be done. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

Re: Quick Dev - Atlas Images

2017-04-25 Thread Michael Miklavcic
I may have missed a discuss thread on this, but it looks like we are no longer using -SNAPSHOT in our current dev master https://github.com/apache/incubator-metron/blob/master/pom.xml#L19 On Tue, Apr 25, 2017 at 12:34 PM, Nick Allen wrote: > >> I hadn't really reasoned about the notion of a "re

Re: Ambari Wizard: Repo Tab

2017-04-25 Thread Otto Fowler
2.4.1 as a matter of fact On April 25, 2017 at 15:29:43, David Lyle (dlyle65...@gmail.com) wrote: Any chance they're running Ambari < 2.4.2? -D... On Tue, Apr 25, 2017 at 3:23 PM, Michael Miklavcic < michael.miklav...@gmail.com> wrote: > Hey Otto, I don't have wizard screenshots in front of

[GitHub] incubator-metron pull request #547: METRON-858 bro-plugin-kafka is throwing ...

2017-04-25 Thread JonZeolla
GitHub user JonZeolla opened a pull request: https://github.com/apache/incubator-metron/pull/547 METRON-858 bro-plugin-kafka is throwing segfaults ## Contributor Comments This PR is a follow-on of #545. Please DO NOT MERGE until the outstanding items are all completed.

Re: Ambari Wizard: Repo Tab

2017-04-25 Thread David Lyle
Any chance they're running Ambari < 2.4.2? -D... On Tue, Apr 25, 2017 at 3:23 PM, Michael Miklavcic < michael.miklav...@gmail.com> wrote: > Hey Otto, I don't have wizard screenshots in front of me right now to say > for sure, but I do see a repoinfo.xml in the mpack. I haven't run into > anythi

[GitHub] incubator-metron issue #535: METRON-859: Use REST application with Kerberos

2017-04-25 Thread mattf-horton
Github user mattf-horton commented on the issue: https://github.com/apache/incubator-metron/pull/535 @merrimanr but is the update of README.md part of the component build? Or can it be made so? That way if a developer makes changes/additions in the annotations, but misses re-committ

Re: Ambari Wizard: Repo Tab

2017-04-25 Thread Michael Miklavcic
Hey Otto, I don't have wizard screenshots in front of me right now to say for sure, but I do see a repoinfo.xml in the mpack. I haven't run into anything like that, but next time I run through the install I can take another look. https://github.com/apache/incubator-metron/blob/master/metron-deploy

[GitHub] incubator-metron issue #535: METRON-859: Use REST application with Kerberos

2017-04-25 Thread mattf-horton
Github user mattf-horton commented on the issue: https://github.com/apache/incubator-metron/pull/535 @merrimanr , okay, thanks. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature en

Re: Quick Dev - Atlas Images

2017-04-25 Thread Nick Allen
>> I hadn't really reasoned about the notion of a "released" Quick Dev image, but I can see a lot of value in having a versioned sandbox type image- but maybe not quick dev, maybe not even Vagrant? We could actually pre-package everything needed and save some provisioning time on released versions

[GitHub] incubator-metron issue #531: METRON-854 create dhcp dump parser

2017-04-25 Thread simonellistonball
Github user simonellistonball commented on the issue: https://github.com/apache/incubator-metron/pull/531 The Bro parsers is actually pretty generic, and will take whatever json bro dumps out. From a quick inspection you should just need to configure the bro instance to send out dhcp,

Re: Ambari Wizard: Repo Tab

2017-04-25 Thread Otto Fowler
stderr: Traceback (most recent call last): File "/var/lib/ambari-agent/cache/common-services/ELASTICSEARCH/2.3.3/package/scripts/elastic_slave.py", line 71, in Elasticsearch().execute() File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 280, in

Re: Ambari Wizard: Repo Tab

2017-04-25 Thread Otto Fowler
Yes, I’m going to run through again and get the logs. The basic issue is, no repo tab -> if I should have one, and the ES yum command failing with nothing to do, which stops deployment on all the nodes. On April 25, 2017 at 13:36:25, Michael Miklavcic ( michael.miklav...@gmail.com) wrote: I've

Re: Status of METRON-153

2017-04-25 Thread Otto Fowler
https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=65144361 This was broken when the rpm builds were introduced ( docker in docker no no ). This should work now ( i can indeed build the rpms etc and run ansible inside of docker now with METRON-857 landed ) with centos 6x ( although I

[GitHub] incubator-metron issue #535: METRON-859: Use REST application with Kerberos

2017-04-25 Thread merrimanr
Github user merrimanr commented on the issue: https://github.com/apache/incubator-metron/pull/535 I think src/main is fine. @mattf-horton there are no new files generated, this simply updates the metron-rest/README.md file. In theory you don't have to use it and it's not part of any

[GitHub] incubator-metron pull request #546: METRON-886 ansible-docker : need to have...

2017-04-25 Thread ottobackwards
GitHub user ottobackwards opened a pull request: https://github.com/apache/incubator-metron/pull/546 METRON-886 ansible-docker : need to have trailing slash for directory in copy for ansible.cfg ## Contributor Comments On a centos 6 host using docker-io, the COPY fails without t

Re: Status of METRON-153

2017-04-25 Thread Matt Foley
FWIW, we know that several of the changes in the (recently committed) METRON-634 Mpack bug fixes and improvements, not impacting Ambari database (PR#532) are necessary for a smooth install on Centos7. These changes are for the manual install wizard, but all the non-“description” changes

Re: Ambari Wizard: Repo Tab

2017-04-25 Thread Michael Miklavcic
I've been doing an internal hwx deployment (HCP) recently for testing purposes. I have some ES tweaks to work out, but other than that it has been running smoothly. Can you share any log info or screenshots? Mike On Apr 25, 2017 11:30 AM, "Otto Fowler" wrote: > Hi, > Should I see the repo tab i

[GitHub] incubator-metron issue #531: METRON-854 create dhcp dump parser

2017-04-25 Thread ottobackwards
Github user ottobackwards commented on the issue: https://github.com/apache/incubator-metron/pull/531 unless of course someone can't use bro for some reason --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

Ambari Wizard: Repo Tab

2017-04-25 Thread Otto Fowler
Hi, Should I see the repo tab in the ambari wizard for metron at this point? Asking for a friend who can’t get the wizard to work :)

Re: Status of METRON-153

2017-04-25 Thread Michael Miklavcic
We didn't come up with another approach Otto. I closed the PR at the time bc I didn't think we wanted to break with Centos 6 yet, and my PR probably would have done so. Can you elaborate on "prior capability wrt centos 6.x work again" a bit? On Apr 25, 2017 10:59 AM, "Otto Fowler" wrote: > Also,

[GitHub] incubator-metron issue #531: METRON-854 create dhcp dump parser

2017-04-25 Thread nickwallen
Github user nickwallen commented on the issue: https://github.com/apache/incubator-metron/pull/531 We also have a `JSONMapParser` that was contributed after the original Bro parser. The data coming out of the Bro plugin can be configured to be JSON. That's how we typically use

[GitHub] incubator-metron issue #535: METRON-859: Use REST application with Kerberos

2017-04-25 Thread mattf-horton
Github user mattf-horton commented on the issue: https://github.com/apache/incubator-metron/pull/535 @merrimanr , does the README.vm generate an html documentation file in target? If so, send me the install location, and I'll make a patch to the site-book tools to also add it to the

[GitHub] incubator-metron issue #535: METRON-859: Use REST application with Kerberos

2017-04-25 Thread mmiklavc
Github user mmiklavc commented on the issue: https://github.com/apache/incubator-metron/pull/535 Agreed this isn't Kerberos related. I just wasn't clear what the file was and why it is in test. Any reason the utility can't go in src/main along with the readme? Separate Jira, of course

[GitHub] incubator-metron issue #535: METRON-859: Use REST application with Kerberos

2017-04-25 Thread merrimanr
Github user merrimanr commented on the issue: https://github.com/apache/incubator-metron/pull/535 To be honest, I don't really care that much. Where would you like it to be? There is a runner class in test/java/org/apache/metron/rest/utils/ReadMeUtils.java that applies the template

[GitHub] incubator-metron issue #535: METRON-859: Use REST application with Kerberos

2017-04-25 Thread mmiklavc
Github user mmiklavc commented on the issue: https://github.com/apache/incubator-metron/pull/535 Should that be in the test src directory? `metron-interface/metron-rest/src/test/resources/README.vm` --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] incubator-metron issue #535: METRON-859: Use REST application with Kerberos

2017-04-25 Thread merrimanr
Github user merrimanr commented on the issue: https://github.com/apache/incubator-metron/pull/535 The REST API is documented with annotations. This is standard practice and powers the in context documentation in the Swagger UI. It was requested that this documentation also be includ

[GitHub] incubator-metron issue #531: METRON-854 create dhcp dump parser

2017-04-25 Thread nickwallen
Github user nickwallen commented on the issue: https://github.com/apache/incubator-metron/pull/531 > As an alternative method for getting DHCP data out of pcap, you might consider the existing Bro sensor, which essentially does what dhcpdump does... The current Bro parser only

Re: Status of METRON-153

2017-04-25 Thread Otto Fowler
Also, I can’t find any list discussion of the issue or other approaches. On April 25, 2017 at 12:48:46, Otto Fowler (ottobackwa...@gmail.com) wrote: What I am looking for is the prior capability wrt centos 6.x work again ( which I think my pr fixing the docker in docker issue would do ), and hav

Re: metron_full_install playbook with small_cluster and multiple hosts

2017-04-25 Thread Otto Fowler
I have a 5 node cluster ( with deploy machine/docker ). I want used to be able to run the full install with a custom inventory for this cluster to deploy. I am trying it again ( post docker fix ), but with centos 7. EC2 doesn’t work for everyone. unless you are saying I can use that to a local

Re: Status of METRON-153

2017-04-25 Thread Otto Fowler
What I am looking for is the prior capability wrt centos 6.x work again ( which I think my pr fixing the docker in docker issue would do ), and have it extended to work with 7.x. I don’t think re-writing all the stuff that is in metron roles is very attractive. On April 25, 2017 at 12:10:47, Dav

Re: metron_full_install playbook with small_cluster and multiple hosts

2017-04-25 Thread David Lyle
Hi Otto, I don't use a custom inventory that I create, but I do run an EC2 build very frequently, most recently on Friday. That does use the small_cluster file. 1. It's used because the Ansible Vagrant provisioner doesn't add localhost (as it should) to the inventory, so [local] works around that

[GitHub] incubator-metron issue #531: METRON-854 create dhcp dump parser

2017-04-25 Thread simonellistonball
Github user simonellistonball commented on the issue: https://github.com/apache/incubator-metron/pull/531 As an alternative method for getting DHCP data out of pcap, you might consider the existing Bro sensor, which essentially does what dhcpdump does, but for a wider range of protoco

[GitHub] incubator-metron issue #531: METRON-854 create dhcp dump parser

2017-04-25 Thread basvdl
Github user basvdl commented on the issue: https://github.com/apache/incubator-metron/pull/531 @nickwallen, these are indeed the options we have discussed... > I am going to lay out all of the possibilities that I can think of just so that we don't leave any stone unturned.

[GitHub] incubator-metron issue #535: METRON-859: Use REST application with Kerberos

2017-04-25 Thread mmiklavc
Github user mmiklavc commented on the issue: https://github.com/apache/incubator-metron/pull/535 @merrimanr Just curious, what's the difference between a vm file and an md file? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as

Re: Status of METRON-153

2017-04-25 Thread David Lyle
So, the current Ansible deployment actually does use blueprints. It constructs them from one of small_cluster.yml or single_node_vm.yml using the ambari_custer_state module. Ambari blueprints [1] and the Ambari Install Wizard are actually two separate things that are only related because both are

[GitHub] incubator-metron issue #507: METRON-819: Document kafka console producer par...

2017-04-25 Thread mmiklavc
Github user mmiklavc commented on the issue: https://github.com/apache/incubator-metron/pull/507 Merge with master again --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled a

Re: Status of METRON-153

2017-04-25 Thread Otto Fowler
Let me clarify, that this is support for automated blueprint deployment with ansible, not the ambari wizard. On April 25, 2017 at 11:31:37, zeo...@gmail.com (zeo...@gmail.com) wrote: Just tagging on here to indicate my interest in this - in order to have someone other than me manage the OSs in

Re: Status of METRON-153

2017-04-25 Thread zeo...@gmail.com
Just tagging on here to indicate my interest in this - in order to have someone other than me manage the OSs in my Metron cluster, I must run on RHEL 7. I assume that will be common across many enterprises. Semi-recentlyI took a stab at CentOS 7 support but it was a bit of a rough go and I dropped

Status of METRON-153

2017-04-25 Thread Otto Fowler
https://issues.apache.org/jira/browse/METRON-153 What is the other approach that is mentioned here? Was it implemented? The ansible build still does not support this and I was thinking of looking at it.

[GitHub] incubator-metron pull request #521: METRON-835 Use Profiler with Kerberos

2017-04-25 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/incubator-metron/pull/521 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the featu

[GitHub] incubator-metron issue #521: METRON-835 Use Profiler with Kerberos

2017-04-25 Thread mmiklavc
Github user mmiklavc commented on the issue: https://github.com/apache/incubator-metron/pull/521 Oh wow, didn't expect that. Thanks @nickwallen looks great! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

metron_full_install playbook with small_cluster and multiple hosts

2017-04-25 Thread Otto Fowler
Now that I have docker sorted with building the rpms, I’m trying to deploy using a custom inventory and metron_full_install. So the question is, has anyone else tried to do this recently? I may be doing something wrong here with a couple of things. 1. what is the [local] host type in the inventor

[GitHub] incubator-metron issue #521: METRON-835 Use Profiler with Kerberos

2017-04-25 Thread nickwallen
Github user nickwallen commented on the issue: https://github.com/apache/incubator-metron/pull/521 Thanks @mmiklavc . I went ahead and addressed all of the items that you mentioned. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHu

[GitHub] incubator-metron issue #542: METRON-873: Stellar string literals do not supp...

2017-04-25 Thread justinleet
Github user justinleet commented on the issue: https://github.com/apache/incubator-metron/pull/542 +1. Thanks for working on this. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this featur

[GitHub] incubator-metron pull request #521: METRON-835 Use Profiler with Kerberos

2017-04-25 Thread mmiklavc
Github user mmiklavc commented on a diff in the pull request: https://github.com/apache/incubator-metron/pull/521#discussion_r113203641 --- Diff: metron-deployment/vagrant/Kerberos-setup.md --- @@ -55,35 +112,245 @@ General Kerberization notes can be found in the metron-deployment

[GitHub] incubator-metron pull request #521: METRON-835 Use Profiler with Kerberos

2017-04-25 Thread mmiklavc
Github user mmiklavc commented on a diff in the pull request: https://github.com/apache/incubator-metron/pull/521#discussion_r113204204 --- Diff: metron-deployment/vagrant/Kerberos-setup.md --- @@ -1,48 +1,105 @@ -# Setting Up Kerberos in Vagrant Full Dev -**Note:** These ar

  1   2   >