Re: [DISCUSS] Regression introduced in Full Dev

http://stackoverflow.com/questions/22150209/maven-changes-the-order-of-plugins-of-different-profiles https://issues.apache.org/jira/browse/MNG-2258 On April 25, 2017 at 22:33:58, Otto Fowler (ottobackwa...@gmail.com) wrote: So, the copy resources configuration is still there, and not governed

Re: [DISCUSS] Regression introduced in Full Dev

So, the copy resources configuration is still there, and not governed by the profile. Is it possible that since they both run in package, that they are out of order? On April 25, 2017 at 22:31:27, Otto Fowler (ottobackwa...@gmail.com) wrote: Having said that, i see the problem. I am very sorry

Re: [DISCUSS] Regression introduced in Full Dev

Having said that, i see the problem. I am very sorry How do I go about resubmitting if I can fix it? On April 25, 2017 at 22:29:56, Otto Fowler (ottobackwa...@gmail.com) wrote: OK, I’ll take a look. Sorry - I thought they read out of the RPMS dir and verified that they were produced both

Re: [DISCUSS] Regression introduced in Full Dev

The problem seems to be that full/quick dev rely on having the RPMs in the /target folder >From metron-deployment/roles/metron-rpms/defaults: metron_rpm_glob: "{{ playbook_dir }}/../packaging/docker/rpm-docker/target/RPMS/noarch/*.rpm" However, that PR appears to no longer copy to the /target

[GitHub] incubator-metron pull request #550: METRON-890: Intermittent unit test error...

GitHub user cestella reopened a pull request: https://github.com/apache/incubator-metron/pull/550 METRON-890: Intermittent unit test errors in shutting down Storm in memory component ## Contributor Comments Cross your fingers. This may or may not work. Please don't merge

[GitHub] incubator-metron pull request #550: METRON-890: Intermittent unit test error...

Github user cestella closed the pull request at: https://github.com/apache/incubator-metron/pull/550 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] incubator-metron pull request #551: METRON-892 add docker to platform info

GitHub user ottobackwards opened a pull request: https://github.com/apache/incubator-metron/pull/551 METRON-892 add docker to platform info ## Contributor Comments Now that docker is a requirement, we need to include the version in the platform info script Tested on

Re: auto-install on bare metal

I failed at this today, but maybe it was the way I tried. An example would be great. On April 25, 2017 at 20:11:26, David Lyle (dlyle65...@gmail.com) wrote: Hi Dima, The same Ansible playbooks that work for EC2 and Vagrant will work for bare metal installations. The only difference is that

[GitHub] incubator-metron pull request #550: METRON-890: Intermittent unit test error...

GitHub user cestella reopened a pull request: https://github.com/apache/incubator-metron/pull/550 METRON-890: Intermittent unit test errors in shutting down Storm in memory component ## Contributor Comments Cross your fingers. This may or may not work. Please don't merge

[GitHub] incubator-metron pull request #550: METRON-890: Intermittent unit test error...

Github user cestella closed the pull request at: https://github.com/apache/incubator-metron/pull/550 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] incubator-metron pull request #542: METRON-873: Stellar string literals do n...

Github user cestella commented on a diff in the pull request: https://github.com/apache/incubator-metron/pull/542#discussion_r113348853 --- Diff: metron-platform/metron-common/src/main/java/org/apache/metron/common/stellar/StellarCompiler.java --- @@ -263,7 +263,56 @@ public void

[GitHub] incubator-metron pull request #542: METRON-873: Stellar string literals do n...

Github user cestella commented on a diff in the pull request: https://github.com/apache/incubator-metron/pull/542#discussion_r113347347 --- Diff: metron-platform/metron-common/src/main/java/org/apache/metron/common/stellar/StellarCompiler.java --- @@ -263,7 +263,56 @@ public void

Re: auto-install on bare metal

Hi Dima, The same Ansible playbooks that work for EC2 and Vagrant will work for bare metal installations. The only difference is that you would need to pre-provision your machines and hand-build your inventory file. The AWS playbooks only provision the machines. All deployment of Metron is

[GitHub] incubator-metron issue #550: METRON-890: Intermittent unit test errors in sh...

Github user cestella commented on the issue: https://github.com/apache/incubator-metron/pull/550 yep, that's the belief. We're going to test it out here, in front of everyone. :) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub

RE: metron UI

Of course, I should disclaim, this is my own project, and not a Telstra related initiative. Kev -Original Message- From: Waterson, Kevin [mailto:kevin.water...@team.telstra.com] Sent: Wednesday, 26 April 2017 8:37 AM To: dev@metron.apache.org Subject: RE: metron UI HTML5/D3 Kev

[GitHub] incubator-metron pull request #545: METRON-883 Capture Bro Plugin Enhancemen...

Github user asfgit closed the pull request at: https://github.com/apache/incubator-metron/pull/545 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

Re: [DISCUSS] Regression introduced in Full Dev

Yeah, I tend to agree that a rundown of the various methods and when you would use them is in order. I will say that full-dev is especially important to have working since it is required for validating PRs. On Tue, Apr 25, 2017 at 18:56 zeo...@gmail.com wrote: > Can somebody

[GitHub] incubator-metron issue #550: METRON-890: Intermittent unit test errors in sh...

Github user mmiklavc commented on the issue: https://github.com/apache/incubator-metron/pull/550 Is the belief here that killing the topologies first might allow full cluster shutdown and cleanup? --- If your project is set up for it, you can reply to this email and have your reply

Re: metron UI

Kevin, Would be interested in learning more about the issues you have with Angular. Which technology would you prefer? Ryan On Tue, Apr 25, 2017 at 5:16 PM, Waterson, Kevin < kevin.water...@team.telstra.com> wrote: > The use of Angular has me writing my own UI for Metron > > Kev > >

RE: metron UI

The use of Angular has me writing my own UI for Metron Kev -Original Message- From: Justin Leet [mailto:justinjl...@gmail.com] Sent: Tuesday, 25 April 2017 12:19 AM To: dev@metron.apache.org Subject: Re: metron UI To elaborate on Ryan's reply a bit, the UI is a fairly new component in

[DISCUSS] Regression introduced in Full Dev

A regression was introduced recently that breaks full dev. I've narrowed down the commit that introduced it and have submitted a PR to revert that commit: https://github.com/apache/incubator-metron/pull/549. Given there has been confusion recently over our deployment build process, I think it's

[GitHub] incubator-metron pull request #550: METRON-890: Intermittent unit test error...

GitHub user cestella opened a pull request: https://github.com/apache/incubator-metron/pull/550 METRON-890: Intermittent unit test errors in shutting down Storm in memory component ## Contributor Comments Cross your fingers. This may or may not work. Please don't merge until

[GitHub] incubator-metron pull request #549: METRON-889: Regression introduced in Ful...

GitHub user merrimanr opened a pull request: https://github.com/apache/incubator-metron/pull/549 METRON-889: Regression introduced in Full Dev ## Contributor Comments Full dev is currently broken because the RPM files are not being copied to vagrant. I've determined the

Re: [DISCUSS] Storm topology.classpath setting

topology.classpath is a topology level config, so Mike's potential alternative is to migrate that down from Ambari into the actual topologies we launch (assuming it works as expected). Basically we're trading ease of use in Ambari vs. additional setup for each topology that's run. I'm in favor

Re: Ambari Wizard: Repo Tab

Nm. sorry. I fixed it. On April 25, 2017 at 16:42:05, Otto Fowler (ottobackwa...@gmail.com) wrote: Ok, now I see the repos in the ‘pick version’ screen, but it is erring on the f://localrepo even though the folder exists, there is no repodata/repomd.xml. What is the command to create a

Re: Ambari Wizard: Repo Tab

createrepo On Tue, Apr 25, 2017 at 4:42 PM, Otto Fowler wrote: > Ok, now I see the repos in the ‘pick version’ screen, but it is erring on > the f://localrepo > even though the folder exists, there is no repodata/repomd.xml. > > What is the command to create a local

[GitHub] incubator-metron issue #541: METRON-870: Add filtering by packet payload to ...

ixed pcap query by executing a command similar to the following: ``` $METRON_HOME/bin/pcap_query.sh fixed --ip_dst_port 8080 -st "20170425" -df "MMdd" ``` * Verify the MR job finishes successfully. Upon completion, you should see multiple files named wi

Re: Quick Dev - Atlas Images

If it is docker, then it is just the Dockerfiles. I think you can have vagrant use docker as a back end too right? On April 25, 2017 at 14:34:14, Nick Allen (n...@nickallen.org) wrote: >> I hadn't really reasoned about the notion of a "released" Quick Dev image, but I can see a lot of value in

Re: Ambari Wizard: Repo Tab

I was going by the HW community page. Ok, Let me try it On April 25, 2017 at 16:04:07, David Lyle (dlyle65...@gmail.com) wrote: That would do it. It requires 2.4.2+. I would have sworn I put that in the README, but I must have only annotated the PR. :( I'll get that in asap. -D... On Tue,

Re: Ambari Wizard: Repo Tab

That would do it. It requires 2.4.2+. I would have sworn I put that in the README, but I must have only annotated the PR. :( I'll get that in asap. -D... On Tue, Apr 25, 2017 at 3:44 PM, Otto Fowler wrote: > 2.4.1 as a matter of fact > > > On April 25, 2017 at

[GitHub] incubator-metron issue #531: METRON-854 create dhcp dump parser

Github user JonZeolla commented on the issue: https://github.com/apache/incubator-metron/pull/531 I would love to see Metron have a solution for both approaches - ingesting DHCP server logs, as well as DHCP observations based on network traffic. Like @ottobackwards mentioned, not

[GitHub] incubator-metron issue #535: METRON-859: Use REST application with Kerberos

Github user merrimanr commented on the issue: https://github.com/apache/incubator-metron/pull/535 It would take some work but I think it could be done. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

Re: Quick Dev - Atlas Images

I may have missed a discuss thread on this, but it looks like we are no longer using -SNAPSHOT in our current dev master https://github.com/apache/incubator-metron/blob/master/pom.xml#L19 On Tue, Apr 25, 2017 at 12:34 PM, Nick Allen wrote: > >> I hadn't really reasoned

Re: Ambari Wizard: Repo Tab

2.4.1 as a matter of fact On April 25, 2017 at 15:29:43, David Lyle (dlyle65...@gmail.com) wrote: Any chance they're running Ambari < 2.4.2? -D... On Tue, Apr 25, 2017 at 3:23 PM, Michael Miklavcic < michael.miklav...@gmail.com> wrote: > Hey Otto, I don't have wizard screenshots in front of

[GitHub] incubator-metron pull request #547: METRON-858 bro-plugin-kafka is throwing ...

GitHub user JonZeolla opened a pull request: https://github.com/apache/incubator-metron/pull/547 METRON-858 bro-plugin-kafka is throwing segfaults ## Contributor Comments This PR is a follow-on of #545. Please DO NOT MERGE until the outstanding items are all completed.

[GitHub] incubator-metron issue #535: METRON-859: Use REST application with Kerberos

Github user mattf-horton commented on the issue: https://github.com/apache/incubator-metron/pull/535 @merrimanr but is the update of README.md part of the component build? Or can it be made so? That way if a developer makes changes/additions in the annotations, but misses

Re: Ambari Wizard: Repo Tab

Hey Otto, I don't have wizard screenshots in front of me right now to say for sure, but I do see a repoinfo.xml in the mpack. I haven't run into anything like that, but next time I run through the install I can take another look.

Re: Quick Dev - Atlas Images

>> I hadn't really reasoned about the notion of a "released" Quick Dev image, but I can see a lot of value in having a versioned sandbox type image- but maybe not quick dev, maybe not even Vagrant? We could actually pre-package everything needed and save some provisioning time on released

[GitHub] incubator-metron issue #531: METRON-854 create dhcp dump parser

Github user simonellistonball commented on the issue: https://github.com/apache/incubator-metron/pull/531 The Bro parsers is actually pretty generic, and will take whatever json bro dumps out. From a quick inspection you should just need to configure the bro instance to send out

Re: Ambari Wizard: Repo Tab

stderr: Traceback (most recent call last): File "/var/lib/ambari-agent/cache/common-services/ELASTICSEARCH/2.3.3/package/scripts/elastic_slave.py", line 71, in Elasticsearch().execute() File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 280,

Re: Status of METRON-153

https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=65144361 This was broken when the rpm builds were introduced ( docker in docker no no ). This should work now ( i can indeed build the rpms etc and run ansible inside of docker now with METRON-857 landed ) with centos 6x ( although

[GitHub] incubator-metron issue #535: METRON-859: Use REST application with Kerberos

Github user merrimanr commented on the issue: https://github.com/apache/incubator-metron/pull/535 I think src/main is fine. @mattf-horton there are no new files generated, this simply updates the metron-rest/README.md file. In theory you don't have to use it and it's not part of

[GitHub] incubator-metron pull request #546: METRON-886 ansible-docker : need to have...

GitHub user ottobackwards opened a pull request: https://github.com/apache/incubator-metron/pull/546 METRON-886 ansible-docker : need to have trailing slash for directory in copy for ansible.cfg ## Contributor Comments On a centos 6 host using docker-io, the COPY fails without

Re: Status of METRON-153

FWIW, we know that several of the changes in the (recently committed) METRON-634 Mpack bug fixes and improvements, not impacting Ambari database (PR#532) are necessary for a smooth install on Centos7. These changes are for the manual install wizard, but all the non-“description” changes

[GitHub] incubator-metron issue #531: METRON-854 create dhcp dump parser

Github user ottobackwards commented on the issue: https://github.com/apache/incubator-metron/pull/531 unless of course someone can't use bro for some reason --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

Ambari Wizard: Repo Tab

Hi, Should I see the repo tab in the ambari wizard for metron at this point? Asking for a friend who can’t get the wizard to work :)

Re: Status of METRON-153

We didn't come up with another approach Otto. I closed the PR at the time bc I didn't think we wanted to break with Centos 6 yet, and my PR probably would have done so. Can you elaborate on "prior capability wrt centos 6.x work again" a bit? On Apr 25, 2017 10:59 AM, "Otto Fowler"

[GitHub] incubator-metron issue #531: METRON-854 create dhcp dump parser

Github user nickwallen commented on the issue: https://github.com/apache/incubator-metron/pull/531 We also have a `JSONMapParser` that was contributed after the original Bro parser. The data coming out of the Bro plugin can be configured to be JSON. That's how we typically use

[GitHub] incubator-metron issue #535: METRON-859: Use REST application with Kerberos

Github user mmiklavc commented on the issue: https://github.com/apache/incubator-metron/pull/535 Agreed this isn't Kerberos related. I just wasn't clear what the file was and why it is in test. Any reason the utility can't go in src/main along with the readme? Separate Jira, of

[GitHub] incubator-metron issue #535: METRON-859: Use REST application with Kerberos

Github user mmiklavc commented on the issue: https://github.com/apache/incubator-metron/pull/535 Should that be in the test src directory? `metron-interface/metron-rest/src/test/resources/README.vm` --- If your project is set up for it, you can reply to this email and have your

[GitHub] incubator-metron issue #535: METRON-859: Use REST application with Kerberos

Github user merrimanr commented on the issue: https://github.com/apache/incubator-metron/pull/535 The REST API is documented with annotations. This is standard practice and powers the in context documentation in the Swagger UI. It was requested that this documentation also be

[GitHub] incubator-metron issue #531: METRON-854 create dhcp dump parser

Github user nickwallen commented on the issue: https://github.com/apache/incubator-metron/pull/531 > As an alternative method for getting DHCP data out of pcap, you might consider the existing Bro sensor, which essentially does what dhcpdump does... The current Bro parser

Re: Status of METRON-153

Also, I can’t find any list discussion of the issue or other approaches. On April 25, 2017 at 12:48:46, Otto Fowler (ottobackwa...@gmail.com) wrote: What I am looking for is the prior capability wrt centos 6.x work again ( which I think my pr fixing the docker in docker issue would do ), and

Re: metron_full_install playbook with small_cluster and multiple hosts

I have a 5 node cluster ( with deploy machine/docker ). I want used to be able to run the full install with a custom inventory for this cluster to deploy. I am trying it again ( post docker fix ), but with centos 7. EC2 doesn’t work for everyone. unless you are saying I can use that to a local

Re: Status of METRON-153

What I am looking for is the prior capability wrt centos 6.x work again ( which I think my pr fixing the docker in docker issue would do ), and have it extended to work with 7.x. I don’t think re-writing all the stuff that is in metron roles is very attractive. On April 25, 2017 at 12:10:47,

Re: metron_full_install playbook with small_cluster and multiple hosts

Hi Otto, I don't use a custom inventory that I create, but I do run an EC2 build very frequently, most recently on Friday. That does use the small_cluster file. 1. It's used because the Ansible Vagrant provisioner doesn't add localhost (as it should) to the inventory, so [local] works around

[GitHub] incubator-metron issue #531: METRON-854 create dhcp dump parser

Github user simonellistonball commented on the issue: https://github.com/apache/incubator-metron/pull/531 As an alternative method for getting DHCP data out of pcap, you might consider the existing Bro sensor, which essentially does what dhcpdump does, but for a wider range of

[GitHub] incubator-metron issue #531: METRON-854 create dhcp dump parser

Github user basvdl commented on the issue: https://github.com/apache/incubator-metron/pull/531 @nickwallen, these are indeed the options we have discussed... > I am going to lay out all of the possibilities that I can think of just so that we don't leave any stone unturned.

[GitHub] incubator-metron issue #535: METRON-859: Use REST application with Kerberos

Github user mmiklavc commented on the issue: https://github.com/apache/incubator-metron/pull/535 @merrimanr Just curious, what's the difference between a vm file and an md file? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as

Re: Status of METRON-153

So, the current Ansible deployment actually does use blueprints. It constructs them from one of small_cluster.yml or single_node_vm.yml using the ambari_custer_state module. Ambari blueprints [1] and the Ambari Install Wizard are actually two separate things that are only related because both are

[GitHub] incubator-metron issue #507: METRON-819: Document kafka console producer par...

Github user mmiklavc commented on the issue: https://github.com/apache/incubator-metron/pull/507 Merge with master again --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

Re: Status of METRON-153

Let me clarify, that this is support for automated blueprint deployment with ansible, not the ambari wizard. On April 25, 2017 at 11:31:37, zeo...@gmail.com (zeo...@gmail.com) wrote: Just tagging on here to indicate my interest in this - in order to have someone other than me manage the OSs in

Re: Status of METRON-153

Just tagging on here to indicate my interest in this - in order to have someone other than me manage the OSs in my Metron cluster, I must run on RHEL 7. I assume that will be common across many enterprises. Semi-recentlyI took a stab at CentOS 7 support but it was a bit of a rough go and I

Status of METRON-153

https://issues.apache.org/jira/browse/METRON-153 What is the other approach that is mentioned here? Was it implemented? The ansible build still does not support this and I was thinking of looking at it.

[GitHub] incubator-metron pull request #521: METRON-835 Use Profiler with Kerberos

Github user asfgit closed the pull request at: https://github.com/apache/incubator-metron/pull/521 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] incubator-metron issue #521: METRON-835 Use Profiler with Kerberos

Github user mmiklavc commented on the issue: https://github.com/apache/incubator-metron/pull/521 Oh wow, didn't expect that. Thanks @nickwallen looks great! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project