Error building in Travis after taking master

2017-05-03 Thread Otto Fowler
Anyone know what this is? [ERROR] No plugin found for prefix 'jacoco' in the current project and in the plugin groups [org.apache.maven.plugins, org.codehaus.mojo] available from the repositories [local (/home/travis/.m2/repository), central ( https://repo.maven.apache.org/maven2)] -> [Help 1]

Re: Error building in Travis after taking master

2017-05-03 Thread Michael Miklavcic
The plugin is definitely in maven central. Otto, do you guys by chance use a dependency management tool to proxy Maven Central? https://github.com/apache/incubator-metron/blob/master/pom.xml#L269 https://search.maven.org/#artifactdetails%7Corg.jacoco%7Cjacoco%7C0.7.9%7Cpom On Wed, May 3, 2017

[GitHub] incubator-metron pull request #563: METRON-840: All "ambari_*" hosts need to...

2017-05-03 Thread dlyle65535
GitHub user dlyle65535 opened a pull request: https://github.com/apache/incubator-metron/pull/563 METRON-840: All "ambari_*" hosts need to have a /localrepo folder ## Contributor Comments Fixes broken EC2 deployment. With the default config, each host running needs a yum repo

Re: Request double-check on Ambari config logic (ES network_host)

2017-05-03 Thread Nick Allen
I think it would be very valuable to split out publish_host and bind_host, but I think it would make sense to do that work as part of a separate JIRA. I would like to see us in a position to ship 0.4.0 as quickly as possible. Is there a quick solution that reasonable balances security and

Re: [GitHub] incubator-metron issue #562: METRON-915 add node and npm to platform_info.sh

2017-05-03 Thread Nick Allen
I think it still makes sense to add these to the platform script. What if the mvn plugin breaks or starts working in an unexpected way? This would help us uncover issues there. But good point to bring up, Ryan. On Wed, May 3, 2017 at 8:32 AM, Ryan Merriman wrote: > We

Re: Error building in Travis after taking master

2017-05-03 Thread Ryan Merriman
You might want to try clearing the mvn cache. I've had travis get into a bad state before because of corrupt maven artifacts. On Wed, May 3, 2017 at 10:26 AM, Otto Fowler wrote: > https://travis-ci.org/ottobackwards/incubator-metron/builds/228364894?utm_ >

Re: [DISCUSS] platform_info.sh and PR template / Dev Guidelines

2017-05-03 Thread Michael Miklavcic
I agree with Nick about the PR template insofar as we should keep it simple. And for any checklist items, we should seek to automate as much as possible. We've already made some strides with Justin Leet's changes to include site-book building in the main build, for instance. I'd prefer to add this

Re: Error building in Travis after taking master

2017-05-03 Thread Justin Leet
Jacoco is introduced from https://github.com/apache/incubator-metron/pull/459. I'm not sure why you'd be getting a plugin error for it though, because it's available in the standard repos and I was able to build off a completely clean maven cache (and Travis has been able to build it as well).

Re: Error building in Travis after taking master

2017-05-03 Thread Otto Fowler
Just for anyone else https://docs.travis-ci.com/user/caching/#Clearing-Caches On May 3, 2017 at 11:45:26, Otto Fowler (ottobackwa...@gmail.com) wrote: > I am not sure I know how to do that in Travis. I will look when I get back > > On May 3, 2017 at 11:35:34, Ryan Merriman (merrim...@gmail.com)

Re: Error building in Travis after taking master

2017-05-03 Thread Justin Leet
I'm also curious if it works if you change the two 'jacoco:prepare-agent' to 'org.jacoco:prepare-agent'. Master has built off of this fine, so I'm wondering if there's a difference in that build that breaks the plugin resolution. On Wed, May 3, 2017 at 11:35 AM, Ryan Merriman

Re: Error building in Travis after taking master

2017-05-03 Thread Michael Miklavcic
Also Otto, are you able to build locally with that branch and same merge with master? On Wed, May 3, 2017 at 9:51 AM, Justin Leet wrote: > I'm also curious if it works if you change the two 'jacoco:prepare-agent' > to 'org.jacoco:prepare-agent'. Master has built off of

Re: Error building in Travis after taking master

2017-05-03 Thread Otto Fowler
I am not sure I know how to do that in Travis. I will look when I get back On May 3, 2017 at 11:35:34, Ryan Merriman (merrim...@gmail.com) wrote: > You might want to try clearing the mvn cache. I've had travis get into a > bad state before because of corrupt maven artifacts. > > On Wed, May 3,

Re: Request double-check on Ambari config logic (ES network_host)

2017-05-03 Thread Otto Fowler
My experience deploying with small_cluster / ansible was that it just worked at the time to my centos 6.9 esxi cluster. On May 3, 2017 at 08:30:59, Nick Allen (n...@nickallen.org) wrote: It only worked "good enough" on Ansible because it was mainly used for deploying to a controlled environment

Re: Normalization topology or separate normalization bolt for parsing topology

2017-05-03 Thread Nick Allen
> Clearly, a generic parser would be useful for the community not a type of parser that is highly customised for our noisy environment. Increasing the number of generic parsers for the community is definitely a good goal. I agree with you there. Could we achieve the same goal by making our

[GitHub] incubator-metron issue #562: METRON-915 add node and npm to platform_info.sh

2017-05-03 Thread nickwallen
Github user nickwallen commented on the issue: https://github.com/apache/incubator-metron/pull/562 +1 Thanks, Otto! Tested on OSX and CentOS. Worked as expected. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

Re: Request double-check on Ambari config logic (ES network_host)

2017-05-03 Thread Otto Fowler
How is the ambari service install configuration different from prior configuration through ansible? This used to work better right? On May 3, 2017 at 07:06:52, zeo...@gmail.com (zeo...@gmail.com) wrote: Thanks for the good write up Matt. Here are my thoughts: D1: I don't see a way to have a

[GitHub] incubator-metron pull request #559: METRON-907: Zeppelin Dashboard to execut...

2017-05-03 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/incubator-metron/pull/559 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

Maven 3.5

2017-05-03 Thread Otto Fowler
Is anyone using this? Are there any issues?

Re: [DISCUSS] platform_info.sh and PR template / Dev Guidelines

2017-05-03 Thread Otto Fowler
For example: we have recently introduced dependencies on Docker, Node.js and NPM that needed to be added later. On May 3, 2017 at 08:25:21, Otto Fowler (ottobackwa...@gmail.com) wrote: I think we should have documented in the template and guidelines something to the extent ( appropriate for the

Re: Request double-check on Ambari config logic (ES network_host)

2017-05-03 Thread David Lyle
Hi Otto, The Ansible settings were preserved by the mpack when deploying with Ansible. Ansible overrides the defaults. -D... On Wed, May 3, 2017 at 8:53 AM, Otto Fowler wrote: > My experience deploying with small_cluster / ansible was that it just > worked at the

Re: Request double-check on Ambari config logic (ES network_host)

2017-05-03 Thread Nick Allen
It only worked "good enough" on Ansible because it was mainly used for deploying to a controlled environment where we know the interface names; aka Vagrant/Single Node. It did not work well at all on environments other than Vagrant/Single Node. The work that was done with Elasticsearch and

Re: Failure to Deploy "Quick Dev"

2017-05-03 Thread Nick Allen
nickwallen Thanks. On Wed, May 3, 2017 at 9:03 AM, David Lyle wrote: > Hi Nick, > > You do. just need to set up an Atlas account and shoot over the name. > > -D... > > > On Wed, May 3, 2017 at 8:44 AM, Nick Allen wrote: > > > Who has the credentials

Re: [GitHub] incubator-metron issue #562: METRON-915 add node and npm to platform_info.sh

2017-05-03 Thread Ryan Merriman
We are using a mvn plugin that automatically installs the correct version of node and npm locally, at least for the management UI. Are there other parts of the project that depend on these tools? Is it desirable to include this even if they aren't a prerequisite for building? I don't

Re: Failure to Deploy "Quick Dev"

2017-05-03 Thread Nick Allen
Who has the credentials to be able to update the images on Atlas? On Fri, Apr 21, 2017 at 11:02 AM, Casey Stella wrote: > I'm betting we need to regenerate the quickdev image after all the mpack > changes. > > On Fri, Apr 21, 2017 at 10:45 AM, Otto Fowler

[DISCUSS] Rest : Delete Sensor

2017-05-03 Thread Otto Fowler
Right now, from what I can see, deleting a sensor just deletes the sensor configuration from ZK. What about the topics and running topologies? Can someone explain the flow for this? The result? For background, I’m working on rest interfaces to install/find/delete parser extensions. When an

Re: Failure to Deploy "Quick Dev"

2017-05-03 Thread Otto Fowler
Achievement unlocked. On May 3, 2017 at 09:03:57, David Lyle (dlyle65...@gmail.com) wrote: Hi Nick, You do. just need to set up an Atlas account and shoot over the name. -D... On Wed, May 3, 2017 at 8:44 AM, Nick Allen wrote: > Who has the credentials to be able to

[GitHub] incubator-metron pull request #562: METRON-915 add node and npm to platform_...

2017-05-03 Thread ottobackwards
GitHub user ottobackwards opened a pull request: https://github.com/apache/incubator-metron/pull/562 METRON-915 add node and npm to platform_info.sh ## Contributor Comments Now that we depend on node and npm, put them into the platform_info.sh for troubleshooting build

Re: MaaS + Apache Twill ?

2017-05-03 Thread Nick Allen
In the long run it is going to make more sense to get on a platform that someone else supports (aka Apache Twill). That is, assuming that the Apache Twill project is sustainable and of high quality. That lets someone else do the work to ensure that MaaS continues to work as YARN evolves,

[GitHub] incubator-metron pull request #562: METRON-915 add node and npm to platform_...

2017-05-03 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/incubator-metron/pull/562 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[DISCUSS] platform_info.sh and PR template / Dev Guidelines

2017-05-03 Thread Otto Fowler
I think we should have documented in the template and guidelines something to the extent ( appropriate for the document ) of : “Metron provides a platform_info.sh script in metron-deployment/scripts that outputs information that may be important to troubleshooting build and deployment issues. It

Re: [DISCUSS] platform_info.sh and PR template / Dev Guidelines

2017-05-03 Thread Nick Allen
+0 I am not a fan of the PR template. In my opinion, there is too much text already and I ended up just skipping over it. I think it gets in the way of a contributor actually describing their change. Just my opinion. I know many others disagree and find value in it. On Wed, May 3, 2017 at

Re: Error building in Travis after taking master

2017-05-03 Thread Otto Fowler
Note: Some input files additionally use unchecked or unsafe operations. 100 warnings [ERROR] No plugin found for prefix 'org.jacoco' in the current project and in the plugin groups [org.apache.maven.plugins, org.codehaus.mojo] available from the repositories [local

Re: Error building in Travis after taking master

2017-05-03 Thread Otto Fowler
I am going to be honest, I don’t usually build locally with : mvn -q -T 2C -DskipTests install && mvn -q -T 2C jacoco:prepare-agent surefire:test@unit-tests && mvn -q jacoco:prepare-agent surefire:test@integration-tests && mvn -q jacoco:prepare-agent test --projects

Re: Error building in Travis after taking master

2017-05-03 Thread Justin Leet
Does it run locally with: mvn -q -T 2C -DskipTests install && mvn -q -T 2C org.jacoco:prepare-agent surefire:test@unit-tests && mvn -q org.jacoco:prepare-agent surefire:test@integration-tests && mvn -q org.jacoco:prepare-agent test --projects metron-interface/metron-config &&

Re: Request double-check on Ambari config logic (ES network_host)

2017-05-03 Thread Matt Foley
Thanks everyone for your input. After sleeping on it, and reviewing Jon’s and Nick’s input, here is my proposal: 1. Keep the parameter network.host as a synonym of network.bind_host. This is backward-compatible with our past usage, and completely predictable in terms of results. 2. Add the

Re: Error building in Travis after taking master

2017-05-03 Thread Michael Miklavcic
One more thing - can you see if this exists in your local maven repo? ls -lh $HOME/.m2/repository/org/jacoco/jacoco-maven-plugin/0.7.9/ <<< total 136 -rw-r--r--+ 1 211B Mar 7 08:21 _remote.repositories -rw-r--r--+ 1 46K Mar 7

Re: Error building in Travis after taking master

2017-05-03 Thread Otto Fowler
> git checkout -b whyyounowork apache/master same issue locally On May 3, 2017 at 13:46:14, Justin Leet (justinjl...@gmail.com) wrote: Is it just that one branch that's failing, or is it also master? To try to narrow things down. On Wed, May 3, 2017 at 1:43 PM, Michael Miklavcic <

Re: Error building in Travis after taking master

2017-05-03 Thread Justin Leet
I'm on 3.5 and not having issues with it. On Wed, May 3, 2017 at 2:19 PM, Otto Fowler wrote: > Did we change the requirements for pre-commit testing to run the command > that way? > I did a commit this morning with the old command. > > > On May 3, 2017 at 14:04:32, Otto

Re: Error building in Travis after taking master

2017-05-03 Thread Otto Fowler
And… I Do have it locally ~/.m2 ottofowler% tree | grep jacoco │ ├── jacoco │ │ ├── jacoco-maven-plugin │ │ │ ├── jacoco-maven-plugin-0.7.9.jar │ │ │ ├── jacoco-maven-plugin-0.7.9.jar.sha1 │ │ │ ├── jacoco-maven-plugin-0.7.9.pom │ │

Re: Error building in Travis after taking master

2017-05-03 Thread Otto Fowler
No I cannot build locally with that command: [ERROR] No plugin found for prefix 'jacoco' in the current project and in the plugin groups [org.apache.maven.plugins, org.codehaus.mojo] available from the repositories [local (/Users/ottofowler/.m2/repository), central (

[GitHub] incubator-metron issue #507: METRON-819: Document kafka console producer par...

2017-05-03 Thread mmiklavc
Github user mmiklavc commented on the issue: https://github.com/apache/incubator-metron/pull/507 @nickwallen can you elaborate on this fix/config a bit? I think we should definitely add this detail to the doc. It looks like you've created a yaf user and principal here. Any additional

[GitHub] incubator-metron issue #507: METRON-819: Document kafka console producer par...

2017-05-03 Thread nickwallen
Github user nickwallen commented on the issue: https://github.com/apache/incubator-metron/pull/507 > can you elaborate on this fix/config a bit? I think we should definitely add this detail to the doc. It looks like you've created a yaf user and prin In the text that I

Re: Error building in Travis after taking master

2017-05-03 Thread Otto Fowler
Did we change the requirements for pre-commit testing to run the command that way? I did a commit this morning with the old command. On May 3, 2017 at 14:04:32, Otto Fowler (ottobackwa...@gmail.com) wrote: > git checkout -b whyyounowork apache/master same issue locally On May 3, 2017 at

Re: Request double-check on Ambari config logic (ES network_host)

2017-05-03 Thread Nick Allen
I like your approach, Matt. +1 from me. On Wed, May 3, 2017 at 1:32 PM, Matt Foley wrote: > Thanks everyone for your input. After sleeping on it, and reviewing Jon’s > and Nick’s input, here is my proposal: > > 1. Keep the parameter network.host as a synonym of

[GitHub] incubator-metron pull request #564: Metron-905 Fix square-bracket behavior a...

2017-05-03 Thread mattf-horton
GitHub user mattf-horton opened a pull request: https://github.com/apache/incubator-metron/pull/564 Metron-905 Fix square-bracket behavior and default network interface bindings for ES ## Contributor Comments Changes as specified in email thread and METRON-905: 1. Document

[DISCUSS] Metron global config

2017-05-03 Thread Michael Miklavcic
I think this has been touched on before, but I've been doing some testing with the MPack and new REST API and Management UI and there are some things I think we can make better. One of those items is the way we manage the global config. I think it would be beneficial to manage the global config

Dataworks summit Sydney

2017-05-03 Thread Casey Stella
The Australia/Pacific version of Dataworks Summit is in Sydney this year, September 20-21. This is a great place to talk about work you are doing in Apache Metron or how you are using Metron. Information on submitting an abstract is at

Re: Error building in Travis after taking master

2017-05-03 Thread Michael Miklavcic
That's great news! I was beginning to think we had a Maven bogeyman. I wonder if Maven is swallowing an exception and wrapping it with something that is user friendly. So, if the cached pom or jars are corrupted, it might simply be reporting this as a "not found" problem. I guess that's user

Re: Error building in Travis after taking master

2017-05-03 Thread Otto Fowler
I’m going to do some more testing here and see, and stop spamming the list. I have seen off of master work, and a build off of my travis branch fail locally. it could be that the problem is repo AND my branch On May 3, 2017 at 16:28:35, Michael Miklavcic (michael.miklav...@gmail.com) wrote:

Re: Request double-check on Ambari config logic (ES network_host)

2017-05-03 Thread Otto Fowler
I can test this on a cluster, but I need a couple of days. Centos7 is such a mess I’m going to switch my cluster back to 6. On May 3, 2017 at 16:46:09, Matt Foley (mfo...@hortonworks.com) wrote: Please see https://github.com/apache/incubator-metron/pull/564 If several of you could grab a

Re: Error building in Travis after taking master

2017-05-03 Thread Otto Fowler
ok, after deleting my repo, and using the travis command I got a good build with the full command. I think I need to delete ALL my travis caches. On May 3, 2017 at 14:52:04, Otto Fowler (ottobackwa...@gmail.com) wrote: And… I Do have it locally ~/.m2 ottofowler% tree | grep jacoco │ ├──

Unable to build Metron, stuck at rpm-docker

2017-05-03 Thread Laurens Vets
Hi List, I'm following this guide: https://cwiki.apache.org/confluence/display/METRON/Metron+with+HDP+2.5+bare-metal+install and Maven seems to fail after this: "cd metron-deployment/packaging/docker/rpm-docker" "mvn clean install -DskipTests -PHDP-2.5.0.0" Removing intermediate container

Re: Request double-check on Ambari config logic (ES network_host)

2017-05-03 Thread Matt Foley
Please see https://github.com/apache/incubator-metron/pull/564 If several of you could grab a config and help with testing it would be hugely appreciated. I’m starting with manual Ambari install, single node. Thanks, --Matt On 5/3/17, 10:45 AM, "Nick Allen" wrote: I

Re: [DISCUSS] Metron global config

2017-05-03 Thread David Lyle
Ambari has to manage the global configuration at least once or we cannot deploy. IIRC, last time this came up, we decided to expose it via Ambari but see if we could turn off the restart. I think we could expose it via an action. I'm pretty happy with any solution that takes that initial

Re: [DISCUSS] Modify bylaws to allow speculative branches

2017-05-03 Thread James Sirota
Hi Guys, I would like to put this up for a vote. We would modify the bylaws with the following: Significant, pervasive features are often developed in a speculative branch of the repository. The PMC may grant commit rights on the branch to its consistent contributors, while the initiative is

Re: Error building in Travis after taking master

2017-05-03 Thread Otto Fowler
I was able to build in travis, with my branch, after changing the command: The command "time mvn -q -T 2C -DskipTests install && time mvn -q -T 2C org.jacoco:jacoco-maven-plugin:prepare-agent surefire:test@unit-tests && mvn -q org.jacoco:jacoco-maven-plugin:prepare-agent

[GitHub] incubator-metron issue #556: METRON-903: Create a connections report in Zepp...

2017-05-03 Thread james-sirota
Github user james-sirota commented on the issue: https://github.com/apache/incubator-metron/pull/556 Thanks for fixing the cumulative report. The histogram looks great +1 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well.

[GitHub] incubator-metron issue #561: METRON-913: Create IP Report in Zeppelin

2017-05-03 Thread james-sirota
Github user james-sirota commented on the issue: https://github.com/apache/incubator-metron/pull/561 + 1 looks great --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] incubator-metron issue #220: Metron-342 Fix permissions of {{ metron_directo...

2017-05-03 Thread gspeter
Github user gspeter commented on the issue: https://github.com/apache/incubator-metron/pull/220 Hi @ottobackwards I run "mvn clean package -DskipTests | tee build.log". I send my log file. Can you help me.

[GitHub] incubator-metron issue #220: Metron-342 Fix permissions of {{ metron_directo...

2017-05-03 Thread ottobackwards
Github user ottobackwards commented on the issue: https://github.com/apache/incubator-metron/pull/220 please open the new jira and attach this log to that. As @simonellistonball has requested, then mention me in the jira --- If your project is set up for it, you can reply to this

Re: Request double-check on Ambari config logic (ES network_host)

2017-05-03 Thread zeo...@gmail.com
Thanks for the good write up Matt. Here are my thoughts: D1: I don't see a way to have a default that works in every scenario. Documenting this and setting a sane default that works most of the time is probably the best path forward. D2: If we use _local_ and _site_, shouldn't it prioritize

[GitHub] incubator-metron issue #220: Metron-342 Fix permissions of {{ metron_directo...

2017-05-03 Thread gspeter
Github user gspeter commented on the issue: https://github.com/apache/incubator-metron/pull/220 ok. thank you. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes