date:20171212

The new Trusty image in travis that just landed breaks the build for one of
my PR’s.  I am not sure why, but I verified that going back to the old
image resolves the problem.
If you see a build fail, and it works locally or it is failing in a strange
way, be wary.

ottO

Re: [DISCUSS] Upcoming Release

2017-12-12 Thread zeo...@gmail.com

So I was looking at some of the docs and saw that Upgrading.md has a
dangling section that maybe should be removed.  Link
,
link 

Jon

On Tue, Dec 12, 2017 at 2:21 PM Matt Foley  wrote:

> Thanks to Jon, Otto, and Nick for looking over RC1.  Tonight I’ll fix the
> RAT glitch, build RC2, and put it to formal vote.
> Regards,
> --Matt
>
> On 12/12/17, 5:14 AM, "Nick Allen"  wrote:
>
> RC1 is looking good to me.  I validated the MD5s, built Metron, built
> the
> Bro plugin and reviewed the other artifacts like release notes.
>
> Running the RAT check on a 'clean' Metron does not produce any errors
> for
> me.  It is only after building Metron, which pulls in additional Node
> dependencies, does the RAT check fail.
>
>
> On Sun, Dec 10, 2017 at 4:41 PM Matt Foley  wrote:
>
> > Yes, but let’s see if anyone else find other issues.
> >
> >
> >
> > From: Otto Fowler 
> > Date: Saturday, December 9, 2017 at 6:16 AM
> > To: Matt Foley , "dev@metron.apache.org" <
> > dev@metron.apache.org>
> > Subject: Re: [DISCUSS] Upcoming Release
> >
> >
> >
> > So RC2 then?
> >
> >
> >
> > On December 8, 2017 at 20:43:21, Matt Foley (mfo...@hortonworks.com)
> > wrote:
> >
> > Hah, here it is: https://github.com/apache/metron/pull/743
> > “This problem seems to only reproduce when one unrolls a tarball
> rather
> > than cloning from github.”
> >
> > Heh, the exclusion at
> > https://github.com/apache/metron/blob/master/pom.xml#L351 is still
> there,
> > but the hashcode in the bundle.css file name has changed from
> > a0b6b99c10d9a13dc67e to f56deed131e58bd7ee04. Sigh. Did the version
> of Font
> > Awesome fonts change?
> >
> >
> > On 12/8/17, 5:26 PM, "Matt Foley"  wrote:
> >
> > I remember having trouble with this bundle.css file on the last
> release,
> > but I can’t remember what we did about it. Anybody?
> >
> > On 12/8/17, 1:41 PM, "Otto Fowler"  wrote:
> >
> > Steps
> >
> > - Downloaded tar.gz’s, asc files and KEYS
> > - Verified signing of both tar.gz’s
> > - searched for rouge 0.4.1 entries
> > - verified the main pom.xml
> > - built :
> >
> > mvn clean && time mvn -q -T 2C -DskipTests install && time mvn -q -T
> > 2C surefire:test@unit-tests && time mvn -q
> > surefire:test@integration-tests && time mvn -q test --projects
> > metron-interface/metron-config && time build_utils/verify_licenses.sh
> >
> > Found rat error:
> >
> >
> > *
> > Summary
> > ---
> > Generated at: 2017-12-08T16:33:27-05:00
> >
> > Notes: 3
> > Binaries: 193
> > Archives: 0
> > Standards: 75
> >
> > Apache Licensed: 74
> > Generated Documents: 0
> >
> > JavaDocs are generated, thus a license header is optional.
> > Generated files do not require license headers.
> >
> > 1 Unknown Licenses
> >
> > *
> >
> > Files with unapproved licenses:
> >
> >
> >
> /Users/batman/tmp/release_ver/apache-metron-0.4.2-rc1/metron-interface/metron-alerts/dist/styles.f56deed131e58bd7ee04.bundle.css
> >
> > *
> >
> >
> >
> >
> >
> > *
> > Summary
> > ---
> > Generated at: 2017-12-08T16:33:27-05:00
> >
> > Notes: 3
> > Binaries: 193
> > Archives: 0
> > Standards: 75
> >
> > Apache Licensed: 74
> > Generated Documents: 0
> >
> > JavaDocs are generated, thus a license header is optional.
> > Generated files do not require license headers.
> >
> > 1 Unknown Licenses
> >
> > *
> >
> > Files with unapproved licenses:
> >
> >
> >
> >
> /Users/ottofowler/tmp/release_ver/apache-metron-0.4.2-rc1/metron-interface/metron-alerts/dist/styles.f56deed131e58bd7ee04.bundle.css
> >
> > *
> >
> >
> >
> > On December 8, 2017 at 04:34:24, Matt Foley (ma...@apache.org)
> wrote:
> >
> > Colleagues,
> > I’ve posted Metron-0.4.2-RC1 and Metron-bro-plugin-kafka-0.1 to
> > https://dist.apache.org/repos/dist/dev/metron/0.4.2-RC1/
> >
> > Given the complexity of this RC, I’d appreciate if a couple people
> would be
> > willing to kick the tires before we put it up for a vote.
> >
> > I will myself be going thru the

[GitHub] metron issue #857: METRON-1340: Improve e2e tests for metron alerts

Github user mmiklavc commented on the issue:

https://github.com/apache/metron/pull/857
  
OK, I was able to get the tests to run. First run I get a single error. 
@merrimanr and I are talking through it now.

```
[INFO]   Test spec for login page
[INFO] â should display error message for invalid credentials
[INFO]   - Failed: Error while waiting for Protractor to sync with the 
page: true
[INFO] .â should login for valid credentials
[INFO] .â should logout
...
[INFO] Failures:
[INFO] 1) Test spec for login page should display error message for invalid 
credentials
[INFO]   Message:
[INFO] Failed: Error while waiting for Protractor to sync with the 
page: true
[INFO]   Stack:
[INFO] Error: Error while waiting for Protractor to sync with the page: 
true
[INFO] at runWaitForAngularScript.then 
(/Users/mmiklavcic/devprojects/metron/metron-interface/metron-alerts/node_modules/protractor/built/browser.js:463:23)
[INFO] at ManagedPromise.invokeCallback_ 
(/Users/mmiklavcic/devprojects/metron/metron-interface/metron-alerts/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:1376:14)
[INFO] at TaskQueue.execute_ 
(/Users/mmiklavcic/devprojects/metron/metron-interface/metron-alerts/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:3084:14)
[INFO] at TaskQueue.executeNext_ 
(/Users/mmiklavcic/devprojects/metron/metron-interface/metron-alerts/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:3067:27)
[INFO] at asyncRun 
(/Users/mmiklavcic/devprojects/metron/metron-interface/metron-alerts/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:2927:27)
[INFO] at 
/Users/mmiklavcic/devprojects/metron/metron-interface/metron-alerts/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:668:7
[INFO] at process._tickCallback 
(internal/process/next_tick.js:109:7)
[INFO] From: Task: 
[INFO] at pollCondition 
(/Users/mmiklavcic/devprojects/metron/metron-interface/metron-alerts/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:2195:19)
[INFO] at 
/Users/mmiklavcic/devprojects/metron/metron-interface/metron-alerts/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:2191:7
[INFO] at new ManagedPromise 
(/Users/mmiklavcic/devprojects/metron/metron-interface/metron-alerts/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:1077:7)
[INFO] at ControlFlow.promise 
(/Users/mmiklavcic/devprojects/metron/metron-interface/metron-alerts/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:2505:12)
[INFO] at 
/Users/mmiklavcic/devprojects/metron/metron-interface/metron-alerts/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:2190:22
[INFO] at TaskQueue.execute_ 
(/Users/mmiklavcic/devprojects/metron/metron-interface/metron-alerts/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:3084:14)
[INFO] at TaskQueue.executeNext_ 
(/Users/mmiklavcic/devprojects/metron/metron-interface/metron-alerts/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:3067:27)
[INFO] at asyncRun 
(/Users/mmiklavcic/devprojects/metron/metron-interface/metron-alerts/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:2974:25)
[INFO] at 
/Users/mmiklavcic/devprojects/metron/metron-interface/metron-alerts/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:668:7
[INFO] at process._tickCallback 
(internal/process/next_tick.js:109:7)
[INFO] From: Task: 
[INFO] at scheduleWait 
(/Users/mmiklavcic/devprojects/metron/metron-interface/metron-alerts/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:2188:20)
[INFO] at ControlFlow.wait 
(/Users/mmiklavcic/devprojects/metron/metron-interface/metron-alerts/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:2517:12)
[INFO] at Driver.wait 
(/Users/mmiklavcic/devprojects/metron/metron-interface/metron-alerts/node_modules/protractor/node_modules/selenium-webdriver/lib/webdriver.js:934:29)
[INFO] at run 
(/Users/mmiklavcic/devprojects/metron/metron-interface/metron-alerts/node_modules/protractor/built/browser.js:59:33)
[INFO] at ProtractorBrowser.to.(anonymous function) [as wait] 
(/Users/mmiklavcic/devprojects/metron/metron-interface/metron-alerts/node_modules/protractor/built/browser.js:67:16)
[INFO] at Object.waitForElementVisibility 
(/Users/mmiklavcic/devprojects/metron/metron-interface/metron-alerts/e2e/utils/e2e_util.ts:43:20)
[INFO] at LoginPage.setUserNameAndPassword

[GitHub] metron issue #774: METRON-1212 The bundle system and maven plugin

Github user ottobackwards commented on the issue:

https://github.com/apache/metron/pull/774
  
The new Trusty image breaks my build. I need to figure out why exactly, so 
I'll be trying a few things.


---

[GitHub] metron issue #857: METRON-1340: Improve e2e tests for metron alerts

Github user mmiklavc commented on the issue:

https://github.com/apache/metron/pull/857
  
Should we also run the tests using the new Maven profile? `mvn test -Pe2e`

The README should probably be updated to reflect this as well.


---

[GitHub] metron issue #857: METRON-1340: Improve e2e tests for metron alerts

Github user mmiklavc commented on the issue:

https://github.com/apache/metron/pull/857
  
A couple notes for the community - if your node/npm versions are out of 
date and you've already built the project, you'll run into trouble with 
versions not matching against the expected node version, e.g. `Node Sass could 
not find a binding for your current environment: OS X 64-bit with Node.js 9.x`. 
I deleted the node_modules directory entirely, `rm -r 
metron-alerts/node_modules`, re-ran the install, `npm install`, and then the 
start server script ran successfully. Just to note, the server script does not 
background the process, so you'll need to manually do that or use 2 windows to 
run the e2e tests.


---

[GitHub] metron pull request #834: METRON-1306: When index template install fails, we...

2017-12-12 Thread asfgit

Github user asfgit closed the pull request at:

https://github.com/apache/metron/pull/834


---

[GitHub] metron issue #857: METRON-1340: Improve e2e tests for metron alerts

Github user mmiklavc commented on the issue:

https://github.com/apache/metron/pull/857
  
Seems that my NodeJS and npm versions got mangled since last time I ran 
this. Perhaps migrating laptops, upgrading to High Sierra, and switching from 
MacPorts to HomeBrew might have something to do with it ;)


---

[GitHub] metron issue #834: METRON-1306: When index template install fails, we should...

Github user ottobackwards commented on the issue:

https://github.com/apache/metron/pull/834
  
I'm good +1


---

“777” Feature Branch Redux

I have created a new feature branch
feature/METRON-1211-extensions-parsers-gradual to track the parser
extension work
and have rebased https://github.com/apache/metron/pull/774 on to that.

I have also updated confluence and jira :
https://cwiki.apache.org/confluence/display/METRON/Metron+Extension+System+and+Parser+Extensions

1. Feature Branch still makes sense
2. Now that we are splitting it up it will work better than the “whole
boat” approach to the original branch
3. I don't want to have to worry about regression for short periods as I
re-implement

Cheers.

[GitHub] metron issue #774: METRON-1212 The bundle system and maven plugin

Github user ottobackwards commented on the issue:

https://github.com/apache/metron/pull/774
  
I have created a new feature branch 
feature/METRON-1211-extensions-parsers-gradual to track this.
and have rebased this PR on to that.

I have also updated confluence and jira : 
https://cwiki.apache.org/confluence/display/METRON/Metron+Extension+System+and+Parser+Extensions

1. Feature Branch still makes sense
2. now that we are splitting it up it will work better
3. I don't want to have to worry about regression for short periods

Will email list


---

[GitHub] metron issue #834: METRON-1306: When index template install fails, we should...

Github user ottobackwards commented on the issue:

https://github.com/apache/metron/pull/834
  
I expect to address this in the "777" feature branch/ parser effort.


---

[GitHub] metron issue #834: METRON-1306: When index template install fails, we should...

Github user merrimanr commented on the issue:

https://github.com/apache/metron/pull/834
  
After reading this whole thread I agree with @ottobackwards.  If we 
installed and started all services and THEN installed our parsers separately 
this whole issue goes away.  If a parser's template doesn't exist or isn't 
installed correctly, it fails and hopefully we know exactly why.  That is 
obviously a major architectural change and won't be solved in this PR.  

Since we're in a situation where we do include default parsers as part of 
the installation, I think they should work out of the box and fail fast if 
there are not installed correctly.  Would some of the concerns be alleviated 
with https://github.com/apache/metron/pull/831?  

+1 from me either way.  


---

Re: [DISCUSS] Community Meetings

2017-12-12 Thread Simon Elliston Ball

Yes, I do. 

I suspect the best bet will be to post recordings somewhere on the apache.org 
 metron site.

Simon

> On 12 Dec 2017, at 18:36, Otto Fowler  wrote:
> 
> Excellent, do you have the > 40 min + record option?
> 
> 
> On December 12, 2017 at 13:19:55, Simon Elliston Ball (
> si...@simonellistonball.com) wrote:
> 
> Happy to volunteer a zoom room. That seems to have worked for most in the
> past.
> 
> Simon
> 
>> On 12 Dec 2017, at 18:09, Otto Fowler  wrote:
>> 
>> Thanks! I think I’d like something hosted though.
>> 
>> 
>> On December 12, 2017 at 11:18:52, Ahmed Shah (ahmeds...@cmail.carleton.ca)
> 
>> wrote:
>> 
>> Hello,
>> 
>> wrt "- How are we going to host it"...
>> 
>> I've used BigBlueButton as an end user at our University.
>> 
>> It is LGPL open source.
>> 
>> https://bigbluebutton.org/
>> https://bigbluebutton.org/developers/
>> 
>> 
>> -Ahmed
>> 
>> ___
>> Ahmed Shah (PMP, M. Eng.)
>> Cybersecurity Analyst & Developer
>> GCR - Cybersecurity Operations Center
>> Carleton University - cugcr.com
>> 
>> 
>> 
>> From: Otto Fowler 
>> Sent: December 11, 2017 4:41 PM
>> To: dev@metron.apache.org
>> Subject: [DISCUSS] Community Meetings
>> 
>> I think that we all want to have regular community meetings. We may be
>> better able to keep to a regular schedule with these meetings if we
> spread
>> out the responsibility for them from James and Casey, both of whom have a
>> lot on their plate already.
>> 
>> I would be willing to coordinate and run the meetings, and would welcome
>> anyone else who wants to help when they can.
>> 
>> The only issue for me is I do not have a web-ex account that I can use to
>> hold the meeting. So I’ll need some recommendations for a suitable
>> alternative. I have not been able to find an Apache Friendly alternative,
>> in the same way that Atlassian is apache friendly.
>> 
>> 
>> So - from what I can see we need to:
>> 
>> - Talk through who is going to do it
>> - How are we going to host it
>> - When are we going to do it
>> 
>> Anything else?
>> 
>> ottO

Re: [DISCUSS] Community Meetings

Excellent, do you have the > 40 min + record option?


On December 12, 2017 at 13:19:55, Simon Elliston Ball (
si...@simonellistonball.com) wrote:

Happy to volunteer a zoom room. That seems to have worked for most in the
past.

Simon

> On 12 Dec 2017, at 18:09, Otto Fowler  wrote:
>
> Thanks! I think I’d like something hosted though.
>
>
> On December 12, 2017 at 11:18:52, Ahmed Shah (ahmeds...@cmail.carleton.ca)

> wrote:
>
> Hello,
>
> wrt "- How are we going to host it"...
>
> I've used BigBlueButton as an end user at our University.
>
> It is LGPL open source.
>
> https://bigbluebutton.org/
> https://bigbluebutton.org/developers/
>
>
> -Ahmed
>
> ___
> Ahmed Shah (PMP, M. Eng.)
> Cybersecurity Analyst & Developer
> GCR - Cybersecurity Operations Center
> Carleton University - cugcr.com
>
>
> 
> From: Otto Fowler 
> Sent: December 11, 2017 4:41 PM
> To: dev@metron.apache.org
> Subject: [DISCUSS] Community Meetings
>
> I think that we all want to have regular community meetings. We may be
> better able to keep to a regular schedule with these meetings if we
spread
> out the responsibility for them from James and Casey, both of whom have a
> lot on their plate already.
>
> I would be willing to coordinate and run the meetings, and would welcome
> anyone else who wants to help when they can.
>
> The only issue for me is I do not have a web-ex account that I can use to
> hold the meeting. So I’ll need some recommendations for a suitable
> alternative. I have not been able to find an Apache Friendly alternative,
> in the same way that Atlassian is apache friendly.
>
>
> So - from what I can see we need to:
>
> - Talk through who is going to do it
> - How are we going to host it
> - When are we going to do it
>
> Anything else?
>
> ottO

Re: [DISCUSS] Community Meetings

2017-12-12 Thread Simon Elliston Ball

Happy to volunteer a zoom room. That seems to have worked for most in the past.

Simon

> On 12 Dec 2017, at 18:09, Otto Fowler  wrote:
> 
> Thanks!  I think I’d like something hosted though.
> 
> 
> On December 12, 2017 at 11:18:52, Ahmed Shah (ahmeds...@cmail.carleton.ca)
> wrote:
> 
> Hello,
> 
> wrt "- How are we going to host it"...
> 
> I've used BigBlueButton as an end user at our University.
> 
> It is LGPL open source.
> 
> https://bigbluebutton.org/
> https://bigbluebutton.org/developers/
> 
> 
> -Ahmed
> 
> ___
> Ahmed Shah (PMP, M. Eng.)
> Cybersecurity Analyst & Developer
> GCR - Cybersecurity Operations Center
> Carleton University - cugcr.com
> 
> 
> 
> From: Otto Fowler 
> Sent: December 11, 2017 4:41 PM
> To: dev@metron.apache.org
> Subject: [DISCUSS] Community Meetings
> 
> I think that we all want to have regular community meetings. We may be
> better able to keep to a regular schedule with these meetings if we spread
> out the responsibility for them from James and Casey, both of whom have a
> lot on their plate already.
> 
> I would be willing to coordinate and run the meetings, and would welcome
> anyone else who wants to help when they can.
> 
> The only issue for me is I do not have a web-ex account that I can use to
> hold the meeting. So I’ll need some recommendations for a suitable
> alternative. I have not been able to find an Apache Friendly alternative,
> in the same way that Atlassian is apache friendly.
> 
> 
> So - from what I can see we need to:
> 
> - Talk through who is going to do it
> - How are we going to host it
> - When are we going to do it
> 
> Anything else?
> 
> ottO

Re: [DISCUSS] Community Meetings

Thanks!  I think I’d like something hosted though.


On December 12, 2017 at 11:18:52, Ahmed Shah (ahmeds...@cmail.carleton.ca)
wrote:

Hello,

wrt "- How are we going to host it"...

I've used BigBlueButton as an end user at our University.

It is LGPL open source.

https://bigbluebutton.org/
https://bigbluebutton.org/developers/


-Ahmed

___
Ahmed Shah (PMP, M. Eng.)
Cybersecurity Analyst & Developer
GCR - Cybersecurity Operations Center
Carleton University - cugcr.com



From: Otto Fowler 
Sent: December 11, 2017 4:41 PM
To: dev@metron.apache.org
Subject: [DISCUSS] Community Meetings

I think that we all want to have regular community meetings. We may be
better able to keep to a regular schedule with these meetings if we spread
out the responsibility for them from James and Casey, both of whom have a
lot on their plate already.

I would be willing to coordinate and run the meetings, and would welcome
anyone else who wants to help when they can.

The only issue for me is I do not have a web-ex account that I can use to
hold the meeting. So I’ll need some recommendations for a suitable
alternative. I have not been able to find an Apache Friendly alternative,
in the same way that Atlassian is apache friendly.


So - from what I can see we need to:

- Talk through who is going to do it
- How are we going to host it
- When are we going to do it

Anything else?

ottO

Re: [DISCUSS] Community Meetings

2017-12-12 Thread Ahmed Shah

Hello,

wrt "- How are we going to host it"...

I've used BigBlueButton as an end user at our University.

It is LGPL open source.

https://bigbluebutton.org/
https://bigbluebutton.org/developers/


-Ahmed

___
Ahmed Shah (PMP, M. Eng.)
Cybersecurity Analyst & Developer
GCR - Cybersecurity Operations Center
Carleton University - cugcr.com



From: Otto Fowler 
Sent: December 11, 2017 4:41 PM
To: dev@metron.apache.org
Subject: [DISCUSS] Community Meetings

I think that we all want to have regular community meetings.  We may be
better able to keep to a regular schedule with these meetings if we spread
out the responsibility for them from James and Casey, both of whom have a
lot on their plate already.

I would be willing to coordinate and run the meetings, and would welcome
anyone else who wants to help when they can.

The only issue for me is I do not have a web-ex account that I can use to
hold the meeting.  So I’ll need some recommendations for a suitable
alternative.  I have not been able to find an Apache Friendly alternative,
in the same way that Atlassian is apache friendly.


So - from what I can see we need to:

- Talk through who is going to do it
- How are we going to host it
- When are we going to do it

Anything else?

ottO

[GitHub] metron issue #853: METRON-1337: List of facets should not be hardcoded

Github user merrimanr commented on the issue:

https://github.com/apache/metron/pull/853
  
Ambari does not manage this config.  It is only included in the base 
application.yml as a default setting.  I don't feel like this setting should be 
in Ambari for a couple reasons:  changing it requires a restart and the list of 
facets cannot be user specific.



---

[GitHub] metron issue #853: METRON-1337: List of facets should not be hardcoded

Github user ottobackwards commented on the issue:

https://github.com/apache/metron/pull/853
  
What I get to was, the setting should be exposed in ambari if ambari 
manages the config.


---

[GitHub] metron pull request #858: METRON-1344: Externalize the infrastructural compo...

GitHub user merrimanr reopened a pull request:

https://github.com/apache/metron/pull/858

METRON-1344: Externalize the infrastructural components using integration
tests

## Contributor Comments
This PR will add infrastructure to our Travis build that will allow the
Alerts UI e2e test to be run. There are several outstanding issues that still
need to be worked out so DO NOT MERGE this yet.

This is a first pass at a potential Docker-based solution and is meant to
be a POC. The intention is to facilitate further discussion around the general
approach and provide a working example that we can build off of.

A good place to start is the .travis file. This provides a good guide on
how this infrastructure is spun up. It is assumed Docker and Docker Compose
are installed. To use outside of travis in a local dev environment (assuming
Mac OSX):

1. Build Metron with `mvn clean install -DskipTests`
1. Create a Docker machine with
`metron-contrib/metron-docker/scripts/create-docker-machine.sh`
1. Set the Docker env variables with `eval $(docker-machine env
metron-machine)`
1. Build the base Metron image that installs Java: `docker build
./metron-centos/ -t "metron-centos"`
1. Spin up the environment: `cd metron-contrib/metron-docker-e2e/compose
&& docker-compose up -d`

A working environment should now be available. To verify get the Docker
machine address with `echo $DOCKER_HOST`. Services should be available on the
ports specified in
`metron-contrib/metron-docker-e2e/compose/docker-compose.yml`. For example,
assuming my $DOCKER_HOST is "tcp://192.168.99.100:2376", Elasticsearch should
be available at http://192.168.99.100:9210.

At this point only a single e2e test is run due to the significant
refactoring being done in https://github.com/apache/metron/pull/857.

Here are my thoughts so far based on work in this PR:

- The Docker environment creation and startup adds about 3 minutes to the
build. This is about what I expected and hopefully we can get this time back
as we move other tests to a reusable environment.
- I'm not convinced spinning up containers for the Alerts UI and REST are
necessary or desired. We may be able to cut some time off the build by running
them directly on the Travis host instead of in Docker
- I experimented with caching the Docker images but pulling them every time
was actually faster

There is still significant work to be done including:
- I was able to get the full suite of tests to run successfully in previous
commits but have all but e2e tests commented out for now to make it easier to
see what's going on in Travis. These will need to be added back once we get
closer to a final solution.
- Once the e2e test refactoring is done those changes will need merged in
and tested
- The license check is commented out right now because I inadvertently
added some new dependency versions (have no idea why). Still need to track
that down.
- There is an intermittent error that happens when starting up REST. Still
working on tracking this down.

Looking forward to some feedback.

## Pull Request Checklist

Thank you for submitting a contribution to Apache Metron.
Please refer to our [Development
Guidelines](https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=61332235)
for the complete guide to follow for contributions.
Please refer also to our [Build Verification
Guidelines](https://cwiki.apache.org/confluence/display/METRON/Verifying+Builds?show-miniview)
for complete smoke testing guides.

In order to streamline the review of the contribution we ask you follow
these guidelines and ask you to double check the following:

### For all changes:
- [x] Is there a JIRA ticket associated with this PR? If not one needs to
be created at [Metron
Jira](https://issues.apache.org/jira/browse/METRON/?selectedTab=com.atlassian.jira.jira-projects-plugin:summary-panel).

- [x] Does your PR title start with METRON- where is the JIRA
number you are trying to resolve? Pay particular attention to the hyphen "-"
character.
- [ ] Has your PR been rebased against the latest commit within the target
branch (typically master)?

### For code changes:
- [ ] Have you included steps to reproduce the behavior or problem that is
being changed or addressed?
- [ ] Have you included steps or a guide to how the change may be verified
and tested manually?
- [ ] Have you ensured that the full suite of tests and checks have been
executed in the root metron folder via:
```
mvn -q clean integration-test install && build_utils/verify_licenses.sh
```

- [ ] Have you written or updated unit tests and or integration tests to
verify your changes?
- [ ] If adding new dependencies to the code, are these

[GitHub] metron pull request #858: METRON-1344: Externalize the infrastructural compo...

Github user merrimanr closed the pull request at:

https://github.com/apache/metron/pull/858


---

[GitHub] metron issue #858: METRON-1344: Externalize the infrastructural components u...

Github user ottobackwards commented on the issue:

https://github.com/apache/metron/pull/858
  
@merrimanr, I have created labels in jira that are applicable to feature 
branch Jiras,
This will allow for tracking these jiras in confluence as I have done here 
: 
https://cwiki.apache.org/confluence/display/METRON/Metron+Extension+System+and+Parser+Extensions.

I would suggest creating a page for this feature branch were we can 
document things in this area, and using the labels on the liras

metron-feature-canidate, {create a metron-feature-externalize-? label}


---

[GitHub] metron issue #853: METRON-1337: List of facets should not be hardcoded

Github user merrimanr commented on the issue:

https://github.com/apache/metron/pull/853
  
Each environment will have it's own application.yaml.  Full dev has one, 
our testing environment has one, Ambari ships one etc.  There is also a base 
application.yaml that has the defaults (default facets are included here).  
Does that make sense?


---

[GitHub] metron issue #579: METRON-941 fix PaloAltoParser

Github user ottobackwards commented on the issue:

https://github.com/apache/metron/pull/579
  
@ctramnitz is there any update on this PR?  Is there something we can do to 
help?


---

[GitHub] metron issue #834: METRON-1306: When index template install fails, we should...

Github user ottobackwards commented on the issue:

https://github.com/apache/metron/pull/834
  
Where are we at with this?


---

[GitHub] metron issue #853: METRON-1337: List of facets should not be hardcoded

Github user ottobackwards commented on the issue:

https://github.com/apache/metron/pull/853
  
Does ambari own the application.yaml currently?


---

[GitHub] metron issue #862: METRON-1343: Swagger UI for User Controller needs request...

Github user ottobackwards commented on the issue:

https://github.com/apache/metron/pull/862
  
Like, 

- Spin up Full Dev
- Go to Swagger at http://node1:4200(?)
- You should see X now


---

[GitHub] metron issue #862: METRON-1343: Swagger UI for User Controller needs request...

Github user ottobackwards commented on the issue:

https://github.com/apache/metron/pull/862
  
Great, can you list that as the manual verification please?


---

[GitHub] metron issue #854: Experimental Improvements - Feedback Only - Do Not Merge

Github user ottobackwards commented on the issue:

https://github.com/apache/metron/pull/854
  
based on @justinleet 's comment, where do we stand with this


---

[GitHub] metron issue #856: METRON-1339 Stellar Shell functionality to verify stored ...

Github user ottobackwards commented on the issue:

https://github.com/apache/metron/pull/856
  
@nickwallen any feedback, does the annotated approach match what you 
imagined?


---

[GitHub] metron pull request #859: METRON-1345: Update EC2 README for custom Ansible ...

Github user ottobackwards commented on a diff in the pull request:

https://github.com/apache/metron/pull/859#discussion_r156374333
  
--- Diff: metron-deployment/roles/ambari_config/vars/small_cluster.yml ---
@@ -87,6 +87,8 @@ configurations:
   topology.classpath: '{{ topology_classpath }}'
   - kafka-broker:
   log.dirs: '{{ kafka_log_dirs | default("/kafka-log") }}'
--- End diff --

Should the EC2 Readme include documentation about the dev env and what it 
means?  Should it link to or document how to  change from dev to 'other', 
or do other from the start?


---

[GitHub] metron issue #862: METRON-1343: Swagger UI for User Controller needs request...

Github user ottobackwards commented on the issue:

https://github.com/apache/metron/pull/862
  
Hi!  Thanks for the contribution.
How can we test these changes?  
Do we need new tests in the code for this change ?


Please consider the items in the PR checklist ( listed in your description 
).


---

[GitHub] metron pull request #863: METRON-1347: Indexing Topology should fail tuples ...

Github user ottobackwards commented on a diff in the pull request:

https://github.com/apache/metron/pull/863#discussion_r156372312
  
--- Diff: metron-platform/metron-indexing/README.md ---
@@ -15,6 +15,12 @@ Indices are written in batch and the batch size and 
batch timeout are specified
 [Sensor Indexing Configuration](#sensor-indexing-configuration) via the 
`batchSize` and `batchTimeout` parameters.
 These configs are variable by sensor type.
 
--- End diff --

Do we care, here in particular, that we have different required fields 
listed?  Should this not be cumulative and include the required fields coming 
out of the parsers?


---

[GitHub] metron pull request #863: METRON-1347: Indexing Topology should fail tuples ...