[GitHub] metron issue #862: METRON-1343: Swagger UI for User Controller needs request...

2017-12-12 Thread MohanDV
Github user MohanDV commented on the issue:

https://github.com/apache/metron/pull/862
  
Thanks @ottobackwards . Updated the manual verification steps.


---


Re: [DISCUSS] Community Meetings

2017-12-12 Thread larry mccay
Not sure about posting the recordings - you will need to check and make
sure that doesn't violate anything.

Just a friendly reminder...
It is important that meetings have notes and a summary that is sent out
describing topics to be decided on the mailing list.
No decisions can be made in the community meeting itself - this gives
others in other timezones and commitments review and voice in the decisions.

If it didn't happen on the mailing lists then it didn't happen. :)


On Tue, Dec 12, 2017 at 1:39 PM, Simon Elliston Ball <
si...@simonellistonball.com> wrote:

> Yes, I do.
>
> I suspect the best bet will be to post recordings somewhere on the
> apache.org  metron site.
>
> Simon
>
> > On 12 Dec 2017, at 18:36, Otto Fowler  wrote:
> >
> > Excellent, do you have the > 40 min + record option?
> >
> >
> > On December 12, 2017 at 13:19:55, Simon Elliston Ball (
> > si...@simonellistonball.com) wrote:
> >
> > Happy to volunteer a zoom room. That seems to have worked for most in the
> > past.
> >
> > Simon
> >
> >> On 12 Dec 2017, at 18:09, Otto Fowler  wrote:
> >>
> >> Thanks! I think I’d like something hosted though.
> >>
> >>
> >> On December 12, 2017 at 11:18:52, Ahmed Shah (
> ahmeds...@cmail.carleton.ca)
> >
> >> wrote:
> >>
> >> Hello,
> >>
> >> wrt "- How are we going to host it"...
> >>
> >> I've used BigBlueButton as an end user at our University.
> >>
> >> It is LGPL open source.
> >>
> >> https://bigbluebutton.org/
> >> https://bigbluebutton.org/developers/
> >>
> >>
> >> -Ahmed
> >>
> >> ___
> >> Ahmed Shah (PMP, M. Eng.)
> >> Cybersecurity Analyst & Developer
> >> GCR - Cybersecurity Operations Center
> >> Carleton University - cugcr.com
> >>
> >>
> >> 
> >> From: Otto Fowler 
> >> Sent: December 11, 2017 4:41 PM
> >> To: dev@metron.apache.org
> >> Subject: [DISCUSS] Community Meetings
> >>
> >> I think that we all want to have regular community meetings. We may be
> >> better able to keep to a regular schedule with these meetings if we
> > spread
> >> out the responsibility for them from James and Casey, both of whom have
> a
> >> lot on their plate already.
> >>
> >> I would be willing to coordinate and run the meetings, and would welcome
> >> anyone else who wants to help when they can.
> >>
> >> The only issue for me is I do not have a web-ex account that I can use
> to
> >> hold the meeting. So I’ll need some recommendations for a suitable
> >> alternative. I have not been able to find an Apache Friendly
> alternative,
> >> in the same way that Atlassian is apache friendly.
> >>
> >>
> >> So - from what I can see we need to:
> >>
> >> - Talk through who is going to do it
> >> - How are we going to host it
> >> - When are we going to do it
> >>
> >> Anything else?
> >>
> >> ottO
>
>


[GitHub] metron pull request #864: METRON-1348 Metron Service Checks Use Wrong Hostna...

2017-12-12 Thread nickwallen
Github user nickwallen commented on a diff in the pull request:

https://github.com/apache/metron/pull/864#discussion_r156526514
  
--- Diff: 
metron-deployment/packaging/ambari/metron-mpack/src/main/resources/common-services/METRON/CURRENT/package/scripts/params/status_params.py
 ---
@@ -80,10 +81,14 @@
 # REST
 metron_rest_port = 
config['configurations']['metron-rest-env']['metron_rest_port']
 
-# UI
-metron_management_ui_port = 
config['configurations']['metron-management-ui-env']['metron_management_ui_port']
+# Alerts UI
+metron_alerts_ui_host = default("/clusterHostInfo/metron_alerts_ui_hosts", 
[hostname])[0]
--- End diff --

I needed to somehow find the hosts running each service and I knew it was 
contained in this `clusterHostInfo` configuration.  But it was really difficult 
to uncover what values Ambari keeps in this `clusterHostInfo` configuration.  I 
have not been able to find any documentation on this.  

I actually had to add some debug statement to a live instance of Ambari to 
find out what values are stored here and how they are named.  Fun, fun.

For the record, here is what is stored in `clusterHostInfo` when spinning 
up the current state of Full Dev.
```
{  
   'snamenode_host':[  
  'node1'
   ],
   'metron_alerts_ui_hosts':[  
  'node1'
   ],
   'nm_hosts':[  
  'node1'
   ],
   'drpc_server_hosts':[  
  'node1'
   ],
   'ambari_server_use_ssl':[  
  'false'
   ],
   'all_ping_ports':[  
  '8670'
   ],
   'all_hosts':[  
  'node1'
   ],
   'rm_host':[  
  'node1'
   ],
   'kafka_broker_hosts':[  
  'node1'
   ],
   'slave_hosts':[  
  'node1'
   ],
   'metron_profiler_hosts':[  
  'node1'
   ],
   'storm_ui_server_hosts':[  
  'node1'
   ],
   'all_racks':[  
  '/default-rack'
   ],
   'all_ipv4_ips':[  
  '127.0.0.1'
   ],
   'app_timeline_server_hosts':[  
  'node1'
   ],
   'hs_host':[  
  'node1'
   ],
   'ambari_server_port':[  
  '8080'
   ],
   'metron_rest_hosts':[  
  'node1'
   ],
   'metron_management_ui_hosts':[  
  'node1'
   ],
   'es_master_hosts':[  
  'node1'
   ],
   'metron_parsers_hosts':[  
  'node1'
   ],
   'kibana_master_hosts':[  
  'node1'
   ],
   'metron_enrichment_master_hosts':[  
  'node1'
   ],
   'hbase_rs_hosts':[  
  'node1'
   ],
   'namenode_host':[  
  'node1'
   ],
   'nimbus_hosts':[  
  'node1'
   ],
   'hbase_master_hosts':[  
  'node1'
   ],
   'metron_indexing_hosts':[  
  'node1'
   ],
   'ambari_server_host':[  
  'node1'
   ],
   'zookeeper_hosts':[  
  'node1'
   ],
   'supervisor_hosts':[  
  'node1'
   ]
}
```


---


[GitHub] metron pull request #864: METRON-1348 Metron Service Checks Use Wrong Hostna...

2017-12-12 Thread nickwallen
GitHub user nickwallen opened a pull request:

https://github.com/apache/metron/pull/864

METRON-1348 Metron Service Checks Use Wrong Hostname

The Metron service check can often use the incorrect hostname when checking 
the Alerts UI, Management UI, and REST services.  This results in a failed 
service check, even when the services are running successfully.

Ambari can run the service check on any node in the cluster, not just the 
node the service is actually running on.  The service check code currently uses 
the hostname on which the service check is running.  If the service is not 
actually installed on that host, the service check will incorrectly fail.

The service check code was updated to find the hostname where the service 
is installed and use that hostname.  

## Testing

This change was tested by deploying Metron on Full Dev and running Metron > 
Service Check in Ambari.  The service check should complete successfully when 
the cluster is healthy.  The fix has also been tested on a multi-node cluster 
in the same manner.

## Pull Request Checklist

- [ ] Is there a JIRA ticket associated with this PR? If not one needs to 
be created at [Metron 
Jira](https://issues.apache.org/jira/browse/METRON/?selectedTab=com.atlassian.jira.jira-projects-plugin:summary-panel).
 
- [ ] Does your PR title start with METRON- where  is the JIRA 
number you are trying to resolve? Pay particular attention to the hyphen "-" 
character.
- [ ] Has your PR been rebased against the latest commit within the target 
branch (typically master)?
- [ ] Have you included steps to reproduce the behavior or problem that is 
being changed or addressed?
- [ ] Have you included steps or a guide to how the change may be verified 
and tested manually?
- [ ] Have you ensured that the full suite of tests and checks have been 
executed in the root metron folder via:
- [ ] Have you written or updated unit tests and or integration tests to 
verify your changes?
- [ ] If adding new dependencies to the code, are these dependencies 
licensed in a way that is compatible for inclusion under [ASF 
2.0](http://www.apache.org/legal/resolved.html#category-a)? 
- [ ] Have you verified the basic functionality of the build by building 
and running locally with Vagrant full-dev environment or the equivalent?


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/nickwallen/metron METRON-1348

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/metron/pull/864.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #864


commit 26f34d33cb4d441d25539597a93de39495507d38
Author: Nick Allen 
Date:   2017-12-11T14:25:17Z

METRON-1348 Metron Service Checks Use Wrong Hostname




---


[GitHub] metron issue #857: METRON-1340: Improve e2e tests for metron alerts

2017-12-12 Thread merrimanr
Github user merrimanr commented on the issue:

https://github.com/apache/metron/pull/857
  
I was reviewing the protractor documentation 
[here](http://www.protractortest.org/#/async-await) and I noticed this:

> Don’t forget to turn off control_flow, you cannot use a mix of 
async/await and the control flow: async/await causes the control flow to become 
unreliable (see github issue). So if you async/await anywhere in a spec, you 
should use the SELENIUM_PROMISE_MANAGER: false

I don't see that setting in protractor.conf.js.  Shouldn't it be included?  
Have all the tests been updated to use async/await instead of web driver flow 
control?


---


New Travis Build Image

2017-12-12 Thread Otto Fowler
The new Trusty image in travis that just landed breaks the build for one of
my PR’s.  I am not sure why, but I verified that going back to the old
image resolves the problem.
If you see a build fail, and it works locally or it is failing in a strange
way, be wary.

ottO


Re: [DISCUSS] Upcoming Release

2017-12-12 Thread zeo...@gmail.com
So I was looking at some of the docs and saw that Upgrading.md has a
dangling section that maybe should be removed.  Link
,
link 

Jon

On Tue, Dec 12, 2017 at 2:21 PM Matt Foley  wrote:

> Thanks to Jon, Otto, and Nick for looking over RC1.  Tonight I’ll fix the
> RAT glitch, build RC2, and put it to formal vote.
> Regards,
> --Matt
>
> On 12/12/17, 5:14 AM, "Nick Allen"  wrote:
>
> RC1 is looking good to me.  I validated the MD5s, built Metron, built
> the
> Bro plugin and reviewed the other artifacts like release notes.
>
> Running the RAT check on a 'clean' Metron does not produce any errors
> for
> me.  It is only after building Metron, which pulls in additional Node
> dependencies, does the RAT check fail.
>
>
> On Sun, Dec 10, 2017 at 4:41 PM Matt Foley  wrote:
>
> > Yes, but let’s see if anyone else find other issues.
> >
> >
> >
> > From: Otto Fowler 
> > Date: Saturday, December 9, 2017 at 6:16 AM
> > To: Matt Foley , "dev@metron.apache.org" <
> > dev@metron.apache.org>
> > Subject: Re: [DISCUSS] Upcoming Release
> >
> >
> >
> > So RC2 then?
> >
> >
> >
> > On December 8, 2017 at 20:43:21, Matt Foley (mfo...@hortonworks.com)
> > wrote:
> >
> > Hah, here it is: https://github.com/apache/metron/pull/743
> > “This problem seems to only reproduce when one unrolls a tarball
> rather
> > than cloning from github.”
> >
> > Heh, the exclusion at
> > https://github.com/apache/metron/blob/master/pom.xml#L351 is still
> there,
> > but the hashcode in the bundle.css file name has changed from
> > a0b6b99c10d9a13dc67e to f56deed131e58bd7ee04. Sigh. Did the version
> of Font
> > Awesome fonts change?
> >
> >
> > On 12/8/17, 5:26 PM, "Matt Foley"  wrote:
> >
> > I remember having trouble with this bundle.css file on the last
> release,
> > but I can’t remember what we did about it. Anybody?
> >
> > On 12/8/17, 1:41 PM, "Otto Fowler"  wrote:
> >
> > Steps
> >
> > - Downloaded tar.gz’s, asc files and KEYS
> > - Verified signing of both tar.gz’s
> > - searched for rouge 0.4.1 entries
> > - verified the main pom.xml
> > - built :
> >
> > mvn clean && time mvn -q -T 2C -DskipTests install && time mvn -q -T
> > 2C surefire:test@unit-tests && time mvn -q
> > surefire:test@integration-tests && time mvn -q test --projects
> > metron-interface/metron-config && time build_utils/verify_licenses.sh
> >
> > Found rat error:
> >
> >
> > *
> > Summary
> > ---
> > Generated at: 2017-12-08T16:33:27-05:00
> >
> > Notes: 3
> > Binaries: 193
> > Archives: 0
> > Standards: 75
> >
> > Apache Licensed: 74
> > Generated Documents: 0
> >
> > JavaDocs are generated, thus a license header is optional.
> > Generated files do not require license headers.
> >
> > 1 Unknown Licenses
> >
> > *
> >
> > Files with unapproved licenses:
> >
> >
> >
> /Users/batman/tmp/release_ver/apache-metron-0.4.2-rc1/metron-interface/metron-alerts/dist/styles.f56deed131e58bd7ee04.bundle.css
> >
> > *
> >
> >
> >
> >
> >
> > *
> > Summary
> > ---
> > Generated at: 2017-12-08T16:33:27-05:00
> >
> > Notes: 3
> > Binaries: 193
> > Archives: 0
> > Standards: 75
> >
> > Apache Licensed: 74
> > Generated Documents: 0
> >
> > JavaDocs are generated, thus a license header is optional.
> > Generated files do not require license headers.
> >
> > 1 Unknown Licenses
> >
> > *
> >
> > Files with unapproved licenses:
> >
> >
> >
> >
> /Users/ottofowler/tmp/release_ver/apache-metron-0.4.2-rc1/metron-interface/metron-alerts/dist/styles.f56deed131e58bd7ee04.bundle.css
> >
> > *
> >
> >
> >
> > On December 8, 2017 at 04:34:24, Matt Foley (ma...@apache.org)
> wrote:
> >
> > Colleagues,
> > I’ve posted Metron-0.4.2-RC1 and Metron-bro-plugin-kafka-0.1 to
> > https://dist.apache.org/repos/dist/dev/metron/0.4.2-RC1/
> >
> > Given the complexity of this RC, I’d appreciate if a couple people
> would be
> > willing to kick the tires before we put it up for a vote.
> >
> > I will myself be going thru the 

[GitHub] metron issue #857: METRON-1340: Improve e2e tests for metron alerts

2017-12-12 Thread mmiklavc
Github user mmiklavc commented on the issue:

https://github.com/apache/metron/pull/857
  
OK, I was able to get the tests to run. First run I get a single error. 
@merrimanr and I are talking through it now.

```
[INFO]   Test spec for login page
[INFO] ✗ should display error message for invalid credentials
[INFO]   - Failed: Error while waiting for Protractor to sync with the 
page: true
[INFO] .✓ should login for valid credentials
[INFO] .✓ should logout
...
[INFO] Failures:
[INFO] 1) Test spec for login page should display error message for invalid 
credentials
[INFO]   Message:
[INFO] Failed: Error while waiting for Protractor to sync with the 
page: true
[INFO]   Stack:
[INFO] Error: Error while waiting for Protractor to sync with the page: 
true
[INFO] at runWaitForAngularScript.then 
(/Users/mmiklavcic/devprojects/metron/metron-interface/metron-alerts/node_modules/protractor/built/browser.js:463:23)
[INFO] at ManagedPromise.invokeCallback_ 
(/Users/mmiklavcic/devprojects/metron/metron-interface/metron-alerts/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:1376:14)
[INFO] at TaskQueue.execute_ 
(/Users/mmiklavcic/devprojects/metron/metron-interface/metron-alerts/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:3084:14)
[INFO] at TaskQueue.executeNext_ 
(/Users/mmiklavcic/devprojects/metron/metron-interface/metron-alerts/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:3067:27)
[INFO] at asyncRun 
(/Users/mmiklavcic/devprojects/metron/metron-interface/metron-alerts/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:2927:27)
[INFO] at 
/Users/mmiklavcic/devprojects/metron/metron-interface/metron-alerts/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:668:7
[INFO] at process._tickCallback 
(internal/process/next_tick.js:109:7)
[INFO] From: Task: 
[INFO] at pollCondition 
(/Users/mmiklavcic/devprojects/metron/metron-interface/metron-alerts/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:2195:19)
[INFO] at 
/Users/mmiklavcic/devprojects/metron/metron-interface/metron-alerts/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:2191:7
[INFO] at new ManagedPromise 
(/Users/mmiklavcic/devprojects/metron/metron-interface/metron-alerts/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:1077:7)
[INFO] at ControlFlow.promise 
(/Users/mmiklavcic/devprojects/metron/metron-interface/metron-alerts/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:2505:12)
[INFO] at 
/Users/mmiklavcic/devprojects/metron/metron-interface/metron-alerts/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:2190:22
[INFO] at TaskQueue.execute_ 
(/Users/mmiklavcic/devprojects/metron/metron-interface/metron-alerts/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:3084:14)
[INFO] at TaskQueue.executeNext_ 
(/Users/mmiklavcic/devprojects/metron/metron-interface/metron-alerts/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:3067:27)
[INFO] at asyncRun 
(/Users/mmiklavcic/devprojects/metron/metron-interface/metron-alerts/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:2974:25)
[INFO] at 
/Users/mmiklavcic/devprojects/metron/metron-interface/metron-alerts/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:668:7
[INFO] at process._tickCallback 
(internal/process/next_tick.js:109:7)
[INFO] From: Task: 
[INFO] at scheduleWait 
(/Users/mmiklavcic/devprojects/metron/metron-interface/metron-alerts/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:2188:20)
[INFO] at ControlFlow.wait 
(/Users/mmiklavcic/devprojects/metron/metron-interface/metron-alerts/node_modules/protractor/node_modules/selenium-webdriver/lib/promise.js:2517:12)
[INFO] at Driver.wait 
(/Users/mmiklavcic/devprojects/metron/metron-interface/metron-alerts/node_modules/protractor/node_modules/selenium-webdriver/lib/webdriver.js:934:29)
[INFO] at run 
(/Users/mmiklavcic/devprojects/metron/metron-interface/metron-alerts/node_modules/protractor/built/browser.js:59:33)
[INFO] at ProtractorBrowser.to.(anonymous function) [as wait] 
(/Users/mmiklavcic/devprojects/metron/metron-interface/metron-alerts/node_modules/protractor/built/browser.js:67:16)
[INFO] at Object.waitForElementVisibility 
(/Users/mmiklavcic/devprojects/metron/metron-interface/metron-alerts/e2e/utils/e2e_util.ts:43:20)
[INFO] at LoginPage.setUserNameAndPassword 

[GitHub] metron issue #774: METRON-1212 The bundle system and maven plugin

2017-12-12 Thread ottobackwards
Github user ottobackwards commented on the issue:

https://github.com/apache/metron/pull/774
  
The new Trusty image breaks my build. I need to figure out why exactly, so 
I'll be trying a few things.


---


[GitHub] metron issue #857: METRON-1340: Improve e2e tests for metron alerts

2017-12-12 Thread mmiklavc
Github user mmiklavc commented on the issue:

https://github.com/apache/metron/pull/857
  
Should we also run the tests using the new Maven profile? `mvn test -Pe2e`

The README should probably be updated to reflect this as well.


---


[GitHub] metron issue #857: METRON-1340: Improve e2e tests for metron alerts

2017-12-12 Thread mmiklavc
Github user mmiklavc commented on the issue:

https://github.com/apache/metron/pull/857
  
A couple notes for the community - if your node/npm versions are out of 
date and you've already built the project, you'll run into trouble with 
versions not matching against the expected node version, e.g. `Node Sass could 
not find a binding for your current environment: OS X 64-bit with Node.js 9.x`. 
I deleted the node_modules directory entirely, `rm -r 
metron-alerts/node_modules`, re-ran the install, `npm install`, and then the 
start server script ran successfully. Just to note, the server script does not 
background the process, so you'll need to manually do that or use 2 windows to 
run the e2e tests.


---


[GitHub] metron pull request #834: METRON-1306: When index template install fails, we...

2017-12-12 Thread asfgit
Github user asfgit closed the pull request at:

https://github.com/apache/metron/pull/834


---


[GitHub] metron issue #857: METRON-1340: Improve e2e tests for metron alerts

2017-12-12 Thread mmiklavc
Github user mmiklavc commented on the issue:

https://github.com/apache/metron/pull/857
  
Seems that my NodeJS and npm versions got mangled since last time I ran 
this. Perhaps migrating laptops, upgrading to High Sierra, and switching from 
MacPorts to HomeBrew might have something to do with it ;)


---


[GitHub] metron issue #834: METRON-1306: When index template install fails, we should...

2017-12-12 Thread ottobackwards
Github user ottobackwards commented on the issue:

https://github.com/apache/metron/pull/834
  
I'm good +1


---


[GitHub] metron issue #857: METRON-1340: Improve e2e tests for metron alerts

2017-12-12 Thread mmiklavc
Github user mmiklavc commented on the issue:

https://github.com/apache/metron/pull/857
  
Something I'm missing?

```
./scripts/start-server-for-e2e.sh


/Users/mmiklavcic/devprojects/metron/metron-interface/metron-alerts/node_modules/@angular/cli/bin/ng:7
const CliConfig = require('../models/config').CliConfig;
^
SyntaxError: Use of const in strict mode.
at Module._compile (module.js:439:25)
at Object.Module._extensions..js (module.js:474:10)
at Module.load (module.js:356:32)
at Function.Module._load (module.js:312:12)
at Function.Module.runMain (module.js:497:10)
at startup (node.js:119:16)
at node.js:945:3
```


---


[GitHub] metron issue #834: METRON-1306: When index template install fails, we should...

2017-12-12 Thread cestella
Github user cestella commented on the issue:

https://github.com/apache/metron/pull/834
  
As far as THIS PR, I think it has the 2 +1s that I was waiting for.  As for 
a long-range fix, I think it's going to happen outside of this PR.  Did I miss 
anything?


---


Re: [DISCUSS] Upcoming Release

2017-12-12 Thread Matt Foley
Thanks to Jon, Otto, and Nick for looking over RC1.  Tonight I’ll fix the RAT 
glitch, build RC2, and put it to formal vote.
Regards,
--Matt

On 12/12/17, 5:14 AM, "Nick Allen"  wrote:

RC1 is looking good to me.  I validated the MD5s, built Metron, built the
Bro plugin and reviewed the other artifacts like release notes.

Running the RAT check on a 'clean' Metron does not produce any errors for
me.  It is only after building Metron, which pulls in additional Node
dependencies, does the RAT check fail.


On Sun, Dec 10, 2017 at 4:41 PM Matt Foley  wrote:

> Yes, but let’s see if anyone else find other issues.
>
>
>
> From: Otto Fowler 
> Date: Saturday, December 9, 2017 at 6:16 AM
> To: Matt Foley , "dev@metron.apache.org" <
> dev@metron.apache.org>
> Subject: Re: [DISCUSS] Upcoming Release
>
>
>
> So RC2 then?
>
>
>
> On December 8, 2017 at 20:43:21, Matt Foley (mfo...@hortonworks.com)
> wrote:
>
> Hah, here it is: https://github.com/apache/metron/pull/743
> “This problem seems to only reproduce when one unrolls a tarball rather
> than cloning from github.”
>
> Heh, the exclusion at
> https://github.com/apache/metron/blob/master/pom.xml#L351 is still there,
> but the hashcode in the bundle.css file name has changed from
> a0b6b99c10d9a13dc67e to f56deed131e58bd7ee04. Sigh. Did the version of 
Font
> Awesome fonts change?
>
>
> On 12/8/17, 5:26 PM, "Matt Foley"  wrote:
>
> I remember having trouble with this bundle.css file on the last release,
> but I can’t remember what we did about it. Anybody?
>
> On 12/8/17, 1:41 PM, "Otto Fowler"  wrote:
>
> Steps
>
> - Downloaded tar.gz’s, asc files and KEYS
> - Verified signing of both tar.gz’s
> - searched for rouge 0.4.1 entries
> - verified the main pom.xml
> - built :
>
> mvn clean && time mvn -q -T 2C -DskipTests install && time mvn -q -T
> 2C surefire:test@unit-tests && time mvn -q
> surefire:test@integration-tests && time mvn -q test --projects
> metron-interface/metron-config && time build_utils/verify_licenses.sh
>
> Found rat error:
>
>
> *
> Summary
> ---
> Generated at: 2017-12-08T16:33:27-05:00
>
> Notes: 3
> Binaries: 193
> Archives: 0
> Standards: 75
>
> Apache Licensed: 74
> Generated Documents: 0
>
> JavaDocs are generated, thus a license header is optional.
> Generated files do not require license headers.
>
> 1 Unknown Licenses
>
> *
>
> Files with unapproved licenses:
>
>
> 
/Users/batman/tmp/release_ver/apache-metron-0.4.2-rc1/metron-interface/metron-alerts/dist/styles.f56deed131e58bd7ee04.bundle.css
>
> *
>
>
>
>
>
> *
> Summary
> ---
> Generated at: 2017-12-08T16:33:27-05:00
>
> Notes: 3
> Binaries: 193
> Archives: 0
> Standards: 75
>
> Apache Licensed: 74
> Generated Documents: 0
>
> JavaDocs are generated, thus a license header is optional.
> Generated files do not require license headers.
>
> 1 Unknown Licenses
>
> *
>
> Files with unapproved licenses:
>
>
>
> 
/Users/ottofowler/tmp/release_ver/apache-metron-0.4.2-rc1/metron-interface/metron-alerts/dist/styles.f56deed131e58bd7ee04.bundle.css
>
> *
>
>
>
> On December 8, 2017 at 04:34:24, Matt Foley (ma...@apache.org) wrote:
>
> Colleagues,
> I’ve posted Metron-0.4.2-RC1 and Metron-bro-plugin-kafka-0.1 to
> https://dist.apache.org/repos/dist/dev/metron/0.4.2-RC1/
>
> Given the complexity of this RC, I’d appreciate if a couple people would 
be
> willing to kick the tires before we put it up for a vote.
>
> I will myself be going thru the Verify Build process this weekend, as I
> won’t be able to do it Friday.
>
> Thanks,
> --Matt
>
>
> On 12/4/17, 2:05 PM, "zeo...@gmail.com"  wrote:
>
> Can we resolve the conversation regarding the second repo? I was waiting
> to get more input/preferences from people There's also a documentation
> update that fixes a few broken Stellar docs that already has aa +1, I just
> need to merge it.
>
> Jon
>
> On Mon, Dec 4, 2017, 17:01 Casey Stella  wrote:
>
> > I would be in favor of a 

“777” Feature Branch Redux

2017-12-12 Thread Otto Fowler
I have created a new feature branch
feature/METRON-1211-extensions-parsers-gradual to track the parser
extension work
and have rebased https://github.com/apache/metron/pull/774 on to that.

I have also updated confluence and jira :
https://cwiki.apache.org/confluence/display/METRON/Metron+Extension+System+and+Parser+Extensions

1. Feature Branch still makes sense
2. Now that we are splitting it up it will work better than the “whole
boat” approach to the original branch
3. I don't want to have to worry about regression for short periods as I
re-implement

Cheers.


[GitHub] metron issue #774: METRON-1212 The bundle system and maven plugin

2017-12-12 Thread ottobackwards
Github user ottobackwards commented on the issue:

https://github.com/apache/metron/pull/774
  
I have created a new feature branch 
feature/METRON-1211-extensions-parsers-gradual to track this.
and have rebased this PR on to that.

I have also updated confluence and jira : 
https://cwiki.apache.org/confluence/display/METRON/Metron+Extension+System+and+Parser+Extensions

1. Feature Branch still makes sense
2. now that we are splitting it up it will work better
3. I don't want to have to worry about regression for short periods

Will email list


---


[GitHub] metron issue #834: METRON-1306: When index template install fails, we should...

2017-12-12 Thread ottobackwards
Github user ottobackwards commented on the issue:

https://github.com/apache/metron/pull/834
  
I expect to address this in the "777" feature branch/ parser effort.


---


[GitHub] metron issue #834: METRON-1306: When index template install fails, we should...

2017-12-12 Thread merrimanr
Github user merrimanr commented on the issue:

https://github.com/apache/metron/pull/834
  
After reading this whole thread I agree with @ottobackwards.  If we 
installed and started all services and THEN installed our parsers separately 
this whole issue goes away.  If a parser's template doesn't exist or isn't 
installed correctly, it fails and hopefully we know exactly why.  That is 
obviously a major architectural change and won't be solved in this PR.  

Since we're in a situation where we do include default parsers as part of 
the installation, I think they should work out of the box and fail fast if 
there are not installed correctly.  Would some of the concerns be alleviated 
with https://github.com/apache/metron/pull/831?  

+1 from me either way.  


---


Re: [DISCUSS] Community Meetings

2017-12-12 Thread Simon Elliston Ball
Yes, I do. 

I suspect the best bet will be to post recordings somewhere on the apache.org 
 metron site.

Simon

> On 12 Dec 2017, at 18:36, Otto Fowler  wrote:
> 
> Excellent, do you have the > 40 min + record option?
> 
> 
> On December 12, 2017 at 13:19:55, Simon Elliston Ball (
> si...@simonellistonball.com) wrote:
> 
> Happy to volunteer a zoom room. That seems to have worked for most in the
> past.
> 
> Simon
> 
>> On 12 Dec 2017, at 18:09, Otto Fowler  wrote:
>> 
>> Thanks! I think I’d like something hosted though.
>> 
>> 
>> On December 12, 2017 at 11:18:52, Ahmed Shah (ahmeds...@cmail.carleton.ca)
> 
>> wrote:
>> 
>> Hello,
>> 
>> wrt "- How are we going to host it"...
>> 
>> I've used BigBlueButton as an end user at our University.
>> 
>> It is LGPL open source.
>> 
>> https://bigbluebutton.org/
>> https://bigbluebutton.org/developers/
>> 
>> 
>> -Ahmed
>> 
>> ___
>> Ahmed Shah (PMP, M. Eng.)
>> Cybersecurity Analyst & Developer
>> GCR - Cybersecurity Operations Center
>> Carleton University - cugcr.com
>> 
>> 
>> 
>> From: Otto Fowler 
>> Sent: December 11, 2017 4:41 PM
>> To: dev@metron.apache.org
>> Subject: [DISCUSS] Community Meetings
>> 
>> I think that we all want to have regular community meetings. We may be
>> better able to keep to a regular schedule with these meetings if we
> spread
>> out the responsibility for them from James and Casey, both of whom have a
>> lot on their plate already.
>> 
>> I would be willing to coordinate and run the meetings, and would welcome
>> anyone else who wants to help when they can.
>> 
>> The only issue for me is I do not have a web-ex account that I can use to
>> hold the meeting. So I’ll need some recommendations for a suitable
>> alternative. I have not been able to find an Apache Friendly alternative,
>> in the same way that Atlassian is apache friendly.
>> 
>> 
>> So - from what I can see we need to:
>> 
>> - Talk through who is going to do it
>> - How are we going to host it
>> - When are we going to do it
>> 
>> Anything else?
>> 
>> ottO



Re: [DISCUSS] Community Meetings

2017-12-12 Thread Otto Fowler
Excellent, do you have the > 40 min + record option?


On December 12, 2017 at 13:19:55, Simon Elliston Ball (
si...@simonellistonball.com) wrote:

Happy to volunteer a zoom room. That seems to have worked for most in the
past.

Simon

> On 12 Dec 2017, at 18:09, Otto Fowler  wrote:
>
> Thanks! I think I’d like something hosted though.
>
>
> On December 12, 2017 at 11:18:52, Ahmed Shah (ahmeds...@cmail.carleton.ca)

> wrote:
>
> Hello,
>
> wrt "- How are we going to host it"...
>
> I've used BigBlueButton as an end user at our University.
>
> It is LGPL open source.
>
> https://bigbluebutton.org/
> https://bigbluebutton.org/developers/
>
>
> -Ahmed
>
> ___
> Ahmed Shah (PMP, M. Eng.)
> Cybersecurity Analyst & Developer
> GCR - Cybersecurity Operations Center
> Carleton University - cugcr.com
>
>
> 
> From: Otto Fowler 
> Sent: December 11, 2017 4:41 PM
> To: dev@metron.apache.org
> Subject: [DISCUSS] Community Meetings
>
> I think that we all want to have regular community meetings. We may be
> better able to keep to a regular schedule with these meetings if we
spread
> out the responsibility for them from James and Casey, both of whom have a
> lot on their plate already.
>
> I would be willing to coordinate and run the meetings, and would welcome
> anyone else who wants to help when they can.
>
> The only issue for me is I do not have a web-ex account that I can use to
> hold the meeting. So I’ll need some recommendations for a suitable
> alternative. I have not been able to find an Apache Friendly alternative,
> in the same way that Atlassian is apache friendly.
>
>
> So - from what I can see we need to:
>
> - Talk through who is going to do it
> - How are we going to host it
> - When are we going to do it
>
> Anything else?
>
> ottO


Re: [DISCUSS] Community Meetings

2017-12-12 Thread Simon Elliston Ball
Happy to volunteer a zoom room. That seems to have worked for most in the past.

Simon

> On 12 Dec 2017, at 18:09, Otto Fowler  wrote:
> 
> Thanks!  I think I’d like something hosted though.
> 
> 
> On December 12, 2017 at 11:18:52, Ahmed Shah (ahmeds...@cmail.carleton.ca)
> wrote:
> 
> Hello,
> 
> wrt "- How are we going to host it"...
> 
> I've used BigBlueButton as an end user at our University.
> 
> It is LGPL open source.
> 
> https://bigbluebutton.org/
> https://bigbluebutton.org/developers/
> 
> 
> -Ahmed
> 
> ___
> Ahmed Shah (PMP, M. Eng.)
> Cybersecurity Analyst & Developer
> GCR - Cybersecurity Operations Center
> Carleton University - cugcr.com
> 
> 
> 
> From: Otto Fowler 
> Sent: December 11, 2017 4:41 PM
> To: dev@metron.apache.org
> Subject: [DISCUSS] Community Meetings
> 
> I think that we all want to have regular community meetings. We may be
> better able to keep to a regular schedule with these meetings if we spread
> out the responsibility for them from James and Casey, both of whom have a
> lot on their plate already.
> 
> I would be willing to coordinate and run the meetings, and would welcome
> anyone else who wants to help when they can.
> 
> The only issue for me is I do not have a web-ex account that I can use to
> hold the meeting. So I’ll need some recommendations for a suitable
> alternative. I have not been able to find an Apache Friendly alternative,
> in the same way that Atlassian is apache friendly.
> 
> 
> So - from what I can see we need to:
> 
> - Talk through who is going to do it
> - How are we going to host it
> - When are we going to do it
> 
> Anything else?
> 
> ottO



Re: [DISCUSS] Community Meetings

2017-12-12 Thread Otto Fowler
Thanks!  I think I’d like something hosted though.


On December 12, 2017 at 11:18:52, Ahmed Shah (ahmeds...@cmail.carleton.ca)
wrote:

Hello,

wrt "- How are we going to host it"...

I've used BigBlueButton as an end user at our University.

It is LGPL open source.

https://bigbluebutton.org/
https://bigbluebutton.org/developers/


-Ahmed

___
Ahmed Shah (PMP, M. Eng.)
Cybersecurity Analyst & Developer
GCR - Cybersecurity Operations Center
Carleton University - cugcr.com



From: Otto Fowler 
Sent: December 11, 2017 4:41 PM
To: dev@metron.apache.org
Subject: [DISCUSS] Community Meetings

I think that we all want to have regular community meetings. We may be
better able to keep to a regular schedule with these meetings if we spread
out the responsibility for them from James and Casey, both of whom have a
lot on their plate already.

I would be willing to coordinate and run the meetings, and would welcome
anyone else who wants to help when they can.

The only issue for me is I do not have a web-ex account that I can use to
hold the meeting. So I’ll need some recommendations for a suitable
alternative. I have not been able to find an Apache Friendly alternative,
in the same way that Atlassian is apache friendly.


So - from what I can see we need to:

- Talk through who is going to do it
- How are we going to host it
- When are we going to do it

Anything else?

ottO


Re: [DISCUSS] Community Meetings

2017-12-12 Thread Ahmed Shah
Hello,

wrt "- How are we going to host it"...

I've used BigBlueButton as an end user at our University.

It is LGPL open source.

https://bigbluebutton.org/
https://bigbluebutton.org/developers/


-Ahmed

___
Ahmed Shah (PMP, M. Eng.)
Cybersecurity Analyst & Developer
GCR - Cybersecurity Operations Center
Carleton University - cugcr.com



From: Otto Fowler 
Sent: December 11, 2017 4:41 PM
To: dev@metron.apache.org
Subject: [DISCUSS] Community Meetings

I think that we all want to have regular community meetings.  We may be
better able to keep to a regular schedule with these meetings if we spread
out the responsibility for them from James and Casey, both of whom have a
lot on their plate already.

I would be willing to coordinate and run the meetings, and would welcome
anyone else who wants to help when they can.

The only issue for me is I do not have a web-ex account that I can use to
hold the meeting.  So I’ll need some recommendations for a suitable
alternative.  I have not been able to find an Apache Friendly alternative,
in the same way that Atlassian is apache friendly.


So - from what I can see we need to:

- Talk through who is going to do it
- How are we going to host it
- When are we going to do it

Anything else?

ottO


[GitHub] metron issue #853: METRON-1337: List of facets should not be hardcoded

2017-12-12 Thread merrimanr
Github user merrimanr commented on the issue:

https://github.com/apache/metron/pull/853
  
Ambari does not manage this config.  It is only included in the base 
application.yml as a default setting.  I don't feel like this setting should be 
in Ambari for a couple reasons:  changing it requires a restart and the list of 
facets cannot be user specific.



---


[GitHub] metron issue #853: METRON-1337: List of facets should not be hardcoded

2017-12-12 Thread ottobackwards
Github user ottobackwards commented on the issue:

https://github.com/apache/metron/pull/853
  
What I get to was, the setting should be exposed in ambari if ambari 
manages the config.


---


[GitHub] metron pull request #858: METRON-1344: Externalize the infrastructural compo...

2017-12-12 Thread merrimanr
GitHub user merrimanr reopened a pull request:

https://github.com/apache/metron/pull/858

METRON-1344: Externalize the infrastructural components using integration 
tests

## Contributor Comments
This PR will add infrastructure to our Travis build that will allow the 
Alerts UI e2e test to be run.  There are several outstanding issues that still 
need to be worked out so DO NOT MERGE this yet.

This is a first pass at a potential Docker-based solution and is meant to 
be a POC.  The intention is to facilitate further discussion around the general 
approach and provide a working example that we can build off of.  

A good place to start is the .travis file.  This provides a good guide on 
how this infrastructure is spun up.  It is assumed Docker and Docker Compose 
are installed.  To use outside of travis in a local dev environment (assuming 
Mac OSX): 

1. Build Metron with `mvn clean install -DskipTests`
1. Create a Docker machine with 
`metron-contrib/metron-docker/scripts/create-docker-machine.sh`
1. Set the Docker env variables with `eval $(docker-machine env 
metron-machine)`
1. Build the base Metron image that installs Java:  `docker build 
./metron-centos/ -t "metron-centos"`
1.  Spin up the environment:  `cd metron-contrib/metron-docker-e2e/compose 
&& docker-compose up -d`

A working environment should now be available.  To verify get the Docker 
machine address with `echo $DOCKER_HOST`.  Services should be available on the 
ports specified in 
`metron-contrib/metron-docker-e2e/compose/docker-compose.yml`.  For example, 
assuming my $DOCKER_HOST is "tcp://192.168.99.100:2376", Elasticsearch should 
be available at http://192.168.99.100:9210.

At this point only a single e2e test is run due to the significant 
refactoring being done in https://github.com/apache/metron/pull/857.  

Here are my thoughts so far based on work in this PR:

- The Docker environment creation and startup adds about 3 minutes to the 
build.  This is about what I expected and hopefully we can get this time back 
as we move other tests to a reusable environment.
- I'm not convinced spinning up containers for the Alerts UI and REST are 
necessary or desired.  We may be able to cut some time off the build by running 
them directly on the Travis host instead of in Docker
- I experimented with caching the Docker images but pulling them every time 
was actually faster

There is still significant work to be done including:
- I was able to get the full suite of tests to run successfully in previous 
commits but have all but e2e tests commented out for now to make it easier to 
see what's going on in Travis.  These will need to be added back once we get 
closer to a final solution.
- Once the e2e test refactoring is done those changes will need merged in 
and tested
- The license check is commented out right now because I inadvertently 
added some new dependency versions (have no idea why).  Still need to track 
that down.
- There is an intermittent error that happens when starting up REST.  Still 
working on tracking this down.
 
Looking forward to some feedback.

## Pull Request Checklist

Thank you for submitting a contribution to Apache Metron.  
Please refer to our [Development 
Guidelines](https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=61332235)
 for the complete guide to follow for contributions.  
Please refer also to our [Build Verification 
Guidelines](https://cwiki.apache.org/confluence/display/METRON/Verifying+Builds?show-miniview)
 for complete smoke testing guides.  


In order to streamline the review of the contribution we ask you follow 
these guidelines and ask you to double check the following:

### For all changes:
- [x] Is there a JIRA ticket associated with this PR? If not one needs to 
be created at [Metron 
Jira](https://issues.apache.org/jira/browse/METRON/?selectedTab=com.atlassian.jira.jira-projects-plugin:summary-panel).
 
- [x] Does your PR title start with METRON- where  is the JIRA 
number you are trying to resolve? Pay particular attention to the hyphen "-" 
character.
- [ ] Has your PR been rebased against the latest commit within the target 
branch (typically master)?


### For code changes:
- [ ] Have you included steps to reproduce the behavior or problem that is 
being changed or addressed?
- [ ] Have you included steps or a guide to how the change may be verified 
and tested manually?
- [ ] Have you ensured that the full suite of tests and checks have been 
executed in the root metron folder via:
  ```
  mvn -q clean integration-test install && build_utils/verify_licenses.sh 
  ```

- [ ] Have you written or updated unit tests and or integration tests to 
verify your changes?
- [ ] If adding new dependencies to the code, are these 

[GitHub] metron pull request #858: METRON-1344: Externalize the infrastructural compo...

2017-12-12 Thread merrimanr
Github user merrimanr closed the pull request at:

https://github.com/apache/metron/pull/858


---


[GitHub] metron issue #858: METRON-1344: Externalize the infrastructural components u...

2017-12-12 Thread ottobackwards
Github user ottobackwards commented on the issue:

https://github.com/apache/metron/pull/858
  
@merrimanr, I have created labels in jira that are applicable to feature 
branch Jiras,
This will allow for tracking these jiras in confluence as I have done here 
: 
https://cwiki.apache.org/confluence/display/METRON/Metron+Extension+System+and+Parser+Extensions.

I would suggest creating a page for this feature branch were we can 
document things in this area, and using the labels on the liras

metron-feature-canidate, {create a metron-feature-externalize-? label}


---


[GitHub] metron issue #853: METRON-1337: List of facets should not be hardcoded

2017-12-12 Thread merrimanr
Github user merrimanr commented on the issue:

https://github.com/apache/metron/pull/853
  
Each environment will have it's own application.yaml.  Full dev has one, 
our testing environment has one, Ambari ships one etc.  There is also a base 
application.yaml that has the defaults (default facets are included here).  
Does that make sense?


---


[GitHub] metron issue #579: METRON-941 fix PaloAltoParser

2017-12-12 Thread ottobackwards
Github user ottobackwards commented on the issue:

https://github.com/apache/metron/pull/579
  
@ctramnitz is there any update on this PR?  Is there something we can do to 
help?


---


[GitHub] metron issue #834: METRON-1306: When index template install fails, we should...

2017-12-12 Thread ottobackwards
Github user ottobackwards commented on the issue:

https://github.com/apache/metron/pull/834
  
Where are we at with this?


---


[GitHub] metron issue #853: METRON-1337: List of facets should not be hardcoded

2017-12-12 Thread ottobackwards
Github user ottobackwards commented on the issue:

https://github.com/apache/metron/pull/853
  
Does ambari own the application.yaml currently?


---


[GitHub] metron issue #862: METRON-1343: Swagger UI for User Controller needs request...

2017-12-12 Thread ottobackwards
Github user ottobackwards commented on the issue:

https://github.com/apache/metron/pull/862
  
Like, 

- Spin up Full Dev
- Go to Swagger at http://node1:4200(?)
- You should see X now


---


[GitHub] metron issue #862: METRON-1343: Swagger UI for User Controller needs request...

2017-12-12 Thread ottobackwards
Github user ottobackwards commented on the issue:

https://github.com/apache/metron/pull/862
  
Great, can you list that as the manual verification please?


---


[GitHub] metron issue #862: METRON-1343: Swagger UI for User Controller needs request...

2017-12-12 Thread MohanDV
Github user MohanDV commented on the issue:

https://github.com/apache/metron/pull/862
  
Hi @ottobackwards , we don't need new test. Currently on the Swagger UI we 
see that API '/api/v1/user' is listed with all the Request methods. After this 
fix we will see it will be listed for only GET request method. I verified by 
building the full dev with this change. 


---


[GitHub] metron issue #854: Experimental Improvements - Feedback Only - Do Not Merge

2017-12-12 Thread ottobackwards
Github user ottobackwards commented on the issue:

https://github.com/apache/metron/pull/854
  
based on @justinleet 's comment, where do we stand with this


---


[GitHub] metron issue #856: METRON-1339 Stellar Shell functionality to verify stored ...

2017-12-12 Thread ottobackwards
Github user ottobackwards commented on the issue:

https://github.com/apache/metron/pull/856
  
@nickwallen any feedback, does the annotated approach match what you 
imagined?


---


[GitHub] metron pull request #859: METRON-1345: Update EC2 README for custom Ansible ...

2017-12-12 Thread ottobackwards
Github user ottobackwards commented on a diff in the pull request:

https://github.com/apache/metron/pull/859#discussion_r156374333
  
--- Diff: metron-deployment/roles/ambari_config/vars/small_cluster.yml ---
@@ -87,6 +87,8 @@ configurations:
   topology.classpath: '{{ topology_classpath }}'
   - kafka-broker:
   log.dirs: '{{ kafka_log_dirs | default("/kafka-log") }}'
--- End diff --

Should the EC2 Readme include documentation about the dev env and what it 
means?  Should it link to or document how to  change from dev to 'other', 
or do other from the start?


---


[GitHub] metron issue #862: METRON-1343: Swagger UI for User Controller needs request...

2017-12-12 Thread ottobackwards
Github user ottobackwards commented on the issue:

https://github.com/apache/metron/pull/862
  
Hi!  Thanks for the contribution.
How can we test these changes?  
Do we need new tests in the code for this change ?


Please consider the items in the PR checklist ( listed in your description 
).


---


[GitHub] metron pull request #863: METRON-1347: Indexing Topology should fail tuples ...

2017-12-12 Thread ottobackwards
Github user ottobackwards commented on a diff in the pull request:

https://github.com/apache/metron/pull/863#discussion_r156372312
  
--- Diff: metron-platform/metron-indexing/README.md ---
@@ -15,6 +15,12 @@ Indices are written in batch and the batch size and 
batch timeout are specified
 [Sensor Indexing Configuration](#sensor-indexing-configuration) via the 
`batchSize` and `batchTimeout` parameters.
 These configs are variable by sensor type.
 
--- End diff --

Do we care, here in particular, that we have different required fields 
listed?  Should this not be cumulative and include the required fields coming 
out of the parsers?


---


[GitHub] metron pull request #863: METRON-1347: Indexing Topology should fail tuples ...

2017-12-12 Thread ottobackwards
Github user ottobackwards commented on a diff in the pull request:

https://github.com/apache/metron/pull/863#discussion_r156372868
  
--- Diff: 
metron-platform/metron-writer/src/main/java/org/apache/metron/writer/bolt/BulkMessageWriterBolt.java
 ---
@@ -229,17 +239,30 @@ public void execute(Tuple tuple) {
   LOG.trace("Writing enrichment message: {}", message);
   WriterConfiguration writerConfiguration = 
configurationTransformation.apply(
   new IndexingWriterConfiguration(bulkMessageWriter.getName(), 
getConfigurations()));
-  if(writerConfiguration.isDefault(sensorType)) {
-//want to warn, but not fail the tuple
-collector.reportError(new Exception("WARNING: Default and (likely) 
unoptimized writer config used for " + bulkMessageWriter.getName() + " writer 
and sensor " + sensorType));
+  if(sensorType == null) {
--- End diff --

Maybe we should validate all the required fields?


---


[GitHub] metron issue #859: METRON-1345: Update EC2 README for custom Ansible tags

2017-12-12 Thread nickwallen
Github user nickwallen commented on the issue:

https://github.com/apache/metron/pull/859
  
Nice.  I'm glad you uncovered the issue with the REST API.  Reaffirming +1


---


Re: [DISCUSS] Upcoming Release

2017-12-12 Thread Nick Allen
RC1 is looking good to me.  I validated the MD5s, built Metron, built the
Bro plugin and reviewed the other artifacts like release notes.

Running the RAT check on a 'clean' Metron does not produce any errors for
me.  It is only after building Metron, which pulls in additional Node
dependencies, does the RAT check fail.



On Sun, Dec 10, 2017 at 4:41 PM Matt Foley  wrote:

> Yes, but let’s see if anyone else find other issues.
>
>
>
> From: Otto Fowler 
> Date: Saturday, December 9, 2017 at 6:16 AM
> To: Matt Foley , "dev@metron.apache.org" <
> dev@metron.apache.org>
> Subject: Re: [DISCUSS] Upcoming Release
>
>
>
> So RC2 then?
>
>
>
> On December 8, 2017 at 20:43:21, Matt Foley (mfo...@hortonworks.com)
> wrote:
>
> Hah, here it is: https://github.com/apache/metron/pull/743
> “This problem seems to only reproduce when one unrolls a tarball rather
> than cloning from github.”
>
> Heh, the exclusion at
> https://github.com/apache/metron/blob/master/pom.xml#L351 is still there,
> but the hashcode in the bundle.css file name has changed from
> a0b6b99c10d9a13dc67e to f56deed131e58bd7ee04. Sigh. Did the version of Font
> Awesome fonts change?
>
>
> On 12/8/17, 5:26 PM, "Matt Foley"  wrote:
>
> I remember having trouble with this bundle.css file on the last release,
> but I can’t remember what we did about it. Anybody?
>
> On 12/8/17, 1:41 PM, "Otto Fowler"  wrote:
>
> Steps
>
> - Downloaded tar.gz’s, asc files and KEYS
> - Verified signing of both tar.gz’s
> - searched for rouge 0.4.1 entries
> - verified the main pom.xml
> - built :
>
> mvn clean && time mvn -q -T 2C -DskipTests install && time mvn -q -T
> 2C surefire:test@unit-tests && time mvn -q
> surefire:test@integration-tests && time mvn -q test --projects
> metron-interface/metron-config && time build_utils/verify_licenses.sh
>
> Found rat error:
>
>
> *
> Summary
> ---
> Generated at: 2017-12-08T16:33:27-05:00
>
> Notes: 3
> Binaries: 193
> Archives: 0
> Standards: 75
>
> Apache Licensed: 74
> Generated Documents: 0
>
> JavaDocs are generated, thus a license header is optional.
> Generated files do not require license headers.
>
> 1 Unknown Licenses
>
> *
>
> Files with unapproved licenses:
>
>
> /Users/batman/tmp/release_ver/apache-metron-0.4.2-rc1/metron-interface/metron-alerts/dist/styles.f56deed131e58bd7ee04.bundle.css
>
> *
>
>
>
>
>
> *
> Summary
> ---
> Generated at: 2017-12-08T16:33:27-05:00
>
> Notes: 3
> Binaries: 193
> Archives: 0
> Standards: 75
>
> Apache Licensed: 74
> Generated Documents: 0
>
> JavaDocs are generated, thus a license header is optional.
> Generated files do not require license headers.
>
> 1 Unknown Licenses
>
> *
>
> Files with unapproved licenses:
>
>
>
> /Users/ottofowler/tmp/release_ver/apache-metron-0.4.2-rc1/metron-interface/metron-alerts/dist/styles.f56deed131e58bd7ee04.bundle.css
>
> *
>
>
>
> On December 8, 2017 at 04:34:24, Matt Foley (ma...@apache.org) wrote:
>
> Colleagues,
> I’ve posted Metron-0.4.2-RC1 and Metron-bro-plugin-kafka-0.1 to
> https://dist.apache.org/repos/dist/dev/metron/0.4.2-RC1/
>
> Given the complexity of this RC, I’d appreciate if a couple people would be
> willing to kick the tires before we put it up for a vote.
>
> I will myself be going thru the Verify Build process this weekend, as I
> won’t be able to do it Friday.
>
> Thanks,
> --Matt
>
>
> On 12/4/17, 2:05 PM, "zeo...@gmail.com"  wrote:
>
> Can we resolve the conversation regarding the second repo? I was waiting
> to get more input/preferences from people There's also a documentation
> update that fixes a few broken Stellar docs that already has aa +1, I just
> need to merge it.
>
> Jon
>
> On Mon, Dec 4, 2017, 17:01 Casey Stella  wrote:
>
> > I would be in favor of a release at this point.
> >
> > On Mon, Dec 4, 2017 at 4:57 PM, Matt Foley  wrote:
> >
> > > Hey all,
> > > I see METRON-1252 was resolved over the weekend. Shall I go ahead and
> > > start the process with 0.4.2 release?
> > > Does anyone have any commits they feel strongly should go in before
> 0.4.2
> > > is done, or are we ready to call it good?
> > >
> > > I believe there is consensus the 0.4.2 release should include a release
> > of
> > > the current state of the metron-bro-plugin-kafka. I will continue the
> > > discussion in that thread as to the process for accomplishing that, but
> > > plan on it happening.
> > >
> > > Regards,
> > > --Matt
> > >
> > > On 11/26/17, 6:26 PM, "Matt Foley"  wrote:
> > >
> > > Hope everyone (at least in the U.S.) had a great