Re: feature branch bumps

[Otto Fowler]

Omit

On September 20, 2017 at 19:20:27, Otto Fowler (ottobackwa...@gmail.com)
wrote:

> With the bundles, archetype, plug-in and parser moves gone 777 will be
> much smaller though.
>
> Also I did not mean to admit Mike’s doc reviews, sorry.
>
> On September 20, 2017 at 18:04:20, Otto Fowler (ottobackwa...@gmail.com)
> wrote:
>
>> I am not taking any of this as starting to do a flame war :)  And your
>> concern is about review not merging right?  If it is all reviewed and
>> accepted, why would 6 merges be preferable?  That doesn’t make sense.
>> Breaking it up only makes sense with regards to review, so that is how
>> I’ll take it.
>>
>>  And I *am* all ears for how to break up the code.  Like literally HOW.
>> Like : ‘I would diff a folder with master, create a new branch for the pr
>> and patch just that’.
>> Just 'break up the massive working thing into smaller chucks' is not
>> enough for my relative experience with git and github to you Nick.
>>
>> So if all we want are code review chunks, regardless of being able to run
>> them and see them work then ( but also not break the regular build and
>> full_dev ):
>>
>> 1. PR for Bundle-Lib And Maven Plugin ( but that is already reviewed )
>> code review only, doesn’t do anything
>> 2. PR for Archetype ( just building the result, unit integration tests )
>> code review and generate project review.   projects built by archetype :
>> => not buildable not installable not testable
>> relies on changes to parser testing etc. to work with non-hard coded
>> paths for test resources etc etc
>> 3. PR with Parsers copied to extensions
>> code review only.  not buildable not installable not testable
>> 4. PR with the rest of 777 + grok fixes ( all non ui non rest ) -> move
>> to new parser structure, load install, grok support etc etc.
>> regression testable, status quo ante functionality.
>> still huge
>> will still be a problem
>> cannot be broken down and built / pass travis at this point
>> this is the integration
>> 5. PR of Rest
>> can install and uninstall via rest
>> 6. PR of UI
>> can list, install uninstall via UI
>>
>> I did not think then that we wanted PRs that don’t build or test or run
>> the changes they have ( but the branch builds and passes travis ), but just
>> have code to review.
>>
>> But, as you said, it has been since April, and only Bundles has been
>> reviewed. If nobody but Matt is even going to _attempt_ code review, then
>> it may now be implicitly required that I do this.
>>
>>
>> On September 20, 2017 at 16:41:46, Nick Allen (n...@nickallen.org) wrote:
>>
>> I was just responding to your statement "Well, if there is an
>> alternative merge strategy, I’m all ears."  I understand that you disagree,
>> but this is an alternative strategy.  I think this approach is feasible and
>> I outlined how here [1].
>>
>> The criticism of this approach is the amount of effort and time to do
>> this, which I agree with.  It will take a good deal of effort.
>> I acquiesced to this criticism at the time because I thought we were trying
>> to push this change in quickly.  But it has been a couple months now and we
>> still have a bunch of code to merge.
>>
>> This is no criticism of you or anyone here.  What Otto has been doing in
>> keeping this branch merged with master is god damn heroic.  And I am so
>> excited to see this functionality hit master.  I just wanted to brainstorm
>> on ideas to help get this in.  Right now, I am unsure of the path and
>> timeline on how we are going to get this merged.
>>
>> Please read this in the kindest possible light.  I am not trying to start
>> a flame war or offend anyone.
>>
>>
>> [1] https://github.com/apache/metron/pull/530#issuecomment-306806217
>>
>>
>>
>> On Wed, Sep 20, 2017 at 3:37 PM Otto Fowler 
>> wrote:
>>
>>> “Bite off small chunks” is what I keep hearing. I have no idea how to do
>>> that from an already integrated and working branch.
>>> Do you mean I…
>>> - create patch files of whole directories and do a pr per directory, but
>>> the build doesn’t work?
>>>
>>>
>>> On September 20, 2017 at 15:07:56, Nick Allen (n...@nickallen.org)
>>> wrote:
>>>
>>> > Otto: Well, if there is an alternative merge strategy, I’m all ears.
>>>
>>> Yes, the alternative strategy is what I mentioned. :)  Copied in below.
>>>
>>> > Nick: Are you going to bite-off small chunks of the feature branch
>>> and introduce PRs against master?
>>>
>>>
>>>
>>>
>>>
>>> On Wed, Sep 20, 2017 at 12:02 PM Otto Fowler 
>>> wrote:
>>>
 Well, if there is an alternative merge strategy, I’m all ears.
 As it stands, the only complete code / conceptual review has been the
 bundles-lib by mattf.
 So there is still a great deal of review activity to do.

 Also conceptually there is ( might not be complete )

 * the discuss thread topic from this morning ( metron parsers v.
 extensions wrt registration and management )
 * default configurations ( parser, 

Re: feature branch bumps

[Otto Fowler]

With the bundles, archetype, plug-in and parser moves gone 777 will be much
smaller though.

Also I did not mean to admit Mike’s doc reviews, sorry.

On September 20, 2017 at 18:04:20, Otto Fowler (ottobackwa...@gmail.com)
wrote:

> I am not taking any of this as starting to do a flame war :)  And your
> concern is about review not merging right?  If it is all reviewed and
> accepted, why would 6 merges be preferable?  That doesn’t make sense.
> Breaking it up only makes sense with regards to review, so that is how
> I’ll take it.
>
>  And I *am* all ears for how to break up the code.  Like literally HOW.
> Like : ‘I would diff a folder with master, create a new branch for the pr
> and patch just that’.
> Just 'break up the massive working thing into smaller chucks' is not
> enough for my relative experience with git and github to you Nick.
>
> So if all we want are code review chunks, regardless of being able to run
> them and see them work then ( but also not break the regular build and
> full_dev ):
>
> 1. PR for Bundle-Lib And Maven Plugin ( but that is already reviewed )
> code review only, doesn’t do anything
> 2. PR for Archetype ( just building the result, unit integration tests )
> code review and generate project review.   projects built by archetype :
> => not buildable not installable not testable
> relies on changes to parser testing etc. to work with non-hard coded paths
> for test resources etc etc
> 3. PR with Parsers copied to extensions
> code review only.  not buildable not installable not testable
> 4. PR with the rest of 777 + grok fixes ( all non ui non rest ) -> move to
> new parser structure, load install, grok support etc etc.
> regression testable, status quo ante functionality.
> still huge
> will still be a problem
> cannot be broken down and built / pass travis at this point
> this is the integration
> 5. PR of Rest
> can install and uninstall via rest
> 6. PR of UI
> can list, install uninstall via UI
>
> I did not think then that we wanted PRs that don’t build or test or run
> the changes they have ( but the branch builds and passes travis ), but just
> have code to review.
>
> But, as you said, it has been since April, and only Bundles has been
> reviewed. If nobody but Matt is even going to _attempt_ code review, then
> it may now be implicitly required that I do this.
>
>
> On September 20, 2017 at 16:41:46, Nick Allen (n...@nickallen.org) wrote:
>
> I was just responding to your statement "Well, if there is an alternative
> merge strategy, I’m all ears."  I understand that you disagree, but this is
> an alternative strategy.  I think this approach is feasible and I outlined
> how here [1].
>
> The criticism of this approach is the amount of effort and time to do
> this, which I agree with.  It will take a good deal of effort.
> I acquiesced to this criticism at the time because I thought we were trying
> to push this change in quickly.  But it has been a couple months now and we
> still have a bunch of code to merge.
>
> This is no criticism of you or anyone here.  What Otto has been doing in
> keeping this branch merged with master is god damn heroic.  And I am so
> excited to see this functionality hit master.  I just wanted to brainstorm
> on ideas to help get this in.  Right now, I am unsure of the path and
> timeline on how we are going to get this merged.
>
> Please read this in the kindest possible light.  I am not trying to start
> a flame war or offend anyone.
>
>
> [1] https://github.com/apache/metron/pull/530#issuecomment-306806217
>
>
>
> On Wed, Sep 20, 2017 at 3:37 PM Otto Fowler 
> wrote:
>
>> “Bite off small chunks” is what I keep hearing. I have no idea how to do
>> that from an already integrated and working branch.
>> Do you mean I…
>> - create patch files of whole directories and do a pr per directory, but
>> the build doesn’t work?
>>
>>
>> On September 20, 2017 at 15:07:56, Nick Allen (n...@nickallen.org) wrote:
>>
>> > Otto: Well, if there is an alternative merge strategy, I’m all ears.
>>
>> Yes, the alternative strategy is what I mentioned. :)  Copied in below.
>>
>> > Nick: Are you going to bite-off small chunks of the feature branch and
>> introduce PRs against master?
>>
>>
>>
>>
>>
>> On Wed, Sep 20, 2017 at 12:02 PM Otto Fowler 
>> wrote:
>>
>>> Well, if there is an alternative merge strategy, I’m all ears.
>>> As it stands, the only complete code / conceptual review has been the
>>> bundles-lib by mattf.
>>> So there is still a great deal of review activity to do.
>>>
>>> Also conceptually there is ( might not be complete )
>>>
>>> * the discuss thread topic from this morning ( metron parsers v.
>>> extensions wrt registration and management )
>>> * default configurations ( parser, enrichment, indexing, elasticsearch)
>>>
>>>
>>>
>>>
>>> On September 20, 2017 at 11:37:17, Nick Allen (n...@nickallen.org)
>>> wrote:
>>>
>>> Ok, so it sounds like you are envisioning a "big bang" merge between 

Re: feature branch bumps

[Otto Fowler]

I am not taking any of this as starting to do a flame war :)  And your
concern is about review not merging right?  If it is all reviewed and
accepted, why would 6 merges be preferable?  That doesn’t make sense.
Breaking it up only makes sense with regards to review, so that is how I’ll
take it.

 And I *am* all ears for how to break up the code.  Like literally HOW.
Like : ‘I would diff a folder with master, create a new branch for the pr
and patch just that’.
Just 'break up the massive working thing into smaller chucks' is not enough
for my relative experience with git and github to you Nick.

So if all we want are code review chunks, regardless of being able to run
them and see them work then ( but also not break the regular build and
full_dev ):

1. PR for Bundle-Lib And Maven Plugin ( but that is already reviewed )
code review only, doesn’t do anything
2. PR for Archetype ( just building the result, unit integration tests )
code review and generate project review.   projects built by archetype : =>
not buildable not installable not testable
relies on changes to parser testing etc. to work with non-hard coded paths
for test resources etc etc
3. PR with Parsers copied to extensions
code review only.  not buildable not installable not testable
4. PR with the rest of 777 + grok fixes ( all non ui non rest ) -> move to
new parser structure, load install, grok support etc etc.
regression testable, status quo ante functionality.
still huge
will still be a problem
cannot be broken down and built / pass travis at this point
this is the integration
5. PR of Rest
can install and uninstall via rest
6. PR of UI
can list, install uninstall via UI

I did not think then that we wanted PRs that don’t build or test or run the
changes they have ( but the branch builds and passes travis ), but just
have code to review.

But, as you said, it has been since April, and only Bundles has been
reviewed. If nobody but Matt is even going to _attempt_ code review, then
it may now be implicitly required that I do this.


On September 20, 2017 at 16:41:46, Nick Allen (n...@nickallen.org) wrote:

I was just responding to your statement "Well, if there is an alternative
merge strategy, I’m all ears."  I understand that you disagree, but this is
an alternative strategy.  I think this approach is feasible and I outlined
how here [1].

The criticism of this approach is the amount of effort and time to do this,
which I agree with.  It will take a good deal of effort.  I acquiesced to
this criticism at the time because I thought we were trying to push this
change in quickly.  But it has been a couple months now and we still have a
bunch of code to merge.

This is no criticism of you or anyone here.  What Otto has been doing in
keeping this branch merged with master is god damn heroic.  And I am so
excited to see this functionality hit master.  I just wanted to brainstorm
on ideas to help get this in.  Right now, I am unsure of the path and
timeline on how we are going to get this merged.

Please read this in the kindest possible light.  I am not trying to start a
flame war or offend anyone.


[1] https://github.com/apache/metron/pull/530#issuecomment-306806217



On Wed, Sep 20, 2017 at 3:37 PM Otto Fowler  wrote:

> “Bite off small chunks” is what I keep hearing. I have no idea how to do
> that from an already integrated and working branch.
> Do you mean I…
> - create patch files of whole directories and do a pr per directory, but
> the build doesn’t work?
>
>
> On September 20, 2017 at 15:07:56, Nick Allen (n...@nickallen.org) wrote:
>
> > Otto: Well, if there is an alternative merge strategy, I’m all ears.
>
> Yes, the alternative strategy is what I mentioned. :)  Copied in below.
>
> > Nick: Are you going to bite-off small chunks of the feature branch and
> introduce PRs against master?
>
>
>
>
>
> On Wed, Sep 20, 2017 at 12:02 PM Otto Fowler 
> wrote:
>
>> Well, if there is an alternative merge strategy, I’m all ears.
>> As it stands, the only complete code / conceptual review has been the
>> bundles-lib by mattf.
>> So there is still a great deal of review activity to do.
>>
>> Also conceptually there is ( might not be complete )
>>
>> * the discuss thread topic from this morning ( metron parsers v.
>> extensions wrt registration and management )
>> * default configurations ( parser, enrichment, indexing, elasticsearch)
>>
>>
>>
>>
>> On September 20, 2017 at 11:37:17, Nick Allen (n...@nickallen.org) wrote:
>>
>> Ok, so it sounds like you are envisioning a "big bang" merge between the
>> feature branch and master at some point.  Not ideal in my mind, but maybe
>> we need to be more pragmatic with this one.
>>
>> Do we have other tasks (like these PRs) to complete before we are ready
>> to consider the merge?
>>
>> I am just looking to help however I can in killing this feature branch;
>> meaning I want your code in master, as soon as possible. :)
>>
>>
>>
>>
>>
>>
>> On Wed, Sep 20, 

[GitHub] metron issue #737: METRON-1161: Add ability to edit parser command line opti...

[merrimanr]

Github user merrimanr commented on the issue:

https://github.com/apache/metron/pull/737
  
Just pushed out a commit that addresses @ottobackwards comments and fixes 
the NUM_WORKERS and NUM_ACKERS issue.  I also added a warning that appears when 
a storm setting changes, telling the user that the topology needs to be 
restarted.  It may seem redundant that the same message appears for each field 
but we might want a different message in the future once we expose a way to 
rebalance using zookeeper values.  At that point this pattern would make sense 
and I don't think it necessarily looks too bad now.

@nickwallen I also started a discussion on the other points you brought up. 
 Hopefully whatever conclusion we come to there can be a follow-on JIRA.


---

[DISCUSS] How should Management UI save changes?

[Ryan Merriman]

Recently @nickwallen brought up some good points about the usability of the
Management UI here:
https://github.com/apache/metron/pull/737#issuecomment-330632113.  The
issues he brings up apply to all child panels so I think it makes sense to
agree on a common approach and apply it to all of them.

Most child panels have a save button that saves changes to the local
(browser) copy of the config.  The save button on the primary panel
persists the changes to zookeeper and closes all panels.  Should we change
the buttons or button text?  What should the different buttons do?  One
idea could be to just skip saving to a local copy, meaning hitting the save
button persists changes in that panel to zookeeper.  Another idea could be
to get rid of the save buttons on child panels and changes to the form
would immediately update the local copy.  In this case we would likely need
an indicator that there are changes to be saved (or should we have that no
matter what?).  Other ideas?

There is also the issue of being able to discard changes and go back to
what they were before.  Now you can close a child or primary panel but you
discard all changes in that panel and all changes period in the case of the
primary panel.  We could be to expose a revert link or button for each form
input (a lot of work probably).  Other ideas?

Ryan

[GitHub] metron pull request #726: METRON-1145: Profiler mpack does not create kafka ...

[nickwallen]

Github user nickwallen commented on a diff in the pull request:

https://github.com/apache/metron/pull/726#discussion_r140089410
  
--- Diff: 
metron-deployment/packaging/ambari/metron-mpack/src/main/resources/common-services/METRON/CURRENT/package/scripts/profiler_master.py
 ---
@@ -59,6 +60,9 @@ def configure(self, env, upgrade_type=None, 
config_dir=None):
 commands.create_hbase_tables()
 if params.security_enabled and not 
commands.is_hbase_acl_configured():
 commands.set_hbase_acls()
--- End diff --

Ah, daft, you're right.  It works different than the Kafka ACL functions.


---

[GitHub] metron pull request #726: METRON-1145: Profiler mpack does not create kafka ...

[justinleet]

Github user justinleet commented on a diff in the pull request:

https://github.com/apache/metron/pull/726#discussion_r140088458
  
--- Diff: 
metron-deployment/packaging/ambari/metron-mpack/src/main/resources/common-services/METRON/CURRENT/package/scripts/profiler_master.py
 ---
@@ -59,6 +60,9 @@ def configure(self, env, upgrade_type=None, 
config_dir=None):
 commands.create_hbase_tables()
 if params.security_enabled and not 
commands.is_hbase_acl_configured():
 commands.set_hbase_acls()
--- End diff --

@nickwallen It happens at the end of `set_hbase_acls()`


---

[GitHub] metron pull request #726: METRON-1145: Profiler mpack does not create kafka ...

[nickwallen]

Github user nickwallen commented on a diff in the pull request:

https://github.com/apache/metron/pull/726#discussion_r140087640
  
--- Diff: 
metron-deployment/packaging/ambari/metron-mpack/src/main/resources/common-services/METRON/CURRENT/package/scripts/profiler_master.py
 ---
@@ -59,6 +60,9 @@ def configure(self, env, upgrade_type=None, 
config_dir=None):
 commands.create_hbase_tables()
 if params.security_enabled and not 
commands.is_hbase_acl_configured():
 commands.set_hbase_acls()
--- End diff --

Sorry to bring this up on an old, closed PR, but I am currently trying to 
track a bug that is giving me an uber headache. :)  Thought this was the 
easiest way to reach out.


---

[GitHub] metron pull request #726: METRON-1145: Profiler mpack does not create kafka ...

[nickwallen]

Github user nickwallen commented on a diff in the pull request:

https://github.com/apache/metron/pull/726#discussion_r140087395
  
--- Diff: 
metron-deployment/packaging/ambari/metron-mpack/src/main/resources/common-services/METRON/CURRENT/package/scripts/profiler_master.py
 ---
@@ -59,6 +60,9 @@ def configure(self, env, upgrade_type=None, 
config_dir=None):
 commands.create_hbase_tables()
 if params.security_enabled and not 
commands.is_hbase_acl_configured():
 commands.set_hbase_acls()
--- End diff --

Is there a reason that we do not call `commands.set_hbase_acl_configured()` 
here?


---

Re: feature branch bumps

[Nick Allen]

I was just responding to your statement "Well, if there is an alternative
merge strategy, I’m all ears."  I understand that you disagree, but this is
an alternative strategy.  I think this approach is feasible and I outlined
how here [1].

The criticism of this approach is the amount of effort and time to do this,
which I agree with.  It will take a good deal of effort.  I acquiesced to
this criticism at the time because I thought we were trying to push this
change in quickly.  But it has been a couple months now and we still have a
bunch of code to merge.

This is no criticism of you or anyone here.  What Otto has been doing in
keeping this branch merged with master is god damn heroic.  And I am so
excited to see this functionality hit master.  I just wanted to brainstorm
on ideas to help get this in.  Right now, I am unsure of the path and
timeline on how we are going to get this merged.

Please read this in the kindest possible light.  I am not trying to start a
flame war or offend anyone.


[1] https://github.com/apache/metron/pull/530#issuecomment-306806217



On Wed, Sep 20, 2017 at 3:37 PM Otto Fowler  wrote:

> “Bite off small chunks” is what I keep hearing. I have no idea how to do
> that from an already integrated and working branch.
> Do you mean I…
> - create patch files of whole directories and do a pr per directory, but
> the build doesn’t work?
>
>
> On September 20, 2017 at 15:07:56, Nick Allen (n...@nickallen.org) wrote:
>
> > Otto: Well, if there is an alternative merge strategy, I’m all ears.
>
> Yes, the alternative strategy is what I mentioned. :)  Copied in below.
>
> > Nick: Are you going to bite-off small chunks of the feature branch and
> introduce PRs against master?
>
>
>
>
>
> On Wed, Sep 20, 2017 at 12:02 PM Otto Fowler 
> wrote:
>
>> Well, if there is an alternative merge strategy, I’m all ears.
>> As it stands, the only complete code / conceptual review has been the
>> bundles-lib by mattf.
>> So there is still a great deal of review activity to do.
>>
>> Also conceptually there is ( might not be complete )
>>
>> * the discuss thread topic from this morning ( metron parsers v.
>> extensions wrt registration and management )
>> * default configurations ( parser, enrichment, indexing, elasticsearch)
>>
>>
>>
>>
>> On September 20, 2017 at 11:37:17, Nick Allen (n...@nickallen.org) wrote:
>>
>> Ok, so it sounds like you are envisioning a "big bang" merge between the
>> feature branch and master at some point.  Not ideal in my mind, but maybe
>> we need to be more pragmatic with this one.
>>
>> Do we have other tasks (like these PRs) to complete before we are ready
>> to consider the merge?
>>
>> I am just looking to help however I can in killing this feature branch;
>> meaning I want your code in master, as soon as possible. :)
>>
>>
>>
>>
>>
>>
>> On Wed, Sep 20, 2017 at 10:27 AM Otto Fowler 
>> wrote:
>>
>>> So, were the functionality is not FB related, I have been doing PR’s
>>> against master ( such as hdfs service ability to set permissions ).
>>> I don’t think we have talked about the end game PR from feature to
>>> master, I am don’t know how we would do multiple PRs to bring it down
>>> once ‘accepted’.  I think that approach needs to to be discussed and
>>> agreed on.
>>>
>>>
>>> On September 20, 2017 at 10:23:44, Nick Allen (n...@nickallen.org)
>>> wrote:
>>>
>>> So it sounds like these PRs do move us closer to bringing the two
>>> branches together.  But I think I am missing your high-level approach
>>> though.
>>>
>>> How are we going to get all of the functionality from the feature branch
>>> into master?  Are you going to bite-off small chunks of the feature branch
>>> and introduce PRs against master?
>>>
>>>
>>>
>>> On Wed, Sep 20, 2017 at 9:23 AM Otto Fowler 
>>> wrote:
>>>
 Right now, these two PR’s are stacked, the UI depends on the
 BundleSystem changes.
 I would rather bring them down to feature before I do master
 integration, than integrate master
 into feature and then bring it out to the stacked PR’s.  Simple as that.



 On September 20, 2017 at 08:35:09, Nick Allen (n...@nickallen.org)
 wrote:

 Hi Otto -

 What is the plan for bringing the feature branch and master together? Do
 these PRs move us closer to bringing the two branches together?

 Thanks



 On Wed, Sep 20, 2017 at 8:19 AM Otto Fowler 
 wrote:

 > Can I get a bump on https://github.com/apache/metron/pull/747 and
 > https://github.com/apache/metron/pull/761?
 >
 > The next time I take master is going to be a bit of integration work,
 and I
 > would like to unstack and get my stuff integrated first.
 >

Re: feature branch bumps

[Otto Fowler]

“Bite off small chunks” is what I keep hearing. I have no idea how to do
that from an already integrated and working branch.
Do you mean I…
- create patch files of whole directories and do a pr per directory, but
the build doesn’t work?


On September 20, 2017 at 15:07:56, Nick Allen (n...@nickallen.org) wrote:

> Otto: Well, if there is an alternative merge strategy, I’m all ears.

Yes, the alternative strategy is what I mentioned. :)  Copied in below.

> Nick: Are you going to bite-off small chunks of the feature branch and
introduce PRs against master?





On Wed, Sep 20, 2017 at 12:02 PM Otto Fowler 
wrote:

> Well, if there is an alternative merge strategy, I’m all ears.
> As it stands, the only complete code / conceptual review has been the
> bundles-lib by mattf.
> So there is still a great deal of review activity to do.
>
> Also conceptually there is ( might not be complete )
>
> * the discuss thread topic from this morning ( metron parsers v.
> extensions wrt registration and management )
> * default configurations ( parser, enrichment, indexing, elasticsearch)
>
>
>
>
> On September 20, 2017 at 11:37:17, Nick Allen (n...@nickallen.org) wrote:
>
> Ok, so it sounds like you are envisioning a "big bang" merge between the
> feature branch and master at some point.  Not ideal in my mind, but maybe
> we need to be more pragmatic with this one.
>
> Do we have other tasks (like these PRs) to complete before we are ready to
> consider the merge?
>
> I am just looking to help however I can in killing this feature branch;
> meaning I want your code in master, as soon as possible. :)
>
>
>
>
>
>
> On Wed, Sep 20, 2017 at 10:27 AM Otto Fowler 
> wrote:
>
>> So, were the functionality is not FB related, I have been doing PR’s
>> against master ( such as hdfs service ability to set permissions ).
>> I don’t think we have talked about the end game PR from feature to
>> master, I am don’t know how we would do multiple PRs to bring it down
>> once ‘accepted’.  I think that approach needs to to be discussed and
>> agreed on.
>>
>>
>> On September 20, 2017 at 10:23:44, Nick Allen (n...@nickallen.org) wrote:
>>
>> So it sounds like these PRs do move us closer to bringing the two
>> branches together.  But I think I am missing your high-level approach
>> though.
>>
>> How are we going to get all of the functionality from the feature branch
>> into master?  Are you going to bite-off small chunks of the feature branch
>> and introduce PRs against master?
>>
>>
>>
>> On Wed, Sep 20, 2017 at 9:23 AM Otto Fowler 
>> wrote:
>>
>>> Right now, these two PR’s are stacked, the UI depends on the
>>> BundleSystem changes.
>>> I would rather bring them down to feature before I do master
>>> integration, than integrate master
>>> into feature and then bring it out to the stacked PR’s.  Simple as that.
>>>
>>>
>>>
>>> On September 20, 2017 at 08:35:09, Nick Allen (n...@nickallen.org)
>>> wrote:
>>>
>>> Hi Otto -
>>>
>>> What is the plan for bringing the feature branch and master together? Do
>>> these PRs move us closer to bringing the two branches together?
>>>
>>> Thanks
>>>
>>>
>>>
>>> On Wed, Sep 20, 2017 at 8:19 AM Otto Fowler 
>>> wrote:
>>>
>>> > Can I get a bump on https://github.com/apache/metron/pull/747 and
>>> > https://github.com/apache/metron/pull/761?
>>> >
>>> > The next time I take master is going to be a bit of integration work,
>>> and I
>>> > would like to unstack and get my stuff integrated first.
>>> >
>>>
>>>

Re: [DISUCUSS] [CALL FOR COMMENT] Metron parsers as actual extensions

[Otto Fowler]

OK,

So, I think that this discussion should be taken up again after the demo.
It will be hopefully
easier then.

Sorry for the static.

Also : remember
https://cwiki.apache.org/confluence/display/METRON/Metron+Extension+System+and+Parser+Extensions


On September 20, 2017 at 14:29:51, Ryan Merriman (merrim...@gmail.com)
wrote:

I will attempt to clarify parsers vs sensors. Parsers refer to concrete
parser classes and sensors refer to configuration + one of the parser
classes (with parser class being defined in the configuration). The
architecture was designed so that a parser class can be made dynamic and
behave differently depending on configuration. This allows multiple use
cases to be handled by a single parser class with different
configurations. A good example of this (and the primary reason this
pattern was chosen) is the GrokParser. This parser can handle multiple
sensors even though it is the same parser class in each instance.

What are some reasons we would want to use the bundle system to deploy a
configuration only sensor? From what I understand creating a bundle is not
a simple process and requires manually editing configuration files (correct
me if I am wrong). The whole reason we created the management UI was to
allow people to create sensors without having to go through this difficult,
error-prone process that initially made Metron hard to use. You can create
sensors right now with the management UI as long as the necessary parser
class is available. You mention test coverage but I would argue the unit
and integration tests for that parser class should cover the edge cases of
configuration. What happens if I change the configuration? Do I have to
also go update the unit/integration tests for that parser? Do I have to
rebuild and reinstall/update the parser?

This bundle system is absolutely necessary for parser classes but for me it
feels a little too tightly coupled to configuration. Not saying
configuration shouldn't be involved at all, there is definitely value in
providing a default/bootstrap configuration for a parser class for someone
to start with. Creating a parser class or a bundle is a complex task
suited for an engineer (with the help of the community). A SOC analyst
should only have to understand the various configuration options to create
and maintain a sensor and be able to make these changes through a user
interface.

This feature branch moves pretty fast so it's entirely possible my
knowledge is incorrect or outdated. Don't hesitate to correct anything I'm
missing or have incorrect. My main concern is that adding this feature
should make Metron easier to use and not harder. Is it possible these 2
options could coexist?

Ryan



On Wed, Sep 20, 2017 at 9:21 AM, Otto Fowler 
wrote:

> Simon, I’m sorry, I may not have answered your question.
> I use parser and sensors as the same thing, but from what you say I think
I
> mean parser.
>
>
> On September 20, 2017 at 10:08:25, Otto Fowler (ottobackwa...@gmail.com)
> wrote:
>
> So,
> The distinction between ‘sensor’ and ‘parser’ is not clear to me either,
if
> it is defined somewhere and I have missed it, please point me in the
right
> direction.
> While I don’t want to fork the discussion, the question is where you find
> it so to speak, so about bundles and configurations.
>
> With the extension system you have two options for ‘config’ only
> parser/sensor installs.
>
> 1. The previous mechanism of creating the configurations and manually
> pushing them zookeeper and possibly patterns still works, although they
> will not be managed as extensions. This I believe is a feature gap that
> already exists and I did not address it as part of this effort.
> 2. Create a new parser from the archetype, but remove all the src/main
code
> and make it configuration only. This I think should be the recommended
way
> to create configuration only parsers, since such parsers will still have
> the unit and integration tests. Flows where you start with the archetype
> and refine through tests or manually deploy, refine and then build from
the
> archetype can be imagined.
>
>
> The main question for this thread however, is should metron parsers and
> 3rd party parsers be treated the same -> they are all extensions,
> manageable through the extensions ui.
>
> I can demo for you whenever you are free if you don’t want to wait for
the
> community demo.
>
>
> On September 20, 2017 at 09:52:56, Simon Elliston Ball (
> si...@simonellistonball.com) wrote:
>
> Otto,
>
> Can you just clarify what you mean by parsers in this instance. To my
mind
> parsers in metron are be classes, and do not have any configuration
> settings. Instances of parsers are referred to in the ui as sensors, and
> are essentially concrete instances of parsers and as such do have config.
>
> The parser vs sensor distinction feels like a valuable one to make here.
> I'd really like a clearer understanding of the role of bundles in config,
> and how we maintain 

Re: feature branch bumps

[zeo...@gmail.com]

But wait, I thought we had established that this was such a fundamental
change that it was hard to chunk it out and keep master working.

Jon

On Wed, Sep 20, 2017 at 3:08 PM Nick Allen  wrote:

> > Otto: Well, if there is an alternative merge strategy, I’m all ears.
>
> Yes, the alternative strategy is what I mentioned. :)  Copied in below.
>
> > Nick: Are you going to bite-off small chunks of the feature branch and
> introduce PRs against master?
>
>
>
>
>
> On Wed, Sep 20, 2017 at 12:02 PM Otto Fowler 
> wrote:
>
> > Well, if there is an alternative merge strategy, I’m all ears.
> > As it stands, the only complete code / conceptual review has been the
> > bundles-lib by mattf.
> > So there is still a great deal of review activity to do.
> >
> > Also conceptually there is ( might not be complete )
> >
> > * the discuss thread topic from this morning ( metron parsers v.
> > extensions wrt registration and management )
> > * default configurations ( parser, enrichment, indexing, elasticsearch)
> >
> >
> >
> >
> > On September 20, 2017 at 11:37:17, Nick Allen (n...@nickallen.org)
> wrote:
> >
> > Ok, so it sounds like you are envisioning a "big bang" merge between the
> > feature branch and master at some point.  Not ideal in my mind, but maybe
> > we need to be more pragmatic with this one.
> >
> > Do we have other tasks (like these PRs) to complete before we are ready
> to
> > consider the merge?
> >
> > I am just looking to help however I can in killing this feature branch;
> > meaning I want your code in master, as soon as possible. :)
> >
> >
> >
> >
> >
> >
> > On Wed, Sep 20, 2017 at 10:27 AM Otto Fowler 
> > wrote:
> >
> >> So, were the functionality is not FB related, I have been doing PR’s
> >> against master ( such as hdfs service ability to set permissions ).
> >> I don’t think we have talked about the end game PR from feature to
> >> master, I am don’t know how we would do multiple PRs to bring it down
> >> once ‘accepted’.  I think that approach needs to to be discussed and
> >> agreed on.
> >>
> >>
> >> On September 20, 2017 at 10:23:44, Nick Allen (n...@nickallen.org)
> wrote:
> >>
> >> So it sounds like these PRs do move us closer to bringing the two
> >> branches together.  But I think I am missing your high-level approach
> >> though.
> >>
> >> How are we going to get all of the functionality from the feature branch
> >> into master?  Are you going to bite-off small chunks of the feature
> branch
> >> and introduce PRs against master?
> >>
> >>
> >>
> >> On Wed, Sep 20, 2017 at 9:23 AM Otto Fowler 
> >> wrote:
> >>
> >>> Right now, these two PR’s are stacked, the UI depends on the
> >>> BundleSystem changes.
> >>> I would rather bring them down to feature before I do master
> >>> integration, than integrate master
> >>> into feature and then bring it out to the stacked PR’s.  Simple as
> that.
> >>>
> >>>
> >>>
> >>> On September 20, 2017 at 08:35:09, Nick Allen (n...@nickallen.org)
> >>> wrote:
> >>>
> >>> Hi Otto -
> >>>
> >>> What is the plan for bringing the feature branch and master together?
> Do
> >>> these PRs move us closer to bringing the two branches together?
> >>>
> >>> Thanks
> >>>
> >>>
> >>>
> >>> On Wed, Sep 20, 2017 at 8:19 AM Otto Fowler 
> >>> wrote:
> >>>
> >>> > Can I get a bump on https://github.com/apache/metron/pull/747 and
> >>> > https://github.com/apache/metron/pull/761?
> >>> >
> >>> > The next time I take master is going to be a bit of integration work,
> >>> and I
> >>> > would like to unstack and get my stuff integrated first.
> >>> >
> >>>
> >>>
>
-- 

Jon

Re: feature branch bumps

[Nick Allen]

> Otto: Well, if there is an alternative merge strategy, I’m all ears.

Yes, the alternative strategy is what I mentioned. :)  Copied in below.

> Nick: Are you going to bite-off small chunks of the feature branch and
introduce PRs against master?





On Wed, Sep 20, 2017 at 12:02 PM Otto Fowler 
wrote:

> Well, if there is an alternative merge strategy, I’m all ears.
> As it stands, the only complete code / conceptual review has been the
> bundles-lib by mattf.
> So there is still a great deal of review activity to do.
>
> Also conceptually there is ( might not be complete )
>
> * the discuss thread topic from this morning ( metron parsers v.
> extensions wrt registration and management )
> * default configurations ( parser, enrichment, indexing, elasticsearch)
>
>
>
>
> On September 20, 2017 at 11:37:17, Nick Allen (n...@nickallen.org) wrote:
>
> Ok, so it sounds like you are envisioning a "big bang" merge between the
> feature branch and master at some point.  Not ideal in my mind, but maybe
> we need to be more pragmatic with this one.
>
> Do we have other tasks (like these PRs) to complete before we are ready to
> consider the merge?
>
> I am just looking to help however I can in killing this feature branch;
> meaning I want your code in master, as soon as possible. :)
>
>
>
>
>
>
> On Wed, Sep 20, 2017 at 10:27 AM Otto Fowler 
> wrote:
>
>> So, were the functionality is not FB related, I have been doing PR’s
>> against master ( such as hdfs service ability to set permissions ).
>> I don’t think we have talked about the end game PR from feature to
>> master, I am don’t know how we would do multiple PRs to bring it down
>> once ‘accepted’.  I think that approach needs to to be discussed and
>> agreed on.
>>
>>
>> On September 20, 2017 at 10:23:44, Nick Allen (n...@nickallen.org) wrote:
>>
>> So it sounds like these PRs do move us closer to bringing the two
>> branches together.  But I think I am missing your high-level approach
>> though.
>>
>> How are we going to get all of the functionality from the feature branch
>> into master?  Are you going to bite-off small chunks of the feature branch
>> and introduce PRs against master?
>>
>>
>>
>> On Wed, Sep 20, 2017 at 9:23 AM Otto Fowler 
>> wrote:
>>
>>> Right now, these two PR’s are stacked, the UI depends on the
>>> BundleSystem changes.
>>> I would rather bring them down to feature before I do master
>>> integration, than integrate master
>>> into feature and then bring it out to the stacked PR’s.  Simple as that.
>>>
>>>
>>>
>>> On September 20, 2017 at 08:35:09, Nick Allen (n...@nickallen.org)
>>> wrote:
>>>
>>> Hi Otto -
>>>
>>> What is the plan for bringing the feature branch and master together? Do
>>> these PRs move us closer to bringing the two branches together?
>>>
>>> Thanks
>>>
>>>
>>>
>>> On Wed, Sep 20, 2017 at 8:19 AM Otto Fowler 
>>> wrote:
>>>
>>> > Can I get a bump on https://github.com/apache/metron/pull/747 and
>>> > https://github.com/apache/metron/pull/761?
>>> >
>>> > The next time I take master is going to be a bit of integration work,
>>> and I
>>> > would like to unstack and get my stuff integrated first.
>>> >
>>>
>>>

Re: [DISUCUSS] [CALL FOR COMMENT] Metron parsers as actual extensions

[Ryan Merriman]

I will attempt to clarify parsers vs sensors.  Parsers refer to concrete
parser classes and sensors refer to configuration + one of the parser
classes (with parser class being defined in the configuration).  The
architecture was designed so that a parser class can be made dynamic and
behave differently depending on configuration.  This allows multiple use
cases to be handled by a single parser class with different
configurations.  A good example of this (and the primary reason this
pattern was chosen) is the GrokParser.  This parser can handle multiple
sensors even though it is the same parser class in each instance.

What are some reasons we would want to use the bundle system to deploy a
configuration only sensor?  From what I understand creating a bundle is not
a simple process and requires manually editing configuration files (correct
me if I am wrong).  The whole reason we created the management UI was to
allow people to create sensors without having to go through this difficult,
error-prone process that initially made Metron hard to use.  You can create
sensors right now with the management UI as long as the necessary parser
class is available.  You mention test coverage but I would argue the unit
and integration tests for that parser class should cover the edge cases of
configuration.  What happens if I change the configuration?  Do I have to
also go update the unit/integration tests for that parser?  Do I have to
rebuild and reinstall/update the parser?

This bundle system is absolutely necessary for parser classes but for me it
feels a little too tightly coupled to configuration.  Not saying
configuration shouldn't be involved at all, there is definitely value in
providing a default/bootstrap configuration for a parser class for someone
to start with.  Creating a parser class or a bundle is a complex task
suited for an engineer (with the help of the community).  A SOC analyst
should only have to understand the various configuration options to create
and maintain a sensor and be able to make these changes through a user
interface.

This feature branch moves pretty fast so it's entirely possible my
knowledge is incorrect or outdated.  Don't hesitate to correct anything I'm
missing or have incorrect.  My main concern is that adding this feature
should make Metron easier to use and not harder.  Is it possible these 2
options could coexist?

Ryan



On Wed, Sep 20, 2017 at 9:21 AM, Otto Fowler 
wrote:

> Simon, I’m sorry, I may not have answered your question.
> I use parser and sensors as the same thing, but from what you say I think I
> mean parser.
>
>
> On September 20, 2017 at 10:08:25, Otto Fowler (ottobackwa...@gmail.com)
> wrote:
>
> So,
> The distinction between ‘sensor’ and ‘parser’ is not clear to me either, if
> it is defined somewhere and I have missed it, please point me in the right
> direction.
> While I don’t want to fork the discussion, the question is where you find
> it so to speak, so about bundles and configurations.
>
> With the extension system you have two options for ‘config’ only
> parser/sensor installs.
>
> 1.  The previous mechanism of creating the configurations and manually
> pushing them zookeeper and possibly patterns still works, although they
> will not be managed as extensions.  This I believe is a feature gap that
> already exists and I did not address it as part of this effort.
> 2. Create a new parser from the archetype, but remove all the src/main code
> and make it configuration only.  This I think should be the recommended way
> to create configuration only parsers, since such parsers will still have
> the unit and integration tests.  Flows where you start with the archetype
> and refine through tests or manually deploy, refine and then build from the
> archetype can be imagined.
>
>
> The main question for this thread however, is should metron parsers and
>  3rd party parsers be treated the same -> they are all extensions,
> manageable through the extensions ui.
>
> I can demo for you whenever you are free if you don’t want to wait for the
> community demo.
>
>
> On September 20, 2017 at 09:52:56, Simon Elliston Ball (
> si...@simonellistonball.com) wrote:
>
> Otto,
>
> Can you just clarify what you mean by parsers in this instance. To my mind
> parsers in metron are be classes, and do not have any configuration
> settings. Instances of parsers are referred to in the ui as sensors, and
> are essentially concrete instances of parsers and as such do have config.
>
> The parser vs sensor distinction feels like a valuable one to make here.
> I'd really like a clearer understanding of the role of bundles in config,
> and how we maintain the ability to control config without the need for
> files in the bundle.
>
> Maybe some samples / a demo would really clear this up, but to be honest
> right now I'm a little confused about how this would work for an average
> SOC team who do not have maven available.
>
> Thanks,
> Simon
>
> Sent from my 

Re: [DISUCUSS] [CALL FOR COMMENT] Metron parsers as actual extensions

[zeo...@gmail.com]

Per our prior conversations, I prefer option 2 - treating third party and
built-in the same way.  I would love to see signing of extensions in the
future as a potential follow-on so we could verify the Metron built-ins
(and even third parties).

Jon

On Wed, Sep 20, 2017 at 10:22 AM Otto Fowler 
wrote:

> Simon, I’m sorry, I may not have answered your question.
> I use parser and sensors as the same thing, but from what you say I think I
> mean parser.
>
>
> On September 20, 2017 at 10:08:25, Otto Fowler (ottobackwa...@gmail.com)
> wrote:
>
> So,
> The distinction between ‘sensor’ and ‘parser’ is not clear to me either, if
> it is defined somewhere and I have missed it, please point me in the right
> direction.
> While I don’t want to fork the discussion, the question is where you find
> it so to speak, so about bundles and configurations.
>
> With the extension system you have two options for ‘config’ only
> parser/sensor installs.
>
> 1.  The previous mechanism of creating the configurations and manually
> pushing them zookeeper and possibly patterns still works, although they
> will not be managed as extensions.  This I believe is a feature gap that
> already exists and I did not address it as part of this effort.
> 2. Create a new parser from the archetype, but remove all the src/main code
> and make it configuration only.  This I think should be the recommended way
> to create configuration only parsers, since such parsers will still have
> the unit and integration tests.  Flows where you start with the archetype
> and refine through tests or manually deploy, refine and then build from the
> archetype can be imagined.
>
>
> The main question for this thread however, is should metron parsers and
>  3rd party parsers be treated the same -> they are all extensions,
> manageable through the extensions ui.
>
> I can demo for you whenever you are free if you don’t want to wait for the
> community demo.
>
>
> On September 20, 2017 at 09:52:56, Simon Elliston Ball (
> si...@simonellistonball.com) wrote:
>
> Otto,
>
> Can you just clarify what you mean by parsers in this instance. To my mind
> parsers in metron are be classes, and do not have any configuration
> settings. Instances of parsers are referred to in the ui as sensors, and
> are essentially concrete instances of parsers and as such do have config.
>
> The parser vs sensor distinction feels like a valuable one to make here.
> I'd really like a clearer understanding of the role of bundles in config,
> and how we maintain the ability to control config without the need for
> files in the bundle.
>
> Maybe some samples / a demo would really clear this up, but to be honest
> right now I'm a little confused about how this would work for an average
> SOC team who do not have maven available.
>
> Thanks,
> Simon
>
> Sent from my iPhone
>
> > On 20 Sep 2017, at 23:43, Otto Fowler  wrote:
> >
> > Note: Grok, CSV and JSONMap would be ‘always there’, as they are still
> > part of the system and not installed individually.
> >
> >
> >
> > On September 20, 2017 at 09:39:38, Otto Fowler (ottobackwa...@gmail.com)
> > wrote:
> >
> > The question has come up about the metron parsers installation vs. parser
> > extension installation differences, and I’d like to get some comments.
> >
> > Right now ( let’s pretend the UI PR get’s merged to the feature branch
> for
> > a minute ) in the original take on this the metron parsers ( bro, yaf,
> > snort etc ) get installed
> > into the system basically as before with regards to zookeeper ( although
> > they ALL get installed since they all have demo compatible default
> configs
> > now). The parser extensions, installed after the fact through ui however
> > have a new parser extension configuration
> > that is registered into a new zookeeper area, and are listed and managed
> in
> > the new UI. They can be installed or uninstalled basically.
> >
> > The question is if the parsers installed by metron should *also* be
> > registered in the same way, such that the parser extension ui lists all
> of
> > the parsers.
> >
> > This would allow removal and installation of metron parsers installed by
> > the system. Also, following on we can customize the install to not
> install
> > everything. It may also be more simple concept wise.
> >
> > That is the basic thing. So the question is if we want to go in this
> > direction.
> >
> > The not so basic thing is to still deploy the extensions into the system
> as
> > packages, but not install them, such that you can add an extension from a
> > file *and also* from a ‘repository’ of
> > extensions. Down the line we can support local and remote repositories
> etc.
> >
> > So the options are:
> >
> > 1. As it is now on the feature branch ( still pretending ;)), the system
> > installed parsers are not the same as the extension installed parsers.
> > While the project can uninstall and replace these parsers, the user/ui
> > cannot.
> > 2. 

Re: feature branch bumps

[Otto Fowler]

Well, if there is an alternative merge strategy, I’m all ears.
As it stands, the only complete code / conceptual review has been the
bundles-lib by mattf.
So there is still a great deal of review activity to do.

Also conceptually there is ( might not be complete )

* the discuss thread topic from this morning ( metron parsers v. extensions
wrt registration and management )
* default configurations ( parser, enrichment, indexing, elasticsearch)




On September 20, 2017 at 11:37:17, Nick Allen (n...@nickallen.org) wrote:

Ok, so it sounds like you are envisioning a "big bang" merge between the
feature branch and master at some point.  Not ideal in my mind, but maybe
we need to be more pragmatic with this one.

Do we have other tasks (like these PRs) to complete before we are ready to
consider the merge?

I am just looking to help however I can in killing this feature branch;
meaning I want your code in master, as soon as possible. :)






On Wed, Sep 20, 2017 at 10:27 AM Otto Fowler 
wrote:

> So, were the functionality is not FB related, I have been doing PR’s
> against master ( such as hdfs service ability to set permissions ).
> I don’t think we have talked about the end game PR from feature to master,
> I am don’t know how we would do multiple PRs to bring it down
> once ‘accepted’.  I think that approach needs to to be discussed and
> agreed on.
>
>
> On September 20, 2017 at 10:23:44, Nick Allen (n...@nickallen.org) wrote:
>
> So it sounds like these PRs do move us closer to bringing the two branches
> together.  But I think I am missing your high-level approach though.
>
> How are we going to get all of the functionality from the feature branch
> into master?  Are you going to bite-off small chunks of the feature branch
> and introduce PRs against master?
>
>
>
> On Wed, Sep 20, 2017 at 9:23 AM Otto Fowler 
> wrote:
>
>> Right now, these two PR’s are stacked, the UI depends on the BundleSystem
>> changes.
>> I would rather bring them down to feature before I do master integration,
>> than integrate master
>> into feature and then bring it out to the stacked PR’s.  Simple as that.
>>
>>
>>
>> On September 20, 2017 at 08:35:09, Nick Allen (n...@nickallen.org) wrote:
>>
>> Hi Otto -
>>
>> What is the plan for bringing the feature branch and master together? Do
>> these PRs move us closer to bringing the two branches together?
>>
>> Thanks
>>
>>
>>
>> On Wed, Sep 20, 2017 at 8:19 AM Otto Fowler 
>> wrote:
>>
>> > Can I get a bump on https://github.com/apache/metron/pull/747 and
>> > https://github.com/apache/metron/pull/761?
>> >
>> > The next time I take master is going to be a bit of integration work,
>> and I
>> > would like to unstack and get my stuff integrated first.
>> >
>>
>>

[GitHub] metron issue #762: METRON-1189: Add alert escalation to the Alerts UI

[iraghumitra]

Github user iraghumitra commented on the issue:

https://github.com/apache/metron/pull/762
  
@merrimanr tested the UI works as explained in the comments


---

[GitHub] metron pull request #762: METRON-1189: Add alert escalation to the Alerts UI

[iraghumitra]

Github user iraghumitra commented on a diff in the pull request:

https://github.com/apache/metron/pull/762#discussion_r139988431
  
--- Diff: 
metron-interface/metron-alerts/e2e/alert-details/alert-status/alert-details-status.e2e-spec.ts
 ---
@@ -0,0 +1,58 @@
+/// 
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+import { MetronAlertDetailsPage } from '../alert-details.po';
+import {customMatchers} from '../../matchers/custom-matchers';
+import {LoginPage} from '../../login/login.po';
+import {loadTestData, deleteTestData} from '../../utils/e2e_util';
+
+describe('metron-alerts alert status', function() {
+  let page: MetronAlertDetailsPage;
+  let loginPage: LoginPage;
+
+  beforeAll(() => {
+loadTestData();
+loginPage = new LoginPage();
+loginPage.login();
+  });
+
+  afterAll(() => {
+loginPage.logout();
--- End diff --

The logout operation is failing as the details window is still open when 
the test case tries to logout. We need the close the window after running the 
specs


---

Re: feature branch bumps

[Otto Fowler]

So, were the functionality is not FB related, I have been doing PR’s
against master ( such as hdfs service ability to set permissions ).
I don’t think we have talked about the end game PR from feature to master,
I am don’t know how we would do multiple PRs to bring it down
once ‘accepted’.  I think that approach needs to to be discussed and agreed
on.


On September 20, 2017 at 10:23:44, Nick Allen (n...@nickallen.org) wrote:

So it sounds like these PRs do move us closer to bringing the two branches
together.  But I think I am missing your high-level approach though.

How are we going to get all of the functionality from the feature branch
into master?  Are you going to bite-off small chunks of the feature branch
and introduce PRs against master?



On Wed, Sep 20, 2017 at 9:23 AM Otto Fowler  wrote:

> Right now, these two PR’s are stacked, the UI depends on the BundleSystem
> changes.
> I would rather bring them down to feature before I do master integration,
> than integrate master
> into feature and then bring it out to the stacked PR’s.  Simple as that.
>
>
>
> On September 20, 2017 at 08:35:09, Nick Allen (n...@nickallen.org) wrote:
>
> Hi Otto -
>
> What is the plan for bringing the feature branch and master together? Do
> these PRs move us closer to bringing the two branches together?
>
> Thanks
>
>
>
> On Wed, Sep 20, 2017 at 8:19 AM Otto Fowler 
> wrote:
>
> > Can I get a bump on https://github.com/apache/metron/pull/747 and
> > https://github.com/apache/metron/pull/761?
> >
> > The next time I take master is going to be a bit of integration work,
> and I
> > would like to unstack and get my stuff integrated first.
> >
>
>

Re: feature branch bumps

[Nick Allen]

So it sounds like these PRs do move us closer to bringing the two branches
together.  But I think I am missing your high-level approach though.

How are we going to get all of the functionality from the feature branch
into master?  Are you going to bite-off small chunks of the feature branch
and introduce PRs against master?



On Wed, Sep 20, 2017 at 9:23 AM Otto Fowler  wrote:

> Right now, these two PR’s are stacked, the UI depends on the BundleSystem
> changes.
> I would rather bring them down to feature before I do master integration,
> than integrate master
> into feature and then bring it out to the stacked PR’s.  Simple as that.
>
>
>
> On September 20, 2017 at 08:35:09, Nick Allen (n...@nickallen.org) wrote:
>
> Hi Otto -
>
> What is the plan for bringing the feature branch and master together? Do
> these PRs move us closer to bringing the two branches together?
>
> Thanks
>
>
>
> On Wed, Sep 20, 2017 at 8:19 AM Otto Fowler 
> wrote:
>
> > Can I get a bump on https://github.com/apache/metron/pull/747 and
> > https://github.com/apache/metron/pull/761?
> >
> > The next time I take master is going to be a bit of integration work,
> and I
> > would like to unstack and get my stuff integrated first.
> >
>
>

Re: [DISUCUSS] [CALL FOR COMMENT] Metron parsers as actual extensions

[Otto Fowler]

Simon, I’m sorry, I may not have answered your question.
I use parser and sensors as the same thing, but from what you say I think I
mean parser.


On September 20, 2017 at 10:08:25, Otto Fowler (ottobackwa...@gmail.com)
wrote:

So,
The distinction between ‘sensor’ and ‘parser’ is not clear to me either, if
it is defined somewhere and I have missed it, please point me in the right
direction.
While I don’t want to fork the discussion, the question is where you find
it so to speak, so about bundles and configurations.

With the extension system you have two options for ‘config’ only
parser/sensor installs.

1.  The previous mechanism of creating the configurations and manually
pushing them zookeeper and possibly patterns still works, although they
will not be managed as extensions.  This I believe is a feature gap that
already exists and I did not address it as part of this effort.
2. Create a new parser from the archetype, but remove all the src/main code
and make it configuration only.  This I think should be the recommended way
to create configuration only parsers, since such parsers will still have
the unit and integration tests.  Flows where you start with the archetype
and refine through tests or manually deploy, refine and then build from the
archetype can be imagined.


The main question for this thread however, is should metron parsers and
 3rd party parsers be treated the same -> they are all extensions,
manageable through the extensions ui.

I can demo for you whenever you are free if you don’t want to wait for the
community demo.


On September 20, 2017 at 09:52:56, Simon Elliston Ball (
si...@simonellistonball.com) wrote:

Otto,

Can you just clarify what you mean by parsers in this instance. To my mind
parsers in metron are be classes, and do not have any configuration
settings. Instances of parsers are referred to in the ui as sensors, and
are essentially concrete instances of parsers and as such do have config.

The parser vs sensor distinction feels like a valuable one to make here.
I'd really like a clearer understanding of the role of bundles in config,
and how we maintain the ability to control config without the need for
files in the bundle.

Maybe some samples / a demo would really clear this up, but to be honest
right now I'm a little confused about how this would work for an average
SOC team who do not have maven available.

Thanks,
Simon

Sent from my iPhone

> On 20 Sep 2017, at 23:43, Otto Fowler  wrote:
>
> Note: Grok, CSV and JSONMap would be ‘always there’, as they are still
> part of the system and not installed individually.
>
>
>
> On September 20, 2017 at 09:39:38, Otto Fowler (ottobackwa...@gmail.com)
> wrote:
>
> The question has come up about the metron parsers installation vs. parser
> extension installation differences, and I’d like to get some comments.
>
> Right now ( let’s pretend the UI PR get’s merged to the feature branch for
> a minute ) in the original take on this the metron parsers ( bro, yaf,
> snort etc ) get installed
> into the system basically as before with regards to zookeeper ( although
> they ALL get installed since they all have demo compatible default configs
> now). The parser extensions, installed after the fact through ui however
> have a new parser extension configuration
> that is registered into a new zookeeper area, and are listed and managed
in
> the new UI. They can be installed or uninstalled basically.
>
> The question is if the parsers installed by metron should *also* be
> registered in the same way, such that the parser extension ui lists all of
> the parsers.
>
> This would allow removal and installation of metron parsers installed by
> the system. Also, following on we can customize the install to not install
> everything. It may also be more simple concept wise.
>
> That is the basic thing. So the question is if we want to go in this
> direction.
>
> The not so basic thing is to still deploy the extensions into the system
as
> packages, but not install them, such that you can add an extension from a
> file *and also* from a ‘repository’ of
> extensions. Down the line we can support local and remote repositories
etc.
>
> So the options are:
>
> 1. As it is now on the feature branch ( still pretending ;)), the system
> installed parsers are not the same as the extension installed parsers.
> While the project can uninstall and replace these parsers, the user/ui
> cannot.
> 2. All parsers are installed as extensions and can be removed, but are all
> initially installed
> 3. All parsers are extensions, but not installed during the original
> deployment, rather they are put into a repository that the ui lets you
> browse to select, in addition to allowing
> install from file ( like intellij and other plugin systems do ). **This
> would have implications for the demo system, since we would want to still
> install the bro, snort and maybe yaf parsers.
>
>
> In considering these questions, we need to keep in mind 

Re: [DISUCUSS] [CALL FOR COMMENT] Metron parsers as actual extensions

[Otto Fowler]

So,
The distinction between ‘sensor’ and ‘parser’ is not clear to me either, if
it is defined somewhere and I have missed it, please point me in the right
direction.
While I don’t want to fork the discussion, the question is where you find
it so to speak, so about bundles and configurations.

With the extension system you have two options for ‘config’ only
parser/sensor installs.

1.  The previous mechanism of creating the configurations and manually
pushing them zookeeper and possibly patterns still works, although they
will not be managed as extensions.  This I believe is a feature gap that
already exists and I did not address it as part of this effort.
2. Create a new parser from the archetype, but remove all the src/main code
and make it configuration only.  This I think should be the recommended way
to create configuration only parsers, since such parsers will still have
the unit and integration tests.  Flows where you start with the archetype
and refine through tests or manually deploy, refine and then build from the
archetype can be imagined.


The main question for this thread however, is should metron parsers and
 3rd party parsers be treated the same -> they are all extensions,
manageable through the extensions ui.

I can demo for you whenever you are free if you don’t want to wait for the
community demo.


On September 20, 2017 at 09:52:56, Simon Elliston Ball (
si...@simonellistonball.com) wrote:

Otto,

Can you just clarify what you mean by parsers in this instance. To my mind
parsers in metron are be classes, and do not have any configuration
settings. Instances of parsers are referred to in the ui as sensors, and
are essentially concrete instances of parsers and as such do have config.

The parser vs sensor distinction feels like a valuable one to make here.
I'd really like a clearer understanding of the role of bundles in config,
and how we maintain the ability to control config without the need for
files in the bundle.

Maybe some samples / a demo would really clear this up, but to be honest
right now I'm a little confused about how this would work for an average
SOC team who do not have maven available.

Thanks,
Simon

Sent from my iPhone

> On 20 Sep 2017, at 23:43, Otto Fowler  wrote:
>
> Note: Grok, CSV and JSONMap would be ‘always there’, as they are still
> part of the system and not installed individually.
>
>
>
> On September 20, 2017 at 09:39:38, Otto Fowler (ottobackwa...@gmail.com)
> wrote:
>
> The question has come up about the metron parsers installation vs. parser
> extension installation differences, and I’d like to get some comments.
>
> Right now ( let’s pretend the UI PR get’s merged to the feature branch
for
> a minute ) in the original take on this the metron parsers ( bro, yaf,
> snort etc ) get installed
> into the system basically as before with regards to zookeeper ( although
> they ALL get installed since they all have demo compatible default
configs
> now). The parser extensions, installed after the fact through ui however
> have a new parser extension configuration
> that is registered into a new zookeeper area, and are listed and managed
in
> the new UI. They can be installed or uninstalled basically.
>
> The question is if the parsers installed by metron should *also* be
> registered in the same way, such that the parser extension ui lists all
of
> the parsers.
>
> This would allow removal and installation of metron parsers installed by
> the system. Also, following on we can customize the install to not
install
> everything. It may also be more simple concept wise.
>
> That is the basic thing. So the question is if we want to go in this
> direction.
>
> The not so basic thing is to still deploy the extensions into the system
as
> packages, but not install them, such that you can add an extension from a
> file *and also* from a ‘repository’ of
> extensions. Down the line we can support local and remote repositories
etc.
>
> So the options are:
>
> 1. As it is now on the feature branch ( still pretending ;)), the system
> installed parsers are not the same as the extension installed parsers.
> While the project can uninstall and replace these parsers, the user/ui
> cannot.
> 2. All parsers are installed as extensions and can be removed, but are
all
> initially installed
> 3. All parsers are extensions, but not installed during the original
> deployment, rather they are put into a repository that the ui lets you
> browse to select, in addition to allowing
> install from file ( like intellij and other plugin systems do ). **This
> would have implications for the demo system, since we would want to still
> install the bro, snort and maybe yaf parsers.
>
>
> In considering these questions, we need to keep in mind where we want to
> go, and how much of this is required for first release of the extension
> system. There is a lot of ‘while we are already doing xxx, we might as
> well do  since it will be harder later’ in this.
>
> Thanks 

Re: [DISUCUSS] [CALL FOR COMMENT] Metron parsers as actual extensions

[Simon Elliston Ball]

Otto,

Can you just clarify what you mean by parsers in this instance. To my mind 
parsers in metron are be classes, and do not have any configuration settings. 
Instances of parsers are referred to in the ui as sensors, and are essentially 
concrete instances of parsers and as such do have config. 

The parser vs sensor distinction feels like a valuable one to make here. I'd 
really like a clearer understanding of the role of bundles in config, and how 
we maintain the ability to control config without the need for files in the 
bundle.

Maybe some samples / a demo would really clear this up, but to be honest right 
now I'm a little confused about how this would work for an average SOC team who 
do not have maven available.

Thanks,
Simon 

Sent from my iPhone

> On 20 Sep 2017, at 23:43, Otto Fowler  wrote:
> 
> Note:  Grok, CSV and JSONMap would be ‘always there’, as they are still
> part of the system and not installed individually.
> 
> 
> 
> On September 20, 2017 at 09:39:38, Otto Fowler (ottobackwa...@gmail.com)
> wrote:
> 
> The question has come up about the metron parsers installation vs. parser
> extension installation differences, and I’d like to get some comments.
> 
> Right now ( let’s pretend the UI PR get’s merged to the feature branch for
> a minute ) in the original take on this the metron parsers ( bro, yaf,
> snort etc ) get installed
> into the system basically as before with regards to zookeeper ( although
> they ALL get installed since they all have demo compatible default configs
> now).  The parser extensions, installed after the fact through ui however
> have a new parser extension configuration
> that is registered into a new zookeeper area, and are listed and managed in
> the new UI.  They can be installed or uninstalled basically.
> 
> The question is if the parsers installed by metron should *also* be
> registered in the same way, such that the parser extension ui lists all of
> the parsers.
> 
> This would allow removal and installation of metron parsers installed by
> the system.  Also, following on we can customize the install to not install
> everything. It may also be more simple concept wise.
> 
> That is the basic thing.  So the question is if we want to go in this
> direction.
> 
> The not so basic thing is to still deploy the extensions into the system as
> packages, but not install them, such that you can add an extension from a
> file *and also* from a ‘repository’ of
> extensions.  Down the line we can support local and remote repositories etc.
> 
> So the options are:
> 
> 1. As it is now on the feature branch ( still pretending ;)), the system
> installed parsers are not the same as the extension installed parsers.
> While the project can uninstall and replace these parsers, the user/ui
> cannot.
> 2. All parsers are installed as extensions and can be removed, but are all
> initially installed
> 3. All parsers are extensions, but not installed during the original
> deployment, rather they are put into a repository that the ui lets you
> browse to select, in addition to allowing
> install from file ( like intellij and other plugin systems do ).  **This
> would have implications for the demo system, since we would want to still
> install the bro, snort and maybe yaf parsers.
> 
> 
> In considering these questions, we need to keep in mind where we want to
> go, and how much of this is required for first release of the extension
> system.  There is a lot of ‘while we are already doing xxx, we might as
> well do  since it will be harder later’  in this.
> 
> Thanks for your time and your timely responses.
> 
> ottO

Re: [DISUCUSS] [CALL FOR COMMENT] Metron parsers as actual extensions

[Otto Fowler]

Note:  Grok, CSV and JSONMap would be ‘always there’, as they are still
part of the system and not installed individually.



On September 20, 2017 at 09:39:38, Otto Fowler (ottobackwa...@gmail.com)
wrote:

The question has come up about the metron parsers installation vs. parser
extension installation differences, and I’d like to get some comments.

Right now ( let’s pretend the UI PR get’s merged to the feature branch for
a minute ) in the original take on this the metron parsers ( bro, yaf,
snort etc ) get installed
into the system basically as before with regards to zookeeper ( although
they ALL get installed since they all have demo compatible default configs
now).  The parser extensions, installed after the fact through ui however
have a new parser extension configuration
that is registered into a new zookeeper area, and are listed and managed in
the new UI.  They can be installed or uninstalled basically.

The question is if the parsers installed by metron should *also* be
registered in the same way, such that the parser extension ui lists all of
the parsers.

This would allow removal and installation of metron parsers installed by
the system.  Also, following on we can customize the install to not install
everything. It may also be more simple concept wise.

That is the basic thing.  So the question is if we want to go in this
direction.

The not so basic thing is to still deploy the extensions into the system as
packages, but not install them, such that you can add an extension from a
file *and also* from a ‘repository’ of
extensions.  Down the line we can support local and remote repositories etc.

So the options are:

1. As it is now on the feature branch ( still pretending ;)), the system
installed parsers are not the same as the extension installed parsers.
While the project can uninstall and replace these parsers, the user/ui
cannot.
2. All parsers are installed as extensions and can be removed, but are all
initially installed
3. All parsers are extensions, but not installed during the original
deployment, rather they are put into a repository that the ui lets you
browse to select, in addition to allowing
install from file ( like intellij and other plugin systems do ).  **This
would have implications for the demo system, since we would want to still
install the bro, snort and maybe yaf parsers.


In considering these questions, we need to keep in mind where we want to
go, and how much of this is required for first release of the extension
system.  There is a lot of ‘while we are already doing xxx, we might as
well do  since it will be harder later’  in this.

Thanks for your time and your timely responses.

ottO

[DISUCUSS] [CALL FOR COMMENT] Metron parsers as actual extensions

[Otto Fowler]

The question has come up about the metron parsers installation vs. parser
extension installation differences, and I’d like to get some comments.

Right now ( let’s pretend the UI PR get’s merged to the feature branch for
a minute ) in the original take on this the metron parsers ( bro, yaf,
snort etc ) get installed
into the system basically as before with regards to zookeeper ( although
they ALL get installed since they all have demo compatible default configs
now).  The parser extensions, installed after the fact through ui however
have a new parser extension configuration
that is registered into a new zookeeper area, and are listed and managed in
the new UI.  They can be installed or uninstalled basically.

The question is if the parsers installed by metron should *also* be
registered in the same way, such that the parser extension ui lists all of
the parsers.

This would allow removal and installation of metron parsers installed by
the system.  Also, following on we can customize the install to not install
everything. It may also be more simple concept wise.

That is the basic thing.  So the question is if we want to go in this
direction.

The not so basic thing is to still deploy the extensions into the system as
packages, but not install them, such that you can add an extension from a
file *and also* from a ‘repository’ of
extensions.  Down the line we can support local and remote repositories etc.

So the options are:

1. As it is now on the feature branch ( still pretending ;)), the system
installed parsers are not the same as the extension installed parsers.
While the project can uninstall and replace these parsers, the user/ui
cannot.
2. All parsers are installed as extensions and can be removed, but are all
initially installed
3. All parsers are extensions, but not installed during the original
deployment, rather they are put into a repository that the ui lets you
browse to select, in addition to allowing
install from file ( like intellij and other plugin systems do ).  **This
would have implications for the demo system, since we would want to still
install the bro, snort and maybe yaf parsers.


In considering these questions, we need to keep in mind where we want to
go, and how much of this is required for first release of the extension
system.  There is a lot of ‘while we are already doing xxx, we might as
well do  since it will be harder later’  in this.

Thanks for your time and your timely responses.

ottO

Re: feature branch bumps

[Otto Fowler]

Right now, these two PR’s are stacked, the UI depends on the BundleSystem
changes.
I would rather bring them down to feature before I do master integration,
than integrate master
into feature and then bring it out to the stacked PR’s.  Simple as that.



On September 20, 2017 at 08:35:09, Nick Allen (n...@nickallen.org) wrote:

Hi Otto -

What is the plan for bringing the feature branch and master together? Do
these PRs move us closer to bringing the two branches together?

Thanks



On Wed, Sep 20, 2017 at 8:19 AM Otto Fowler 
wrote:

> Can I get a bump on https://github.com/apache/metron/pull/747 and
> https://github.com/apache/metron/pull/761?
>
> The next time I take master is going to be a bit of integration work, and
I
> would like to unstack and get my stuff integrated first.
>

[GitHub] metron pull request #757: METRON-938: "service metron-rest start "...

[justinleet]

Github user justinleet commented on a diff in the pull request:

https://github.com/apache/metron/pull/757#discussion_r139957912
  
--- Diff: 
metron-deployment/packaging/ambari/metron-mpack/src/main/resources/common-services/METRON/CURRENT/service_advisor.py
 ---
@@ -69,6 +69,10 @@ def getServiceComponentLayoutValidations(self, services, 
hosts):
 message = "Metron REST must be colocated with an instance of 
STORM SUPERVISOR"
 items.append({ "type": 'host-component', "level": 'WARN', 
"message": message, "component-name": 'METRON_REST', "host": metronRESTHost })
 
+if metronRESTHost not in hbaseClientHosts:
--- End diff --

It's more a shorthand for the fact that REST requires requires basically 
everything else (HDFS, ZK, HBase, all the topologies, etc.).  We could do all 
the requirements in specifics if we want (and I'm not necessarily opposed to 
doing it for REST while I'm already in here), but I'd argue that's a good 
candidate for a follow on task (since we do it on multiple items).

I'd prefer to do what the Hive service does and just automatically place 
every required item on the same node automatically (the selector on install 
automatically selects the same thing for all the components), which then means 
that this colocation check on something like parsers is more robust.  Having 
said that, I never quite dug deep enough into how Ambari actually enforces that 
for Hive.


---

Re: feature branch bumps

[Nick Allen]

Hi Otto -

What is the plan for bringing the feature branch and master together?  Do
these PRs move us closer to bringing the two branches together?

Thanks



On Wed, Sep 20, 2017 at 8:19 AM Otto Fowler  wrote:

> Can I get a bump on https://github.com/apache/metron/pull/747 and
> https://github.com/apache/metron/pull/761?
>
> The next time I take master is going to be a bit of integration work, and I
> would like to unstack and get my stuff integrated first.
>

feature branch bumps

[Otto Fowler]

Can I get a bump on https://github.com/apache/metron/pull/747 and
https://github.com/apache/metron/pull/761?

The next time I take master is going to be a bit of integration work, and I
would like to unstack and get my stuff integrated first.

[GitHub] metron pull request #757: METRON-938: "service metron-rest start "...

[ottobackwards]

Github user ottobackwards commented on a diff in the pull request:

https://github.com/apache/metron/pull/757#discussion_r139951556
  
--- Diff: 
metron-deployment/packaging/ambari/metron-mpack/src/main/resources/common-services/METRON/CURRENT/service_advisor.py
 ---
@@ -69,6 +69,10 @@ def getServiceComponentLayoutValidations(self, services, 
hosts):
 message = "Metron REST must be colocated with an instance of 
STORM SUPERVISOR"
 items.append({ "type": 'host-component', "level": 'WARN', 
"message": message, "component-name": 'METRON_REST', "host": metronRESTHost })
 
+if metronRESTHost not in hbaseClientHosts:
--- End diff --

isn't making everything require everything else on the same node kind of an 
anti pattern?


---

[GitHub] metron pull request #757: METRON-938: "service metron-rest start "...

[justinleet]

Github user justinleet commented on a diff in the pull request:

https://github.com/apache/metron/pull/757#discussion_r139943087
  
--- Diff: 
metron-deployment/packaging/ambari/metron-mpack/src/main/resources/common-services/METRON/CURRENT/service_advisor.py
 ---
@@ -69,6 +69,10 @@ def getServiceComponentLayoutValidations(self, services, 
hosts):
 message = "Metron REST must be colocated with an instance of 
STORM SUPERVISOR"
 items.append({ "type": 'host-component', "level": 'WARN', 
"message": message, "component-name": 'METRON_REST', "host": metronRESTHost })
 
+if metronRESTHost not in hbaseClientHosts:
--- End diff --

Updated the service advisor to check on metronParsersHost


---

[GitHub] metron pull request #757: METRON-938: "service metron-rest start "...

[justinleet]

Github user justinleet commented on a diff in the pull request:

https://github.com/apache/metron/pull/757#discussion_r139942579
  
--- Diff: 
metron-deployment/packaging/ambari/metron-mpack/src/main/resources/common-services/METRON/CURRENT/package/scripts/rest_commands.py
 ---
@@ -58,17 +74,94 @@ def init_kafka_acls(self):
 
 def start_rest_application(self):
 Logger.info('Starting REST application')
-command = format("service metron-rest start 
{metron_jdbc_password!p}")
-Execute(command)
+
+# Build the spring options
+# the vagrant Spring profile provides configuration values, 
otherwise configuration is provided by rest_application.yml
+metron_spring_options = format(" --server.port={metron_rest_port}")
--- End diff --

Ignore the things I say, it would have helped if I'd done stuff right.  PR 
is updated to primarily run from a script, that's largely a renamed and 
slightly modified metron-rest (rename is to metron-rest.sh).  Need to update 
docs to reflect it.

I also killed the metron version variable by doing some shell globbing.  If 
you have multiple metron-rest jars in the lib directory it could be 
problematic, but I don't think it's unreasonable to not add random crap to the 
dirs.


---