subject:"\[DISCUSS\] Upcoming Release"

Re: [DISCUSS] Upcoming Release

2017-12-18 Thread Otto Fowler

Once we have the area, I can do the same for the RC check script


On December 18, 2017 at 11:11:18, Nick Allen (n...@nickallen.org) wrote:

Sure, I can clean up the script a bit and submit a PR for it.

Jon and Otto asked that I open a PR on the script that I use for merging
PRs too.

On Fri, Dec 15, 2017 at 2:30 PM, Matt Foley <mfo...@hortonworks.com> wrote:

> Perhaps under “build_utils” we should add a subdirectory for
> “release_utils”.
>
> From: Casey Stella <ceste...@gmail.com>
> Date: Friday, December 15, 2017 at 10:50 AM
> To: "dev@metron.apache.org" <dev@metron.apache.org>
> Cc: Matt Foley <mfo...@hortonworks.com>
> Subject: Re: [DISCUSS] Upcoming Release
>
> That script seems great, nick! Perhaps we should adjust the wiki around
> releases to point to it? Thoughts?
>
> On Fri, Dec 15, 2017 at 1:47 PM, Nick Allen <n...@nickallen.org<mailto:nic
> k...@nickallen.org>> wrote:
> Thanks, Matt.
>
> Maybe you already have something that does this. I wrote a quick script
> that validates each JIRA since the last release tag to make sure they are
> marked "Done" and with the correct fix version. I would expect that for
> the next release, each JIRA should have status="Done",
fix-version="0.4.2".
>
> Unless I am mistaken, we have quite a few that need cleaned up. In the
> following output, any line that has a URL indicates that a fix is needed.
> Or it least, **I think** a fix is needed.
>
> To the community If you see your name with a URL next to it, it would
> be great if you could follow that link and fix the JIRA. Otherwise, I
will
> volunteer to help clean some of these up should some not get addressed.
>
>
> *$ ./validate-jira-for-release*
> *Cloning into 'metron-0.4.2'...*
> *remote: Counting objects: 35046, done.*
> *remote: Compressing objects: 100% (13698/13698), done.*
> *remote: Total 35046 (delta 15708), reused 31645 (delta 12822)*
> *Receiving objects: 100% (35046/35046), 53.05 MiB | 6.48 MiB/s, done.*
> *Resolving deltas: 100% (15708/15708), done.*
> *Fetching origin*
> * JIRA STATUS FIX VERSION
> ASSIGNEE FIX*
> * METRON-1345 Done Michael
> Miklavcic https://issues.apache.org/jira/browse/METRON-1345
> <https://issues.apache.org/jira/browse/METRON-1345>*
> * METRON-1349 Done Next + 1 Nick
> Allen https://issues.apache.org/jira/browse/METRON-1349
> <https://issues.apache.org/jira/browse/METRON-1349>*
> * METRON-1343 Done
> Mohan https://issues.apache.org/jira/browse/METRON-1343
> <https://issues.apache.org/jira/browse/METRON-1343>*
> * METRON-1306 To Do
> Unassigned https://issues.apache.org/jira/browse/METRON-1306
> <https://issues.apache.org/jira/browse/METRON-1306>*
> * METRON-1341 Done Simon Elliston
> Ball https://issues.apache.org/jira/browse/METRON-1341
> <https://issues.apache.org/jira/browse/METRON-1341>*
> * METRON-1313 Done Jon
> Zeolla https://issues.apache.org/jira/browse/METRON-1313
> <https://issues.apache.org/jira/browse/METRON-1313>*
> * METRON-1346 Done Otto
> Fowler https://issues.apache.org/jira/browse/METRON-1346
> <https://issues.apache.org/jira/browse/METRON-1346>*
> * METRON-1336 Done 0.4.2 Nick
> Allen*
> * METRON-1335 Done Anand
> Subramanian https://issues.apache.org/jira/browse/METRON-1335
> <https://issues.apache.org/jira/browse/METRON-1335>*
> * METRON-1308 Done Jon
> Zeolla https://issues.apache.org/jira/browse/METRON-1308
> <https://issues.apache.org/jira/browse/METRON-1308>*
> * METRON-1338 Done 0.4.2 Nick
> Allen*
> * METRON-1286 To Do 0.4.2
> Unassigned https://issues.apache.org/jira/browse/METRON-1286
> <https://issues.apache.org/jira/browse/METRON-1286>*
> * METRON-1334 Done 0.4.2 Nick
> Allen*
> * METRON-1277 Done Otto
> Fowler https://issues.apache.org/jira/browse/METRON-1277
> <https://issues.apache.org/jira/browse/METRON-1277>*
> * METRON-1239 To Do
> Unassigned https://issues.apache.org/jira/browse/METRON-1239
> <https://issues.apache.org/jira/browse/METRON-1239>*
> * METRON-1328 Done Anand
> Subramanian https://issues.apache.org/jira/browse/METRON-1328
> <https://issues.apache.org/jira/browse/METRON-1328>*
> * METRON-1333 Done Otto
> Fowler https://issues.apache.org/jira/browse/METRON-1333
> <https://issues.apache.org/jira/browse/METRON-1333>*
> * METRON-1252 Done
> RaghuMitra https://issues.apache.org/jira/browse/METRON-1252
> <https://issues.apache.org/jira/browse/METRON-1252>*
> * METRON-1316 To Do Next + 1
> Unassigned https://issues.apache.org/jira/browse/METRON-1316
> <https://issues.apache.org/jira/browse/METRON-1316>*
> * METRON-1088 Done Jon
> Zeolla https://issues.apache.org/jira/b

Re: [DISCUSS] Upcoming Release

2017-12-18 Thread Nick Allen

Sure, I can clean up the script a bit and submit a PR for it.

Jon and Otto asked that I open a PR on the script that I use for merging
PRs too.

On Fri, Dec 15, 2017 at 2:30 PM, Matt Foley <mfo...@hortonworks.com> wrote:

> Perhaps under “build_utils” we should add a subdirectory for
> “release_utils”.
>
> From: Casey Stella <ceste...@gmail.com>
> Date: Friday, December 15, 2017 at 10:50 AM
> To: "dev@metron.apache.org" <dev@metron.apache.org>
> Cc: Matt Foley <mfo...@hortonworks.com>
> Subject: Re: [DISCUSS] Upcoming Release
>
> That script seems great, nick!  Perhaps we should adjust the wiki around
> releases to point to it?  Thoughts?
>
> On Fri, Dec 15, 2017 at 1:47 PM, Nick Allen <n...@nickallen.org<mailto:nic
> k...@nickallen.org>> wrote:
> Thanks, Matt.
>
> Maybe you already have something that does this.  I wrote a quick script
> that validates each JIRA since the last release tag to make sure they are
> marked "Done" and with the correct fix version.  I would expect that for
> the next release, each JIRA should have status="Done", fix-version="0.4.2".
>
> Unless I am mistaken, we have quite a few that need cleaned up. In the
> following output, any line that has a URL indicates that a fix is needed.
> Or it least, **I think** a fix is needed.
>
> To the community If you see your name with a URL next to it, it would
> be great if you could follow that link and fix the JIRA.  Otherwise, I will
> volunteer to help clean some of these up should some not get addressed.
>
>
> *$ ./validate-jira-for-release*
> *Cloning into 'metron-0.4.2'...*
> *remote: Counting objects: 35046, done.*
> *remote: Compressing objects: 100% (13698/13698), done.*
> *remote: Total 35046 (delta 15708), reused 31645 (delta 12822)*
> *Receiving objects: 100% (35046/35046), 53.05 MiB | 6.48 MiB/s, done.*
> *Resolving deltas: 100% (15708/15708), done.*
> *Fetching origin*
> *   JIRA  STATUS FIX VERSION
>  ASSIGNEEFIX*
> *METRON-1345Done  Michael
> Miklavcic  https://issues.apache.org/jira/browse/METRON-1345
> <https://issues.apache.org/jira/browse/METRON-1345>*
> *METRON-1349DoneNext + 1 Nick
> Allen  https://issues.apache.org/jira/browse/METRON-1349
> <https://issues.apache.org/jira/browse/METRON-1349>*
> *METRON-1343Done
> Mohan  https://issues.apache.org/jira/browse/METRON-1343
> <https://issues.apache.org/jira/browse/METRON-1343>*
> *METRON-1306   To Do
>  Unassigned  https://issues.apache.org/jira/browse/METRON-1306
> <https://issues.apache.org/jira/browse/METRON-1306>*
> *METRON-1341DoneSimon Elliston
> Ball  https://issues.apache.org/jira/browse/METRON-1341
> <https://issues.apache.org/jira/browse/METRON-1341>*
> *METRON-1313Done Jon
> Zeolla  https://issues.apache.org/jira/browse/METRON-1313
> <https://issues.apache.org/jira/browse/METRON-1313>*
> *METRON-1346DoneOtto
> Fowler  https://issues.apache.org/jira/browse/METRON-1346
> <https://issues.apache.org/jira/browse/METRON-1346>*
> *METRON-1336Done   0.4.2 Nick
> Allen*
> *METRON-1335Done  Anand
> Subramanian  https://issues.apache.org/jira/browse/METRON-1335
> <https://issues.apache.org/jira/browse/METRON-1335>*
> *METRON-1308Done Jon
> Zeolla  https://issues.apache.org/jira/browse/METRON-1308
> <https://issues.apache.org/jira/browse/METRON-1308>*
> *METRON-1338Done   0.4.2 Nick
> Allen*
> *METRON-1286   To Do   0.4.2
>  Unassigned  https://issues.apache.org/jira/browse/METRON-1286
> <https://issues.apache.org/jira/browse/METRON-1286>*
> *METRON-1334Done   0.4.2 Nick
> Allen*
> *METRON-1277DoneOtto
> Fowler  https://issues.apache.org/jira/browse/METRON-1277
> <https://issues.apache.org/jira/browse/METRON-1277>*
> *METRON-1239   To Do
>  Unassigned  https://issues.apache.org/jira/browse/METRON-1239
> <https://issues.apache.org/jira/browse/METRON-1239>*
> *METRON-1328Done  Anand
> Subramanian  https://issues.apache.org/jira/browse/METRON-1328
> <https://issues.apache.org/jira/browse/MET

Re: [DISCUSS] Upcoming Release

2017-12-15 Thread Casey Stella

Yeah, that might work too.  Let's also not forget otto's scripting around
release activity: https://github.com/ottobackwards/Metron-and-Nifi-
Scripts/blob/master/metron/metron-rc-check

We seem to be accumulating a lot of automation to do this, which is great.

On Fri, Dec 15, 2017 at 2:30 PM, Matt Foley <mfo...@hortonworks.com> wrote:

> Perhaps under “build_utils” we should add a subdirectory for
> “release_utils”.
>
> From: Casey Stella <ceste...@gmail.com>
> Date: Friday, December 15, 2017 at 10:50 AM
> To: "dev@metron.apache.org" <dev@metron.apache.org>
> Cc: Matt Foley <mfo...@hortonworks.com>
> Subject: Re: [DISCUSS] Upcoming Release
>
> That script seems great, nick!  Perhaps we should adjust the wiki around
> releases to point to it?  Thoughts?
>
> On Fri, Dec 15, 2017 at 1:47 PM, Nick Allen <n...@nickallen.org<mailto:nic
> k...@nickallen.org>> wrote:
> Thanks, Matt.
>
> Maybe you already have something that does this.  I wrote a quick script
> that validates each JIRA since the last release tag to make sure they are
> marked "Done" and with the correct fix version.  I would expect that for
> the next release, each JIRA should have status="Done", fix-version="0.4.2".
>
> Unless I am mistaken, we have quite a few that need cleaned up. In the
> following output, any line that has a URL indicates that a fix is needed.
> Or it least, **I think** a fix is needed.
>
> To the community If you see your name with a URL next to it, it would
> be great if you could follow that link and fix the JIRA.  Otherwise, I will
> volunteer to help clean some of these up should some not get addressed.
>
>
> *$ ./validate-jira-for-release*
> *Cloning into 'metron-0.4.2'...*
> *remote: Counting objects: 35046, done.*
> *remote: Compressing objects: 100% (13698/13698), done.*
> *remote: Total 35046 (delta 15708), reused 31645 (delta 12822)*
> *Receiving objects: 100% (35046/35046), 53.05 MiB | 6.48 MiB/s, done.*
> *Resolving deltas: 100% (15708/15708), done.*
> *Fetching origin*
> *   JIRA  STATUS FIX VERSION
>  ASSIGNEEFIX*
> *METRON-1345Done  Michael
> Miklavcic  https://issues.apache.org/jira/browse/METRON-1345
> <https://issues.apache.org/jira/browse/METRON-1345>*
> *METRON-1349DoneNext + 1 Nick
> Allen  https://issues.apache.org/jira/browse/METRON-1349
> <https://issues.apache.org/jira/browse/METRON-1349>*
> *METRON-1343Done
> Mohan  https://issues.apache.org/jira/browse/METRON-1343
> <https://issues.apache.org/jira/browse/METRON-1343>*
> *METRON-1306   To Do
>  Unassigned  https://issues.apache.org/jira/browse/METRON-1306
> <https://issues.apache.org/jira/browse/METRON-1306>*
> *METRON-1341DoneSimon Elliston
> Ball  https://issues.apache.org/jira/browse/METRON-1341
> <https://issues.apache.org/jira/browse/METRON-1341>*
> *METRON-1313Done Jon
> Zeolla  https://issues.apache.org/jira/browse/METRON-1313
> <https://issues.apache.org/jira/browse/METRON-1313>*
> *METRON-1346DoneOtto
> Fowler  https://issues.apache.org/jira/browse/METRON-1346
> <https://issues.apache.org/jira/browse/METRON-1346>*
> *METRON-1336Done   0.4.2 Nick
> Allen*
> *METRON-1335Done  Anand
> Subramanian  https://issues.apache.org/jira/browse/METRON-1335
> <https://issues.apache.org/jira/browse/METRON-1335>*
> *METRON-1308Done Jon
> Zeolla  https://issues.apache.org/jira/browse/METRON-1308
> <https://issues.apache.org/jira/browse/METRON-1308>*
> *METRON-1338Done   0.4.2 Nick
> Allen*
> *METRON-1286   To Do   0.4.2
>  Unassigned  https://issues.apache.org/jira/browse/METRON-1286
> <https://issues.apache.org/jira/browse/METRON-1286>*
> *METRON-1334Done   0.4.2 Nick
> Allen*
> *METRON-1277DoneOtto
> Fowler  https://issues.apache.org/jira/browse/METRON-1277
> <https://issues.apache.org/jira/browse/METRON-1277>*
> *METRON-1239   To Do
>  Unassigned  https://issues.apache.org/jira/browse/METRON-1239
> <https://issues.apache.org/jira/browse/METRON-1239>*
> *METRON-1328Done  Anand
> Su

Re: [DISCUSS] Upcoming Release

2017-12-15 Thread Matt Foley

Perhaps under “build_utils” we should add a subdirectory for “release_utils”.

From: Casey Stella <ceste...@gmail.com>
Date: Friday, December 15, 2017 at 10:50 AM
To: "dev@metron.apache.org" <dev@metron.apache.org>
Cc: Matt Foley <mfo...@hortonworks.com>
Subject: Re: [DISCUSS] Upcoming Release

That script seems great, nick!  Perhaps we should adjust the wiki around 
releases to point to it?  Thoughts?

On Fri, Dec 15, 2017 at 1:47 PM, Nick Allen 
<n...@nickallen.org<mailto:n...@nickallen.org>> wrote:
Thanks, Matt.

Maybe you already have something that does this.  I wrote a quick script
that validates each JIRA since the last release tag to make sure they are
marked "Done" and with the correct fix version.  I would expect that for
the next release, each JIRA should have status="Done", fix-version="0.4.2".

Unless I am mistaken, we have quite a few that need cleaned up. In the
following output, any line that has a URL indicates that a fix is needed.
Or it least, **I think** a fix is needed.

To the community If you see your name with a URL next to it, it would
be great if you could follow that link and fix the JIRA.  Otherwise, I will
volunteer to help clean some of these up should some not get addressed.


*$ ./validate-jira-for-release*
*Cloning into 'metron-0.4.2'...*
*remote: Counting objects: 35046, done.*
*remote: Compressing objects: 100% (13698/13698), done.*
*remote: Total 35046 (delta 15708), reused 31645 (delta 12822)*
*Receiving objects: 100% (35046/35046), 53.05 MiB | 6.48 MiB/s, done.*
*Resolving deltas: 100% (15708/15708), done.*
*Fetching origin*
*   JIRA  STATUS FIX VERSION
 ASSIGNEEFIX*
*METRON-1345Done  Michael
Miklavcic  https://issues.apache.org/jira/browse/METRON-1345
<https://issues.apache.org/jira/browse/METRON-1345>*
*METRON-1349DoneNext + 1 Nick
Allen  https://issues.apache.org/jira/browse/METRON-1349
<https://issues.apache.org/jira/browse/METRON-1349>*
*METRON-1343Done
Mohan  https://issues.apache.org/jira/browse/METRON-1343
<https://issues.apache.org/jira/browse/METRON-1343>*
*METRON-1306   To Do
 Unassigned  https://issues.apache.org/jira/browse/METRON-1306
<https://issues.apache.org/jira/browse/METRON-1306>*
*METRON-1341DoneSimon Elliston
Ball  https://issues.apache.org/jira/browse/METRON-1341
<https://issues.apache.org/jira/browse/METRON-1341>*
*METRON-1313Done Jon
Zeolla  https://issues.apache.org/jira/browse/METRON-1313
<https://issues.apache.org/jira/browse/METRON-1313>*
*METRON-1346DoneOtto
Fowler  https://issues.apache.org/jira/browse/METRON-1346
<https://issues.apache.org/jira/browse/METRON-1346>*
*METRON-1336Done   0.4.2 Nick
Allen*
*METRON-1335Done  Anand
Subramanian  https://issues.apache.org/jira/browse/METRON-1335
<https://issues.apache.org/jira/browse/METRON-1335>*
*METRON-1308Done Jon
Zeolla  https://issues.apache.org/jira/browse/METRON-1308
<https://issues.apache.org/jira/browse/METRON-1308>*
*METRON-1338Done   0.4.2 Nick
Allen*
*METRON-1286   To Do   0.4.2
 Unassigned  https://issues.apache.org/jira/browse/METRON-1286
<https://issues.apache.org/jira/browse/METRON-1286>*
*METRON-1334Done   0.4.2 Nick
Allen*
*METRON-1277DoneOtto
Fowler  https://issues.apache.org/jira/browse/METRON-1277
<https://issues.apache.org/jira/browse/METRON-1277>*
*METRON-1239   To Do
 Unassigned  https://issues.apache.org/jira/browse/METRON-1239
<https://issues.apache.org/jira/browse/METRON-1239>*
*METRON-1328Done  Anand
Subramanian  https://issues.apache.org/jira/browse/METRON-1328
<https://issues.apache.org/jira/browse/METRON-1328>*
*METRON-1333DoneOtto
Fowler  https://issues.apache.org/jira/browse/METRON-1333
<https://issues.apache.org/jira/browse/METRON-1333>*
*METRON-1252Done
 RaghuMitra  https://issues.apache.org/jira/browse/METRON-1252
<https://issues.apache.org/jira/browse/METRON-1252>*
*METRON-1316   To DoNext + 1
 Unassigned  https://issues.apache.org/jira/browse/METRON-1316
<https://issues.apache.org/jira/browse/METRON-1316>*
*METRON-1088Done Jon
Zeolla  https://issues.apache.org/jira/browse/METRO

Re: [DISCUSS] Upcoming Release

2017-12-15 Thread Casey Stella

pletion helps here – OR "Docs
> Text"
> > is not EMPTY.  Each of these needs to be looked at carefully, verified,
> and
> > if Doc Text is missing, the contributor and I cooperate to write a couple
> > sentences.  The Doc Text then needs to go in the RELEASE_NOTES for the
> > release, in the section “## Non-backward-compatible Changes in this
> > Release, and Upgrade Suggestions”.
> >
> > That’s about it.  I guess I should add the above as an appendix doc to
> the
> > Release Process documentation.  I’ll do that.
> > Cheers,
> > --Matt
> >
> >
> > On 12/15/17, 9:15 AM, "Nick Allen" <n...@nickallen.org> wrote:
> >
> > Hi Matt -
> >
> > I just updated like 15+ JIRAs of my own JIRAs that I completed and
> > merged,
> > but failed to mark as resolved.  All of these will be included in
> > 0.4.2.  I
> > updated each to be "fixed" in version 0.4.2 and marked as resolved.
> > Hopefully the next RC will report those as fixed.
> >
> > (Q) Where does the list of JIRAs that get attached to the release
> > originate
> > from?  Does it get pulled out of JIRA or do they come from the commit
> > log?
> >
> > My apologies for not staying on top of my JIRAs.
> >
> >
> >
> >
> > On Tue, Dec 12, 2017 at 2:21 PM, Matt Foley <ma...@apache.org>
> wrote:
> >
> > > Thanks to Jon, Otto, and Nick for looking over RC1.  Tonight I’ll
> > fix the
> > > RAT glitch, build RC2, and put it to formal vote.
> > > Regards,
> > > --Matt
> > >
> > > On 12/12/17, 5:14 AM, "Nick Allen" <n...@nickallen.org> wrote:
> > >
> > > RC1 is looking good to me.  I validated the MD5s, built Metron,
> > built
> > > the
> > > Bro plugin and reviewed the other artifacts like release notes.
> > >
> > > Running the RAT check on a 'clean' Metron does not produce any
> > errors
> > > for
> > > me.  It is only after building Metron, which pulls in
> additional
> > Node
> > > dependencies, does the RAT check fail.
> > >
> > >
> > > On Sun, Dec 10, 2017 at 4:41 PM Matt Foley <ma...@apache.org>
> > wrote:
> > >
> > > > Yes, but let’s see if anyone else find other issues.
> > > >
> > > >
> > > >
> > > > From: Otto Fowler <ottobackwa...@gmail.com>
> > > > Date: Saturday, December 9, 2017 at 6:16 AM
> > > > To: Matt Foley <mfo...@hortonworks.com>, "
> > dev@metron.apache.org" <
> > > > dev@metron.apache.org>
> > > > Subject: Re: [DISCUSS] Upcoming Release
> > > >
> > > >
> > > >
> > > > So RC2 then?
> > > >
> > > >
> > > >
> > > > On December 8, 2017 at 20:43:21, Matt Foley (
> > mfo...@hortonworks.com)
> > > > wrote:
> > > >
> > > > Hah, here it is: https://github.com/apache/metron/pull/743
> > > > “This problem seems to only reproduce when one unrolls a
> > tarball
> > > rather
> > > > than cloning from github.”
> > > >
> > > > Heh, the exclusion at
> > > > https://github.com/apache/metron/blob/master/pom.xml#L351 is
> > still
> > > there,
> > > > but the hashcode in the bundle.css file name has changed from
> > > > a0b6b99c10d9a13dc67e to f56deed131e58bd7ee04. Sigh. Did the
> > version
> > > of Font
> > > > Awesome fonts change?
> > > >
> > > >
> > > > On 12/8/17, 5:26 PM, "Matt Foley" <ma...@apache.org> wrote:
> > > >
> > > > I remember having trouble with this bundle.css file on the
> last
> > > release,
> > > > but I can’t remember what we did about it. Anybody?
> > > >
> > > > On 12/8/17, 1:41 PM, "Otto Fowler" <ottobackwa...@gmail.com>
> > wrote:
> > > >
> > > > Steps
> > > >
> > > > - Downloaded tar.gz’s, asc f

Re: [DISCUSS] Upcoming Release

2017-12-15 Thread Nick Allen

 defined before I started managing
> releases, but I think it’s a reasonable approach.  It’s specified in our
> Release Process document, https://cwiki.apache.org/
> confluence/display/METRON/Release+Process , Step 5, the last bullet:
> “The artifacts for a release or RC [include]… A CHANGES file
> denoting the changes [since the last release].
> We usually construct this by taking the output of git log | grep
> METRON | sed 's/\[//g' | sed 's/\]//g' | grep -v "http" and removing the
> JIRAs from the previous releases (it’s in time sorted order so this is
> easy).”
>
> However, the release manager also has a responsibility to “Monitor and
> Verify Jiras” (Step 2 and various other places in the doc).  Altho the doc
> isn’t explicit about it, I take this responsibility a little farther and do
> the following as part of Jira verification:
>
> 1. Extract the jira id’s from the CHANGES file with a simple awk script,
> put them in a Jira query, and confirm they are all actually marked
> “Fixed/Resolved” with Fix Version =  in Jira.  If
> there are exceptions, I dun the contributors to fix it.  You’ve seen these
> emails from me in the past couple releases.  I don’t just mark them fixed
> myself, because some contributions (perfectly legitimately) only partially
> address a problem, and just marking a ticket “Fixed” may not be the right
> thing to do.
>
> 2. Query jira for any tickets NOT included in the prior list, that ARE
> marked fixed in the current release (or later, or “Next”, or similar
> “future” version constructs).  These don’t usually happen (considering that
> Metron contributors mostly live inside github PR processes rather than
> Jiras) but when they do they also need to be resolved.  Cases I’ve seen (in
> Metron and other projects) include:
> a) If they were indeed fixed in the current release, but not reflected in
> the commit message, I’ll annotate this fact in the Jira ticket, and
> hand-edit the CHANGES file for this release.  (Doesn’t seem worth trying to
> edit old commit messages, since that mucks up the SHA1’s.)
> b) If they were fixed in a previous release but mis-marked as to Fix
> Version in the ticket, fix the ticket.
> c) If they aren’t really fixed, fix the ticket.
>
> 3. Once the set of Fixed tickets is complete and correct, query them for
> any that have “labels in (backward-incompatible, backwards-compatibility,
> backwards-incompatible)” -- yes, there shouldn’t be 3 labels, but it’s hard
> to fix because Jira never forgets, so every time someone gets it wrong, it
> adds to the set; fortunately, query completion helps here – OR "Docs Text"
> is not EMPTY.  Each of these needs to be looked at carefully, verified, and
> if Doc Text is missing, the contributor and I cooperate to write a couple
> sentences.  The Doc Text then needs to go in the RELEASE_NOTES for the
> release, in the section “## Non-backward-compatible Changes in this
> Release, and Upgrade Suggestions”.
>
> That’s about it.  I guess I should add the above as an appendix doc to the
> Release Process documentation.  I’ll do that.
> Cheers,
> --Matt
>
>
> On 12/15/17, 9:15 AM, "Nick Allen" <n...@nickallen.org> wrote:
>
> Hi Matt -
>
> I just updated like 15+ JIRAs of my own JIRAs that I completed and
> merged,
> but failed to mark as resolved.  All of these will be included in
> 0.4.2.  I
> updated each to be "fixed" in version 0.4.2 and marked as resolved.
> Hopefully the next RC will report those as fixed.
>
> (Q) Where does the list of JIRAs that get attached to the release
> originate
> from?  Does it get pulled out of JIRA or do they come from the commit
> log?
>
> My apologies for not staying on top of my JIRAs.
>
>
>
>
> On Tue, Dec 12, 2017 at 2:21 PM, Matt Foley <ma...@apache.org> wrote:
>
> > Thanks to Jon, Otto, and Nick for looking over RC1.  Tonight I’ll
> fix the
> > RAT glitch, build RC2, and put it to formal vote.
> > Regards,
> > --Matt
> >
> > On 12/12/17, 5:14 AM, "Nick Allen" <n...@nickallen.org> wrote:
> >
> > RC1 is looking good to me.  I validated the MD5s, built Metron,
> built
> > the
> > Bro plugin and reviewed the other artifacts like release notes.
> >
> > Running the RAT check on a 'clean' Metron does not produce any
> errors
> > for
> > me.  It is only after building Metron, which pulls in additional
> Node
> > dependencies, does the RAT check fail.
> >
> >
> > On Sun, Dec 10, 2017 at 4:41 PM Matt Foley <ma...@apache.org&g

Re: [DISCUSS] Upcoming Release

2017-12-15 Thread Nick Allen

Also, METRON-1349 [1] was merged into master, but I am not sure if that
will be included in the RC.  So I resolved it and marked it as "Next + 1".

[1] https://issues.apache.org/jira/browse/METRON-1349

On Fri, Dec 15, 2017 at 12:14 PM, Nick Allen <n...@nickallen.org> wrote:

> Hi Matt -
>
> I just updated like 15+ JIRAs of my own JIRAs that I completed and merged,
> but failed to mark as resolved.  All of these will be included in 0.4.2.  I
> updated each to be "fixed" in version 0.4.2 and marked as resolved.
> Hopefully the next RC will report those as fixed.
>
> (Q) Where does the list of JIRAs that get attached to the release
> originate from?  Does it get pulled out of JIRA or do they come from the
> commit log?
>
> My apologies for not staying on top of my JIRAs.
>
>
>
>
> On Tue, Dec 12, 2017 at 2:21 PM, Matt Foley <ma...@apache.org> wrote:
>
>> Thanks to Jon, Otto, and Nick for looking over RC1.  Tonight I’ll fix the
>> RAT glitch, build RC2, and put it to formal vote.
>> Regards,
>> --Matt
>>
>> On 12/12/17, 5:14 AM, "Nick Allen" <n...@nickallen.org> wrote:
>>
>> RC1 is looking good to me.  I validated the MD5s, built Metron, built
>> the
>> Bro plugin and reviewed the other artifacts like release notes.
>>
>> Running the RAT check on a 'clean' Metron does not produce any errors
>> for
>> me.  It is only after building Metron, which pulls in additional Node
>> dependencies, does the RAT check fail.
>>
>>
>> On Sun, Dec 10, 2017 at 4:41 PM Matt Foley <ma...@apache.org> wrote:
>>
>> > Yes, but let’s see if anyone else find other issues.
>> >
>> >
>> >
>> > From: Otto Fowler <ottobackwa...@gmail.com>
>> > Date: Saturday, December 9, 2017 at 6:16 AM
>> > To: Matt Foley <mfo...@hortonworks.com>, "dev@metron.apache.org" <
>> > dev@metron.apache.org>
>> > Subject: Re: [DISCUSS] Upcoming Release
>> >
>> >
>> >
>> > So RC2 then?
>> >
>> >
>> >
>> > On December 8, 2017 at 20:43:21, Matt Foley (mfo...@hortonworks.com
>> )
>> > wrote:
>> >
>> > Hah, here it is: https://github.com/apache/metron/pull/743
>> > “This problem seems to only reproduce when one unrolls a tarball
>> rather
>> > than cloning from github.”
>> >
>> > Heh, the exclusion at
>> > https://github.com/apache/metron/blob/master/pom.xml#L351 is still
>> there,
>> > but the hashcode in the bundle.css file name has changed from
>> > a0b6b99c10d9a13dc67e to f56deed131e58bd7ee04. Sigh. Did the version
>> of Font
>> > Awesome fonts change?
>> >
>> >
>> > On 12/8/17, 5:26 PM, "Matt Foley" <ma...@apache.org> wrote:
>> >
>> > I remember having trouble with this bundle.css file on the last
>> release,
>> > but I can’t remember what we did about it. Anybody?
>> >
>> > On 12/8/17, 1:41 PM, "Otto Fowler" <ottobackwa...@gmail.com> wrote:
>> >
>> > Steps
>> >
>> > - Downloaded tar.gz’s, asc files and KEYS
>> > - Verified signing of both tar.gz’s
>> > - searched for rouge 0.4.1 entries
>> > - verified the main pom.xml
>> > - built :
>> >
>> > mvn clean && time mvn -q -T 2C -DskipTests install && time mvn -q -T
>> > 2C surefire:test@unit-tests && time mvn -q
>> > surefire:test@integration-tests && time mvn -q test --projects
>> > metron-interface/metron-config && time
>> build_utils/verify_licenses.sh
>> >
>> > Found rat error:
>> >
>> >
>> > *
>> > Summary
>> > ---
>> > Generated at: 2017-12-08T16:33:27-05:00
>> >
>> > Notes: 3
>> > Binaries: 193
>> > Archives: 0
>> > Standards: 75
>> >
>> > Apache Licensed: 74
>> > Generated Documents: 0
>> >
>> > JavaDocs are generated, thus a license header is optional.
>> > Generated files do not require license headers.
>> >
>> > 1 Unknown Licenses
>>

Re: [DISCUSS] Upcoming Release

2017-12-15 Thread Matt Foley

Hi Nick,
Good timing, you’ve saved me some work! :-)

Origin of the list:  The approach was defined before I started managing 
releases, but I think it’s a reasonable approach.  It’s specified in our 
Release Process document, 
https://cwiki.apache.org/confluence/display/METRON/Release+Process , Step 5, 
the last bullet:
“The artifacts for a release or RC [include]… A CHANGES file denoting 
the changes [since the last release].
We usually construct this by taking the output of git log | grep METRON 
| sed 's/\[//g' | sed 's/\]//g' | grep -v "http" and removing the JIRAs from 
the previous releases (it’s in time sorted order so this is easy).”

However, the release manager also has a responsibility to “Monitor and Verify 
Jiras” (Step 2 and various other places in the doc).  Altho the doc isn’t 
explicit about it, I take this responsibility a little farther and do the 
following as part of Jira verification:

1. Extract the jira id’s from the CHANGES file with a simple awk script, put 
them in a Jira query, and confirm they are all actually marked “Fixed/Resolved” 
with Fix Version =  in Jira.  If there are exceptions, I 
dun the contributors to fix it.  You’ve seen these emails from me in the past 
couple releases.  I don’t just mark them fixed myself, because some 
contributions (perfectly legitimately) only partially address a problem, and 
just marking a ticket “Fixed” may not be the right thing to do.

2. Query jira for any tickets NOT included in the prior list, that ARE marked 
fixed in the current release (or later, or “Next”, or similar “future” version 
constructs).  These don’t usually happen (considering that Metron contributors 
mostly live inside github PR processes rather than Jiras) but when they do they 
also need to be resolved.  Cases I’ve seen (in Metron and other projects) 
include:
a) If they were indeed fixed in the current release, but not reflected in the 
commit message, I’ll annotate this fact in the Jira ticket, and hand-edit the 
CHANGES file for this release.  (Doesn’t seem worth trying to edit old commit 
messages, since that mucks up the SHA1’s.)
b) If they were fixed in a previous release but mis-marked as to Fix Version in 
the ticket, fix the ticket.
c) If they aren’t really fixed, fix the ticket.

3. Once the set of Fixed tickets is complete and correct, query them for any 
that have “labels in (backward-incompatible, backwards-compatibility, 
backwards-incompatible)” -- yes, there shouldn’t be 3 labels, but it’s hard to 
fix because Jira never forgets, so every time someone gets it wrong, it adds to 
the set; fortunately, query completion helps here – OR "Docs Text" is not 
EMPTY.  Each of these needs to be looked at carefully, verified, and if Doc 
Text is missing, the contributor and I cooperate to write a couple sentences.  
The Doc Text then needs to go in the RELEASE_NOTES for the release, in the 
section “## Non-backward-compatible Changes in this Release, and Upgrade 
Suggestions”.

That’s about it.  I guess I should add the above as an appendix doc to the 
Release Process documentation.  I’ll do that.
Cheers,
--Matt

On 12/15/17, 9:15 AM, "Nick Allen" <n...@nickallen.org> wrote:

Hi Matt -

I just updated like 15+ JIRAs of my own JIRAs that I completed and merged,
but failed to mark as resolved.  All of these will be included in 0.4.2.  I
updated each to be "fixed" in version 0.4.2 and marked as resolved.
Hopefully the next RC will report those as fixed.

(Q) Where does the list of JIRAs that get attached to the release originate
from?  Does it get pulled out of JIRA or do they come from the commit log?

My apologies for not staying on top of my JIRAs.

On Tue, Dec 12, 2017 at 2:21 PM, Matt Foley <ma...@apache.org> wrote:

> Thanks to Jon, Otto, and Nick for looking over RC1.  Tonight I’ll fix the
> RAT glitch, build RC2, and put it to formal vote.
> Regards,
> --Matt
>
> On 12/12/17, 5:14 AM, "Nick Allen" <n...@nickallen.org> wrote:
>
> RC1 is looking good to me.  I validated the MD5s, built Metron, built
> the
> Bro plugin and reviewed the other artifacts like release notes.
>
> Running the RAT check on a 'clean' Metron does not produce any errors
> for
> me.  It is only after building Metron, which pulls in additional Node
> dependencies, does the RAT check fail.
>
>
> On Sun, Dec 10, 2017 at 4:41 PM Matt Foley <ma...@apache.org> wrote:
>
> > Yes, but let’s see if anyone else find other issues.
> >
> >
> >
> > From: Otto Fowler <ottobackwa...@gmail.com>
> > Date: Saturday, December 9, 2017 at 6:16 AM
> > To: Matt Foley <mfo...@hor

Re: [DISCUSS] Upcoming Release

2017-12-15 Thread Nick Allen

Hi Matt -

I just updated like 15+ JIRAs of my own JIRAs that I completed and merged,
but failed to mark as resolved.  All of these will be included in 0.4.2.  I
updated each to be "fixed" in version 0.4.2 and marked as resolved.
Hopefully the next RC will report those as fixed.

(Q) Where does the list of JIRAs that get attached to the release originate
from?  Does it get pulled out of JIRA or do they come from the commit log?

My apologies for not staying on top of my JIRAs.




On Tue, Dec 12, 2017 at 2:21 PM, Matt Foley <ma...@apache.org> wrote:

> Thanks to Jon, Otto, and Nick for looking over RC1.  Tonight I’ll fix the
> RAT glitch, build RC2, and put it to formal vote.
> Regards,
> --Matt
>
> On 12/12/17, 5:14 AM, "Nick Allen" <n...@nickallen.org> wrote:
>
> RC1 is looking good to me.  I validated the MD5s, built Metron, built
> the
> Bro plugin and reviewed the other artifacts like release notes.
>
> Running the RAT check on a 'clean' Metron does not produce any errors
> for
> me.  It is only after building Metron, which pulls in additional Node
> dependencies, does the RAT check fail.
>
>
> On Sun, Dec 10, 2017 at 4:41 PM Matt Foley <ma...@apache.org> wrote:
>
> > Yes, but let’s see if anyone else find other issues.
> >
> >
> >
> > From: Otto Fowler <ottobackwa...@gmail.com>
> > Date: Saturday, December 9, 2017 at 6:16 AM
> > To: Matt Foley <mfo...@hortonworks.com>, "dev@metron.apache.org" <
> > dev@metron.apache.org>
> > Subject: Re: [DISCUSS] Upcoming Release
> >
> >
> >
> > So RC2 then?
> >
> >
> >
> > On December 8, 2017 at 20:43:21, Matt Foley (mfo...@hortonworks.com)
> > wrote:
> >
> > Hah, here it is: https://github.com/apache/metron/pull/743
> > “This problem seems to only reproduce when one unrolls a tarball
> rather
> > than cloning from github.”
> >
> > Heh, the exclusion at
> > https://github.com/apache/metron/blob/master/pom.xml#L351 is still
> there,
> > but the hashcode in the bundle.css file name has changed from
> > a0b6b99c10d9a13dc67e to f56deed131e58bd7ee04. Sigh. Did the version
> of Font
> > Awesome fonts change?
> >
> >
> > On 12/8/17, 5:26 PM, "Matt Foley" <ma...@apache.org> wrote:
> >
> > I remember having trouble with this bundle.css file on the last
> release,
> > but I can’t remember what we did about it. Anybody?
> >
> > On 12/8/17, 1:41 PM, "Otto Fowler" <ottobackwa...@gmail.com> wrote:
> >
> > Steps
> >
> > - Downloaded tar.gz’s, asc files and KEYS
> > - Verified signing of both tar.gz’s
> > - searched for rouge 0.4.1 entries
> > - verified the main pom.xml
> > - built :
> >
> > mvn clean && time mvn -q -T 2C -DskipTests install && time mvn -q -T
> > 2C surefire:test@unit-tests && time mvn -q
> > surefire:test@integration-tests && time mvn -q test --projects
> > metron-interface/metron-config && time build_utils/verify_licenses.sh
> >
> > Found rat error:
> >
> >
> > *
> > Summary
> > ---
> > Generated at: 2017-12-08T16:33:27-05:00
> >
> > Notes: 3
> > Binaries: 193
> > Archives: 0
> > Standards: 75
> >
> > Apache Licensed: 74
> > Generated Documents: 0
> >
> > JavaDocs are generated, thus a license header is optional.
> > Generated files do not require license headers.
> >
> > 1 Unknown Licenses
> >
> > *
> >
> > Files with unapproved licenses:
> >
> >
> > /Users/batman/tmp/release_ver/apache-metron-0.4.2-rc1/
> metron-interface/metron-alerts/dist/styles.f56deed131e58bd7ee04.bundle.css
> >
> > *
> >
> >
> >
> >
> >
> > *
> > Summary
> > ---
> > Generated at: 2017-12-08T16:33:27-05:00
> >
> > Notes: 3
> > Binaries: 193
> > Archives: 0
> > Standards: 75
> >
> > Apache Licensed: 74
>

Re: [DISCUSS] Upcoming Release

2017-12-12 Thread zeo...@gmail.com

So I was looking at some of the docs and saw that Upgrading.md has a
dangling section that maybe should be removed.  Link
<https://github.com/apache/metron/blob/master/Upgrading.md#description>,
link <https://github.com/apache/metron/pull/780>

Jon

On Tue, Dec 12, 2017 at 2:21 PM Matt Foley <ma...@apache.org> wrote:

> Thanks to Jon, Otto, and Nick for looking over RC1.  Tonight I’ll fix the
> RAT glitch, build RC2, and put it to formal vote.
> Regards,
> --Matt
>
> On 12/12/17, 5:14 AM, "Nick Allen" <n...@nickallen.org> wrote:
>
> RC1 is looking good to me.  I validated the MD5s, built Metron, built
> the
> Bro plugin and reviewed the other artifacts like release notes.
>
> Running the RAT check on a 'clean' Metron does not produce any errors
> for
> me.  It is only after building Metron, which pulls in additional Node
> dependencies, does the RAT check fail.
>
>
> On Sun, Dec 10, 2017 at 4:41 PM Matt Foley <ma...@apache.org> wrote:
>
> > Yes, but let’s see if anyone else find other issues.
> >
> >
> >
> > From: Otto Fowler <ottobackwa...@gmail.com>
> > Date: Saturday, December 9, 2017 at 6:16 AM
>     > To: Matt Foley <mfo...@hortonworks.com>, "dev@metron.apache.org" <
> > dev@metron.apache.org>
> > Subject: Re: [DISCUSS] Upcoming Release
> >
> >
> >
> > So RC2 then?
> >
> >
> >
> > On December 8, 2017 at 20:43:21, Matt Foley (mfo...@hortonworks.com)
> > wrote:
> >
> > Hah, here it is: https://github.com/apache/metron/pull/743
> > “This problem seems to only reproduce when one unrolls a tarball
> rather
> > than cloning from github.”
> >
> > Heh, the exclusion at
> > https://github.com/apache/metron/blob/master/pom.xml#L351 is still
> there,
> > but the hashcode in the bundle.css file name has changed from
> > a0b6b99c10d9a13dc67e to f56deed131e58bd7ee04. Sigh. Did the version
> of Font
> > Awesome fonts change?
> >
> >
> > On 12/8/17, 5:26 PM, "Matt Foley" <ma...@apache.org> wrote:
> >
> > I remember having trouble with this bundle.css file on the last
> release,
> > but I can’t remember what we did about it. Anybody?
> >
> > On 12/8/17, 1:41 PM, "Otto Fowler" <ottobackwa...@gmail.com> wrote:
> >
> > Steps
> >
> > - Downloaded tar.gz’s, asc files and KEYS
> > - Verified signing of both tar.gz’s
> > - searched for rouge 0.4.1 entries
> > - verified the main pom.xml
> > - built :
> >
> > mvn clean && time mvn -q -T 2C -DskipTests install && time mvn -q -T
> > 2C surefire:test@unit-tests && time mvn -q
> > surefire:test@integration-tests && time mvn -q test --projects
> > metron-interface/metron-config && time build_utils/verify_licenses.sh
> >
> > Found rat error:
> >
> >
> > *
> > Summary
> > ---
> > Generated at: 2017-12-08T16:33:27-05:00
> >
> > Notes: 3
> > Binaries: 193
> > Archives: 0
> > Standards: 75
> >
> > Apache Licensed: 74
> > Generated Documents: 0
> >
> > JavaDocs are generated, thus a license header is optional.
> > Generated files do not require license headers.
> >
> > 1 Unknown Licenses
> >
> > *
> >
> > Files with unapproved licenses:
> >
> >
> >
> /Users/batman/tmp/release_ver/apache-metron-0.4.2-rc1/metron-interface/metron-alerts/dist/styles.f56deed131e58bd7ee04.bundle.css
> >
> > *
> >
> >
> >
> >
> >
> > *
> > Summary
> > ---
> > Generated at: 2017-12-08T16:33:27-05:00
> >
> > Notes: 3
> > Binaries: 193
> > Archives: 0
> > Standards: 75
> >
> > Apache Licensed: 74
> > Generated Documents: 0
> >
> > JavaDocs are generated, thus a license header is optional.
> > Generated files do not require license headers.
> >
> > 1 Unknown Licenses
>

Re: [DISCUSS] Upcoming Release

2017-12-12 Thread Nick Allen

RC1 is looking good to me.  I validated the MD5s, built Metron, built the
Bro plugin and reviewed the other artifacts like release notes.

Running the RAT check on a 'clean' Metron does not produce any errors for
me.  It is only after building Metron, which pulls in additional Node
dependencies, does the RAT check fail.



On Sun, Dec 10, 2017 at 4:41 PM Matt Foley <ma...@apache.org> wrote:

> Yes, but let’s see if anyone else find other issues.
>
>
>
> From: Otto Fowler <ottobackwa...@gmail.com>
> Date: Saturday, December 9, 2017 at 6:16 AM
> To: Matt Foley <mfo...@hortonworks.com>, "dev@metron.apache.org" <
> dev@metron.apache.org>
> Subject: Re: [DISCUSS] Upcoming Release
>
>
>
> So RC2 then?
>
>
>
> On December 8, 2017 at 20:43:21, Matt Foley (mfo...@hortonworks.com)
> wrote:
>
> Hah, here it is: https://github.com/apache/metron/pull/743
> “This problem seems to only reproduce when one unrolls a tarball rather
> than cloning from github.”
>
> Heh, the exclusion at
> https://github.com/apache/metron/blob/master/pom.xml#L351 is still there,
> but the hashcode in the bundle.css file name has changed from
> a0b6b99c10d9a13dc67e to f56deed131e58bd7ee04. Sigh. Did the version of Font
> Awesome fonts change?
>
>
> On 12/8/17, 5:26 PM, "Matt Foley" <ma...@apache.org> wrote:
>
> I remember having trouble with this bundle.css file on the last release,
> but I can’t remember what we did about it. Anybody?
>
> On 12/8/17, 1:41 PM, "Otto Fowler" <ottobackwa...@gmail.com> wrote:
>
> Steps
>
> - Downloaded tar.gz’s, asc files and KEYS
> - Verified signing of both tar.gz’s
> - searched for rouge 0.4.1 entries
> - verified the main pom.xml
> - built :
>
> mvn clean && time mvn -q -T 2C -DskipTests install && time mvn -q -T
> 2C surefire:test@unit-tests && time mvn -q
> surefire:test@integration-tests && time mvn -q test --projects
> metron-interface/metron-config && time build_utils/verify_licenses.sh
>
> Found rat error:
>
>
> *
> Summary
> ---
> Generated at: 2017-12-08T16:33:27-05:00
>
> Notes: 3
> Binaries: 193
> Archives: 0
> Standards: 75
>
> Apache Licensed: 74
> Generated Documents: 0
>
> JavaDocs are generated, thus a license header is optional.
> Generated files do not require license headers.
>
> 1 Unknown Licenses
>
> *
>
> Files with unapproved licenses:
>
>
> /Users/batman/tmp/release_ver/apache-metron-0.4.2-rc1/metron-interface/metron-alerts/dist/styles.f56deed131e58bd7ee04.bundle.css
>
> *
>
>
>
>
>
> *
> Summary
> ---
> Generated at: 2017-12-08T16:33:27-05:00
>
> Notes: 3
> Binaries: 193
> Archives: 0
> Standards: 75
>
> Apache Licensed: 74
> Generated Documents: 0
>
> JavaDocs are generated, thus a license header is optional.
> Generated files do not require license headers.
>
> 1 Unknown Licenses
>
> *
>
> Files with unapproved licenses:
>
>
>
> /Users/ottofowler/tmp/release_ver/apache-metron-0.4.2-rc1/metron-interface/metron-alerts/dist/styles.f56deed131e58bd7ee04.bundle.css
>
> *
>
>
>
> On December 8, 2017 at 04:34:24, Matt Foley (ma...@apache.org) wrote:
>
> Colleagues,
> I’ve posted Metron-0.4.2-RC1 and Metron-bro-plugin-kafka-0.1 to
> https://dist.apache.org/repos/dist/dev/metron/0.4.2-RC1/
>
> Given the complexity of this RC, I’d appreciate if a couple people would be
> willing to kick the tires before we put it up for a vote.
>
> I will myself be going thru the Verify Build process this weekend, as I
> won’t be able to do it Friday.
>
> Thanks,
> --Matt
>
>
> On 12/4/17, 2:05 PM, "zeo...@gmail.com" <zeo...@gmail.com> wrote:
>
> Can we resolve the conversation regarding the second repo? I was waiting
> to get more input/preferences from people There's also a documentation
> update that fixes a few broken Stellar docs that already has aa +1, I just
> need to merge it.
>
> Jon
>
> On Mon, Dec 4, 2017, 17:01 Casey Stella <ceste...@gmail.com> wrote:
>
> > I would be in favor of a release at this point.
> >
> > On Mon, Dec 4, 2017 at 4:57 PM, Matt Foley <ma...@apache.org> wrote:
> >
> > > Hey all,
> > > I see METRON-1252 was resolved over the weekend. Shall I go ahead and
> > >

Re: [DISCUSS] Upcoming Release

2017-12-10 Thread Matt Foley

Yes, but let’s see if anyone else find other issues.

 

From: Otto Fowler <ottobackwa...@gmail.com>
Date: Saturday, December 9, 2017 at 6:16 AM
To: Matt Foley <mfo...@hortonworks.com>, "dev@metron.apache.org" 
<dev@metron.apache.org>
Subject: Re: [DISCUSS] Upcoming Release

 

So RC2 then?

 

On December 8, 2017 at 20:43:21, Matt Foley (mfo...@hortonworks.com) wrote:

Hah, here it is: https://github.com/apache/metron/pull/743 
“This problem seems to only reproduce when one unrolls a tarball rather than 
cloning from github.” 

Heh, the exclusion at https://github.com/apache/metron/blob/master/pom.xml#L351 
is still there, but the hashcode in the bundle.css file name has changed from 
a0b6b99c10d9a13dc67e to f56deed131e58bd7ee04. Sigh. Did the version of Font 
Awesome fonts change? 


On 12/8/17, 5:26 PM, "Matt Foley" <ma...@apache.org> wrote: 

I remember having trouble with this bundle.css file on the last release, but I 
can’t remember what we did about it. Anybody? 

On 12/8/17, 1:41 PM, "Otto Fowler" <ottobackwa...@gmail.com> wrote: 

Steps 

- Downloaded tar.gz’s, asc files and KEYS 
- Verified signing of both tar.gz’s 
- searched for rouge 0.4.1 entries 
- verified the main pom.xml 
- built : 

mvn clean && time mvn -q -T 2C -DskipTests install && time mvn -q -T 
2C surefire:test@unit-tests && time mvn -q 
surefire:test@integration-tests && time mvn -q test --projects 
metron-interface/metron-config && time build_utils/verify_licenses.sh 

Found rat error: 


* 
Summary 
--- 
Generated at: 2017-12-08T16:33:27-05:00 

Notes: 3 
Binaries: 193 
Archives: 0 
Standards: 75 

Apache Licensed: 74 
Generated Documents: 0 

JavaDocs are generated, thus a license header is optional. 
Generated files do not require license headers. 

1 Unknown Licenses 

* 

Files with unapproved licenses: 

/Users/batman/tmp/release_ver/apache-metron-0.4.2-rc1/metron-interface/metron-alerts/dist/styles.f56deed131e58bd7ee04.bundle.css
 

* 





* 
Summary 
--- 
Generated at: 2017-12-08T16:33:27-05:00 

Notes: 3 
Binaries: 193 
Archives: 0 
Standards: 75 

Apache Licensed: 74 
Generated Documents: 0 

JavaDocs are generated, thus a license header is optional. 
Generated files do not require license headers. 

1 Unknown Licenses 

* 

Files with unapproved licenses: 


/Users/ottofowler/tmp/release_ver/apache-metron-0.4.2-rc1/metron-interface/metron-alerts/dist/styles.f56deed131e58bd7ee04.bundle.css
 

* 



On December 8, 2017 at 04:34:24, Matt Foley (ma...@apache.org) wrote: 

Colleagues, 
I’ve posted Metron-0.4.2-RC1 and Metron-bro-plugin-kafka-0.1 to 
https://dist.apache.org/repos/dist/dev/metron/0.4.2-RC1/ 

Given the complexity of this RC, I’d appreciate if a couple people would be 
willing to kick the tires before we put it up for a vote. 

I will myself be going thru the Verify Build process this weekend, as I 
won’t be able to do it Friday. 

Thanks, 
--Matt 


On 12/4/17, 2:05 PM, "zeo...@gmail.com" <zeo...@gmail.com> wrote: 

Can we resolve the conversation regarding the second repo? I was waiting 
to get more input/preferences from people There's also a documentation 
update that fixes a few broken Stellar docs that already has aa +1, I just 
need to merge it. 

Jon 

On Mon, Dec 4, 2017, 17:01 Casey Stella <ceste...@gmail.com> wrote: 

> I would be in favor of a release at this point. 
> 
> On Mon, Dec 4, 2017 at 4:57 PM, Matt Foley <ma...@apache.org> wrote: 
> 
> > Hey all, 
> > I see METRON-1252 was resolved over the weekend. Shall I go ahead and 
> > start the process with 0.4.2 release? 
> > Does anyone have any commits they feel strongly should go in before 
0.4.2 
> > is done, or are we ready to call it good? 
> > 
> > I believe there is consensus the 0.4.2 release should include a release 
> of 
> > the current state of the metron-bro-plugin-kafka. I will continue the 
> > discussion in that thread as to the process for accomplishing that, but 
> > plan on it happening. 
> > 
> > Regards, 
> > --Matt 
> > 
> > On 11/26/17, 6:26 PM, "Matt Foley" <ma...@apache.org> wrote: 
> > 
> > Hope everyone (at least in the U.S.) had a great Thanksgiving 
> holiday. 
> > Regarding status of the release effort, still pending METRON-1252, so 
> > not making the release branch yet. 
> > 
> > Regards, 
> > --Matt 
> > 
> > On 11/17/17, 1:32 PM, "Matt Foley" <ma...@apache.org> wrote: 
&g

Re: [DISCUSS] Upcoming Release

2017-12-09 Thread Otto Fowler

So RC2 then?

On December 8, 2017 at 20:43:21, Matt Foley (mfo...@hortonworks.com) wrote:

Hah, here it is: https://github.com/apache/metron/pull/743
“This problem seems to only reproduce when one unrolls a tarball rather
than cloning from github.”

Heh, the exclusion at
https://github.com/apache/metron/blob/master/pom.xml#L351 is still there,
but the hashcode in the bundle.css file name has changed from
a0b6b99c10d9a13dc67e to f56deed131e58bd7ee04. Sigh. Did the version of Font
Awesome fonts change?

On 12/8/17, 5:26 PM, "Matt Foley"  wrote:

I remember having trouble with this bundle.css file on the last release,
but I can’t remember what we did about it. Anybody?

On 12/8/17, 1:41 PM, "Otto Fowler"  wrote:

Steps

- Downloaded tar.gz’s, asc files and KEYS
- Verified signing of both tar.gz’s
- searched for rouge 0.4.1 entries
- verified the main pom.xml
- built :

mvn clean && time mvn -q -T 2C -DskipTests install && time mvn -q -T
2C surefire:test@unit-tests && time mvn -q
surefire:test@integration-tests && time mvn -q test --projects
metron-interface/metron-config && time build_utils/verify_licenses.sh

Found rat error:

*
Summary
---
Generated at: 2017-12-08T16:33:27-05:00

Notes: 3
Binaries: 193
Archives: 0
Standards: 75

Apache Licensed: 74
Generated Documents: 0

JavaDocs are generated, thus a license header is optional.
Generated files do not require license headers.

1 Unknown Licenses

*

Files with unapproved licenses:

/Users/batman/tmp/release_ver/apache-metron-0.4.2-rc1/metron-interface/metron-alerts/dist/styles.f56deed131e58bd7ee04.bundle.css

*

*
Summary
---
Generated at: 2017-12-08T16:33:27-05:00

Notes: 3
Binaries: 193
Archives: 0
Standards: 75

Apache Licensed: 74
Generated Documents: 0

JavaDocs are generated, thus a license header is optional.
Generated files do not require license headers.

1 Unknown Licenses

*

Files with unapproved licenses:

/Users/ottofowler/tmp/release_ver/apache-metron-0.4.2-rc1/metron-interface/metron-alerts/dist/styles.f56deed131e58bd7ee04.bundle.css

*

On December 8, 2017 at 04:34:24, Matt Foley (ma...@apache.org) wrote:

Colleagues,
I’ve posted Metron-0.4.2-RC1 and Metron-bro-plugin-kafka-0.1 to
https://dist.apache.org/repos/dist/dev/metron/0.4.2-RC1/

Given the complexity of this RC, I’d appreciate if a couple people would be
willing to kick the tires before we put it up for a vote.

I will myself be going thru the Verify Build process this weekend, as I
won’t be able to do it Friday.

Thanks,
--Matt

On 12/4/17, 2:05 PM, "zeo...@gmail.com"  wrote:

Can we resolve the conversation regarding the second repo? I was waiting
to get more input/preferences from people There's also a documentation
update that fixes a few broken Stellar docs that already has aa +1, I just
need to merge it.

Jon

On Mon, Dec 4, 2017, 17:01 Casey Stella  wrote:

> I would be in favor of a release at this point.
>
> On Mon, Dec 4, 2017 at 4:57 PM, Matt Foley  wrote:
>
> > Hey all,
> > I see METRON-1252 was resolved over the weekend. Shall I go ahead and
> > start the process with 0.4.2 release?
> > Does anyone have any commits they feel strongly should go in before
0.4.2
> > is done, or are we ready to call it good?
> >
> > I believe there is consensus the 0.4.2 release should include a release
> of
> > the current state of the metron-bro-plugin-kafka. I will continue the
> > discussion in that thread as to the process for accomplishing that, but
> > plan on it happening.
> >
> > Regards,
> > --Matt
> >
> > On 11/26/17, 6:26 PM, "Matt Foley"  wrote:
> >
> > Hope everyone (at least in the U.S.) had a great Thanksgiving
> holiday.
> > Regarding status of the release effort, still pending METRON-1252, so
> > not making the release branch yet.
> >
> > Regards,
> > --Matt
> >
> > On 11/17/17, 1:32 PM, "Matt Foley"  wrote:
> >
> > (With release manager hat on)
> >
> > The community has proposed a release of Metron in the near
> future,
> > focusing on Meta-alerts running in Elasticsearch.
> > Congrats on getting so many of the below already done. At this
> > point, only METRON-1252, and the discussion of how to handle joint
> release
> > of the Metron bro plugin, remain as gating items for the release. I
> > project these will be resolved next week, so let’s propose the
following:
> >
> > Sometime next week, after the last bits are done, I’ll start the
> > release process and create the release branch.
> >
> > The proposed new version will be 0.4.2, unless there are backward
> > incompatible changes that support making it

Re: [DISCUSS] Upcoming Release

2017-12-08 Thread Matt Foley

Hah, here it is: https://github.com/apache/metron/pull/743
“This problem seems to only reproduce when one unrolls a tarball rather than 
cloning from github.”

Heh, the exclusion at https://github.com/apache/metron/blob/master/pom.xml#L351 
is still there, but the hashcode in the bundle.css file name has changed from 
a0b6b99c10d9a13dc67e to f56deed131e58bd7ee04.  Sigh.  Did the version of Font 
Awesome fonts change?

On 12/8/17, 5:26 PM, "Matt Foley"  wrote:

I remember having trouble with this bundle.css file on the last release, 
but I can’t remember what we did about it.  Anybody?

On 12/8/17, 1:41 PM, "Otto Fowler"  wrote:

Steps

   - Downloaded tar.gz’s, asc files and KEYS
   - Verified signing of both tar.gz’s
   - searched for rouge 0.4.1 entries
   - verified the main pom.xml
   - built :

mvn clean && time mvn -q -T 2C -DskipTests install && time mvn -q -T
2C surefire:test@unit-tests && time mvn -q
surefire:test@integration-tests  && time mvn -q test --projects
metron-interface/metron-config && time build_utils/verify_licenses.sh

Found rat error:

*
Summary
---
Generated at: 2017-12-08T16:33:27-05:00

Notes: 3
Binaries: 193
Archives: 0
Standards: 75

Apache Licensed: 74
Generated Documents: 0

JavaDocs are generated, thus a license header is optional.
Generated files do not require license headers.

1 Unknown Licenses

*

Files with unapproved licenses:

/Users/batman/tmp/release_ver/apache-metron-0.4.2-rc1/metron-interface/metron-alerts/dist/styles.f56deed131e58bd7ee04.bundle.css

*

*
Summary
---
Generated at: 2017-12-08T16:33:27-05:00

Notes: 3
Binaries: 193
Archives: 0
Standards: 75

Apache Licensed: 74
Generated Documents: 0

JavaDocs are generated, thus a license header is optional.
Generated files do not require license headers.

1 Unknown Licenses

*

Files with unapproved licenses:

/Users/ottofowler/tmp/release_ver/apache-metron-0.4.2-rc1/metron-interface/metron-alerts/dist/styles.f56deed131e58bd7ee04.bundle.css

*

On December 8, 2017 at 04:34:24, Matt Foley (ma...@apache.org) wrote:

Colleagues,
I’ve posted Metron-0.4.2-RC1 and Metron-bro-plugin-kafka-0.1 to
https://dist.apache.org/repos/dist/dev/metron/0.4.2-RC1/

Given the complexity of this RC, I’d appreciate if a couple people 
would be
willing to kick the tires before we put it up for a vote.

I will myself be going thru the Verify Build process this weekend, as I
won’t be able to do it Friday.

Thanks,
--Matt

On 12/4/17, 2:05 PM, "zeo...@gmail.com"  wrote:

Can we resolve the conversation regarding the second repo? I was waiting
to get more input/preferences from people There's also a documentation
update that fixes a few broken Stellar docs that already has aa +1, I 
just
need to merge it.

Jon

On Mon, Dec 4, 2017, 17:01 Casey Stella  wrote:

> I would be in favor of a release at this point.
>
> On Mon, Dec 4, 2017 at 4:57 PM, Matt Foley  wrote:
>
> > Hey all,
> > I see METRON-1252 was resolved over the weekend. Shall I go ahead 
and
> > start the process with 0.4.2 release?
> > Does anyone have any commits they feel strongly should go in before
0.4.2
> > is done, or are we ready to call it good?
> >
> > I believe there is consensus the 0.4.2 release should include a 
release
> of
> > the current state of the metron-bro-plugin-kafka. I will continue 
the
> > discussion in that thread as to the process for accomplishing that, 
but
> > plan on it happening.
> >
> > Regards,
> > --Matt
> >
> > On 11/26/17, 6:26 PM, "Matt Foley"  wrote:
> >
> > Hope everyone (at least in the U.S.) had a

Re: [DISCUSS] Upcoming Release

2017-12-08 Thread Matt Foley

I remember having trouble with this bundle.css file on the last release, but I 
can’t remember what we did about it.  Anybody?

On 12/8/17, 1:41 PM, "Otto Fowler"  wrote:

Steps

   - Downloaded tar.gz’s, asc files and KEYS
   - Verified signing of both tar.gz’s
   - searched for rouge 0.4.1 entries
   - verified the main pom.xml
   - built :

mvn clean && time mvn -q -T 2C -DskipTests install && time mvn -q -T
2C surefire:test@unit-tests && time mvn -q
surefire:test@integration-tests  && time mvn -q test --projects
metron-interface/metron-config && time build_utils/verify_licenses.sh

Found rat error:


*
Summary
---
Generated at: 2017-12-08T16:33:27-05:00

Notes: 3
Binaries: 193
Archives: 0
Standards: 75

Apache Licensed: 74
Generated Documents: 0

JavaDocs are generated, thus a license header is optional.
Generated files do not require license headers.

1 Unknown Licenses

*

Files with unapproved licenses:

  
/Users/batman/tmp/release_ver/apache-metron-0.4.2-rc1/metron-interface/metron-alerts/dist/styles.f56deed131e58bd7ee04.bundle.css

*





*
Summary
---
Generated at: 2017-12-08T16:33:27-05:00

Notes: 3
Binaries: 193
Archives: 0
Standards: 75

Apache Licensed: 74
Generated Documents: 0

JavaDocs are generated, thus a license header is optional.
Generated files do not require license headers.

1 Unknown Licenses

*

Files with unapproved licenses:



/Users/ottofowler/tmp/release_ver/apache-metron-0.4.2-rc1/metron-interface/metron-alerts/dist/styles.f56deed131e58bd7ee04.bundle.css

*



On December 8, 2017 at 04:34:24, Matt Foley (ma...@apache.org) wrote:

Colleagues,
I’ve posted Metron-0.4.2-RC1 and Metron-bro-plugin-kafka-0.1 to
https://dist.apache.org/repos/dist/dev/metron/0.4.2-RC1/

Given the complexity of this RC, I’d appreciate if a couple people would be
willing to kick the tires before we put it up for a vote.

I will myself be going thru the Verify Build process this weekend, as I
won’t be able to do it Friday.

Thanks,
--Matt


On 12/4/17, 2:05 PM, "zeo...@gmail.com"  wrote:

Can we resolve the conversation regarding the second repo? I was waiting
to get more input/preferences from people There's also a documentation
update that fixes a few broken Stellar docs that already has aa +1, I just
need to merge it.

Jon

On Mon, Dec 4, 2017, 17:01 Casey Stella  wrote:

> I would be in favor of a release at this point.
>
> On Mon, Dec 4, 2017 at 4:57 PM, Matt Foley  wrote:
>
> > Hey all,
> > I see METRON-1252 was resolved over the weekend. Shall I go ahead and
> > start the process with 0.4.2 release?
> > Does anyone have any commits they feel strongly should go in before
0.4.2
> > is done, or are we ready to call it good?
> >
> > I believe there is consensus the 0.4.2 release should include a release
> of
> > the current state of the metron-bro-plugin-kafka. I will continue the
> > discussion in that thread as to the process for accomplishing that, but
> > plan on it happening.
> >
> > Regards,
> > --Matt
> >
> > On 11/26/17, 6:26 PM, "Matt Foley"  wrote:
> >
> > Hope everyone (at least in the U.S.) had a great Thanksgiving
> holiday.
> > Regarding status of the release effort, still pending METRON-1252, so
> > not making the release branch yet.
> >
> > Regards,
> > --Matt
> >
> > On 11/17/17, 1:32 PM, "Matt Foley"  wrote:
> >
> > (With release manager hat on)
> >
> > The community has proposed a release of Metron in the near
> future,
> > focusing on Meta-alerts running in Elasticsearch.
> > Congrats on getting so many of the below already done. At this
> > point, only METRON-1252, and the discussion of how to handle joint
> release
> > of the Metron bro plugin, remain as gating items for the release. I
> > project these will be resolved next week, so let’s propose the
following:
> >
> > Sometime next week, after the last bits are done, I’ll start the
> > release process and create the release branch.
> >
> > The proposed new version will be 0.4.2, unless there are backward

Re: [DISCUSS] Upcoming Release

2017-12-08 Thread Otto Fowler

Steps

   - Downloaded tar.gz’s, asc files and KEYS
   - Verified signing of both tar.gz’s
   - searched for rouge 0.4.1 entries
   - verified the main pom.xml
   - built :

mvn clean && time mvn -q -T 2C -DskipTests install && time mvn -q -T
2C surefire:test@unit-tests && time mvn -q
surefire:test@integration-tests  && time mvn -q test --projects
metron-interface/metron-config && time build_utils/verify_licenses.sh

Found rat error:


*
Summary
---
Generated at: 2017-12-08T16:33:27-05:00

Notes: 3
Binaries: 193
Archives: 0
Standards: 75

Apache Licensed: 74
Generated Documents: 0

JavaDocs are generated, thus a license header is optional.
Generated files do not require license headers.

1 Unknown Licenses

*

Files with unapproved licenses:

  
/Users/batman/tmp/release_ver/apache-metron-0.4.2-rc1/metron-interface/metron-alerts/dist/styles.f56deed131e58bd7ee04.bundle.css

*





*
Summary
---
Generated at: 2017-12-08T16:33:27-05:00

Notes: 3
Binaries: 193
Archives: 0
Standards: 75

Apache Licensed: 74
Generated Documents: 0

JavaDocs are generated, thus a license header is optional.
Generated files do not require license headers.

1 Unknown Licenses

*

Files with unapproved licenses:


/Users/ottofowler/tmp/release_ver/apache-metron-0.4.2-rc1/metron-interface/metron-alerts/dist/styles.f56deed131e58bd7ee04.bundle.css

*



On December 8, 2017 at 04:34:24, Matt Foley (ma...@apache.org) wrote:

Colleagues,
I’ve posted Metron-0.4.2-RC1 and Metron-bro-plugin-kafka-0.1 to
https://dist.apache.org/repos/dist/dev/metron/0.4.2-RC1/

Given the complexity of this RC, I’d appreciate if a couple people would be
willing to kick the tires before we put it up for a vote.

I will myself be going thru the Verify Build process this weekend, as I
won’t be able to do it Friday.

Thanks,
--Matt


On 12/4/17, 2:05 PM, "zeo...@gmail.com"  wrote:

Can we resolve the conversation regarding the second repo? I was waiting
to get more input/preferences from people There's also a documentation
update that fixes a few broken Stellar docs that already has aa +1, I just
need to merge it.

Jon

On Mon, Dec 4, 2017, 17:01 Casey Stella  wrote:

> I would be in favor of a release at this point.
>
> On Mon, Dec 4, 2017 at 4:57 PM, Matt Foley  wrote:
>
> > Hey all,
> > I see METRON-1252 was resolved over the weekend. Shall I go ahead and
> > start the process with 0.4.2 release?
> > Does anyone have any commits they feel strongly should go in before
0.4.2
> > is done, or are we ready to call it good?
> >
> > I believe there is consensus the 0.4.2 release should include a release
> of
> > the current state of the metron-bro-plugin-kafka. I will continue the
> > discussion in that thread as to the process for accomplishing that, but
> > plan on it happening.
> >
> > Regards,
> > --Matt
> >
> > On 11/26/17, 6:26 PM, "Matt Foley"  wrote:
> >
> > Hope everyone (at least in the U.S.) had a great Thanksgiving
> holiday.
> > Regarding status of the release effort, still pending METRON-1252, so
> > not making the release branch yet.
> >
> > Regards,
> > --Matt
> >
> > On 11/17/17, 1:32 PM, "Matt Foley"  wrote:
> >
> > (With release manager hat on)
> >
> > The community has proposed a release of Metron in the near
> future,
> > focusing on Meta-alerts running in Elasticsearch.
> > Congrats on getting so many of the below already done. At this
> > point, only METRON-1252, and the discussion of how to handle joint
> release
> > of the Metron bro plugin, remain as gating items for the release. I
> > project these will be resolved next week, so let’s propose the
following:
> >
> > Sometime next week, after the last bits are done, I’ll start the
> > release process and create the release branch.
> >
> > The proposed new version will be 0.4.2, unless there are backward
> > incompatible changes that support making it 0.5.0.
> > Currently there are NO included Jiras labeled
> > ‘backward-incompatible’, nor having Docs Text indicating so.
> > ***If anyone knows that some of the commits included since 0.4.1
> > introduce backward incompatibility, please say so now on this thread,
and
> > mark the Jira as such.***
> >
> > The 90 or so jiras/commits already in master branch since 0.4.1
> > are listed below.
> > Thanks,
> > --Matt
> >
> > METRON-1301 Alerts UI - Sorting on Triage Score Unexpectedly
> > Filters Some Records (nickwallen) closes apache/metron#832
> > METRON-1294 IP addresses are not formatted correctly in facet
> > and group results (merrimanr) closes apache/metron#827
> > METRON-1291 Kafka produce REST endpoint does not work in a
> > Kerberized

Re: [DISCUSS] Upcoming Release

2017-12-08 Thread zeo...@gmail.com

I poked around the RC briefly and spun up full dev with and without
sensors, no issues so far.

Jon

On Fri, Dec 8, 2017 at 4:34 AM Matt Foley  wrote:

> Colleagues,
> I’ve posted Metron-0.4.2-RC1 and Metron-bro-plugin-kafka-0.1 to
> https://dist.apache.org/repos/dist/dev/metron/0.4.2-RC1/
>
> Given the complexity of this RC, I’d appreciate if a couple people would
> be willing to kick the tires before we put it up for a vote.
>
> I will myself be going thru the Verify Build process this weekend, as I
> won’t be able to do it Friday.
>
> Thanks,
> --Matt
>
>
> On 12/4/17, 2:05 PM, "zeo...@gmail.com"  wrote:
>
> Can we resolve the conversation regarding the second repo?  I was
> waiting
> to get more input/preferences from people  There's also a documentation
> update that fixes a few broken Stellar docs that already has aa +1, I
> just
> need to merge it.
>
> Jon
>
> On Mon, Dec 4, 2017, 17:01 Casey Stella  wrote:
>
> > I would be in favor of a release at this point.
> >
> > On Mon, Dec 4, 2017 at 4:57 PM, Matt Foley  wrote:
> >
> > > Hey all,
> > > I see METRON-1252 was resolved over the weekend.  Shall I go ahead
> and
> > > start the process with 0.4.2 release?
> > > Does anyone have any commits they feel strongly should go in
> before 0.4.2
> > > is done, or are we ready to call it good?
> > >
> > > I believe there is consensus the 0.4.2 release should include a
> release
> > of
> > > the current state of the metron-bro-plugin-kafka.  I will continue
> the
> > > discussion in that thread as to the process for accomplishing
> that, but
> > > plan on it happening.
> > >
> > > Regards,
> > > --Matt
> > >
> > > On 11/26/17, 6:26 PM, "Matt Foley"  wrote:
> > >
> > > Hope everyone (at least in the U.S.) had a great Thanksgiving
> > holiday.
> > > Regarding status of the release effort, still pending
> METRON-1252, so
> > > not making the release branch yet.
> > >
> > > Regards,
> > > --Matt
> > >
> > > On 11/17/17, 1:32 PM, "Matt Foley"  wrote:
> > >
> > > (With release manager hat on)
> > >
> > > The community has proposed a release of Metron in the near
> > future,
> > > focusing on Meta-alerts running in Elasticsearch.
> > > Congrats on getting so many of the below already done.  At
> this
> > > point, only METRON-1252, and the discussion of how to handle joint
> > release
> > > of the Metron bro plugin, remain as gating items for the release.
> I
> > > project these will be resolved next week, so let’s propose the
> following:
> > >
> > > Sometime next week, after the last bits are done, I’ll
> start the
> > > release process and create the release branch.
> > >
> > > The proposed new version will be 0.4.2, unless there are
> backward
> > > incompatible changes that support making it 0.5.0.
> > > Currently there are NO included Jiras labeled
> > > ‘backward-incompatible’, nor having Docs Text indicating so.
> > > ***If anyone knows that some of the commits included since
> 0.4.1
> > > introduce backward incompatibility, please say so now on this
> thread, and
> > > mark the Jira as such.***
> > >
> > > The 90 or so jiras/commits already in master branch since
> 0.4.1
> > > are listed below.
> > > Thanks,
> > > --Matt
> > >
> > > METRON-1301 Alerts UI - Sorting on Triage Score
> Unexpectedly
> > > Filters Some Records (nickwallen) closes apache/metron#832
> > > METRON-1294 IP addresses are not formatted correctly
> in facet
> > > and group results (merrimanr) closes apache/metron#827
> > > METRON-1291 Kafka produce REST endpoint does not work
> in a
> > > Kerberized cluster (merrimanr) closes apache/metron#826
> > > METRON-1290 Only first 10 alerts are update when a
> MetaAlert
> > > status is changed to inactive (justinleet) closes apache/metron#842
> > > METRON-1311 Service Check Should Check Elasticsearch
> Index
> > > Templates (nickwallen) closes apache/metron#839
> > > METRON-1289 Alert fields are lost when a MetaAlert is
> created
> > > (merrimanr) closes apache/metron#824
> > > METRON-1309 Change metron-deployment to pull the
> plugin from
> > > apache/metron-bro-plugin-kafka (JonZeolla) closes apache/metron#837
> > > METRON-1310 Template Delete Action Deletes Search
> Indices
> > > (nickwallen) closes apache/metron#838
> > > METRON-1275: Fix Metron Documentation closes
> > > apache/incubator-metron#833
> > > METRON-1295 Unable to Configure

Re: [DISCUSS] Upcoming Release

2017-12-08 Thread Matt Foley

Colleagues,
I’ve posted Metron-0.4.2-RC1 and Metron-bro-plugin-kafka-0.1 to 
https://dist.apache.org/repos/dist/dev/metron/0.4.2-RC1/

Given the complexity of this RC, I’d appreciate if a couple people would be 
willing to kick the tires before we put it up for a vote.

I will myself be going thru the Verify Build process this weekend, as I won’t 
be able to do it Friday.

Thanks,
--Matt


On 12/4/17, 2:05 PM, "zeo...@gmail.com"  wrote:

Can we resolve the conversation regarding the second repo?  I was waiting
to get more input/preferences from people  There's also a documentation
update that fixes a few broken Stellar docs that already has aa +1, I just
need to merge it.

Jon

On Mon, Dec 4, 2017, 17:01 Casey Stella  wrote:

> I would be in favor of a release at this point.
>
> On Mon, Dec 4, 2017 at 4:57 PM, Matt Foley  wrote:
>
> > Hey all,
> > I see METRON-1252 was resolved over the weekend.  Shall I go ahead and
> > start the process with 0.4.2 release?
> > Does anyone have any commits they feel strongly should go in before 
0.4.2
> > is done, or are we ready to call it good?
> >
> > I believe there is consensus the 0.4.2 release should include a release
> of
> > the current state of the metron-bro-plugin-kafka.  I will continue the
> > discussion in that thread as to the process for accomplishing that, but
> > plan on it happening.
> >
> > Regards,
> > --Matt
> >
> > On 11/26/17, 6:26 PM, "Matt Foley"  wrote:
> >
> > Hope everyone (at least in the U.S.) had a great Thanksgiving
> holiday.
> > Regarding status of the release effort, still pending METRON-1252, 
so
> > not making the release branch yet.
> >
> > Regards,
> > --Matt
> >
> > On 11/17/17, 1:32 PM, "Matt Foley"  wrote:
> >
> > (With release manager hat on)
> >
> > The community has proposed a release of Metron in the near
> future,
> > focusing on Meta-alerts running in Elasticsearch.
> > Congrats on getting so many of the below already done.  At this
> > point, only METRON-1252, and the discussion of how to handle joint
> release
> > of the Metron bro plugin, remain as gating items for the release.  I
> > project these will be resolved next week, so let’s propose the 
following:
> >
> > Sometime next week, after the last bits are done, I’ll start the
> > release process and create the release branch.
> >
> > The proposed new version will be 0.4.2, unless there are 
backward
> > incompatible changes that support making it 0.5.0.
> > Currently there are NO included Jiras labeled
> > ‘backward-incompatible’, nor having Docs Text indicating so.
> > ***If anyone knows that some of the commits included since 0.4.1
> > introduce backward incompatibility, please say so now on this thread, 
and
> > mark the Jira as such.***
> >
> > The 90 or so jiras/commits already in master branch since 0.4.1
> > are listed below.
> > Thanks,
> > --Matt
> >
> > METRON-1301 Alerts UI - Sorting on Triage Score Unexpectedly
> > Filters Some Records (nickwallen) closes apache/metron#832
> > METRON-1294 IP addresses are not formatted correctly in 
facet
> > and group results (merrimanr) closes apache/metron#827
> > METRON-1291 Kafka produce REST endpoint does not work in a
> > Kerberized cluster (merrimanr) closes apache/metron#826
> > METRON-1290 Only first 10 alerts are update when a MetaAlert
> > status is changed to inactive (justinleet) closes apache/metron#842
> > METRON-1311 Service Check Should Check Elasticsearch Index
> > Templates (nickwallen) closes apache/metron#839
> > METRON-1289 Alert fields are lost when a MetaAlert is 
created
> > (merrimanr) closes apache/metron#824
> > METRON-1309 Change metron-deployment to pull the plugin from
> > apache/metron-bro-plugin-kafka (JonZeolla) closes apache/metron#837
> > METRON-1310 Template Delete Action Deletes Search Indices
> > (nickwallen) closes apache/metron#838
> > METRON-1275: Fix Metron Documentation closes
> > apache/incubator-metron#833
> > METRON-1295 Unable to Configure Logging for REST API
> > (nickwallen) closes apache/metron#828
> > METRON-1307 Force install of java8 since java9 does not
> appear
> > to work with the scripts (brianhurley via ottobackwards) closes
> > apache/metron#835
> > METRON-1296 Full Dev Fails to Deploy Index Templates
> > (nickwallen via cestella) closes apache/incubator-metron#829

Re: [DISCUSS] Upcoming Release

2017-12-04 Thread zeo...@gmail.com

Can we resolve the conversation regarding the second repo?  I was waiting
to get more input/preferences from people  There's also a documentation
update that fixes a few broken Stellar docs that already has aa +1, I just
need to merge it.

Jon

On Mon, Dec 4, 2017, 17:01 Casey Stella  wrote:

> I would be in favor of a release at this point.
>
> On Mon, Dec 4, 2017 at 4:57 PM, Matt Foley  wrote:
>
> > Hey all,
> > I see METRON-1252 was resolved over the weekend.  Shall I go ahead and
> > start the process with 0.4.2 release?
> > Does anyone have any commits they feel strongly should go in before 0.4.2
> > is done, or are we ready to call it good?
> >
> > I believe there is consensus the 0.4.2 release should include a release
> of
> > the current state of the metron-bro-plugin-kafka.  I will continue the
> > discussion in that thread as to the process for accomplishing that, but
> > plan on it happening.
> >
> > Regards,
> > --Matt
> >
> > On 11/26/17, 6:26 PM, "Matt Foley"  wrote:
> >
> > Hope everyone (at least in the U.S.) had a great Thanksgiving
> holiday.
> > Regarding status of the release effort, still pending METRON-1252, so
> > not making the release branch yet.
> >
> > Regards,
> > --Matt
> >
> > On 11/17/17, 1:32 PM, "Matt Foley"  wrote:
> >
> > (With release manager hat on)
> >
> > The community has proposed a release of Metron in the near
> future,
> > focusing on Meta-alerts running in Elasticsearch.
> > Congrats on getting so many of the below already done.  At this
> > point, only METRON-1252, and the discussion of how to handle joint
> release
> > of the Metron bro plugin, remain as gating items for the release.  I
> > project these will be resolved next week, so let’s propose the following:
> >
> > Sometime next week, after the last bits are done, I’ll start the
> > release process and create the release branch.
> >
> > The proposed new version will be 0.4.2, unless there are backward
> > incompatible changes that support making it 0.5.0.
> > Currently there are NO included Jiras labeled
> > ‘backward-incompatible’, nor having Docs Text indicating so.
> > ***If anyone knows that some of the commits included since 0.4.1
> > introduce backward incompatibility, please say so now on this thread, and
> > mark the Jira as such.***
> >
> > The 90 or so jiras/commits already in master branch since 0.4.1
> > are listed below.
> > Thanks,
> > --Matt
> >
> > METRON-1301 Alerts UI - Sorting on Triage Score Unexpectedly
> > Filters Some Records (nickwallen) closes apache/metron#832
> > METRON-1294 IP addresses are not formatted correctly in facet
> > and group results (merrimanr) closes apache/metron#827
> > METRON-1291 Kafka produce REST endpoint does not work in a
> > Kerberized cluster (merrimanr) closes apache/metron#826
> > METRON-1290 Only first 10 alerts are update when a MetaAlert
> > status is changed to inactive (justinleet) closes apache/metron#842
> > METRON-1311 Service Check Should Check Elasticsearch Index
> > Templates (nickwallen) closes apache/metron#839
> > METRON-1289 Alert fields are lost when a MetaAlert is created
> > (merrimanr) closes apache/metron#824
> > METRON-1309 Change metron-deployment to pull the plugin from
> > apache/metron-bro-plugin-kafka (JonZeolla) closes apache/metron#837
> > METRON-1310 Template Delete Action Deletes Search Indices
> > (nickwallen) closes apache/metron#838
> > METRON-1275: Fix Metron Documentation closes
> > apache/incubator-metron#833
> > METRON-1295 Unable to Configure Logging for REST API
> > (nickwallen) closes apache/metron#828
> > METRON-1307 Force install of java8 since java9 does not
> appear
> > to work with the scripts (brianhurley via ottobackwards) closes
> > apache/metron#835
> > METRON-1296 Full Dev Fails to Deploy Index Templates
> > (nickwallen via cestella) closes apache/incubator-metron#829
> > METRON-1281 Remove hard-coded indices from the Alerts UI
> > (merrimanr) closes apache/metron#821
> > METRON-1287 Full Dev Fails When Installing EPEL Repository
> > (nickwallen) closes apache/metron#820
> > METRON-1267 Alerts UI returns a 404 when refreshing the
> > alerts-list page (iraghumitra via merrimanr) closes apache/metron#819
> > METRON-1283 Install Elasticsearch template as a part of the
> > mpack startup scripts (anandsubbu via nickwallen) closes
> apache/metron#817
> > METRON-1254: Conditionals as map keys do not function in
> > Stellar closes apache/incubator-metron#801
> > METRON-1261 Apply bro security patch (JonZeolla via
> > ottobackwards) closes apache/metron#805
> > METRON-1284 Remove extraneous dead query in

Re: [DISCUSS] Upcoming Release

2017-12-04 Thread Casey Stella

I would be in favor of a release at this point.

On Mon, Dec 4, 2017 at 4:57 PM, Matt Foley  wrote:

> Hey all,
> I see METRON-1252 was resolved over the weekend.  Shall I go ahead and
> start the process with 0.4.2 release?
> Does anyone have any commits they feel strongly should go in before 0.4.2
> is done, or are we ready to call it good?
>
> I believe there is consensus the 0.4.2 release should include a release of
> the current state of the metron-bro-plugin-kafka.  I will continue the
> discussion in that thread as to the process for accomplishing that, but
> plan on it happening.
>
> Regards,
> --Matt
>
> On 11/26/17, 6:26 PM, "Matt Foley"  wrote:
>
> Hope everyone (at least in the U.S.) had a great Thanksgiving holiday.
> Regarding status of the release effort, still pending METRON-1252, so
> not making the release branch yet.
>
> Regards,
> --Matt
>
> On 11/17/17, 1:32 PM, "Matt Foley"  wrote:
>
> (With release manager hat on)
>
> The community has proposed a release of Metron in the near future,
> focusing on Meta-alerts running in Elasticsearch.
> Congrats on getting so many of the below already done.  At this
> point, only METRON-1252, and the discussion of how to handle joint release
> of the Metron bro plugin, remain as gating items for the release.  I
> project these will be resolved next week, so let’s propose the following:
>
> Sometime next week, after the last bits are done, I’ll start the
> release process and create the release branch.
>
> The proposed new version will be 0.4.2, unless there are backward
> incompatible changes that support making it 0.5.0.
> Currently there are NO included Jiras labeled
> ‘backward-incompatible’, nor having Docs Text indicating so.
> ***If anyone knows that some of the commits included since 0.4.1
> introduce backward incompatibility, please say so now on this thread, and
> mark the Jira as such.***
>
> The 90 or so jiras/commits already in master branch since 0.4.1
> are listed below.
> Thanks,
> --Matt
>
> METRON-1301 Alerts UI - Sorting on Triage Score Unexpectedly
> Filters Some Records (nickwallen) closes apache/metron#832
> METRON-1294 IP addresses are not formatted correctly in facet
> and group results (merrimanr) closes apache/metron#827
> METRON-1291 Kafka produce REST endpoint does not work in a
> Kerberized cluster (merrimanr) closes apache/metron#826
> METRON-1290 Only first 10 alerts are update when a MetaAlert
> status is changed to inactive (justinleet) closes apache/metron#842
> METRON-1311 Service Check Should Check Elasticsearch Index
> Templates (nickwallen) closes apache/metron#839
> METRON-1289 Alert fields are lost when a MetaAlert is created
> (merrimanr) closes apache/metron#824
> METRON-1309 Change metron-deployment to pull the plugin from
> apache/metron-bro-plugin-kafka (JonZeolla) closes apache/metron#837
> METRON-1310 Template Delete Action Deletes Search Indices
> (nickwallen) closes apache/metron#838
> METRON-1275: Fix Metron Documentation closes
> apache/incubator-metron#833
> METRON-1295 Unable to Configure Logging for REST API
> (nickwallen) closes apache/metron#828
> METRON-1307 Force install of java8 since java9 does not appear
> to work with the scripts (brianhurley via ottobackwards) closes
> apache/metron#835
> METRON-1296 Full Dev Fails to Deploy Index Templates
> (nickwallen via cestella) closes apache/incubator-metron#829
> METRON-1281 Remove hard-coded indices from the Alerts UI
> (merrimanr) closes apache/metron#821
> METRON-1287 Full Dev Fails When Installing EPEL Repository
> (nickwallen) closes apache/metron#820
> METRON-1267 Alerts UI returns a 404 when refreshing the
> alerts-list page (iraghumitra via merrimanr) closes apache/metron#819
> METRON-1283 Install Elasticsearch template as a part of the
> mpack startup scripts (anandsubbu via nickwallen) closes apache/metron#817
> METRON-1254: Conditionals as map keys do not function in
> Stellar closes apache/incubator-metron#801
> METRON-1261 Apply bro security patch (JonZeolla via
> ottobackwards) closes apache/metron#805
> METRON-1284 Remove extraneous dead query in ElasticsearchDao
> (justinleet) closes apache/metron#818
> METRON-1270: fix for warnings missing @return tag argument in
> metron-analytics/metron-profiler-common and metron-profiler-client closes
> apache/incubator-metron#810
> METRON-1272 Hide child alerts from searches and grouping if
> they belong to meta alerts (justinleet) closes apache/metron#811
> METRON-1224 Add time range selection to search control
> (iraghumitra via james-sirota) closes apache/metron#796
>

Re: [DISCUSS] Upcoming Release

2017-12-04 Thread Matt Foley

Hey all, 
I see METRON-1252 was resolved over the weekend.  Shall I go ahead and start 
the process with 0.4.2 release?
Does anyone have any commits they feel strongly should go in before 0.4.2 is 
done, or are we ready to call it good?

I believe there is consensus the 0.4.2 release should include a release of the 
current state of the metron-bro-plugin-kafka.  I will continue the discussion 
in that thread as to the process for accomplishing that, but plan on it 
happening.

Regards,
--Matt

On 11/26/17, 6:26 PM, "Matt Foley"  wrote:

Hope everyone (at least in the U.S.) had a great Thanksgiving holiday.
Regarding status of the release effort, still pending METRON-1252, so not 
making the release branch yet.

Regards,
--Matt

On 11/17/17, 1:32 PM, "Matt Foley"  wrote:

(With release manager hat on)

The community has proposed a release of Metron in the near future, 
focusing on Meta-alerts running in Elasticsearch.
Congrats on getting so many of the below already done.  At this point, 
only METRON-1252, and the discussion of how to handle joint release of the 
Metron bro plugin, remain as gating items for the release.  I project these 
will be resolved next week, so let’s propose the following:

Sometime next week, after the last bits are done, I’ll start the 
release process and create the release branch.

The proposed new version will be 0.4.2, unless there are backward 
incompatible changes that support making it 0.5.0.
Currently there are NO included Jiras labeled ‘backward-incompatible’, 
nor having Docs Text indicating so.
***If anyone knows that some of the commits included since 0.4.1 
introduce backward incompatibility, please say so now on this thread, and mark 
the Jira as such.***

The 90 or so jiras/commits already in master branch since 0.4.1 are 
listed below.
Thanks,
--Matt

METRON-1301 Alerts UI - Sorting on Triage Score Unexpectedly 
Filters Some Records (nickwallen) closes apache/metron#832
METRON-1294 IP addresses are not formatted correctly in facet and 
group results (merrimanr) closes apache/metron#827
METRON-1291 Kafka produce REST endpoint does not work in a 
Kerberized cluster (merrimanr) closes apache/metron#826
METRON-1290 Only first 10 alerts are update when a MetaAlert status 
is changed to inactive (justinleet) closes apache/metron#842
METRON-1311 Service Check Should Check Elasticsearch Index 
Templates (nickwallen) closes apache/metron#839
METRON-1289 Alert fields are lost when a MetaAlert is created 
(merrimanr) closes apache/metron#824
METRON-1309 Change metron-deployment to pull the plugin from 
apache/metron-bro-plugin-kafka (JonZeolla) closes apache/metron#837
METRON-1310 Template Delete Action Deletes Search Indices 
(nickwallen) closes apache/metron#838
METRON-1275: Fix Metron Documentation closes 
apache/incubator-metron#833
METRON-1295 Unable to Configure Logging for REST API (nickwallen) 
closes apache/metron#828
METRON-1307 Force install of java8 since java9 does not appear to 
work with the scripts (brianhurley via ottobackwards) closes apache/metron#835
METRON-1296 Full Dev Fails to Deploy Index Templates (nickwallen 
via cestella) closes apache/incubator-metron#829
METRON-1281 Remove hard-coded indices from the Alerts UI 
(merrimanr) closes apache/metron#821
METRON-1287 Full Dev Fails When Installing EPEL Repository 
(nickwallen) closes apache/metron#820
METRON-1267 Alerts UI returns a 404 when refreshing the alerts-list 
page (iraghumitra via merrimanr) closes apache/metron#819
METRON-1283 Install Elasticsearch template as a part of the mpack 
startup scripts (anandsubbu via nickwallen) closes apache/metron#817
METRON-1254: Conditionals as map keys do not function in Stellar 
closes apache/incubator-metron#801
METRON-1261 Apply bro security patch (JonZeolla via ottobackwards) 
closes apache/metron#805
METRON-1284 Remove extraneous dead query in ElasticsearchDao 
(justinleet) closes apache/metron#818
METRON-1270: fix for warnings missing @return tag argument in 
metron-analytics/metron-profiler-common and metron-profiler-client closes 
apache/incubator-metron#810
METRON-1272 Hide child alerts from searches and grouping if they 
belong to meta alerts (justinleet) closes apache/metron#811
METRON-1224 Add time range selection to search control (iraghumitra 
via james-sirota) closes apache/metron#796
METRON-1280 0.4.1 -> 0.4.2 missed a couple of projects (cestella 
via justinleet) closes apache/metron#816
METRON-1243: Add a REST endpoint which allows us to get a list of 
all indice closes

Re: [DISCUSS] Upcoming Release

2017-11-27 Thread Otto Fowler

Considering the problems we are having with people building the node stuff
on centos,  would it make sense to wait to take
a potential PR that allows full metron builds in our ansible docker image?



On November 26, 2017 at 21:26:27, Matt Foley (ma...@apache.org) wrote:

Hope everyone (at least in the U.S.) had a great Thanksgiving holiday.
Regarding status of the release effort, still pending METRON-1252, so not
making the release branch yet.

Regards,
--Matt

On 11/17/17, 1:32 PM, "Matt Foley"  wrote:

(With release manager hat on)

The community has proposed a release of Metron in the near future, focusing
on Meta-alerts running in Elasticsearch.
Congrats on getting so many of the below already done. At this point, only
METRON-1252, and the discussion of how to handle joint release of the
Metron bro plugin, remain as gating items for the release. I project these
will be resolved next week, so let’s propose the following:

Sometime next week, after the last bits are done, I’ll start the release
process and create the release branch.

The proposed new version will be 0.4.2, unless there are backward
incompatible changes that support making it 0.5.0.
Currently there are NO included Jiras labeled ‘backward-incompatible’, nor
having Docs Text indicating so.
***If anyone knows that some of the commits included since 0.4.1 introduce
backward incompatibility, please say so now on this thread, and mark the
Jira as such.***

The 90 or so jiras/commits already in master branch since 0.4.1 are listed
below.
Thanks,
--Matt

METRON-1301 Alerts UI - Sorting on Triage Score Unexpectedly Filters Some
Records (nickwallen) closes apache/metron#832
METRON-1294 IP addresses are not formatted correctly in facet and group
results (merrimanr) closes apache/metron#827
METRON-1291 Kafka produce REST endpoint does not work in a Kerberized
cluster (merrimanr) closes apache/metron#826
METRON-1290 Only first 10 alerts are update when a MetaAlert status is
changed to inactive (justinleet) closes apache/metron#842
METRON-1311 Service Check Should Check Elasticsearch Index Templates
(nickwallen) closes apache/metron#839
METRON-1289 Alert fields are lost when a MetaAlert is created (merrimanr)
closes apache/metron#824
METRON-1309 Change metron-deployment to pull the plugin from
apache/metron-bro-plugin-kafka (JonZeolla) closes apache/metron#837
METRON-1310 Template Delete Action Deletes Search Indices (nickwallen)
closes apache/metron#838
METRON-1275: Fix Metron Documentation closes apache/incubator-metron#833
METRON-1295 Unable to Configure Logging for REST API (nickwallen) closes
apache/metron#828
METRON-1307 Force install of java8 since java9 does not appear to work with
the scripts (brianhurley via ottobackwards) closes apache/metron#835
METRON-1296 Full Dev Fails to Deploy Index Templates (nickwallen via
cestella) closes apache/incubator-metron#829
METRON-1281 Remove hard-coded indices from the Alerts UI (merrimanr) closes
apache/metron#821
METRON-1287 Full Dev Fails When Installing EPEL Repository (nickwallen)
closes apache/metron#820
METRON-1267 Alerts UI returns a 404 when refreshing the alerts-list page
(iraghumitra via merrimanr) closes apache/metron#819
METRON-1283 Install Elasticsearch template as a part of the mpack startup
scripts (anandsubbu via nickwallen) closes apache/metron#817
METRON-1254: Conditionals as map keys do not function in Stellar closes
apache/incubator-metron#801
METRON-1261 Apply bro security patch (JonZeolla via ottobackwards) closes
apache/metron#805
METRON-1284 Remove extraneous dead query in ElasticsearchDao (justinleet)
closes apache/metron#818
METRON-1270: fix for warnings missing @return tag argument in
metron-analytics/metron-profiler-common and metron-profiler-client closes
apache/incubator-metron#810
METRON-1272 Hide child alerts from searches and grouping if they belong to
meta alerts (justinleet) closes apache/metron#811
METRON-1224 Add time range selection to search control (iraghumitra via
james-sirota) closes apache/metron#796
METRON-1280 0.4.1 -> 0.4.2 missed a couple of projects (cestella via
justinleet) closes apache/metron#816
METRON-1243: Add a REST endpoint which allows us to get a list of all
indice closes apache/incubator-metron#797
METRON-1196 Increment master version number to 0.4.2 for on-going
development (mattf-horton) closes apache/metron#767
METRON-1278 Strip Build Status widget from root README.md in
site-book build (mattf-horton) closes apache/metron#815
METRON-1274 Master has failure in StormControllerIntegrationTest
(merrimanr) closes apache/metron#813
METRON-1266 Profiler - SASL Authentication Failed (nickwallen) closes
apache/metron#809
METRON-1260 Include Alerts UI in Ambari Service Check (nickwallen) closes
apache/metron#804
METRON-1251: Typo and formatting fixes for metron-rest README closes
apache/incubator-metron#800
METRON-1241: Enable the REST API to use a cache for the zookeeper config
similar to the Bolts closes apache/incubator-metron#795

Re: [DISCUSS] Upcoming Release

2017-11-26 Thread Matt Foley

Hope everyone (at least in the U.S.) had a great Thanksgiving holiday.
Regarding status of the release effort, still pending METRON-1252, so not 
making the release branch yet.

Regards,
--Matt

On 11/17/17, 1:32 PM, "Matt Foley"  wrote:

(With release manager hat on)

The community has proposed a release of Metron in the near future, focusing 
on Meta-alerts running in Elasticsearch.
Congrats on getting so many of the below already done.  At this point, only 
METRON-1252, and the discussion of how to handle joint release of the Metron 
bro plugin, remain as gating items for the release.  I project these will be 
resolved next week, so let’s propose the following:

Sometime next week, after the last bits are done, I’ll start the release 
process and create the release branch.

The proposed new version will be 0.4.2, unless there are backward 
incompatible changes that support making it 0.5.0.
Currently there are NO included Jiras labeled ‘backward-incompatible’, nor 
having Docs Text indicating so.
***If anyone knows that some of the commits included since 0.4.1 introduce 
backward incompatibility, please say so now on this thread, and mark the Jira 
as such.***

The 90 or so jiras/commits already in master branch since 0.4.1 are listed 
below.
Thanks,
--Matt

METRON-1301 Alerts UI - Sorting on Triage Score Unexpectedly Filters 
Some Records (nickwallen) closes apache/metron#832
METRON-1294 IP addresses are not formatted correctly in facet and group 
results (merrimanr) closes apache/metron#827
METRON-1291 Kafka produce REST endpoint does not work in a Kerberized 
cluster (merrimanr) closes apache/metron#826
METRON-1290 Only first 10 alerts are update when a MetaAlert status is 
changed to inactive (justinleet) closes apache/metron#842
METRON-1311 Service Check Should Check Elasticsearch Index Templates 
(nickwallen) closes apache/metron#839
METRON-1289 Alert fields are lost when a MetaAlert is created 
(merrimanr) closes apache/metron#824
METRON-1309 Change metron-deployment to pull the plugin from 
apache/metron-bro-plugin-kafka (JonZeolla) closes apache/metron#837
METRON-1310 Template Delete Action Deletes Search Indices (nickwallen) 
closes apache/metron#838
METRON-1275: Fix Metron Documentation closes apache/incubator-metron#833
METRON-1295 Unable to Configure Logging for REST API (nickwallen) 
closes apache/metron#828
METRON-1307 Force install of java8 since java9 does not appear to work 
with the scripts (brianhurley via ottobackwards) closes apache/metron#835
METRON-1296 Full Dev Fails to Deploy Index Templates (nickwallen via 
cestella) closes apache/incubator-metron#829
METRON-1281 Remove hard-coded indices from the Alerts UI (merrimanr) 
closes apache/metron#821
METRON-1287 Full Dev Fails When Installing EPEL Repository (nickwallen) 
closes apache/metron#820
METRON-1267 Alerts UI returns a 404 when refreshing the alerts-list 
page (iraghumitra via merrimanr) closes apache/metron#819
METRON-1283 Install Elasticsearch template as a part of the mpack 
startup scripts (anandsubbu via nickwallen) closes apache/metron#817
METRON-1254: Conditionals as map keys do not function in Stellar closes 
apache/incubator-metron#801
METRON-1261 Apply bro security patch (JonZeolla via ottobackwards) 
closes apache/metron#805
METRON-1284 Remove extraneous dead query in ElasticsearchDao 
(justinleet) closes apache/metron#818
METRON-1270: fix for warnings missing @return tag argument in 
metron-analytics/metron-profiler-common and metron-profiler-client closes 
apache/incubator-metron#810
METRON-1272 Hide child alerts from searches and grouping if they belong 
to meta alerts (justinleet) closes apache/metron#811
METRON-1224 Add time range selection to search control (iraghumitra via 
james-sirota) closes apache/metron#796
METRON-1280 0.4.1 -> 0.4.2 missed a couple of projects (cestella via 
justinleet) closes apache/metron#816
METRON-1243: Add a REST endpoint which allows us to get a list of all 
indice closes apache/incubator-metron#797
METRON-1196 Increment master version number to 0.4.2 for on-going 
development (mattf-horton) closes apache/metron#767
METRON-1278 Strip Build Status widget from root README.md 
in site-book build (mattf-horton) closes apache/metron#815
METRON-1274 Master has failure in StormControllerIntegrationTest 
(merrimanr) closes apache/metron#813
METRON-1266 Profiler - SASL Authentication Failed (nickwallen) closes 
apache/metron#809
METRON-1260 Include Alerts UI in Ambari Service Check (nickwallen) 
closes apache/metron#804
METRON-1251: Typo and formatting fixes for metron-rest README closes 
apache/incubator-metron#800
METRON-1241: Enable the REST API to use a cache for

Re: [DISCUSS] Upcoming Release

2017-11-18 Thread zeo...@gmail.com

Should we consider drafting some bare metal install instructions for 0.4.2
that can be a part of the release?  Similar to
https://github.com/apache/metron/tree/master/metron-deployment/other-examples/manual-install

Or do we want to continue to have those instructions outside of the
releases themselves (wiki, etc.)?  Just a thought, maybe something to do
next release and not this time.

I have a documentation PR that fixes some stellar documentation issues that
I would like to get in (broken links, nonexistent functions, reordering,
etc.), and I am about done with the bro 2.5.2 upgrade in full dev (PR soon,
probably Monday).  If we can get that in we will have some cleaner
documentation and a nice clean bro plugin scenario without needing to do a
one off.


Jon

On Fri, Nov 17, 2017, 21:28 Ryan Merriman  wrote:

> Makes sense now.  Thanks Matt.
>
> > On Nov 17, 2017, at 4:25 PM, Matt Foley  wrote:
> >
> > Hi Ryan,
> > Yes and no.  The last release (see
> https://dist.apache.org/repos/dist/release/metron/ ) was 0.4.1, announced
> on 9/19.
> > Immediately after that we bumped the  of builds from master
> branch, per https://issues.apache.org/jira/browse/METRON-1196 .  This is
> consistent with the Release Process “Clean up” phase: “It is good practice
> to increment the build version in master immediately after a Feature
> Release, so that dev builds with new stuff from master cannot be mistaken
> for builds of the release version. So, immediately after a release,
> increment the MINOR version number (eg, with the 0.4.0 just released, set
> the new version number to 0.4.1)” (
> https://cwiki.apache.org/confluence/display/METRON/Release+Process#ReleaseProcess-Step15-Cleanup
> )
> >
> > So you’re correct that the version of development builds is currently
> set to 0.4.2.  After we actually make a release of 0.4.2, we’ll change dev
> build  to 0.4.3 (regardless of whether we expect the following
> release to be 0.4.3 or 0.5.0).
> >
> > Hope this clarifies,
> > --Matt
> >
> > On 11/17/17, 1:59 PM, "Ryan Merriman"  wrote:
> >
> >Matt,
> >
> >I think we are currently on version 0.4.2.  If that is the case would
> the
> >next version be 0.4.3?
> >
> >Ryan
> >
> >>On Fri, Nov 17, 2017 at 3:31 PM, Matt Foley 
> wrote:
> >>
> >> (With release manager hat on)
> >>
> >> The community has proposed a release of Metron in the near future,
> >> focusing on Meta-alerts running in Elasticsearch.
> >> Congrats on getting so many of the below already done.  At this point,
> >> only METRON-1252, and the discussion of how to handle joint release of
> the
> >> Metron bro plugin, remain as gating items for the release.  I project
> these
> >> will be resolved next week, so let’s propose the following:
> >>
> >> Sometime next week, after the last bits are done, I’ll start the release
> >> process and create the release branch.
> >>
> >> The proposed new version will be 0.4.2, unless there are backward
> >> incompatible changes that support making it 0.5.0.
> >> Currently there are NO included Jiras labeled ‘backward-incompatible’,
> nor
> >> having Docs Text indicating so.
> >> ***If anyone knows that some of the commits included since 0.4.1
> introduce
> >> backward incompatibility, please say so now on this thread, and mark the
> >> Jira as such.***
> >>
> >> The 90 or so jiras/commits already in master branch since 0.4.1 are
> listed
> >> below.
> >> Thanks,
> >> --Matt
> >>
> >>METRON-1301 Alerts UI - Sorting on Triage Score Unexpectedly Filters
> >> Some Records (nickwallen) closes apache/metron#832
> >>METRON-1294 IP addresses are not formatted correctly in facet and
> >> group results (merrimanr) closes apache/metron#827
> >>METRON-1291 Kafka produce REST endpoint does not work in a Kerberized
> >> cluster (merrimanr) closes apache/metron#826
> >>METRON-1290 Only first 10 alerts are update when a MetaAlert status
> is
> >> changed to inactive (justinleet) closes apache/metron#842
> >>METRON-1311 Service Check Should Check Elasticsearch Index Templates
> >> (nickwallen) closes apache/metron#839
> >>METRON-1289 Alert fields are lost when a MetaAlert is created
> >> (merrimanr) closes apache/metron#824
> >>METRON-1309 Change metron-deployment to pull the plugin from
> >> apache/metron-bro-plugin-kafka (JonZeolla) closes apache/metron#837
> >>METRON-1310 Template Delete Action Deletes Search Indices
> (nickwallen)
> >> closes apache/metron#838
> >>METRON-1275: Fix Metron Documentation closes
> >> apache/incubator-metron#833
> >>METRON-1295 Unable to Configure Logging for REST API (nickwallen)
> >> closes apache/metron#828
> >>METRON-1307 Force install of java8 since java9 does not appear to
> work
> >> with the scripts (brianhurley via ottobackwards) closes
> apache/metron#835
> >>METRON-1296 Full Dev Fails to Deploy Index Templates (nickwallen via
> >> cestella) closes

Re: [DISCUSS] Upcoming Release

2017-11-17 Thread Ryan Merriman

Makes sense now.  Thanks Matt.

> On Nov 17, 2017, at 4:25 PM, Matt Foley  wrote:
> 
> Hi Ryan,
> Yes and no.  The last release (see 
> https://dist.apache.org/repos/dist/release/metron/ ) was 0.4.1, announced on 
> 9/19.
> Immediately after that we bumped the  of builds from master branch, 
> per https://issues.apache.org/jira/browse/METRON-1196 .  This is consistent 
> with the Release Process “Clean up” phase: “It is good practice to increment 
> the build version in master immediately after a Feature Release, so that dev 
> builds with new stuff from master cannot be mistaken for builds of the 
> release version. So, immediately after a release, increment the MINOR version 
> number (eg, with the 0.4.0 just released, set the new version number to 
> 0.4.1)” 
> (https://cwiki.apache.org/confluence/display/METRON/Release+Process#ReleaseProcess-Step15-Cleanup
>  )
> 
> So you’re correct that the version of development builds is currently set to 
> 0.4.2.  After we actually make a release of 0.4.2, we’ll change dev build 
>  to 0.4.3 (regardless of whether we expect the following release to 
> be 0.4.3 or 0.5.0).
> 
> Hope this clarifies,
> --Matt
> 
> On 11/17/17, 1:59 PM, "Ryan Merriman"  wrote:
> 
>Matt,
> 
>I think we are currently on version 0.4.2.  If that is the case would the
>next version be 0.4.3?
> 
>Ryan
> 
>>On Fri, Nov 17, 2017 at 3:31 PM, Matt Foley  wrote:
>> 
>> (With release manager hat on)
>> 
>> The community has proposed a release of Metron in the near future,
>> focusing on Meta-alerts running in Elasticsearch.
>> Congrats on getting so many of the below already done.  At this point,
>> only METRON-1252, and the discussion of how to handle joint release of the
>> Metron bro plugin, remain as gating items for the release.  I project these
>> will be resolved next week, so let’s propose the following:
>> 
>> Sometime next week, after the last bits are done, I’ll start the release
>> process and create the release branch.
>> 
>> The proposed new version will be 0.4.2, unless there are backward
>> incompatible changes that support making it 0.5.0.
>> Currently there are NO included Jiras labeled ‘backward-incompatible’, nor
>> having Docs Text indicating so.
>> ***If anyone knows that some of the commits included since 0.4.1 introduce
>> backward incompatibility, please say so now on this thread, and mark the
>> Jira as such.***
>> 
>> The 90 or so jiras/commits already in master branch since 0.4.1 are listed
>> below.
>> Thanks,
>> --Matt
>> 
>>METRON-1301 Alerts UI - Sorting on Triage Score Unexpectedly Filters
>> Some Records (nickwallen) closes apache/metron#832
>>METRON-1294 IP addresses are not formatted correctly in facet and
>> group results (merrimanr) closes apache/metron#827
>>METRON-1291 Kafka produce REST endpoint does not work in a Kerberized
>> cluster (merrimanr) closes apache/metron#826
>>METRON-1290 Only first 10 alerts are update when a MetaAlert status is
>> changed to inactive (justinleet) closes apache/metron#842
>>METRON-1311 Service Check Should Check Elasticsearch Index Templates
>> (nickwallen) closes apache/metron#839
>>METRON-1289 Alert fields are lost when a MetaAlert is created
>> (merrimanr) closes apache/metron#824
>>METRON-1309 Change metron-deployment to pull the plugin from
>> apache/metron-bro-plugin-kafka (JonZeolla) closes apache/metron#837
>>METRON-1310 Template Delete Action Deletes Search Indices (nickwallen)
>> closes apache/metron#838
>>METRON-1275: Fix Metron Documentation closes
>> apache/incubator-metron#833
>>METRON-1295 Unable to Configure Logging for REST API (nickwallen)
>> closes apache/metron#828
>>METRON-1307 Force install of java8 since java9 does not appear to work
>> with the scripts (brianhurley via ottobackwards) closes apache/metron#835
>>METRON-1296 Full Dev Fails to Deploy Index Templates (nickwallen via
>> cestella) closes apache/incubator-metron#829
>>METRON-1281 Remove hard-coded indices from the Alerts UI (merrimanr)
>> closes apache/metron#821
>>METRON-1287 Full Dev Fails When Installing EPEL Repository
>> (nickwallen) closes apache/metron#820
>>METRON-1267 Alerts UI returns a 404 when refreshing the alerts-list
>> page (iraghumitra via merrimanr) closes apache/metron#819
>>METRON-1283 Install Elasticsearch template as a part of the mpack
>> startup scripts (anandsubbu via nickwallen) closes apache/metron#817
>>METRON-1254: Conditionals as map keys do not function in Stellar
>> closes apache/incubator-metron#801
>>METRON-1261 Apply bro security patch (JonZeolla via ottobackwards)
>> closes apache/metron#805
>>METRON-1284 Remove extraneous dead query in ElasticsearchDao
>> (justinleet) closes apache/metron#818
>>METRON-1270: fix for warnings missing @return tag argument in
>> metron-analytics/metron-profiler-common and metron-profiler-client closes
>>

Re: [DISCUSS] Upcoming Release

2017-11-17 Thread Matt Foley

Hi Ryan,
Yes and no.  The last release (see 
https://dist.apache.org/repos/dist/release/metron/ ) was 0.4.1, announced on 
9/19.
Immediately after that we bumped the  of builds from master branch, 
per https://issues.apache.org/jira/browse/METRON-1196 .  This is consistent 
with the Release Process “Clean up” phase: “It is good practice to increment 
the build version in master immediately after a Feature Release, so that dev 
builds with new stuff from master cannot be mistaken for builds of the release 
version. So, immediately after a release, increment the MINOR version number 
(eg, with the 0.4.0 just released, set the new version number to 0.4.1)” 
(https://cwiki.apache.org/confluence/display/METRON/Release+Process#ReleaseProcess-Step15-Cleanup
 )

So you’re correct that the version of development builds is currently set to 
0.4.2.  After we actually make a release of 0.4.2, we’ll change dev build 
 to 0.4.3 (regardless of whether we expect the following release to be 
0.4.3 or 0.5.0).

Hope this clarifies,
--Matt

On 11/17/17, 1:59 PM, "Ryan Merriman"  wrote:

Matt,

I think we are currently on version 0.4.2.  If that is the case would the
next version be 0.4.3?

Ryan

On Fri, Nov 17, 2017 at 3:31 PM, Matt Foley  wrote:

> (With release manager hat on)
>
> The community has proposed a release of Metron in the near future,
> focusing on Meta-alerts running in Elasticsearch.
> Congrats on getting so many of the below already done.  At this point,
> only METRON-1252, and the discussion of how to handle joint release of the
> Metron bro plugin, remain as gating items for the release.  I project 
these
> will be resolved next week, so let’s propose the following:
>
> Sometime next week, after the last bits are done, I’ll start the release
> process and create the release branch.
>
> The proposed new version will be 0.4.2, unless there are backward
> incompatible changes that support making it 0.5.0.
> Currently there are NO included Jiras labeled ‘backward-incompatible’, nor
> having Docs Text indicating so.
> ***If anyone knows that some of the commits included since 0.4.1 introduce
> backward incompatibility, please say so now on this thread, and mark the
> Jira as such.***
>
> The 90 or so jiras/commits already in master branch since 0.4.1 are listed
> below.
> Thanks,
> --Matt
>
> METRON-1301 Alerts UI - Sorting on Triage Score Unexpectedly Filters
> Some Records (nickwallen) closes apache/metron#832
> METRON-1294 IP addresses are not formatted correctly in facet and
> group results (merrimanr) closes apache/metron#827
> METRON-1291 Kafka produce REST endpoint does not work in a Kerberized
> cluster (merrimanr) closes apache/metron#826
> METRON-1290 Only first 10 alerts are update when a MetaAlert status is
> changed to inactive (justinleet) closes apache/metron#842
> METRON-1311 Service Check Should Check Elasticsearch Index Templates
> (nickwallen) closes apache/metron#839
> METRON-1289 Alert fields are lost when a MetaAlert is created
> (merrimanr) closes apache/metron#824
> METRON-1309 Change metron-deployment to pull the plugin from
> apache/metron-bro-plugin-kafka (JonZeolla) closes apache/metron#837
> METRON-1310 Template Delete Action Deletes Search Indices (nickwallen)
> closes apache/metron#838
> METRON-1275: Fix Metron Documentation closes
> apache/incubator-metron#833
> METRON-1295 Unable to Configure Logging for REST API (nickwallen)
> closes apache/metron#828
> METRON-1307 Force install of java8 since java9 does not appear to work
> with the scripts (brianhurley via ottobackwards) closes apache/metron#835
> METRON-1296 Full Dev Fails to Deploy Index Templates (nickwallen via
> cestella) closes apache/incubator-metron#829
> METRON-1281 Remove hard-coded indices from the Alerts UI (merrimanr)
> closes apache/metron#821
> METRON-1287 Full Dev Fails When Installing EPEL Repository
> (nickwallen) closes apache/metron#820
> METRON-1267 Alerts UI returns a 404 when refreshing the alerts-list
> page (iraghumitra via merrimanr) closes apache/metron#819
> METRON-1283 Install Elasticsearch template as a part of the mpack
> startup scripts (anandsubbu via nickwallen) closes apache/metron#817
> METRON-1254: Conditionals as map keys do not function in Stellar
> closes apache/incubator-metron#801
> METRON-1261 Apply bro security patch (JonZeolla via ottobackwards)
> closes apache/metron#805
> METRON-1284 Remove extraneous dead query in ElasticsearchDao
> (justinleet) closes apache/metron#818
> METRON-1270: fix for warnings missing @return tag argument in
>

Re: [DISCUSS] Upcoming Release

2017-11-17 Thread Nick Allen

Our last release was 0.4.1, so the next would be at least 0.4.2.  We
recently have been keeping master at the next presumed release version.

On Fri, Nov 17, 2017 at 4:59 PM, Ryan Merriman  wrote:

> Matt,
>
> I think we are currently on version 0.4.2.  If that is the case would the
> next version be 0.4.3?
>
> Ryan
>
> On Fri, Nov 17, 2017 at 3:31 PM, Matt Foley  wrote:
>
> > (With release manager hat on)
> >
> > The community has proposed a release of Metron in the near future,
> > focusing on Meta-alerts running in Elasticsearch.
> > Congrats on getting so many of the below already done.  At this point,
> > only METRON-1252, and the discussion of how to handle joint release of
> the
> > Metron bro plugin, remain as gating items for the release.  I project
> these
> > will be resolved next week, so let’s propose the following:
> >
> > Sometime next week, after the last bits are done, I’ll start the release
> > process and create the release branch.
> >
> > The proposed new version will be 0.4.2, unless there are backward
> > incompatible changes that support making it 0.5.0.
> > Currently there are NO included Jiras labeled ‘backward-incompatible’,
> nor
> > having Docs Text indicating so.
> > ***If anyone knows that some of the commits included since 0.4.1
> introduce
> > backward incompatibility, please say so now on this thread, and mark the
> > Jira as such.***
> >
> > The 90 or so jiras/commits already in master branch since 0.4.1 are
> listed
> > below.
> > Thanks,
> > --Matt
> >
> > METRON-1301 Alerts UI - Sorting on Triage Score Unexpectedly Filters
> > Some Records (nickwallen) closes apache/metron#832
> > METRON-1294 IP addresses are not formatted correctly in facet and
> > group results (merrimanr) closes apache/metron#827
> > METRON-1291 Kafka produce REST endpoint does not work in a Kerberized
> > cluster (merrimanr) closes apache/metron#826
> > METRON-1290 Only first 10 alerts are update when a MetaAlert status
> is
> > changed to inactive (justinleet) closes apache/metron#842
> > METRON-1311 Service Check Should Check Elasticsearch Index Templates
> > (nickwallen) closes apache/metron#839
> > METRON-1289 Alert fields are lost when a MetaAlert is created
> > (merrimanr) closes apache/metron#824
> > METRON-1309 Change metron-deployment to pull the plugin from
> > apache/metron-bro-plugin-kafka (JonZeolla) closes apache/metron#837
> > METRON-1310 Template Delete Action Deletes Search Indices
> (nickwallen)
> > closes apache/metron#838
> > METRON-1275: Fix Metron Documentation closes
> > apache/incubator-metron#833
> > METRON-1295 Unable to Configure Logging for REST API (nickwallen)
> > closes apache/metron#828
> > METRON-1307 Force install of java8 since java9 does not appear to
> work
> > with the scripts (brianhurley via ottobackwards) closes apache/metron#835
> > METRON-1296 Full Dev Fails to Deploy Index Templates (nickwallen via
> > cestella) closes apache/incubator-metron#829
> > METRON-1281 Remove hard-coded indices from the Alerts UI (merrimanr)
> > closes apache/metron#821
> > METRON-1287 Full Dev Fails When Installing EPEL Repository
> > (nickwallen) closes apache/metron#820
> > METRON-1267 Alerts UI returns a 404 when refreshing the alerts-list
> > page (iraghumitra via merrimanr) closes apache/metron#819
> > METRON-1283 Install Elasticsearch template as a part of the mpack
> > startup scripts (anandsubbu via nickwallen) closes apache/metron#817
> > METRON-1254: Conditionals as map keys do not function in Stellar
> > closes apache/incubator-metron#801
> > METRON-1261 Apply bro security patch (JonZeolla via ottobackwards)
> > closes apache/metron#805
> > METRON-1284 Remove extraneous dead query in ElasticsearchDao
> > (justinleet) closes apache/metron#818
> > METRON-1270: fix for warnings missing @return tag argument in
> > metron-analytics/metron-profiler-common and metron-profiler-client
> closes
> > apache/incubator-metron#810
> > METRON-1272 Hide child alerts from searches and grouping if they
> > belong to meta alerts (justinleet) closes apache/metron#811
> > METRON-1224 Add time range selection to search control (iraghumitra
> > via james-sirota) closes apache/metron#796
> > METRON-1280 0.4.1 -> 0.4.2 missed a couple of projects (cestella via
> > justinleet) closes apache/metron#816
> > METRON-1243: Add a REST endpoint which allows us to get a list of all
> > indice closes apache/incubator-metron#797
> > METRON-1196 Increment master version number to 0.4.2 for on-going
> > development (mattf-horton) closes apache/metron#767
> > METRON-1278 Strip Build Status widget from root README.md
> > in site-book build (mattf-horton) closes apache/metron#815
> > METRON-1274 Master has failure in StormControllerIntegrationTest
> > (merrimanr) closes apache/metron#813
> > METRON-1266 Profiler - SASL Authentication Failed (nickwallen) closes

Re: [DISCUSS] Upcoming Release

2017-11-17 Thread Ryan Merriman

Matt,

I think we are currently on version 0.4.2.  If that is the case would the
next version be 0.4.3?

Ryan

On Fri, Nov 17, 2017 at 3:31 PM, Matt Foley  wrote:

> (With release manager hat on)
>
> The community has proposed a release of Metron in the near future,
> focusing on Meta-alerts running in Elasticsearch.
> Congrats on getting so many of the below already done.  At this point,
> only METRON-1252, and the discussion of how to handle joint release of the
> Metron bro plugin, remain as gating items for the release.  I project these
> will be resolved next week, so let’s propose the following:
>
> Sometime next week, after the last bits are done, I’ll start the release
> process and create the release branch.
>
> The proposed new version will be 0.4.2, unless there are backward
> incompatible changes that support making it 0.5.0.
> Currently there are NO included Jiras labeled ‘backward-incompatible’, nor
> having Docs Text indicating so.
> ***If anyone knows that some of the commits included since 0.4.1 introduce
> backward incompatibility, please say so now on this thread, and mark the
> Jira as such.***
>
> The 90 or so jiras/commits already in master branch since 0.4.1 are listed
> below.
> Thanks,
> --Matt
>
> METRON-1301 Alerts UI - Sorting on Triage Score Unexpectedly Filters
> Some Records (nickwallen) closes apache/metron#832
> METRON-1294 IP addresses are not formatted correctly in facet and
> group results (merrimanr) closes apache/metron#827
> METRON-1291 Kafka produce REST endpoint does not work in a Kerberized
> cluster (merrimanr) closes apache/metron#826
> METRON-1290 Only first 10 alerts are update when a MetaAlert status is
> changed to inactive (justinleet) closes apache/metron#842
> METRON-1311 Service Check Should Check Elasticsearch Index Templates
> (nickwallen) closes apache/metron#839
> METRON-1289 Alert fields are lost when a MetaAlert is created
> (merrimanr) closes apache/metron#824
> METRON-1309 Change metron-deployment to pull the plugin from
> apache/metron-bro-plugin-kafka (JonZeolla) closes apache/metron#837
> METRON-1310 Template Delete Action Deletes Search Indices (nickwallen)
> closes apache/metron#838
> METRON-1275: Fix Metron Documentation closes
> apache/incubator-metron#833
> METRON-1295 Unable to Configure Logging for REST API (nickwallen)
> closes apache/metron#828
> METRON-1307 Force install of java8 since java9 does not appear to work
> with the scripts (brianhurley via ottobackwards) closes apache/metron#835
> METRON-1296 Full Dev Fails to Deploy Index Templates (nickwallen via
> cestella) closes apache/incubator-metron#829
> METRON-1281 Remove hard-coded indices from the Alerts UI (merrimanr)
> closes apache/metron#821
> METRON-1287 Full Dev Fails When Installing EPEL Repository
> (nickwallen) closes apache/metron#820
> METRON-1267 Alerts UI returns a 404 when refreshing the alerts-list
> page (iraghumitra via merrimanr) closes apache/metron#819
> METRON-1283 Install Elasticsearch template as a part of the mpack
> startup scripts (anandsubbu via nickwallen) closes apache/metron#817
> METRON-1254: Conditionals as map keys do not function in Stellar
> closes apache/incubator-metron#801
> METRON-1261 Apply bro security patch (JonZeolla via ottobackwards)
> closes apache/metron#805
> METRON-1284 Remove extraneous dead query in ElasticsearchDao
> (justinleet) closes apache/metron#818
> METRON-1270: fix for warnings missing @return tag argument in
> metron-analytics/metron-profiler-common and metron-profiler-client closes
> apache/incubator-metron#810
> METRON-1272 Hide child alerts from searches and grouping if they
> belong to meta alerts (justinleet) closes apache/metron#811
> METRON-1224 Add time range selection to search control (iraghumitra
> via james-sirota) closes apache/metron#796
> METRON-1280 0.4.1 -> 0.4.2 missed a couple of projects (cestella via
> justinleet) closes apache/metron#816
> METRON-1243: Add a REST endpoint which allows us to get a list of all
> indice closes apache/incubator-metron#797
> METRON-1196 Increment master version number to 0.4.2 for on-going
> development (mattf-horton) closes apache/metron#767
> METRON-1278 Strip Build Status widget from root README.md
> in site-book build (mattf-horton) closes apache/metron#815
> METRON-1274 Master has failure in StormControllerIntegrationTest
> (merrimanr) closes apache/metron#813
> METRON-1266 Profiler - SASL Authentication Failed (nickwallen) closes
> apache/metron#809
> METRON-1260 Include Alerts UI in Ambari Service Check (nickwallen)
> closes apache/metron#804
> METRON-1251: Typo and formatting fixes for metron-rest README closes
> apache/incubator-metron#800
> METRON-1241: Enable the REST API to use a cache for the zookeeper
> config similar to the Bolts closes apache/incubator-metron#795
> METRON-1267 Alerts UI returns a 404 when

Re: [DISCUSS] Upcoming Release

2017-11-17 Thread Matt Foley

(With release manager hat on)

The community has proposed a release of Metron in the near future, focusing on 
Meta-alerts running in Elasticsearch.
Congrats on getting so many of the below already done.  At this point, only 
METRON-1252, and the discussion of how to handle joint release of the Metron 
bro plugin, remain as gating items for the release.  I project these will be 
resolved next week, so let’s propose the following:

Sometime next week, after the last bits are done, I’ll start the release 
process and create the release branch.

The proposed new version will be 0.4.2, unless there are backward incompatible 
changes that support making it 0.5.0.
Currently there are NO included Jiras labeled ‘backward-incompatible’, nor 
having Docs Text indicating so.
***If anyone knows that some of the commits included since 0.4.1 introduce 
backward incompatibility, please say so now on this thread, and mark the Jira 
as such.***

The 90 or so jiras/commits already in master branch since 0.4.1 are listed 
below.
Thanks,
--Matt

METRON-1301 Alerts UI - Sorting on Triage Score Unexpectedly Filters Some 
Records (nickwallen) closes apache/metron#832
METRON-1294 IP addresses are not formatted correctly in facet and group 
results (merrimanr) closes apache/metron#827
METRON-1291 Kafka produce REST endpoint does not work in a Kerberized 
cluster (merrimanr) closes apache/metron#826
METRON-1290 Only first 10 alerts are update when a MetaAlert status is 
changed to inactive (justinleet) closes apache/metron#842
METRON-1311 Service Check Should Check Elasticsearch Index Templates 
(nickwallen) closes apache/metron#839
METRON-1289 Alert fields are lost when a MetaAlert is created (merrimanr) 
closes apache/metron#824
METRON-1309 Change metron-deployment to pull the plugin from 
apache/metron-bro-plugin-kafka (JonZeolla) closes apache/metron#837
METRON-1310 Template Delete Action Deletes Search Indices (nickwallen) 
closes apache/metron#838
METRON-1275: Fix Metron Documentation closes apache/incubator-metron#833
METRON-1295 Unable to Configure Logging for REST API (nickwallen) closes 
apache/metron#828
METRON-1307 Force install of java8 since java9 does not appear to work with 
the scripts (brianhurley via ottobackwards) closes apache/metron#835
METRON-1296 Full Dev Fails to Deploy Index Templates (nickwallen via 
cestella) closes apache/incubator-metron#829
METRON-1281 Remove hard-coded indices from the Alerts UI (merrimanr) closes 
apache/metron#821
METRON-1287 Full Dev Fails When Installing EPEL Repository (nickwallen) 
closes apache/metron#820
METRON-1267 Alerts UI returns a 404 when refreshing the alerts-list page 
(iraghumitra via merrimanr) closes apache/metron#819
METRON-1283 Install Elasticsearch template as a part of the mpack startup 
scripts (anandsubbu via nickwallen) closes apache/metron#817
METRON-1254: Conditionals as map keys do not function in Stellar closes 
apache/incubator-metron#801
METRON-1261 Apply bro security patch (JonZeolla via ottobackwards) closes 
apache/metron#805
METRON-1284 Remove extraneous dead query in ElasticsearchDao (justinleet) 
closes apache/metron#818
METRON-1270: fix for warnings missing @return tag argument in 
metron-analytics/metron-profiler-common and metron-profiler-client closes 
apache/incubator-metron#810
METRON-1272 Hide child alerts from searches and grouping if they belong to 
meta alerts (justinleet) closes apache/metron#811
METRON-1224 Add time range selection to search control (iraghumitra via 
james-sirota) closes apache/metron#796
METRON-1280 0.4.1 -> 0.4.2 missed a couple of projects (cestella via 
justinleet) closes apache/metron#816
METRON-1243: Add a REST endpoint which allows us to get a list of all 
indice closes apache/incubator-metron#797
METRON-1196 Increment master version number to 0.4.2 for on-going 
development (mattf-horton) closes apache/metron#767
METRON-1278 Strip Build Status widget from root README.md in 
site-book build (mattf-horton) closes apache/metron#815
METRON-1274 Master has failure in StormControllerIntegrationTest 
(merrimanr) closes apache/metron#813
METRON-1266 Profiler - SASL Authentication Failed (nickwallen) closes 
apache/metron#809
METRON-1260 Include Alerts UI in Ambari Service Check (nickwallen) closes 
apache/metron#804
METRON-1251: Typo and formatting fixes for metron-rest README closes 
apache/incubator-metron#800
METRON-1241: Enable the REST API to use a cache for the zookeeper config 
similar to the Bolts closes apache/incubator-metron#795
METRON-1267 Alerts UI returns a 404 when refreshing the alerts-list page 
(merrimanr) closes apache/metron#808
METRON-1262 Unable to add comment for a alert in a meta-alert (merrimanr) 
closes apache/metron#806
METRON-1263 Start Alerts UI service after Metron REST (anandsubbu via 
nickwallen) closes apache/metron#807
METRON-1255 MetaAlert

Re: [DISCUSS] Upcoming Release

2017-11-17 Thread Nick Allen

I just wanted to send an update on where we are at.  We've gotten a lot
done here recently as you can see below.

  ✓ DONE (1) First, METRON-1289 needs to go in.  This one was a fairly big
effort and I am hearing that we are pretty close.

  ✓ DONE (2) METRON-1294 fixes an issue in how field types are looked-up.

  ✓ DONE (3) METRON-1290 is next.  While this may have been fixed in
M-1289, there may be some test cases we want from this PR.

  ✓ DONE (4) METRON-1301 addresses a problem with the sorting logic.

  ✓ DONE (5) METRON-1291 fixes an issue with escalation of metaalerts.

  (6) That leads us to Raghu's UI work in METRON-1252.  This introduces the
UI bits that depend on all the previous backend work.

  (7) At this point, we should have our best effort at running Metaalerts
on Elasticsearch 2.x. I propose that we cut a release here.

  (8) After we cut the release, we can introduce the work for ES 5.x in
METRON-939.  I know we will need lots of help testing and reviewing this
one.

We also have an outstanding question that needs resolved BEFORE we
release.  We need to come to a consensus on how to release having moved our
Bro Plugin to a separate repo.  I don't think we've heard from everyone on
this.  I'd urge everyone to chime in so we can choose a path forward.

If anyone is totally confused in regards to that discussion, I can try and
send an options summary again as a separate discuss thread.  The original
chain was somewhere around here [1].

[1]
https://lists.apache.org/thread.html/54a4474881b97e559df24728b3a0e923a58345a282451085eef832ef@%3Cdev.metron.apache.org%3E

On Wed, Nov 15, 2017 at 10:04 AM, Nick Allen  wrote:

> Hi Guys -
>
> I want to follow-up on this discussion.  It sounds like most people are in
> agreement with the general approach.
>
> A lot of people have been working hard on Metaalerts and Elasticsearch.  I
> have checked-in with those doing the heavy lifting and have compiled a more
> detailed plan based on where we are at now.  To the best of my knowledge
> here is the plan of attack for finishing out this effort.
>
>   (1) First, METRON-1289 needs to go in.  This one was a fairly big effort
> and I am hearing that we are pretty close.
>
>   (2) METRON-1294 fixes an issue in how field types are looked-up.
>
>   (3) METRON-1290 is next.  While this may have been fixed in M-1289,
> there may be some test cases we want from this PR.
>
>   (4) METRON-1301 addresses a problem with the sorting logic.
>
>   (5) METRON-1291 fixes an issue with escalation of metaalerts.
>
>   (6) That leads us to Raghu's UI work in METRON-1252.  This introduces
> the UI bits that depend on all the previous backend work.
>
>   (7) At this point, we should have our best effort at running Metaalerts
> on Elasticsearch 2.x. I propose that we cut a release here.
>
>   (8) After we cut the release, we can introduce the work for ES 5.x in
> METRON-939.  I know we will need lots of help testing and reviewing this
> one.
>
> Please correct me if I am wrong.  I will try and send out updates as we
> make progress.
>
>
>
>
>
> On Mon, Nov 6, 2017 at 1:03 PM, zeo...@gmail.com  wrote:
>
>> I agree, I think it's very reasonable to move in line with Nick's
>> proposal.  I would also suggest that we outline what the target versions
>> would be to add in the METRON-777 components, since it has been functional
>> for a very long time but not reviewed and has some really rockstar
>> improvements.
>>
>> Jon
>>
>> On Mon, Nov 6, 2017 at 12:56 PM Otto Fowler 
>> wrote:
>>
>> > I think the ES cutover should be the start of the 0.5.x series, and we
>> > continue on with 0.4.x for the
>> > metadata improvements etc.  We could chose to focus 0.5.x’s first
>> releases
>> > on not only ES but
>> > getting a handle on kibana and the mpack situation as well.
>> >
>> >
>> >
>> >
>> > On November 6, 2017 at 12:48:45, Michael Miklavcic (
>> > michael.miklav...@gmail.com) wrote:
>> >
>> > I agree with your proposal, Nick. I think having a stabilizing release
>> > prior to upgrading ES/Kibana makes sense.
>> >
>> > On Mon, Nov 6, 2017 at 9:16 AM, Nick Allen  wrote:
>> >
>> > > I would like to start a discussion around upcoming releases. We have a
>> > > couple separate significant tracks of work that we need to reconcile
>> in
>> > our
>> > > release schedule.
>> > >
>> > > (1) We have had (and have in review) a good number of bug fixes
>> required
>> > to
>> > > support Metaalerts on the existing Elasticsearch 2.x infrastructure.
>> > >
>> > >
>> > > (2) We also have ongoing work to upgrade our infrastructure to
>> > > Elasticsearch 5.x, which will not be backwards compatible.
>> > >
>> > >
>> > > I would like to see a release that has our best work on ES 2.x before
>> we
>> > > migrate to 5.x. I would propose the following.
>> > >
>> > > Release N+1: Introduce Metaalerts running on ES 2.x
>> > >
>> > > Release N+2: Cut-over to ES 5.x
>> > >
>> > >

Re: [DISCUSS] Upcoming Release

2017-11-16 Thread Nick Allen

Sorry, missed one.  I think this is the one Jon prefers.

(5) The other alternative is just to complete Jon's roadmap BEFORE the next
release.

On Thu, Nov 16, 2017 at 10:37 AM, Nick Allen  wrote:

> Just to layout some other alternatives...
>
> (1) [Jon] Make a release in apache/metron-bro-plugin-kafka, then update
> Ansible to pin to that release.
>
> (2) [Jon] Update Ansible to pin to a specific commit.
>
> (3) Wouldn't the submodule approach solve this?  I imagine that if we use
> a submodule, then all of the Bro plugin code will be contained directly in
> each Metron release.  We would just need to change that small bit of
> Ansible code so that it uses the code directly (like before) instead of
> going to Git and checking it out.
>
> (4) Revert PR #837 because as you pointed out this approach does not work
> well with releases.  That would give us enough time to come up with a
> sensible approach and do that.
>
>
>
>
>
>
> On Thu, Nov 16, 2017 at 10:21 AM, zeo...@gmail.com 
> wrote:
>
>> The problem is that we're currently pinning to master and if something in
>> the plugin breaks backward compatibility, our prior releases will be
>> perpetually broken, which is why I suggest it blocks the upcoming release.
>>
>> The alternative would be to update ansible to pin to a specific commit or
>> to make a release in apache/metron-bro-plugin-kafka sooner rather than
>> later and pin to its branch/tag.  That feels like a waste of time though,
>> as the 2.5.2 upgrade is somewhat trivial.
>>
>> Jon
>>
>> On Thu, Nov 16, 2017 at 10:14 AM Nick Allen  wrote:
>>
>> > While I think that the Bro work is super valuable, Jon, I am not sure
>> that
>> > we need to block the next release for it.  In my mind, the "theme" of
>> the
>> > next release is Metaalerts running on ES 2.x.  I'd prefer to just stick
>> > with that.
>> >
>> > I was hoping we could get the remaining "Metaalerts + ES 2.x" PRs merged
>> > and start cutting release candidates ASAP.  That could be possible by
>> end
>> > of week based on the "big one" (METRON-1289) getting merged in last
>> night.
>> >
>> > Of course, if you happen to get all the Bro work done in time, then
>> > definitely let's include it in the next release.  But I am wary of
>> blocking
>> > the release for that work.  No need for you to rush through it.
>> >
>> > Just one man's opinion.  Would like to hear feedback from more of the
>> > community.
>> >
>> >
>> >
>> > On Thu, Nov 16, 2017 at 8:01 AM, zeo...@gmail.com 
>> > wrote:
>> >
>> > > The way master's full-dev is set up right now is non optimal for the
>> bro
>> > > plugin configuration, and I would like to complete the roadmap I
>> outlined
>> > > in my discuss thread before a release.  I have a PR open against
>> > > Apache/metron-bro-plugin-kafka right now to turn it into a plugin,
>> and I
>> > > expect it will take me until end of next week at the latest to have
>> the
>> > > rest of the work done to move us to 2.5.2, and to pin to a specific
>> > package
>> > > version.  At that point I'm comfortable with a release.
>> > >
>> > > Jon
>> > >
>> > > On Wed, Nov 15, 2017, 18:57 Matt Foley  wrote:
>> > >
>> > > > I’ve been listening.  Looks like there are still a number of major
>> > issues
>> > > > to be committed first, right?
>> > > > The discussion on this thread constitutes sufficient engagement, I
>> > think,
>> > > > especially given the Subject line :-)
>> > > > Would the folks working on the 6 issues listed by Nick care to
>> suggest
>> > a
>> > > > cut-off date by which they’ll probably have those fixes in?
>> > > > I’ll be happy to run the release process, if the community so
>> wishes,
>> > as
>> > > > soon as those issues are committed.
>> > > >
>> > > > I guess I should, pro forma, send the list of commits already in
>> since
>> > > the
>> > > > last release.  I’ll do that today.
>> > > > Also, if anyone wishes to raise a hand and propose additional
>> commits
>> > are
>> > > > needed, please do so on this thread.
>> > > > Thanks,
>> > > > --Matt
>> > > >
>> > > >
>> > > > On 11/15/17, 2:09 PM, "Casey Stella"  wrote:
>> > > >
>> > > > I'd say that if a release is this imminent that we had better
>> > notify
>> > > > the
>> > > > release manager who will make a release announcement, Nick.
>> Matt,
>> > > are
>> > > > you
>> > > > tuning in to this?
>> > > >
>> > > > On Wed, Nov 15, 2017 at 10:04 AM, Nick Allen <
>> n...@nickallen.org>
>> > > > wrote:
>> > > >
>> > > > > Hi Guys -
>> > > > >
>> > > > > I want to follow-up on this discussion.  It sounds like most
>> > people
>> > > > are in
>> > > > > agreement with the general approach.
>> > > > >
>> > > > > A lot of people have been working hard on Metaalerts and
>> > > > Elasticsearch.  I
>> > > > > have checked-in with those doing the heavy lifting and have
>> > > compiled
>> > > > a more
>> > > > >

Re: [DISCUSS] Upcoming Release

2017-11-16 Thread Nick Allen

> (1) [Jon] Make a release in apache/metron-bro-plugin-kafka, then update
Ansible to pin to that release.

The problem with this approach, as I see it, is that it commits us to
having separate releases for the Bro plugin.  Which I am not sure if or how
long it will take to come to a consensus on that.  We also would need to
invent that release process rather quickly.

> (4) Revert PR #837 because as you pointed out this approach does not work
well with releases.  That would give us enough time to come up with a
sensible approach and do that.

I am kind of leaning towards (4) right now.  The approach taken in this PR,
doesn't work.  Let's just revert it and give ourselves some time to come up
with an approach that does work and has consensus.







On Thu, Nov 16, 2017 at 10:37 AM, Nick Allen  wrote:

> Just to layout some other alternatives...
>
> (1) [Jon] Make a release in apache/metron-bro-plugin-kafka, then update
> Ansible to pin to that release.
>
> (2) [Jon] Update Ansible to pin to a specific commit.
>
> (3) Wouldn't the submodule approach solve this?  I imagine that if we use
> a submodule, then all of the Bro plugin code will be contained directly in
> each Metron release.  We would just need to change that small bit of
> Ansible code so that it uses the code directly (like before) instead of
> going to Git and checking it out.
>
> (4) Revert PR #837 because as you pointed out this approach does not work
> well with releases.  That would give us enough time to come up with a
> sensible approach and do that.
>
>
>
>
>
>
> On Thu, Nov 16, 2017 at 10:21 AM, zeo...@gmail.com 
> wrote:
>
>> The problem is that we're currently pinning to master and if something in
>> the plugin breaks backward compatibility, our prior releases will be
>> perpetually broken, which is why I suggest it blocks the upcoming release.
>>
>> The alternative would be to update ansible to pin to a specific commit or
>> to make a release in apache/metron-bro-plugin-kafka sooner rather than
>> later and pin to its branch/tag.  That feels like a waste of time though,
>> as the 2.5.2 upgrade is somewhat trivial.
>>
>> Jon
>>
>> On Thu, Nov 16, 2017 at 10:14 AM Nick Allen  wrote:
>>
>> > While I think that the Bro work is super valuable, Jon, I am not sure
>> that
>> > we need to block the next release for it.  In my mind, the "theme" of
>> the
>> > next release is Metaalerts running on ES 2.x.  I'd prefer to just stick
>> > with that.
>> >
>> > I was hoping we could get the remaining "Metaalerts + ES 2.x" PRs merged
>> > and start cutting release candidates ASAP.  That could be possible by
>> end
>> > of week based on the "big one" (METRON-1289) getting merged in last
>> night.
>> >
>> > Of course, if you happen to get all the Bro work done in time, then
>> > definitely let's include it in the next release.  But I am wary of
>> blocking
>> > the release for that work.  No need for you to rush through it.
>> >
>> > Just one man's opinion.  Would like to hear feedback from more of the
>> > community.
>> >
>> >
>> >
>> > On Thu, Nov 16, 2017 at 8:01 AM, zeo...@gmail.com 
>> > wrote:
>> >
>> > > The way master's full-dev is set up right now is non optimal for the
>> bro
>> > > plugin configuration, and I would like to complete the roadmap I
>> outlined
>> > > in my discuss thread before a release.  I have a PR open against
>> > > Apache/metron-bro-plugin-kafka right now to turn it into a plugin,
>> and I
>> > > expect it will take me until end of next week at the latest to have
>> the
>> > > rest of the work done to move us to 2.5.2, and to pin to a specific
>> > package
>> > > version.  At that point I'm comfortable with a release.
>> > >
>> > > Jon
>> > >
>> > > On Wed, Nov 15, 2017, 18:57 Matt Foley  wrote:
>> > >
>> > > > I’ve been listening.  Looks like there are still a number of major
>> > issues
>> > > > to be committed first, right?
>> > > > The discussion on this thread constitutes sufficient engagement, I
>> > think,
>> > > > especially given the Subject line :-)
>> > > > Would the folks working on the 6 issues listed by Nick care to
>> suggest
>> > a
>> > > > cut-off date by which they’ll probably have those fixes in?
>> > > > I’ll be happy to run the release process, if the community so
>> wishes,
>> > as
>> > > > soon as those issues are committed.
>> > > >
>> > > > I guess I should, pro forma, send the list of commits already in
>> since
>> > > the
>> > > > last release.  I’ll do that today.
>> > > > Also, if anyone wishes to raise a hand and propose additional
>> commits
>> > are
>> > > > needed, please do so on this thread.
>> > > > Thanks,
>> > > > --Matt
>> > > >
>> > > >
>> > > > On 11/15/17, 2:09 PM, "Casey Stella"  wrote:
>> > > >
>> > > > I'd say that if a release is this imminent that we had better
>> > notify
>> > > > the
>> > > > release manager who will make a release announcement, Nick.

Re: [DISCUSS] Upcoming Release

2017-11-16 Thread Nick Allen

Just to layout some other alternatives...

(1) [Jon] Make a release in apache/metron-bro-plugin-kafka, then update
Ansible to pin to that release.

(2) [Jon] Update Ansible to pin to a specific commit.

(3) Wouldn't the submodule approach solve this?  I imagine that if we use a
submodule, then all of the Bro plugin code will be contained directly in
each Metron release.  We would just need to change that small bit of
Ansible code so that it uses the code directly (like before) instead of
going to Git and checking it out.

(4) Revert PR #837 because as you pointed out this approach does not work
well with releases.  That would give us enough time to come up with a
sensible approach and do that.






On Thu, Nov 16, 2017 at 10:21 AM, zeo...@gmail.com  wrote:

> The problem is that we're currently pinning to master and if something in
> the plugin breaks backward compatibility, our prior releases will be
> perpetually broken, which is why I suggest it blocks the upcoming release.
>
> The alternative would be to update ansible to pin to a specific commit or
> to make a release in apache/metron-bro-plugin-kafka sooner rather than
> later and pin to its branch/tag.  That feels like a waste of time though,
> as the 2.5.2 upgrade is somewhat trivial.
>
> Jon
>
> On Thu, Nov 16, 2017 at 10:14 AM Nick Allen  wrote:
>
> > While I think that the Bro work is super valuable, Jon, I am not sure
> that
> > we need to block the next release for it.  In my mind, the "theme" of the
> > next release is Metaalerts running on ES 2.x.  I'd prefer to just stick
> > with that.
> >
> > I was hoping we could get the remaining "Metaalerts + ES 2.x" PRs merged
> > and start cutting release candidates ASAP.  That could be possible by end
> > of week based on the "big one" (METRON-1289) getting merged in last
> night.
> >
> > Of course, if you happen to get all the Bro work done in time, then
> > definitely let's include it in the next release.  But I am wary of
> blocking
> > the release for that work.  No need for you to rush through it.
> >
> > Just one man's opinion.  Would like to hear feedback from more of the
> > community.
> >
> >
> >
> > On Thu, Nov 16, 2017 at 8:01 AM, zeo...@gmail.com 
> > wrote:
> >
> > > The way master's full-dev is set up right now is non optimal for the
> bro
> > > plugin configuration, and I would like to complete the roadmap I
> outlined
> > > in my discuss thread before a release.  I have a PR open against
> > > Apache/metron-bro-plugin-kafka right now to turn it into a plugin, and
> I
> > > expect it will take me until end of next week at the latest to have the
> > > rest of the work done to move us to 2.5.2, and to pin to a specific
> > package
> > > version.  At that point I'm comfortable with a release.
> > >
> > > Jon
> > >
> > > On Wed, Nov 15, 2017, 18:57 Matt Foley  wrote:
> > >
> > > > I’ve been listening.  Looks like there are still a number of major
> > issues
> > > > to be committed first, right?
> > > > The discussion on this thread constitutes sufficient engagement, I
> > think,
> > > > especially given the Subject line :-)
> > > > Would the folks working on the 6 issues listed by Nick care to
> suggest
> > a
> > > > cut-off date by which they’ll probably have those fixes in?
> > > > I’ll be happy to run the release process, if the community so wishes,
> > as
> > > > soon as those issues are committed.
> > > >
> > > > I guess I should, pro forma, send the list of commits already in
> since
> > > the
> > > > last release.  I’ll do that today.
> > > > Also, if anyone wishes to raise a hand and propose additional commits
> > are
> > > > needed, please do so on this thread.
> > > > Thanks,
> > > > --Matt
> > > >
> > > >
> > > > On 11/15/17, 2:09 PM, "Casey Stella"  wrote:
> > > >
> > > > I'd say that if a release is this imminent that we had better
> > notify
> > > > the
> > > > release manager who will make a release announcement, Nick.
> Matt,
> > > are
> > > > you
> > > > tuning in to this?
> > > >
> > > > On Wed, Nov 15, 2017 at 10:04 AM, Nick Allen  >
> > > > wrote:
> > > >
> > > > > Hi Guys -
> > > > >
> > > > > I want to follow-up on this discussion.  It sounds like most
> > people
> > > > are in
> > > > > agreement with the general approach.
> > > > >
> > > > > A lot of people have been working hard on Metaalerts and
> > > > Elasticsearch.  I
> > > > > have checked-in with those doing the heavy lifting and have
> > > compiled
> > > > a more
> > > > > detailed plan based on where we are at now.  To the best of my
> > > > knowledge
> > > > > here is the plan of attack for finishing out this effort.
> > > > >
> > > > >   (1) First, METRON-1289 needs to go in.  This one was a fairly
> > big
> > > > effort
> > > > > and I am hearing that we are pretty close.
> > > > >
> > > > >   (2) METRON-1294 fixes an

Re: [DISCUSS] Upcoming Release

2017-11-16 Thread zeo...@gmail.com

The problem is that we're currently pinning to master and if something in
the plugin breaks backward compatibility, our prior releases will be
perpetually broken, which is why I suggest it blocks the upcoming release.

The alternative would be to update ansible to pin to a specific commit or
to make a release in apache/metron-bro-plugin-kafka sooner rather than
later and pin to its branch/tag.  That feels like a waste of time though,
as the 2.5.2 upgrade is somewhat trivial.

Jon

On Thu, Nov 16, 2017 at 10:14 AM Nick Allen  wrote:

> While I think that the Bro work is super valuable, Jon, I am not sure that
> we need to block the next release for it.  In my mind, the "theme" of the
> next release is Metaalerts running on ES 2.x.  I'd prefer to just stick
> with that.
>
> I was hoping we could get the remaining "Metaalerts + ES 2.x" PRs merged
> and start cutting release candidates ASAP.  That could be possible by end
> of week based on the "big one" (METRON-1289) getting merged in last night.
>
> Of course, if you happen to get all the Bro work done in time, then
> definitely let's include it in the next release.  But I am wary of blocking
> the release for that work.  No need for you to rush through it.
>
> Just one man's opinion.  Would like to hear feedback from more of the
> community.
>
>
>
> On Thu, Nov 16, 2017 at 8:01 AM, zeo...@gmail.com 
> wrote:
>
> > The way master's full-dev is set up right now is non optimal for the bro
> > plugin configuration, and I would like to complete the roadmap I outlined
> > in my discuss thread before a release.  I have a PR open against
> > Apache/metron-bro-plugin-kafka right now to turn it into a plugin, and I
> > expect it will take me until end of next week at the latest to have the
> > rest of the work done to move us to 2.5.2, and to pin to a specific
> package
> > version.  At that point I'm comfortable with a release.
> >
> > Jon
> >
> > On Wed, Nov 15, 2017, 18:57 Matt Foley  wrote:
> >
> > > I’ve been listening.  Looks like there are still a number of major
> issues
> > > to be committed first, right?
> > > The discussion on this thread constitutes sufficient engagement, I
> think,
> > > especially given the Subject line :-)
> > > Would the folks working on the 6 issues listed by Nick care to suggest
> a
> > > cut-off date by which they’ll probably have those fixes in?
> > > I’ll be happy to run the release process, if the community so wishes,
> as
> > > soon as those issues are committed.
> > >
> > > I guess I should, pro forma, send the list of commits already in since
> > the
> > > last release.  I’ll do that today.
> > > Also, if anyone wishes to raise a hand and propose additional commits
> are
> > > needed, please do so on this thread.
> > > Thanks,
> > > --Matt
> > >
> > >
> > > On 11/15/17, 2:09 PM, "Casey Stella"  wrote:
> > >
> > > I'd say that if a release is this imminent that we had better
> notify
> > > the
> > > release manager who will make a release announcement, Nick.  Matt,
> > are
> > > you
> > > tuning in to this?
> > >
> > > On Wed, Nov 15, 2017 at 10:04 AM, Nick Allen 
> > > wrote:
> > >
> > > > Hi Guys -
> > > >
> > > > I want to follow-up on this discussion.  It sounds like most
> people
> > > are in
> > > > agreement with the general approach.
> > > >
> > > > A lot of people have been working hard on Metaalerts and
> > > Elasticsearch.  I
> > > > have checked-in with those doing the heavy lifting and have
> > compiled
> > > a more
> > > > detailed plan based on where we are at now.  To the best of my
> > > knowledge
> > > > here is the plan of attack for finishing out this effort.
> > > >
> > > >   (1) First, METRON-1289 needs to go in.  This one was a fairly
> big
> > > effort
> > > > and I am hearing that we are pretty close.
> > > >
> > > >   (2) METRON-1294 fixes an issue in how field types are
> looked-up.
> > > >
> > > >   (3) METRON-1290 is next.  While this may have been fixed in
> > > M-1289, there
> > > > may be some test cases we want from this PR.
> > > >
> > > >   (4) METRON-1301 addresses a problem with the sorting logic.
> > > >
> > > >   (5) METRON-1291 fixes an issue with escalation of metaalerts.
> > > >
> > > >   (6) That leads us to Raghu's UI work in METRON-1252.  This
> > > introduces the
> > > > UI bits that depend on all the previous backend work.
> > > >
> > > >   (7) At this point, we should have our best effort at running
> > > Metaalerts
> > > > on Elasticsearch 2.x. I propose that we cut a release here.
> > > >
> > > >   (8) After we cut the release, we can introduce the work for ES
> > 5.x
> > > in
> > > > METRON-939.  I know we will need lots of help testing and
> reviewing
> > > this
> > > > one.
> > > >
> > > > Please correct me if I am wrong.  I will try

Re: [DISCUSS] Upcoming Release

2017-11-16 Thread Nick Allen

While I think that the Bro work is super valuable, Jon, I am not sure that
we need to block the next release for it.  In my mind, the "theme" of the
next release is Metaalerts running on ES 2.x.  I'd prefer to just stick
with that.

I was hoping we could get the remaining "Metaalerts + ES 2.x" PRs merged
and start cutting release candidates ASAP.  That could be possible by end
of week based on the "big one" (METRON-1289) getting merged in last night.

Of course, if you happen to get all the Bro work done in time, then
definitely let's include it in the next release.  But I am wary of blocking
the release for that work.  No need for you to rush through it.

Just one man's opinion.  Would like to hear feedback from more of the
community.



On Thu, Nov 16, 2017 at 8:01 AM, zeo...@gmail.com  wrote:

> The way master's full-dev is set up right now is non optimal for the bro
> plugin configuration, and I would like to complete the roadmap I outlined
> in my discuss thread before a release.  I have a PR open against
> Apache/metron-bro-plugin-kafka right now to turn it into a plugin, and I
> expect it will take me until end of next week at the latest to have the
> rest of the work done to move us to 2.5.2, and to pin to a specific package
> version.  At that point I'm comfortable with a release.
>
> Jon
>
> On Wed, Nov 15, 2017, 18:57 Matt Foley  wrote:
>
> > I’ve been listening.  Looks like there are still a number of major issues
> > to be committed first, right?
> > The discussion on this thread constitutes sufficient engagement, I think,
> > especially given the Subject line :-)
> > Would the folks working on the 6 issues listed by Nick care to suggest a
> > cut-off date by which they’ll probably have those fixes in?
> > I’ll be happy to run the release process, if the community so wishes, as
> > soon as those issues are committed.
> >
> > I guess I should, pro forma, send the list of commits already in since
> the
> > last release.  I’ll do that today.
> > Also, if anyone wishes to raise a hand and propose additional commits are
> > needed, please do so on this thread.
> > Thanks,
> > --Matt
> >
> >
> > On 11/15/17, 2:09 PM, "Casey Stella"  wrote:
> >
> > I'd say that if a release is this imminent that we had better notify
> > the
> > release manager who will make a release announcement, Nick.  Matt,
> are
> > you
> > tuning in to this?
> >
> > On Wed, Nov 15, 2017 at 10:04 AM, Nick Allen 
> > wrote:
> >
> > > Hi Guys -
> > >
> > > I want to follow-up on this discussion.  It sounds like most people
> > are in
> > > agreement with the general approach.
> > >
> > > A lot of people have been working hard on Metaalerts and
> > Elasticsearch.  I
> > > have checked-in with those doing the heavy lifting and have
> compiled
> > a more
> > > detailed plan based on where we are at now.  To the best of my
> > knowledge
> > > here is the plan of attack for finishing out this effort.
> > >
> > >   (1) First, METRON-1289 needs to go in.  This one was a fairly big
> > effort
> > > and I am hearing that we are pretty close.
> > >
> > >   (2) METRON-1294 fixes an issue in how field types are looked-up.
> > >
> > >   (3) METRON-1290 is next.  While this may have been fixed in
> > M-1289, there
> > > may be some test cases we want from this PR.
> > >
> > >   (4) METRON-1301 addresses a problem with the sorting logic.
> > >
> > >   (5) METRON-1291 fixes an issue with escalation of metaalerts.
> > >
> > >   (6) That leads us to Raghu's UI work in METRON-1252.  This
> > introduces the
> > > UI bits that depend on all the previous backend work.
> > >
> > >   (7) At this point, we should have our best effort at running
> > Metaalerts
> > > on Elasticsearch 2.x. I propose that we cut a release here.
> > >
> > >   (8) After we cut the release, we can introduce the work for ES
> 5.x
> > in
> > > METRON-939.  I know we will need lots of help testing and reviewing
> > this
> > > one.
> > >
> > > Please correct me if I am wrong.  I will try and send out updates
> as
> > we
> > > make progress.
> > >
> > >
> > >
> > >
> > >
> > > On Mon, Nov 6, 2017 at 1:03 PM, zeo...@gmail.com  >
> > wrote:
> > >
> > > > I agree, I think it's very reasonable to move in line with Nick's
> > > > proposal.  I would also suggest that we outline what the target
> > versions
> > > > would be to add in the METRON-777 components, since it has been
> > > functional
> > > > for a very long time but not reviewed and has some really
> rockstar
> > > > improvements.
> > > >
> > > > Jon
> > > >
> > > > On Mon, Nov 6, 2017 at 12:56 PM Otto Fowler <
> > ottobackwa...@gmail.com>
> > > > wrote:
> > > >
> > > > > I think the ES cutover should be the

Re: [DISCUSS] Upcoming Release

2017-11-16 Thread zeo...@gmail.com

My PR is to turn it into a package containing a plugin*

On Thu, Nov 16, 2017, 08:01 zeo...@gmail.com  wrote:

> The way master's full-dev is set up right now is non optimal for the bro
> plugin configuration, and I would like to complete the roadmap I outlined
> in my discuss thread before a release.  I have a PR open against
> Apache/metron-bro-plugin-kafka right now to turn it into a plugin, and I
> expect it will take me until end of next week at the latest to have the
> rest of the work done to move us to 2.5.2, and to pin to a specific package
> version.  At that point I'm comfortable with a release.
>
> Jon
>
> On Wed, Nov 15, 2017, 18:57 Matt Foley  wrote:
>
>> I’ve been listening.  Looks like there are still a number of major issues
>> to be committed first, right?
>> The discussion on this thread constitutes sufficient engagement, I think,
>> especially given the Subject line :-)
>> Would the folks working on the 6 issues listed by Nick care to suggest a
>> cut-off date by which they’ll probably have those fixes in?
>> I’ll be happy to run the release process, if the community so wishes, as
>> soon as those issues are committed.
>>
>> I guess I should, pro forma, send the list of commits already in since
>> the last release.  I’ll do that today.
>> Also, if anyone wishes to raise a hand and propose additional commits are
>> needed, please do so on this thread.
>> Thanks,
>> --Matt
>>
>>
>> On 11/15/17, 2:09 PM, "Casey Stella"  wrote:
>>
>> I'd say that if a release is this imminent that we had better notify
>> the
>> release manager who will make a release announcement, Nick.  Matt,
>> are you
>> tuning in to this?
>>
>> On Wed, Nov 15, 2017 at 10:04 AM, Nick Allen 
>> wrote:
>>
>> > Hi Guys -
>> >
>> > I want to follow-up on this discussion.  It sounds like most people
>> are in
>> > agreement with the general approach.
>> >
>> > A lot of people have been working hard on Metaalerts and
>> Elasticsearch.  I
>> > have checked-in with those doing the heavy lifting and have
>> compiled a more
>> > detailed plan based on where we are at now.  To the best of my
>> knowledge
>> > here is the plan of attack for finishing out this effort.
>> >
>> >   (1) First, METRON-1289 needs to go in.  This one was a fairly big
>> effort
>> > and I am hearing that we are pretty close.
>> >
>> >   (2) METRON-1294 fixes an issue in how field types are looked-up.
>> >
>> >   (3) METRON-1290 is next.  While this may have been fixed in
>> M-1289, there
>> > may be some test cases we want from this PR.
>> >
>> >   (4) METRON-1301 addresses a problem with the sorting logic.
>> >
>> >   (5) METRON-1291 fixes an issue with escalation of metaalerts.
>> >
>> >   (6) That leads us to Raghu's UI work in METRON-1252.  This
>> introduces the
>> > UI bits that depend on all the previous backend work.
>> >
>> >   (7) At this point, we should have our best effort at running
>> Metaalerts
>> > on Elasticsearch 2.x. I propose that we cut a release here.
>> >
>> >   (8) After we cut the release, we can introduce the work for ES
>> 5.x in
>> > METRON-939.  I know we will need lots of help testing and reviewing
>> this
>> > one.
>> >
>> > Please correct me if I am wrong.  I will try and send out updates
>> as we
>> > make progress.
>> >
>> >
>> >
>> >
>> >
>> > On Mon, Nov 6, 2017 at 1:03 PM, zeo...@gmail.com 
>> wrote:
>> >
>> > > I agree, I think it's very reasonable to move in line with Nick's
>> > > proposal.  I would also suggest that we outline what the target
>> versions
>> > > would be to add in the METRON-777 components, since it has been
>> > functional
>> > > for a very long time but not reviewed and has some really rockstar
>> > > improvements.
>> > >
>> > > Jon
>> > >
>> > > On Mon, Nov 6, 2017 at 12:56 PM Otto Fowler <
>> ottobackwa...@gmail.com>
>> > > wrote:
>> > >
>> > > > I think the ES cutover should be the start of the 0.5.x series,
>> and we
>> > > > continue on with 0.4.x for the
>> > > > metadata improvements etc.  We could chose to focus 0.5.x’s
>> first
>> > > releases
>> > > > on not only ES but
>> > > > getting a handle on kibana and the mpack situation as well.
>> > > >
>> > > >
>> > > >
>> > > >
>> > > > On November 6, 2017 at 12:48:45, Michael Miklavcic (
>> > > > michael.miklav...@gmail.com) wrote:
>> > > >
>> > > > I agree with your proposal, Nick. I think having a stabilizing
>> release
>> > > > prior to upgrading ES/Kibana makes sense.
>> > > >
>> > > > On Mon, Nov 6, 2017 at 9:16 AM, Nick Allen 
>> wrote:
>> > > >
>> > > > > I would like to start a discussion around upcoming releases.
>> We

Re: [DISCUSS] Upcoming Release

2017-11-16 Thread zeo...@gmail.com

The way master's full-dev is set up right now is non optimal for the bro
plugin configuration, and I would like to complete the roadmap I outlined
in my discuss thread before a release.  I have a PR open against
Apache/metron-bro-plugin-kafka right now to turn it into a plugin, and I
expect it will take me until end of next week at the latest to have the
rest of the work done to move us to 2.5.2, and to pin to a specific package
version.  At that point I'm comfortable with a release.

Jon

On Wed, Nov 15, 2017, 18:57 Matt Foley  wrote:

> I’ve been listening.  Looks like there are still a number of major issues
> to be committed first, right?
> The discussion on this thread constitutes sufficient engagement, I think,
> especially given the Subject line :-)
> Would the folks working on the 6 issues listed by Nick care to suggest a
> cut-off date by which they’ll probably have those fixes in?
> I’ll be happy to run the release process, if the community so wishes, as
> soon as those issues are committed.
>
> I guess I should, pro forma, send the list of commits already in since the
> last release.  I’ll do that today.
> Also, if anyone wishes to raise a hand and propose additional commits are
> needed, please do so on this thread.
> Thanks,
> --Matt
>
>
> On 11/15/17, 2:09 PM, "Casey Stella"  wrote:
>
> I'd say that if a release is this imminent that we had better notify
> the
> release manager who will make a release announcement, Nick.  Matt, are
> you
> tuning in to this?
>
> On Wed, Nov 15, 2017 at 10:04 AM, Nick Allen 
> wrote:
>
> > Hi Guys -
> >
> > I want to follow-up on this discussion.  It sounds like most people
> are in
> > agreement with the general approach.
> >
> > A lot of people have been working hard on Metaalerts and
> Elasticsearch.  I
> > have checked-in with those doing the heavy lifting and have compiled
> a more
> > detailed plan based on where we are at now.  To the best of my
> knowledge
> > here is the plan of attack for finishing out this effort.
> >
> >   (1) First, METRON-1289 needs to go in.  This one was a fairly big
> effort
> > and I am hearing that we are pretty close.
> >
> >   (2) METRON-1294 fixes an issue in how field types are looked-up.
> >
> >   (3) METRON-1290 is next.  While this may have been fixed in
> M-1289, there
> > may be some test cases we want from this PR.
> >
> >   (4) METRON-1301 addresses a problem with the sorting logic.
> >
> >   (5) METRON-1291 fixes an issue with escalation of metaalerts.
> >
> >   (6) That leads us to Raghu's UI work in METRON-1252.  This
> introduces the
> > UI bits that depend on all the previous backend work.
> >
> >   (7) At this point, we should have our best effort at running
> Metaalerts
> > on Elasticsearch 2.x. I propose that we cut a release here.
> >
> >   (8) After we cut the release, we can introduce the work for ES 5.x
> in
> > METRON-939.  I know we will need lots of help testing and reviewing
> this
> > one.
> >
> > Please correct me if I am wrong.  I will try and send out updates as
> we
> > make progress.
> >
> >
> >
> >
> >
> > On Mon, Nov 6, 2017 at 1:03 PM, zeo...@gmail.com 
> wrote:
> >
> > > I agree, I think it's very reasonable to move in line with Nick's
> > > proposal.  I would also suggest that we outline what the target
> versions
> > > would be to add in the METRON-777 components, since it has been
> > functional
> > > for a very long time but not reviewed and has some really rockstar
> > > improvements.
> > >
> > > Jon
> > >
> > > On Mon, Nov 6, 2017 at 12:56 PM Otto Fowler <
> ottobackwa...@gmail.com>
> > > wrote:
> > >
> > > > I think the ES cutover should be the start of the 0.5.x series,
> and we
> > > > continue on with 0.4.x for the
> > > > metadata improvements etc.  We could chose to focus 0.5.x’s first
> > > releases
> > > > on not only ES but
> > > > getting a handle on kibana and the mpack situation as well.
> > > >
> > > >
> > > >
> > > >
> > > > On November 6, 2017 at 12:48:45, Michael Miklavcic (
> > > > michael.miklav...@gmail.com) wrote:
> > > >
> > > > I agree with your proposal, Nick. I think having a stabilizing
> release
> > > > prior to upgrading ES/Kibana makes sense.
> > > >
> > > > On Mon, Nov 6, 2017 at 9:16 AM, Nick Allen 
> wrote:
> > > >
> > > > > I would like to start a discussion around upcoming releases.
> We have
> > a
> > > > > couple separate significant tracks of work that we need to
> reconcile
> > in
> > > > our
> > > > > release schedule.
> > > > >
> > > > > (1) We have had (and have in review) a good number of bug fixes
> > > required
>

Re: [DISCUSS] Upcoming Release

2017-11-15 Thread Matt Foley

I’ve been listening.  Looks like there are still a number of major issues to be 
committed first, right?
The discussion on this thread constitutes sufficient engagement, I think, 
especially given the Subject line :-)
Would the folks working on the 6 issues listed by Nick care to suggest a 
cut-off date by which they’ll probably have those fixes in?
I’ll be happy to run the release process, if the community so wishes, as soon 
as those issues are committed.

I guess I should, pro forma, send the list of commits already in since the last 
release.  I’ll do that today.
Also, if anyone wishes to raise a hand and propose additional commits are 
needed, please do so on this thread.
Thanks,
--Matt


On 11/15/17, 2:09 PM, "Casey Stella"  wrote:

I'd say that if a release is this imminent that we had better notify the
release manager who will make a release announcement, Nick.  Matt, are you
tuning in to this?

On Wed, Nov 15, 2017 at 10:04 AM, Nick Allen  wrote:

> Hi Guys -
>
> I want to follow-up on this discussion.  It sounds like most people are in
> agreement with the general approach.
>
> A lot of people have been working hard on Metaalerts and Elasticsearch.  I
> have checked-in with those doing the heavy lifting and have compiled a 
more
> detailed plan based on where we are at now.  To the best of my knowledge
> here is the plan of attack for finishing out this effort.
>
>   (1) First, METRON-1289 needs to go in.  This one was a fairly big effort
> and I am hearing that we are pretty close.
>
>   (2) METRON-1294 fixes an issue in how field types are looked-up.
>
>   (3) METRON-1290 is next.  While this may have been fixed in M-1289, 
there
> may be some test cases we want from this PR.
>
>   (4) METRON-1301 addresses a problem with the sorting logic.
>
>   (5) METRON-1291 fixes an issue with escalation of metaalerts.
>
>   (6) That leads us to Raghu's UI work in METRON-1252.  This introduces 
the
> UI bits that depend on all the previous backend work.
>
>   (7) At this point, we should have our best effort at running Metaalerts
> on Elasticsearch 2.x. I propose that we cut a release here.
>
>   (8) After we cut the release, we can introduce the work for ES 5.x in
> METRON-939.  I know we will need lots of help testing and reviewing this
> one.
>
> Please correct me if I am wrong.  I will try and send out updates as we
> make progress.
>
>
>
>
>
> On Mon, Nov 6, 2017 at 1:03 PM, zeo...@gmail.com  wrote:
>
> > I agree, I think it's very reasonable to move in line with Nick's
> > proposal.  I would also suggest that we outline what the target versions
> > would be to add in the METRON-777 components, since it has been
> functional
> > for a very long time but not reviewed and has some really rockstar
> > improvements.
> >
> > Jon
> >
> > On Mon, Nov 6, 2017 at 12:56 PM Otto Fowler 
> > wrote:
> >
> > > I think the ES cutover should be the start of the 0.5.x series, and we
> > > continue on with 0.4.x for the
> > > metadata improvements etc.  We could chose to focus 0.5.x’s first
> > releases
> > > on not only ES but
> > > getting a handle on kibana and the mpack situation as well.
> > >
> > >
> > >
> > >
> > > On November 6, 2017 at 12:48:45, Michael Miklavcic (
> > > michael.miklav...@gmail.com) wrote:
> > >
> > > I agree with your proposal, Nick. I think having a stabilizing release
> > > prior to upgrading ES/Kibana makes sense.
> > >
> > > On Mon, Nov 6, 2017 at 9:16 AM, Nick Allen  wrote:
> > >
> > > > I would like to start a discussion around upcoming releases. We have
> a
> > > > couple separate significant tracks of work that we need to reconcile
> in
> > > our
> > > > release schedule.
> > > >
> > > > (1) We have had (and have in review) a good number of bug fixes
> > required
> > > to
> > > > support Metaalerts on the existing Elasticsearch 2.x infrastructure.
> > > >
> > > >
> > > > (2) We also have ongoing work to upgrade our infrastructure to
> > > > Elasticsearch 5.x, which will not be backwards compatible.
> > > >
> > > >
> > > > I would like to see a release that has our best work on ES 2.x 
before
> > we
> > > > migrate to 5.x. I would propose the following.
> > > >
> > > > Release N+1: Introduce Metaalerts running on ES 2.x
> > > >
> > > > Release N+2: Cut-over to ES 5.x
> > > >
> > > >
> > > > (Q) Is it worth cutting a separate release for ES 2.x? Is there a
> > better
> > > > way to handle the cut-over to 5.x?
> > > >
> > >
> > --
> >
> > Jon
> >

Re: [DISCUSS] Upcoming Release

2017-11-15 Thread Casey Stella

I'd say that if a release is this imminent that we had better notify the
release manager who will make a release announcement, Nick.  Matt, are you
tuning in to this?

On Wed, Nov 15, 2017 at 10:04 AM, Nick Allen  wrote:

> Hi Guys -
>
> I want to follow-up on this discussion.  It sounds like most people are in
> agreement with the general approach.
>
> A lot of people have been working hard on Metaalerts and Elasticsearch.  I
> have checked-in with those doing the heavy lifting and have compiled a more
> detailed plan based on where we are at now.  To the best of my knowledge
> here is the plan of attack for finishing out this effort.
>
>   (1) First, METRON-1289 needs to go in.  This one was a fairly big effort
> and I am hearing that we are pretty close.
>
>   (2) METRON-1294 fixes an issue in how field types are looked-up.
>
>   (3) METRON-1290 is next.  While this may have been fixed in M-1289, there
> may be some test cases we want from this PR.
>
>   (4) METRON-1301 addresses a problem with the sorting logic.
>
>   (5) METRON-1291 fixes an issue with escalation of metaalerts.
>
>   (6) That leads us to Raghu's UI work in METRON-1252.  This introduces the
> UI bits that depend on all the previous backend work.
>
>   (7) At this point, we should have our best effort at running Metaalerts
> on Elasticsearch 2.x. I propose that we cut a release here.
>
>   (8) After we cut the release, we can introduce the work for ES 5.x in
> METRON-939.  I know we will need lots of help testing and reviewing this
> one.
>
> Please correct me if I am wrong.  I will try and send out updates as we
> make progress.
>
>
>
>
>
> On Mon, Nov 6, 2017 at 1:03 PM, zeo...@gmail.com  wrote:
>
> > I agree, I think it's very reasonable to move in line with Nick's
> > proposal.  I would also suggest that we outline what the target versions
> > would be to add in the METRON-777 components, since it has been
> functional
> > for a very long time but not reviewed and has some really rockstar
> > improvements.
> >
> > Jon
> >
> > On Mon, Nov 6, 2017 at 12:56 PM Otto Fowler 
> > wrote:
> >
> > > I think the ES cutover should be the start of the 0.5.x series, and we
> > > continue on with 0.4.x for the
> > > metadata improvements etc.  We could chose to focus 0.5.x’s first
> > releases
> > > on not only ES but
> > > getting a handle on kibana and the mpack situation as well.
> > >
> > >
> > >
> > >
> > > On November 6, 2017 at 12:48:45, Michael Miklavcic (
> > > michael.miklav...@gmail.com) wrote:
> > >
> > > I agree with your proposal, Nick. I think having a stabilizing release
> > > prior to upgrading ES/Kibana makes sense.
> > >
> > > On Mon, Nov 6, 2017 at 9:16 AM, Nick Allen  wrote:
> > >
> > > > I would like to start a discussion around upcoming releases. We have
> a
> > > > couple separate significant tracks of work that we need to reconcile
> in
> > > our
> > > > release schedule.
> > > >
> > > > (1) We have had (and have in review) a good number of bug fixes
> > required
> > > to
> > > > support Metaalerts on the existing Elasticsearch 2.x infrastructure.
> > > >
> > > >
> > > > (2) We also have ongoing work to upgrade our infrastructure to
> > > > Elasticsearch 5.x, which will not be backwards compatible.
> > > >
> > > >
> > > > I would like to see a release that has our best work on ES 2.x before
> > we
> > > > migrate to 5.x. I would propose the following.
> > > >
> > > > Release N+1: Introduce Metaalerts running on ES 2.x
> > > >
> > > > Release N+2: Cut-over to ES 5.x
> > > >
> > > >
> > > > (Q) Is it worth cutting a separate release for ES 2.x? Is there a
> > better
> > > > way to handle the cut-over to 5.x?
> > > >
> > >
> > --
> >
> > Jon
> >
>

Re: [DISCUSS] Upcoming Release

2017-11-06 Thread Otto Fowler

When I speak of the mpack situation, I’m speaking of the discussion around
removing them.


On November 6, 2017 at 15:14:59, Michael Miklavcic (
michael.miklav...@gmail.com) wrote:

Kibana and MPack are currently being handled as part of the ES upgrade. I'm
recreating the Kibana dashboards right now. Long story short, there was an
issue upgrading from our pickle file, so I'm recreating it from scratch and
will attempt an alternative approach to storing the dashboard templates.

On Mon, Nov 6, 2017 at 10:56 AM, Otto Fowler 
wrote:

> I think the ES cutover should be the start of the 0.5.x series, and we
> continue on with 0.4.x for the
> metadata improvements etc.  We could chose to focus 0.5.x’s first releases
> on not only ES but
> getting a handle on kibana and the mpack situation as well.
>
>
>
>
> On November 6, 2017 at 12:48:45, Michael Miklavcic (
> michael.miklav...@gmail.com) wrote:
>
> I agree with your proposal, Nick. I think having a stabilizing release
> prior to upgrading ES/Kibana makes sense.
>
> On Mon, Nov 6, 2017 at 9:16 AM, Nick Allen  wrote:
>
> > I would like to start a discussion around upcoming releases. We have a
> > couple separate significant tracks of work that we need to reconcile in
> our
> > release schedule.
> >
> > (1) We have had (and have in review) a good number of bug fixes required
> to
> > support Metaalerts on the existing Elasticsearch 2.x infrastructure.
> >
> >
> > (2) We also have ongoing work to upgrade our infrastructure to
> > Elasticsearch 5.x, which will not be backwards compatible.
> >
> >
> > I would like to see a release that has our best work on ES 2.x before we
> > migrate to 5.x. I would propose the following.
> >
> > Release N+1: Introduce Metaalerts running on ES 2.x
> >
> > Release N+2: Cut-over to ES 5.x
> >
> >
> > (Q) Is it worth cutting a separate release for ES 2.x? Is there a better
> > way to handle the cut-over to 5.x?
> >
>
>

Re: [DISCUSS] Upcoming Release

2017-11-06 Thread Michael Miklavcic

Kibana and MPack are currently being handled as part of the ES upgrade. I'm
recreating the Kibana dashboards right now. Long story short, there was an
issue upgrading from our pickle file, so I'm recreating it from scratch and
will attempt an alternative approach to storing the dashboard templates.

On Mon, Nov 6, 2017 at 10:56 AM, Otto Fowler 
wrote:

> I think the ES cutover should be the start of the 0.5.x series, and we
> continue on with 0.4.x for the
> metadata improvements etc.  We could chose to focus 0.5.x’s first releases
> on not only ES but
> getting a handle on kibana and the mpack situation as well.
>
>
>
>
> On November 6, 2017 at 12:48:45, Michael Miklavcic (
> michael.miklav...@gmail.com) wrote:
>
> I agree with your proposal, Nick. I think having a stabilizing release
> prior to upgrading ES/Kibana makes sense.
>
> On Mon, Nov 6, 2017 at 9:16 AM, Nick Allen  wrote:
>
> > I would like to start a discussion around upcoming releases. We have a
> > couple separate significant tracks of work that we need to reconcile in
> our
> > release schedule.
> >
> > (1) We have had (and have in review) a good number of bug fixes required
> to
> > support Metaalerts on the existing Elasticsearch 2.x infrastructure.
> >
> >
> > (2) We also have ongoing work to upgrade our infrastructure to
> > Elasticsearch 5.x, which will not be backwards compatible.
> >
> >
> > I would like to see a release that has our best work on ES 2.x before we
> > migrate to 5.x. I would propose the following.
> >
> > Release N+1: Introduce Metaalerts running on ES 2.x
> >
> > Release N+2: Cut-over to ES 5.x
> >
> >
> > (Q) Is it worth cutting a separate release for ES 2.x? Is there a better
> > way to handle the cut-over to 5.x?
> >
>
>

Re: [DISCUSS] Upcoming Release

2017-11-06 Thread zeo...@gmail.com

I agree, I think it's very reasonable to move in line with Nick's
proposal.  I would also suggest that we outline what the target versions
would be to add in the METRON-777 components, since it has been functional
for a very long time but not reviewed and has some really rockstar
improvements.

Jon

On Mon, Nov 6, 2017 at 12:56 PM Otto Fowler  wrote:

> I think the ES cutover should be the start of the 0.5.x series, and we
> continue on with 0.4.x for the
> metadata improvements etc.  We could chose to focus 0.5.x’s first releases
> on not only ES but
> getting a handle on kibana and the mpack situation as well.
>
>
>
>
> On November 6, 2017 at 12:48:45, Michael Miklavcic (
> michael.miklav...@gmail.com) wrote:
>
> I agree with your proposal, Nick. I think having a stabilizing release
> prior to upgrading ES/Kibana makes sense.
>
> On Mon, Nov 6, 2017 at 9:16 AM, Nick Allen  wrote:
>
> > I would like to start a discussion around upcoming releases. We have a
> > couple separate significant tracks of work that we need to reconcile in
> our
> > release schedule.
> >
> > (1) We have had (and have in review) a good number of bug fixes required
> to
> > support Metaalerts on the existing Elasticsearch 2.x infrastructure.
> >
> >
> > (2) We also have ongoing work to upgrade our infrastructure to
> > Elasticsearch 5.x, which will not be backwards compatible.
> >
> >
> > I would like to see a release that has our best work on ES 2.x before we
> > migrate to 5.x. I would propose the following.
> >
> > Release N+1: Introduce Metaalerts running on ES 2.x
> >
> > Release N+2: Cut-over to ES 5.x
> >
> >
> > (Q) Is it worth cutting a separate release for ES 2.x? Is there a better
> > way to handle the cut-over to 5.x?
> >
>
-- 

Jon

Re: [DISCUSS] Upcoming Release

2017-11-06 Thread Otto Fowler

I think the ES cutover should be the start of the 0.5.x series, and we
continue on with 0.4.x for the
metadata improvements etc.  We could chose to focus 0.5.x’s first releases
on not only ES but
getting a handle on kibana and the mpack situation as well.

On November 6, 2017 at 12:48:45, Michael Miklavcic (
michael.miklav...@gmail.com) wrote:

I agree with your proposal, Nick. I think having a stabilizing release
prior to upgrading ES/Kibana makes sense.

On Mon, Nov 6, 2017 at 9:16 AM, Nick Allen  wrote:

> I would like to start a discussion around upcoming releases. We have a
> couple separate significant tracks of work that we need to reconcile in
our
> release schedule.
>
> (1) We have had (and have in review) a good number of bug fixes required
to
> support Metaalerts on the existing Elasticsearch 2.x infrastructure.
>
>
> (2) We also have ongoing work to upgrade our infrastructure to
> Elasticsearch 5.x, which will not be backwards compatible.
>
>
> I would like to see a release that has our best work on ES 2.x before we
> migrate to 5.x. I would propose the following.
>
> Release N+1: Introduce Metaalerts running on ES 2.x
>
> Release N+2: Cut-over to ES 5.x
>
>
> (Q) Is it worth cutting a separate release for ES 2.x? Is there a better
> way to handle the cut-over to 5.x?
>

Re: [DISCUSS] Upcoming Release

2017-11-06 Thread Michael Miklavcic

I agree with your proposal, Nick. I think having a stabilizing release
prior to upgrading ES/Kibana makes sense.

On Mon, Nov 6, 2017 at 9:16 AM, Nick Allen  wrote:

> I would like to start a discussion around upcoming releases.  We have a
> couple separate significant tracks of work that we need to reconcile in our
> release schedule.
>
> (1) We have had (and have in review) a good number of bug fixes required to
> support Metaalerts on the existing Elasticsearch 2.x infrastructure.
>
>
> (2) We also have ongoing work to upgrade our infrastructure to
> Elasticsearch 5.x, which will not be backwards compatible.
>
>
> I would like to see a release that has our best work on ES 2.x before we
> migrate to 5.x.  I would propose the following.
>
> Release N+1: Introduce Metaalerts running on ES 2.x
>
> Release N+2: Cut-over to ES 5.x
>
>
> (Q) Is it worth cutting a separate release for ES 2.x?  Is there a better
> way to handle the cut-over to 5.x?
>

[DISCUSS] Upcoming Release

2017-11-06 Thread Nick Allen

I would like to start a discussion around upcoming releases.  We have a
couple separate significant tracks of work that we need to reconcile in our
release schedule.

(1) We have had (and have in review) a good number of bug fixes required to
support Metaalerts on the existing Elasticsearch 2.x infrastructure.


(2) We also have ongoing work to upgrade our infrastructure to
Elasticsearch 5.x, which will not be backwards compatible.


I would like to see a release that has our best work on ES 2.x before we
migrate to 5.x.  I would propose the following.

Release N+1: Introduce Metaalerts running on ES 2.x

Release N+2: Cut-over to ES 5.x


(Q) Is it worth cutting a separate release for ES 2.x?  Is there a better
way to handle the cut-over to 5.x?

45 matches

Mail list logo