Re: [DISCUSS] Internal Metron fields

2018-09-07 Thread Ryan Merriman
Internal means it’s not configurable, doesn’t contain our default separator (dots) and is namespaced with metron. We can definitely improve on DRY but there’s more to it than that. For example, having 2 different versions of this field name (ES and Solr) adds a significant amount of

Re: [DISCUSS] Internal Metron fields

2018-09-07 Thread Michael Miklavcic
Can you elaborate on what you mean by "convert to internal?" From your description, it looks like the challenge is from our violations of DRY when it comes to constants referencing those keys, which would be eliminated by refactoring. On Fri, Sep 7, 2018, 3:50 PM Ryan Merriman wrote: > I

[DISCUSS] Internal Metron fields

2018-09-07 Thread Ryan Merriman
I recently worked on a PR that involved changing the default behavior of the ElasticsearchWriter to store data using field names with the default Metron separator, dots. One of the unfortunate consequences of this is that although dots are allowed in more recent versions of ES, it changes how

Re: [GitHub] metron issue #1188: METRON-1769: Script creation of a release candidate

2018-09-07 Thread Michael Miklavcic
Whoops, yeah. Meant for a GitHub thread, not a email thread On Fri, Sep 7, 2018, 1:19 PM Casey Stella wrote: > Mike, did you mean to reply to this on the dev list or were you aiming to > make this comment on the PR? If you were aiming to make this comment on > the PR, then I think you need to

Re: [GitHub] metron issue #1188: METRON-1769: Script creation of a release candidate

2018-09-07 Thread Casey Stella
Mike, did you mean to reply to this on the dev list or were you aiming to make this comment on the PR? If you were aiming to make this comment on the PR, then I think you need to go through github's UI. On Fri, Sep 7, 2018 at 1:34 PM Michael Miklavcic < michael.miklav...@gmail.com> wrote: >

Re: [DISCUSS] Split apart releases for core Metron and the Bro plugin

2018-09-07 Thread Casey Stella
+1 to defer for this release and complete separation. Good fences make good submodules. ;) On Fri, Sep 7, 2018 at 2:33 PM zeo...@gmail.com wrote: > +1 to defer for this release and +1 to Justin's suggested release/dist > directory breakout and complete separation. > > Jon > > On Fri, Sep 7,

Re: [DISCUSS] Split apart releases for core Metron and the Bro plugin

2018-09-07 Thread zeo...@gmail.com
+1 to defer for this release and +1 to Justin's suggested release/dist directory breakout and complete separation. Jon On Fri, Sep 7, 2018 at 1:43 PM Michael Miklavcic < michael.miklav...@gmail.com> wrote: > +1 to deferring for this release and having the separation like NiFi. Since > we're

Re: [DISCUSS] Split apart releases for core Metron and the Bro plugin

2018-09-07 Thread Michael Miklavcic
+1 to deferring for this release and having the separation like NiFi. Since we're bootstrapping from their process, what are they doing? I would assume we'd want some sort of vote for the plugin version change as well. On Fri, Sep 7, 2018 at 10:15 AM Nick Allen wrote: > +1 for complete

Re: [GitHub] metron issue #1188: METRON-1769: Script creation of a release candidate

2018-09-07 Thread Michael Miklavcic
Yeah, the Angular upgrade was the other bit that came to mind. Shane's PR for the Angular upgrade has the necessary +1's, but @nickwallen you had requested we hold off on that for this release (which I completely agree with). https://github.com/apache/metron/pull/1096 On Fri, Sep 7, 2018 at 10:24

Re: [DISCUSS] Feature branches post-merge

2018-09-07 Thread Michael Miklavcic
Ok, I'm +1 to this as well. Jira and our Git history does anything we need for posterity (aside from those wanting to re-live old feature branch glory days - who am I to judge?), so no need for the extra cruft. I've participated on but not created a FB yet - is it an infra request to deal with the

Re: [DISCUSS] Metron Release 0.6.0?

2018-09-07 Thread Michael Miklavcic
I'm a little rusty on my C++, but to summarize what I've gathered from this thread and linked code: The Bro plugin infrastructure does not support a patch version currently, only major.minor (x.y). But bro-pkg, the bro package installation mechanism, does support it now. Which is unfortunately

Re: [DISCUSS] Feature branches post-merge

2018-09-07 Thread zeo...@gmail.com
Yeah I don't have a good reason to suggest we keep 'em. so +1 to deleting old FBs. Jon On Fri, Sep 7, 2018 at 12:14 PM Nick Allen wrote: > +1 delete old feature branches. > > BTW, there is a branch out there called METRON-113 that we probably need to > clean-up. I'm not sure where that came

Re: [DISCUSS] Split apart releases for core Metron and the Bro plugin

2018-09-07 Thread Nick Allen
+1 for complete separation as you've described. On Fri, Sep 7, 2018 at 11:31 AM Justin Leet wrote: > I would like this to be a complete separation. Complete with separate RCs, > separate call to vote, etc. There's a bit more overhead, but plugin > releases should be rarer and as the release

Re: [DISCUSS] Feature branches post-merge

2018-09-07 Thread Nick Allen
+1 delete old feature branches. BTW, there is a branch out there called METRON-113 that we probably need to clean-up. I'm not sure where that came from or why its still around. Probably a fat-finger from long ago. On Fri, Sep 7, 2018 at 12:00 PM Otto Fowler wrote: > I would drop them. > I’ve

Re: [DISCUSS] Feature branches post-merge

2018-09-07 Thread Otto Fowler
I would drop them. I’ve already clean up FB’s around dead things. On September 6, 2018 at 13:42:55, Michael Miklavcic ( michael.miklav...@gmail.com) wrote: What are we doing with feature branches once they're complete and merged into master? Is our expectation that we'll keep feature branches

Re: [DISCUSS] Split apart releases for core Metron and the Bro plugin

2018-09-07 Thread Justin Leet
I would like this to be a complete separation. Complete with separate RCs, separate call to vote, etc. There's a bit more overhead, but plugin releases should be rarer and as the release infra gets improved and scripted out more, I don't think it'll end up being much more than bundling it

Re: [DISCUSS] Split apart releases for core Metron and the Bro plugin

2018-09-07 Thread Nick Allen
> Other projects, e.g. NiFi , split apart these releases within their dist directories. I prefer the way Nifi organizes it. Definitely seems more logically organized. > If we split them apart, we can make the releases independently. This fixes the problem of

[DISCUSS] Split apart releases for core Metron and the Bro plugin

2018-09-07 Thread Justin Leet
Right now, we tie together our main release and the Bro plugin, as seen in our 0.4.2 release https://archive.apache.org/dist/metron/0.4.2/ and the current RC. Other projects, e.g. NiFi , split apart these releases within their dist directories. In our case this

Re: [VOTE] Metron Release Candidate 0.6.0-RC1

2018-09-07 Thread Nick Allen
I think the change you made was valuable clean-up. I don't think we need to cancel the vote. We can either manually validate the RC or just use the script from your branch to do the validation. I will be testing the RC this morning. On Fri, Sep 7, 2018 at 9:34 AM Justin Leet wrote: > As full

Re: [VOTE] Metron Release Candidate 0.6.0-RC1

2018-09-07 Thread Justin Leet
As full disclosure, there's a slight difference in the naming of the of the RC artifacts for the plugin. Namely, that it actually the RC number. This breaks the verification script as-is. I have an updated version added here https://github.com/apache/metron/pull/1188/files. If we feel it's

[VOTE] Metron Release Candidate 0.6.0-RC1

2018-09-07 Thread Justin Leet
This is a call to vote on releasing Apache Metron 0.6.0 and the Bro-Kafka plugin 0.2.0 Full list of changes in this release: https://dist.apache.org/repos/dist/dev/metron/0.6.0-RC1/CHANGES https://dist.apache.org/repos/dist/dev/metron/0.6.0-RC1/CHANGES.bro-plugin The tags to be voted upon are:

Re: [DISCUSS] Getting to a 1.0 release

2018-09-07 Thread Nick Allen
We should also publish binaries for each release. That might include any of the following... - Publish the jars to a Maven repository - Publish the RPMs and DEBs to a repository - Host a pre-built "Full Dev" in Vagrant Cloud On Wed, Aug 15, 2018 at 2:49 PM zeo...@gmail.com wrote: >