Re: [DISCUSS] Metron Release 0.6.0?

+1 to a 0.6.0 release that includes the Pcap Panel and Solr work. +1 to doing a 0.2.0 release for metron-bro-plugin-kafka. I *think* we need to do the plugin release first, so that the 0.6.0 Metron release will point to plugin 0.2.0. FWIW, here are the changes since the last release. 6 days

Re: [ANNOUNCE] - Apache Metron Slack channel

It's another option with different features. I imagine many people will use both. On Wed, Aug 15, 2018, 9:14 AM Simon Elliston Ball < si...@simonellistonball.com> wrote: > Since this is committers only, would it make more sense to stick to IRC? Or > is exclusivity the idea? > > On 15 August 2018

[DISCUSS] Release cadence

Hi all, In concert with the discuss thread on a potential 0.6.0 release, I'd also like start a discussion about our release cadence. We've generally been pretty relaxed around doing releases, and I'm curious what people's thoughts are on adopting a somewhat more regular schedule. Couple

Re: [ANNOUNCE] - Apache Metron Slack channel

Since this is committers only, would it make more sense to stick to IRC? Or is exclusivity the idea? On 15 August 2018 at 16:09, Nick Allen wrote: > Thanks for the instructions! > > On Wed, Aug 15, 2018 at 10:22 AM, Michael Miklavcic < > michael.miklav...@gmail.com> wrote: > > > The Metron

[ANNOUNCE] - Apache Metron Slack channel

The Metron community has a Slack channel available for communication (similar to the existing IRC channel, only on Slack). To join: 1. Go to slack.com. 2. For organization/group, you'll enter "the-asf" 3. Use your Apache email for your login 4. Click "Channels" and look for #metron

Re: [DISCUSS] Metron Release 0.6.0?

I was actually going to kick out another thread in a little bit around our release schedule, it should be out shortly. Good point on the metron-bro-plugin-kafka. I'm in favor of putting out a 0.2.0 release of it simultaneously. On Wed, Aug 15, 2018 at 9:48 AM zeo...@gmail.com wrote: > I agree

Re: [DISCUSS] Metron Release 0.6.0?

I just sent a thread about release cadence. Jon, I'd recommend starting a thread on a 1.0 roadmap. I thought about merging the threads, but I think that's just going to result in more crosstalk, so I'll let you start that conversation. On Wed, Aug 15, 2018 at 10:37 AM Nick Allen wrote: > +1 to

Re: [ANNOUNCE] - Apache Metron Slack channel

Thanks for the instructions! On Wed, Aug 15, 2018 at 10:22 AM, Michael Miklavcic < michael.miklav...@gmail.com> wrote: > The Metron community has a Slack channel available for communication > (similar to the existing IRC channel, only on Slack). > > To join: > >1. Go to slack.com. >2.

Re: [DISCUSS] Metron Release 0.6.0?

I agree - I would love to see a release not long after the PCAP FB gets into master, and 0.6.0 makes sense to me. I'd also like to see a 0.2 release of metron-bro-plugin-kafka. There is one new commit, and I have a PR open which is waiting on some tests before it's ready to be evaluated/merged.

Re: [DISCUSS] Getting to a 1.0 release

Agreed, should we add TDE by default, and get the ranger policies on by default? That leaves secured in Kafka, which would have to be built into the consumers and producers to encrypt into the on disk Kafka topics. Does that seem necessary to people? It would have performance implications for

[DISCUSS] Metron Release 0.6.0?

Hi all, It's been a little while since the last release, and a couple major items have gone in since then (or are hopefully close to going in!). In particular, I'd personally like to see a release with our Solr work and the close-to-completion

Re: [DISCUSS] Getting to a 1.0 release

+1 to that. That’s more the TDE bit, we would also need Kafka SSL, and the Knox stuff (METRON-1663 adds SSL to all the UI and rest api stuff) Simon > On 15 Aug 2018, at 21:03, Otto Fowler wrote: > > https://issues.apache.org/jira/browse/METRON-106 > At least making sure it is met and closing

Re: [DISCUSS] Getting to a 1.0 release

Well, I look at it like this. The Secure Vault was part of the original metron pitch, and many may have used that as part of their evaluations. “Look, it is going to have a security vault type thing, it is on the roadmap”. Regardless of the implementation, conceptually, security of data at rest

Re: [DISCUSS] Getting to a 1.0 release

https://issues.apache.org/jira/browse/METRON-343 On August 15, 2018 at 15:47:24, Simon Elliston Ball ( si...@simonellistonball.com) wrote: What would you see as secure? I’ve seen people use TDE for the HDFS store, but it’s harder to encrypt storage with solr / es. Something I was thinking of

Re: [DISCUSS] Getting to a 1.0 release

What would you see as secure? I’ve seen people use TDE for the HDFS store, but it’s harder to encrypt storage with solr / es. Something I was thinking of doing to follow up on the Knox Feature was to add Ranger integration for securing and auditing configs, and potentially extending to the

Re: [DISCUSS] Getting to a 1.0 release

That’s going back a way. I always saw that concept as begin about the formats, e.g. Orc, and meta data around it plus the data service api to get at it. I’m all for that too, but think it needs more thought than the ticket captures. Simon > On 15 Aug 2018, at 20:53, Otto Fowler wrote: > >

Re: [DISCUSS] Getting to a 1.0 release

https://issues.apache.org/jira/browse/METRON-106 At least making sure it is met and closing it On August 15, 2018 at 15:53:02, Otto Fowler (ottobackwa...@gmail.com) wrote: https://issues.apache.org/jira/browse/METRON-343 On August 15, 2018 at 15:47:24, Simon Elliston Ball (

Re: [DISCUSS] Metron Release 0.6.0?

+1 to both releases, this is plenty for an 0.6.0 and a 0.2.0 On Wed, Aug 15, 2018 at 11:04 AM Justin Leet wrote: > I just sent a thread about release cadence. Jon, I'd recommend starting a > thread on a 1.0 roadmap. I thought about merging the threads, but I think > that's just going to result

Re: [ANNOUNCE] - Apache Metron Slack channel

+ Metron user list On Wed, Aug 15, 2018 at 10:38 AM Michael Miklavcic < michael.miklav...@gmail.com> wrote: > Turns out we are able to invite folks on an ad-hoc basis. See instructions > here - > https://cwiki.apache.org/confluence/display/METRON/Community+Resources > > > On Wed, Aug 15, 2018 at

Re: [DISCUSS] Release cadence

I'm also a fan of the 2-3 month time frame for releases. And I agree it fits nicely with our board report. That said, I think we should minimally kick off a DISCUSS at least every 2 months per the recommendations above. If it's warranted, great. If not, then we bring it up at a stated later time

Re: [DISCUSS] Release cadence

Works for me, that would be great. On Wed, Aug 15, 2018 at 12:22 PM Casey Stella wrote: > If you like, I can volunteer to kick off a discuss thread when I submit the > board report. > > On Wed, Aug 15, 2018 at 2:21 PM Michael Miklavcic < > michael.miklav...@gmail.com> wrote: > > > I'm also a

Re: [ANNOUNCE] - Apache Metron Slack channel

Could I be invited? On 15-Aug-18 09:48, Michael Miklavcic wrote: > + Metron user list > > On Wed, Aug 15, 2018 at 10:38 AM Michael Miklavcic < > michael.miklav...@gmail.com> wrote: > >> Turns out we are able to invite folks on an ad-hoc basis. See instructions >> here - >>

Re: [DISCUSS] Release cadence

If you like, I can volunteer to kick off a discuss thread when I submit the board report. On Wed, Aug 15, 2018 at 2:21 PM Michael Miklavcic < michael.miklav...@gmail.com> wrote: > I'm also a fan of the 2-3 month time frame for releases. And I agree it > fits nicely with our board report. That

Re: [DISCUSS] Getting to a 1.0 release

Secure storage off the top of my head On August 15, 2018 at 14:49:26, zeo...@gmail.com (zeo...@gmail.com) wrote: So, as has been discussed in a few < https://lists.apache.org/thread.html/0445cd8f94dfb844cd5a23ac3eeca04c9f44c9d8f269c6ef12cb3598@%3Cdev.metron.apache.org%3E> other <

Re: [DISCUSS] Release cadence

Strictly selfishly, I'd love for a release to happen quickly enough to have something to announce to the board during the reports. Once every 2 months or when a sufficiently complicated change happens sounds like a sensible cadence. I very much support a "how do we get to 1.0" discussion, maybe

Re: [DISCUSS] Metron Release 0.6.0?

+1 here as well to the proposed releases. On Wed, Aug 15, 2018 at 11:06 AM Casey Stella wrote: > +1 to both releases, this is plenty for an 0.6.0 and a 0.2.0 > > On Wed, Aug 15, 2018 at 11:04 AM Justin Leet > wrote: > > > I just sent a thread about release cadence. Jon, I'd recommend starting

Re: [ANNOUNCE] - Apache Metron Slack channel

Done On August 15, 2018 at 14:22:45, Vets, Laurens (laur...@daemon.be) wrote: Could I be invited? On 15-Aug-18 09:48, Michael Miklavcic wrote: > + Metron user list > > On Wed, Aug 15, 2018 at 10:38 AM Michael Miklavcic < > michael.miklav...@gmail.com> wrote: > >> Turns out we are able to

[DISCUSS] Getting to a 1.0 release

So, as has been discussed in a few other recent

Re: Slack Channel

Invite sent On Wed, Aug 15, 2018 at 10:57 AM Simon Elliston Ball < si...@simonellistonball.com> wrote: > Hello dev team, may I please join your slack channel :) >

Re: Slack Channel

Sorry Simon, I retract the comment! I didn't realize it was possible, but it is possible to invite. On Wed, Aug 15, 2018 at 1:01 PM Casey Stella wrote: > Sadly, it's the ASF slack and I believe it requires an @apache.org email > address. > > On Wed, Aug 15, 2018 at 12:57 PM Simon Elliston Ball

Re: [DISCUSS] Pcap query branch completion

Otto, I believe the items you requested are in the feature branch now. Is there anything outstanding that we missed? The Jiras for the Pcap feature branch should be up to date: https://issues.apache.org/jira/browse/METRON-1554 On Mon, Aug 13, 2018 at 5:13 PM, Ryan Merriman wrote: > - Date

Re: Change field separator in Metron to make it Hive and ORC friendly

Hi Simon, I think it is a hard trade-off. Even right now without any ability to customise separator/Metron internal field names, Metron users need to put a mapping in place at the integration layer (At least this is what we are doing :) ). Every organisation/user may need to follow different