Re: [DISCUSS] Upcoming Release

2017-11-15 Thread Casey Stella
I'd say that if a release is this imminent that we had better notify the release manager who will make a release announcement, Nick. Matt, are you tuning in to this? On Wed, Nov 15, 2017 at 10:04 AM, Nick Allen wrote: > Hi Guys - > > I want to follow-up on this discussion. It sounds like most

Re: [DISCUSS] Release Procedure + 'Kafka Plugin for Bro'

2017-11-16 Thread Casey Stella
I'd recommend restarting this thread with this subject and including [MENTORS] in the subject line. At least I don't know the answer to this and I'd want broader visibility so we get more responses. On Thu, Nov 16, 2017 at 9:10 AM, Nick Allen wrote: > The code of the 'Kafka Plugin for Bro' is n

Re: [DISCUSS] e2e test infrastructure

2017-11-29 Thread Casey Stella
Honestly, I'm ok with either the in-memory component approach or the docker approach as long as: - It runs in travis - The infrastructure components are spun up in a way that isolates their classpath - The UI e2e test and the integration tests both use the same infrastructure I hon

Re: DISCUSS: Quick change to parser config

2017-12-04 Thread Casey Stella
So, just chiming in here. It seems to me that we have a problem with extraneous fields in a couple of different ways: * Temporary Variables I think that the problem of temporary variables is one beyond just the parser. What I'd like to see is the Stellar field transformations operate similar to

Re: [DISCUSS] Upcoming Release

2017-12-04 Thread Casey Stella
I would be in favor of a release at this point. On Mon, Dec 4, 2017 at 4:57 PM, Matt Foley wrote: > Hey all, > I see METRON-1252 was resolved over the weekend. Shall I go ahead and > start the process with 0.4.2 release? > Does anyone have any commits they feel strongly should go in before 0.4.

New PMC members

2017-12-07 Thread Casey Stella
The Project Management Committee (PMC) for Apache Impala has invited Otto Fowler, Michael Miklavcic and Justin Leet to become a PMC member and we are pleased to announce that they have accepted. Congratulations and welcome!

Re: New PMC members

2017-12-07 Thread Casey Stella
Well, obviously, I meant Metron instead of Impala. To this point, we should have a wiki page around templates for this, similar to the impala project. :) On Thu, Dec 7, 2017 at 10:06 AM, Casey Stella wrote: > The Project Management Committee (PMC) for Apache Impala has invited Otto >

Re: [DISCUSS] Stellar Documentation Autogeneration

2017-12-14 Thread Casey Stella
chiming in with a +1 on my end too. This would be fantastic. On Thu, Dec 14, 2017 at 2:51 PM, Nick Allen wrote: > +1 I think it is a great idea, Justin and the only way that we'll keep the > docs in-sync with the code. > > > > > > On Thu, Dec 14, 2017 at 2:32 PM Justin Leet wrote: > > > I thin

Re: [DISCUSS] Support Ubuntu Installs in the MPack

2017-12-15 Thread Casey Stella
Nick is right that the ASF does not provide support in an explicit way (i.e. there are no pathways to get *prioritized* support via SLAs, etc.), but it is expected that apache projects provide support via mailing lists and answered by volunteers. Specifically, this is the crux of the "community ov

Re: [DISCUSS] Upcoming Release

2017-12-15 Thread Casey Stella
*METRON-1228Done > Unassigned https://issues.apache.org/jira/browse/METRON-1228 > <https://issues.apache.org/jira/browse/METRON-1228>* > *METRON-1218 To Do Ryan > Merriman https://issues.apache.org/jira/browse/METRON-1218 > <https://issues.

Re: [DISCUSS] Upcoming Release

2017-12-15 Thread Casey Stella
at 2:30 PM, Matt Foley wrote: > Perhaps under “build_utils” we should add a subdirectory for > “release_utils”. > > From: Casey Stella > Date: Friday, December 15, 2017 at 10:50 AM > To: "dev@metron.apache.org" > Cc: Matt Foley > Subject: Re: [DISCUSS] Upcomi

Re: [DISCUSS] Stellar in a Zeppelin Notebook

2017-12-19 Thread Casey Stella
I love it! I wonder if we could get more of the REPL-like experience (i.e. I crave autocomplete ;) if we integrated it with jquery shell like they did with nanook (https://github.com/aeshell/nanook). I know zeppelin lets you integrate with more complex javascript. Regardless, this is awesome, g

Re: [VOTE] Metron Release Candidate 0.4.2-RC2

2017-12-19 Thread Casey Stella
+1 validated via Otto's script * Checksums * Sigs * Build * Full dev validation On Tue, Dec 19, 2017 at 2:45 PM, Nick Allen wrote: > +1 I validated using Otto's great script. > > * Validated the list of changes > * Checksums > * Sigs > * Build > * Tests > * Full Dev > > On Tue, Dec 19, 2017 at

Re: [DISCUSS] Resources for how to contribute to Apache Metron

2017-12-20 Thread Casey Stella
That's really good feedback, Jon. I agree that we have a significant barrier to get to the point of tinkering. Full-dev really wasn't intended to be that entry point; it's more of a way to test PRs in something resembling a realistic scenario (and it is still not super realistic). I would welcom

[DISCUSS] Generating and Interacting with serialized summary objects

2017-12-24 Thread Casey Stella
Hi all, I wanted to get some feedback on a sensible plan for something. It occurred to me the other day when considering the use-case of detecting typosquatted domains, that one approach was to generate the set of typosquatted domains for some set of reference domains and compare domains as they

Re: [DISCUSS] Generating and Interacting with serialized summary objects

2017-12-24 Thread Casey Stella
ve a place and I started with the configuration based approach as it was a more natural extension of what we already had. I'd love to hear thoughts about this idea too. On Sun, Dec 24, 2017 at 8:20 PM, Casey Stella wrote: > Hi all, > > I wanted to get some feedback on a sensible

Re: [DISCUSS] Generating and Interacting with serialized summary objects

2017-12-24 Thread Casey Stella
, 2017 at 8:20 PM, Casey Stella wrote: > Hi all, > > I wanted to get some feedback on a sensible plan for something. It > occurred to me the other day when considering the use-case of detecting > typosquatted domains, that one approach was to generate the set of > typosquatted domai

Re: [DISCUSS] Generating and Interacting with serialized summary objects

2018-01-03 Thread Casey Stella
;https://github.com/apache/metron/pull/879>. This would allow me to > integrate Stellar with whatever external tools that I want. > > $ cat top-1m.csv | awk -F, '{print $2}' | sed '/^$/d' | stellar -i > 'DOMAIN_REMOVE_TLD(_)' | stellar -i 'BLOOM_ADD(_)&#

Re: [DISCUSS] Generating and Interacting with serialized summary objects

2018-01-03 Thread Casey Stella
not understanding something. > > > > > > > On Wed, Jan 3, 2018 at 10:17 AM, Casey Stella wrote: > > > Thanks for the feedback, Nick. > > > > Regarding "IMHO, I'd rather not reinvent the wheel for text > manipulation." > > > > I

Re: [DISCUSS] Generating and Interacting with serialized summary objects

2018-01-04 Thread Casey Stella
park, which again, I am really very very keen on > seeing > >>> happen. That said, not sure how that would happen in this context, > unless > >>> you’re talking about pushing to something like livy for example > (eminently > >>> sensible for th

Re: [DISCUSS] Generating and Interacting with serialized summary objects

2018-01-04 Thread Casey Stella
ilter *for* typosquatting use-case. Hard coding this would prevent things like bloom filters containing malicious IPs from a reference source, for instance. On Thu, Jan 4, 2018 at 10:46 AM, Casey Stella wrote: > So, there is value outside of just bloom usage. The most specific example > of

Re: [DISCUSS] Generating and Interacting with serialized summary objects

2018-01-05 Thread Casey Stella
t; HiveQL, should the need arise. > > > > In summary, my impressions are that at this point the features and level > of > > abstraction feel appropriate to me. I think it buys us 1) learning from a > > starting typosquatting use case, 2) flexibility to change and adap

Re: [DISCUSS] Generating and Interacting with serialized summary objects

2018-01-05 Thread Casey Stella
ine the ‘stellar-object-repo’ would be part of the global > configuration or configuration passed to the command. > why specify in the function itself? > > > > > On January 5, 2018 at 11:22:32, Casey Stella (ceste...@gmail.com) wrote: > > I like that, specifically the reposi

Re: Full Dev -> Heartbeat issues

2018-01-08 Thread Casey Stella
I haven't seen that one. I spun one up from master on Friday and it seemed ok. Sorry, "works for me!" isn't super helpful, but it may be relevant since master is close to 0.4.2 :) On Mon, Jan 8, 2018 at 11:11 AM, Otto Fowler wrote: > I just started up full dev from the 0.4.2 release tag, and e

Anand is a new Committer!

2018-01-11 Thread Casey Stella
The Project Management Committee (PMC) for Apache Metron has invited Anand Subramanian to become a committer and we are pleased to announce that they have accepted. Congratulations and welcome, Anand!

Re: Travis for Apache/Metron is in trouble

2018-01-18 Thread Casey Stella
I made an infra ticket: https://issues.apache.org/jira/browse/INFRA-15865 On Thu, Jan 18, 2018 at 11:42 AM, Otto Fowler wrote: > 24hr long build is blocking up master’s travis build. > Who can nuke it? > > ottO >

Re: Some more upgrade fallout... Can't restart Metron Indexing

2018-01-18 Thread Casey Stella
So, the challenge here is that our install script isn't smart enough right now to skip creating tables that are already created. One thing you could do is 1. rename the hbase tables for metron (see https://stackoverflow.com/questions/27966072/how-do-you-rename-a-table-in-hbase ) 2. l

Re: [DISCUSS] Upgrading Solr

2018-01-18 Thread Casey Stella
+1 to both the feature branch and user@ announcement. On Thu, Jan 18, 2018 at 2:45 PM, Otto Fowler wrote: > +1 to the feature branch. > > Also, there have been some questions about solr support recently, I think > when the feature branch > is ready you should announce it on user@ too, we may get

Re: [DISCUSS] Time to remove github updates from dev?

2018-01-19 Thread Casey Stella
I could get behind that. On Fri, Jan 19, 2018 at 3:31 PM, Andre wrote: > Folks, > > May I suggest Metron follows the NiFi mailing list strategy (we got > inspired by another project but I don't recall the name) and remove the > github comments from the dev list? > > Within NiFi we have both the

Re: Master is failed in Travis

2018-01-22 Thread Casey Stella
This could be one of those intermittent test failures related to timing. Specifically this: test(org.apache.metron.rest.controller.SensorIndexingConfigControllerIntegrationTest) Time elapsed: 0.064 sec <<< FAILURE! java.lang.AssertionError: Status expected:<404> but was:<200> at org.spr

Re: When things change in hdfs, how do we know

2018-01-25 Thread Casey Stella
Right now you have to restart the parser topology. On Thu, Jan 25, 2018 at 10:15 PM, Otto Fowler wrote: > At the moment, when a grok file or something changes in HDFS, how do we > know? Do we have to restart the parser topology to pick it up? > Just trying to clarify for myself. > > ottO >

Re: Metron User Community Meeting Call

2018-01-26 Thread Casey Stella
I can't wait! This is going to be really cool :) On Fri, Jan 26, 2018 at 5:25 PM, James Sirota wrote: > Yeah very interested in the presentation as well > > 26.01.2018, 15:15, "Simon Elliston Ball" : > > This is going to be a really exciting call. Looking forward to seeing > how the GCR Canary

Re: [DISCUSS] Move SHELL type functions from management to stellar common

2018-01-31 Thread Casey Stella
I'd be in favor of that. That is general purpose stuff. On Wed, Jan 31, 2018 at 9:12 AM, Otto Fowler wrote: > Per: https://issues.apache.org/jira/browse/METRON-876 > > I think we should move the shell/console type functions from stellar > management to stellar-common, and guard them with CONSO

Re: [DISCUSS] Move SHELL type functions from management to stellar common

2018-01-31 Thread Casey Stella
I assumed he was talking about the SHELL_EDIT stuff and maybe the file loading bits. The config stuff is metron specific On Wed, Jan 31, 2018 at 10:06 AM, Nick Allen wrote: > > I think we should move the shell/console type functions from stellar > > What functions specifically? Are you talking

Re: When things change in hdfs, how do we know

2018-01-31 Thread Casey Stella
Hmm, I have heard this feedback before. Perhaps a more low-key approach would be either a static timer that checked or a timer bolt that sent a periodic timer and the parser bolt reconfigured the parser (or indeed we added a Reloadable interface with a 'reload' method). We could be smart also and

Re: When things change in hdfs, how do we know

2018-01-31 Thread Casey Stella
and there should be 1 listeners. > > Having 1 listener shouldn’t be an issue. > > > > On January 31, 2018 at 11:45:54, Casey Stella (ceste...@gmail.com) wrote: > > Hmm, I have heard this feedback before. Perhaps a more low-key approach > would be either a static

Re: [DISCUSS] Persistence store for user profile settings

2018-02-01 Thread Casey Stella
So, I'll answer your question with some questions: - No matter the data store we use upgrading will take some care, right? - Do we currently depend on a RDBMS anywhere? I want to say that we do in the REST layer already, right? - If we don't use a RDBMs, what's the other option? What

Re: Apache Website Required Links

2018-02-15 Thread Casey Stella
Just reporting back that Anand's PR METRON-1386 ( https://github.com/apache/metron/pull/935) has been merged into master and the asf-site branch. Kudos to Anand! Casey On Wed, Feb 7, 2018 at 9:11 AM, Anand Subramanian < asubraman...@hortonworks.com> wrote: > I can take a shot at this if there ar

[DISCUSS] Alternatives to split/join enrichment

2018-02-22 Thread Casey Stella
Hi all, I've been thinking and working on something that I wanted to get some feedback on. The way that we do our enrichments, the split/join architecture was created to effectively to parallel enrichments in a storm-like way in contrast to OpenSoc. There are some good parts to this architecture

Re: [DISCUSS] Alternatives to split/join enrichment

2018-02-22 Thread Casey Stella
A couple of questions: > > * how does this effect the distribution of work through the cluster, and > resiliency of the topologies? > * Is anyone else doing it like this? > * Can we have multiple thread pools and group tasks together ( or separate > them ) wrt hbase? >

Re: [DISCUSS] Alternatives to split/join enrichment

2018-02-22 Thread Casey Stella
FYI, the PR for this is up at https://github.com/apache/metron/pull/940 For those interested, please comment on the actual implementation there. On Thu, Feb 22, 2018 at 12:43 PM, Casey Stella wrote: > So, these are good questions, as usual Otto :) > > > how does this effect the dis

Re: [DISCUSS] Time to remove github updates from dev?

2018-03-19 Thread Casey Stella
> > >> 1. Creation of the new "issues" list > >> 2. redirect both GitHub and JIRA integrations to the new list > >> > >> Cheers > >> > >> On Sat, Jan 27, 2018 at 9:40 AM, James Sirota > >> wrote: > >> > >&

Re: [VOTE] Metron Release Candidate 0.4.2-RC2

2018-04-10 Thread Casey Stella
erated RPMs > > * Bro, YAF and snort - ingest into kafka topics and validated indices > > * Add squid telemetry, ingest into kafka topic and validated indices > > * Management UI, Alerts UI and Swagger UI sanity check > > > > +1 (non-binding) > > > > > >

Re: [VOTE] Metron Release Candidate 0.4.2-RC2

2018-04-10 Thread Casey Stella
Nevermind, it's just the internal apache release status wasn't updated. Sorry, I updated it as part of the board report. Let me make sure I update teh docs for releasing. On Tue, Apr 10, 2018 at 10:35 AM Casey Stella wrote: > It seems that 0.4.2 never got released. Is there a re

Re: Another intermittant build failure?

2018-04-11 Thread Casey Stella
I have not personally seen that one yet, but I will not deny that it exists. It could be very intermittent or triggered under load in travis too. Either way, we should probably investigate and fix. On Wed, Apr 11, 2018 at 3:57 PM Otto Fowler wrote: > I had a PR build fail with an issue with th

Re: [DISCUSS] Metron RPM spec changelog

2018-04-18 Thread Casey Stella
I think I'd prefer to see the changelog only include the release entries, rather than individual entries per dev. We keep the spec file in source control to determine the individual changes between releases. I'm happy to have my mind changed, though. On Wed, Apr 18, 2018 at 9:47 AM Michael Mikla

Re: [VOTE] Development Guidelines Addendum on Inactive Pull Requests

2018-04-20 Thread Casey Stella
+1 On Fri, Apr 20, 2018 at 11:17 AM David Lyle wrote: > +1 sounds good to me. > > -D... > > > On Fri, Apr 20, 2018 at 11:09 AM, zeo...@gmail.com > wrote: > > > +1 (non-binding) > > > > On Fri, Apr 20, 2018 at 9:42 AM Michel Sumbul > > wrote: > > > > > +1 > > > > > > 2018-04-20 14:40 GMT+01:00

[DISCUSS] Release?

2018-05-09 Thread Casey Stella
Is it about time for a release? I know we got some substantial performance changes in since the last release. I think we might have a justification for a release. Casey

Re: [DISCUSS] Release?

2018-05-09 Thread Casey Stella
I wasn't aware we had a script for that..is that in dev-utilities/release-utils? On Wed, May 9, 2018 at 11:41 AM Otto Fowler wrote: > Can you run the issues included script and post that for us to see? > > > On May 9, 2018 at 11:14:11, Casey Stella (ceste...@gmail.com) wrote

Re: [DISCUSS] Release?

2018-05-09 Thread Casey Stella
oses apache/metron#859 > 5 months ago METRON-1349 Full Dev Builds Metron Twice (nickwallen) closes > apache/metron#866 > 5 months ago METRON-1343 Swagger UI for User Controller needs request > method (MohanDV via ottobackwards) closes apache/metron#862 > 5 months ago METRON-1306

Re: [DISCUSS] Release?

2018-05-09 Thread Casey Stella
> >> (mmiklavc via mmiklavc) closes apache/metron#908 > > > > > >> 4 months ago METRON-1302: Split up Indexing Topology into batch > > and > > > > > >> random access sections closes apache/incubator-metron#831 > > > > > >>

Re: [DISCUSS] Release?

2018-05-09 Thread Casey Stella
oses apache/metron#888 > > >> > > > > >> 3 months ago METRON-1432 JDK Install Fails on Ubuntu > > >> Development > > >> > > > > >> Environment (nickwallen) closes apache/metron#913 > > >> > > &g

Re: [DISCUSS] Pcap UI user requirements

2018-05-09 Thread Casey Stella
A couple of thoughts on cluster overuse: * Definitely can't pause/resume MR jobs, unfortunately * The traditional approach to managing overuse of cluster resources and prioritization in Yarn is via the scheduler. I'd suggest rather than building this ourselves, we allow users to be associated with

[DISCUSS] Release Manager

2018-05-10 Thread Casey Stella
Hi All, Matt Foley, our esteemed Release manager for the last couple releases, has asked to be relieved. So, I'm calling on volunteers for the next release manager. It should be a committer and there are a few things that require a PMC member, I believe, but the release manager can ask for help

Re: [DISCUSS] Release Manager

2018-05-10 Thread Casey Stella
9:07 AM, Justin Leet > > wrote: > > > > > I'd be happy to to volunteer to take over for a while. > > > > > > Thanks to Matt for all the help through the last couple releases! > > > > > > Justin > > > > > > On Thu, May 10

Re: [DISCUSS] Metron release 0.5.0

2018-05-16 Thread Casey Stella
So, I definitely hear what you're saying Otto, but I think since we had Solr support before (though it didn't really work as well as it is going to), I think we can do the solr work in a point release. I'd also be willing to move to 0.6 directly after Solr comes in if we think its' big enough. Al

Re: Request for Comment on new Syslog 5424 Parsing library

2018-05-18 Thread Casey Stella
Cool! I'd welcome a syslog parser! On Fri, May 18, 2018 at 10:02 AM Otto Fowler wrote: > There have been some issues and talk about they way we parse syslog, and > the deficiencies of our grok and regex based approaches, mainly not > supporting structured data as I recall. > I played around wit

Re: [VOTE] Metron Release Candidate 0.5.0-RC1

2018-05-29 Thread Casey Stella
Just a question, do we need anything new in the Upgrading.md for this release? Any migration that we expect people to do? On Tue, May 29, 2018 at 11:30 AM Nick Allen wrote: > METRON-1544 was just merged into master. > > > On Tue, May 29, 2018 at 2:16 PM, Justin Leet > wrote: > > > I'm going to

Re: [DISCUSS] Refactoring

2018-05-30 Thread Casey Stella
I'm torn on this, honestly. I completely agree that cosmetic refactoring gets in the way of review and the risk can be more than the reward, especially in a subtle bit of code. That being said, I'm a big fan of opportunistically refactoring to generalize or correct faulty assumptions. Often, I ca

Re: [DISCUSS] Refactoring

2018-05-30 Thread Casey Stella
up a vote > thread following with the final proposed changes? > > On Wed, May 30, 2018 at 9:46 AM, Casey Stella wrote: > >> I'm torn on this, honestly. I completely agree that cosmetic refactoring >> gets in the way of review and the risk can be more than the reward

Re: [DISCUSS] Refactoring

2018-05-30 Thread Casey Stella
t; > On May 30, 2018 at 11:53:51, Casey Stella (ceste...@gmail.com) wrote: > > Yep, I think we can, mike. > > Let me start with a emendation: > > "Don’t combine code changes with lots of edits of whitespace, comments, or > code changes specifically for cosmetic refactorin

Re: [DISCUSS] Field conversions

2018-06-04 Thread Casey Stella
Before we construct a super generic solution, can we get an analysis of all the places in the UI where we're hard-coding fields? It seems like pulling the field from the global config is the strategy that we've gone with that could be expanded upon in https://github.com/apache/metron/pull/1010 (th

Re: [DISCUSS] Field conversions

2018-06-05 Thread Casey Stella
Well, on write it is a transformation, on read it's a translation. This is to say that you're providing a mapping on read to translate field names given the index you're using. The other approach that I was considering last night is a field transformation REST call which translates field names th

Re: [DISCUSS] Field conversions

2018-06-05 Thread Casey Stella
nd transforming existing data. > > > > On Tue, Jun 5, 2018 at 8:21 AM, Casey Stella wrote: > > > > > Well, on write it is a transformation, on read it's a translation. > This > > is > > > to say that you're providing a mapping on read to transl

Re: Writing enrichment data directly from NiFi with PutHBaseJSON

2018-06-05 Thread Casey Stella
The problem, as you correctly diagnosed, is the key in HBase. We construct the key very specifically in Metron, so it's unlikely to work out of the box with the NiFi processor unfortunately. The key that we use is formed here in the codebase: https://github.com/cestella/incubator-metron/blob/mast

Re: [DISCUSS] Field conversions

2018-06-05 Thread Casey Stella
ould be nice to have a script that read and > > transformed fields for templates and indices to replace the colons with > > dots in ES. > > > > Simon > > > > On 5 June 2018 at 06:40, Casey Stella wrote: > > > > > +1 to that, Simon. Do we have a sense

Re: Writing enrichment data directly from NiFi with PutHBaseJSON

2018-06-05 Thread Casey Stella
ball.com> wrote: > Do we, the community, think it would be a good idea to create a > PutMetronEnrichment NiFi processor for this use case? It seems a number of > people want to use NiFi to manage and schedule loading of enrichments for > example. > > Simon > > On 5 June

Re: [DISCUSS] Field conversions

2018-06-05 Thread Casey Stella
38 AM Otto Fowler wrote: > It is still our user list and dev list that will have the burden of > talking folks through that. > > > On June 5, 2018 at 09:58:32, Casey Stella (ceste...@gmail.com) wrote: > > To be clear, I'm not even suggesting that we create any tooling here.

Re: Writing enrichment data directly from NiFi with PutHBaseJSON

2018-06-05 Thread Casey Stella
gt; > > Is it worth us creating an nifi-metron-bundle. Happy to kick that off, > > > since I'm half way there. > > > > > > Simon > > > > > > > > > > > > On 5 June 2018 at 08:41, Otto Fowler wrote: > > > > > >

Re: [ANNOUNCE] Apache Metron release 0.5.0

2018-06-08 Thread Casey Stella
Great job all! This was a big release with a lot of good stuff. I especially like the performance improvements :) Casey On Fri, Jun 8, 2018 at 8:54 AM Justin Leet wrote: > Hi All, > > I’m happy to announce the release of Metron 0.5.0! Everyone has put in a > lot of working into improvements,

Re: Writing enrichment data directly from NiFi with PutHBaseJSON

2018-06-13 Thread Casey Stella
owed your instructions in this blog: > >>> https://cwiki.apache.org/confluence/display/METRON/ > >>> 2016/06/16/Metron+Tutorial+-+Fundamentals+Part+6%3A+Streaming+Enrichm > >>> ent > >>> > >>> 4. Create a new "dns" sensor in Metron 5. Use th

Re: Using Java Rest Client instead of Transport Client for Elasticsearch

2018-06-13 Thread Casey Stella
It was my understanding was that ES x-pack only supports the transport client (e.g. https://www.elastic.co/guide/en/x-pack/current/java-clients.html). I think that was a major reason why we chose to go that route. I might be wrong though. On Wed, Jun 13, 2018 at 10:30 AM Ali Nazemian wrote: >

[DISCUSS] Treating null as false in boolean expressions in Stellar

2018-06-16 Thread Casey Stella
Right now, because fields may not exist, users can have an awkward time. For instance, checking for is_alert, you end up having to preface checks with exists(is_alert). For instance, in one of our use-cases: https://github.com/apache/metron/tree/master/use-cases/geographic_login_outliers we use "

Re: [DISCUSS] Treating null as false in boolean expressions in Stellar

2018-06-16 Thread Casey Stella
eturn foo. Thoughts? On Sat, Jun 16, 2018 at 10:17 AM Casey Stella wrote: > Right now, because fields may not exist, users can have an awkward time. > For instance, checking for is_alert, you end up having to preface checks > with exists(is_alert). > > For instance, in one of ou

Re: [DISCUSS] Treating null as false in boolean expressions in Stellar

2018-06-16 Thread Casey Stella
I created a PR for the empty collection falseyness as well: https://github.com/apache/metron/pull/1064 so we can choose either of them if we so desire. On Sat, Jun 16, 2018 at 1:10 PM Casey Stella wrote: > I created a PR for this functionality, in case we decided for it: > https://gith

Re: [DISCUSS] Merging Solr feature branch (METRON-1416) into master

2018-06-21 Thread Casey Stella
I think that we should merge now, but I’m perhaps biased since I did one of the hard merges. I think that since the major outstanding bug is being worked and we are otherwise feature complete, the feature branch did its job and we are ready to merge. On Thu, Jun 21, 2018 at 10:21 Justin Leet wrote

Re: Architectural reason to split in 4 topologies / impact on the kafka ressources

2018-06-22 Thread Casey Stella
Hey Michel, Those are good questions and there were some reasons surrounding that. In fact, historically, we had fewer topologies (e.g. indexing and enrichment were merged). Even earlier on, we had just one giant topology per parser that enriched and indexed. The long story short is that we move

Re: [DISCUSS] Deprecating metron-api

2018-06-29 Thread Casey Stella
I have no objection and would consider it to be a prerequisite to bringing in the PR unless there's someone depending on it out there. You might want to cc user@ as well, to get a broader set of input for the "are people using it?" question. On Fri, Jun 29, 2018 at 5:21 PM Ryan Merriman wrote:

Re: Security Feature Branch?

2018-07-12 Thread Casey Stella
I would support this being a feature branch. It sounds like a valuable but large contribution. On Thu, Jul 12, 2018 at 10:51 AM Simon Elliston Ball < si...@simonellistonball.com> wrote: > I've been doing some work on getting the Metron UIs and REST layers to work > with Apache KnoxSSO, and LDAP

Re: Security Feature Branch?

2018-07-12 Thread Casey Stella
I added the feature branch: feature/METRON-1663-knoxsso https://git-wip-us.apache.org/repos/asf?p=metron.git;a=shortlog;h=refs/heads/feature/METRON-1663-knoxsso On Thu, Jul 12, 2018 at 11:13 AM Otto Fowler wrote: > I think I understand what you are saying very very very well Simon. I am > not

Re: Security Feature Branch?

2018-07-13 Thread Casey Stella
nd Metron wherein possible threat details can be > > communicated > > > to Knox to take action on at authentication/authorization time. > > > Knox could also potentially push interesting events like possible brute > > > force login attempts to Metron. > > >

Good press for Metron!

2018-08-09 Thread Casey Stella
https://www.darkreading.com/endpoint/oh-no-not-another-security-product/a/d-id/1332453

Re: Slack Channel

2018-08-15 Thread Casey Stella
Sadly, it's the ASF slack and I believe it requires an @apache.org email address. On Wed, Aug 15, 2018 at 12:57 PM Simon Elliston Ball < si...@simonellistonball.com> wrote: > Hello dev team, may I please join your slack channel :) >

Re: Slack Channel

2018-08-15 Thread Casey Stella
Sorry Simon, I retract the comment! I didn't realize it was possible, but it is possible to invite. On Wed, Aug 15, 2018 at 1:01 PM Casey Stella wrote: > Sadly, it's the ASF slack and I believe it requires an @apache.org email > address. > > On Wed, Aug 15, 2018 at 12:57

Re: [DISCUSS] Metron Release 0.6.0?

2018-08-15 Thread Casey Stella
+1 to both releases, this is plenty for an 0.6.0 and a 0.2.0 On Wed, Aug 15, 2018 at 11:04 AM Justin Leet wrote: > I just sent a thread about release cadence. Jon, I'd recommend starting a > thread on a 1.0 roadmap. I thought about merging the threads, but I think > that's just going to result

Re: [DISCUSS] Release cadence

2018-08-15 Thread Casey Stella
Strictly selfishly, I'd love for a release to happen quickly enough to have something to announce to the board during the reports. Once every 2 months or when a sufficiently complicated change happens sounds like a sensible cadence. I very much support a "how do we get to 1.0" discussion, maybe a

Re: [DISCUSS] Release cadence

2018-08-15 Thread Casey Stella
27;m seeing which are also > large-ish and will fit nicely into the next cycle (pending completion, of > course): > >1. NiFi Metron parsers >2. Profiler enhancements - bootstrapping, etc. > 3. Knox SSO > > > > On Wed, Aug 15, 2018 at 11:10 AM Casey Stella w

Re: [DISCUSS] Pcap query branch completion

2018-08-16 Thread Casey Stella
I'm +1 on the merge. This is great work and congrats to those who contributed to it! On Thu, Aug 16, 2018 at 8:27 AM Otto Fowler wrote: > Looks good, thanks! > > > On August 15, 2018 at 19:38:12, Ryan Merriman (merrim...@gmail.com) wrote: > > Otto, I believe the items you requested are in the fe

Re: [DISCUSS] Getting to a 1.0 release

2018-08-20 Thread Casey Stella
I completely agree, Mike. Our docs are either very high level or very low level (and possibly stale) and, worse, aren't aimed at the actors that you've stated. I think that the HBase project does a good job of providing coherent and useable documentation in their "HBase Book" (see https://hbase.ap

Re: package.lock changes during build?

2018-08-25 Thread Casey Stella
I have looked into this for other reasons and the guidance that I've seen is to check in package-lock.json into source control. I'll leave this stack overflow thread here: https://stackoverflow.com/questions/44206782/do-i-commit-the-package-lock-json-file-created-by-npm-5 I want to point out that

Re: package.lock changes during build?

2018-08-25 Thread Casey Stella
t; that's unacceptable, imo. > I've glanced at the package file and see carrots littering the > dependencies, which as I understand it means "get me anything later than > this version." I do not think we should be doing that. > > > On Sat, Aug 25, 2018, 9:1

Re: package.lock changes during build?

2018-08-25 Thread Casey Stella
json update whenever we build locally. Coincidentally, I > just > > addressed this by switching to `npm ci` in an open PR of mine because I > > noticed the same happening locally and I was already updating npm > commands > > in the pom.xml. > > > > > https

Re: [DISCUSS] Contributing a General Purpose Regex Parser

2018-08-29 Thread Casey Stella
+1, I look forward to the PR. On Tue, Aug 28, 2018 at 8:37 AM Nick Allen wrote: > I'd love to see a PR for this. I know there are others in the community > looking for something similar. > > On Sun, Aug 26, 2018 at 7:28 PM wrote: > > > Hello, > > > > > > > > We have implemented a general purpo

Re: IRC Channel -> OPS?

2018-08-31 Thread Casey Stella
wait, I'm an op? Coming up in the world! Do we need this still? I'm currently afk, but will get to it tomorrow. On Wed, Aug 29, 2018 at 4:23 PM Otto Fowler wrote: > Damn, I was hoping not. It will never happen now > > > On August 29, 2018 at 15:49:26, zeo...@gmail.com (zeo...@gmail.com) wrot

Re: [DISCUSS] Feature branches post-merge

2018-09-06 Thread Casey Stella
I’d get rid of them. On Thu, Sep 6, 2018 at 13:42 Michael Miklavcic wrote: > What are we doing with feature branches once they're complete and merged > into master? Is our expectation that we'll keep feature branches in > perpetuity, or should we plan to do some house cleaning once they've been >

Re: [DISCUSS] Split apart releases for core Metron and the Bro plugin

2018-09-07 Thread Casey Stella
+1 to defer for this release and complete separation. Good fences make good submodules. ;) On Fri, Sep 7, 2018 at 2:33 PM zeo...@gmail.com wrote: > +1 to defer for this release and +1 to Justin's suggested release/dist > directory breakout and complete separation. > > Jon > > On Fri, Sep 7, 201

Re: [GitHub] metron issue #1188: METRON-1769: Script creation of a release candidate

2018-09-07 Thread Casey Stella
Mike, did you mean to reply to this on the dev list or were you aiming to make this comment on the PR? If you were aiming to make this comment on the PR, then I think you need to go through github's UI. On Fri, Sep 7, 2018 at 1:34 PM Michael Miklavcic < michael.miklav...@gmail.com> wrote: > Yeah

Re: [MENTORS][DISCUSS] LICENSE and NOTICE likely outdated

2018-09-12 Thread Casey Stella
v...@gmail.com> wrote: > I'm not sure I fully understand what is out of date. I know I have > personally modified our licenses a couple times in the past and used an > automated script that, I believe, Casey Stella had created for doing the > check. I even made some improvements

Re: [DISCUSS] Replacing Moment.js with date-fns or native functions

2018-09-26 Thread Casey Stella
I think it's fine. My only concern would be that we aren't accidentally using moment.js somewhere for something that date-fns doesn't do. I suspect whoever picks up the ticket will figure that out pretty quick though. ;) . I'm +1 on the move; you convinced me. On Wed, Sep 26, 2018 at 8:36 AM Tam

Re: [DISCUSS] Knox SSO feature branch review and features

2018-09-27 Thread Casey Stella
I'm coming in late to the game here, but for my mind a feature branch should involve the minimum architectural change to accomplish a given feature. The feature in question is SSO integration. It seems to me that the operative question is can we do the feature without making the OTHER architectura

  1   2   3   >