Re: [DISCUSS] Release?

2018-05-09 Thread Casey Stella
> > > >> 4 months ago METRON-1428: Travis build failing from > metron-config > > > > > >> (mmiklavc via mmiklavc) closes apache/metron#908 > > > > > >> 4 months ago METRON-1302: Split up Indexing Topology into batch > > and > > > &g

Re: [DISCUSS] Release Manager

2018-05-10 Thread Casey Stella
18 at 9:07 AM, Justin Leet <justinjl...@gmail.com> > > wrote: > > > > > I'd be happy to to volunteer to take over for a while. > > > > > > Thanks to Matt for all the help through the last couple releases! > > > > > > Justin > >

[DISCUSS] Release Manager

2018-05-10 Thread Casey Stella
Hi All, Matt Foley, our esteemed Release manager for the last couple releases, has asked to be relieved. So, I'm calling on volunteers for the next release manager. It should be a committer and there are a few things that require a PMC member, I believe, but the release manager can ask for help

Re: [DISCUSS] Treating null as false in boolean expressions in Stellar

2018-06-16 Thread Casey Stella
I created a PR for the empty collection falseyness as well: https://github.com/apache/metron/pull/1064 so we can choose either of them if we so desire. On Sat, Jun 16, 2018 at 1:10 PM Casey Stella wrote: > I created a PR for this functionality, in case we decided for it: > https://gith

Re: Writing enrichment data directly from NiFi with PutHBaseJSON

2018-06-13 Thread Casey Stella
https://cwiki.apache.org/confluence/display/METRON/ > >>> 2016/06/16/Metron+Tutorial+-+Fundamentals+Part+6%3A+Streaming+Enrichm > >>> ent > >>> > >>> 4. Create a new "dns" sensor in Metron 5. Use the CSVParser and > >>> SimpleHbaseEnrichment

Re: [DISCUSS] Refactoring

2018-05-30 Thread Casey Stella
up a vote > thread following with the final proposed changes? > > On Wed, May 30, 2018 at 9:46 AM, Casey Stella wrote: > >> I'm torn on this, honestly. I completely agree that cosmetic refactoring >> gets in the way of review and the risk can be more than the re

Re: [DISCUSS] Refactoring

2018-05-30 Thread Casey Stella
I'm torn on this, honestly. I completely agree that cosmetic refactoring gets in the way of review and the risk can be more than the reward, especially in a subtle bit of code. That being said, I'm a big fan of opportunistically refactoring to generalize or correct faulty assumptions. Often, I

Re: [DISCUSS] Refactoring

2018-05-30 Thread Casey Stella
> On May 30, 2018 at 11:53:51, Casey Stella (ceste...@gmail.com) wrote: > > Yep, I think we can, mike. > > Let me start with a emendation: > > "Don’t combine code changes with lots of edits of whitespace, comments, or > code changes specifically for cosmetic refactoring pur

Re: [VOTE] Metron Release Candidate 0.5.0-RC1

2018-05-29 Thread Casey Stella
Just a question, do we need anything new in the Upgrading.md for this release? Any migration that we expect people to do? On Tue, May 29, 2018 at 11:30 AM Nick Allen wrote: > METRON-1544 was just merged into master. > > > On Tue, May 29, 2018 at 2:16 PM, Justin Leet > wrote: > > > I'm going

Re: [DISCUSS] Field conversions

2018-06-05 Thread Casey Stella
Well, on write it is a transformation, on read it's a translation. This is to say that you're providing a mapping on read to translate field names given the index you're using. The other approach that I was considering last night is a field transformation REST call which translates field names

Re: [DISCUSS] Field conversions

2018-06-05 Thread Casey Stella
hat read and > > transformed fields for templates and indices to replace the colons with > > dots in ES. > > > > Simon > > > > On 5 June 2018 at 06:40, Casey Stella wrote: > > > > > +1 to that, Simon. Do we have a sense of if there are utilities > prov

Re: Writing enrichment data directly from NiFi with PutHBaseJSON

2018-06-05 Thread Casey Stella
The problem, as you correctly diagnosed, is the key in HBase. We construct the key very specifically in Metron, so it's unlikely to work out of the box with the NiFi processor unfortunately. The key that we use is formed here in the codebase:

Re: Writing enrichment data directly from NiFi with PutHBaseJSON

2018-06-05 Thread Casey Stella
com> wrote: > Do we, the community, think it would be a good idea to create a > PutMetronEnrichment NiFi processor for this use case? It seems a number of > people want to use NiFi to manage and schedule loading of enrichments for > example. > > Simon > > On 5 June 2018

Re: [DISCUSS] Field conversions

2018-06-05 Thread Casey Stella
Fowler wrote: > It is still our user list and dev list that will have the burden of > talking folks through that. > > > On June 5, 2018 at 09:58:32, Casey Stella (ceste...@gmail.com) wrote: > > To be clear, I'm not even suggesting that we create any tooling here. I'd > say ju

Re: Writing enrichment data directly from NiFi with PutHBaseJSON

2018-06-05 Thread Casey Stella
s it worth us creating an nifi-metron-bundle. Happy to kick that off, > > > since I'm half way there. > > > > > > Simon > > > > > > > > > > > > On 5 June 2018 at 08:41, Otto Fowler wrote: > > > > > > > We have jiras a

Re: [DISCUSS] Deprecating metron-api

2018-06-29 Thread Casey Stella
I have no objection and would consider it to be a prerequisite to bringing in the PR unless there's someone depending on it out there. You might want to cc user@ as well, to get a broader set of input for the "are people using it?" question. On Fri, Jun 29, 2018 at 5:21 PM Ryan Merriman wrote:

Re: Architectural reason to split in 4 topologies / impact on the kafka ressources

2018-06-22 Thread Casey Stella
Hey Michel, Those are good questions and there were some reasons surrounding that. In fact, historically, we had fewer topologies (e.g. indexing and enrichment were merged). Even earlier on, we had just one giant topology per parser that enriched and indexed. The long story short is that we

Re: [DISCUSS] Generating and Interacting with serialized summary objects

2018-01-05 Thread Casey Stella
features and level > of > > abstraction feel appropriate to me. I think it buys us 1) learning from a > > starting typosquatting use case, 2) flexibility to change and adapt it > > without affecting users, and 3) enough concrete capability to make more > > specific use ca

Re: [DISCUSS] Generating and Interacting with serialized summary objects

2018-01-05 Thread Casey Stella
com> wrote: > I would imagine the ‘stellar-object-repo’ would be part of the global > configuration or configuration passed to the command. > why specify in the function itself? > > > > > On January 5, 2018 at 11:22:32, Casey Stella (ceste...@gmail.com) wrote:

[DISCUSS] Generating and Interacting with serialized summary objects

2017-12-24 Thread Casey Stella
Hi all, I wanted to get some feedback on a sensible plan for something. It occurred to me the other day when considering the use-case of detecting typosquatted domains, that one approach was to generate the set of typosquatted domains for some set of reference domains and compare domains as they

Re: [DISCUSS] Generating and Interacting with serialized summary objects

2017-12-24 Thread Casey Stella
and I started with the configuration based approach as it was a more natural extension of what we already had. I'd love to hear thoughts about this idea too. On Sun, Dec 24, 2017 at 8:20 PM, Casey Stella <ceste...@gmail.com> wrote: > Hi all, > > I wanted to get some feedback on

Re: [DISCUSS] Generating and Interacting with serialized summary objects

2017-12-24 Thread Casey Stella
, 2017 at 8:20 PM, Casey Stella <ceste...@gmail.com> wrote: > Hi all, > > I wanted to get some feedback on a sensible plan for something. It > occurred to me the other day when considering the use-case of detecting > typosquatted domains, that one approach was to generate the

Anand is a new Committer!

2018-01-11 Thread Casey Stella
The Project Management Committee (PMC) for Apache Metron has invited Anand Subramanian to become a committer and we are pleased to announce that they have accepted. Congratulations and welcome, Anand!

Re: [DISCUSS] Generating and Interacting with serialized summary objects

2018-01-04 Thread Casey Stella
am really very very keen on > seeing > >>> happen. That said, not sure how that would happen in this context, > unless > >>> you’re talking about pushing to something like livy for example > (eminently > >>> sensible for things like cross instance caching a

Re: [DISCUSS] Generating and Interacting with serialized summary objects

2018-01-04 Thread Casey Stella
*for* typosquatting use-case. Hard coding this would prevent things like bloom filters containing malicious IPs from a reference source, for instance. On Thu, Jan 4, 2018 at 10:46 AM, Casey Stella <ceste...@gmail.com> wrote: > So, there is value outside of just bloom usage. The most specifi

Re: Full Dev -> Heartbeat issues

2018-01-08 Thread Casey Stella
I haven't seen that one. I spun one up from master on Friday and it seemed ok. Sorry, "works for me!" isn't super helpful, but it may be relevant since master is close to 0.4.2 :) On Mon, Jan 8, 2018 at 11:11 AM, Otto Fowler wrote: > I just started up full dev from

Re: Travis for Apache/Metron is in trouble

2018-01-18 Thread Casey Stella
I made an infra ticket: https://issues.apache.org/jira/browse/INFRA-15865 On Thu, Jan 18, 2018 at 11:42 AM, Otto Fowler wrote: > 24hr long build is blocking up master’s travis build. > Who can nuke it? > > ottO >

Re: Some more upgrade fallout... Can't restart Metron Indexing

2018-01-18 Thread Casey Stella
So, the challenge here is that our install script isn't smart enough right now to skip creating tables that are already created. One thing you could do is 1. rename the hbase tables for metron (see https://stackoverflow.com/questions/27966072/how-do-you-rename-a-table-in-hbase ) 2.

Re: [DISCUSS] Upgrading Solr

2018-01-18 Thread Casey Stella
+1 to both the feature branch and user@ announcement. On Thu, Jan 18, 2018 at 2:45 PM, Otto Fowler wrote: > +1 to the feature branch. > > Also, there have been some questions about solr support recently, I think > when the feature branch > is ready you should announce

Re: [DISCUSS] Time to remove github updates from dev?

2018-01-19 Thread Casey Stella
I could get behind that. On Fri, Jan 19, 2018 at 3:31 PM, Andre wrote: > Folks, > > May I suggest Metron follows the NiFi mailing list strategy (we got > inspired by another project but I don't recall the name) and remove the > github comments from the dev list? > > Within

Re: Metron User Community Meeting Call

2018-01-26 Thread Casey Stella
I can't wait! This is going to be really cool :) On Fri, Jan 26, 2018 at 5:25 PM, James Sirota wrote: > Yeah very interested in the presentation as well > > 26.01.2018, 15:15, "Simon Elliston Ball" : > > This is going to be a really exciting

Re: [DISCUSS] Move SHELL type functions from management to stellar common

2018-01-31 Thread Casey Stella
I'd be in favor of that. That is general purpose stuff. On Wed, Jan 31, 2018 at 9:12 AM, Otto Fowler wrote: > Per: https://issues.apache.org/jira/browse/METRON-876 > > I think we should move the shell/console type functions from stellar > management to stellar-common,

Re: [DISCUSS] Move SHELL type functions from management to stellar common

2018-01-31 Thread Casey Stella
I assumed he was talking about the SHELL_EDIT stuff and maybe the file loading bits. The config stuff is metron specific On Wed, Jan 31, 2018 at 10:06 AM, Nick Allen wrote: > > I think we should move the shell/console type functions from stellar > > What functions

Re: When things change in hdfs, how do we know

2018-01-31 Thread Casey Stella
Hmm, I have heard this feedback before. Perhaps a more low-key approach would be either a static timer that checked or a timer bolt that sent a periodic timer and the parser bolt reconfigured the parser (or indeed we added a Reloadable interface with a 'reload' method). We could be smart also

Re: When things change in hdfs, how do we know

2018-01-31 Thread Casey Stella
is a newish-api, and there should be 1 listeners. > > Having 1 listener shouldn’t be an issue. > > > > On January 31, 2018 at 11:45:54, Casey Stella (ceste...@gmail.com) wrote: > > Hmm, I have heard this feedback before. Perhaps a more low-key approach > would be eit

Re: [DISCUSS] Persistence store for user profile settings

2018-02-01 Thread Casey Stella
So, I'll answer your question with some questions: - No matter the data store we use upgrading will take some care, right? - Do we currently depend on a RDBMS anywhere? I want to say that we do in the REST layer already, right? - If we don't use a RDBMs, what's the other option?

Re: [DISCUSS] Alternatives to split/join enrichment

2018-02-22 Thread Casey Stella
tions: > > * how does this effect the distribution of work through the cluster, and > resiliency of the topologies? > * Is anyone else doing it like this? > * Can we have multiple thread pools and group tasks together ( or separate > them ) wrt hbase? > > > > On February 22,

[DISCUSS] Alternatives to split/join enrichment

2018-02-22 Thread Casey Stella
Hi all, I've been thinking and working on something that I wanted to get some feedback on. The way that we do our enrichments, the split/join architecture was created to effectively to parallel enrichments in a storm-like way in contrast to OpenSoc. There are some good parts to this

Re: [DISCUSS] Alternatives to split/join enrichment

2018-02-22 Thread Casey Stella
FYI, the PR for this is up at https://github.com/apache/metron/pull/940 For those interested, please comment on the actual implementation there. On Thu, Feb 22, 2018 at 12:43 PM, Casey Stella <ceste...@gmail.com> wrote: > So, these are good questions, as usual Otto :) > > > ho

Re: Apache Website Required Links

2018-02-15 Thread Casey Stella
Just reporting back that Anand's PR METRON-1386 ( https://github.com/apache/metron/pull/935) has been merged into master and the asf-site branch. Kudos to Anand! Casey On Wed, Feb 7, 2018 at 9:11 AM, Anand Subramanian < asubraman...@hortonworks.com> wrote: > I can take a shot at this if there

Re: [DISCUSS] Generating and Interacting with serialized summary objects

2018-01-03 Thread Casey Stella
would allow me to > integrate Stellar with whatever external tools that I want. > > $ cat top-1m.csv | awk -F, '{print $2}' | sed '/^$/d' | stellar -i > 'DOMAIN_REMOVE_TLD(_)' | stellar -i 'BLOOM_ADD(_)' > > > > > > > > > On Sun, Dec 24, 2017 at 8:28 PM, Casey

Re: [DISCUSS] Generating and Interacting with serialized summary objects

2018-01-03 Thread Casey Stella
really high, unless I am not understanding something. > > > > > > > On Wed, Jan 3, 2018 at 10:17 AM, Casey Stella <ceste...@gmail.com> wrote: > > > Thanks for the feedback, Nick. > > > > Regarding "IMHO, I'd rather not reinvent the wheel for text > man

Re: [DISCUSS] Merging Solr feature branch (METRON-1416) into master

2018-06-21 Thread Casey Stella
I think that we should merge now, but I’m perhaps biased since I did one of the hard merges. I think that since the major outstanding bug is being worked and we are otherwise feature complete, the feature branch did its job and we are ready to merge. On Thu, Jun 21, 2018 at 10:21 Justin Leet

Re: [DISCUSS] Metron Release 0.6.0?

2018-08-15 Thread Casey Stella
+1 to both releases, this is plenty for an 0.6.0 and a 0.2.0 On Wed, Aug 15, 2018 at 11:04 AM Justin Leet wrote: > I just sent a thread about release cadence. Jon, I'd recommend starting a > thread on a 1.0 roadmap. I thought about merging the threads, but I think > that's just going to result

Re: [DISCUSS] Release cadence

2018-08-15 Thread Casey Stella
which are also > large-ish and will fit nicely into the next cycle (pending completion, of > course): > >1. NiFi Metron parsers >2. Profiler enhancements - bootstrapping, etc. >3. Knox SSO > > > > On Wed, Aug 15, 2018 at 11:10 AM Casey Stella wrote: > >

Re: [DISCUSS] Release cadence

2018-08-15 Thread Casey Stella
Strictly selfishly, I'd love for a release to happen quickly enough to have something to announce to the board during the reports. Once every 2 months or when a sufficiently complicated change happens sounds like a sensible cadence. I very much support a "how do we get to 1.0" discussion, maybe

Re: Slack Channel

2018-08-15 Thread Casey Stella
Sorry Simon, I retract the comment! I didn't realize it was possible, but it is possible to invite. On Wed, Aug 15, 2018 at 1:01 PM Casey Stella wrote: > Sadly, it's the ASF slack and I believe it requires an @apache.org email > address. > > On Wed, Aug 15, 2018 at 12:57 PM Simon E

Re: [DISCUSS] Pcap query branch completion

2018-08-16 Thread Casey Stella
I'm +1 on the merge. This is great work and congrats to those who contributed to it! On Thu, Aug 16, 2018 at 8:27 AM Otto Fowler wrote: > Looks good, thanks! > > > On August 15, 2018 at 19:38:12, Ryan Merriman (merrim...@gmail.com) wrote: > > Otto, I believe the items you requested are in the

Re: package.lock changes during build?

2018-08-25 Thread Casey Stella
eptable, imo. > I've glanced at the package file and see carrots littering the > dependencies, which as I understand it means "get me anything later than > this version." I do not think we should be doing that. > > > On Sat, Aug 25, 2018, 9:14 AM Casey Stella

Re: package.lock changes during build?

2018-08-25 Thread Casey Stella
I have looked into this for other reasons and the guidance that I've seen is to check in package-lock.json into source control. I'll leave this stack overflow thread here: https://stackoverflow.com/questions/44206782/do-i-commit-the-package-lock-json-file-created-by-npm-5 I want to point out

Re: package.lock changes during build?

2018-08-25 Thread Casey Stella
never we build locally. Coincidentally, I > just > > addressed this by switching to `npm ci` in an open PR of mine because I > > noticed the same happening locally and I was already updating npm > commands > > in the pom.xml. > > > > > https://github.com/a

Re: [DISCUSS] Getting to a 1.0 release

2018-08-20 Thread Casey Stella
I completely agree, Mike. Our docs are either very high level or very low level (and possibly stale) and, worse, aren't aimed at the actors that you've stated. I think that the HBase project does a good job of providing coherent and useable documentation in their "HBase Book" (see

Re: [DISCUSS] Contributing a General Purpose Regex Parser

2018-08-29 Thread Casey Stella
+1, I look forward to the PR. On Tue, Aug 28, 2018 at 8:37 AM Nick Allen wrote: > I'd love to see a PR for this. I know there are others in the community > looking for something similar. > > On Sun, Aug 26, 2018 at 7:28 PM wrote: > > > Hello, > > > > > > > > We have implemented a general

Re: [DISCUSS] Feature branches post-merge

2018-09-06 Thread Casey Stella
I’d get rid of them. On Thu, Sep 6, 2018 at 13:42 Michael Miklavcic wrote: > What are we doing with feature branches once they're complete and merged > into master? Is our expectation that we'll keep feature branches in > perpetuity, or should we plan to do some house cleaning once they've been

Re: IRC Channel -> OPS?

2018-08-31 Thread Casey Stella
wait, I'm an op? Coming up in the world! Do we need this still? I'm currently afk, but will get to it tomorrow. On Wed, Aug 29, 2018 at 4:23 PM Otto Fowler wrote: > Damn, I was hoping not. It will never happen now > > > On August 29, 2018 at 15:49:26, zeo...@gmail.com (zeo...@gmail.com)

Re: [GitHub] metron issue #1188: METRON-1769: Script creation of a release candidate

2018-09-07 Thread Casey Stella
Mike, did you mean to reply to this on the dev list or were you aiming to make this comment on the PR? If you were aiming to make this comment on the PR, then I think you need to go through github's UI. On Fri, Sep 7, 2018 at 1:34 PM Michael Miklavcic < michael.miklav...@gmail.com> wrote: >

Re: Security Feature Branch?

2018-07-12 Thread Casey Stella
I would support this being a feature branch. It sounds like a valuable but large contribution. On Thu, Jul 12, 2018 at 10:51 AM Simon Elliston Ball < si...@simonellistonball.com> wrote: > I've been doing some work on getting the Metron UIs and REST layers to work > with Apache KnoxSSO, and LDAP

Re: Security Feature Branch?

2018-07-12 Thread Casey Stella
I added the feature branch: feature/METRON-1663-knoxsso https://git-wip-us.apache.org/repos/asf?p=metron.git;a=shortlog;h=refs/heads/feature/METRON-1663-knoxsso On Thu, Jul 12, 2018 at 11:13 AM Otto Fowler wrote: > I think I understand what you are saying very very very well Simon. I am > not

Re: Master is failed in Travis

2018-01-22 Thread Casey Stella
This could be one of those intermittent test failures related to timing. Specifically this: test(org.apache.metron.rest.controller.SensorIndexingConfigControllerIntegrationTest) Time elapsed: 0.064 sec <<< FAILURE! java.lang.AssertionError: Status expected:<404> but was:<200> at

Re: [VOTE] Metron Release Candidate 0.4.2-RC2

2018-04-10 Thread Casey Stella
k cluster using the > > generated RPMs > > * Bro, YAF and snort - ingest into kafka topics and validated indices > > * Add squid telemetry, ingest into kafka topic and validated indices > > * Management UI, Alerts UI and Swagger UI sanity check > > > > +1 (non-binding)

Re: [VOTE] Metron Release Candidate 0.4.2-RC2

2018-04-10 Thread Casey Stella
Nevermind, it's just the internal apache release status wasn't updated. Sorry, I updated it as part of the board report. Let me make sure I update teh docs for releasing. On Tue, Apr 10, 2018 at 10:35 AM Casey Stella <ceste...@gmail.com> wrote: > It seems that 0.4.2 never got

Re: Another intermittant build failure?

2018-04-11 Thread Casey Stella
I have not personally seen that one yet, but I will not deny that it exists. It could be very intermittent or triggered under load in travis too. Either way, we should probably investigate and fix. On Wed, Apr 11, 2018 at 3:57 PM Otto Fowler wrote: > I had a PR build

Re: [DISCUSS] Metron RPM spec changelog

2018-04-18 Thread Casey Stella
I think I'd prefer to see the changelog only include the release entries, rather than individual entries per dev. We keep the spec file in source control to determine the individual changes between releases. I'm happy to have my mind changed, though. On Wed, Apr 18, 2018 at 9:47 AM Michael

Re: [VOTE] Development Guidelines Addendum on Inactive Pull Requests

2018-04-20 Thread Casey Stella
+1 On Fri, Apr 20, 2018 at 11:17 AM David Lyle wrote: > +1 sounds good to me. > > -D... > > > On Fri, Apr 20, 2018 at 11:09 AM, zeo...@gmail.com > wrote: > > > +1 (non-binding) > > > > On Fri, Apr 20, 2018 at 9:42 AM Michel Sumbul

Re: [DISCUSS] Time to remove github updates from dev?

2018-03-19 Thread Casey Stella
get this actioned. > >> > >> They would need to assist with: > >> > >> 1. Creation of the new "issues" list > >> 2. redirect both GitHub and JIRA integrations to the new list > >> > >> Cheers > >> > >> On Sat, Jan 27, 201

Re: [DISCUSS] Knox SSO feature branch review and features

2018-09-27 Thread Casey Stella
I'm coming in late to the game here, but for my mind a feature branch should involve the minimum architectural change to accomplish a given feature. The feature in question is SSO integration. It seems to me that the operative question is can we do the feature without making the OTHER

Re: [DISCUSS] Slack Channel Use

2018-10-22 Thread Casey Stella
I am of 2 minds, but I tend to agree. On the one hand, it's definitely the preference that we use the mailing lists for the reasons you stated (and also because not everyone has access to slack generally). On the other hand, I think an interactive medium like Slack has a lot of advantages in

Re: [DISCUSS] Slack Channel Use

2018-10-22 Thread Casey Stella
t; > > > > > On Mon, Oct 22, 2018 at 10:51 AM Justin Leet > > > wrote: > > > > > >> If we want to push more discussion to the dev list, my obvious follow > up > > >> question then is "What are we hoping to get out of Slack/irc/other

Re: [DISCUSS] Stellar REST client

2018-10-19 Thread Casey Stella
I think it makes a lot of sense. A couple of questions: - What actions do you see the REST verbs corresponding to? I would understand GET (which is in effect "evaluate an expression", right?), but I'm not sure about the others. - We should probably be careful about caching stellar

Re: [DISCUSS] Slack Channel Use

2018-10-24 Thread Casey Stella
t; thought > > Metron is sort of dead just because the mailing list is not so active > > anymore! > > > > Cheers, > > Ali > > > > On Tue, Oct 23, 2018 at 8:23 AM Casey Stella wrote: > > > > > Agreed, the benefit of the mailing list is that it’s searchable

Re: [DISCUSS] Slack Channel Use

2018-10-24 Thread Casey Stella
quick clarification, I said "a lot of dev discussion happens on github and JIRA". I want to make sure I didn't mean to imply that larger decisions were being made outside of the appropriate place, the dev list. On Wed, Oct 24, 2018 at 10:08 AM Casey Stella wrote: > No

Re: [DISCUSS] Deprecate split-join enrichment topology in favor of unified enrichment topology

2018-11-01 Thread Casey Stella
+1 On Thu, Nov 1, 2018 at 18:34 Nick Allen wrote: > +1 > > On Thu, Nov 1, 2018, 6:27 PM Justin Leet wrote: > > > +1, I haven't seen any case where the split-join topology isn't made > > obsolete by the unified topology. > > > > On Thu, Nov 1, 2018 at 6:17 PM Michael Miklavcic < > >

Re: [DISCUSS] Attribution and merging the Elasticsearch client migration

2018-11-15 Thread Casey Stella
for Git to handle without > requiring redoing merges and another full round of testing. I'd much prefer > to avoid that in this instance. > > This PR is ready to be merged into master. It's recent and very close to > fully up to date in the branch. Latest master merges cleanly. There is an >

Re: [DISCUSS] Add e2e step to PR checklist

2018-10-05 Thread Casey Stella
gt;> so >> > > people shouldn't get failing tests when they are done running. I just >> > meant >> > > that we often re-run flaky tests with protractor-flake, so it can >> take a >> > > while to run and could increase the build time considerably.

Re: [DISCUSS] Feature Branch guidance

2018-09-29 Thread Casey Stella
ed along the way and formalize them as Casey is recommending > > in our bylaws. I'll be following up with more specific thoughts on > > language. > > > > Best, > > Mike > > > > > > On Fri, Sep 28, 2018 at 10:13 AM Justin Leet > > wrote: > > &g

Re: [DISCUSS] Add e2e step to PR checklist

2018-10-01 Thread Casey Stella
doesn't have much or any knowledge of the UI be able to run that without assistance? For instance, we use full-dev, do we need to stop data from being played into full-dev for the tests to work? Casey On Mon, Oct 1, 2018 at 8:29 AM Casey Stella wrote: > I'm not super keen on expanding the st

Re: [DISCUSS] Add e2e step to PR checklist

2018-10-01 Thread Casey Stella
I'm not super keen on expanding the steps to contribute, especially in an avenue that should be automated. That being said, I think that until we get to the point of automating the e2e tests, it's sensible to add them to the checklist. So, I would support it, but I would also urge us to move

Re: [DISCUSS] Replacing Moment.js with date-fns or native functions

2018-09-26 Thread Casey Stella
I think it's fine. My only concern would be that we aren't accidentally using moment.js somewhere for something that date-fns doesn't do. I suspect whoever picks up the ticket will figure that out pretty quick though. ;) . I'm +1 on the move; you convinced me. On Wed, Sep 26, 2018 at 8:36 AM

[DISCUSS] Feature Branch guidance

2018-09-28 Thread Casey Stella
Hi All, Given discussions about the current high-profile feature branch (Knox SSO), I thought it might be appropriate to have a conversation about what constitutes a feature branch and get some of this encoded in the community guidelines. Specifically, there was the request made that we split up

Re: [DISCUSS] Feature Branch guidance

2018-09-28 Thread Casey Stella
I just noticed this, but googling "metron bylaws" yields http://metron.apache.org/develop/bylaws.html which is not our bylaws. Our bylaws are on https://cwiki.apache.org/confluence/display/METRON/Apache+Metron+Bylaws We should fix that. On Fri, Sep 28, 2018 at 12:02 PM Casey Ste

Re: [ANNOUNCE] Apache Metron release 0.7.0

2018-12-17 Thread Casey Stella
+1 to that!! On Mon, Dec 17, 2018 at 13:16 Michael Miklavcic wrote: > And a big thanks to Justin Leet for being our release manager. Great work > Justin! > > On Mon, Dec 17, 2018 at 10:07 AM Justin Leet wrote: > >> Hi all, >> >> I’m pleased to announce the release of Metron 0.7.0! There's been

Re: [DISCUSS] Metron documentation improvements

2018-12-20 Thread Casey Stella
I definitely agree with option 3; that's a no-brainer IMO. I thought for sure this was already happening, honestly. As for 2, we could even script the broken link check by: - Serving up the site locally via python with `python -m http.server` from the site-book output directory -

Re: [DISCUSS] Metron documentation improvements

2018-12-20 Thread Casey Stella
On Thu, Dec 20, 2018 at 1:08 PM Casey Stella wrote: > > > I definitely agree with option 3; that's a no-brainer IMO. I thought for > > sure this was already happening, honestly. > > > > As for 2, we could even script the broken link check by: > > > >-

Re: [DISCUSS] Handling dropped messages in REGEX_SELECT with Kafka topic routing

2018-12-19 Thread Casey Stella
We absolutely should be acking the dropped messages otherwise they'll be in a replay loop. Not acking is a flat-out bug IMO. On Wed, Dec 19, 2018 at 2:37 PM Michael Miklavcic < michael.miklav...@gmail.com> wrote: > When a message is filtered by the message filtering mechanism, we > explicitly

Re: [DISCUSS] Managing intermittent test failures

2018-11-29 Thread Casey Stella
+1, I'd say mention it on the dev list and slack channel. On Thu, Nov 29, 2018 at 10:26 AM Michael Miklavcic < michael.miklav...@gmail.com> wrote: > Every now and then we see intermittent test failures, and rather than > sweeping them under the rug, we should have a simple method to track and >

Re: [ANNOUNCE] Shane Ardell is a committer

2018-11-27 Thread Casey Stella
Congrats Shane! Well deserved! On Tue, Nov 27, 2018 at 7:24 AM Tamás Fodor wrote: > Congratulations Shane!  > > On Mon, Nov 19, 2018 at 5:58 PM Mohan Venkateshaiah < > mvenkatesha...@hortonworks.com> wrote: > > > Congrats Shane !! > > > > Thanks > > Mohan DV > > > > On 11/19/18, 10:26 PM,

Re: [MENTORS][DISCUSS] LICENSE and NOTICE likely outdated

2018-09-12 Thread Casey Stella
gt; wrote: > I'm not sure I fully understand what is out of date. I know I have > personally modified our licenses a couple times in the past and used an > automated script that, I believe, Casey Stella had created for doing the > check. I even made some improvements to it a long ways ba

Re: [DISCUSS] Shaded jar classifiers

2019-06-03 Thread Casey Stella
This looks good to me, honestly. Anything to make the build more understandable and help find classpath issues easier is a good idea IMO. Just curious, did you test that PR in both solr and ES (you added an exclude in the ES portion of the code) and did you spin it up in full-dev (to ensure

Re: [DISCUSS] Metron Release - 0.7.1 next steps

2019-05-02 Thread Casey Stella
FWIW, I'm in favor of 2. I think it's a relatively minor bug and the impact is limited. I do agree that it should be a blocker for 0.8.0 though. On Thu, May 2, 2019 at 9:31 AM Michael Miklavcic < michael.miklav...@gmail.com> wrote: > I am still in favor of option 2. I will volunteer and submit

Re: [DISCUSS] Full-dev role in PR testign

2019-05-03 Thread Casey Stella
I just want to chime in and say I'm STRONGLY in favor of a docker-based approach to testing (I specifically like the JUnit 5 extensions suggestion). I think that forcing a full-dev evaluation for every small PR is a barrier to entry that I'd like to overcome. I also think that this is going to

Re: [DISCUSS] Deprecate Least Recently Used Pruner

2019-08-13 Thread Casey Stella
Ah, that feature. Yes, it never seemed to catch on. It actually wasn't from OpenSOC, but a very early feature of Metron. The use-case was that enrichments may go stale and removing them based on TTL was easy to do, but not ideal. The LeastRecentlyUsedPruner was a MR job which would allow

Dev list/commit statistics

2020-01-10 Thread Casey Stella
Hi all, First off, thank every one of our amazing committers and contributors. You guys are awesome! That being said, we seem to have a decline in development activity recently both in terms of dev list discussions (this has been sorta historic) as well as commits. I saw then when going

Re: [DISCUSS] Next Release - Life After 0.7.1

2020-01-15 Thread Casey Stella
I'd recommend pulling this into a separate thread and tagging the question with [MENTORS]. FWIW, I'm of the opinion that you should just denote in the commit that it was a dependabot contribution, squash like we normally do and not rewrite the user for attribution. dependabot does not appear to

Re: Development Activity has dropped to effectively 0, what should we do?

2020-04-08 Thread Casey Stella
ion were to contribute I don't know that it would be soon > enough or at the volume that is recognized as viable, which is why I ask > the question. > > > On 2020-04-08 15:05:51-07:00 Casey Stella wrote: > > Hi all, > > When composing the board report today, I realized

Development Activity has dropped to effectively 0, what should we do?

2020-04-08 Thread Casey Stella
Hi all, When composing the board report today, I realized that we have effectively had no development in the last quarter on this project. Please be aware that I say this without a shred of blame or judgement (especially so considering I have not contributed in a long time). That being said, I

Re: Metron-2340 - Geo Database

2020-05-20 Thread Casey Stella
Yeah, that Perl utility appears to be GPL, but the thing that it creates need not be GPL, I presume. Am I reading that correctly? On Wed, May 20, 2020 at 21:55 Otto Fowler wrote: > I “believe” that those files apply to the database that they ship, looking > at the perl It doesn’t seem to

Re: [VOTE] Move Apache Metron to the Apache Attic and Dissolve PMC

2020-11-16 Thread Casey Stella
+1 On Mon, Nov 16, 2020 at 09:01 Justin Leet wrote: > Hi all, > > This is a vote thread to retire Metron to the Attic, and dissolve the PMC. > This follows a discussion thread on the dev list ([DISCUSS] Retire Metron > to the Attic > < >

Fwd: [VOTE] Move Apache Metron to the Apache Attic and Dissolve PMC

2020-11-19 Thread Casey Stella
etron" is hereby terminated; and be it further RESOLVED, that the Apache Metron PMC is hereby terminated. Best, Casey Stella VP Apache Metron [1] https://lists.apache.org/thread.html/r81c4b8ee3f4075938883728c2bd5f7f8897d4ef797699ff1a85998be%40%3Cdev.metron.apa

Re: [DISCUSS] Retire Metron to the Attic

2020-11-09 Thread Casey Stella
Hi all, I'm in complete support of this. Given the current level of interest, I believe that we should move this project to the attic. Best, Casey Stella On Mon, Nov 2, 2020 at 7:28 PM Justin Leet wrote: > Hi all, > > I want to start a discussion in the community to consider retiri

<    1   2