Re: [DISCUSS] Shaded jar classifiers

2019-06-03 Thread Casey Stella
This looks good to me, honestly. Anything to make the build more understandable and help find classpath issues easier is a good idea IMO. Just curious, did you test that PR in both solr and ES (you added an exclude in the ES portion of the code) and did you spin it up in full-dev (to ensure

Re: [DISCUSS] Full-dev role in PR testign

2019-05-03 Thread Casey Stella
I just want to chime in and say I'm STRONGLY in favor of a docker-based approach to testing (I specifically like the JUnit 5 extensions suggestion). I think that forcing a full-dev evaluation for every small PR is a barrier to entry that I'd like to overcome. I also think that this is going to

Re: [DISCUSS] Metron Release - 0.7.1 next steps

2019-05-02 Thread Casey Stella
FWIW, I'm in favor of 2. I think it's a relatively minor bug and the impact is limited. I do agree that it should be a blocker for 0.8.0 though. On Thu, May 2, 2019 at 9:31 AM Michael Miklavcic < michael.miklav...@gmail.com> wrote: > I am still in favor of option 2. I will volunteer and submit

Re: [DISCUSS] Metron documentation improvements

2018-12-20 Thread Casey Stella
On Thu, Dec 20, 2018 at 1:08 PM Casey Stella wrote: > > > I definitely agree with option 3; that's a no-brainer IMO. I thought for > > sure this was already happening, honestly. > > > > As for 2, we could even script the broken link check by: > > > >-

Re: [DISCUSS] Metron documentation improvements

2018-12-20 Thread Casey Stella
I definitely agree with option 3; that's a no-brainer IMO. I thought for sure this was already happening, honestly. As for 2, we could even script the broken link check by: - Serving up the site locally via python with `python -m http.server` from the site-book output directory -

Re: [DISCUSS] Handling dropped messages in REGEX_SELECT with Kafka topic routing

2018-12-19 Thread Casey Stella
We absolutely should be acking the dropped messages otherwise they'll be in a replay loop. Not acking is a flat-out bug IMO. On Wed, Dec 19, 2018 at 2:37 PM Michael Miklavcic < michael.miklav...@gmail.com> wrote: > When a message is filtered by the message filtering mechanism, we > explicitly

Re: [ANNOUNCE] Apache Metron release 0.7.0

2018-12-17 Thread Casey Stella
+1 to that!! On Mon, Dec 17, 2018 at 13:16 Michael Miklavcic wrote: > And a big thanks to Justin Leet for being our release manager. Great work > Justin! > > On Mon, Dec 17, 2018 at 10:07 AM Justin Leet wrote: > >> Hi all, >> >> I’m pleased to announce the release of Metron 0.7.0! There's been

Re: [DISCUSS] Managing intermittent test failures

2018-11-29 Thread Casey Stella
+1, I'd say mention it on the dev list and slack channel. On Thu, Nov 29, 2018 at 10:26 AM Michael Miklavcic < michael.miklav...@gmail.com> wrote: > Every now and then we see intermittent test failures, and rather than > sweeping them under the rug, we should have a simple method to track and >

Re: [ANNOUNCE] Shane Ardell is a committer

2018-11-27 Thread Casey Stella
Congrats Shane! Well deserved! On Tue, Nov 27, 2018 at 7:24 AM Tamás Fodor wrote: > Congratulations Shane!  > > On Mon, Nov 19, 2018 at 5:58 PM Mohan Venkateshaiah < > mvenkatesha...@hortonworks.com> wrote: > > > Congrats Shane !! > > > > Thanks > > Mohan DV > > > > On 11/19/18, 10:26 PM,

Re: [DISCUSS] Attribution and merging the Elasticsearch client migration

2018-11-15 Thread Casey Stella
for Git to handle without > requiring redoing merges and another full round of testing. I'd much prefer > to avoid that in this instance. > > This PR is ready to be merged into master. It's recent and very close to > fully up to date in the branch. Latest master merges cleanly. There is an >

Re: [DISCUSS] Slack Channel Use

2018-11-12 Thread Casey Stella
ussions > > > like this in Slack should be redirected to the user list. > > > > > > Is this a reasonable way separate our concerns here? > > > > > > On Wed, Oct 24, 2018 at 11:37 AM Michael Miklavcic < > > > michael.miklav...@g

Re: [DISCUSS] Deprecate split-join enrichment topology in favor of unified enrichment topology

2018-11-01 Thread Casey Stella
+1 On Thu, Nov 1, 2018 at 18:34 Nick Allen wrote: > +1 > > On Thu, Nov 1, 2018, 6:27 PM Justin Leet wrote: > > > +1, I haven't seen any case where the split-join topology isn't made > > obsolete by the unified topology. > > > > On Thu, Nov 1, 2018 at 6:17 PM Michael Miklavcic < > >

Re: [DISCUSS] Slack Channel Use

2018-10-24 Thread Casey Stella
quick clarification, I said "a lot of dev discussion happens on github and JIRA". I want to make sure I didn't mean to imply that larger decisions were being made outside of the appropriate place, the dev list. On Wed, Oct 24, 2018 at 10:08 AM Casey Stella wrote: > No

Re: [DISCUSS] Slack Channel Use

2018-10-24 Thread Casey Stella
t; thought > > Metron is sort of dead just because the mailing list is not so active > > anymore! > > > > Cheers, > > Ali > > > > On Tue, Oct 23, 2018 at 8:23 AM Casey Stella wrote: > > > > > Agreed, the benefit of the mailing list is that it’s searchable

Re: [DISCUSS] Slack Channel Use

2018-10-22 Thread Casey Stella
t; > > > > > On Mon, Oct 22, 2018 at 10:51 AM Justin Leet > > > wrote: > > > > > >> If we want to push more discussion to the dev list, my obvious follow > up > > >> question then is "What are we hoping to get out of Slack/irc/other

Re: [DISCUSS] Slack Channel Use

2018-10-22 Thread Casey Stella
I am of 2 minds, but I tend to agree. On the one hand, it's definitely the preference that we use the mailing lists for the reasons you stated (and also because not everyone has access to slack generally). On the other hand, I think an interactive medium like Slack has a lot of advantages in

Re: [DISCUSS] Stellar REST client

2018-10-19 Thread Casey Stella
I think it makes a lot of sense. A couple of questions: - What actions do you see the REST verbs corresponding to? I would understand GET (which is in effect "evaluate an expression", right?), but I'm not sure about the others. - We should probably be careful about caching stellar

Re: [DISCUSS] Add e2e step to PR checklist

2018-10-05 Thread Casey Stella
gt;> so >> > > people shouldn't get failing tests when they are done running. I just >> > meant >> > > that we often re-run flaky tests with protractor-flake, so it can >> take a >> > > while to run and could increase the build time considerably.

Re: [DISCUSS] Add e2e step to PR checklist

2018-10-02 Thread Casey Stella
uce locally. In addition, our current Protractor tests are > brittle and extremely slow. > > All that said, it seems we agree that we could add another PR checklist > item in the meantime. Clarifying those e2e test instructions should be part > of that task. > > On Mon, Oct 1, 2

Re: [DISCUSS] Add e2e step to PR checklist

2018-10-01 Thread Casey Stella
doesn't have much or any knowledge of the UI be able to run that without assistance? For instance, we use full-dev, do we need to stop data from being played into full-dev for the tests to work? Casey On Mon, Oct 1, 2018 at 8:29 AM Casey Stella wrote: > I'm not super keen on expanding the st

Re: [DISCUSS] Add e2e step to PR checklist

2018-10-01 Thread Casey Stella
I'm not super keen on expanding the steps to contribute, especially in an avenue that should be automated. That being said, I think that until we get to the point of automating the e2e tests, it's sensible to add them to the checklist. So, I would support it, but I would also urge us to move

Re: [DISCUSS] Feature Branch guidance

2018-09-29 Thread Casey Stella
ed along the way and formalize them as Casey is recommending > > in our bylaws. I'll be following up with more specific thoughts on > > language. > > > > Best, > > Mike > > > > > > On Fri, Sep 28, 2018 at 10:13 AM Justin Leet > > wrote: > > &g

Re: [DISCUSS] Feature Branch guidance

2018-09-28 Thread Casey Stella
I just noticed this, but googling "metron bylaws" yields http://metron.apache.org/develop/bylaws.html which is not our bylaws. Our bylaws are on https://cwiki.apache.org/confluence/display/METRON/Apache+Metron+Bylaws We should fix that. On Fri, Sep 28, 2018 at 12:02 PM Casey Ste

[DISCUSS] Feature Branch guidance

2018-09-28 Thread Casey Stella
Hi All, Given discussions about the current high-profile feature branch (Knox SSO), I thought it might be appropriate to have a conversation about what constitutes a feature branch and get some of this encoded in the community guidelines. Specifically, there was the request made that we split up

Re: [DISCUSS] Knox SSO feature branch review and features

2018-09-27 Thread Casey Stella
I'm coming in late to the game here, but for my mind a feature branch should involve the minimum architectural change to accomplish a given feature. The feature in question is SSO integration. It seems to me that the operative question is can we do the feature without making the OTHER

Re: [DISCUSS] Replacing Moment.js with date-fns or native functions

2018-09-26 Thread Casey Stella
I think it's fine. My only concern would be that we aren't accidentally using moment.js somewhere for something that date-fns doesn't do. I suspect whoever picks up the ticket will figure that out pretty quick though. ;) . I'm +1 on the move; you convinced me. On Wed, Sep 26, 2018 at 8:36 AM

Re: [MENTORS][DISCUSS] LICENSE and NOTICE likely outdated

2018-09-12 Thread Casey Stella
gt; wrote: > I'm not sure I fully understand what is out of date. I know I have > personally modified our licenses a couple times in the past and used an > automated script that, I believe, Casey Stella had created for doing the > check. I even made some improvements to it a long ways ba

Re: [GitHub] metron issue #1188: METRON-1769: Script creation of a release candidate

2018-09-07 Thread Casey Stella
Mike, did you mean to reply to this on the dev list or were you aiming to make this comment on the PR? If you were aiming to make this comment on the PR, then I think you need to go through github's UI. On Fri, Sep 7, 2018 at 1:34 PM Michael Miklavcic < michael.miklav...@gmail.com> wrote: >

Re: [DISCUSS] Split apart releases for core Metron and the Bro plugin

2018-09-07 Thread Casey Stella
+1 to defer for this release and complete separation. Good fences make good submodules. ;) On Fri, Sep 7, 2018 at 2:33 PM zeo...@gmail.com wrote: > +1 to defer for this release and +1 to Justin's suggested release/dist > directory breakout and complete separation. > > Jon > > On Fri, Sep 7,

Re: [DISCUSS] Feature branches post-merge

2018-09-06 Thread Casey Stella
I’d get rid of them. On Thu, Sep 6, 2018 at 13:42 Michael Miklavcic wrote: > What are we doing with feature branches once they're complete and merged > into master? Is our expectation that we'll keep feature branches in > perpetuity, or should we plan to do some house cleaning once they've been

Re: IRC Channel -> OPS?

2018-08-31 Thread Casey Stella
wait, I'm an op? Coming up in the world! Do we need this still? I'm currently afk, but will get to it tomorrow. On Wed, Aug 29, 2018 at 4:23 PM Otto Fowler wrote: > Damn, I was hoping not. It will never happen now > > > On August 29, 2018 at 15:49:26, zeo...@gmail.com (zeo...@gmail.com)

Re: [DISCUSS] Contributing a General Purpose Regex Parser

2018-08-29 Thread Casey Stella
+1, I look forward to the PR. On Tue, Aug 28, 2018 at 8:37 AM Nick Allen wrote: > I'd love to see a PR for this. I know there are others in the community > looking for something similar. > > On Sun, Aug 26, 2018 at 7:28 PM wrote: > > > Hello, > > > > > > > > We have implemented a general

Re: package.lock changes during build?

2018-08-25 Thread Casey Stella
never we build locally. Coincidentally, I > just > > addressed this by switching to `npm ci` in an open PR of mine because I > > noticed the same happening locally and I was already updating npm > commands > > in the pom.xml. > > > > > https://github.com/a

Re: package.lock changes during build?

2018-08-25 Thread Casey Stella
eptable, imo. > I've glanced at the package file and see carrots littering the > dependencies, which as I understand it means "get me anything later than > this version." I do not think we should be doing that. > > > On Sat, Aug 25, 2018, 9:14 AM Casey Stella

Re: package.lock changes during build?

2018-08-25 Thread Casey Stella
I have looked into this for other reasons and the guidance that I've seen is to check in package-lock.json into source control. I'll leave this stack overflow thread here: https://stackoverflow.com/questions/44206782/do-i-commit-the-package-lock-json-file-created-by-npm-5 I want to point out

Re: [DISCUSS] Getting to a 1.0 release

2018-08-20 Thread Casey Stella
I completely agree, Mike. Our docs are either very high level or very low level (and possibly stale) and, worse, aren't aimed at the actors that you've stated. I think that the HBase project does a good job of providing coherent and useable documentation in their "HBase Book" (see

Re: [DISCUSS] Pcap query branch completion

2018-08-16 Thread Casey Stella
I'm +1 on the merge. This is great work and congrats to those who contributed to it! On Thu, Aug 16, 2018 at 8:27 AM Otto Fowler wrote: > Looks good, thanks! > > > On August 15, 2018 at 19:38:12, Ryan Merriman (merrim...@gmail.com) wrote: > > Otto, I believe the items you requested are in the

Re: [DISCUSS] Release cadence

2018-08-15 Thread Casey Stella
which are also > large-ish and will fit nicely into the next cycle (pending completion, of > course): > >1. NiFi Metron parsers >2. Profiler enhancements - bootstrapping, etc. >3. Knox SSO > > > > On Wed, Aug 15, 2018 at 11:10 AM Casey Stella wrote: > >

Re: [DISCUSS] Release cadence

2018-08-15 Thread Casey Stella
Strictly selfishly, I'd love for a release to happen quickly enough to have something to announce to the board during the reports. Once every 2 months or when a sufficiently complicated change happens sounds like a sensible cadence. I very much support a "how do we get to 1.0" discussion, maybe

Re: [DISCUSS] Metron Release 0.6.0?

2018-08-15 Thread Casey Stella
+1 to both releases, this is plenty for an 0.6.0 and a 0.2.0 On Wed, Aug 15, 2018 at 11:04 AM Justin Leet wrote: > I just sent a thread about release cadence. Jon, I'd recommend starting a > thread on a 1.0 roadmap. I thought about merging the threads, but I think > that's just going to result

Re: Slack Channel

2018-08-15 Thread Casey Stella
Sorry Simon, I retract the comment! I didn't realize it was possible, but it is possible to invite. On Wed, Aug 15, 2018 at 1:01 PM Casey Stella wrote: > Sadly, it's the ASF slack and I believe it requires an @apache.org email > address. > > On Wed, Aug 15, 2018 at 12:57 PM Simon E

Re: Slack Channel

2018-08-15 Thread Casey Stella
Sadly, it's the ASF slack and I believe it requires an @apache.org email address. On Wed, Aug 15, 2018 at 12:57 PM Simon Elliston Ball < si...@simonellistonball.com> wrote: > Hello dev team, may I please join your slack channel :) >

Re: Security Feature Branch?

2018-07-13 Thread Casey Stella
nd Metron wherein possible threat details can be > > communicated > > > to Knox to take action on at authentication/authorization time. > > > Knox could also potentially push interesting events like possible brute > > > force login attempts to Metron. > > >

Re: Security Feature Branch?

2018-07-12 Thread Casey Stella
I added the feature branch: feature/METRON-1663-knoxsso https://git-wip-us.apache.org/repos/asf?p=metron.git;a=shortlog;h=refs/heads/feature/METRON-1663-knoxsso On Thu, Jul 12, 2018 at 11:13 AM Otto Fowler wrote: > I think I understand what you are saying very very very well Simon. I am > not

Re: Security Feature Branch?

2018-07-12 Thread Casey Stella
I would support this being a feature branch. It sounds like a valuable but large contribution. On Thu, Jul 12, 2018 at 10:51 AM Simon Elliston Ball < si...@simonellistonball.com> wrote: > I've been doing some work on getting the Metron UIs and REST layers to work > with Apache KnoxSSO, and LDAP

Re: [DISCUSS] Deprecating metron-api

2018-06-29 Thread Casey Stella
I have no objection and would consider it to be a prerequisite to bringing in the PR unless there's someone depending on it out there. You might want to cc user@ as well, to get a broader set of input for the "are people using it?" question. On Fri, Jun 29, 2018 at 5:21 PM Ryan Merriman wrote:

Re: Architectural reason to split in 4 topologies / impact on the kafka ressources

2018-06-22 Thread Casey Stella
Hey Michel, Those are good questions and there were some reasons surrounding that. In fact, historically, we had fewer topologies (e.g. indexing and enrichment were merged). Even earlier on, we had just one giant topology per parser that enriched and indexed. The long story short is that we

Re: [DISCUSS] Merging Solr feature branch (METRON-1416) into master

2018-06-21 Thread Casey Stella
I think that we should merge now, but I’m perhaps biased since I did one of the hard merges. I think that since the major outstanding bug is being worked and we are otherwise feature complete, the feature branch did its job and we are ready to merge. On Thu, Jun 21, 2018 at 10:21 Justin Leet

Re: [DISCUSS] Treating null as false in boolean expressions in Stellar

2018-06-16 Thread Casey Stella
I created a PR for the empty collection falseyness as well: https://github.com/apache/metron/pull/1064 so we can choose either of them if we so desire. On Sat, Jun 16, 2018 at 1:10 PM Casey Stella wrote: > I created a PR for this functionality, in case we decided for it: > https://gith

Re: [DISCUSS] Treating null as false in boolean expressions in Stellar

2018-06-16 Thread Casey Stella
? On Sat, Jun 16, 2018 at 10:17 AM Casey Stella wrote: > Right now, because fields may not exist, users can have an awkward time. > For instance, checking for is_alert, you end up having to preface checks > with exists(is_alert). > > For instance, in one of our use-cases: > https:/

[DISCUSS] Treating null as false in boolean expressions in Stellar

2018-06-16 Thread Casey Stella
Right now, because fields may not exist, users can have an awkward time. For instance, checking for is_alert, you end up having to preface checks with exists(is_alert). For instance, in one of our use-cases: https://github.com/apache/metron/tree/master/use-cases/geographic_login_outliers we use

Re: Writing enrichment data directly from NiFi with PutHBaseJSON

2018-06-13 Thread Casey Stella
https://cwiki.apache.org/confluence/display/METRON/ > >>> 2016/06/16/Metron+Tutorial+-+Fundamentals+Part+6%3A+Streaming+Enrichm > >>> ent > >>> > >>> 4. Create a new "dns" sensor in Metron 5. Use the CSVParser and > >>> SimpleHbaseEnrichment

Re: [ANNOUNCE] Apache Metron release 0.5.0

2018-06-08 Thread Casey Stella
Great job all! This was a big release with a lot of good stuff. I especially like the performance improvements :) Casey On Fri, Jun 8, 2018 at 8:54 AM Justin Leet wrote: > Hi All, > > I’m happy to announce the release of Metron 0.5.0! Everyone has put in a > lot of working into

Re: Writing enrichment data directly from NiFi with PutHBaseJSON

2018-06-05 Thread Casey Stella
s it worth us creating an nifi-metron-bundle. Happy to kick that off, > > > since I'm half way there. > > > > > > Simon > > > > > > > > > > > > On 5 June 2018 at 08:41, Otto Fowler wrote: > > > > > > > We have jiras a

Re: [DISCUSS] Field conversions

2018-06-05 Thread Casey Stella
Fowler wrote: > It is still our user list and dev list that will have the burden of > talking folks through that. > > > On June 5, 2018 at 09:58:32, Casey Stella (ceste...@gmail.com) wrote: > > To be clear, I'm not even suggesting that we create any tooling here. I'd > say ju

Re: Writing enrichment data directly from NiFi with PutHBaseJSON

2018-06-05 Thread Casey Stella
com> wrote: > Do we, the community, think it would be a good idea to create a > PutMetronEnrichment NiFi processor for this use case? It seems a number of > people want to use NiFi to manage and schedule loading of enrichments for > example. > > Simon > > On 5 June 2018

Re: [DISCUSS] Field conversions

2018-06-05 Thread Casey Stella
hat read and > > transformed fields for templates and indices to replace the colons with > > dots in ES. > > > > Simon > > > > On 5 June 2018 at 06:40, Casey Stella wrote: > > > > > +1 to that, Simon. Do we have a sense of if there are utilities > prov

Re: Writing enrichment data directly from NiFi with PutHBaseJSON

2018-06-05 Thread Casey Stella
The problem, as you correctly diagnosed, is the key in HBase. We construct the key very specifically in Metron, so it's unlikely to work out of the box with the NiFi processor unfortunately. The key that we use is formed here in the codebase:

Re: [DISCUSS] Field conversions

2018-06-05 Thread Casey Stella
nd transforming existing data. > > > > On Tue, Jun 5, 2018 at 8:21 AM, Casey Stella wrote: > > > > > Well, on write it is a transformation, on read it's a translation. > This > > is > > > to say that you're providing a mapping on read to translate field

Re: [DISCUSS] Field conversions

2018-06-05 Thread Casey Stella
Well, on write it is a transformation, on read it's a translation. This is to say that you're providing a mapping on read to translate field names given the index you're using. The other approach that I was considering last night is a field transformation REST call which translates field names

Re: [DISCUSS] Field conversions

2018-06-04 Thread Casey Stella
Before we construct a super generic solution, can we get an analysis of all the places in the UI where we're hard-coding fields? It seems like pulling the field from the global config is the strategy that we've gone with that could be expanded upon in https://github.com/apache/metron/pull/1010

Re: [DISCUSS] Refactoring

2018-05-30 Thread Casey Stella
> On May 30, 2018 at 11:53:51, Casey Stella (ceste...@gmail.com) wrote: > > Yep, I think we can, mike. > > Let me start with a emendation: > > "Don’t combine code changes with lots of edits of whitespace, comments, or > code changes specifically for cosmetic refactoring pur

Re: [DISCUSS] Refactoring

2018-05-30 Thread Casey Stella
up a vote > thread following with the final proposed changes? > > On Wed, May 30, 2018 at 9:46 AM, Casey Stella wrote: > >> I'm torn on this, honestly. I completely agree that cosmetic refactoring >> gets in the way of review and the risk can be more than the re

Re: [DISCUSS] Refactoring

2018-05-30 Thread Casey Stella
I'm torn on this, honestly. I completely agree that cosmetic refactoring gets in the way of review and the risk can be more than the reward, especially in a subtle bit of code. That being said, I'm a big fan of opportunistically refactoring to generalize or correct faulty assumptions. Often, I

Re: [VOTE] Metron Release Candidate 0.5.0-RC1

2018-05-29 Thread Casey Stella
Just a question, do we need anything new in the Upgrading.md for this release? Any migration that we expect people to do? On Tue, May 29, 2018 at 11:30 AM Nick Allen wrote: > METRON-1544 was just merged into master. > > > On Tue, May 29, 2018 at 2:16 PM, Justin Leet > wrote: > > > I'm going

Re: Request for Comment on new Syslog 5424 Parsing library

2018-05-18 Thread Casey Stella
Cool! I'd welcome a syslog parser! On Fri, May 18, 2018 at 10:02 AM Otto Fowler wrote: > There have been some issues and talk about they way we parse syslog, and > the deficiencies of our grok and regex based approaches, mainly not > supporting structured data as I

Re: [DISCUSS] Release Manager

2018-05-10 Thread Casey Stella
18 at 9:07 AM, Justin Leet <justinjl...@gmail.com> > > wrote: > > > > > I'd be happy to to volunteer to take over for a while. > > > > > > Thanks to Matt for all the help through the last couple releases! > > > > > > Justin > >

[DISCUSS] Release Manager

2018-05-10 Thread Casey Stella
Hi All, Matt Foley, our esteemed Release manager for the last couple releases, has asked to be relieved. So, I'm calling on volunteers for the next release manager. It should be a committer and there are a few things that require a PMC member, I believe, but the release manager can ask for help

Re: [DISCUSS] Pcap UI user requirements

2018-05-09 Thread Casey Stella
A couple of thoughts on cluster overuse: * Definitely can't pause/resume MR jobs, unfortunately * The traditional approach to managing overuse of cluster resources and prioritization in Yarn is via the scheduler. I'd suggest rather than building this ourselves, we allow users to be associated

Re: [DISCUSS] Release?

2018-05-09 Thread Casey Stella
e/metron#888 > > >> > > > > >> 3 months ago METRON-1432 JDK Install Fails on Ubuntu > > >> Development > > >> > > > > >> Environment (nickwallen) closes apache/metron#913 > > >> > >

Re: [DISCUSS] Release?

2018-05-09 Thread Casey Stella
> > > >> 4 months ago METRON-1428: Travis build failing from > metron-config > > > > > >> (mmiklavc via mmiklavc) closes apache/metron#908 > > > > > >> 4 months ago METRON-1302: Split up Indexing Topology into batch > > and > > > &g

Re: [DISCUSS] Release?

2018-05-09 Thread Casey Stella
n#859 > 5 months ago METRON-1349 Full Dev Builds Metron Twice (nickwallen) closes > apache/metron#866 > 5 months ago METRON-1343 Swagger UI for User Controller needs request > method (MohanDV via ottobackwards) closes apache/metron#862 > 5 months ago METRON-1306: When index template install

Re: [DISCUSS] Release?

2018-05-09 Thread Casey Stella
I wasn't aware we had a script for that..is that in dev-utilities/release-utils? On Wed, May 9, 2018 at 11:41 AM Otto Fowler <ottobackwa...@gmail.com> wrote: > Can you run the issues included script and post that for us to see? > > > On May 9, 2018 at 11:14:11, Casey Stella (

[DISCUSS] Release?

2018-05-09 Thread Casey Stella
Is it about time for a release? I know we got some substantial performance changes in since the last release. I think we might have a justification for a release. Casey

Re: [VOTE] Development Guidelines Addendum on Inactive Pull Requests

2018-04-20 Thread Casey Stella
+1 On Fri, Apr 20, 2018 at 11:17 AM David Lyle wrote: > +1 sounds good to me. > > -D... > > > On Fri, Apr 20, 2018 at 11:09 AM, zeo...@gmail.com > wrote: > > > +1 (non-binding) > > > > On Fri, Apr 20, 2018 at 9:42 AM Michel Sumbul

Re: [DISCUSS] Metron RPM spec changelog

2018-04-18 Thread Casey Stella
I think I'd prefer to see the changelog only include the release entries, rather than individual entries per dev. We keep the spec file in source control to determine the individual changes between releases. I'm happy to have my mind changed, though. On Wed, Apr 18, 2018 at 9:47 AM Michael

Re: Another intermittant build failure?

2018-04-11 Thread Casey Stella
I have not personally seen that one yet, but I will not deny that it exists. It could be very intermittent or triggered under load in travis too. Either way, we should probably investigate and fix. On Wed, Apr 11, 2018 at 3:57 PM Otto Fowler wrote: > I had a PR build

Re: [VOTE] Metron Release Candidate 0.4.2-RC2

2018-04-10 Thread Casey Stella
Nevermind, it's just the internal apache release status wasn't updated. Sorry, I updated it as part of the board report. Let me make sure I update teh docs for releasing. On Tue, Apr 10, 2018 at 10:35 AM Casey Stella <ceste...@gmail.com> wrote: > It seems that 0.4.2 never got

Re: [VOTE] Metron Release Candidate 0.4.2-RC2

2018-04-10 Thread Casey Stella
k cluster using the > > generated RPMs > > * Bro, YAF and snort - ingest into kafka topics and validated indices > > * Add squid telemetry, ingest into kafka topic and validated indices > > * Management UI, Alerts UI and Swagger UI sanity check > > > > +1 (non-binding)

Re: [DISCUSS] Time to remove github updates from dev?

2018-03-19 Thread Casey Stella
get this actioned. > >> > >> They would need to assist with: > >> > >> 1. Creation of the new "issues" list > >> 2. redirect both GitHub and JIRA integrations to the new list > >> > >> Cheers > >> > >> On Sat, Jan 27, 201

Re: [DISCUSS] Alternatives to split/join enrichment

2018-02-22 Thread Casey Stella
FYI, the PR for this is up at https://github.com/apache/metron/pull/940 For those interested, please comment on the actual implementation there. On Thu, Feb 22, 2018 at 12:43 PM, Casey Stella <ceste...@gmail.com> wrote: > So, these are good questions, as usual Otto :) > > > ho

Re: [DISCUSS] Alternatives to split/join enrichment

2018-02-22 Thread Casey Stella
tions: > > * how does this effect the distribution of work through the cluster, and > resiliency of the topologies? > * Is anyone else doing it like this? > * Can we have multiple thread pools and group tasks together ( or separate > them ) wrt hbase? > > > > On February 22,

[DISCUSS] Alternatives to split/join enrichment

2018-02-22 Thread Casey Stella
Hi all, I've been thinking and working on something that I wanted to get some feedback on. The way that we do our enrichments, the split/join architecture was created to effectively to parallel enrichments in a storm-like way in contrast to OpenSoc. There are some good parts to this

Re: Apache Website Required Links

2018-02-15 Thread Casey Stella
Just reporting back that Anand's PR METRON-1386 ( https://github.com/apache/metron/pull/935) has been merged into master and the asf-site branch. Kudos to Anand! Casey On Wed, Feb 7, 2018 at 9:11 AM, Anand Subramanian < asubraman...@hortonworks.com> wrote: > I can take a shot at this if there

Re: [DISCUSS] Persistence store for user profile settings

2018-02-01 Thread Casey Stella
So, I'll answer your question with some questions: - No matter the data store we use upgrading will take some care, right? - Do we currently depend on a RDBMS anywhere? I want to say that we do in the REST layer already, right? - If we don't use a RDBMs, what's the other option?

Re: When things change in hdfs, how do we know

2018-01-31 Thread Casey Stella
is a newish-api, and there should be 1 listeners. > > Having 1 listener shouldn’t be an issue. > > > > On January 31, 2018 at 11:45:54, Casey Stella (ceste...@gmail.com) wrote: > > Hmm, I have heard this feedback before. Perhaps a more low-key approach > would be eit

Re: When things change in hdfs, how do we know

2018-01-31 Thread Casey Stella
Hmm, I have heard this feedback before. Perhaps a more low-key approach would be either a static timer that checked or a timer bolt that sent a periodic timer and the parser bolt reconfigured the parser (or indeed we added a Reloadable interface with a 'reload' method). We could be smart also

Re: [DISCUSS] Move SHELL type functions from management to stellar common

2018-01-31 Thread Casey Stella
I assumed he was talking about the SHELL_EDIT stuff and maybe the file loading bits. The config stuff is metron specific On Wed, Jan 31, 2018 at 10:06 AM, Nick Allen wrote: > > I think we should move the shell/console type functions from stellar > > What functions

Re: [DISCUSS] Move SHELL type functions from management to stellar common

2018-01-31 Thread Casey Stella
I'd be in favor of that. That is general purpose stuff. On Wed, Jan 31, 2018 at 9:12 AM, Otto Fowler wrote: > Per: https://issues.apache.org/jira/browse/METRON-876 > > I think we should move the shell/console type functions from stellar > management to stellar-common,

Re: Metron User Community Meeting Call

2018-01-26 Thread Casey Stella
I can't wait! This is going to be really cool :) On Fri, Jan 26, 2018 at 5:25 PM, James Sirota wrote: > Yeah very interested in the presentation as well > > 26.01.2018, 15:15, "Simon Elliston Ball" : > > This is going to be a really exciting

Re: Master is failed in Travis

2018-01-22 Thread Casey Stella
This could be one of those intermittent test failures related to timing. Specifically this: test(org.apache.metron.rest.controller.SensorIndexingConfigControllerIntegrationTest) Time elapsed: 0.064 sec <<< FAILURE! java.lang.AssertionError: Status expected:<404> but was:<200> at

Re: [DISCUSS] Time to remove github updates from dev?

2018-01-19 Thread Casey Stella
I could get behind that. On Fri, Jan 19, 2018 at 3:31 PM, Andre wrote: > Folks, > > May I suggest Metron follows the NiFi mailing list strategy (we got > inspired by another project but I don't recall the name) and remove the > github comments from the dev list? > > Within

Re: [DISCUSS] Upgrading Solr

2018-01-18 Thread Casey Stella
+1 to both the feature branch and user@ announcement. On Thu, Jan 18, 2018 at 2:45 PM, Otto Fowler wrote: > +1 to the feature branch. > > Also, there have been some questions about solr support recently, I think > when the feature branch > is ready you should announce

Re: Some more upgrade fallout... Can't restart Metron Indexing

2018-01-18 Thread Casey Stella
So, the challenge here is that our install script isn't smart enough right now to skip creating tables that are already created. One thing you could do is 1. rename the hbase tables for metron (see https://stackoverflow.com/questions/27966072/how-do-you-rename-a-table-in-hbase ) 2.

Re: Travis for Apache/Metron is in trouble

2018-01-18 Thread Casey Stella
I made an infra ticket: https://issues.apache.org/jira/browse/INFRA-15865 On Thu, Jan 18, 2018 at 11:42 AM, Otto Fowler wrote: > 24hr long build is blocking up master’s travis build. > Who can nuke it? > > ottO >

Anand is a new Committer!

2018-01-11 Thread Casey Stella
The Project Management Committee (PMC) for Apache Metron has invited Anand Subramanian to become a committer and we are pleased to announce that they have accepted. Congratulations and welcome, Anand!

Re: Full Dev -> Heartbeat issues

2018-01-08 Thread Casey Stella
I haven't seen that one. I spun one up from master on Friday and it seemed ok. Sorry, "works for me!" isn't super helpful, but it may be relevant since master is close to 0.4.2 :) On Mon, Jan 8, 2018 at 11:11 AM, Otto Fowler wrote: > I just started up full dev from

Re: [DISCUSS] Generating and Interacting with serialized summary objects

2018-01-05 Thread Casey Stella
com> wrote: > I would imagine the ‘stellar-object-repo’ would be part of the global > configuration or configuration passed to the command. > why specify in the function itself? > > > > > On January 5, 2018 at 11:22:32, Casey Stella (ceste...@gmail.com) wrote:

Re: [DISCUSS] Generating and Interacting with serialized summary objects

2018-01-05 Thread Casey Stella
features and level > of > > abstraction feel appropriate to me. I think it buys us 1) learning from a > > starting typosquatting use case, 2) flexibility to change and adapt it > > without affecting users, and 3) enough concrete capability to make more > > specific use ca

Re: [DISCUSS] Generating and Interacting with serialized summary objects

2018-01-04 Thread Casey Stella
*for* typosquatting use-case. Hard coding this would prevent things like bloom filters containing malicious IPs from a reference source, for instance. On Thu, Jan 4, 2018 at 10:46 AM, Casey Stella <ceste...@gmail.com> wrote: > So, there is value outside of just bloom usage. The most specifi

  1   2   3   >