So we graduated...

For anyone paying attention to incubator-general, it will come as no surprise that we graduated as of last night's board meeting. We have a press released queued up and planned for monday along with a PR (METRON-687 at https://github.com/apache/incubator-metron/pull/539). It escaped my notice

Re: Failure to Deploy "Quick Dev"

I'm betting we need to regenerate the quickdev image after all the mpack changes. On Fri, Apr 21, 2017 at 10:45 AM, Otto Fowler wrote: > From lira: > > I 'think' that quickdev is actually build from full_dev, with metron > installed already. So it may be that we need a

Re: Ansible Docker fails to build latest

Sorry, can you go through how you're getting to this error? I'm not super familiar with this part of the stack..is this compiling Metron inside of a docker image? On Tue, Apr 18, 2017 at 1:45 PM, Otto Fowler wrote: > Is it something to do with the relocation of

Re: Silly question about zk_load_configs.sh

, Otto Fowler <ottobackwa...@gmail.com> wrote: > OK, not to be thick, I just don’t see where any class from metron-parsers > gets called or instantiated. > ¯\_(ツ)_/¯ > > > On April 24, 2017 at 09:55:38, Casey Stella (ceste...@gmail.com) wrote: > > All of the current

Re: [DISCUSS] Persisting user data

I'd vote for a DB-based solution, but I'd argue that any solution shouldn't be database specific (i.e. postgres), but JDBC-generic. People and organizations have very strong views regarding databases and I'd prefer to side-step those holy wars by being agnostic. On Wed, Aug 2, 2017 at 9:36 PM,

Re: MaaS and Metron Architecture talks at DataWorks Summit SJ 2017

Ok, those talks are added. On Thu, Aug 3, 2017 at 3:44 PM, Casey Stella <ceste...@gmail.com> wrote: > Absolutely! > > On Thu, Aug 3, 2017 at 3:41 PM, Justin Leet <justinjl...@gmail.com> wrote: > >> Could we put these up on the wiki page for tech talks in the

Re: SimpleEnrichmentFlatFileLoaderIntegrationTest …..

That is puzzling for sure. Another intermittent failing test mystery (dum dum dum), brought to you by the letter 'o' and the number Math.rand(). On Thu, Aug 3, 2017 at 3:15 PM, Otto Fowler wrote: > Results : > > Failed tests: > >

Re: MaaS and Metron Architecture talks at DataWorks Summit SJ 2017

tent. > > https://cwiki.apache.org/confluence/display/METRON/Tech+Talks > > On Thu, Aug 3, 2017 at 10:32 AM, Casey Stella <ceste...@gmail.com> wrote: > >> The Videos of talks that Simon Ball and I gave at DataWorks Summit are >> now up and on youtube: >> >>

Re: Metron Alerts bombing in Travis?

Yes, I'm getting it locally too, here's more context: [INFO] ERROR in multi script-loader!./~/jquery/dist/jquery.js script-loader!./~/tether/dist/js/tether.js script-loader!./~/ace-builds/src-noconflict/ace.js [INFO] Module not found: Error: Can't resolve

Re: Metron Alerts bombing in Travis?

t; > wrote: > > > Where are the bootstrap files? > > > > the @import ~bootstrap/….. looks like it is failing? > > > > > > > > On August 11, 2017 at 09:51:03, RaghuMitra Kandikonda ( > > raghumitra....@gmail.com) wrote: > > > > I am ru

Re: threatintel_taxii_load.sh throws exception

Thanks Vladimir, https://github.com/apache/metron/pull/643 was submitted a few moments ago. On Mon, Jul 10, 2017 at 3:35 PM, Vladimir Shlyakhtin < vladimir.shlyakh...@sstech.us> wrote: > Filed METRON-1026 > > - Vladimir > > From: Otto Fowler

Re: UI pivotting / aggregation backend

I just want to chime in and support the notion of an abstraction layer between the UI and the indexed stores. I think that having an API that people can conform to is going to be important as people want to plug in their own backing indices in the future. Casey On Thu, Jul 6, 2017 at 2:11 PM,

Re: [DISCUSSION] METRON-1046 -> Stellar Files for multiple statement execution

Just chiming in on a part of this: definitely we do not want to lose automatic config updates (at least, I'd be strongly, strongly STRONGLY against it). I definitely agree that JSON files could easily get unwieldy. I don't know anything about JSON pointers, could you cover that briefly, Matt?

Re: threatintel_taxii_load.sh throws exception

Also, please reply here with the JIRA and I'll submit a PR. It's a very easy fix. On Mon, Jul 10, 2017 at 1:24 PM, Casey Stella <ceste...@gmail.com> wrote: > This is absolutely a bug and you should open a JIRA. > > On Mon, Jul 10, 2017 at 1:11 PM, Vladimir Shlyakhtin <

Re: threatintel_taxii_load.sh throws exception

This is absolutely a bug and you should open a JIRA. On Mon, Jul 10, 2017 at 1:11 PM, Vladimir Shlyakhtin < vladimir.shlyakh...@sstech.us> wrote: > Hello, > > After upgrading from 0.3.0 version we noticed that taxii loader does not > work. > > Here is details: > > #

Re: [VOTE] Apache Metron 0.4.0 release

+1 (binding) * Verified keys * Verified mvn build * Verified unit and integration tests run * Verified license check runs * Verified fulldev spun up with smoketest On Wed, Jun 28, 2017 at 8:10 PM, Anand Subramanian < asubraman...@hortonworks.com> wrote: > +1 (non-binding) > > * Brought up Metron

Re: [DISCUSS] Regression introduced in Full Dev

Yeah, I tend to agree that a rundown of the various methods and when you would use them is in order. I will say that full-dev is especially important to have working since it is required for validating PRs. On Tue, Apr 25, 2017 at 18:56 zeo...@gmail.com wrote: > Can somebody

Re: Normalization topology or separate normalization bolt for parsing topology

nd give us parsing > exception. The timestamp example was not a good one because that is > actually a post-parse exception. > > On Wed, Apr 26, 2017 at 11:28 PM, Casey Stella <ceste...@gmail.com> wrote: > > > So, further transformation post-parse was one of th

Re: Normalization topology or separate normalization bolt for parsing topology

you going to deal with normalization after the parsing if > that noise affects the parsing? For some reason, the incoming data do not > look like in the way that has to be. > > On Wed, Apr 26, 2017 at 11:37 PM, Casey Stella <ceste...@gmail.com> wrote: > > > Ok, that's an

Re: Normalization topology or separate normalization bolt for parsing topology

y field. We do after all > store the original_string for you if you really absolutely have to had > everything, so a more schema-on-read philosophy certainly applies and will > likely side-step a lot of your issues. > > Simon > > > On 26 Apr 2017, at 14:37, Casey Ste

Re: Normalization topology or separate normalization bolt for parsing topology

So, further transformation post-parse was one of the motivating reasons for Stellar (to do that transformation post-parse). Is there a capability that it's lacking that we can add to fit your usecase? On Wed, Apr 26, 2017 at 9:24 AM, Ali Nazemian wrote: > I've created a

Re: Normalization topology or separate normalization bolt for parsing topology

So, having stellar operate on the whole message is definitely something that would be cool. That being said, it's also nice to motivate the construction of functions to do simple transformations/normalizations. That way, common useful capabilities may be reused all the places Stellar is used

Re: Silly question about zk_load_configs.sh

rs, except what is brought in from > common, why doesn’t this just call common? > You don’t need parsers jar for any of this do you? > > > On April 24, 2017 at 09:22:06, Casey Stella (ceste...@gmail.com) wrote: > > Because stellar statements are validated prior to pushing, we needed

Re: Profiler statistics NaN

Ok, so the problem here is that your profile is returning integers (specifically HLLP cardinalities) rather than stats objects. When you're doing: STATS_PERCENTILE(STATS_MERGE( PROFILE_GET('host-talks-to', '99.191.183.156', PROFILE_FIXED(10, 'HOURS')), 90) You are calling STATS_MERGE on a

Re: Profiler statistics NaN

t; > "result": "HLLP_CARDINALITY(outcoming)" > > to this... > > "result": "outcoming" > > ? > > On Wed, Aug 9, 2017 at 3:48 PM Casey Stella <ceste...@gmail.com> wrote: > > > Ok, so the problem here is that your profile is re

Re: Profiler statistics NaN

describe or complain that it doesn't know how to handle a > > list. Easy fix though. > > > > [Stellar]>>> STATS_MEAN(STATS_ADD(null, 1, 2, 3)) > > 2.0 > > > > [Stellar]>>> STATS_MEAN(STATS_ADD(null, [1,2,3])) > > 1.0 > &g

Re: Profiler statistics NaN

friendly set of > facade functions, > or some thing would work. > > > > On August 9, 2017 at 16:38:48, Casey Stella (ceste...@gmail.com) wrote: > > Yeah, I'm leaning toward STATS_ADD or STATS_INIT taking a list of numbers. > STATS_MERGE seems confusing. > >

Dataworks summit Sydney

The Australia/Pacific version of Dataworks Summit is in Sydney this year, September 20-21. This is a great place to talk about work you are doing in Apache Metron or how you are using Metron. Information on submitting an abstract is at

Re: Infrastructure migrated

in the JIRA, please let me know and I'll add you. On Mon, May 15, 2017 at 9:30 AM, Casey Stella <ceste...@gmail.com> wrote: > For those who haven't noticed, the git infrastructure has migrated over. > If you are a committer, you will need to adjust the URLs for the apache > remote re

Infrastructure migrated

For those who haven't noticed, the git infrastructure has migrated over. If you are a committer, you will need to adjust the URLs for the apache remote repo via: git remote set-url apache https://git-wip-us.apache.org/repos/asf/metron.git Obviously substitute "apache" for "upstream" if you

[DISCUSS] Enrichment Split/Join issues

Hi All, Last week, I encountered some weirdness in the Enrichment topology. Doing some somewhat high-latency enrichment work, I noticed that at some point, data stopped flowing through the enrichment topology. I tracked down the problem to the join bolt. For those who aren't aware, we do a

Re: [DISCUSS] Enrichment Split/Join issues

the knobs to tune, this architecture works, I believe. On Tue, May 16, 2017 at 12:09 PM, Casey Stella <ceste...@gmail.com> wrote: > We could definitely parallelize within the bolt, but you're right, it does > break the storm model. I also like making things other people's problems

Re: [DISCUSS] Enrichment Split/Join issues

efficient (i.e. first labeling, and > “completing” a message and then dependent of label and completeness do > different other enrichments). > > So you have a +1 from me for serial rather than parallel enrichment. > > > BR, >Christian > > On 16.05.17, 16:58, "Casey Stel

Re: [DISCUSS] Enrichment Split/Join issues

simplify the Enrichment > topology. I don't think we would not need the split/join pattern if we did > this. No? > > On Tue, May 16, 2017 at 11:54 AM, Casey Stella <ceste...@gmail.com> wrote: > > > The problem is that an enrichment type won't necessarily have a fixed >

Re: [INCOMING] Metron 0.4.0 release (RC3)

I'm willing to be shot down about this, but METRON-995 fixes a pretty significant bug around using map variables and temporary variables in stellar enrichments. I'd appreciate if it could get in, but I'll understand if we think it shouldn't. On Sat, Jun 3, 2017 at 4:31 AM, Christian Tramnitz

Re: [INCOMING] Metron 0.4.0 release (RC3)

Ok METRON-995 is in, so we should be able to release any time from my perspective. I think we're just waiting on some resolution for 941 at this point (that thread seems dead). On Fri, Jun 9, 2017 at 6:38 PM, Casey Stella <ceste...@gmail.com> wrote: > I'm willing to be shot d

[DISCUSS] Mutation of Indexed Data

Hi All, I know we've had a couple of these already, but we're due for another discussion of a sensible approach to mutating indexed data. The motivation for this is users will want to update fields to correct and augment data. These corrections are invaluable for things like feedback for ML

Re: storm-kafka-client use in Metron

Thanks for the heads up, kris. This is really good context and we should be aware of it. So we could avoid this by avoiding subscription objects which rely on consumer.subscribe and prefer consumer.assign, correct? On Tue, Jun 20, 2017 at 10:07 PM, Kristopher Kane wrote:

[DISCUSS] Metadata Ingest

Hi All, I wanted to call attention to a JIRA (METRON-1001) that I just submitted and possibly discuss it more broader than on the PR. Currently, we only ingest data in Metron. Often, there is valuable metadata constructed up-stream of Metron that is relevant to enrichment and cross-cuts many

Re: [DISCUSS] Metadata Ingest

d like to talk about if some of this > activity > > is more enrichment than not, and should be handled/exposed there, where > we > > have the splitter/joiner pattern already. > > > > - Other than exposing the metadata, I am not sure I understand the > > difference

Re: [DISCUSS] Mutation of Indexed Data

be > painful. > >> > > >> > Justin > >> > > >> > > >> > On Wed, Jun 21, 2017 at 10:18 PM, Simon Elliston Ball < > >> > si...@simonellistonball.com> wrote: > >> > > >> > > I'd say that w

Re: [DISCUSS] Metron IRC channel

ask for what we > > mentioned above? > > > > Jon > > > > On Sun, Dec 18, 2016 at 1:58 PM Kyle Richardson < > kylerichards...@gmail.com> > > wrote: > > > >> I'll second the JIRA and Git integrations. I also like the meeting > minutes > >

Re: [DISCUSS] Enrichment Split/Join issues

and Geo enrichment bolts from the Enrichment > > topology. Stellar provides a user with much greater flexibility than the > > existing HBase and Geo enrichment bolts. > > > > A side effect of this would be to greatly simplify the Enrichment > > topology. I don't thi

Re: Question about the customization of Metron with my machine learining algo.

We do not ship any ML models currently with metron, just the infrastructure to deploy your own models and interact with those models from within Metron. That being said, you might be interested in https://gist.github.com/cestella/8dd83031b8898a732b6a5a60fce1b616 That's the code to take a DGA

Re: Question about the customization of Metron with my machine learining algo.

tation process. > > For example if I have an SVM algo that I would test into Metron and that > ML algortihm has been developed in python using scikit-py. > > How can I do that? > > Thank you and I'm sorry for the very basic question. > > Best Regards, > > Simone >

Re: [ANNOUNCE] Apache Metron Release 0.4.1

Fantastic! I'm really proud of this release and a great job was done by Matt and the community for getting this out! On Tue, Sep 19, 2017 at 1:24 PM, Frank Horsfall < frankhorsf...@cunet.carleton.ca> wrote: > Congrats guys! > > > > Frank > > > > > > *From:* zeo...@gmail.com

Re: [VOTE] Metron Release Candidate 0.4.1-RC4

I ran it up and forgot to vote. +1 binding - ran in full-dev - verified signatures - stellar works On Thu, Sep 14, 2017 at 9:52 AM, Matt Foley wrote: > Hi Team, > Unfortunately, it appears I was incorrect to say it’s passing just yet. > On reviewing the rules, I find that a

Re: [DISCUSS] Upgrading Elasticsearch from 2.x to 5.x

Yeah, I agree with what Michael "fine whine" Miklavcic said; I'm in favor of the high level client. On Thu, Oct 5, 2017 at 3:35 PM, Michael Miklavcic < michael.miklav...@gmail.com> wrote: > Justin, thanks for the feedback! I'm inclined to agree with you about using > the high level client. It's

Re: Cloudtrail use case

There is actually a use-cases top level directory with worked examples in them. They get picked up by the doc book too! I'd suggest putting it there, thoughts? On Fri, Oct 6, 2017 at 8:44 AM, Nick Allen wrote: > Yes, agreed, Justin. I guess my main point to Laurens was

Re: [DISCUSS] Dropping support for elastic 2.x

So, how would this work in an upgrade scenario that does not involve losing the existing indexed data? On Wed, Oct 4, 2017 at 12:55 PM, Michael Miklavcic < michael.miklav...@gmail.com> wrote: > The client I'm currently working on moving towards would *not* be backwards > compatible. >

Re: [DISCUSS] Dropping support for elastic 2.x

rk, but it looks like it's just calling > > shell commands from python, e.g. https://github.com/ > > apache/metron/blob/master/metron-deployment/packaging/ > > ambari/metron-mpack/src/main/resources/common-services/ > > ELASTICSEARCH/2.3.3/package/scripts/elastic_master.p

Re: [DISCUSS] Dropping support for elastic 2.x

> > from the HDFS store. Alternatively there are means to do inplace upgrades > > from 2.x to 5.x I believe. > > > > Simon > > > > > On 4 Oct 2017, at 18:05, Casey Stella <ceste...@gmail.com> wrote: > > > > > > So, how would this work in an upgrad

Re: [VOTE] Metron Release Candidate 0.4.1-RC3

Yeah, I fixed the dependencies_with_url problem in METRON-1169. Let's pull that one in and the rat check one and cut a new RC. I'm -1 On Fri, Sep 8, 2017 at 7:38 PM, Matt Foley wrote: > Couple pieces of info that may affect your vote: > > 1. This does not include today’s

Re: Unclear recent commit

My guess is that what happened was otto did a pull of METRON-1061 into his master, but it wasn't *quite* in sync with master in apache and there was a conflict in the dependencies_with_url.csv, so that commit was made locally to fix the conflict. both commits, the squashed and the merge conflict

Re: Unclear recent commit

I don't have an issue with this; it's good to have the explanation. I also found that we weren't running the dependency analyzer as part of travis (or, we thought we were, but we weren't) until METRON-1169. Thanks otto for the explanation and jon for catching it. On Fri, Sep 8, 2017 at 10:09

Re: [VOTE] Metron Release Candidate 0.4.1-RC3

Just a FYI, I think all the relevant PRs to fix the rat check and dependency analyzer are committed now and we can cut another RC whenever we're ready. On Fri, Sep 8, 2017 at 8:24 PM, Casey Stella <ceste...@gmail.com> wrote: > Yeah, I fixed the dependencies_with_url problem in ME

Re: Unclear recent commit

That's very weird. On Fri, Sep 8, 2017 at 10:12 PM, Otto Fowler <ottobackwa...@gmail.com> wrote: > without re-writing my other reply: > I use the prepare-commit into a clean repo every time, so I believe that > this issue > *was* in the apache repo. > > > On September

Re: [DISCUSS] Metron release 0.4.1

For my PRs, I'd vote for METRON-1122 being in (commit very imminent). I'd very much like METRON-1134 to be in as well. Beyond that, I'm ok On Tue, Aug 22, 2017 at 4:37 PM, Nick Allen wrote: > Thanks for starting the process, Matt. > > These are my own open PRs that I would

Re: [DISCUSS] Metron release 0.4.1

do you have an ETA for these two PRs? (PR#709 and 717) > Thanks, > --Matt > > On 8/29/17, 9:34 AM, "Casey Stella" <ceste...@gmail.com> wrote: > > For my PRs, I'd vote for METRON-1122 being in (commit very imminent). > I'd very much like METRON-1134 to be in as well. &

Re: Upgrade from Metron 0.4.1-rc to 0.4.1-rc4 problems.

So the escalation topic is a new parameter for the REST service in 0.4.1. It appears that the ambari upgrade story is a bit weak. Is it possible to modify /var/lib/ambari-agent/cache/ common-services/METRON/0.4.1/package/templates/metron.j2 and - create a kafka topic called 'metron_escalation'

Re: [DISCUSS] Metron release 0.4.1

t; wrote: >> >> > > >> >> > >> The following PRs are usability enhancements for the >> Profiler. They >> >> are >> >> > >> fairly simple and I think are very helpful for >> troubleshooting. I >> &g

Re: Unclear recent commit

d against policy? What should I have done? > I think the committer’s guide is archived btw. > > > On September 8, 2017 at 22:12:27, Casey Stella (ceste...@gmail.com) wrote: > > I don't have an issue with this; it's good to have the explanation. I also > found that we weren't r

[DISCUSS] Splitting up the Indexing Topology

One of the lessons that have bubbled up in doing some performance analysis is that having the indexing topology share both the ES and the HDFS writer in the same topology can be problematic from a tuning perspective. Specifically, it's hard to square that circle and make both perform fast enough

[DISCUSS] Build broken due to transitive dependencies

Ok, the build is broken in metron-config due to some transitive changes that happened in npm-land: [INFO] /Users/cstella/Documents/workspace/metron/fork/incubator-metron/metron-interface/metron-config/node_modules/toposort/index.js:32 [INFO] throw new Error('Cyclic dependency:

Re: [DISCUSS] Build broken due to transitive dependencies

rking at any moment because its dependencies can change. Prevent this by > migrating to Yarn: https://bower.io/blog/2017/how-to-migrate-away-from- > bower/ > > > On October 2, 2017 at 11:29:50, Casey Stella (ceste...@gmail.com) wrote: > > Ok, I can verify that 0.4.1 did build

Re: [DISCUSS] Build broken due to transitive dependencies

ntly in a pr, though I can’t > remember which one > > > On October 2, 2017 at 11:45:48, Casey Stella (ceste...@gmail.com) wrote: > > Yeah, seems like it might be worth while to seriously investigate > migrating to yarn if that gets us a consistent build. > > On Mon, Oct

Re: [ANNOUNCE] Metron community meeting

forever as far as I'm concerned. They don't really close so much as they are forgotten. :) On Thu, Aug 24, 2017 at 11:08 AM, Otto Fowler <ottobackwa...@gmail.com> wrote: > Casey, what is the time frame for the discuss thread to ‘close’? > > > On August 24, 2017 at 10:06:12, C

Re: [ANNOUNCE] Metron community meeting

Yeah, sorry about that; technology (and webex in particular) is such a fragile thing sometimes. I think the synopsis is pretty complete. That being said, if there's anything that you want clarity on, we can hammer it out on the discuss thread too. On Wed, Aug 23, 2017 at 4:28 PM, James Sirota

Re: [ANNOUNCE] Metron community meeting

weekend plus a business day has been elapsed - Some general consensus is reached - A full day has gone by without any dissent If nobody responds, then treat that as consent by silence. That's what I normally use. :) On Thu, Aug 24, 2017 at 11:09 AM, Casey Stella <ceste...@gmail.com>

Re: Question on Hdfs Writer

The storm metron-writer bolt has to write to many different sensor's HDFS files. The hdfs bolts provided by storm only support writing to one file (or set of files). The alternative would have been a bolt per sensor, which did not support our architecture. In short, the hdfs bolt from storm

Re: [DISCUSS] Metron release 0.4.1

ait until Friday, if > necessary, to cut an RC with 717 in it? > Thanks, > --Matt > > On 8/29/17, 11:45 AM, "Casey Stella" <ceste...@gmail.com> wrote: > > 709 is in and 717 is under concerted review by Otto. I'd like to see > it in > by Friday. >

Re: SUM aggregator not working?

Ok, so this is subtle. Your rules are wrong and I totally understand why you thought they were right. When we index into ES, we take . and convert them to :, however PRIOR to indexing (when threat triage is running) those fields have .'s not :'s Therefore, your rules should be:

Re: Stellar support for switch/case style conditionals

it. > > So default would be a keyword? > > and a lambda that uses x can be used on the right side of the : > > > > On October 17, 2017 at 14:21:01, Casey Stella (ceste...@gmail.com) wrote: > > So, just to map this onto the example, you mean: > match(longer_variable ->

Re: Stellar support for switch/case style conditionals

nd it’s value is returned > * no matches returns null or return of optional final statement, which is > a LAMBDA without a BOOLEAN_STATEMENT > > > On October 17, 2017 at 12:06:05, Casey Stella (ceste...@gmail.com) wrote: > > Ugh, I forgot to preface this with DISCUSS: Sorry! &

Re: Stellar support for switch/case style conditionals

There's no string concat candy, so that'd probably be default: FORMAT('critical-%s%s', x, y) On Tue, Oct 17, 2017 at 2:28 PM, Otto Fowler wrote: > match(longer_variable -> x, other_variable -> y) { x < 10 : 'info', x <= > 20 : 'warn’, x < y : ‘oh boy’, default:

Re: Master build failures in Travis

Yeah, that could be a consequence. With the cache in place, the calls to delete are async. This isn't generally a problem in an actual installation, but in the integration tests, it can take some time to sync up (depending on the load). I ran it 20 or so times teasing these out, but it's never

Re: Master build failures in Travis

Looks like Ryan got there first, which is awesome. Thanks for cleaning up my mess :) On Mon, Oct 23, 2017 at 10:04 AM, Casey Stella <ceste...@gmail.com> wrote: > Yeah, that could be a consequence. With the cache in place, the calls to > delete are async. This isn't general

Re: [DISCUSS] Upcoming Release

I'd say that if a release is this imminent that we had better notify the release manager who will make a release announcement, Nick. Matt, are you tuning in to this? On Wed, Nov 15, 2017 at 10:04 AM, Nick Allen wrote: > Hi Guys - > > I want to follow-up on this discussion.

Re: [DISCUSS] e2e test infrastructure

Honestly, I'm ok with either the in-memory component approach or the docker approach as long as: - It runs in travis - The infrastructure components are spun up in a way that isolates their classpath - The UI e2e test and the integration tests both use the same infrastructure I

Re: [DISCUSS] Upcoming Release

I would be in favor of a release at this point. On Mon, Dec 4, 2017 at 4:57 PM, Matt Foley wrote: > Hey all, > I see METRON-1252 was resolved over the weekend. Shall I go ahead and > start the process with 0.4.2 release? > Does anyone have any commits they feel strongly

Re: DISCUSS: Quick change to parser config

So, just chiming in here. It seems to me that we have a problem with extraneous fields in a couple of different ways: * Temporary Variables I think that the problem of temporary variables is one beyond just the parser. What I'd like to see is the Stellar field transformations operate similar

Re: [DISCUSS] Stellar Documentation Autogeneration

chiming in with a +1 on my end too. This would be fantastic. On Thu, Dec 14, 2017 at 2:51 PM, Nick Allen wrote: > +1 I think it is a great idea, Justin and the only way that we'll keep the > docs in-sync with the code. > > > > > > On Thu, Dec 14, 2017 at 2:32 PM Justin Leet

Re: [DISCUSS] Support Ubuntu Installs in the MPack

Nick is right that the ASF does not provide support in an explicit way (i.e. there are no pathways to get *prioritized* support via SLAs, etc.), but it is expected that apache projects provide support via mailing lists and answered by volunteers. Specifically, this is the crux of the "community

Re: [DISCUSS] Upcoming Release

, Matt Foley <mfo...@hortonworks.com> wrote: > Perhaps under “build_utils” we should add a subdirectory for > “release_utils”. > > From: Casey Stella <ceste...@gmail.com> > Date: Friday, December 15, 2017 at 10:50 AM > To: "dev@metron.apache.org" <d

Re: [DISCUSS] Upcoming Release

METRON-1228Done > Unassigned https://issues.apache.org/jira/browse/METRON-1228 > <https://issues.apache.org/jira/browse/METRON-1228>* > *METRON-1218 To Do Ryan > Merriman https://issues.apache.org/jira/browse/METRON-1

Re: [VOTE] Metron Release Candidate 0.4.2-RC2

+1 validated via Otto's script * Checksums * Sigs * Build * Full dev validation On Tue, Dec 19, 2017 at 2:45 PM, Nick Allen wrote: > +1 I validated using Otto's great script. > > * Validated the list of changes > * Checksums > * Sigs > * Build > * Tests > * Full Dev > > On

Re: [DISCUSS] Stellar in a Zeppelin Notebook

I love it! I wonder if we could get more of the REPL-like experience (i.e. I crave autocomplete ;) if we integrated it with jquery shell like they did with nanook (https://github.com/aeshell/nanook). I know zeppelin lets you integrate with more complex javascript. Regardless, this is awesome,

Re: [DISCUSS] Release Procedure + 'Kafka Plugin for Bro'

I'd recommend restarting this thread with this subject and including [MENTORS] in the subject line. At least I don't know the answer to this and I'd want broader visibility so we get more responses. On Thu, Nov 16, 2017 at 9:10 AM, Nick Allen wrote: > The code of the 'Kafka

New PMC members

The Project Management Committee (PMC) for Apache Impala has invited Otto Fowler, Michael Miklavcic and Justin Leet to become a PMC member and we are pleased to announce that they have accepted. Congratulations and welcome!

Re: New PMC members

Well, obviously, I meant Metron instead of Impala. To this point, we should have a wiki page around templates for this, similar to the impala project. :) On Thu, Dec 7, 2017 at 10:06 AM, Casey Stella <ceste...@gmail.com> wrote: > The Project Management Committee (PMC) for Apache I

Re: MASTER is Broken

I got it. I made a JIRA and have a fix in the middle of testing: https://issues.apache.org/jira/browse/METRON-1280 On Wed, Oct 25, 2017 at 3:09 PM, Otto Fowler wrote: > https://travis-ci.org/search/apache%252Fmetron > > I believe it is to do with

Re: MASTER is Broken

sions to set > that up though, so if anyone does that'd be super helpful. > > On Wed, Oct 25, 2017 at 3:14 PM, Casey Stella <ceste...@gmail.com> wrote: > > > I noticed it on my personal travis the when I merged in apache master. > > BTW: the PR for the fix is

Re: MASTER is Broken

who does? > > > On October 25, 2017 at 15:12:46, Otto Fowler (ottobackwa...@gmail.com) > wrote: > > Thanks Casey > > > On October 25, 2017 at 15:11:34, Casey Stella (ceste...@gmail.com) wrote: > > I got it. I made a JIRA and have a fix in the middle of testing: > http

Re: Stellar support for switch/case style conditionals

nly would prefer > to use the less explicit syntax ( e.g. match { foo == 0: true, bar == 1: > false, default: false } ). > > -Kyle > > On Tue, Oct 24, 2017 at 11:30 AM, Casey Stella <ceste...@gmail.com> wrote: > > > So, I do like it. My only issue is the explicit

Re: [DISCUSS] Resources for how to contribute to Apache Metron

That's really good feedback, Jon. I agree that we have a significant barrier to get to the point of tinkering. Full-dev really wasn't intended to be that entry point; it's more of a way to test PRs in something resembling a realistic scenario (and it is still not super realistic). I would

[DISCUSS] Release?

Is it about time for a release? I know we got some substantial performance changes in since the last release. I think we might have a justification for a release. Casey

Re: [DISCUSS] Release?

I wasn't aware we had a script for that..is that in dev-utilities/release-utils? On Wed, May 9, 2018 at 11:41 AM Otto Fowler <ottobackwa...@gmail.com> wrote: > Can you run the issues included script and post that for us to see? > > > On May 9, 2018 at 11:14:11, Casey Stella (

Re: [DISCUSS] Release?

n#859 > 5 months ago METRON-1349 Full Dev Builds Metron Twice (nickwallen) closes > apache/metron#866 > 5 months ago METRON-1343 Swagger UI for User Controller needs request > method (MohanDV via ottobackwards) closes apache/metron#862 > 5 months ago METRON-1306: When index template install

Re: [DISCUSS] Release?

e/metron#888 > > >> > > > > >> 3 months ago METRON-1432 JDK Install Fails on Ubuntu > > >> Development > > >> > > > > >> Environment (nickwallen) closes apache/metron#913 > > >> > >

Re: [DISCUSS] Pcap UI user requirements

A couple of thoughts on cluster overuse: * Definitely can't pause/resume MR jobs, unfortunately * The traditional approach to managing overuse of cluster resources and prioritization in Yarn is via the scheduler. I'd suggest rather than building this ourselves, we allow users to be associated

  1   2   >