[GitHub] metron issue #857: METRON-1340: Improve e2e tests for metron alerts

Github user ottobackwards commented on the issue: https://github.com/apache/metron/pull/857 What is the status of this pr? it is 29 day without comment, and conflicted, literally, and perhaps figuratively ---

[GitHub] metron issue #915: METRON-1433: Only emit debugging timing fields in enrichm...

Github user cestella commented on the issue: https://github.com/apache/metron/pull/915 @mraliagha Well, if it's being used explicitly and on an on-going manner, then I'll remove this PR. Do you think there's some value to being able to slice and dice in the global config the ability

Re: [DISCUSS] Move SHELL type functions from management to stellar common

> I think we should move the shell/console type functions from stellar What functions specifically? Are you talking about `metron-platform/metron-management`? If you are talking about he functions like 'CONFIG_GET' and 'CONFIG_PUT', those seem specific to interacting with Metron. They don't

[GitHub] metron pull request #856: METRON-1339 Stellar Shell functionality to verify ...

Github user ottobackwards commented on a diff in the pull request: https://github.com/apache/metron/pull/856#discussion_r165048687 --- Diff: metron-stellar/stellar-common/src/main/java/org/apache/metron/stellar/common/utils/validation/StellarZookeeperBasedValidator.java --- @@

Re: [DISCUSS] Move SHELL type functions from management to stellar common

I'd be in favor of that. That is general purpose stuff. On Wed, Jan 31, 2018 at 9:12 AM, Otto Fowler wrote: > Per: https://issues.apache.org/jira/browse/METRON-876 > > I think we should move the shell/console type functions from stellar > management to stellar-common,

Re: [DISCUSS] Move SHELL type functions from management to stellar common

Agreed On Jan 31, 2018 7:51 AM, "Justin Leet" wrote: > Agreed, I think it makes sense to move them there. > > On Wed, Jan 31, 2018 at 9:28 AM, Casey Stella wrote: > > > I'd be in favor of that. That is general purpose stuff. > > > > On Wed, Jan 31,

[GitHub] metron issue #915: METRON-1433: Only emit debugging timing fields in enrichm...

Github user mmiklavc commented on the issue: https://github.com/apache/metron/pull/915 Perhaps we can leverage parts of the performance logging code for this purpose? The frequency/sampling bit was something I had wanted to add for some time now.

Re: [DISCUSS] Move SHELL type functions from management to stellar common

Agreed, I think it makes sense to move them there. On Wed, Jan 31, 2018 at 9:28 AM, Casey Stella wrote: > I'd be in favor of that. That is general purpose stuff. > > On Wed, Jan 31, 2018 at 9:12 AM, Otto Fowler > wrote: > > > Per:

Re: [DISCUSS] Move SHELL type functions from management to stellar common

I assumed he was talking about the SHELL_EDIT stuff and maybe the file loading bits. The config stuff is metron specific On Wed, Jan 31, 2018 at 10:06 AM, Nick Allen wrote: > > I think we should move the shell/console type functions from stellar > > What functions

Re: [DISCUSS] Move SHELL type functions from management to stellar common

And File loading, as casey stated On January 31, 2018 at 10:18:22, Otto Fowler (ottobackwa...@gmail.com) wrote: Yeah, ShellFunctions to start. Later, KafkaFunctions Then GrokFunctions. I would propose to split the loading of grok out from the eval for that etc. On January 31, 2018 at

Re: [DISCUSS] Move SHELL type functions from management to stellar common

Yeah, ShellFunctions to start. Later, KafkaFunctions Then GrokFunctions. I would propose to split the loading of grok out from the eval for that etc. On January 31, 2018 at 10:14:45, Casey Stella (ceste...@gmail.com) wrote: I assumed he was talking about the SHELL_EDIT stuff and maybe the

Re: When things change in hdfs, how do we know

Hmm, I have heard this feedback before. Perhaps a more low-key approach would be either a static timer that checked or a timer bolt that sent a periodic timer and the parser bolt reconfigured the parser (or indeed we added a Reloadable interface with a 'reload' method). We could be smart also

Re: When things change in hdfs, how do we know

I don’t think the Unstable means the implementation will crash. I think it means it is a newish-api, and there should be 1 listeners. Having 1 listener shouldn’t be an issue. On January 31, 2018 at 11:45:54, Casey Stella (ceste...@gmail.com) wrote: Hmm, I have heard this feedback before.

Re: When things change in hdfs, how do we know

I take it your service would just be a thin daemon along the lines of the PoC you linked, which makes a lot of sense, delegating the actual notification to the zookeeper bits we already have. That makes sense to me. One other question would be around the availability of that service (which is

Re: When things change in hdfs, how do we know

Well, it'll be one listener per worker and if you have a lot of workers, it's going to be a bad time probably. On Wed, Jan 31, 2018 at 11:50 AM, Otto Fowler wrote: > I don’t think the Unstable means the implementation will crash. I think > it means > it is a

Re: When things change in hdfs, how do we know

No, I would propose a new Ambari Service, the did the notify->zookeeper stuff. Did you not see my awesome ascii art diagram? On January 31, 2018 at 11:51:51, Casey Stella (ceste...@gmail.com) wrote: Well, it'll be one listener per worker and if you have a lot of workers, it's going to be a

[GitHub] metron pull request #911: METRON-1419: Create a SolrDao

Github user merrimanr commented on a diff in the pull request: https://github.com/apache/metron/pull/911#discussion_r165141296 --- Diff: metron-platform/metron-indexing/src/test/java/org/apache/metron/indexing/dao/SearchIntegrationTest.java --- @@ -655,83 +699,54 @@ public void

[GitHub] metron pull request #911: METRON-1419: Create a SolrDao

Github user merrimanr commented on a diff in the pull request: https://github.com/apache/metron/pull/911#discussion_r165141322 --- Diff: metron-platform/metron-solr/src/main/java/org/apache/metron/solr/dao/SolrDao.java --- @@ -0,0 +1,118 @@ +/** + * Licensed to the Apache

Re: When things change in hdfs, how do we know

It depends if the transaction id is the same per instance. If it is, then we can de-doupe as long as we put the tx id into the event json. On January 31, 2018 at 12:40:15, Simon Elliston Ball ( si...@simonellistonball.com) wrote: I take it your service would just be a thin daemon along the

[GitHub] metron pull request #911: METRON-1419: Create a SolrDao

Github user merrimanr commented on a diff in the pull request: https://github.com/apache/metron/pull/911#discussion_r165142091 --- Diff: metron-platform/metron-solr/src/test/java/org/apache/metron/solr/integration/components/SolrComponent.java --- @@ -158,4 +162,16 @@ public

[GitHub] metron pull request #911: METRON-1419: Create a SolrDao

Github user merrimanr commented on a diff in the pull request: https://github.com/apache/metron/pull/911#discussion_r165141979 --- Diff: metron-platform/metron-solr/src/main/java/org/apache/metron/solr/dao/SolrSearchDao.java --- @@ -0,0 +1,315 @@ +/** + * Licensed to the

[GitHub] metron pull request #911: METRON-1419: Create a SolrDao

Github user merrimanr commented on a diff in the pull request: https://github.com/apache/metron/pull/911#discussion_r165142049 --- Diff: metron-platform/metron-solr/src/test/java/org/apache/metron/solr/integration/SolrSearchIntegrationTest.java --- @@ -0,0 +1,152 @@ +/**

[GitHub] metron pull request #911: METRON-1419: Create a SolrDao

Github user merrimanr commented on a diff in the pull request: https://github.com/apache/metron/pull/911#discussion_r165141915 --- Diff: metron-platform/metron-solr/src/main/java/org/apache/metron/solr/dao/SolrSearchDao.java --- @@ -0,0 +1,315 @@ +/** + * Licensed to the

[GitHub] metron pull request #918: METRON-1436: Manually Install Solr Cloud in Full D...

Github user justinleet commented on a diff in the pull request: https://github.com/apache/metron/pull/918#discussion_r165192051 --- Diff: metron-platform/metron-solr/README.md --- @@ -0,0 +1,52 @@ + +# Solr in Metron + +## Table of Contents + +*

[GitHub] metron issue #918: METRON-1436: Manually Install Solr Cloud in Full Dev

Github user merrimanr commented on the issue: https://github.com/apache/metron/pull/918 I tested it out and worked fine. I think it's a good start. +1 as long as other commenters are satisfied. ---

Re: [DISCUSS] Quick Poll - How You are Using & Contributing to Metron? - 2018-01-30

Hello Metron developers and users; Here are the results of the survey so far. --- Where respondents are located: Canada, USA Types of data collected: Various Logs, Honeypot Data, Firewall, IDS, Server Data Kind of

[GitHub] metron pull request #911: METRON-1419: Create a SolrDao

Github user merrimanr commented on a diff in the pull request: https://github.com/apache/metron/pull/911#discussion_r165141548 --- Diff: metron-platform/metron-solr/src/main/java/org/apache/metron/solr/dao/SolrSearchDao.java --- @@ -0,0 +1,315 @@ +/** + * Licensed to the