[GitHub] metron issue #940: METRON-1460: Create a complementary non-split-join enrich...

Github user mraliagha commented on the issue: https://github.com/apache/metron/pull/940 Is there any document somewhere to show how the previous approach was implemented? I would like to understand the previous architecture in details. Becuase some of the pros/cons didn't make sense

[GitHub] metron pull request #853: METRON-1337: List of facets should not be hardcode...

Github user ottobackwards commented on a diff in the pull request: https://github.com/apache/metron/pull/853#discussion_r170125479 --- Diff: metron-interface/metron-rest/src/main/java/org/apache/metron/rest/service/impl/AlertServiceImpl.java --- @@ -37,15 +47,21 @@ @Service

[GitHub] metron pull request #853: METRON-1337: List of facets should not be hardcode...

Github user merrimanr commented on a diff in the pull request: https://github.com/apache/metron/pull/853#discussion_r170120909 --- Diff: metron-interface/metron-rest/src/main/java/org/apache/metron/rest/service/impl/AlertServiceImpl.java --- @@ -37,15 +47,21 @@ @Service

[GitHub] metron pull request #858: METRON-1344: Externalize the infrastructural compo...

Github user merrimanr closed the pull request at: https://github.com/apache/metron/pull/858 ---

[GitHub] metron pull request #941: METRON-1355: Convert metron-elasticsearch to new i...

GitHub user merrimanr opened a pull request: https://github.com/apache/metron/pull/941 METRON-1355: Convert metron-elasticsearch to new infrastructure ## Contributor Comments This PR switches metron-elasticsearch integration tests from using in-memory components to the e2e

[GitHub] metron pull request #940: Single bolt split join poc

Github user cestella commented on a diff in the pull request: https://github.com/apache/metron/pull/940#discussion_r170089790 --- Diff: metron-platform/metron-enrichment/src/main/java/org/apache/metron/enrichment/bolt/UnifiedEnrichmentBolt.java --- @@ -0,0 +1,323 @@ +/**

[GitHub] metron pull request #940: Single bolt split join poc

Github user cestella commented on a diff in the pull request: https://github.com/apache/metron/pull/940#discussion_r170057327 --- Diff: metron-platform/metron-enrichment/src/main/java/org/apache/metron/enrichment/bolt/UnifiedEnrichmentBolt.java --- @@ -0,0 +1,323 @@ +/**

Re: [DISCUSS] Alternatives to split/join enrichment

FYI, the PR for this is up at https://github.com/apache/metron/pull/940 For those interested, please comment on the actual implementation there. On Thu, Feb 22, 2018 at 12:43 PM, Casey Stella wrote: > So, these are good questions, as usual Otto :) > > > how does this effect

[GitHub] metron pull request #940: Single bolt split join poc

GitHub user cestella opened a pull request: https://github.com/apache/metron/pull/940 Single bolt split join poc ## Contributor Comments There are some deficiencies to the split/join topology. It's hard to reason about * Understanding the latency of enriching a

Re: [DISCUSS] Alternatives to split/join enrichment

So, these are good questions, as usual Otto :) > how does this effect the distribution of work through the cluster, and resiliency of the topologies? This moves us to a data parallelism scheme rather than a task parallelism scheme. This, in effect means, that we will not be distributing the

Re: [DISCUSS] Alternatives to split/join enrichment

Also, how are we to measure the effect? Not to get all six sigma ;) On February 22, 2018 at 11:48:41, Otto Fowler (ottobackwa...@gmail.com) wrote: This sounds worth exploring. A couple of questions: * how does this effect the distribution of work through the cluster, and resiliency of the

Re: [DISCUSS] Alternatives to split/join enrichment

This sounds worth exploring. A couple of questions: * how does this effect the distribution of work through the cluster, and resiliency of the topologies? * Is anyone else doing it like this? * Can we have multiple thread pools and group tasks together ( or separate them ) wrt hbase? On

[DISCUSS] Alternatives to split/join enrichment

Hi all, I've been thinking and working on something that I wanted to get some feedback on. The way that we do our enrichments, the split/join architecture was created to effectively to parallel enrichments in a storm-like way in contrast to OpenSoc. There are some good parts to this

[GitHub] metron issue #924: METRON-1299 In MetronError tests, don't test for HostName...

Github user merrimanr commented on the issue: https://github.com/apache/metron/pull/924 +1 pending @cestella's approval. Thanks @ottobackwards. ---