[GitHub] metron issue #923: METRON-1442: Split rest end points for indexing topology ...

2018-02-02 Thread MohanDV
Github user MohanDV commented on the issue: https://github.com/apache/metron/pull/923 @cestella I merged your PR ---

[GitHub] metron issue #923: METRON-1442: Split rest end points for indexing topology ...

2018-02-02 Thread cestella
Github user cestella commented on the issue: https://github.com/apache/metron/pull/923 Looks like you didn't quite get all of the mock test infrastructure for `metron-rest` set up properly. I went ahead and submitted a PR against your branch to help out. :) If you can merge

[GitHub] metron issue #922: METRON-1441: Create complementary Solr schemas for the ma...

2018-02-02 Thread merrimanr
Github user merrimanr commented on the issue: https://github.com/apache/metron/pull/922 I tested this in full dev using the install script in https://github.com/apache/metron/pull/918. I was able to create collections for each schema except for "error". For that to work properly, I

Re: [DISCUSS] Profiler Enhancement

2018-02-02 Thread Otto Fowler
You know, I am going to back this up. I usually thing of replay as replay, profiler or not, but that is not true. Replay of data through the full pipeline (parsers/enrichement) has more consequences or concerns, so we can drop this. I don’t want to expand the scope of your idea. We can

Re: [DISCUSS] Profiler Enhancement

2018-02-02 Thread Nick Allen
I think that is definitely a reasonable extension. In this case would we need any additional actions to indicate that data will be overwritten? I am trying to think of other additional needs that this use case has over the others. On Feb 2, 2018 12:38 PM, "Otto Fowler"

[GitHub] metron pull request #925: METRON-1443 Missing Critical MPack Install Instruc...

2018-02-02 Thread nickwallen
GitHub user nickwallen opened a pull request: https://github.com/apache/metron/pull/925 METRON-1443 Missing Critical MPack Install Instruction for Ubuntu When installing Elasticsearch with the MPack on Ubuntu, you must manually install the Elasticsearch repositories. The Mpack

[GitHub] metron pull request #924: METRON-1299 In MetronError tests, don't test for H...

2018-02-02 Thread ottobackwards
GitHub user ottobackwards opened a pull request: https://github.com/apache/metron/pull/924 METRON-1299 In MetronError tests, don't test for HostName if getHostName wouldn't work MetronError ignores exceptions from InetAddress.getLocalHost().getHostName() and leaves the field

[GitHub] metron issue #922: METRON-1441: Create complementary Solr schemas for the ma...

2018-02-02 Thread cestella
Github user cestella commented on the issue: https://github.com/apache/metron/pull/922 Ok, I did the following: * Augmented the readme to point to the Solr documentation around schemas. Keep in mind, this is intermediate work that will feed into the "install Solr" work *

[GitHub] metron issue #865: METRON-1212 The bundle System and Maven Plugin (Feature B...

2018-02-02 Thread ottobackwards
Github user ottobackwards commented on the issue: https://github.com/apache/metron/pull/865 Hey @JonZeolla how is it going? ---

[GitHub] metron pull request #923: METRON-1442: Split rest end points for indexing to...

2018-02-02 Thread MohanDV
GitHub user MohanDV opened a pull request: https://github.com/apache/metron/pull/923 METRON-1442: Split rest end points for indexing topology into random access indexing and batch indexing ## Contributor Comments Split rest end points for indexing topology into random

[GitHub] metron pull request #922: METRON-1441: Create complementary Solr schemas for...

2018-02-02 Thread cestella
Github user cestella commented on a diff in the pull request: https://github.com/apache/metron/pull/922#discussion_r165711148 --- Diff: metron-platform/metron-solr/src/test/java/org/apache/metron/solr/schema/SchemaTranslatorTest.java --- @@ -0,0 +1,188 @@ +/** + *

Re: [DISCUSS] Profiler Enhancement

2018-02-02 Thread Otto Fowler
Scenario 3: As a Security ? I have modified a profile or parser configuration ( replay is replay ), and I want to run the new version against my old data. On February 2, 2018 at 12:19:54, Nick Allen (n...@nickallen.org) wrote: I have been thinking about an enhancement to the Profiler for

[DISCUSS] Profiler Enhancement

2018-02-02 Thread Nick Allen
I have been thinking about an enhancement to the Profiler for quite some time. Actually, my first pass at defining this was called "Replay Telemetry through Profiler" back in METRON-594 [1]. I'd like to first discuss the use case to make sure we start out on the right foot. Here is how I would

[GitHub] metron pull request #920: METRON-1438 Move SHELL functions from metron-manag...

2018-02-02 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/metron/pull/920 ---

[GitHub] metron pull request #918: METRON-1436: Manually Install Solr Cloud in Full D...

2018-02-02 Thread mmiklavc
Github user mmiklavc closed the pull request at: https://github.com/apache/metron/pull/918 ---

[GitHub] metron issue #920: METRON-1438 Move SHELL functions from metron-management t...

2018-02-02 Thread nickwallen
Github user nickwallen commented on the issue: https://github.com/apache/metron/pull/920 +1 Thanks @ottobackwards ---

[GitHub] metron pull request #920: METRON-1438 Move SHELL functions from metron-manag...

2018-02-02 Thread nickwallen
Github user nickwallen commented on a diff in the pull request: https://github.com/apache/metron/pull/920#discussion_r165663581 --- Diff: metron-stellar/stellar-common/src/test/java/org/apache/metron/stellar/dsl/functions/ShellFunctionsTest.java --- @@ -40,8 +45,8 @@ );

[GitHub] metron pull request #922: METRON-1441: Create complementary Solr schemas for...

2018-02-02 Thread ottobackwards
Github user ottobackwards commented on a diff in the pull request: https://github.com/apache/metron/pull/922#discussion_r165662614 --- Diff: metron-platform/metron-solr/src/test/java/org/apache/metron/solr/schema/SchemaTranslatorTest.java --- @@ -0,0 +1,188 @@ +/** + *

[GitHub] metron pull request #920: METRON-1438 Move SHELL functions from metron-manag...

2018-02-02 Thread ottobackwards
Github user ottobackwards commented on a diff in the pull request: https://github.com/apache/metron/pull/920#discussion_r165662017 --- Diff: metron-stellar/stellar-common/src/test/java/org/apache/metron/stellar/dsl/functions/ShellFunctionsTest.java --- @@ -40,8 +45,8 @@ );

[GitHub] metron pull request #920: METRON-1438 Move SHELL functions from metron-manag...

2018-02-02 Thread ottobackwards
Github user ottobackwards commented on a diff in the pull request: https://github.com/apache/metron/pull/920#discussion_r165661676 --- Diff: metron-stellar/stellar-common/src/test/java/org/apache/metron/stellar/dsl/functions/ShellFunctionsTest.java --- @@ -40,8 +45,8 @@ );

[GitHub] metron pull request #917: METRON-1435: Management UI cannot save json object...

2018-02-02 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/metron/pull/917 ---

[GitHub] metron pull request #920: METRON-1438 Move SHELL functions from metron-manag...

2018-02-02 Thread nickwallen
Github user nickwallen commented on a diff in the pull request: https://github.com/apache/metron/pull/920#discussion_r165660358 --- Diff: metron-stellar/stellar-common/src/main/java/org/apache/metron/stellar/common/shell/cli/PausableInput.java --- @@ -36,8 +37,8 @@ *

[GitHub] metron pull request #920: METRON-1438 Move SHELL functions from metron-manag...

2018-02-02 Thread ottobackwards
Github user ottobackwards commented on a diff in the pull request: https://github.com/apache/metron/pull/920#discussion_r165659854 --- Diff: metron-stellar/stellar-common/src/main/java/org/apache/metron/stellar/common/shell/cli/PausableInput.java --- @@ -36,8 +37,8 @@ *

[GitHub] metron issue #918: METRON-1436: Manually Install Solr Cloud in Full Dev

2018-02-02 Thread justinleet
Github user justinleet commented on the issue: https://github.com/apache/metron/pull/918 Thanks for the updates. I'm +1 on including this in the feature branch. ---

Re: [DISCUSS] Persistence store for user profile settings

2018-02-02 Thread Nick Allen
> Glad you agree with me that this isn’t HBase scale… it’s clearly not. I would never suggest introducing HBase for something like this, but since it’s there. Ah, gotcha. Misunderstood your statement. On Fri, Feb 2, 2018 at 9:01 AM Simon Elliston Ball < si...@simonellistonball.com> wrote: >

[GitHub] metron issue #922: METRON-1441: Create complementary Solr schemas for the ma...

2018-02-02 Thread cestella
Github user cestella commented on the issue: https://github.com/apache/metron/pull/922 @ottobackwards Very likely these schema files won't stay in this spot, but the final resting spot won't be apparent until we figure out how to automatically apply the schemas. Treat this PR as

[GitHub] metron pull request #922: METRON-1441: Create complementary Solr schemas for...

2018-02-02 Thread cestella
Github user cestella commented on a diff in the pull request: https://github.com/apache/metron/pull/922#discussion_r165656636 --- Diff: metron-platform/metron-solr/src/test/java/org/apache/metron/solr/schema/SchemaTranslatorTest.java --- @@ -0,0 +1,188 @@ +/** + *

[GitHub] metron pull request #922: METRON-1441: Create complementary Solr schemas for...

2018-02-02 Thread cestella
Github user cestella commented on a diff in the pull request: https://github.com/apache/metron/pull/922#discussion_r165656511 --- Diff: metron-platform/metron-solr/src/test/java/org/apache/metron/solr/schema/SchemaTranslatorTest.java --- @@ -0,0 +1,188 @@ +/** + *

[GitHub] metron pull request #920: METRON-1438 Move SHELL functions from metron-manag...

2018-02-02 Thread nickwallen
Github user nickwallen commented on a diff in the pull request: https://github.com/apache/metron/pull/920#discussion_r165654991 --- Diff: metron-stellar/stellar-common/src/main/java/org/apache/metron/stellar/common/shell/cli/PausableInput.java --- @@ -36,8 +37,8 @@ *

[GitHub] metron pull request #920: METRON-1438 Move SHELL functions from metron-manag...

2018-02-02 Thread nickwallen
Github user nickwallen commented on a diff in the pull request: https://github.com/apache/metron/pull/920#discussion_r165653517 --- Diff: metron-stellar/stellar-common/src/test/java/org/apache/metron/stellar/dsl/functions/ShellFunctionsTest.java --- @@ -40,8 +45,8 @@ );

Re: [DISCUSS] Persistence store for user profile settings

2018-02-02 Thread Simon Elliston Ball
Glad you agree with me that this isn’t HBase scale… it’s clearly not. I would never suggest introducing HBase for something like this, but since it’s there. On the idea of using the Ambari RDBMS for the same basis of it being there, I see your point. That said, it can be postgres, sql server,

Re: [DISCUSS] Persistence store for user profile settings

2018-02-02 Thread Nick Allen
I fall marginally on the side of an RDBMS. There is definitely a case to be made on both sides, but I'll point out a few things for the RDBMS. (1) Flexibility. Using an RDBMS is going to provide us with much greater flexibility going forward. We really don't know what the specific use cases

Re: [DISCUSS] Persistence store for user profile settings

2018-02-02 Thread Simon Elliston Ball
Couldn’t agree with you more Otto! On the perms / ACLs / AXOs / groups / users etc concerns though, there are other Apache projects (such as Ranger) which have already done a lot of the hard thinking and architecture / data structure / admin ui and persistence pieces for us, so I’d say we lean

Re: [DISCUSS] Persistence store for user profile settings

2018-02-02 Thread Otto Fowler
Fair enough, I don’t have a preference. I think my point is that we need to understand the use cases we can think of more, especially if we are going to be having permissions, grouping and crud around that, and preloading, before just throwing everything in RDBMS -or- HBASE. On February 2,

Re: [DISCUSS] Persistence store for user profile settings

2018-02-02 Thread Simon Elliston Ball
True, and that is a requirement I’ve heard a lot (standard views or field sets in shared sets of saved search for example). That would definitely rule out sticking with the current approach (browser local storage, per Casey’s suggestion below). That said, I’m not sure that changes my views on

Re: [DISCUSS] Persistence store for user profile settings

2018-02-02 Thread Otto Fowler
It is not uncommon to want to have ‘shared’ preferences or setups. Think of shared dashboards or queries vs. personal version in jira. Would RDBMS help with that? On February 2, 2018 at 07:17:04, Simon Elliston Ball ( si...@simonellistonball.com) wrote: Introducing a RDBMS to the stack

Re: [DISCUSS] Persistence store for user profile settings

2018-02-02 Thread Simon Elliston Ball
Introducing a RDBMS to the stack seems unnecessary for this. If we consider the data access patterns for user profiles, we are unlikely to query into them, or indeed do anything other than look them up, or write them out by a username key. To that end, using an ORM to translate a a nested

Re: Disable Metron parser output writer entirely

2018-02-02 Thread Otto Fowler
You cannot. On February 1, 2018 at 23:51:28, Ali Nazemian (alinazem...@gmail.com) wrote: Hi All, I am trying to investigate whether we can disable a Metron parser output writer entirely and manage it via KAFKA_PUT Stellar function instead. First, is it possible via configuration? Second, will