[GitHub] metron issue #869: METRON-1362 Improve Metron Deployment README

Github user justinleet commented on the issue: https://github.com/apache/metron/pull/869 I'm still +1 after the latest changes. Thanks @nickwallen! ---

[GitHub] metron pull request #869: METRON-1362 Improve Metron Deployment README

Github user nickwallen commented on a diff in the pull request: https://github.com/apache/metron/pull/869#discussion_r159434045 --- Diff: metron-deployment/packaging/ambari/metron-mpack/README.md --- @@ -0,0 +1,127 @@ + + +This provides a Management Pack (MPack)

[GitHub] metron pull request #869: METRON-1362 Improve Metron Deployment README

Github user anandsubbu commented on a diff in the pull request: https://github.com/apache/metron/pull/869#discussion_r159430163 --- Diff: metron-deployment/packaging/ambari/metron-mpack/README.md --- @@ -0,0 +1,127 @@ + + +This provides a Management Pack (MPack)

[GitHub] metron pull request #869: METRON-1362 Improve Metron Deployment README

Github user anandsubbu commented on a diff in the pull request: https://github.com/apache/metron/pull/869#discussion_r159426522 --- Diff: metron-deployment/packaging/ambari/metron-mpack/README.md --- @@ -0,0 +1,127 @@ + + +This provides a Management Pack (MPack)

[GitHub] metron pull request #869: METRON-1362 Improve Metron Deployment README

Github user nickwallen commented on a diff in the pull request: https://github.com/apache/metron/pull/869#discussion_r159433756 --- Diff: metron-deployment/packaging/ambari/metron-mpack/README.md --- @@ -0,0 +1,127 @@ + + +This provides a Management Pack (MPack)

[GitHub] metron issue #869: METRON-1362 Improve Metron Deployment README

Github user anandsubbu commented on the issue: https://github.com/apache/metron/pull/869 +1 (non-binding). Looks good @nickwallen ! ---

[GitHub] metron pull request #869: METRON-1362 Improve Metron Deployment README

Github user nickwallen commented on a diff in the pull request: https://github.com/apache/metron/pull/869#discussion_r159431190 --- Diff: metron-deployment/packaging/ambari/metron-mpack/README.md --- @@ -0,0 +1,127 @@ + + +This provides a Management Pack (MPack)

[GitHub] metron pull request #869: METRON-1362 Improve Metron Deployment README

Github user nickwallen commented on a diff in the pull request: https://github.com/apache/metron/pull/869#discussion_r159431002 --- Diff: metron-deployment/packaging/ambari/metron-mpack/README.md --- @@ -0,0 +1,127 @@ + + +This provides a Management Pack (MPack)

[GitHub] metron pull request #869: METRON-1362 Improve Metron Deployment README

Github user asfgit closed the pull request at: https://github.com/apache/metron/pull/869 ---

[GitHub] metron issue #884: METRON-1382 Run Stellar in a Zeppelin Notebook

Github user nickwallen commented on the issue: https://github.com/apache/metron/pull/884 The problem is on the UI side of things; for both Zeppelin and the CLI. When I get the result back from Stellar, I was using `ConversionUtils.convert(value, String.class)` to get me a result

[GitHub] metron issue #884: METRON-1382 Run Stellar in a Zeppelin Notebook

Github user merrimanr commented on the issue: https://github.com/apache/metron/pull/884 I also noticed that functions returning a list only display the first item in both the shell and Zeppelin output. For example, the expression `MAP([ 'foo', 'bar'], (x) -> TO_UPPER(x) )`

Re: [DISCUSS] Generating and Interacting with serialized summary objects

Thanks for the feedback, Nick. Regarding "IMHO, I'd rather not reinvent the wheel for text manipulation." I would argue that we are not reinventing the wheel for text manipulation as the extractor config exists already and we are doing a similar thing in the flatfile loader (in fact, the code is

Re: [DISCUSS] Generating and Interacting with serialized summary objects

> Even with 5 threads, it takes an hour for the full Alexa 1m, so I think this will impact performance What exactly takes an hour? Adding 1M entries to a bloom filter? That seems really high, unless I am not understanding something. On Wed, Jan 3, 2018 at 10:17 AM, Casey Stella

[GitHub] metron issue #884: METRON-1382 Run Stellar in a Zeppelin Notebook

Github user nickwallen commented on the issue: https://github.com/apache/metron/pull/884 > @merrimanr: Is there a way to pass in the zookeeper url in Zeppelin? No, I did not implement that in this PR. As a next step I was going to do whatever needs done to get the management

[GitHub] metron issue #884: METRON-1382 Run Stellar in a Zeppelin Notebook

Github user nickwallen commented on the issue: https://github.com/apache/metron/pull/884 Thanks for digging into it @merrimanr . You uncovered some good stuff. > I found that the functions available in Zeppelin are a subset of what's available in the Stellar shell. The

[GitHub] metron issue #884: METRON-1382 Run Stellar in a Zeppelin Notebook

Github user merrimanr commented on the issue: https://github.com/apache/metron/pull/884 I think this is an excellent start. So far I have only reviewed it from a user perspective and it's working well so far. I've spun it up in full dev (not sure that's even necessary) and

[GitHub] metron issue #884: METRON-1382 Run Stellar in a Zeppelin Notebook

Github user nickwallen commented on the issue: https://github.com/apache/metron/pull/884 > @merrimanr: From what I can tell the problem is only in how the result is displayed. For example, the expression 'BAR' in MAP([ 'foo', 'bar'], (x) -> TO_UPPER(x) ) returns true as expected.

[GitHub] metron issue #884: METRON-1382 Run Stellar in a Zeppelin Notebook

Github user nickwallen commented on the issue: https://github.com/apache/metron/pull/884 @merrimanr Fixed that bug and was able to add a good amount of unit tests around it. Also figured out a way to unit test the Aesh-driven StellarShell class. ---

Re: [DISCUSS] Generating and Interacting with serialized summary objects

I just went through these pull requests as well and also agree this is good work. I think it's a good first pass. I would be careful with trying to boil the ocean here. I think for the initial use case I would only support loading the bloom filters from HDFS. If people want to pre-process

Re: Metron Alert UI and zero-down time Elasticsearch re-index

Hi Ali, I am not sure I understand what you are trying to do. Are you trying to change the name on the old index, add it to the alias, and then re-index and give the new index the name of the old index? 01.01.2018, 22:30, "Ali Nazemian" : > Hi All, > > We are using an

Re: [DISCUSS] Generating and Interacting with serialized summary objects

It's actually many more than 1M. There are 1M domains, each domain could have upwards of 300 - 1000 possible typosquatted domains. You will notice from https://github.com/cestella/incubator-metron/tree/typosquat_merge/use-cases/typosquat_detection#generate-the-bloom-filter that we are not adding

Re: [DISCUSS] Generating and Interacting with serialized summary objects

Oh, gotcha. That makes sense. Thanks for clarifying. On Wed, Jan 3, 2018 at 12:15 PM, Casey Stella wrote: > It's actually many more than 1M. There are 1M domains, each domain could > have upwards of 300 - 1000 possible typosquatted domains. > > You will notice from >

Re: [DISCUSS] Generating and Interacting with serialized summary objects

There is some really cool stuff happening here, if only I’d been allowed to see the lists over Christmas... :) A few thoughts... I like Otto’s generalisation of the problem to include specific local stellar objects in a cache loaded from a store (HDFS seems a natural, but not only place,

[GitHub] metron pull request #887: METRON-1388 update public web site to point at 0.4...

GitHub user mattf-horton opened a pull request: https://github.com/apache/metron/pull/887 METRON-1388 update public web site to point at 0.4.2 new release ## Contributor Comments Update the public web site to point at new release 0.4.2 (to be pushed simultaneously with the

[GitHub] metron issue #884: METRON-1382 Run Stellar in a Zeppelin Notebook

Github user ottobackwards commented on the issue: https://github.com/apache/metron/pull/884 This is quite a bit to go over can I trade you a similarly large review task? ---

[GitHub] metron issue #873: METRON-1367 Stellar should have some instrumentation of f...

Github user ottobackwards commented on the issue: https://github.com/apache/metron/pull/873 This has been submitted to Commons Lang : https://github.com/apache/commons-lang/pull/311 The version there is refactored and also java 7 comparable ( so I had to lose Optional and Lambdas

Re: [DISCUSS] Generating and Interacting with serialized summary objects

I'm liking this design and growth strategy, Casey. I also think Nick and Otto have some valid points. I always find there's a natural tension between too little, just enough, and boiling the ocean and these discuss threads really help drive what the short and long term visions should look like.

[GitHub] metron pull request #884: METRON-1382 Run Stellar in a Zeppelin Notebook

Github user nickwallen commented on a diff in the pull request: https://github.com/apache/metron/pull/884#discussion_r15947 --- Diff: metron-stellar/stellar-zeppelin/src/main/java/org/apache/metron/stellar/zeppelin/StellarInterpreter.java --- @@ -0,0 +1,158 @@ +/* + *

[GitHub] metron pull request #884: METRON-1382 Run Stellar in a Zeppelin Notebook

Github user nickwallen commented on a diff in the pull request: https://github.com/apache/metron/pull/884#discussion_r159497124 --- Diff: metron-stellar/stellar-common/src/main/java/org/apache/metron/stellar/common/shell/StellarExecutionListeners.java --- @@ -0,0 +1,51 @@ +/*

[GitHub] metron pull request #884: METRON-1382 Run Stellar in a Zeppelin Notebook

Github user nickwallen commented on a diff in the pull request: https://github.com/apache/metron/pull/884#discussion_r159497380 --- Diff: metron-stellar/stellar-common/src/main/java/org/apache/metron/stellar/common/shell/StellarExecutionNotifier.java --- @@ -0,0 +1,44 @@ +/*

[GitHub] metron pull request #884: METRON-1382 Run Stellar in a Zeppelin Notebook

Github user nickwallen commented on a diff in the pull request: https://github.com/apache/metron/pull/884#discussion_r159498954 --- Diff: metron-stellar/stellar-common/src/main/java/org/apache/metron/stellar/common/shell/specials/QuitCommand.java --- @@ -0,0 +1,51 @@ +/*

[GitHub] metron pull request #884: METRON-1382 Run Stellar in a Zeppelin Notebook

Github user nickwallen commented on a diff in the pull request: https://github.com/apache/metron/pull/884#discussion_r159498085 --- Diff: metron-stellar/stellar-common/src/main/java/org/apache/metron/stellar/common/shell/StellarResult.java --- @@ -0,0 +1,185 @@ +/* + *

[GitHub] metron pull request #884: METRON-1382 Run Stellar in a Zeppelin Notebook

Github user nickwallen commented on a diff in the pull request: https://github.com/apache/metron/pull/884#discussion_r159495486 --- Diff: metron-platform/metron-management/src/main/java/org/apache/metron/management/ShellFunctions.java --- @@ -90,19 +95,19 @@ public Object

[GitHub] metron pull request #884: METRON-1382 Run Stellar in a Zeppelin Notebook

Github user nickwallen commented on a diff in the pull request: https://github.com/apache/metron/pull/884#discussion_r159496039 --- Diff: metron-stellar/stellar-common/README.md --- @@ -1377,7 +1377,7 @@ IS_EMAIL To run the Stellar Shell directly from the Metron source code,

[GitHub] metron pull request #884: METRON-1382 Run Stellar in a Zeppelin Notebook

Github user nickwallen commented on a diff in the pull request: https://github.com/apache/metron/pull/884#discussion_r159496645 --- Diff: metron-stellar/stellar-common/src/main/java/org/apache/metron/stellar/common/shell/DefaultStellarShellExecutor.java --- @@ -0,0 +1,398 @@

[GitHub] metron pull request #884: METRON-1382 Run Stellar in a Zeppelin Notebook

Github user nickwallen commented on a diff in the pull request: https://github.com/apache/metron/pull/884#discussion_r159498766 --- Diff: metron-stellar/stellar-common/src/main/java/org/apache/metron/stellar/common/shell/specials/DocCommand.java --- @@ -0,0 +1,105 @@ +/*

[GitHub] metron pull request #884: METRON-1382 Run Stellar in a Zeppelin Notebook

Github user nickwallen commented on a diff in the pull request: https://github.com/apache/metron/pull/884#discussion_r159499691 --- Diff: metron-stellar/stellar-zeppelin/src/main/java/org/apache/metron/stellar/zeppelin/StellarInterpreter.java --- @@ -0,0 +1,158 @@ +/* + *

Re: [DISCUSS] Generating and Interacting with serialized summary objects

I just finished stepping through the typosquatting use case README in your merge branch. This is really, really good work Casey. I see most of our previous documentation issues addressed up front, e.g. special variables are cited, all new fields explained, side effects documented. The use case doc