Re: [DISCUSS] Metron Alerts UI

2017-02-23 Thread Casey Stella
- What does "Escalate" do exactly? - Where does the Alert ID come from? - Are the fields displayed configurable? - It'd be nice to be able to select a set of fields for a message and have the list of messages filter to just those where those fields are the same as the one viewed.

[GitHub] incubator-metron issue #453: METRON-694: Index Errors from Topologies

2017-02-23 Thread JonZeolla
Github user JonZeolla commented on the issue: https://github.com/apache/incubator-metron/pull/453 I'm sorry if this was already covered and I missed it, I'm writing this from my phone. How does this handle errors during indexing itself? For instance, if the failure is due to the con

[DISCUSS] Metron Alerts UI

2017-02-23 Thread Houshang Livian
Hello Metron Community, We have mocked up an Alerts UI for Metron for your consideration. Please take a look and share your thoughts. Here is a link to our thoughts on this: http://imgur.com/a/KMTKN Does this look like a reasonable place to start? Is there anything that is an absolute MUST have

[GitHub] incubator-metron issue #316: METRON-503: Metron REST API

2017-02-23 Thread merrimanr
Github user merrimanr commented on the issue: https://github.com/apache/incubator-metron/pull/316 Hmm I just tried deploying on quick-dev the same way and was able to hit the stellar endpoints without issue. I will continue testing to see if I can get it to break tomorrow. --- If y

Re: new committer: Josh Meyer

2017-02-23 Thread zeo...@gmail.com
Congrats Josh! On Thu, Feb 23, 2017, 2:38 PM Otto Fowler wrote: > Nice! Congratulations! > > > On February 23, 2017 at 15:14:42, James Sirota (jsir...@apache.org) wrote: > > The Podling Project Management Committee (PPMC) for Apache Metron > (Incubating) > has asked Kyle Richardson to become a

Re: new committer: Josh Meyer

2017-02-23 Thread Otto Fowler
Nice! Congratulations! On February 23, 2017 at 15:14:42, James Sirota (jsir...@apache.org) wrote: The Podling Project Management Committee (PPMC) for Apache Metron (Incubating) has asked Kyle Richardson to become a committer and we are pleased to announce that they have accepted. Being a comm

Re: [DISCUSS] Top domains enrichment config/extractor management

2017-02-23 Thread David Lyle
I don't really have a firm opinion either way except for the normal default- if Ambari is going to own a part of it, it needs to maintain the full (load/update/upgrade/remove) lifecycle. So, it sounds like you've already decided you want Ambari to do the initial install and load. The consequence o

Re: new committer: Josh Meyer

2017-02-23 Thread Kyle Richardson
Welcome, Josh! -Kyle On Thu, Feb 23, 2017 at 3:17 PM, James Sirota wrote: > Oops. Sorry. Cut and paste error... These Apache templates are hard :) > > The Podling Project Management Committee (PPMC) for Apache Metron > (Incubating) > has asked Josh Meyer to become a committer and we are please

[GitHub] incubator-metron issue #316: METRON-503: Metron REST API

2017-02-23 Thread jjmeyer0
Github user jjmeyer0 commented on the issue: https://github.com/apache/incubator-metron/pull/316 @merrimanr I'm using quick-dev to do my tests. I built everything from metron root with `mvn clean install -DskipTests`. Then I setup `quick-dev`. Extracted the tar and ran the following:

Re: new committer: Josh Meyer

2017-02-23 Thread James Sirota
Oops. Sorry. Cut and paste error... These Apache templates are hard :) The Podling Project Management Committee (PPMC) for Apache Metron (Incubating) has asked Josh Meyer to become a committer and we are pleased to announce that they have accepted. Being a committer enables easier contribution

new committer: Josh Meyer

2017-02-23 Thread James Sirota
The Podling Project Management Committee (PPMC) for Apache Metron (Incubating) has asked Kyle Richardson to become a committer and we are pleased to announce that they have accepted. Being a committer enables easier contribution to the project since there is no need to go via the patch submissio

[GitHub] incubator-metron issue #463: METRON-728: ReaderSpliteratorTest fails randoml...

2017-02-23 Thread justinleet
Github user justinleet commented on the issue: https://github.com/apache/incubator-metron/pull/463 Yep, my +1 is still in place. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] incubator-metron issue #463: METRON-728: ReaderSpliteratorTest fails randoml...

2017-02-23 Thread mmiklavc
Github user mmiklavc commented on the issue: https://github.com/apache/incubator-metron/pull/463 I'm +1 on this latest incarnation. Pulled the branch locally, reviewed, and verified metron-common `mvn clean install` runs successfully. Thanks for taking this on @cestella. The code is c

[GitHub] incubator-metron issue #463: METRON-728: ReaderSpliteratorTest fails randoml...

2017-02-23 Thread cestella
Github user cestella commented on the issue: https://github.com/apache/incubator-metron/pull/463 @justinleet Thanks! I replaced the polymorphic delegate with the cleaner mock version *and* I learned a bit about how mocking works. --- If your project is set up for it, you can reply t

[GitHub] incubator-metron issue #463: METRON-728: ReaderSpliteratorTest fails randoml...

2017-02-23 Thread mmiklavc
Github user mmiklavc commented on the issue: https://github.com/apache/incubator-metron/pull/463 Good catch @justinleet! That was my next suspicion and you beat me to it. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If

[GitHub] incubator-metron issue #463: METRON-728: ReaderSpliteratorTest fails randoml...

2017-02-23 Thread justinleet
Github user justinleet commented on the issue: https://github.com/apache/incubator-metron/pull/463 I figured out why From the docs: ``` List list = new LinkedList(); List spy = spy(list); //Impossible: real method is called so spy.get(0) throw

[GitHub] incubator-metron issue #463: METRON-728: ReaderSpliteratorTest fails randoml...

2017-02-23 Thread mmiklavc
Github user mmiklavc commented on the issue: https://github.com/apache/incubator-metron/pull/463 @cestella @justinleet Hm, those should theoretically be the same in this case. doAnswer() is just a different implementation for dealing with void return methods. http://static.javadoc.io

[GitHub] incubator-metron issue #463: METRON-728: ReaderSpliteratorTest fails randoml...

2017-02-23 Thread cestella
Github user cestella commented on the issue: https://github.com/apache/incubator-metron/pull/463 Thanks! I don't quite get why that works and mine returns one fewer split though. Anyone have an insight on that? --- If your project is set up for it, you can reply to this email and h

[GitHub] incubator-metron issue #463: METRON-728: ReaderSpliteratorTest fails randoml...

2017-02-23 Thread justinleet
Github user justinleet commented on the issue: https://github.com/apache/incubator-metron/pull/463 @cestella Spy() syntax ends up working differently than mock() from what I can tell. This worked for me ``` Spliterator delegatingSpliterator = spy(spliterator);

[GitHub] incubator-metron issue #463: METRON-728: ReaderSpliteratorTest fails randoml...

2017-02-23 Thread cestella
Github user cestella commented on the issue: https://github.com/apache/incubator-metron/pull/463 I reacted to your request, @justinleet . Also, I would like to put out a request for the community: The `delegatingSpliterator` pattern that I used uses polymorphism to construct

[GitHub] incubator-metron issue #463: METRON-728: ReaderSpliteratorTest fails randoml...

2017-02-23 Thread justinleet
Github user justinleet commented on the issue: https://github.com/apache/incubator-metron/pull/463 +1, I appreciate you going ahead and taking this ticket, given that I've been bitten by it twice now. Looks great. --- If your project is set up for it, you can reply to this email and

[GitHub] incubator-metron pull request #463: METRON-728: ReaderSpliteratorTest fails ...

2017-02-23 Thread justinleet
Github user justinleet commented on a diff in the pull request: https://github.com/apache/incubator-metron/pull/463#discussion_r102776136 --- Diff: metron-platform/metron-common/src/test/java/org/apache/metron/common/utils/file/ReaderSpliteratorTest.java --- @@ -97,88 +110,73 @@ p

[GitHub] incubator-metron issue #463: METRON-728: ReaderSpliteratorTest fails randoml...

2017-02-23 Thread cestella
Github user cestella commented on the issue: https://github.com/apache/incubator-metron/pull/463 @mmiklavc The stream can consider itself parallel without it actually functioning in parallel. If we have one big split, only one thread will be used in the threadpool, so it'll be de fac

[GitHub] incubator-metron issue #463: METRON-728: ReaderSpliteratorTest fails randoml...

2017-02-23 Thread mmiklavc
Github user mmiklavc commented on the issue: https://github.com/apache/incubator-metron/pull/463 > We actually need it for all of the tests that ensure actual parallelism and I've strengthened the other test impacted. We had gotten around it by being too lenient, honestly. The rest of

[GitHub] incubator-metron issue #316: METRON-503: Metron REST API

2017-02-23 Thread merrimanr
Github user merrimanr commented on the issue: https://github.com/apache/incubator-metron/pull/316 Just merged in the latest from master including the maxmind fix that was breaking the build. I am not able to recreate the NoSuchError exception. Here is what I'm doing to test:

[GitHub] incubator-metron issue #463: METRON-728: ReaderSpliteratorTest fails randoml...

2017-02-23 Thread justinleet
Github user justinleet commented on the issue: https://github.com/apache/incubator-metron/pull/463 @cestella that is a much better way of stating it, and exactly what I was alluding to. I'll look through the new commit. --- If your project is set up for it, you can reply to this ema

[GitHub] incubator-metron issue #463: METRON-728: ReaderSpliteratorTest fails randoml...

2017-02-23 Thread cestella
Github user cestella commented on the issue: https://github.com/apache/incubator-metron/pull/463 Yeah, I can get behind that, @justinleet I'd state it slightly differently though. It's not that we want to test the Stream so much as we want to test that the Spliterator, when interacti

[GitHub] incubator-metron issue #463: METRON-728: ReaderSpliteratorTest fails randoml...

2017-02-23 Thread justinleet
Github user justinleet commented on the issue: https://github.com/apache/incubator-metron/pull/463 @cestella The more I'm thinking about this, the more I wonder if this test is inherently structured incorrectly. My thinking is that it seems more like we're testing whether or not a str

[VOTE] Releasing Apache Metron (incubating) 0.3.1-RC5

2017-02-23 Thread Casey Stella
This is a call to vote on releasing Apache Metron 0.3.1-RC5 incubating Full list of changes in this release: https://dist.apache.org/repos/dist/dev/incubator/metron/0.3.1-RC5-incubating/CHANGES The tag/commit to be voted upon is apache-metron-0.3.1-rc5-incubating: https://git-wip-us.apache.org/re

[GitHub] incubator-metron issue #449: METRON-701 Triage Metrics Produced by the Profi...

2017-02-23 Thread nickwallen
Github user nickwallen commented on the issue: https://github.com/apache/incubator-metron/pull/449 > Am I missing something? Is there a way to define the topic dynamically while using the BulkMessageWriterBolt & KafkaMessageWriter classes unchanged? Created [METRON-738](https:

[GitHub] incubator-metron issue #449: METRON-701 Triage Metrics Produced by the Profi...

2017-02-23 Thread nickwallen
Github user nickwallen commented on the issue: https://github.com/apache/incubator-metron/pull/449 Ok, good. I don't see any problems there. Will use the 'enrichments' queue. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as w

[GitHub] incubator-metron issue #449: METRON-701 Triage Metrics Produced by the Profi...

2017-02-23 Thread cestella
Github user cestella commented on the issue: https://github.com/apache/incubator-metron/pull/449 Ok, I'm ok with that. The writer should be more adaptable here and that shouldn't hold your PR up, agreed. Can we make it the enrichment queue, though? --- If your project is set up for

[GitHub] incubator-metron issue #449: METRON-701 Triage Metrics Produced by the Profi...

2017-02-23 Thread nickwallen
Github user nickwallen commented on the issue: https://github.com/apache/incubator-metron/pull/449 > Ninja Edit: I think the kafka topic written to should be pulled from zookeeper... @cestella I remember now why I settled on making the topic name a static configuration from t

[GitHub] incubator-metron pull request #462: METRON-734 Builds failing because of Max...

2017-02-23 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/incubator-metron/pull/462 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the featu

Travis is taking a long time to start Metron builds

2017-02-23 Thread Casey Stella
Yesterday we waited all day for the METRON-734 PR to even start (only to have it fail in a sporadic failure, but there's a PR for that too ;). On the PR, I moved to allow the travis result to be an indication of a good build after waiting 3 hours for Travis to start our build. I would prefer to n

[GitHub] incubator-metron issue #462: METRON-734 Builds failing because of MaxMind DB...

2017-02-23 Thread nickwallen
Github user nickwallen commented on the issue: https://github.com/apache/incubator-metron/pull/462 > Given the fact that we are without a working build, the failure here is a known sporadic failure and the Travis queue seems to be very backed up lately, I move that we wait for 3 hours

[GitHub] incubator-metron issue #462: METRON-734 Builds failing because of MaxMind DB...

2017-02-23 Thread merrimanr
Github user merrimanr commented on the issue: https://github.com/apache/incubator-metron/pull/462 +1 on committing based on Justin's Travis results --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] incubator-metron issue #463: METRON-728: ReaderSpliteratorTest fails randoml...

2017-02-23 Thread cestella
Github user cestella commented on the issue: https://github.com/apache/incubator-metron/pull/463 We actually need it for all of the tests that ensure actual parallelism and I've strengthened the other test impacted. We had gotten around it by being too lenient, honestly. The rest of

[GitHub] incubator-metron issue #463: METRON-728: ReaderSpliteratorTest fails randoml...

2017-02-23 Thread nickwallen
Github user nickwallen commented on the issue: https://github.com/apache/incubator-metron/pull/463 Why do we need this for only one of our tests? Why are other tests not impacted by this issue? Do you think this is the best fix or a 'good-enough' band-aid? --- If your project is

[GitHub] incubator-metron issue #449: METRON-701 Triage Metrics Produced by the Profi...

2017-02-23 Thread cestella
Github user cestella commented on the issue: https://github.com/apache/incubator-metron/pull/449 @nickwallen yes, I was --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled an

[GitHub] incubator-metron issue #449: METRON-701 Triage Metrics Produced by the Profi...

2017-02-23 Thread nickwallen
Github user nickwallen commented on the issue: https://github.com/apache/incubator-metron/pull/449 > Be backwards compatible with the current syntax. This proposed syntax isn't directly backwards compatible. Were you assuming we would do a translation of sorts? Lik

[GitHub] incubator-metron issue #463: METRON-728: ReaderSpliteratorTest fails randoml...

2017-02-23 Thread cestella
Github user cestella commented on the issue: https://github.com/apache/incubator-metron/pull/463 Yep, whole test 100k times red-green style. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have th

[GitHub] incubator-metron issue #463: METRON-728: ReaderSpliteratorTest fails randoml...

2017-02-23 Thread justinleet
Github user justinleet commented on the issue: https://github.com/apache/incubator-metron/pull/463 Nevermind, I can't read. You ran the whole test 100k times, correct? I'm fine with that. --- If your project is set up for it, you can reply to this email and have your reply appear o

[GitHub] incubator-metron issue #449: METRON-701 Triage Metrics Produced by the Profi...

2017-02-23 Thread ottobackwards
Github user ottobackwards commented on the issue: https://github.com/apache/incubator-metron/pull/449 I'm sorry, no Nick I don't --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] incubator-metron issue #463: METRON-728: ReaderSpliteratorTest fails randoml...

2017-02-23 Thread justinleet
Github user justinleet commented on the issue: https://github.com/apache/incubator-metron/pull/463 Are we settling on "less sporadic"? Like I noted in the ticket, I had the original test run for over a minute (~90 seconds) before the JVM decided to actually be single threaded. It's

[GitHub] incubator-metron issue #449: METRON-701 Triage Metrics Produced by the Profi...

2017-02-23 Thread nickwallen
Github user nickwallen commented on the issue: https://github.com/apache/incubator-metron/pull/449 @cestella Thanks for laying out your other ideas for Medium and Longer term. We can open those up for community debate on separate JIRAs, but it was very worthwhile for you to begin lay

[GitHub] incubator-metron pull request #463: METRON-728: ReaderSpliteratorTest fails ...

2017-02-23 Thread cestella
GitHub user cestella opened a pull request: https://github.com/apache/incubator-metron/pull/463 METRON-728: ReaderSpliteratorTest fails randomly and extremely rarely This unit test is sporadic, so it's less sporadic now. * I ran the medium batch tests in question 100k times

[GitHub] incubator-metron issue #449: METRON-701 Triage Metrics Produced by the Profi...

2017-02-23 Thread nickwallen
Github user nickwallen commented on the issue: https://github.com/apache/incubator-metron/pull/449 **Near Term:** I like it. I think we've converged on "near term". Yay! I will tackle these items as part of this PR. > **Longer Term:** ... In this world, the profiler is si

[GitHub] incubator-metron issue #462: METRON-734 Builds failing because of MaxMind DB...

2017-02-23 Thread cestella
Github user cestella commented on the issue: https://github.com/apache/incubator-metron/pull/462 I'm also going to tackle METRON-728 this morning, so we can avoid this in the future. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHu

[GitHub] incubator-metron issue #316: METRON-503: Metron REST API

2017-02-23 Thread jjmeyer0
Github user jjmeyer0 commented on the issue: https://github.com/apache/incubator-metron/pull/316 @merrimanr For some reason I am still getting the following error on the endpoints. Also, the no such method error turned into this as well. How are you running it? I'm running it using ja

[GitHub] incubator-metron issue #462: METRON-734 Builds failing because of MaxMind DB...

2017-02-23 Thread kylerichardson
Github user kylerichardson commented on the issue: https://github.com/apache/incubator-metron/pull/462 +1 to proceeded with commit based on @justinleet's Travis results On Thu, Feb 23, 2017 at 7:39 AM, Casey Stella wrote: > Given the fact that we are without a wor

[GitHub] incubator-metron pull request #316: METRON-503: Metron REST API

2017-02-23 Thread jjmeyer0
Github user jjmeyer0 commented on a diff in the pull request: https://github.com/apache/incubator-metron/pull/316#discussion_r102705316 --- Diff: metron-interface/metron-rest/src/main/java/org/apache/metron/rest/controller/GrokController.java --- @@ -0,0 +1,56 @@ +/** + *

[GitHub] incubator-metron issue #462: METRON-734 Builds failing because of MaxMind DB...

2017-02-23 Thread cestella
Github user cestella commented on the issue: https://github.com/apache/incubator-metron/pull/462 Given the fact that we are without a working build, the failure here is a known sporadic failure and the Travis queue seems to be very backed up lately, I move that we wait for 3 hours and

[GitHub] incubator-metron issue #462: METRON-734 Builds failing because of MaxMind DB...

2017-02-23 Thread justinleet
Github user justinleet commented on the issue: https://github.com/apache/incubator-metron/pull/462 Apparently https://issues.apache.org/jira/browse/METRON-728 occurs more frequently on travis than my local machine. The Travis running on my personal account already succeed (ht

[GitHub] incubator-metron pull request #462: METRON-734 Builds failing because of Max...

2017-02-23 Thread justinleet
Github user justinleet closed the pull request at: https://github.com/apache/incubator-metron/pull/462 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the f

[GitHub] incubator-metron pull request #462: METRON-734 Builds failing because of Max...

2017-02-23 Thread justinleet
GitHub user justinleet reopened a pull request: https://github.com/apache/incubator-metron/pull/462 METRON-734 Builds failing because of MaxMind DB transitive dependency The pure exclude solution. Still being tested on Vagrant, but making it available for viewing. You can merge th