Re: [DISCUSS] Metron Alerts UI

[Casey Stella]

   - What does "Escalate" do exactly?
   - Where does the Alert ID come from?
   - Are the fields displayed configurable?
   - It'd be nice to be able to select a set of fields for a message and
   have the list of messages filter to just those where those fields are the
   same as the one viewed.


On Thu, Feb 23, 2017 at 3:24 PM, Houshang Livian 
wrote:

> Hello Metron Community,
>
> We have mocked up an Alerts UI for Metron for your consideration. Please
> take a look and share your thoughts.
>
> Here is a link to our thoughts on this:
> http://imgur.com/a/KMTKN
>
> Does this look like a reasonable place to start?
> Is there anything that is an absolute MUST have or MUST NOT have?
>
> Houshang Livian
>
>
>

[GitHub] incubator-metron issue #453: METRON-694: Index Errors from Topologies

[JonZeolla]

Github user JonZeolla commented on the issue:

https://github.com/apache/incubator-metron/pull/453
  
I'm sorry if this was already covered and I missed it, I'm writing this 
from my phone.  How does this handle errors during indexing itself?  For 
instance, if the failure is due to the content of a specific field and that is 
placed in rawMessage/raw_message, wouldn't we have a failure loop?  I was under 
the assumption that rawMessage/raw_message was not going to be included because 
of this sort of cyclical concern.  Just wanted to double check on that because 
I've definitely seen those sort of failures in my environment.  

This also may be mitigated by the outcome of the ongoing dev list 
conversation "[DISCUSS] Management of Elastic and other index schemas" because 
we could attempt to fix those known sort of issues in the abstraction, but it 
still leaves us open to unknown issues.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[DISCUSS] Metron Alerts UI

[Houshang Livian]

Hello Metron Community,

We have mocked up an Alerts UI for Metron for your consideration. Please take a 
look and share your thoughts.

Here is a link to our thoughts on this:
http://imgur.com/a/KMTKN

Does this look like a reasonable place to start?
Is there anything that is an absolute MUST have or MUST NOT have?

Houshang Livian

[GitHub] incubator-metron issue #316: METRON-503: Metron REST API

[merrimanr]

Github user merrimanr commented on the issue:

https://github.com/apache/incubator-metron/pull/316
  
Hmm I just tried deploying on quick-dev the same way and was able to hit 
the stellar endpoints without issue.  I will continue testing to see if I can 
get it to break tomorrow.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: new committer: Josh Meyer

[zeo...@gmail.com]

Congrats Josh!

On Thu, Feb 23, 2017, 2:38 PM Otto Fowler  wrote:

> Nice!  Congratulations!
>
>
> On February 23, 2017 at 15:14:42, James Sirota (jsir...@apache.org) wrote:
>
> The Podling Project Management Committee (PPMC) for Apache Metron
> (Incubating)
> has asked Kyle Richardson to become a committer and we are pleased
> to announce that they have accepted.
>
>
> Being a committer enables easier contribution to the
> project since there is no need to go via the patch
> submission process. This should enable better productivity.
> Being a PMC member enables assistance with the management
> and to guide the direction of the project.
>
> ---
> Thank you,
>
> James Sirota
> PPMC- Apache Metron (Incubating)
> jsirota AT apache DOT org
>
-- 

Jon

Sent from my mobile device

Re: new committer: Josh Meyer

[Otto Fowler]

Nice!  Congratulations!


On February 23, 2017 at 15:14:42, James Sirota (jsir...@apache.org) wrote:

The Podling Project Management Committee (PPMC) for Apache Metron
(Incubating)
has asked Kyle Richardson to become a committer and we are pleased
to announce that they have accepted.


Being a committer enables easier contribution to the
project since there is no need to go via the patch
submission process. This should enable better productivity.
Being a PMC member enables assistance with the management
and to guide the direction of the project.

---
Thank you,

James Sirota
PPMC- Apache Metron (Incubating)
jsirota AT apache DOT org

Re: [DISCUSS] Top domains enrichment config/extractor management

[David Lyle]

I don't really have a firm opinion either way except for the normal
default- if Ambari is going to own a part of it, it needs to maintain the
full (load/update/upgrade/remove) lifecycle.

So, it sounds like you've already decided you want Ambari to do the initial
install and load. The consequence of that is that Ambari should expose the
rest.

-D...


On Tue, Feb 21, 2017 at 6:31 PM, Michael Miklavcic <
michael.miklav...@gmail.com> wrote:

> With the work committed in
> https://github.com/apache/incubator-metron/pull/445 and
> https://github.com/apache/incubator-metron/pull/432, we now have a robust
> and flexible means to import enrichment sources and transform their
> contents as they are inserted into HBase. One of the main motivators for
> this new functionality was to add the ability to load top domain rankings
> from sources such as Alexa. The proposal is to make this type of enrichment
> a top-level feature in Metron by introducing it to the Ambari management UI
> as a configurable set of properties in the MPack install. This comes with
> some options and challenges in how we want to manage the configurations,
> which I will outline below.
>
> *Use cases:*
>
>- Single load of top domains file
>- Re-loading top domains file - need to be able to cleanup properly
>- Cleaning up/deleting old enrichment data (this is a general feature
>that we currently lack - I think it is worth a separate Jira/PR for
>creating a MapReduce job that enables cleanup to occur).
>- Modifying default top domains file source - there are other options
>besides Alexa. And users may want to load a file from local URI since
> many
>data centers do not have direct access to the internet.
>- Ability to modify the default extractor config JSON and tune the
>Stellar transformations for both the value and indicator transforms.
> Allows
>more flexible handling of data based on other sources.
>- Loading multiple top domains source enrichments. (Maybe a separate PR
>for this if we even think it would be useful)
>- Updating the top domain enrichment - This needs to be an atomic
>operation in order to prevent incorrect data.
>- Rolling back to an older version of the top domains enrichment. Also
>needs to be atomic.
>- Ability to schedule an enrichment load on schedule - we would like to
>defer this to an external scheduling mechanism, e.g. cron or Control M.
> The
>enrichment loading system should have the necessary features to enable
> this
>type of automation without data integrity issues.
>
> *Considerations:*
>
>- As mentioned above, we want to add this feature to the Ambari MPack.
>This requires at least 2 parameters to work. We need the ability to
> specify
>a URI as well as an extractor config.
>- How do we want to manage the extractor config? The most obvious
>solution is to provide a text field in Ambari with a default JSON
> config.
>When a load is initiated, Ambari would place a fresh copy of the
> extractor
>config in the /tmp/ directory. This is an ephemeral file that isn't
> needed
>other than during a load.
>- It seems easy enough to have the load occur during the initial
>install, however subsequent loads would require a different workflow.
> How
>do folks feel about adding a set of dropdown options in the Ambari UI
> for
>loading, updating, and deleting the top domains enrichment? I believe we
>are doing something similar for the ElasticSearch templates currently.
>- In the case of atomic operations for updates and rollbacks, I propose
>we add a property to Zookeeper that is reference-able in the enrichment
>itself. The idea would be to create a "top-domains" property in ZK that
>points to an enrichment key with a load timestamp associated with it,
> e.g.
>top-domains_20170221042000. This would also allow a mapreduce job to be
>written that cleans up old enrichments. Another option is to create a
> new
>table in HBase if/when you update the enrichment and change the
> enrichment
>config manually. Deleting an old enrichment would simply be a matter of
>dropping the table in HBase. A relevant discussion of the tradeoffs of
>having many small tables versus 1 large table can be found here -
>http://grokbase.com/t/hbase/user/11bjbdw94q/multiple-
> tables-vs-big-fat-table
>- In order to update or rollback an enrichment as mentioned above, we
>would also ideally provide a mechanism for changing the rowkey pointed
> to
>by the enrichment.
>
> In summary of the use cases and considerations above, this boils down to
> how we'd like to leverage Ambari here. Do we want Ambari to handle only the
> initial install/load and have end users be responsible on an ongoing basis
> for updates (users would be responsible for copying or distributing the
> extractor_config.json for instance), or do we want to enable Ambari to
> manage the configuration 

Re: new committer: Josh Meyer

[Kyle Richardson]

Welcome, Josh!

-Kyle

On Thu, Feb 23, 2017 at 3:17 PM, James Sirota  wrote:

> Oops. Sorry.  Cut and paste error... These Apache templates are hard :)
>
> The Podling Project Management Committee (PPMC) for Apache Metron
> (Incubating)
> has asked Josh Meyer to become a committer and we are pleased
> to announce that they have accepted.
>
>
> Being a committer enables easier contribution to the
> project since there is no need to go via the patch
> submission process. This should enable better productivity.
> Being a PMC member enables assistance with the management
> and to guide the direction of the project.
>
>
> 23.02.2017, 13:14, "James Sirota" :
> > The Podling Project Management Committee (PPMC) for Apache Metron
> (Incubating)
> > has asked Kyle Richardson to become a committer and we are pleased
> > to announce that they have accepted.
> >
> > Being a committer enables easier contribution to the
> > project since there is no need to go via the patch
> > submission process. This should enable better productivity.
> > Being a PMC member enables assistance with the management
> > and to guide the direction of the project.
> >
> > ---
> > Thank you,
> >
> > James Sirota
> > PPMC- Apache Metron (Incubating)
> > jsirota AT apache DOT org
>
> ---
> Thank you,
>
> James Sirota
> PPMC- Apache Metron (Incubating)
> jsirota AT apache DOT org
>

[GitHub] incubator-metron issue #316: METRON-503: Metron REST API

[jjmeyer0]

Github user jjmeyer0 commented on the issue:

https://github.com/apache/incubator-metron/pull/316
  
@merrimanr I'm using quick-dev to do my tests. I built everything from 
metron root with `mvn clean install -DskipTests`. Then I setup `quick-dev`. 
Extracted the tar and ran the following:

```
java -jar ./lib/metron-rest-$METRON_VERSION.jar 
--spring.profiles.active=vagrant,dev --server.port=8082
```


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: new committer: Josh Meyer

[James Sirota]

Oops. Sorry.  Cut and paste error... These Apache templates are hard :)

The Podling Project Management Committee (PPMC) for Apache Metron (Incubating)
has asked Josh Meyer to become a committer and we are pleased 
to announce that they have accepted.


Being a committer enables easier contribution to the
project since there is no need to go via the patch
submission process. This should enable better productivity.
Being a PMC member enables assistance with the management
and to guide the direction of the project.


23.02.2017, 13:14, "James Sirota" :
> The Podling Project Management Committee (PPMC) for Apache Metron (Incubating)
> has asked Kyle Richardson to become a committer and we are pleased
> to announce that they have accepted.
>
> Being a committer enables easier contribution to the
> project since there is no need to go via the patch
> submission process. This should enable better productivity.
> Being a PMC member enables assistance with the management
> and to guide the direction of the project.
>
> ---
> Thank you,
>
> James Sirota
> PPMC- Apache Metron (Incubating)
> jsirota AT apache DOT org

--- 
Thank you,

James Sirota
PPMC- Apache Metron (Incubating)
jsirota AT apache DOT org

new committer: Josh Meyer

[James Sirota]

The Podling Project Management Committee (PPMC) for Apache Metron (Incubating)
has asked Kyle Richardson to become a committer and we are pleased 
to announce that they have accepted.


Being a committer enables easier contribution to the
project since there is no need to go via the patch
submission process. This should enable better productivity.
Being a PMC member enables assistance with the management
and to guide the direction of the project.

--- 
Thank you,

James Sirota
PPMC- Apache Metron (Incubating)
jsirota AT apache DOT org

[GitHub] incubator-metron issue #463: METRON-728: ReaderSpliteratorTest fails randoml...

[justinleet]

Github user justinleet commented on the issue:

https://github.com/apache/incubator-metron/pull/463
  
Yep, my +1 is still in place.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-metron issue #463: METRON-728: ReaderSpliteratorTest fails randoml...

[mmiklavc]

Github user mmiklavc commented on the issue:

https://github.com/apache/incubator-metron/pull/463
  
I'm +1 on this latest incarnation. Pulled the branch locally, reviewed, and 
verified metron-common `mvn clean install` runs successfully. Thanks for taking 
this on @cestella. The code is cleaner and the intent is clear to me.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-metron issue #463: METRON-728: ReaderSpliteratorTest fails randoml...

[cestella]

Github user cestella commented on the issue:

https://github.com/apache/incubator-metron/pull/463
  
@justinleet Thanks!  I replaced the polymorphic delegate with the cleaner 
mock version *and* I learned a bit about how mocking works.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-metron issue #463: METRON-728: ReaderSpliteratorTest fails randoml...

[justinleet]

Github user justinleet commented on the issue:

https://github.com/apache/incubator-metron/pull/463
  
I figured out why

From the docs:
```

   List list = new LinkedList();
   List spy = spy(list);

   //Impossible: real method is called so spy.get(0) throws 
IndexOutOfBoundsException (the list is yet empty)
   when(spy.get(0)).thenReturn("foo");

   //You have to use doReturn() for stubbing
   doReturn("foo").when(spy).get(0);
```
So the first call to @cestella's trySplit() is in the when(), not in the 
lambda.  The subsequent calls are.  So everything shifts by one.

As the docs note "Sometimes it's impossible or impractical to use 
when(Object) for stubbing spies. Therefore when using spies please consider 
doReturn|Answer|Throw() family of methods for stubbing."

I just happened to try this, and now I know why it works.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-metron issue #463: METRON-728: ReaderSpliteratorTest fails randoml...

[mmiklavc]

Github user mmiklavc commented on the issue:

https://github.com/apache/incubator-metron/pull/463
  
@cestella @justinleet Hm, those should theoretically be the same in this 
case. doAnswer() is just a different implementation for dealing with void 
return methods. 
http://static.javadoc.io/org.mockito/mockito-core/2.7.11/org/mockito/Mockito.html#doAnswer(org.mockito.stubbing.Answer)


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-metron issue #463: METRON-728: ReaderSpliteratorTest fails randoml...

[cestella]

Github user cestella commented on the issue:

https://github.com/apache/incubator-metron/pull/463
  
Thanks!  I don't quite get why that works and mine returns one fewer split 
though.  Anyone have an insight on that?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-metron issue #463: METRON-728: ReaderSpliteratorTest fails randoml...

[justinleet]

Github user justinleet commented on the issue:

https://github.com/apache/incubator-metron/pull/463
  
@cestella Spy() syntax ends up working differently than mock() from what I 
can tell.

This worked for me
```
Spliterator delegatingSpliterator = spy(spliterator);
doAnswer(invocationOnMock -> {
  Spliterator ret = spliterator.trySplit();
  if(ret != null) {
numSplits.incrementAndGet();
  }
  return ret;
}).when(delegatingSpliterator).trySplit();
```


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-metron issue #463: METRON-728: ReaderSpliteratorTest fails randoml...

[cestella]

Github user cestella commented on the issue:

https://github.com/apache/incubator-metron/pull/463
  
I reacted to your request, @justinleet .  Also, I would like to put out a 
request for the community:

The `delegatingSpliterator` pattern that I used uses polymorphism to 
construct a delegate which allows me to count splits.  I wanted to investigate 
using mocks to do this in a cleaner way, but I kept getting off-by-one issues.  
Here's what I did as a replacement for `delegatingSpliterator`

```
Spliterator delegatingSpliterator = spy(spliterator);
when(delegatingSpliterator.trySplit()).thenAnswer(answer -> {
  Spliterator ret = spliterator.trySplit();
if(ret != null) {
  numSplits.incrementAndGet();
}
return ret;
});
```

What I see is that I am processing 1 fewer splits than I expect.  What we 
have in the PR is perfectly sufficient and inoffensive, but I wanted to learn a 
bit more about mockito and thought if anyone had insight, we could all get 
better. :)


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-metron issue #463: METRON-728: ReaderSpliteratorTest fails randoml...

[justinleet]

Github user justinleet commented on the issue:

https://github.com/apache/incubator-metron/pull/463
  
+1, I appreciate you going ahead and taking this ticket, given that I've 
been bitten by it twice now.  Looks great.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-metron pull request #463: METRON-728: ReaderSpliteratorTest fails ...

[justinleet]

Github user justinleet commented on a diff in the pull request:

https://github.com/apache/incubator-metron/pull/463#discussion_r102776136
  
--- Diff: 
metron-platform/metron-common/src/test/java/org/apache/metron/common/utils/file/ReaderSpliteratorTest.java
 ---
@@ -97,88 +110,73 @@ public void testSequentialStreamLargeBatch() throws 
FileNotFoundException {
   Map count =
   stream.map(s -> s.trim())
   .collect(Collectors.toMap(s -> s, s -> 1, 
Integer::sum));
-  Assert.assertEquals(5, count.size());
-  Assert.assertEquals(3, (int) count.get("foo"));
-  Assert.assertEquals(2, (int) count.get("bar"));
-  Assert.assertEquals(1, (int) count.get("and"));
-  Assert.assertEquals(1, (int) count.get("the"));
+  validateMapCount(count);
 }
   }
 
-  @Test
-  public void testActuallyParallel() throws ExecutionException, 
InterruptedException, FileNotFoundException {
-//With 9 elements and a batch of 2, we should only ceil(9/2) = 5 
batches, so at most min(5, 2) = 2 threads will be used
-try( Stream stream = ReaderSpliterator.lineStream(getReader(), 
2)) {
-  ForkJoinPool forkJoinPool = new ForkJoinPool(2);
-  forkJoinPool.submit(() -> {
-Map threads =
-stream.parallel().map(s -> 
Thread.currentThread().getName())
-.collect(Collectors.toMap(s -> s, s -> 1, 
Integer::sum));
-Assert.assertTrue(threads.size() <= 2);
-  }
-  ).get();
-}
-  }
+  private int getNumberOfBatches(final ReaderSpliterator spliterator) 
throws ExecutionException, InterruptedException {
+final AtomicInteger numSplits = new AtomicInteger(0);
+//we want to wrap the spliterator and count the (valid) splits
+Spliterator delegatingSpliterator = new Spliterator() {
+  @Override
+  public boolean tryAdvance(Consumer action) {
+return spliterator.tryAdvance(action);
+  }
 
-  @Test
-  public void testActuallyParallel_mediumBatch() throws 
ExecutionException, InterruptedException, FileNotFoundException {
-//With 9 elements and a batch of 2, we should only ceil(9/2) = 5 
batches, so at most 5 threads of the pool of 10 will be used
-try( Stream stream = ReaderSpliterator.lineStream(getReader(), 
2)) {
-  ForkJoinPool forkJoinPool = new ForkJoinPool(10);
-  forkJoinPool.submit(() -> {
-Map threads =
-stream.parallel().map(s -> 
Thread.currentThread().getName())
-.collect(Collectors.toMap(s -> s, s -> 1, 
Integer::sum));
-Assert.assertTrue(threads.size() <= (int) Math.ceil(9.0 / 
2) && threads.size() > 1);
-  }
-  ).get();
-}
+  @Override
+  public Spliterator trySplit() {
+Spliterator ret = spliterator.trySplit();
+if(ret != null) {
+  numSplits.incrementAndGet();
+}
+return ret;
+  }
+
+  @Override
+  public long estimateSize() {
+return spliterator.estimateSize();
+  }
+
+  @Override
+  public int characteristics() {
+return spliterator.characteristics();
+  }
+};
+
+Stream stream = StreamSupport.stream(delegatingSpliterator, 
true);
+
+//now run it in a parallel pool and do some calculation that doesn't 
really matter.
+ForkJoinPool forkJoinPool = new ForkJoinPool(10);
--- End diff --

Incredibly minor point, but since we no longer care about the actual 
execution and aren't running it a lot, it seems appropriate to just use 
ForkJoinPool.commonPool(), and drop the shutdown line.

This is entirely up to you if you want to change, I don't consider it 
blocking by any means.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-metron issue #463: METRON-728: ReaderSpliteratorTest fails randoml...

[cestella]

Github user cestella commented on the issue:

https://github.com/apache/incubator-metron/pull/463
  
@mmiklavc The stream can consider itself parallel without it actually 
functioning in parallel.  If we have one big split, only one thread will be 
used in the threadpool, so it'll be de facto serial.  Also, we don't actually 
implement Stream, we *construct* a stream from a Spliterator.

The issue that @justinleet alluded to was that we were testing 
`Spliterator` indirectly by testing `Stream`s in a threadpool.  If we just did 
the direct test, we'd narrow the scope and remove the variability as well.  If 
we are, then we should be able to trust `Stream.parallel()` to do the right 
thing rather than testing that.  What we really want to know is are we 
splitting properly. 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-metron issue #463: METRON-728: ReaderSpliteratorTest fails randoml...

[mmiklavc]

Github user mmiklavc commented on the issue:

https://github.com/apache/incubator-metron/pull/463
  
> We actually need it for all of the tests that ensure actual parallelism 
and I've strengthened the other test impacted. We had gotten around it by being 
too lenient, honestly. The rest of the test cases are non-parallel, so they are 
correct as implemented.

> This is the correct fix, IMO. stream.parallel() does not give any 
explicit guarantees about parallelism, just that it can parallelize batches 
across a thread pool. This test essentially ensures that given enough tries, it 
does parallelize.

Ok, I follow what's going on here and support the need to test for 
instances where you may end up with the same thread by looping. 

Just for some clarification, what is the reason that we are not leveraging 
stream.isParallel() to check for parallelism? Would the concern here be that 
because we're returning an object the implements the Stream interface, the 
ReaderSpliterator could be changed to return something that  does not abide by 
the isParallel() contract?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-metron issue #316: METRON-503: Metron REST API

[merrimanr]

Github user merrimanr commented on the issue:

https://github.com/apache/incubator-metron/pull/316
  
Just merged in the latest from master including the maxmind fix that was 
breaking the build.

I am not able to recreate the NoSuchError exception.  Here is what I'm 
doing to test:
//from project root
mvn clean install -DskipTests
java -jar ./metron-interface/metron-rest/target/metron-rest-0.3.1.jar 
--spring.profiles.active=docker,dev

When I hit the stellar endpoints, the functions come back without error.  
How are you testing it?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-metron issue #463: METRON-728: ReaderSpliteratorTest fails randoml...

[justinleet]

Github user justinleet commented on the issue:

https://github.com/apache/incubator-metron/pull/463
  
@cestella that is a much better way of stating it, and exactly what I was 
alluding to.  I'll look through the new commit.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-metron issue #463: METRON-728: ReaderSpliteratorTest fails randoml...

[cestella]

Github user cestella commented on the issue:

https://github.com/apache/incubator-metron/pull/463
  
Yeah, I can get behind that, @justinleet I'd state it slightly differently 
though.  It's not that we want to test the Stream so much as we want to test 
that the Spliterator, when interacting with a stream, makes the right number of 
splits.  That was the core problem with `Files.lines()` was that it made one 
split, not many splits.

What we really want to test here is that the spliterator splits the 
expected number of times.  What we have indirectly tests that, but it relies on 
the ForkPool to work in a consistent way, which isn't going to happen.  Anyway, 
I pushed a commit, see how you like it.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-metron issue #463: METRON-728: ReaderSpliteratorTest fails randoml...

[justinleet]

Github user justinleet commented on the issue:

https://github.com/apache/incubator-metron/pull/463
  
@cestella The more I'm thinking about this, the more I wonder if this test 
is inherently structured incorrectly. My thinking is that it seems more like 
we're testing whether or not a stream can run in parallel, rather than that the 
stream produced by the spliterator meets the appropriate contracts for a stream.

Is there a way to restructure this so that it just tests "Does this meet 
the criteria of a Java stream?", rather than "Can a stream in Java run in 
parallel?"


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[VOTE] Releasing Apache Metron (incubating) 0.3.1-RC5

[Casey Stella]

This is a call to vote on releasing Apache Metron 0.3.1-RC5 incubating

Full list of changes in this release:
https://dist.apache.org/repos/dist/dev/incubator/metron/0.3.1-RC5-incubating/CHANGES

The tag/commit to be voted upon is apache-metron-0.3.1-rc5-incubating:
https://git-wip-us.apache.org/repos/asf?p=incubator-metron.git;a=shortlog;h=refs/tags/apache-metron-0.3.1-rc5-incubating

The source archive being voted upon can be found here:
https://dist.apache.org/repos/dist/dev/incubator/metron/0.3.1-RC5-incubating/apache-metron-0.3.1-rc5-incubating.tar.gz

Other release files, signatures and digests can be found here:
https://dist.apache.org/repos/dist/dev/incubator/metron/0.3.1-RC5-incubating/

The release artifacts are signed with the following key:
https://git-wip-us.apache.org/repos/asf?p=incubator-metron.git;a=blob;f=KEYS;h=8381e96d64c249a0c1b489bc0c234d9c260ba55e;hb=refs/tags/apache-metron-0.3.1-rc5-incubating

The book associated with this RC is located at
https://dist.apache.org/repos/dist/dev/incubator/metron/0.3.1-RC5-incubating/book-site/index.html

Please vote on releasing this package as Apache Metron 0.3.1-RC5 incubating

When voting, please list the actions taken to verify the release.

Recommended build validation and verification instructions are posted here:
https://cwiki.apache.org/confluence/display/METRON/Verifying+Builds


This vote will be open for at least 72 hours.

[ ] +1 Release this package as Apache Metron 0.3.1-RC5 incubating

[ ]  0 No opinion

[ ] -1 Do not release this package because...

[GitHub] incubator-metron issue #449: METRON-701 Triage Metrics Produced by the Profi...

[nickwallen]

Github user nickwallen commented on the issue:

https://github.com/apache/incubator-metron/pull/449
  
> Am I missing something? Is there a way to define the topic dynamically 
while using the BulkMessageWriterBolt & KafkaMessageWriter classes unchanged?

Created [METRON-738](https://issues.apache.org/jira/browse/METRON-738) to 
track this 'wish list' enhancement.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-metron issue #449: METRON-701 Triage Metrics Produced by the Profi...

[cestella]

Github user cestella commented on the issue:

https://github.com/apache/incubator-metron/pull/449
  
Ok, I'm ok with that.  The writer should be more adaptable here and that 
shouldn't hold your PR up, agreed.  Can we make it the enrichment queue, though?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-metron issue #449: METRON-701 Triage Metrics Produced by the Profi...

[nickwallen]

Github user nickwallen commented on the issue:

https://github.com/apache/incubator-metron/pull/449
  
> Ninja Edit: I think the kafka topic written to should be pulled from 
zookeeper...

@cestella I remember now why I settled on making the topic name a static 
configuration from the topology properties.  Our `BulkMessageWriterBolt` & 
`KafkaMessageWriter` classes seem to have been designed to only work with 
statically defined topic names.  I would have to change those core classes, if 
I want to set the topic name dynamically from Zookeeper.  And I tend to get 
smacked around when coming close to core classes :)

Am I missing something?  Is there a way to define the topic dynamically 
while using the `BulkMessageWriterBolt` & `KafkaMessageWriter` classes 
unchanged?

If not, I'd prefer to keep the topic name as a static property defined in 
the topology properties, at least for this PR. My second choice would be to 
open a completely separate PR to update those core classes to accept dynamic 
topic names.  It does not seem like a complex change, but those are core 
classes and will likely cause some debate.






---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-metron pull request #462: METRON-734 Builds failing because of Max...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/incubator-metron/pull/462


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Travis is taking a long time to start Metron builds

[Casey Stella]

Yesterday we waited all day for the METRON-734 PR to even start (only to
have it fail in a sporadic failure, but there's a PR for that too ;).

On the PR, I moved to allow the travis result to be an indication of a good
build after waiting 3 hours for Travis to start our build.  I would prefer
to not be required to take extreme measures to get a good build in a timely
manner.

As such, I have submitted an Apache INFRA ticket requesting some guidance
on the Travis situation: https://issues.apache.org/jira/browse/INFRA-13571

For those interested, please watch that JIRA and hopefully we can work with
INFRA to make this tolerable again.

[GitHub] incubator-metron issue #462: METRON-734 Builds failing because of MaxMind DB...

[nickwallen]

Github user nickwallen commented on the issue:

https://github.com/apache/incubator-metron/pull/462
  
> Given the fact that we are without a working build, the failure here is a 
known sporadic failure and the Travis queue seems to be very backed up lately, 
I move that we wait for 3 hours and then commit this PR on the basis of 
Justin's Travis results linked above.

+1 to commit based on previous successful travis run.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-metron issue #462: METRON-734 Builds failing because of MaxMind DB...

[merrimanr]

Github user merrimanr commented on the issue:

https://github.com/apache/incubator-metron/pull/462
  
+1 on committing based on Justin's Travis results


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-metron issue #463: METRON-728: ReaderSpliteratorTest fails randoml...

[cestella]

Github user cestella commented on the issue:

https://github.com/apache/incubator-metron/pull/463
  
We actually need it for all of the tests that ensure actual parallelism and 
I've strengthened the other test impacted.  We had gotten around it by being 
too lenient, honestly.  The rest of the test cases are non-parallel, so they 
are correct as implemented.

This is the correct fix, IMO.  `stream.parallel()` does not give any 
explicit guarantees about parallelism, just that it can parallelize batches 
across a thread pool.  This test essentially ensures that given enough tries, 
it does parallelize.  


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-metron issue #463: METRON-728: ReaderSpliteratorTest fails randoml...

[nickwallen]

Github user nickwallen commented on the issue:

https://github.com/apache/incubator-metron/pull/463
  
Why do we need this for only one of our tests?  Why are other tests not 
impacted by this issue?
Do you think this is the best fix or a 'good-enough' band-aid?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-metron issue #449: METRON-701 Triage Metrics Produced by the Profi...

[cestella]

Github user cestella commented on the issue:

https://github.com/apache/incubator-metron/pull/449
  
@nickwallen yes, I was


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-metron issue #449: METRON-701 Triage Metrics Produced by the Profi...

[nickwallen]

Github user nickwallen commented on the issue:

https://github.com/apache/incubator-metron/pull/449
  
> Be backwards compatible with the current syntax.

This proposed syntax isn't directly backwards compatible.  Were you 
assuming we would do a translation of sorts?  

Like translate this...
```
{ "profiles": [
{
  ...
  "result":  "stats"
}
  ]
}
```

To this...
```
{ "profiles": [
{
  ...
  "result":  {
 "profile" : "stats",
   } 
}
  ]
}
```


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-metron issue #463: METRON-728: ReaderSpliteratorTest fails randoml...

[cestella]

Github user cestella commented on the issue:

https://github.com/apache/incubator-metron/pull/463
  
Yep, whole test 100k times red-green style.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-metron issue #463: METRON-728: ReaderSpliteratorTest fails randoml...

[justinleet]

Github user justinleet commented on the issue:

https://github.com/apache/incubator-metron/pull/463
  
Nevermind, I can't read.  You ran the whole test 100k times, correct?  I'm 
fine with that.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-metron issue #449: METRON-701 Triage Metrics Produced by the Profi...

[ottobackwards]

Github user ottobackwards commented on the issue:

https://github.com/apache/incubator-metron/pull/449
  
I'm sorry, no Nick I don't


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-metron issue #463: METRON-728: ReaderSpliteratorTest fails randoml...

[justinleet]

Github user justinleet commented on the issue:

https://github.com/apache/incubator-metron/pull/463
  
Are we settling on "less sporadic"?  Like I noted in the ticket, I had the 
original test run for over a minute (~90 seconds) before the JVM decided to 
actually be single threaded.  It's not the usual case but I probably only ran 
it 20 or so times before I hit the 90 second case.

It seems more likely to fail in Travis, which is fine, but I'm not sure I 
want my local build failing that often.



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-metron issue #449: METRON-701 Triage Metrics Produced by the Profi...

[nickwallen]

Github user nickwallen commented on the issue:

https://github.com/apache/incubator-metron/pull/449
  
@cestella Thanks for laying out your other ideas for Medium and Longer 
term.  We can open those up for community debate on separate JIRAs, but it was 
very worthwhile for you to begin laying those out here.  They provided some 
good context.

@ottobackwards Not sure which part your comment was in reference to.  Do 
you have any concerns specifically with the "Near Term" items that I will 
tackle as part of this PR?  I'd like to make sure we reach consensus on those 
first.  We can debate the other items later (if that works for you.)




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-metron pull request #463: METRON-728: ReaderSpliteratorTest fails ...

[cestella]

GitHub user cestella opened a pull request:

https://github.com/apache/incubator-metron/pull/463

METRON-728: ReaderSpliteratorTest fails randomly and extremely rarely

This unit test is sporadic, so it's less sporadic now.

* I ran the medium batch tests in question 100k times and validated that it 
fails without the change at least once.
* I ran the medium batch tests in question 100k time after the change and 
validated that it does not fail.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/cestella/incubator-metron METRON-728

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-metron/pull/463.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #463


commit 46e7df89210a608e936082e1fa1acb21447f92fd
Author: cstella 
Date:   2017-02-23T13:51:33Z

METRON-728: ReaderSpliteratorTest fails randomly and extremely rarely




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-metron issue #449: METRON-701 Triage Metrics Produced by the Profi...

[nickwallen]

Github user nickwallen commented on the issue:

https://github.com/apache/incubator-metron/pull/449
  
**Near Term:**  I like it.   I think we've converged on "near term".  Yay!  
I will tackle these items as part of this PR.

> **Longer Term:** ... In this world, the profiler is simple, it just 
writes messages out to the indexing topology. 

Great! Yep, this is what I was hoping to do when we first starting batting 
this idea around.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-metron issue #462: METRON-734 Builds failing because of MaxMind DB...

[cestella]

Github user cestella commented on the issue:

https://github.com/apache/incubator-metron/pull/462
  
I'm also going to tackle METRON-728 this morning, so we can avoid this in 
the future.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-metron issue #316: METRON-503: Metron REST API

[jjmeyer0]

Github user jjmeyer0 commented on the issue:

https://github.com/apache/incubator-metron/pull/316
  
@merrimanr For some reason I am still getting the following error on the 
endpoints. Also, the no such method error turned into this as well. How are you 
running it? I'm running it using java cli. Are you using IntelliJ or something 
else?

```
Feb 23, 2017 12:51:24 PM org.apache.catalina.core.StandardWrapperValve 
invoke
SEVERE: Servlet.service() for servlet [dispatcherServlet] in context with 
path [] threw exception [Handler dispatch failed; nested exception is 
java.lang.NoSuchMethodError: 
org.reflections.util.ConfigurationBuilder.filterInputsBy(Lorg/apache/metron/guava/base/Predicate;)Lorg/reflections/util/ConfigurationBuilder;]
 with root cause
java.lang.NoSuchMethodError: 
org.reflections.util.ConfigurationBuilder.filterInputsBy(Lorg/apache/metron/guava/base/Predicate;)Lorg/reflections/util/ConfigurationBuilder;
at 
org.apache.metron.common.dsl.functions.resolver.ClasspathFunctionResolver.resolvables(ClasspathFunctionResolver.java:167)
at 
org.apache.metron.common.dsl.functions.resolver.BaseFunctionResolver.resolveFunctions(BaseFunctionResolver.java:119)
at 
org.apache.metron.guava.base.Suppliers$MemoizingSupplier.get(Suppliers.java:125)
at 
org.apache.metron.common.dsl.functions.resolver.BaseFunctionResolver.getFunctionInfo(BaseFunctionResolver.java:77)
at 
org.apache.metron.rest.service.impl.StellarServiceImpl.getStellarFunctions(StellarServiceImpl.java:74)
at 
org.apache.metron.rest.controller.StellarController.listFunctions(StellarController.java:71)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:221)
at 
org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:136)
at 
org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:114)
at 
org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:827)
at 
org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:738)
at 
org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:85)
at 
org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:963)
at 
org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:897)
at 
org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:970)
at 
org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:861)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:622)
at 
org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:846)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:729)
at 
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:230)
at 
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:165)
at 
org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
at 
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:192)
at 
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:165)
at 
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:317)
at 
org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:127)
at 
org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:91)
at 
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
at 
org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:115)
at 
org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:331)
at 
org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:137)
at 

[GitHub] incubator-metron issue #462: METRON-734 Builds failing because of MaxMind DB...

[kylerichardson]

Github user kylerichardson commented on the issue:

https://github.com/apache/incubator-metron/pull/462
  
+1 to proceeded with commit based on @justinleet's Travis results

On Thu, Feb 23, 2017 at 7:39 AM, Casey Stella 
wrote:

> Given the fact that we are without a working build, the failure here is a
> known sporadic failure and the Travis queue seems to be very backed up
> lately, I move that we wait for 3 hours and then commit this PR on the
> basis of Justin's Travis results linked above.
>
> Objections?
>
> —
> You are receiving this because you were mentioned.
> Reply to this email directly, view it on GitHub
> 
,
> or mute the thread
> 

> .
>



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-metron pull request #316: METRON-503: Metron REST API

[jjmeyer0]

Github user jjmeyer0 commented on a diff in the pull request:

https://github.com/apache/incubator-metron/pull/316#discussion_r102705316
  
--- Diff: 
metron-interface/metron-rest/src/main/java/org/apache/metron/rest/controller/GrokController.java
 ---
@@ -0,0 +1,56 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.metron.rest.controller;
+
+import io.swagger.annotations.ApiOperation;
+import io.swagger.annotations.ApiParam;
+import io.swagger.annotations.ApiResponse;
+import org.apache.metron.rest.RestException;
+import org.apache.metron.rest.model.GrokValidation;
+import org.apache.metron.rest.service.GrokService;
+import org.springframework.beans.factory.annotation.Autowired;
+import org.springframework.http.HttpStatus;
+import org.springframework.http.ResponseEntity;
+import org.springframework.web.bind.annotation.RequestBody;
+import org.springframework.web.bind.annotation.RequestMapping;
+import org.springframework.web.bind.annotation.RequestMethod;
+import org.springframework.web.bind.annotation.RestController;
+
+import java.util.Map;
+
+@RestController
+@RequestMapping("/api/v1/grok")
+public class GrokController {
+
+@Autowired
+private GrokService grokService;
+
+@ApiOperation(value = "Applies a Grok statement to a sample message")
+@ApiResponse(message = "JSON results", code = 200)
+@RequestMapping(value = "/validate", method = RequestMethod.POST)
+ResponseEntity post(@ApiParam(name = "grokValidation", 
value = "Object containing Grok statement and sample message", required = true) 
@RequestBody GrokValidation grokValidation) throws RestException {
--- End diff --

Looks like it is working now


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-metron issue #462: METRON-734 Builds failing because of MaxMind DB...

[cestella]

Github user cestella commented on the issue:

https://github.com/apache/incubator-metron/pull/462
  
Given the fact that we are without a working build, the failure here is a 
known sporadic failure and the Travis queue seems to be very backed up lately, 
I move that we wait for 3 hours and then commit this PR on the basis of 
Justin's Travis results linked above.

Objections?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-metron issue #462: METRON-734 Builds failing because of MaxMind DB...

[justinleet]

Github user justinleet commented on the issue:

https://github.com/apache/incubator-metron/pull/462
  
Apparently https://issues.apache.org/jira/browse/METRON-728 occurs more 
frequently on travis than my local machine.

The Travis running on my personal account already succeed 
(https://travis-ci.org/justinleet/incubator-metron/builds/204210174)

I'll kick Travis and hopefully we aren't waiting 16 hours for a build


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-metron pull request #462: METRON-734 Builds failing because of Max...

[justinleet]

Github user justinleet closed the pull request at:

https://github.com/apache/incubator-metron/pull/462


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-metron pull request #462: METRON-734 Builds failing because of Max...

[justinleet]

GitHub user justinleet reopened a pull request:

https://github.com/apache/incubator-metron/pull/462

METRON-734 Builds failing because of MaxMind DB transitive dependency

The pure exclude solution.  Still being tested on Vagrant, but making it 
available for viewing.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/justinleet/incubator-metron METRON-734

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-metron/pull/462.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #462


commit e3edf3773e2d1bd2029aa12be3a612be7487d809
Author: justinjleet 
Date:   2017-02-22T14:19:50Z

Run Travis for integration while spinning up Vagrant




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---