[GitHub] metron issue #636: METRON-1022: Elasticsearch REST endpoint

2017-07-19 Thread ottobackwards
Github user ottobackwards commented on the issue: https://github.com/apache/metron/pull/636 +1 - great job --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so,

[GitHub] metron issue #636: METRON-1022: Elasticsearch REST endpoint

2017-07-19 Thread merrimanr
Github user merrimanr commented on the issue: https://github.com/apache/metron/pull/636 I added this endpoint to the README with the latest commit and also tested on full dev with Kerberos enabled. --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] metron issue #636: METRON-1022: Elasticsearch REST endpoint

2017-07-19 Thread merrimanr
Github user merrimanr commented on the issue: https://github.com/apache/metron/pull/636 What do you think @ottobackwards? Are you good with this? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] metron issue #636: METRON-1022: Elasticsearch REST endpoint

2017-07-19 Thread cestella
Github user cestella commented on the issue: https://github.com/apache/metron/pull/636 Ok, this appears to me like good infrastructure to base further index-oriented REST calls atop of. I'm +1; good job, @merrimanr ! --- If your project is set up for it, you can reply to this emai

[GitHub] metron issue #636: METRON-1022: Elasticsearch REST endpoint

2017-07-13 Thread cestella
Github user cestella commented on the issue: https://github.com/apache/metron/pull/636 Bear with me, @merrimanr, I am going to submit a PR with the DAO abstraction I was talking about so we can hash it out. I started it in collaboration with @justinleet to ensure the ideas in the mut

[GitHub] metron issue #636: METRON-1022: Elasticsearch REST endpoint

2017-07-12 Thread cestella
Github user cestella commented on the issue: https://github.com/apache/metron/pull/636 @ottobackwards agreed, we should work to that goal. I think we might want to make baby steps, though, and create the abstractions first and move to making it pluggable later. --- If your project

[GitHub] metron issue #636: METRON-1022: Elasticsearch REST endpoint

2017-07-12 Thread ottobackwards
Github user ottobackwards commented on the issue: https://github.com/apache/metron/pull/636 @cestella https://issues.apache.org/jira/browse/METRON-956 Is the jira I had created on this topic. --- If your project is set up for it, you can reply to this email and have your repl

[GitHub] metron issue #636: METRON-1022: Elasticsearch REST endpoint

2017-07-12 Thread cestella
Github user cestella commented on the issue: https://github.com/apache/metron/pull/636 I just want to follow-up with something a bit more concrete suggestions. I think the beginnings of an abstraction are there. You pulled out a bunch of utility methods from `ElasticsearchWriter` wh

[GitHub] metron issue #636: METRON-1022: Elasticsearch REST endpoint

2017-07-12 Thread cestella
Github user cestella commented on the issue: https://github.com/apache/metron/pull/636 Looking at this implementation and working a bit on the PoC for index data mutation, I think the abstraction here isn't in the right place. It's too bound-up in the REST layer whereas data access h

[GitHub] metron issue #636: METRON-1022: Elasticsearch REST endpoint

2017-07-11 Thread merrimanr
Github user merrimanr commented on the issue: https://github.com/apache/metron/pull/636 The latest commit enforces a maximum page size. A value of 1000 is set by default. As it currently stands, you would need to add this property to the rest_application.yml file manually if it need

[GitHub] metron issue #636: METRON-1022: Elasticsearch REST endpoint

2017-07-10 Thread ottobackwards
Github user ottobackwards commented on the issue: https://github.com/apache/metron/pull/636 If the es or solr code was packaged as a bundle ( it with it's dependencies -> the es and solr libs ), the bundle system let you load each implementation as a plugin, and will create a new clas

[GitHub] metron issue #636: METRON-1022: Elasticsearch REST endpoint

2017-07-10 Thread merrimanr
Github user merrimanr commented on the issue: https://github.com/apache/metron/pull/636 The core issue is that both Solr and Elasticsearch depend on Lucene. Our version of Elasticsearch (2.3.3) is old so there are lots of Lucene class version conflicts when adding Solr as a dependenc

[GitHub] metron issue #636: METRON-1022: Elasticsearch REST endpoint

2017-07-10 Thread ottobackwards
Github user ottobackwards commented on the issue: https://github.com/apache/metron/pull/636 What kind of isolation we do need? The 777 introduces bundle loading gives classloader isolation If they were loaded as plugins then it may work. --- If your project is set up for it, you

[GitHub] metron issue #636: METRON-1022: Elasticsearch REST endpoint

2017-07-10 Thread merrimanr
Github user merrimanr commented on the issue: https://github.com/apache/metron/pull/636 @ottobackwards just created a discuss thread on the general approach here. @cestella you're absolutely correct. We need a way for solr and ES to coexist. A precedent has already been set

[GitHub] metron issue #636: METRON-1022: Elasticsearch REST endpoint

2017-07-10 Thread cestella
Github user cestella commented on the issue: https://github.com/apache/metron/pull/636 This is good work! Thanks, Ryan. We desperately need this abstraction. I think the confusion may be that this PR is the general abstraction as well as a concrete implementation for one index. I

[GitHub] metron issue #636: METRON-1022: Elasticsearch REST endpoint

2017-07-06 Thread ottobackwards
Github user ottobackwards commented on the issue: https://github.com/apache/metron/pull/636 @merrimanr, I'm obviously feeling a little confused about what seems like 3 related things going on. I don't mean to say this is the wrong approach. Would you feel comfortable opening a [DISC

[GitHub] metron issue #636: METRON-1022: Elasticsearch REST endpoint

2017-07-06 Thread ottobackwards
Github user ottobackwards commented on the issue: https://github.com/apache/metron/pull/636 I feel like this and the alert UI are building the transcontinental railroad from each end of the county, and are going to miss each other by a state. How does this relate to that effo