Re: [DISCUSS] Update Metron Elasticsearch index names to metron_

2018-01-26 Thread James Sirota
i am +1 on it then 26.01.2018, 15:56, "Michael Miklavcic" : > Just checked on the length issue - we should be good - > https://github.com/elastic/elasticsearch/issues/8079 > > On Fri, Jan 26, 2018 at 3:37 PM, James Sirota wrote: > >>  Seems

Re: [DISCUSS] Update Metron Elasticsearch index names to metron_

2018-01-26 Thread Michael Miklavcic
Just checked on the length issue - we should be good - https://github.com/elastic/elasticsearch/issues/8079 On Fri, Jan 26, 2018 at 3:37 PM, James Sirota wrote: > Seems reasonable to me. The only thing is that it may make the index > names too long. Not sure if that matters

Re: [DISCUSS] Update Metron Elasticsearch index names to metron_

2018-01-26 Thread James Sirota
Seems reasonable to me. The only thing is that it may make the index names too long. Not sure if that matters to ES or not 26.01.2018, 15:32, "Simon Elliston Ball" : > +1 on this. The idea of a default broad matching template should also include > an order entry

Re: [DISCUSS] Update Metron Elasticsearch index names to metron_

2018-01-26 Thread Simon Elliston Ball
+1 on this. The idea of a default broad matching template should also include an order entry to avoid conflicts with more specific templates, and we should then document the need for a higher order value in all per-source index templates. In terms of production migration, I think we may want

Re: [DISCUSS] Update Metron Elasticsearch index names to metron_

2018-01-24 Thread Michael Miklavcic
One other benefit of this revised approach - we can more effectively use index template patterns to specify our base set of Metron property types. Call me crazy, but I think we should be able to do something like: { *"template": "metron_*",* "mappings": { "metron_doc": {

Re: [DISCUSS] Update Metron Elasticsearch index names to metron_

2018-01-24 Thread Michael Miklavcic
I hear you Ali. I think this type of change would actually ease issues with downtime because it offers an easy path to migrating existing indices. I'd have to review the specifics in the ES docs again, but I believe you could duplicate the old indexes and migrate them to "metron_" in advance of

Re: [DISCUSS] Update Metron Elasticsearch index names to metron_

2018-01-24 Thread Ali Nazemian
Hi All, I just wanted to say it would be great if we can be careful with these type of changes. From the development point of view, it is just a few lines of code which can provide multiple advantages, but for live large-scale Metron platforms, some of these changes might be really expensive to

Re: [DISCUSS] Update Metron Elasticsearch index names to metron_

2018-01-24 Thread Otto Fowler
+1 On January 24, 2018 at 16:28:42, Nick Allen (n...@nickallen.org) wrote: +1 to a standard prefix for all Metron indices. I've had the same thought myself and you laid out the advantages well. On Wed, Jan 24, 2018 at 3:47 PM zeo...@gmail.com wrote: > I agree with

Re: [DISCUSS] Update Metron Elasticsearch index names to metron_

2018-01-24 Thread Nick Allen
+1 to a standard prefix for all Metron indices. I've had the same thought myself and you laid out the advantages well. On Wed, Jan 24, 2018 at 3:47 PM zeo...@gmail.com wrote: > I agree with having a metron_ prefix for ES indexes, and the timing. > > Jon > > On Wed, Jan

Re: [DISCUSS] Update Metron Elasticsearch index names to metron_

2018-01-24 Thread zeo...@gmail.com
I agree with having a metron_ prefix for ES indexes, and the timing. Jon On Wed, Jan 24, 2018 at 3:20 PM Michael Miklavcic < michael.miklav...@gmail.com> wrote: > With the completion of https://github.com/apache/metron/pull/840 > (METRON-939: Upgrade ElasticSearch and Kibana), we have the

[DISCUSS] Update Metron Elasticsearch index names to metron_

2018-01-24 Thread Michael Miklavcic
With the completion of https://github.com/apache/metron/pull/840 (METRON-939: Upgrade ElasticSearch and Kibana), we have the makings for a major release rev of Metron in the upcoming release (currently slotted to 0.4.3, I believe). Since there are non-backwards compatible changes pertaining to ES