I agree with having a metron_ prefix for ES indexes, and the timing. Jon
On Wed, Jan 24, 2018 at 3:20 PM Michael Miklavcic < michael.miklav...@gmail.com> wrote: > With the completion of https://github.com/apache/metron/pull/840 > (METRON-939: Upgrade ElasticSearch and Kibana), we have the makings for a > major release rev of Metron in the upcoming release (currently slotted to > 0.4.3, I believe). Since there are non-backwards compatible changes > pertaining to ES indexing, it seems like a good opportunity to revisit our > index naming standards. > > I propose we add a simple prefix "metron_" to all Metron indexes. There are > numerous reasons for doing so > > - removes the likelihood of index name collisions when we perform > operations on index wildcard names, e.g. "enrichment_*, indexing_*, > etc.". > - ie, this allows us to be more friendly in a multi-tenant ES > environment for relatively low engineering cost. > - simplifies the Kibana dashboard a bit. We currently needed to create a > special index pattern in order to accommodate multi-index pattern > matching > across all metron-specific indexes. Using metron_* would be much simpler > and less prone to error. > - easier for customers to debug and identify Metron-specific indexes and > associated data > > > The reason for making these changes now is that we already have breaking > changes with ES. Leveraging existing indexed data rather than deleting > indexes and starting from scractch already requires a re-indexing/migration > step, so there is no additional effort on the part of users if they choose > to attempt a migration. It further makes sense with our current work > towards upgrading Solr. > > We already have a battery of integration and manual tests after the ES > upgrade work that can be leveraged to validate the changes. > > Mike Miklavcic > -- Jon
