With the completion of https://github.com/apache/metron/pull/840 (METRON-939: Upgrade ElasticSearch and Kibana), we have the makings for a major release rev of Metron in the upcoming release (currently slotted to 0.4.3, I believe). Since there are non-backwards compatible changes pertaining to ES indexing, it seems like a good opportunity to revisit our index naming standards.
I propose we add a simple prefix "metron_" to all Metron indexes. There are numerous reasons for doing so - removes the likelihood of index name collisions when we perform operations on index wildcard names, e.g. "enrichment_*, indexing_*, etc.". - ie, this allows us to be more friendly in a multi-tenant ES environment for relatively low engineering cost. - simplifies the Kibana dashboard a bit. We currently needed to create a special index pattern in order to accommodate multi-index pattern matching across all metron-specific indexes. Using metron_* would be much simpler and less prone to error. - easier for customers to debug and identify Metron-specific indexes and associated data The reason for making these changes now is that we already have breaking changes with ES. Leveraging existing indexed data rather than deleting indexes and starting from scractch already requires a re-indexing/migration step, so there is no additional effort on the part of users if they choose to attempt a migration. It further makes sense with our current work towards upgrading Solr. We already have a battery of integration and manual tests after the ES upgrade work that can be leveraged to validate the changes. Mike Miklavcic
