We've been dealing with a reoccurring challenge in Metron. It is common for various fields to contain '.' characters for the purpose of making them more readable, namespacing, etc. At one point we only supported Elasticsearch 2.3 which did not allow dots and forced us to use ':' instead. This limitation does not exist in later versions of Elasticsearch or Solr.
Now we're in a situation where we need to allow a user to use either one because they may still be using ES 2.3 or have data with ':' characters in field names. We've attempted to make this configurable in a couple different PRs: https://github.com/apache/metron/pull/1022 https://github.com/apache/metron/pull/1010 https://github.com/apache/metron/pull/1038 The approaches taken in these are not consistent and fall short in different ways. The first (METRON-1569 Allow user to change field name conversion when indexing) only applies to indexing and not querying. The others only apply to a single field which does not scale well. Now we have an issue with another field in https://issues.apache.org/jira/browse/METRON-1600. Rather than continuing with a patchwork of different fixes I want to attempt to design a system-wide solution. My first thought is to expand https://github.com/apache/metron/pull/1022 to apply globally. However this is not trivial and would require significant changes. It would also make https://github.com/apache/metron/pull/1010 obsolete and we might end up having to revert all of it. Does anyone have any ideas or opinions? I am still researching solutions but would love some guidance from the community.
