Jon, What would you use case be for comparison? Reconciliation of the sources? In theory both should be identical since they’re indexed from the same source. There should never be any reason to combine ES and HDFS indexing, unless there is a use case I’m missing...
Simon > On 9 May 2017, at 15:00, zeo...@gmail.com <zeo...@gmail.com> wrote: > > Have we ever considered the use case where we might want to compare data > across our stores, like ES and HDFS? I think there is some merit in > keeping the keys consistent across them (and doing transforms earlier in > the stack rather than later). > > Jon > > On Tue, May 9, 2017 at 9:53 AM Otto Fowler <ottobackwa...@gmail.com> wrote: > >> ElasticsearchFieldNameConverter.java >> >> >> >> On May 9, 2017 at 09:46:09, Nick Allen (n...@nickallen.org) wrote: >> >> The ES indexer replaces periods, I believe. Are you seeing periods hit >> Elasticsearch? >> >> And architecturally that kind of logic should be done in the indexers >> anyways. >> >> On Tue, May 9, 2017 at 9:41 AM, zeo...@gmail.com <zeo...@gmail.com> wrote: >> >>> Is there a reason why the bro parser allows periods >>> <https://github.com/apache/incubator-metron/blob/master/ >>> metron-platform/metron-parsers/src/main/java/org/ >>> apache/metron/parsers/bro/JSONCleaner.java#L56> >>> in the keys if we can't index it (ES 2.3.3 does not allow periods in >>> indexes)? Would anybody be opposed to me modifying the bro parser to >>> rewrite "."s to "_"s for the short term, until we get ES upgraded >>> <https://issues.apache.org/jira/browse/METRON-939>? >>> >>> Jon >>> -- >>> >>> Jon >>> >> > -- > > Jon
