Totally agree with replacing dot with something else. We have had so much
drama to use either dot or column with ORC either via Hive or Spark.
Although we have replaced it with an underscore, it may not be a good idea
as it can be confusing with underscores in the internal field names.
Cheers,
I propose that we just disallow having dots in the field name. Dots seem to
have a special meaning and as we keep adding data stores we may run into some
unintended behavior. We should have logic in our code to check for it and
either auto-correct it (replace with underscores?) or at least
Internal means it’s not configurable, doesn’t contain our default separator
(dots) and is namespaced with metron. We can definitely improve on DRY but
there’s more to it than that. For example, having 2 different versions of this
field name (ES and Solr) adds a significant amount of
Can you elaborate on what you mean by "convert to internal?" From your
description, it looks like the challenge is from our violations of DRY when
it comes to constants referencing those keys, which would be eliminated by
refactoring.
On Fri, Sep 7, 2018, 3:50 PM Ryan Merriman wrote:
> I
