I agree Ali. May be it can be configuration parameter. On Tue, Aug 14, 2018 at 3:24 PM Ali Nazemian <alinazem...@gmail.com> wrote:
> Hi Simon, > > We have temporarily decided to just change it with "_" for HDFS to avoid > all the headaches of the bugs and issues that can be raised by using > unsupported separators for ORC/Hive and Spark. However, I am not quite > confident with "_" as an option for the community as it becomes similar to > normal Metron separator. Maybe it would be nice to have an ability to > change the separator to any other character and let users decide what they > want to use. > > Cheers, > Ali > > On Tue, Aug 14, 2018 at 12:14 AM Simon Elliston Ball < > si...@simonellistonball.com> wrote: > > > Do you have any suggestions for what would make sense as a delimiter? > > > > On 9 August 2018 at 05:57, Ali Nazemian <alinazem...@gmail.com> wrote: > > > > > Hi All, > > > > > > I was wondering if we can change the field separators in Metron to be > > able > > > to make it Hive/ORC friendly. I could find the following PR, but > neither > > > dot nor colon is very Hive and ORC friendly and they will cause some > > > issues. Hence, I wanted to see if it is possible to change the field > > > separator to something else or even give users an ability to define > what > > > separator to be used to make the data model consistent across > > Elasticsearch > > > and HDFS. > > > > > > https://github.com/apache/metron/pull/1022 > > > > > > Cheers, > > > Ali > > > > > > > > > > > -- > > -- > > simon elliston ball > > @sireb > > > > > -- > A.Nazemian >
