This has come up a few times in the past, maybe this is an opportunity to figure out the best way to handle poorly created JSON records at scale? Without having to transform it?
Chris Matta [email protected] <javascript:_e(%7B%7D,'cvml','[email protected]');> 215-701-3146 On Mon, Sep 21, 2015 at 3:53 PM, John Omernik <[email protected] <javascript:_e(%7B%7D,'cvml','[email protected]');>> wrote: > Sorry about that, premature send. Here are some records. As you can see > the myvalue1-3 is in the top level of the record, ideally I'd run thte > kvgen on the myvalue records, but I have no way to address those. I tried > kvgen() on * for and that failed. Not sure how to address this in json, > yes, I know it's poorly formatted, but it's what I have been given. > > > > > > { "_id" : "127.0.0.1", "created_on" : "2014-02-18 14:52:23", "myvalue1" : { > "source" : "somestuff", "context" : "Context here", "last_seen" : > "2014-02-11 00:00:00", "refreshed" : "2014-03-12 18:14:23" } } > { "_id" : "127.0.0.2", "created_on" : "2014-02-18 14:52:08", "myvalue2" : { > "source" : "otherstuff", "context" : "Special context", "last_seen" : > "2014-02-26 18:14:05", "refreshed" : "2014-02-26 18:14:05" } } > { "_id" : "127.0.0.3", "created_on" : "2014-04-25 00:08:17", "myvalue3" : { > "source" : "oops", "context" : "Other Context, "last_seen" : "2014-04-25 > 05:32:08", "refreshed" : "2014-04-25 05:32:08" } } > > On Mon, Sep 21, 2015 at 2:52 PM, John Omernik <[email protected] > <javascript:_e(%7B%7D,'cvml','[email protected]');>> wrote: > > > The challenge I have is the data was poorly formatted, here's some > records > > > > > > > > > > > > On Mon, Sep 21, 2015 at 2:14 PM, Jim Scott <[email protected] > <javascript:_e(%7B%7D,'cvml','[email protected]');>> wrote: > > > >> I do believe KVGEN will meet your needs: > >> https://drill.apache.org/docs/kvgen/ > >> > >> On Mon, Sep 21, 2015 at 2:11 PM, John Omernik <[email protected] > <javascript:_e(%7B%7D,'cvml','[email protected]');>> wrote: > >> > >> > I have some poorly developed json where the developer used data for > key > >> > names > >> > > >> > {"created":"2015-12-01", "ZYS":"BLAH"} > >> > {"created":"2015-12-01", "ZYX":"BLAH"} > >> > {"created":"2015-12-01", "ABC":"BLAH"} > >> > {"created":"2015-12-01", "ADS":"BLAH"} > >> > > >> > I'd like to somehow map the key name to a value and give it a generic > >> name > >> > > >> > select `created`, somemagic() as value1 from table > >> > > >> > Not sure how this would work, or if it's possible, or how I'd even > >> > reference that, but thought I would ask. > >> > > >> > John > >> > > >> > >> > >> > >> -- > >> *Jim Scott* > >> Director, Enterprise Strategy & Architecture > >> +1 (347) 746-9281 > >> @kingmesal <https://twitter.com/kingmesal> > >> > >> <http://www.mapr.com/> > >> [image: MapR Technologies] <http://www.mapr.com> > >> > >> Now Available - Free Hadoop On-Demand Training > >> < > >> > http://www.mapr.com/training?utm_source=Email&utm_medium=Signature&utm_campaign=Free%20available > >> > > >> > > > > > -- Chris Matta [email protected] 215-701-3146
