Re: get_json_object for nested field returning a String instead of an Array

2014-04-08 Thread Narayanan K
Thanks Yong! On Mon, Apr 7, 2014 at 5:07 PM, java8964 java8...@hotmail.com wrote: Hi, Narayanan: The current problem is that for a generic solution, there is no way that we know that element in the Json is an array. Keep in mind that in any element of Json, it could be any valid structure.

Re: get_json_object for nested field returning a String instead of an Array

2014-04-08 Thread David Quigley
Hi Narayanan, We have had some success with a similar use case using a custom input format / record reader to recursively split arbitrary json into a set of discreet records at runtime. No schema is needed. Doing something similar might give you the functionality you are looking for.

get_json_object for nested field returning a String instead of an Array

2014-04-07 Thread Narayanan K
Hi all I am using get_json_object to read a json text file. I have created the external table as below : CREATE EXTERNAL TABLE EXT_TABLE ( json string) PARTITIONED BY (dt string) LOCATION '/users/abc/'; The json data has some fields that are not simple fields but fields which are nested fields

Re: get_json_object for nested field returning a String instead of an Array

2014-04-07 Thread Peyman Mohajerian
perhaps: https://github.com/rcongiu/Hive-JSON-Serde On Mon, Apr 7, 2014 at 6:52 PM, Narayanan K knarayana...@gmail.com wrote: Hi all I am using get_json_object to read a json text file. I have created the external table as below : CREATE EXTERNAL TABLE EXT_TABLE ( json string)

Re: get_json_object for nested field returning a String instead of an Array

2014-04-07 Thread Narayanan K
Thanks Peyman. Actually the problem with Hive-Json-Serde is that we need to provide the entire schema upfront while creating the table. My requirement is that we just project/aggregate on the fields using get_json_object after creating the external table without schema. This way the external