Using json_tuple for Nested json Arrays

Hi, Is it possible to use json_tuple function to extract data from json arrays (nested too). I am trying to process json data as string and avoid using serdes since user data may be malformed. Please see a sample json data given below: { "filter_level": "low", "retweeted": false,

Re: Using json_tuple for Nested json Arrays

I tried using EXPLODE function on the nested json array but it doesn't work and throws following error: FAILED: UDFArgumentException explode() takes an array or a map as a parameter Thanks, Joel On Tue, Oct 27, 2015 at 3:20 PM, Sam Joe wrote: > Hi, > > Is it possible

RE: Using json_tuple for Nested json Arrays

10; You should also be able to use json_tuple(), but start simple From: Sam Joe [mailto:games2013@gmail.com] Sent: Tuesday, October 27, 2015 1:43 PM To: user@hive.apache.org Subject: Re: Using json_tuple for Nested json Arrays Hi Ryan, Thanks for your reply! I didn't try using

RE: Using json_tuple for Nested json Arrays

ubject: Re: Using json_tuple for Nested json Arrays Hi Ryan, The simple query is running fine as shown below: hive> SELECT tr2.id<http://tr2.id>, tr2.possibly_sensitive > FROM tweets_raw tr1 > LATERAL VIEW json_tuple(tr1.text_col, 'id', 'extended_entities', 'possi

RE: Using json_tuple for Nested json Arrays

, October 27, 2015 1:29 PM To: user@hive.apache.org Subject: Re: Using json_tuple for Nested json Arrays I tried using EXPLODE function on the nested json array but it doesn't work and throws following error: FAILED: UDFArgumentException explode() takes an array or a map as a parameter Thanks, Joel

Re: Using json_tuple for Nested json Arrays

T get_json_object(text_col, '$.id') as id FROM tweets_raw limit 10; > > > > You should also be able to use json_tuple(), but start simple > > > > *From:* Sam Joe [mailto:games2013@gmail.com] > *Sent:* Tuesday, October 27, 2015 1:43 PM > > *To:* use

Re: Using json_tuple for Nested json Arrays

Tuesday, October 27, 2015 1:29 PM > *To:* user@hive.apache.org > *Subject:* Re: Using json_tuple for Nested json Arrays > > > > I tried using EXPLODE function on the nested json array but it doesn't > work and throws following error: > > > > FAILED: UDFArgumentExcepti

Re: Using json_tuple for Nested json Arrays

55},"large":{"w":1024,"resize":"fit","h":768},"medium":{"w":600,"resize":"fit","h":450}},"source_user_id":16864598,"media_url":" http://pbs.twimg.com/media/CRSL2M8WcAEXowZ.jpg&q

RE: Using json_tuple for Nested json Arrays

as a parameter Apparently, hive doesn't think tr3.media is an array or map..so what is it? From: Sam Joe [mailto:games2013@gmail.com] Sent: Tuesday, October 27, 2015 2:56 PM To: user@hive.apache.org Subject: Re: Using json_tuple for Nested json Arrays Hi Ryan, The statement returns null for media

Re: Using json_tuple for Nested json Arrays

..so what is > it? > > > > *From:* Sam Joe [mailto:games2013@gmail.com] > *Sent:* Tuesday, October 27, 2015 2:56 PM > > *To:* user@hive.apache.org > *Subject:* Re: Using json_tuple for Nested json Arrays > > > > Hi Ryan, > > > > The statement returns

Re: Using json_tuple for Nested json Arrays

Hello Sam, You can easily achieve this by using elephant-bird.jars in pig. We are also caturing tweets via flume and filter them using pig and elephant-jars. You can find the related jars over internet. Cheers, Nishant Aggarwal On 28 Oct 2015 00:50, "Sam Joe" wrote: >

RE: Using json_tuple for Nested json Arrays

just need to troubleshoot the explode() From: Sam Joe [mailto:games2013@gmail.com] Sent: Tuesday, October 27, 2015 7:37 PM To: user@hive.apache.org Subject: Re: Using json_tuple for Nested json Arrays Hi Ryan, I think tr3.media a complex json array having nested json tuple objects

Re: Using json_tuple for Nested json Arrays

Thanks Nishant! Will try using Pig json loader too to achieve this requirement. If you have any tutorial for extracting data from complex nested json arrays (as the example given in my previous email), please send it. Appreciate your help! Thanks, Joel On Tue, Oct 27, 2015 at 10:20 PM, Nishant

Re: Using json_tuple for Nested json Arrays

Hi, > If you have any tutorial for extracting data from complex nested json >arrays (as the example given in my previous email), please send it. 90% of working with the real world is cleansing bad data. People under-sell hive's flexibility in situations like this. This is what I do hive>

Re: Using json_tuple for Nested json Arrays

Hello Sam, Please find attached PIG script for the same. You may find the necessary jars below. http://mvnrepository.com/artifact/com.twitter.elephantbird/elephant-bird-pig Note: Same functionality can be achieved in Hive as well. Thanks and Regards Nishant Aggarwal, PMP Cell No:- +91 99588