Re: Re: how to extract arraytype data to file

2016-10-18 Thread lk_spark
Thank you, all of you. explode() is helpful: df.selectExpr("explode(bizs) as e").select("e.*").show() 2016-10-19 lk_spark 发件人:Hyukjin Kwon <gurwls...@gmail.com> 发送时间:2016-10-19 13:16 主题:Re: how to extract arraytype data to file 收件人:"Divya Gehlot"&l

Re: how to extract arraytype data to file

2016-10-18 Thread Hyukjin Kwon
This reminds me of https://github.com/databricks/spark-xml/issues/141#issuecomment-234835577 Maybe using explode() would be helpful. Thanks! 2016-10-19 14:05 GMT+09:00 Divya Gehlot : > http://stackoverflow.com/questions/33864389/how-can-i- >

Re: how to extract arraytype data to file

2016-10-18 Thread Divya Gehlot
http://stackoverflow.com/questions/33864389/how-can-i-create-a-spark-dataframe-from-a-nested-array-of-struct-element Hope this helps Thanks, Divya On 19 October 2016 at 11:35, lk_spark wrote: > hi,all: > I want to read a json file and search it by sql . > the data struct

RE: how to extract arraytype data to file

2016-10-18 Thread Kappaganthu, Sivaram (ES)
There is an option called Explode for this . From: lk_spark [mailto:lk_sp...@163.com] Sent: Wednesday, October 19, 2016 9:06 AM To: user.spark Subject: how to extract arraytype data to file hi,all: I want to read a json file and search it by sql . the data struct should be : bid: string

how to extract arraytype data to file

2016-10-18 Thread lk_spark
hi,all: I want to read a json file and search it by sql . the data struct should be : bid: string (nullable = true) code: string (nullable = true) and the json file data should be like : {bid":"MzI4MTI5MzcyNw==","code":"罗甸网警"} {"bid":"MzI3MzQ5Nzc2Nw==","code":"西早君"} but in fact my json