Re: spark dataframe transform JSON to ORC meet “column ambigous exception”

2015-09-12 Thread Ted Yu
Can you take a look at SPARK-5278 where ambiguity is shown between field names which differ only by case ? Cheers On Sat, Sep 12, 2015 at 3:40 AM, Fengdong Yu wrote: > Hi Ted, > I checked the JSON, there aren't duplicated key in JSON. > > > Azuryy Yu > Sr. Infrastructure Engineer > > cel: 158-0

Re: spark dataframe transform JSON to ORC meet “column ambigous exception”

2015-09-12 Thread Fengdong Yu
Hi Ted, I checked the JSON, there aren't duplicated key in JSON. Azuryy Yu Sr. Infrastructure Engineer cel: 158-0164-9103 wetchat: azuryy On Sat, Sep 12, 2015 at 5:52 PM, Ted Yu wrote: > Is it possible that Canonical_URL occurs more than once in your json ? > > Can you check your json input

Re: spark dataframe transform JSON to ORC meet “column ambigous exception”

2015-09-12 Thread Ted Yu
Is it possible that Canonical_URL occurs more than once in your json ? Can you check your json input ? Thanks On Sat, Sep 12, 2015 at 2:05 AM, Fengdong Yu wrote: > Hi, > > I am using spark1.4.1 data frame, read JSON data, then save it to orc. the > code is very simple: > > DataFrame json = sql

spark dataframe transform JSON to ORC meet “column ambigous exception”

2015-09-12 Thread Fengdong Yu
Hi, I am using spark1.4.1 data frame, read JSON data, then save it to orc. the code is very simple: DataFrame json = sqlContext.read().json(input); json.write().format("orc").save(output); the job failed. what's wrong with this exception? Thanks. Exception in thread "main" org.apache.spark.sq