Re: Dataframe in apache spark Error java.lang.ArrayIndexOutOfBoundsException:

http://stackoverflow.com/questions/33864389/how-can-i-create-a-spark-dataframe-from-a-nested-array-of-struct-element This is similar to your use case Hope this helps Thanks, Divya On 16 October 2016 at 19:03, Mark Mikolajczak - 07855 306 064 < m...@flayranalytics.co.uk> wrote: > > > down votef

Re: Dataframe in apache spark Error java.lang.ArrayIndexOutOfBoundsException:

thanks Divya, I tried that and still get same error - array out of bounds. Full code. import org.apache.spark.sql.sources.EqualTo import com.couchbase.spark.sql._ import com.couchbase.spark.sql.N1QLRelation \val bkEvent = Map("bucket" -> "events") val eve = sqlc.read.couchbase(schemaFilter = Eq

Re: Dataframe in apache spark Error java.lang.ArrayIndexOutOfBoundsException:

Try with eve.select(col("id"),col("customerEvent.customerContext. anonymousCustomerChanges.analyticsId.id ")).show This should work Thanks, Divya On 16 October 2016 at 19:59, Mark Mikolajczak - 07855 306 064 < m..

Re: Dataframe in apache spark Error java.lang.ArrayIndexOutOfBoundsException:

Thanks Kim. Sorry It was a typo as I was doing the second part from my phone (Laptop battery died). The code I was using in zeppelin is: eve.select($"id",$"customerEvent.customerContext.anonymousCustomerChanges.analyticsId.id").show I tried with and with out the $ sign but still get the same

Re: Dataframe in apache spark Error java.lang.ArrayIndexOutOfBoundsException:

Hi, Mark Mikolajczak I think it should be eve.select("id", " customerEvent.customerContext.anonymousCustomerChanges.analyticsId.id") You missed 'customerEvent'! :-) 2016년 10월 16일 (일) 오후 8:32, Flayranalytics 님이 작성: Hi, Sorry forgot to include the code. eve.select($"id",$"customerContext.anony

Re: Dataframe in apache spark Error java.lang.ArrayIndexOutOfBoundsException:

Hi, Sorry forgot to include the code. eve.select($"id",$"customerContext.anonymousCustomerChanges.analyticsId.id").show() If I just use Id it works. I am seeing this happen when the data is an array within my data. Sent from my 📲 > On 16 Oct 2016, at 12:03, Mark Mikolajczak - 07855 306 06

Dataframe in apache spark Error java.lang.ArrayIndexOutOfBoundsException:

down vote <>favorite Hi, I have setup a sp