Re: SparkSQL returns ArrayBuffer for fields of type Array

2014-08-27 Thread Michael Armbrust
> > From: Michael Armbrust > Date: Wednesday, August 27, 2014 at 5:21 PM > To: Du Li > Cc: "user@spark.apache.org" > Subject: Re: SparkSQL returns ArrayBuffer for fields of type Array > > Arrays in the JVM are also mutable. However, you should not be relyi

Re: SparkSQL returns ArrayBuffer for fields of type Array

2014-08-27 Thread Du Li
mailto:mich...@databricks.com>> Date: Wednesday, August 27, 2014 at 5:21 PM To: Du Li mailto:l...@yahoo-inc.com>> Cc: "user@spark.apache.org<mailto:user@spark.apache.org>" mailto:user@spark.apache.org>> Subject: Re: SparkSQL returns ArrayBuffer for fields of type

Re: SparkSQL returns ArrayBuffer for fields of type Array

2014-08-27 Thread Michael Armbrust
Arrays in the JVM are also mutable. However, you should not be relying on the exact type here. The only promise is that you will get back something of type Seq[_]. On Wed, Aug 27, 2014 at 4:27 PM, Du Li wrote: > Hi, Michael. > > I used HiveContext to create a table with a field of type Arra

SparkSQL returns ArrayBuffer for fields of type Array

2014-08-27 Thread Du Li
Hi, Michael. I used HiveContext to create a table with a field of type Array. However, in the hql results, this field was returned as type ArrayBuffer which is mutable. Would it make more sense to be an Array? The Spark version of my test is 1.0.2. I haven’t tested it on SQLContext nor newer v