RE: Using Map and Basic Operators yield java.lang.ClassCastException (Parquet + Hive + Spark SQL 1.5.0 + Thrift)

optional int64 consumed; } } } Thanks! Dominic Ricard Triton Digital -Original Message- From: Cheng Lian [mailto:lian.cs@gmail.com] Sent: Friday, September 25, 2015 1:41 PM To: Dominic Ricard; user@spark.apache.org Subject: Re: Using Map and Basic Operators yield java.lang.ClassCa

Using Map and Basic Operators yield java.lang.ClassCastException (Parquet + Hive + Spark SQL 1.5.0 + Thrift)

Hi, I stumbled on the following today. We have Parquet files that expose a column in a Map format. This is very convenient as we have data parts that can vary in time. Not knowing what the data will be, we simply split it in tuples and insert it as a map inside 1 column. Retrieving the data is

RE: Using Map and Basic Operators yield java.lang.ClassCastException (Parquet + Hive + Spark SQL 1.5.0 + Thrift)

some testing, that my problem is on how casting a Map value to the primitives Float and Double are handled. Handling INT is all good but float and double are causing the exception. Thanks. Dominic Ricard Triton Digital -Original Message- From: Cheng Lian [mailto:lian.cs@gmail.com] Sent:

HiveQL Compatibility (0.12.0, 0.13.0???)

Hi, here's a statement from the Spark 1.5.0 Spark SQL and DataFrame Guide : *Compatibility with Apache Hive* Spark SQL is designed to be compatible with the Hive Metastore, SerDes and UDFs.