Kevin Mader created ZEPPELIN-43:
-----------------------------------
Summary: SQL and automatic tuple conversion
Key: ZEPPELIN-43
URL: https://issues.apache.org/jira/browse/ZEPPELIN-43
Project: Zeppelin
Issue Type: Bug
Components: Core
Affects Versions: 0.5.0
Reporter: Kevin Mader
Priority: Minor
If you let the toDF and implicit conversion in SparkSQL convert a case class
with a tuple as one of the fields, you cannot access this in the %sql
environment.
The test example I have is a field structured as follows
```
case class ShapeInformation2D(label: (Long, Double), comX: Double, comY:
Double, extentsX: Double,extentsY: Double, area: Long)
```
which in SparkSQL is represented as
```
org.apache.spark.sql.DataFrame = [label: struct<_1:bigint,_2:double>, comX:
double, comY: double, extentsX: double, extentsY: double, area: bigint]
```
```
%sql
select comX, comY, label._1 from mapdata
```
Results in error ```cannot recognize input near 'label' '.' '_1' in select
expression; line 1 pos 25```
While running the same command in pure spark
```
%spark
{
import sqlContext.implicits.rddToDataFrameHolder
rddToDataFrameHolder(mapTestRDD).toDF().registerTempTable("test")
}
val data = sqlContext.sql("select comX, comY, label._1 from test")
```
Results (as expected) in
```
data: org.apache.spark.sql.DataFrame = [comX: double, comY: double, _1: bigint]
```
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
