[
https://issues.apache.org/jira/browse/SPARK-1922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Reynold Xin resolved SPARK-1922.
--------------------------------
Resolution: Fixed
> hql query throws "RuntimeException: Unsupported dataType" if struct field of
> a table has a column with underscore in name
> -------------------------------------------------------------------------------------------------------------------------
>
> Key: SPARK-1922
> URL: https://issues.apache.org/jira/browse/SPARK-1922
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 1.0.0
> Reporter: llai
> Assignee: llai
> Fix For: 1.1.0, 1.0.1
>
>
> If table A has a struct field A_strct <a_col: string>, when doing an hql
> query like "select", "RuntimeException: Unsupported dataType" is thrown.
> Running a query wiht "sbt/sbt hive/console":
> {code}
> scala> hql("SELECT utc_time, pkg FROM pkg_table where year=2014 and month=1
> limit 10").collect().foreach(x => println(x(1)))
> {code}
> Console output:
> {code}
> 14/05/25 19:50:27 INFO parse.ParseDriver: Parsing command: SELECT utc_time,
> pkg FROM pkg_table where year=2014 and month=1 limit 10
> 14/05/25 19:50:27 INFO parse.ParseDriver: Parse Completed
> 14/05/25 19:50:28 INFO analysis.Analyzer: Max iterations (2) reached for
> batch MultiInstanceRelations
> 14/05/25 19:50:28 INFO analysis.Analyzer: Max iterations (2) reached for
> batch CaseInsensitiveAttributeReferences
> 14/05/25 19:50:28 INFO hive.metastore: Trying to connect to metastore with
> URI thrift://xxxxx
> 14/05/25 19:50:28 INFO hive.metastore: Waiting 1 seconds before next
> connection attempt.
> 14/05/25 19:50:29 INFO hive.metastore: Connected to metastore.
> java.lang.RuntimeException: Unsupported dataType:
> struct<adv_id:string,u_lat:double,u_lon:double>
> at scala.sys.package$.error(package.scala:27)
> at
> org.apache.spark.sql.hive.HiveMetastoreTypes$.toDataType(HiveMetastoreCatalog.scala:219)
> at
> org.apache.spark.sql.hive.MetastoreRelation$SchemaAttribute.toAttribute(HiveMetastoreCatalog.scala:273)
> at
> org.apache.spark.sql.hive.MetastoreRelation$$anonfun$8.apply(HiveMetastoreCatalog.scala:283)
> at
> org.apache.spark.sql.hive.MetastoreRelation$$anonfun$8.apply(HiveMetastoreCatalog.scala:283)
> at
> scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244)
> {code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
