[jira] [Resolved] (SPARK-9685) "Unsupported dataType: char(X)" in Hive

2016-10-08 Thread Sean Owen (JIRA) 

 [ 
https://issues.apache.org/jira/browse/SPARK-9685?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sean Owen resolved SPARK-9685.
--
Resolution: Duplicate

> "Unsupported dataType: char(X)" in Hive
> ---
>
> Key: SPARK-9685
> URL: https://issues.apache.org/jira/browse/SPARK-9685
> Project: Spark
>  Issue Type: Bug
>  Components: SQL
>Affects Versions: 1.5.0
>Reporter: Ángel Álvarez
> Attachments: SPARK-9685.1.patch.txt
>
>
> I'm getting the following error when I try to read a Hive table with char(X) 
> fields:
> {code}
> 15/08/06 11:38:51 INFO parse.ParseDriver: Parse Completed
> org.apache.spark.sql.types.DataTypeException: Unsupported dataType: char(8). 
> If you have a struct and a field name of it has any special characters, 
> please use backticks (`) to quote that field name, e.g. `x+y`. Please note 
> that backtick itself is not supported in a field name.
> at 
> org.apache.spark.sql.types.DataTypeParser$class.toDataType(DataTypeParser.scala:95)
> at 
> org.apache.spark.sql.types.DataTypeParser$$anon$1.toDataType(DataTypeParser.scala:107)
> at 
> org.apache.spark.sql.types.DataTypeParser$.parse(DataTypeParser.scala:111)
> at 
> org.apache.spark.sql.hive.HiveMetastoreTypes$.toDataType(HiveMetastoreCatalog.scala:769)
> at 
> org.apache.spark.sql.hive.MetastoreRelation$SchemaAttribute.toAttribute(HiveMetastoreCatalog.scala:742)
> at 
> org.apache.spark.sql.hive.MetastoreRelation$$anonfun$44.apply(HiveMetastoreCatalog.scala:752)
> at 
> org.apache.spark.sql.hive.MetastoreRelation$$anonfun$44.apply(HiveMetastoreCatalog.scala:752)
> {code}
> It seems there is no "char" DataType defined in the DataTypeParser class
> {code}
>   protected lazy val primitiveType: Parser[DataType] =
> "(?i)string".r ^^^ StringType |
> "(?i)float".r ^^^ FloatType |
> "(?i)(?:int|integer)".r ^^^ IntegerType |
> "(?i)tinyint".r ^^^ ByteType |
> "(?i)smallint".r ^^^ ShortType |
> "(?i)double".r ^^^ DoubleType |
> "(?i)(?:bigint|long)".r ^^^ LongType |
> "(?i)binary".r ^^^ BinaryType |
> "(?i)boolean".r ^^^ BooleanType |
> fixedDecimalType |
> "(?i)decimal".r ^^^ DecimalType.USER_DEFAULT |
> "(?i)date".r ^^^ DateType |
> "(?i)timestamp".r ^^^ TimestampType |
> varchar
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org

[jira] [Resolved] (SPARK-9685) "Unsupported dataType: char(X)" in Hive

2016-10-08 Thread Xiao Li (JIRA) 

 [ 
https://issues.apache.org/jira/browse/SPARK-9685?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Xiao Li resolved SPARK-9685.

Resolution: Fixed

> "Unsupported dataType: char(X)" in Hive
> ---
>
> Key: SPARK-9685
> URL: https://issues.apache.org/jira/browse/SPARK-9685
> Project: Spark
>  Issue Type: Bug
>  Components: SQL
>Affects Versions: 1.5.0
>Reporter: Ángel Álvarez
> Attachments: SPARK-9685.1.patch.txt
>
>
> I'm getting the following error when I try to read a Hive table with char(X) 
> fields:
> {code}
> 15/08/06 11:38:51 INFO parse.ParseDriver: Parse Completed
> org.apache.spark.sql.types.DataTypeException: Unsupported dataType: char(8). 
> If you have a struct and a field name of it has any special characters, 
> please use backticks (`) to quote that field name, e.g. `x+y`. Please note 
> that backtick itself is not supported in a field name.
> at 
> org.apache.spark.sql.types.DataTypeParser$class.toDataType(DataTypeParser.scala:95)
> at 
> org.apache.spark.sql.types.DataTypeParser$$anon$1.toDataType(DataTypeParser.scala:107)
> at 
> org.apache.spark.sql.types.DataTypeParser$.parse(DataTypeParser.scala:111)
> at 
> org.apache.spark.sql.hive.HiveMetastoreTypes$.toDataType(HiveMetastoreCatalog.scala:769)
> at 
> org.apache.spark.sql.hive.MetastoreRelation$SchemaAttribute.toAttribute(HiveMetastoreCatalog.scala:742)
> at 
> org.apache.spark.sql.hive.MetastoreRelation$$anonfun$44.apply(HiveMetastoreCatalog.scala:752)
> at 
> org.apache.spark.sql.hive.MetastoreRelation$$anonfun$44.apply(HiveMetastoreCatalog.scala:752)
> {code}
> It seems there is no "char" DataType defined in the DataTypeParser class
> {code}
>   protected lazy val primitiveType: Parser[DataType] =
> "(?i)string".r ^^^ StringType |
> "(?i)float".r ^^^ FloatType |
> "(?i)(?:int|integer)".r ^^^ IntegerType |
> "(?i)tinyint".r ^^^ ByteType |
> "(?i)smallint".r ^^^ ShortType |
> "(?i)double".r ^^^ DoubleType |
> "(?i)(?:bigint|long)".r ^^^ LongType |
> "(?i)binary".r ^^^ BinaryType |
> "(?i)boolean".r ^^^ BooleanType |
> fixedDecimalType |
> "(?i)decimal".r ^^^ DecimalType.USER_DEFAULT |
> "(?i)date".r ^^^ DateType |
> "(?i)timestamp".r ^^^ TimestampType |
> varchar
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org