[ https://issues.apache.org/jira/browse/SPARK-1982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14014945#comment-14014945 ]
Andre Schumacher commented on SPARK-1982: ----------------------------------------- It turns out that ByteType primitive types weren't correctly treated earlier. Since Parquet doesn't have these one fix is to use fixed-length byte arrays (which are treated as primitives also). This is fine until there will be support for nested types. Even then I think one may want to treat these as actual arrays and not primitives. Anyway.. PR available here: https://github.com/apache/spark/pull/934 > saveToParquetFile doesn't support ByteType > ------------------------------------------ > > Key: SPARK-1982 > URL: https://issues.apache.org/jira/browse/SPARK-1982 > Project: Spark > Issue Type: Bug > Components: SQL > Affects Versions: 1.0.0 > Reporter: Michael Armbrust > Assignee: Andre Schumacher > > {code} > java.lang.RuntimeException: Unsupported datatype ByteType > at scala.sys.package$.error(package.scala:27) > at > org.apache.spark.sql.parquet.ParquetTypesConverter$.fromDataType(ParquetRelation.scala:201) > ... > {code} -- This message was sent by Atlassian JIRA (v6.2#6252)