Cheng Lian created SPARK-8848:
---------------------------------
Summary: Write Parquet LISTs and MAPs conforming to Parquet format
spec
Key: SPARK-8848
URL: https://issues.apache.org/jira/browse/SPARK-8848
Project: Spark
Issue Type: Sub-task
Components: SQL
Affects Versions: 1.4.0, 1.3.1, 1.2.2, 1.1.1, 1.0.2
Reporter: Cheng Lian
Assignee: Cheng Lian
[Parquet format PR #17|https://github.com/apache/parquet-format/pull/17]
standardized structures of Parquet complex types (LIST & MAP). Spark SQL should
follow this spec and write Parquet data conforming to the standard.
Note that although currently Parquet files written by Spark SQL is non-standard
(because Parquet format spec wasn't clear about this part when Spark SQL
Parquet support was authored), it's still compatible with the most recent
Parquet format spec, because the format we use is covered by the
backwards-compatibility rules.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
