Cheng Lian created SPARK-6775:
---------------------------------
Summary: Simplify CatalystConverter class hierarchy and pass in
Parquet schema
Key: SPARK-6775
URL: https://issues.apache.org/jira/browse/SPARK-6775
Project: Spark
Issue Type: Sub-task
Components: SQL
Affects Versions: 1.3.0, 1.2.1, 1.1.1, 1.0.2
Reporter: Cheng Lian
Assignee: Cheng Lian
{{CataystConverter}} classes are used to convert Parquet records to Spark SQL
row objects. Current converter implementations have the following problems:
# They simply ignore original Parquet schema, which makes adding Parquet
backwards-compatibility rules impossible.
# They are unnecessary over complicated.
# {{SpecificMutableRow}} is only used for structs whose fields are all of
primitive types.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
