Thomas Omans created SPARK-7743:
-----------------------------------
Summary: Upgrade parquet dependency
Key: SPARK-7743
URL: https://issues.apache.org/jira/browse/SPARK-7743
Project: Spark
Issue Type: Bug
Reporter: Thomas Omans
There are many outstanding issues with the parquet format that have been
resolved between the version depended on by spark (1.6.0rc3 as of spark 1.3.1)
and the most recent parquet release (1.6.0).
Some of these are things include not supporting schema migration when using
parquet with avro, not supporting summary metadata in the parquet footers
causing null pointer exceptions reading, and many others.
See https://github.com/apache/parquet-mr/blob/master/CHANGES.md#version-160 for
the full list of fixes.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
