Cheng Lian created SPARK-10400:
----------------------------------
Summary: Rename or deprecate SQL option
"spark.sql.parquet.followParquetFormatSpec"
Key: SPARK-10400
URL: https://issues.apache.org/jira/browse/SPARK-10400
Project: Spark
Issue Type: Improvement
Components: SQL
Affects Versions: 1.5.0
Reporter: Cheng Lian
Assignee: Cheng Lian
Priority: Minor
We introduced SQL option "spark.sql.parquet.followParquetFormatSpec" while
working on implementing Parquet backwards-compatibility rules in SPARK-6777. It
indicates whether we should use legacy Parquet format adopted by Spark 1.4 and
prior versions or the standard format defined in parquet-format spec. However,
the name of this option is somewhat confusing, because it's not super intuitive
why we shouldn't follow the spec. Would be nice to rename it to
"spark.sql.parquet.writeLegacyFormat" and invert its default value (they have
opposite meanings). Note that this option is not "public" ({{isPublic}} is
false).
At the moment of writing, 1.5 RC3 has already been cut. If we can't make this
one into 1.5, we can deprecate the old option with the new one.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
