Github user liancheng commented on a diff in the pull request:
https://github.com/apache/spark/pull/7070#discussion_r33704967
--- Diff: sql/core/src/main/scala/org/apache/spark/sql/SQLConf.scala ---
@@ -227,6 +227,13 @@ private[spark] object SQLConf {
defaultValue = Some(true),
doc = "<TODO>")
+ val PARQUET_MERGE_SCHEMA_ENABLED =
booleanConf("spark.sql.parquet.mergeSchema",
+ defaultValue = Some(true),
+ doc = "Turn on the schema merge feature of parquet datasource API. " +
+ "Enable it will merge different schema of parquet" +
+ "Disable it will spped up the parquet schema loading time if all
your parquet " +
+ "schema is the same" )
--- End diff --
Please reword `doc` as following:
> When true, the Parquet data source merges schemas collected from all data
files, otherwise the schema is picked from the summary file or a random data
file if no summary file is available.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
