[
https://issues.apache.org/jira/browse/SPARK-38651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-38651:
------------------------------------
Assignee: Apache Spark
> Writing out empty or nested empty schemas in Datasource should be configurable
> ------------------------------------------------------------------------------
>
> Key: SPARK-38651
> URL: https://issues.apache.org/jira/browse/SPARK-38651
> Project: Spark
> Issue Type: Task
> Components: SQL
> Affects Versions: 2.4.0
> Reporter: Thejdeep Gudivada
> Assignee: Apache Spark
> Priority: Major
>
> In SPARK-23372, we introduced a backwards incompatible change that would
> remove support for writing out empty or nested empty schemas in file based
> datasources. This introduces backward incompatibility for users who have been
> using a schema that met the above condition since the datasource supported
> it. Except for Parquet and text, other file based sources support this
> behavior.
>
> We should either :
> * Make it configurable to enable/disable writing out empty schemas
> * Enable the validation check only for sources that do not support it -
> Parquet / Text
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
