[
https://issues.apache.org/jira/browse/SPARK-38651?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17512202#comment-17512202
]
Apache Spark commented on SPARK-38651:
--------------------------------------
User 'thejdeep' has created a pull request for this issue:
https://github.com/apache/spark/pull/35969
> Writing out empty or nested empty schemas in Datasource should be configurable
> ------------------------------------------------------------------------------
>
> Key: SPARK-38651
> URL: https://issues.apache.org/jira/browse/SPARK-38651
> Project: Spark
> Issue Type: Task
> Components: SQL
> Affects Versions: 2.4.0
> Reporter: Thejdeep Gudivada
> Priority: Major
>
> In SPARK-23372, we introduced a backwards incompatible change that would
> remove support for writing out empty or nested empty schemas in file based
> datasources. This introduces backward incompatibility for users who have been
> using a schema that met the above condition since the datasource supported
> it. Except for Parquet and text, other file based sources support this
> behavior.
>
> We should either :
> * Make it configurable to enable/disable writing out empty schemas
> * Enable the validation check only for sources that do not support it -
> Parquet / Text
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
