Bo Zhang created SPARK-33316:
--------------------------------
Summary: Support nullable Avro schemas for non-nullable data in
Avro writing
Key: SPARK-33316
URL: https://issues.apache.org/jira/browse/SPARK-33316
Project: Spark
Issue Type: Improvement
Components: SQL
Affects Versions: 3.0.1, 3.0.0, 2.4.0
Reporter: Bo Zhang
Currently when users try to use nullable Avro schemas for non-nullable data in
Avro writing, Spark will throw a IncompatibleSchemaException.
There are some cases when users do not have full control over the nullability
of the data, or the nullability of the Avro schemas they have to use. We should
support nullable Avro schemas for non-nullable data in Avro writing for better
usability.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
