koert kuipers created SPARK-39770:
-------------------------------------
Summary: Support Avro schema evolution
Key: SPARK-39770
URL: https://issues.apache.org/jira/browse/SPARK-39770
Project: Spark
Issue Type: Improvement
Components: SQL
Affects Versions: 3.3.0
Reporter: koert kuipers
currently the avro source in connector/avro does not yet support schema
evolution.
from source code of AvroUtils:
{code:java}
// Schema evolution is not supported yet. Here we only pick first random
readable sample file to
// figure out the schema of the whole dataset.
{code}
i added schema evolution for our inhouse spark version. if there is interest in
this i could contribute it.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
