ganeshashree commented on PR #3963: URL: https://github.com/apache/avro/pull/3963#issuecomment-6010513758
Spark 4.1.0, which uses Avro 1.12.1, is affected by this issue. Avro 1.12.1 enables the fast reader by default. This affects long-running jobs, such as Spark Structured Streaming, that read many Avro files with distinct writer schemas, due to the accumulation of reader objects on the executor heap. @RyanSkraba @iemejia Could we have this reviewed and merged soon? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
