xushiyan commented on issue #3635: URL: https://github.com/apache/hudi/issues/3635#issuecomment-926332253
@deshpandeanoop noticed that you used a different version `"spark-avro" % "2.4.4"` rather than your main spark version 2.3.1 A more important matter: it has been discussed long time back where we only support spark 2.4+ https://lists.apache.org/thread.html/r19ec206e33f8b63e95a840bfba519ab89d1b7af790adef0bc369d618%40%3Cdev.hudi.apache.org%3E Strongly suggest that you make efforts to upgrade Spark in some way. there are important avro updates in newer version. Spark 2.3.1 uses avro 1.7.7 which can be very problematic https://github.com/apache/spark/blob/30aaa5a3a1076ca52439a905274b1fcf498bc562/pom.xml#L142 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
