Hello Gabor, This Spark PR upgrades Parquet but does not upgrade Avro, note the exclusion for parquet-avro
https://github.com/apache/spark/pull/26804/files#diff-600376dffeb79835ede4a0b285078036R2104 <https://github.com/apache/spark/pull/26804/files#diff-600376dffeb79835ede4a0b285078036R2104> Parquet 1.11.0/1.11.1 depends on Avro 1.9.1 and Spark depends on Avro 1.8.2, how will this Spark PR be compatible? michael > On Jan 23, 2020, at 3:38 AM, Gabor Szadovszky <[email protected]> wrote: > > Thanks, Fokko. I've linked the related issues to the release jira as > blockers. > Currently, every issue is resolved. Waiting for feedback if the > fixes/descriptions are correct and if we need to fix anything else for > Spark. > > On Wed, Jan 22, 2020 at 5:18 PM Driesprong, Fokko <[email protected]> > wrote: > >> Thank you Gabor, >> >> What kind of issues are found? Let me know if I can help in any way. >> >> Cheers, Fokko >> >> Op wo 22 jan. 2020 om 11:10 schreef Gabor Szadovszky <[email protected]>: >> >>> Dear All, >>> >>> During the migration to 1.11.0 in Spark we discovered some issues in the >>> parquet release. I am preparing the minor release 1.11.1 to fix these >>> issues. Created the jira >>> https://issues.apache.org/jira/browse/PARQUET-1774 to >>> track this effort. Feel free to link any bug jiras if they are >> regressions >>> in 1.11.0. >>> The release will be prepared in the separate branch "parquet-1.11.x". >> I'll >>> do the backports as required. >>> >>> Regards, >>> Gabor >>> >>
