For the interested there was also some extra context in the discussion at: https://github.com/apache/beam/pull/9779
Gleb mentioned the key two points: 1. The fact that Avro is exposed in the User API in beam-sdks-java-core was a mistake and makes fixing this issue backwards incompatible. 2. Shading is not an option because Avro compiler would generate specific records that won't match the non-vendored version so we will break user records compatibility (for example for users with a schema registry). So save if I am missing something and someone can give an alternative, we are in a situation where the only solution to the issue is to do (1), move Avro out of core as an extension but then the question is would we sacrifice breaking backwards compatibility for this issue. I am in the 'we should do it' camp. What do others think? On Thu, Jan 16, 2020 at 10:17 AM Gleb Kanterov <g...@spotify.com> wrote: > There are significant changes between Avro 1.8 and Avro 1.9. I'm not sure > it's possible for beam-sdks-java-core to support both at the same time. The > fact that AvroIO is a part of the beam-sdks-java-core doesn't make it > simpler. However, I can see how we can build two binary artifacts with the > same user-facing API each supporting own version of Avro. > > Shading or vendoring would be a breaking change because public signatures > of AvroIO (and few other IOs, for instance, BigQueryIO) refer to classes > from Avro, for instance, GenericRecord. Furthermore, a lot of Beam users > use Avro compiler to generate Java code for SpecificRecord, which would > refer to non-vendored version. >
