Balhau opened a new pull request #31809: URL: https://github.com/apache/spark/pull/31809
Currently Avro serialization is being done in two distinct ways. One is by using the BinaryEncoder/BinaryDecoder strategies provided by avro. However Avro also has the BinaryMessageEncoder/BinaryMessageDecoder strategies that uses the BinaryEncoder/BinaryDecoder to codify avro payloads but [adds a fingerprint in the beginning of the binary](https://github.com/apache/avro/blob/master/lang/java/avro/src/main/java/org/apache/avro/message/BinaryMessageEncoder.java) in this PR we use the MAGIC_NUMBER to decide if we need to strip or not the first 10 bytes that represent the MAGIC_NUMBER plus the 8 bytes of fingerprint. This way we can, transparently decode avro payloads that are encoded via one of those strategies -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
