Here is the schema https://gitlab.com/chad.preisler/avrojavabinaryencoderexample/-/blob/main/avroTestSchema.avsc
On Fri, May 28, 2021 at 9:13 AM Michael A. Smith <mich...@smith-li.com> wrote: > Hi, Chad, > > Did you share the schema somewhere? Is that something you're able to share? > > On Fri, May 28, 2021 at 10:00 AM Chad Preisler <chad.preis...@gmail.com> > wrote: > > > > Hi, > > I created a simple example in Java and wrote some Python to try to read > the record. I am getting the following error when trying to read the Java > record in Python. > > > > Traceback (most recent call last): > > File "/home/chad/python/avroReadTest/avro_read_binary_java.py", line > 18, in <module> > > message = read_datum(java_binary_data, schema) > > File "/home/chad/python/avroReadTest/avro_read_binary_java.py", line > 10, in read_datum > > return datum_reader.read(decoder) > > File "/home/chad/.local/lib/python3.8/site-packages/avro/io.py", line > 626, in read > > return self.read_data(self.writers_schema, self.readers_schema, > decoder) > > File "/home/chad/.local/lib/python3.8/site-packages/avro/io.py", line > 698, in read_data > > return self.read_record(writers_schema, readers_schema, decoder) > > File "/home/chad/.local/lib/python3.8/site-packages/avro/io.py", line > 898, in read_record > > field_val = self.read_data(field.type, readers_field.type, decoder) > > File "/home/chad/.local/lib/python3.8/site-packages/avro/io.py", line > 655, in read_data > > return decoder.read_utf8() > > File "/home/chad/.local/lib/python3.8/site-packages/avro/io.py", line > 312, in read_utf8 > > return unicode(self.read_bytes(), "utf-8") > > UnicodeDecodeError: 'utf-8' codec can't decode byte 0xc0 in position 2: > invalid start byte > > > > Here is a link to the Java code. > > > https://gitlab.com/chad.preisler/avrojavabinaryencoderexample/-/blob/main/src/main/java/chad/preisler/avro/eamples/AvroWriteReadBinary.java > > > > I'll admit I'm fairly new to Python. Here is my Python code. > > > > import avro.io > > import avro.schema > > import io > > > > > > def read_datum(buffer, writers_schema, readers_schema=None): > > reader = io.BytesIO(buffer) > > decoder = avro.io.BinaryDecoder(reader) > > datum_reader = avro.io.DatumReader(writers_schema, readers_schema) > > return datum_reader.read(decoder) > > > > > > java_binary_data = > open("/home/chad/app_shared_resources/avroBinaryEncoderTest/java_binary_output.avo", > "rb").read() > > schemaBytes = > open("/home/chad/app_shared_resources/avroBinaryEncoderTest/avroTestSchema.avsc", > "rb").read() > > print ("Schema read in: " + schemaBytes.decode('UTF-8')) > > schema = avro.schema.parse(schemaBytes) > > print("Schema " + schema.__str__()) > > message = read_datum(java_binary_data, schema) > > print(message) > > > > I appreciate any help getting this working. > > > > Thanks, > > Chad > > > > On Thu, May 27, 2021 at 12:56 PM Michael A. Smith <mich...@smith-li.com> > wrote: > >> > >> They should be compatible. > >> > >> Take a look at lang/py/avro/test/test_io.py in > >> > >> https://github.com/apache/avro > >> > >> Line 239 has a simple function that lays it out. > >> > >> If you encounter a way in which Java and Python are producing > incompatible results, please let us know. > >> > >> On Thu, May 27, 2021 at 13:05 Chad Preisler <chad.preis...@gmail.com> > wrote: > >>> > >>> Hello, > >>> > >>> I am writing messages in Java using the BinaryMessageEncoder. I would > like to read the message in python. Is this supported, or is the format > written with BinaryMessageEncoder only supported in Java? > >>> > >>> If it is supported can you point me to a python example that reads the > binary message format in python? > >>> > >>> Thanks, > >>> Chad >
