Hi,
I created a simple example in Java and wrote some Python to try to read the
record. I am getting the following error when trying to read the Java
record in Python.
Traceback (most recent call last):
File "/home/chad/python/avroReadTest/avro_read_binary_java.py", line 18,
in <module>
message = read_datum(java_binary_data, schema)
File "/home/chad/python/avroReadTest/avro_read_binary_java.py", line 10,
in read_datum
return datum_reader.read(decoder)
File "/home/chad/.local/lib/python3.8/site-packages/avro/io.py", line
626, in read
return self.read_data(self.writers_schema, self.readers_schema, decoder)
File "/home/chad/.local/lib/python3.8/site-packages/avro/io.py", line
698, in read_data
return self.read_record(writers_schema, readers_schema, decoder)
File "/home/chad/.local/lib/python3.8/site-packages/avro/io.py", line
898, in read_record
field_val = self.read_data(field.type, readers_field.type, decoder)
File "/home/chad/.local/lib/python3.8/site-packages/avro/io.py", line
655, in read_data
return decoder.read_utf8()
File "/home/chad/.local/lib/python3.8/site-packages/avro/io.py", line
312, in read_utf8
return unicode(self.read_bytes(), "utf-8")
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xc0 in position 2:
invalid start byte
Here is a link to the Java code.
https://gitlab.com/chad.preisler/avrojavabinaryencoderexample/-/blob/main/src/main/java/chad/preisler/avro/eamples/AvroWriteReadBinary.java
I'll admit I'm fairly new to Python. Here is my Python code.
import avro.io
import avro.schema
import io
def read_datum(buffer, writers_schema, readers_schema=None):
reader = io.BytesIO(buffer)
decoder = avro.io.BinaryDecoder(reader)
datum_reader = avro.io.DatumReader(writers_schema, readers_schema)
return datum_reader.read(decoder)
java_binary_data =
open("/home/chad/app_shared_resources/avroBinaryEncoderTest/java_binary_output.avo",
"rb").read()
schemaBytes =
open("/home/chad/app_shared_resources/avroBinaryEncoderTest/avroTestSchema.avsc",
"rb").read()
print ("Schema read in: " + schemaBytes.decode('UTF-8'))
schema = avro.schema.parse(schemaBytes)
print("Schema " + schema.__str__())
message = read_datum(java_binary_data, schema)
print(message)
I appreciate any help getting this working.
Thanks,
Chad
On Thu, May 27, 2021 at 12:56 PM Michael A. Smith <mich...@smith-li.com>
wrote:
> They should be compatible.
>
> Take a look at lang/py/avro/test/test_io.py in
>
> https://github.com/apache/avro
>
> Line 239 has a simple function that lays it out.
>
> If you encounter a way in which Java and Python are producing incompatible
> results, please let us know.
>
> On Thu, May 27, 2021 at 13:05 Chad Preisler <chad.preis...@gmail.com>
> wrote:
>
>> Hello,
>>
>> I am writing messages in Java using the BinaryMessageEncoder. I would
>> like to read the message in python. Is this supported, or is the format
>> written with BinaryMessageEncoder only supported in Java?
>>
>> If it is supported can you point me to a python example that reads the
>> binary message format in python?
>>
>> Thanks,
>> Chad
>>
>
