bmsilva opened a new issue #10174: URL: https://github.com/apache/pulsar/issues/10174
Hi, I step into this problem when using AvroSchema and connecting to pulsar for read. The writers are implemented in Java, and in my case that I'm reading using python with the pulsar-client, I noticed that the schema doesn't respect the field order and it sorts the field alphabetically, link to the code here: https://github.com/apache/pulsar/blob/3d7a6bae1d4de4eaa399d45b8f59b92b0f090faa/pulsar-client-cpp/python/pulsar/schema/definition.py#L83 This is a problem because the java writer doesn't sort the fields, so 2 bugs happen, first when subscribing the topic, pulsar doesn't validate the Schema correctly, and then you get an error when calling method `value` in the read message. A workaround for me was reimplement the `schema` method from `Record` class. example: ```python class MyRecord(pschema.Record): c = pschema.String(required=False) b = pschema.String(required=False) a = pschema.String(required=False) @classmethod def schema(cls): schema = { 'name': str(cls.__name__), 'type': 'record', 'fields': [] } for name in cls._fields.keys(): # removed "sorted" from here field = cls._fields[name] field_type = field.schema() if field._required else ['null', field.schema()] field_schema = { 'name': name, 'type': field_type } # also add problem when `required=False` and had to add `default` if not field._required: field_schema['default'] = None schema['fields'].append(field_schema) return schema ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
