benkonz commented on code in PR #30770:
URL: https://github.com/apache/beam/pull/30770#discussion_r1566452436
##########
sdks/python/apache_beam/io/avroio.py:
##########
@@ -649,7 +684,11 @@ def beam_schema_to_avro_schema(
def beam_type_to_avro_type(beam_type: schema_pb2.FieldType) -> _AvroSchemaType:
type_info = beam_type.WhichOneof("type_info")
if type_info == "atomic_type":
- return {'type': BEAM_PRIMITIVES_TO_AVRO_PRIMITIVES[beam_type.atomic_type]}
+ avro_primitive = BEAM_PRIMITIVES_TO_AVRO_PRIMITIVES[beam_type.atomic_type]
+ if beam_type.nullable:
+ return ['null', avro_primitive]
Review Comment:
@ahmedabu98 I tried to get the function to work by returning `{ 'type':
['null', primitive] }`, however kept getting that `TypeError: unhashable type:
'list'` exception with fastavro unfortunately. I updated the code to only
return the list if we encounter a nullable field, so it shouldn't affect any
previously working code hopefully. If you know of a way to get avro to accept
something like:
```json
{
"type": "record",
"name": "",
"fields": [
{
"name": "name",
"type": {
"type": "string"
}
},
{
"name": "favorite_number",
"type": {
"type": [
"int",
"null"
]
}
},
{
"name": "favorite_color",
"type": {
"type": [
"string",
"null"
]
}
}
]
}
```
please let me know!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
