Abacn commented on code in PR #34542:
URL: https://github.com/apache/beam/pull/34542#discussion_r2034348191
##########
sdks/python/apache_beam/io/avroio.py:
##########
@@ -646,9 +646,9 @@ def avro_atomic_value_to_beam_atomic_value(avro_type: str,
value):
if value is None:
return value
elif avro_type == "int":
- return ctypes.c_uint32(value).value
Review Comment:
We no longer need avro_atomic_value_to_beam_atomic_value (and
beam_atomic_value_to_avro_atomic_value) after #30770. I tested in a branch:
https://github.com/Abacn/beam/commit/979940b1c7b2cf083c654199dc56cae1efa2670d
where I find `test_avro_schema_to_beam_schema_with_nullable_atomic_fields`
can pass, and no longer experience int overflow in intermediate results:
```
_______________________________________________________________________
TestFastAvro.test_avro_schema_to_beam_schema_with_nullable_atomic_fields
________________________________________________________________________
--------- Captured stdout call ----------
Row(name='Thomas', favorite_number=1, favorite_color='blue')
Row(name='Henry', favorite_number=3, favorite_color='green')
Row(name='Toby', favorite_number=7, favorite_color='brown')
Row(name='Gordon', favorite_number=4, favorite_color='blue')
Row(name='Emily', favorite_number=-1, favorite_color='Red')
Row(name='Percy', favorite_number=6, favorite_color='Green')
Row(name='Bruce', favorite_number=None, favorite_color=None)
BeamSchema_(name='Thomas', favorite_number=1, favorite_color='blue')
BeamSchema_(name='Henry', favorite_number=3, favorite_color='green')
BeamSchema_(name='Toby', favorite_number=7, favorite_color='brown')
BeamSchema_(name='Gordon', favorite_number=4, favorite_color='blue')
BeamSchema_(name='Emily', favorite_number=-1, favorite_color='Red')
BeamSchema_(name='Percy', favorite_number=6, favorite_color='Green')
BeamSchema_(name='Bruce', favorite_number=None, favorite_color=None)
======= 1 passed, 1 warning in 11.70s =======
```
To test, build sql expansion service first, then run `pytest -rP -v
apache_beam/io/avroio_test.py::TestFastAvro::test_avro_schema_to_beam_schema_with_nullable_atomic_fields`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
