mina-asham commented on code in PR #26008:
URL: https://github.com/apache/flink/pull/26008#discussion_r1928952489
##########
flink-python/setup.py:
##########
@@ -318,7 +318,7 @@ def extracted_output_files(base_dir, file_path,
output_directory):
install_requires = ['py4j==0.10.9.7', 'python-dateutil>=2.8.0,<3',
'apache-beam>=2.54.0,<=2.61.0',
- 'cloudpickle>=2.2.0', 'avro-python3>=1.8.1,!=1.9.2',
+ 'cloudpickle>=2.2.0', 'avro>=1.12.0',
Review Comment:
> There seems to be slight differences between the old avro-python3 and the
new preferred avro packages. I assume the other changes in this pr relate to
the slight changes. The original code seems to be tolerating a range of
versions, do we know why? Or is the latest fine?
I think we are fine, basically `avro-python3` was deprecated in favour of
`avro`, it's the same package just renamed, but they also did some breaking
changes in these updates (changing package names, slightly changing some names,
etc...), that's what the code changes here account for. We still account for a
range of versions here though, which afaik is a Python best practice so that
you don't lock up consumers to a certain library.
> I am curious whether fastavro needs to be updated to be compatible with
avro 1.12.0 in any way. fastavro 1.1 is 5 years old - should we push this to
the latest as well.
`fastavro` is completely unrelated (beats me with Flink is using two
separate Avro libraries tbh), `avro-python3/avro` is a pure Python avro
implementation, `fastavro` is written in C to offer faster execution. Might be
worth using a single Python package for avro or upgrading avro, but I think
that's a bigger effort and should be out of scope of this smaller change here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
