I would suggest to make fastavro a default option on Python 3, for the lack
of alternative at the moment, but keep current default on Python 2.
I would also keep both avro and avro-python3 dependencies and associated
codepaths.
This way, we will gradually increase the usage of fastavro, but keep
I agree with Ahmet.
Fastavro seems to be well maintained and has good, tested
compatibility. Unless we expect significant performance improvements
in the standard Avro Python package (a significant undertaking, likely
not one we have the bandwidth to take on, and my impression is that
it's
Hi all,
Thank you for the feedback. Looking at the responses, it seems like there
is a consensus to move forward with fastavro as the default implementation
on Python 3.
There are 2 questions left however:
- Should fastavro also become the default implementation on Python 2?
This is a trade-off
Hi Ismaël,
It is great to hear that Avro is planning to make a release soon.
To answer your concerns, fastavro has a set of tests using regular avro
files[1] and it also has a large set of users (with 675470 package
downloads). This is in addition to it being a py2 & py3 compatible package
and
Hello,
The problem of switching implementations is the risk of losing
interoperability, and this is more important than performance. Does
fastavro have tests that guarantee that it is fully compatible with
Avro’s Java version? (given that it is the de-facto implementation
used everywhere).
If
+1 for making use_fastavro the default for Python3. I don't see any
significant drawbacks in doing this from Beam's point of view. One concern
is whether avro and fastavro can safely co-exist in the same environment so
that Beam continues to work for users who already have avro library
installed.
Thanks, Robbe and Frederik, for raising this.
Over the course of making Beam Python 3 compatible this is at least the
second time [1] we have to deal with an error in avro-python3 package. The
release cadence of Apache Avro (1 release a year)
is concerning to me [2]. Even if we have a new release
