This is an automated email from the ASF dual-hosted git repository.
potiuk pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/airflow.git
The following commit(s) were added to refs/heads/main by this push:
new 0261bc57bc Updates release notes for snakebite-py3 incompatibility
with protobuf (#31756)
0261bc57bc is described below
commit 0261bc57bcad3c1f3c0b7f3a4683584462cf1e0e
Author: Jarek Potiuk <[email protected]>
AuthorDate: Wed Jun 7 16:55:49 2023 +0200
Updates release notes for snakebite-py3 incompatibility with protobuf
(#31756)
Users that would like to use previous version of hdfs provider with
new Airlfow will have to configure their environment temporarily to use
Python based protobuf seralization.
---
airflow/providers/apache/hdfs/CHANGELOG.rst | 19 +++++++++++++++++++
1 file changed, 19 insertions(+)
diff --git a/airflow/providers/apache/hdfs/CHANGELOG.rst
b/airflow/providers/apache/hdfs/CHANGELOG.rst
index 3425068011..3fa097f90e 100644
--- a/airflow/providers/apache/hdfs/CHANGELOG.rst
+++ b/airflow/providers/apache/hdfs/CHANGELOG.rst
@@ -49,6 +49,25 @@ you can use 3.* version of the provider, but the
recommendation is to switch to
* ``Remove snakebite-py3 based HDFS hooks and sensors (#31262)``
+
+.. note::
+
+ Protobuf 3 required by the snakebite-py3 library has ended its life in June
2023 and Airflow and it's
+ providers stopped supporting it. If you would like to continue using HDFS
hooks and sensors
+ based on snakebite-py3 library when you have protobuf library 4.+ you can
install the 3.* version
+ of the provider but due to Protobuf incompatibility, you need to do one of
the the two things:
+
+ * set ``PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python`` variable in your
environment.
+ * downgrade protobuf to latest 3.* version (3.20.3 at this time)
+
+ Setting ``PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python`` will make many
libraries using protobuf
+ much slower - including multiple Google client libraries and Kubernetes.
Downgrading protobuf to
+ (already End-Of-Life) 3.* version will make some of the latest versions of
the new providers
+ incompatible (for example google and grpc) and you will have to downgrade
those providers as well.
+ Both should be treated as a temporary workaround only, and you should
migrate to WebHDFS
+ as soon as possible.
+
+
Misc
~~~~
