seongwon77 opened a new issue, #25929: URL: https://github.com/apache/airflow/issues/25929
### Apache Airflow Provider(s) google ### Versions of Apache Airflow Providers alembic==1.8.1 anyio==3.6.1 apache-airflow==2.3.4 apache-airflow-providers-common-sql==1.1.0 apache-airflow-providers-ftp==3.1.0 apache-airflow-providers-google==8.3.0 apache-airflow-providers-http==4.0.0 apache-airflow-providers-imap==3.0.0 apache-airflow-providers-sqlite==3.2.0 apispec==3.3.2 argcomplete==2.0.0 attrs==22.1.0 Babel==2.10.3 blinker==1.5 cachelib==0.9.0 cachetools==5.2.0 cattrs==22.1.0 certifi==2022.6.15 cffi==1.15.1 charset-normalizer==2.1.1 click==8.1.3 clickclick==20.10.2 colorama==0.4.5 colorlog==4.8.0 commonmark==0.9.1 connexion==2.14.1 cron-descriptor==1.2.31 croniter==1.3.5 cryptography==37.0.4 db-dtypes==1.0.3 Deprecated==1.2.13 dill==0.3.5.1 dnspython==2.2.1 docutils==0.19 email-validator==1.2.1 exceptiongroup==1.0.0rc8 Flask==2.2.2 Flask-AppBuilder==4.1.3 Flask-Babel==2.0.0 Flask-Caching==2.0.1 Flask-JWT-Extended==4.4.4 Flask-Login==0.6.2 Flask-Session==0.4.0 Flask-SQLAlchemy==2.5.1 Flask-WTF==0.15.1 future==0.18.2 google-ads==18.1.0 google-api-core==2.8.2 google-api-python-client==1.12.11 google-auth==2.11.0 google-auth-httplib2==0.1.0 google-auth-oauthlib==0.5.2 google-cloud-aiplatform==1.13.1 google-cloud-appengine-logging==1.1.4 google-cloud-audit-log==0.2.4 google-cloud-automl==2.8.1 google-cloud-bigquery==2.34.3 google-cloud-bigquery-datatransfer==3.7.1 google-cloud-bigquery-storage==2.14.2 google-cloud-bigtable==1.7.1 google-cloud-build==3.9.1 google-cloud-container==2.11.2 google-cloud-core==2.3.2 google-cloud-datacatalog==3.9.1 google-cloud-dataform==0.2.1 google-cloud-dataplex==1.1.1 google-cloud-dataproc==5.0.1 google-cloud-dataproc-metastore==1.6.1 google-cloud-dlp==1.0.1 google-cloud-kms==2.12.1 google-cloud-language==1.3.1 google-cloud-logging==3.2.2 google-cloud-memcache==1.4.2 google-cloud-monitoring==2.11.1 google-cloud-orchestration-airflow==1.4.2 google-cloud-os-login==2.7.2 google-cloud-pubsub==2.13.6 google-cloud-redis==2.9.1 google-cloud-resource-manager==1.6.1 google-cloud-secret-manager==1.0.1 google-cloud-spanner==1.19.2 google-cloud-speech==1.3.3 google-cloud-storage==1.44.0 google-cloud-tasks==2.10.2 google-cloud-texttospeech==1.0.2 google-cloud-translate==1.7.1 google-cloud-videointelligence==1.16.2 google-cloud-vision==1.0.1 google-cloud-workflows==1.7.2 google-crc32c==1.3.0 google-resumable-media==2.3.3 googleapis-common-protos==1.56.4 graphviz==0.20.1 grpc-google-iam-v1==0.12.4 grpcio==1.47.0 grpcio-gcp==0.2.2 grpcio-status==1.47.0 gunicorn==20.1.0 h11==0.12.0 httpcore==0.15.0 httplib2==0.20.4 httpx==0.23.0 idna==3.3 importlib-metadata==4.12.0 importlib-resources==5.9.0 inflection==0.5.1 itsdangerous==2.1.2 Jinja2==3.1.2 json-merge-patch==0.2 jsonschema==4.14.0 lazy-object-proxy==1.7.1 linkify-it-py==2.0.0 lockfile==0.12.2 looker-sdk==22.10.0 Mako==1.2.1 Markdown==3.4.1 markdown-it-py==2.1.0 MarkupSafe==2.1.1 marshmallow==3.17.1 marshmallow-enum==1.5.1 marshmallow-oneofschema==3.0.1 marshmallow-sqlalchemy==0.26.1 mdit-py-plugins==0.3.0 mdurl==0.1.2 numpy==1.23.2 oauthlib==3.2.0 packaging==21.3 pandas==1.4.3 pandas-gbq==0.17.8 pathspec==0.9.0 pendulum==2.1.2 pkgutil_resolve_name==1.3.10 pluggy==1.0.0 prison==0.2.1 proto-plus==1.22.0 protobuf==4.21.5 psutil==5.9.1 pyarrow==6.0.1 pyasn1==0.4.8 pyasn1-modules==0.2.8 pycparser==2.21 pydata-google-auth==1.4.0 Pygments==2.13.0 PyJWT==2.4.0 pyOpenSSL==22.0.0 pyparsing==3.0.9 pyrsistent==0.18.1 python-daemon==2.3.1 python-dateutil==2.8.2 python-nvd3==0.15.0 python-slugify==6.1.2 pytz==2022.2.1 pytzdata==2020.1 PyYAML==6.0 requests==2.28.1 requests-oauthlib==1.3.1 requests-toolbelt==0.9.1 rfc3986==1.5.0 rich==12.5.1 rsa==4.9 setproctitle==1.3.2 six==1.16.0 sniffio==1.2.0 SQLAlchemy==1.4.27 sqlalchemy-bigquery==1.4.4 SQLAlchemy-JSONField==1.0.0 SQLAlchemy-Utils==0.38.3 sqlparse==0.4.2 swagger-ui-bundle==0.0.9 tabulate==0.8.10 tenacity==8.0.1 termcolor==1.1.0 text-unidecode==1.3 typing_extensions==4.3.0 uc-micro-py==1.0.1 unicodecsv==0.14.1 uritemplate==3.0.1 urllib3==1.26.12 Werkzeug==2.2.2 wrapt==1.14.1 WTForms==2.3.3 zipp==3.8.1 ### Apache Airflow version 2.3.4 ### Operating System Debian 11 bullseye ### Deployment Official Apache Airflow Helm Chart ### Deployment details _No response_ ### What happened Starting from 18.1.0 of the package `google-ads`, one of its constraint has change, which is `protobuf >= 4.21.5`. - Before: https://github.com/googleads/google-ads-python/blob/18.0.0/setup.py#L34 - After: https://github.com/googleads/google-ads-python/blob/18.1.0/setup.py#L31 With 18.1.0 of `google-ads`, it tries to install `protobuf==4.21.5`. Then, the Airflow DB migration job fails when initiating an Airflow instance with the following log. (I deployed the instance to a k8s cluster using the official Helm chart) ``` Info 2022-08-24 02:33:47.645 PDTTraceback (most recent call last): File "/home/airflow/.local/bin/airflow", line 5, in <module> from airflow.__main__ import main File "/home/airflow/.local/lib/python3.8/site-packages/airflow/__init__.py", line 35, in <module> from airflow import settings File "/home/airflow/.local/lib/python3.8/site-packages/airflow/settings.py", line 35, in <module> from airflow.configuration import AIRFLOW_HOME, WEBSERVER_CONFIG, conf # NOQA F401 File "/home/airflow/.local/lib/python3.8/site-packages/airflow/configuration.py", line 1618, in <module> secrets_backend_list = initialize_secrets_backends() File "/home/airflow/.local/lib/python3.8/site-packages/airflow/configuration.py", line 1540, in initialize_secrets_backends custom_secret_backend = get_custom_secret_backend() File "/home/airflow/.local/lib/python3.8/site-packages/airflow/configuration.py", line 1515, in get_custom_secret_backend secrets_backend_cls = conf.getimport(section='secrets', key='backend') Fil e "/home/airflow/.local/lib/python3.8/site-packages/airflow/configuration.py", line 684, in getimport return import_string(full_qualified_path) File "/home/airflow/.local/lib/python3.8/site-packages/airflow/utils/module_loading.py", line 32, in import_string module = import_module(module_path) File "/usr/local/lib/python3.8/importlib/__init__.py", line 127, in import_module return _bootstrap._gcd_import(name[level:], package, level) File "/home/airflow/.local/lib/python3.8/site-packages/airflow/providers/google/cloud/secrets/secret_manager.py", line 27, in <module> from airflow.providers.google.cloud._internal_client.secret_manager_client import _SecretManagerClient File "/home/airflow/.local/lib/python3.8/site-packages/airflow/providers/google/cloud/_internal_client/secret_manager_client.py", line 23, in <module> from google.cloud.secretmanager_v1 import SecretManagerServiceClient File "/home/airflow/.local/lib/python3.8/site-packages/google/cloud/secretmanager_v1/__init__.py", lin e 22, in <module> from google.cloud.secretmanager_v1 import types File "/home/airflow/.local/lib/python3.8/site-packages/google/cloud/secretmanager_v1/types.py", line 23, in <module> from google.cloud.secretmanager_v1.proto import resources_pb2 File "/home/airflow/.local/lib/python3.8/site-packages/google/cloud/secretmanager_v1/proto/resources_pb2.py", line 57, in <module> _descriptor.EnumValueDescriptor( File "/home/airflow/.local/lib/python3.8/site-packages/google/protobuf/descriptor.py", line 755, in __new__ _message.Message._CheckCalledFromGeneratedFile() TypeError: Descriptors cannot not be created directly. Info 2022-08-24 02:33:47.645 PDTIf this call came from a _pb2.py file, your generated code is out of date and must be regenerated with protoc >= 3.19.0. Info 2022-08-24 02:33:47.645 PDTIf you cannot immediately regenerate your protos, some other possible workarounds are: Info 2022-08-24 02:33:47.645 PDT 1. Downgrade the protobuf package to 3.20.x or lower. Info 2022-08-24 02:33:47.645 PDT 2. Set PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python (but this will use pure-Python parsing and will be much slower). Info 2022-08-24 02:33:47.645 PDT{} Info 2022-08-24 02:33:47.645 PDTMore information: https://developers.google.com/protocol-buffers/docs/news/2022-05-06#python-updates Info 2022-08-24 02:33:47.645 PDT{} ``` ### What you think should happen instead Looks like the version of the package must be equal to or lower than `3.20.x`. ### How to reproduce 1. Build a custom Airflow image with the following Dockerfile. ``` FROM apache/airflow:slim-2.3.4-python3.8 RUN pip install apache-airflow[google] ``` 2. Deploy a new Airflow instance to a k8s cluster using the official Helm chart. ### Anything else _No response_ ### Are you willing to submit PR? - [ ] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
