amoghrajesh opened a new pull request, #59587:
URL: https://github.com/apache/airflow/pull/59587
<!--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-->
<!--
Thank you for contributing! Please make sure that your code changes
are covered with tests. And in case of new features or big changes
remember to adjust the documentation.
Feel free to ping committers for the review!
In case of an existing issue, reference it using one of the following:
closes: #ISSUE
related: #ISSUE
How to write a good git commit message:
http://chris.beams.io/posts/git-commit/
-->
This is an attempt to fix CI failures here:
https://github.com/apache/airflow/actions/runs/20317942783/job/58367633157
Error:
```python
____________________________________________
test_should_be_importable[providers/google/tests/system/google/cloud/gen_ai/example_gen_ai_generative_model.py]
_____________________________________________
airflow-core/tests/unit/always/test_example_dags.py:177: in
test_should_be_importable
assert len(dagbag.import_errors) == 0,
f"import_errors={str(dagbag.import_errors)}"
E AssertionError:
import_errors={'/opt/airflow/providers/google/tests/system/google/cloud/gen_ai/example_gen_ai_generative_model.py':
'Traceback (most recent call last):\n File
"/usr/python/lib/python3.12/site-packages/importlib_metadata/__init__.py", line
765, in children\n return os.listdir(self.root or \'.\')\n
^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File
"/opt/airflow/airflow-core/src/airflow/dag_processing/dagbag.py", line 122, in
handle_timeout\n raise
AirflowTaskTimeout(error_message)\nairflow.sdk.exceptions.AirflowTaskTimeout:
DagBag import timeout for
/opt/airflow/providers/google/tests/system/google/cloud/gen_ai/example_gen_ai_generative_model.py
after 30.0s.\nPlease take a look at these docs to improve your DAG import
time:\n*
https://airflow.apache.org/docs/apache-airflow/3.2.0/best-practices.html#top-level-python-code\n*
https://airflow.apache.org/docs/apache-airflow/3.2.0/best-practices.html#reducing-dag-complexity,
PID: 81\n'}
E assert 1 == 0
E + where 1 =
len({'/opt/airflow/providers/google/tests/system/google/cloud/gen_ai/example_gen_ai_generative_model.py':
'Traceback (most...n-code\n*
https://airflow.apache.org/docs/apache-airflow/3.2.0/best-practices.html#reducing-dag-complexity,
PID: 81\n'})
E + where
{'/opt/airflow/providers/google/tests/system/google/cloud/gen_ai/example_gen_ai_generative_model.py':
'Traceback (most...n-code\n*
https://airflow.apache.org/docs/apache-airflow/3.2.0/best-practices.html#reducing-dag-complexity,
PID: 81\n'} = <airflow.dag_processing.dagbag.DagBag object at
0x7fb924ba51f0>.import_errors
------------------------------------------------------------------------------------------
Captured stdout call
------------------------------------------------------------------------------------------
2025-12-17T22:01:30.694081Z [info ] Filling up the DagBag from
/opt/airflow/providers/google/tests/system/google/cloud/gen_ai/example_gen_ai_generative_model.py
[airflow.dag_processing.dagbag.DagBag]
2025-12-17T22:02:00.695495Z [error ] Failed to import:
/opt/airflow/providers/google/tests/system/google/cloud/gen_ai/example_gen_ai_generative_model.py
[airflow.dag_processing.dagbag.DagBag]
Traceback (most recent call last):
File "/opt/airflow/airflow-core/src/airflow/dag_processing/dagbag.py",
line 438, in parse
loader.exec_module(new_module)
File "<frozen importlib._bootstrap_external>", line 999, in exec_module
File "<frozen importlib._bootstrap>", line 488, in
_call_with_frames_removed
File
"/opt/airflow/providers/google/tests/system/google/cloud/gen_ai/example_gen_ai_generative_model.py",
line 29, in <module>
from vertexai.preview.evaluation import MetricPromptTemplateExamples
File "/usr/python/lib/python3.12/site-packages/vertexai/__init__.py", line
20, in <module>
from google.cloud.aiplatform import version as aiplatform_version
File
"/usr/python/lib/python3.12/site-packages/google/cloud/aiplatform/__init__.py",
line 34, in <module>
from google.cloud.aiplatform import gapic
File
"/usr/python/lib/python3.12/site-packages/google/cloud/aiplatform/gapic/__init__.py",
line 20, in <module>
from google.cloud.aiplatform.gapic import schema
File
"/usr/python/lib/python3.12/site-packages/google/cloud/aiplatform/gapic/schema/__init__.py",
line 20, in <module>
from google.cloud.aiplatform.v1beta1.schema import predict as
predict_v1beta1
File
"/usr/python/lib/python3.12/site-packages/google/cloud/aiplatform/v1beta1/schema/__init__.py",
line 18, in <module>
from google.cloud.aiplatform.v1beta1.schema import predict
File
"/usr/python/lib/python3.12/site-packages/google/cloud/aiplatform/v1beta1/schema/predict/__init__.py",
line 18, in <module>
from google.cloud.aiplatform.v1beta1.schema.predict import instance
File
"/usr/python/lib/python3.12/site-packages/google/cloud/aiplatform/v1beta1/schema/predict/instance/__init__.py",
line 23, in <module>
from
google.cloud.aiplatform.v1beta1.schema.predict.instance_v1beta1.types.image_classification
import (
File
"/usr/python/lib/python3.12/site-packages/google/cloud/aiplatform/v1beta1/schema/predict/instance_v1beta1/__init__.py",
line 47, in <module>
api_core.check_dependency_versions("google.cloud.aiplatform.v1beta1.schema.predict.instance_v1beta1")
# type: ignore
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File
"/usr/python/lib/python3.12/site-packages/google/api_core/_python_package_support.py",
line 229, in check_dependency_versions
warn_deprecation_for_versions_less_than(
File
"/usr/python/lib/python3.12/site-packages/google/api_core/_python_package_support.py",
line 157, in warn_deprecation_for_versions_less_than
dependency_version = get_dependency_version(dependency_import_package)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File
"/usr/python/lib/python3.12/site-packages/google/api_core/_python_package_support.py",
line 102, in get_dependency_version
version_string: str = metadata.version(dependency_name)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/python/lib/python3.12/importlib/metadata/__init__.py", line
889, in version
return distribution(distribution_name).version
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/python/lib/python3.12/importlib/metadata/__init__.py", line
862, in distribution
return Distribution.from_name(distribution_name)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/python/lib/python3.12/importlib/metadata/__init__.py", line
397, in from_name
return next(cls.discover(name=name))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File
"/usr/python/lib/python3.12/site-packages/importlib_metadata/__init__.py", line
919, in <genexpr>
path.search(prepared) for path in map(FastPath, paths)
^^^^^^^^^^^^^^^^^^^^^
File
"/usr/python/lib/python3.12/site-packages/importlib_metadata/__init__.py", line
778, in search
return self.lookup(self.mtime).search(name)
^^^^^^^^^^^^^^^^^^^^^^^
File
"/usr/python/lib/python3.12/site-packages/importlib_metadata/_functools.py",
line 80, in wrapper
return cached_method(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File
"/usr/python/lib/python3.12/site-packages/importlib_metadata/__init__.py", line
788, in lookup
return Lookup(self)
^^^^^^^^^^^^
File
"/usr/python/lib/python3.12/site-packages/importlib_metadata/__init__.py", line
810, in __init__
for child in path.children():
^^^^^^^^^^^^^^^
File
"/usr/python/lib/python3.12/site-packages/importlib_metadata/__init__.py", line
765, in children
return os.listdir(self.root or '.')
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/airflow/airflow-core/src/airflow/dag_processing/dagbag.py",
line 122, in handle_timeout
raise AirflowTaskTimeout(error_message)
airflow.sdk.exceptions.AirflowTaskTimeout: DagBag import timeout for
/opt/airflow/providers/google/tests/system/google/cloud/gen_ai/example_gen_ai_generative_model.py
after 30.0s.
Please take a look at these docs to improve your DAG import time:
*
https://airflow.apache.org/docs/apache-airflow/3.2.0/best-practices.html#top-level-python-code
*
https://airflow.apache.org/docs/apache-airflow/3.2.0/best-practices.html#reducing-dag-complexity,
PID: 81
------------------------------------------------------------------------------------------
Captured stderr call
------------------------------------------------------------------------------------------
2025-12-17T22:02:00.694977Z [error ] Process timed out, PID: 81
[airflow.dag_processing.dagbag]
-------------------------------------------------------------------------------------------
Captured log call
--------------------------------------------------------------------------------------------
ERROR airflow.dag_processing.dagbag:dagbag.py:119 Process timed out, PID:
81
```
The errors seems to be that the DAG was timing out during parsing with newer
google cloud versions (`google-cloud-aiplatform==1.132.0`). The timeout
seemingly occurs because:
- The DAG imports `from vertexai.preview.evaluation import
MetricPromptTemplateExamples` at module level
- This triggers a deep import chain through `google.cloud.aiplatform`
- During import, `api_core.check_dependency_versions()` calls
`importlib.metadata.version()`
- This performs discover package metadata
I attempt to fix this by moving those imports lazy which will defer the
expensive import until the operator is actually created, avoiding the timeout
during initial DAG file parsing.
<!-- Please keep an empty line above the dashes. -->
---
**^ Add meaningful description above**
Read the **[Pull Request
Guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#pull-request-guidelines)**
for more information.
In case of fundamental code changes, an Airflow Improvement Proposal
([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvement+Proposals))
is needed.
In case of a new dependency, check compliance with the [ASF 3rd Party
License Policy](https://www.apache.org/legal/resolved.html#category-x).
In case of backwards incompatible changes please leave a note in a
newsfragment file, named `{pr_number}.significant.rst` or
`{issue_number}.significant.rst`, in
[airflow-core/newsfragments](https://github.com/apache/airflow/tree/main/airflow-core/newsfragments).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
