GitHub user iscipar added a comment to the discussion: TypeError when importing
operators from airflow.providers.google.cloud.operators.dataproc
Hello.
I would like to comment that I have updated the sources corresponding to the PR
in my virtual environment to check if the error was resolved and this was the
result:
Although the error has disappeared in the dag that imported the dataproc
operators, the dags that imported the BigQueryInsertJobOperator have now
stopped working, producing the following new exception.
Can you check if, as it seems, the resolution of the error has affected the
import of this other operator?
[2025-02-08T19:30:17.251+0100] {logging_mixin.py:190} INFO -
[2025-02-08T19:30:17.249+0100] {dagbag.py:387} ERROR - Failed to import:
/home/iscipar/Projects/airflow-tutorial/dags/2_simple_storage_bigquery.py
Traceback (most recent call last):
File
"/home/iscipar/Proyectos/airflow-tutorial/airflow_env/lib/python3.12/site-packages/airflow/models/dagbag.py",
line 383, in parse
loader.exec_module(new_module)
File "<frozen importlib._bootstrap_external>", line 995, in exec_module
File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
File
"/home/iscipar/Proyectos/airflow-tutorial/dags/2_simple_storage_bigquery.py",
line 2, in <module>
from airflow.providers.google.cloud.operators.bigquery import
BigQueryInsertJobOperator
File
"/home/iscipar/Proyectos/airflow-tutorial/airflow_env/lib/python3.12/site-packages/airflow/providers/google/cloud/operators/bigquery.py",
line 47, in <module>
from airflow.providers.google.cloud.openlineage.mixins import
_BigQueryOpenLineageMixin
File
"/home/iscipar/Proyectos/airflow-tutorial/airflow_env/lib/python3.12/site-packages/airflow/providers/google/cloud/openlineage/mixins.py",
line 39, in <module>
from airflow.providers.google.cloud.openlineage.utils import (
File
"/home/iscipar/Proyectos/airflow-tutorial/airflow_env/lib/python3.12/site-packages/airflow/providers/google/cloud/openlineage/utils.py",
line 40, in <module>
from airflow.providers.common.compat.openlineage.utils.spark import (
ImportError: cannot import name
'inject_transport_information_into_spark_properties' from
'airflow.providers.common.compat.openlineage.utils.spark'
(/home/iscipar/Proyectos/airflow-tutorial/airflow_env/lib/python3.12/site-packages/airflow/providers/common/compat/openlineage/utils/spark.py)
GitHub link:
https://github.com/apache/airflow/discussions/46478#discussioncomment-12104714
----
This is an automatically sent email for [email protected].
To unsubscribe, please send an email to: [email protected]
