dgouju commented on PR #28230:
URL: https://github.com/apache/airflow/pull/28230#issuecomment-1353242270
Hello, I'm trying to test this code on my Airflow 2.3.4 environment (GCP
Cloud Composer).
In google.cloud.utils.kubernetes_engine_config.py,
`write_permanent_gke_config_file` wants to write into /files/ , which doesn't
exist in my environment:
`[2022-12-15, 14:18:35 UTC] {taskinstance.py:1904} ERROR - Task failed with
exception
Traceback (most recent call last):
File
"/opt/python3.8/lib/python3.8/site-packages/adasdgouju/google/cloud/operators/kubernetes_engine.py",
line 358, in execute
config_file = write_permanent_gke_config_file(**config_args) # type:
ignore[arg-type]
File
"/opt/python3.8/lib/python3.8/site-packages/adasdgouju/google/cloud/utils/kubernetes_engine_config.py",
line 100, in write_permanent_gke_config_file
with open(filename, "w") as conf_file, patch_environ(
FileNotFoundError: [Errno 2] No such file or directory:
'/files/kube_config_948b954c'`
I tried to use /home/airflow/gcs/data/ instead of /files/ . Pods are
scheduled but almost all immediately terminated, some continue to run (as
expected) while Airflow fails all tasks:
`[2022-12-15, 14:51:12 UTC] {taskinstance.py:1904} ERROR - Task failed with
exception
Traceback (most recent call last):
File
"/opt/python3.8/lib/python3.8/site-packages/adasdgouju/google/cloud/operators/kubernetes_engine.py",
line 372, in execute_complete
return super().execute_complete(context, event)
File
"/opt/python3.8/lib/python3.8/site-packages/adasdgouju/cncf/kubernetes/operators/kubernetes_pod.py",
line 575, in execute_complete
self.cleanup(
File
"/opt/python3.8/lib/python3.8/site-packages/adasdgouju/cncf/kubernetes/operators/kubernetes_pod.py",
line 644, in cleanup
raise AirflowException(
airflow.exceptions.AirflowException: Pod run-zf-pod-whesa52w returned a
failure:
remote_pod: {'api_version': 'v1',
...`
What is your guidance to fix this?
Thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
