Usuychik commented on issue #59602: URL: https://github.com/apache/airflow/issues/59602#issuecomment-3759828027
> > or try to upload all logs before shutdown > > This would be ideal case. However, it’s possible the pod was terminated before the worker process finished uploading the log. > > [airflow/task-sdk/src/airflow/sdk/execution_time/supervisor.py](https://github.com/apache/airflow/blob/7bb3f203c24fca914cd296af47d6f00fa735feba/task-sdk/src/airflow/sdk/execution_time/supervisor.py#L1006-L1010) > > Lines 1006 to 1010 in [7bb3f20](/apache/airflow/commit/7bb3f203c24fca914cd296af47d6f00fa735feba) > # Now at the last possible moment, when all logs and comms with the subprocess has finished, lets > # upload the remote logs > self._upload_logs() > > return self._exit_code But its about task itself. And I am speaking about CelertyExecutor shutdown feature. In general as for me Airflow miss some lightweight Celery Worker for k8s -- which will store all task task info in Redis so it will be able to get info on restart or to be captured by other executor replicas, in case if existing one executor removed. Airflow K8S executor in general require image to be based on Airflow image. For our use case we are using CeleryWorker executor to run DAGs which creates custom pods with something like: ` kubernetes_task = KubernetesPodOperator( task_id=k8s_task_id, image=IMAGE, cmds=["python3", "-u", "xxxx.py"], arguments=args, namespace="airflow", in_cluster=True, image_pull_secrets=[k8s.V1LocalObjectReference("xxxx")], env_vars=base_env, configmaps=["xxxx"], container_resources=k8s.V1ResourceRequirements( requests={"memory": "1Gi", "cpu": "0.5"} ), image_pull_policy="Always", deferrable=False, get_logs=True, dag=mydag, logging_interval=5, ) ` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
