Ferdinanddb commented on issue #57416: URL: https://github.com/apache/airflow/issues/57416#issuecomment-3456344811
Sure, here is a snippet, and some remarks: - If the bug happens again -> I will provide some screenshots and copy-paste the logs that I see. - I strongly think that the behavior that I am trying to explain is similar to the one I described [here](https://github.com/apache/airflow/issues/56291), but I am not using the deferrable mode, just to be clear. ```python import os from datetime import datetime, timedelta from pathlib import Path from airflow.providers.cncf.kubernetes.operators.spark_kubernetes import SparkKubernetesOperator from airflow.sdk import DAG, Asset default_args = { "owner": "airflow", "depends_on_past": False, "email_on_failure": False, "email_on_retry": False, "retries": 1, "start_date": datetime(2016, 1, 4), "retry_delay": timedelta(minutes=1), "weight_rule": "upstream", } with DAG( dag_id=Path(__file__).parent.name, schedule=None, default_args=default_args, max_active_runs=1, max_active_tasks=50, catchup=False, on_failure_callback=default_dag_failure_slack_webhook_notification, ) as dag: full_reload_job = SparkKubernetesOperator( task_id="full_reload_job", namespace="spark-operator", application_file="spark_app/full_reload_job/spark_application_config.yml", kubernetes_conn_id="kubernetes_default", random_name_suffix=True, get_logs=True, reattach_on_restart=True, delete_on_termination=True, do_xcom_push=False, deferrable=False, retries=2, ) ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
