dhkim1920 commented on issue #71175: URL: https://github.com/apache/airflow/issues/71175#issuecomment-5854338348
A potentially related issue is #56271 (https://github.com/apache/airflow/issues/56271). In that case, the worker Pod had AIRFLOW__CORE__EXECUTOR set to LocalExecutor, so tasks explicitly specifying executor="KubernetesExecutor" failed executor validation when the worker parsed the Dag again. Adding KubernetesExecutor to the executor configuration in the Pod template was reported to resolve the failure. There is also a related proposed fix in #73172 (https://github.com/apache/airflow/pull/73172). However, we have not confirmed whether this issue has the same underlying cause. Could you share the following details to help investigate and reproduce it? - A minimal reproduction Dag: A small example containing a task with executor="KubernetesExecutor". - The full stack trace and log source: Does the error occur in the Dag processor, or inside the worker Pod after it starts? - The effective executor configuration in each component: The output of airflow config get-value core executor and the value of AIRFLOW__CORE__EXECUTOR, if set, in the scheduler, Dag processor, and worker Pod. - Pod template configuration: Whether you use a custom podTemplate or pod_template_file, including the relevant executor environment variable settings. - The worker image and package versions: Whether the worker Pod uses the same Airflow and apache-airflow-providers-cncf-kubernetes versions as the scheduler. Even if the ConfigMap is correct, environment variables in the Pod can override it, so checking the effective configuration inside the failing container would help narrow this down. Also, the correct configuration key is [core] executor (singular). The misleading [core] executors wording in the error message was corrected in #71336 (https://github.com/apache/airflow/pull/71336), but that change only fixes the message, not the execution failure. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
