kaxil commented on PR #49433:
URL: https://github.com/apache/airflow/pull/49433#issuecomment-2826742662
OK, I see what's going on. That `LocalExecutor` should never have been
required. We already pass `--local` in the CLI. But the bug was that we showed
a different command for K8s Rendered Template on the UI (which does not include
local).
From a test with the Helm Chart with KE and 2.10.5.
```
Containers:
base:
Container ID:
docker://5861ce7cefaf3359356060379033c496712187ef5dd84cb8644ca4766fe1ed24
Image: apache/airflow:2.10.5
Image ID:
docker-pullable://apache/airflow@sha256:6499a680a93463846d3a6be980e85d601dc97b0d81e82eed9ef5e5cb9da31b79
Port: <none>
Host Port: <none>
Args:
airflow
tasks
run
example_bash_operator
runme_0
manual__2025-04-24T07:57:01+00:00
--local
--subdir
/home/airflow/.local/lib/python3.12/site-packages/airflow/example_dags/example_bash_operator.py
```
But the UI shows
<img width="1061" alt="image"
src="https://github.com/user-attachments/assets/004eecae-afeb-4e9c-b141-7114b2bc5d36";
/>
Tested it with ` helm install $RELEASE_NAME apache-airflow/airflow
--namespace $NAMESPACE -f values.yaml`
`values.yaml`:
```yaml
executor: KubernetesExecutor
env:
- name: AIRFLOW__CORE__LOAD_EXAMPLES
value: "True"
- name: AIRFLOW__CORE__EXECUTOR
value: "KubernetesExecutor"
podTemplate: |-
apiVersion: v1
kind: Pod
metadata:
name: placeholder-name
labels:
tier: airflow
component: worker
release: example-release
annotations:
cluster-autoscaler.kubernetes.io/safe-to-evict: "false"
spec:
initContainers:
containers:
- envFrom:
[]
env:
- name: AIRFLOW__CORE__FERNET_KEY
valueFrom:
secretKeyRef:
name: example-release-fernet-key
key: fernet-key
- name: AIRFLOW_HOME
value: /opt/airflow
# For Airflow <2.3, backward compatibility; moved to [database] in
2.3
- name: AIRFLOW__CORE__SQL_ALCHEMY_CONN
valueFrom:
secretKeyRef:
name: example-release-metadata
key: connection
- name: AIRFLOW__DATABASE__SQL_ALCHEMY_CONN
valueFrom:
secretKeyRef:
name: example-release-metadata
key: connection
- name: AIRFLOW_CONN_AIRFLOW_DB
valueFrom:
secretKeyRef:
name: example-release-metadata
key: connection
- name: AIRFLOW__WEBSERVER__SECRET_KEY
valueFrom:
secretKeyRef:
name: example-release-webserver-secret-key
key: webserver-secret-key
image: apache/airflow:2.10.5
imagePullPolicy: IfNotPresent
securityContext:
allowPrivilegeEscalation: false
capabilities:
drop:
- ALL
name: base
volumeMounts:
- mountPath: "/opt/airflow/logs"
name: logs
- name: config
mountPath: "/opt/airflow/airflow.cfg"
subPath: airflow.cfg
readOnly: true
- name: config
mountPath: "/opt/airflow/config/airflow_local_settings.py"
subPath: airflow_local_settings.py
readOnly: true
restartPolicy: Never
securityContext:
runAsUser: 50000
fsGroup: 0
terminationGracePeriodSeconds: 600
serviceAccountName: "example-release-airflow-worker"
volumes:
- emptyDir:
{}
name: logs
- configMap:
name: example-release-config
name: config
```
That `--local` is passed from the following place in the Scheduler, which
then sends it to all the executors:
https://github.com/apache/airflow/blob/2.10.5/airflow/jobs/scheduler_job_runner.py#L685C13-L688
```py
command = ti.command_as_list(
local=True,
pickle_id=ti.dag_model.pickle_id,
)
```
https://github.com/apache/airflow/blob/2.10.5/airflow/executors/base_executor.py#L160C9-L172
https://github.com/apache/airflow/blob/2.10.5/airflow/executors/base_executor.py#L408
and used by KE here
https://github.com/apache/airflow/blob/2.10.5/airflow/providers/cncf/kubernetes/executors/kubernetes_executor.py#L342
https://github.com/apache/airflow/blob/2.10.5/airflow/providers/cncf/kubernetes/executors/kubernetes_executor.py#L396-L400
which then generates pod spec here:
https://github.com/apache/airflow/blob/main/providers/cncf/kubernetes/src/airflow/providers/cncf/kubernetes/executors/kubernetes_executor_utils.py#L429
But the UI code is generated directly using
`args=task_instance.command_as_list()` instead of
`args=task_instance.command_as_list(local=True)` (like the scheduler), hence
the mis-match
https://github.com/apache/airflow/blob/1b27c3b5d5d0b20508e5bc174797613aa85064af/providers/cncf/kubernetes/src/airflow/providers/cncf/kubernetes/template_rendering.py#L52
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
