GitHub user niraj8241 created a discussion: Dag Processor Liveness Probe Keeps
Failing Intermittently
# Issue with Dag Processor
## Environment Details:
Cluster: AWS EKS Running on 1.35
Component: Dag Processor
Number of Dags: 900+
Airflow Version: *3.1.8*
Issue Observed: `No alive jobs found.`
I have recently deployed Airflox 3x. This is running on AWS EKS. The dag
process is currently running with 2 replicas with below settings:
```
dagProcessor:
enabled: true
replicas: ${AIRFLOW_DAG_PROCESSOR_REPLICAS:=2}
podAnnotations:
ad.datadoghq.com/dag-processor.logs: |
[
{
"service":"airflow-dag-processor",
"source":"airflow",
"env":"${AIRFLOW_ENVIRONMENT:=dev}",
}
]
resources:
requests:
cpu: ${AIRFLOW_DAG_PROCESSOR_CPU_REQUEST:=2}
memory: ${AIRFLOW_DAG_PROCESSOR_MEMORY_REQUEST:=2000Mi}
limits:
cpu: ${AIRFLOW_DAG_PROCESSOR_CPU_LIMIT:=3}
memory: ${AIRFLOW_DAG_PROCESSOR_MEMORY_LIMIT:=4000Mi}
livenessProbe:
timeoutSeconds: 60
failureThreshold: 10
periodSeconds: 60
env:
- name: AIRFLOW__DAG_PROCESSOR__DAG_FILE_PROCESSOR_TIMEOUT
value: "600"
- name: AIRFLOW__DAG_PROCESSOR__PARSING_PROCESSES
value: "4"
- name: AIRFLOW__CORE__DAGBAG_IMPORT_TIMEOUT
value: "180"
- name: AIRFLOW__DAG_PROCESSOR__FILE_PARSING_SORT_MODE
value: "random_seeded_by_host"
- name: AIRFLOW__DAG_PROCESSOR__MIN_FILE_PROCESS_INTERVAL
value: "900"
- name: AIRFLOW__DAG_PROCESSOR__STALE_DAG_THRESHOLD
value: "600"
```
Every now and then the Dag Processor just keeping passing and failing causing
the new dags and update to existing dags be not updated for several hours.
-- Snippet of Restarts --
<img width="1414" height="85" alt="image"
src="https://github.com/user-attachments/assets/70f99381-f3cb-4c8d-aa68-aa756666663e";
/>
Observation:
I do see that the dag processor is able to see the files in my dags folder but
then it does not processor them. During dags update when I go inside the
container and see the files, the files are updated files but when the jobs run
on Airflow, it refers to the older version of the file which earlier had
issues. So I have a feeling that the code is never registered or acknowledged
by Airflow.
```
dags-folder classic/processor.py
4627 252.88s
0 0
================================================================================
[airflow.dag_processing.manager.DagFileProcessorManager] dd.env=
dd.service=airflow-dag-processor dd.span_id=0 dd.trace_id=0 dd.version=
loc=manager.py:815
2026-06-05T17:41:28.823070Z [info ] **Heartbeat recovered after 240.17
seconds** [airflow.jobs.job.Job] dd.env= dd.service=airflow-dag-processor
dd.span_id=0 dd.trace_id=0 dd.version= loc=job.py:253
2026-06-05T17:41:28.833219Z [info ] Not time to refresh bundle dags-folder
[airflow.dag_processing.manager.DagFileProcessorManager] dd.env=
dd.service=airflow-dag-processor dd.span_id=0 dd.trace_id=0 dd.version=
loc=manager.py:585
2026-06-05T17:41:30.248570Z [info ] DAG bundles loaded: dags-folder
[airflow.dag_processing.bundles.manager.DagBundlesManager] dd.env=
dd.service=airflow-dag-processor dd.span_id=0 dd.trace_id=0 dd.version=
loc=manager.py:179
2026-06-05T17:41:30.277134Z [info ] DAG bundles loaded: dags-folder
[airflow.dag_processing.bundles.manager.DagBundlesManager] dd.env=
dd.service=airflow-dag-processor dd.span_id=0 dd.trace_id=0 dd.version=
loc=manager.py:179
2026-06-05T17:41:33.281455Z [info ] DAG bundles loaded: dags-folder
[airflow.dag_processing.bundles.manager.DagBundlesManager] dd.env=
dd.service=airflow-dag-processor dd.span_id=0 dd.trace_id=0 dd.version=
loc=manager.py:179
2026-06-05T17:41:33.297554Z [info ] DAG bundles loaded: dags-folder
[airflow.dag_processing.bundles.manager.DagBundlesManager] dd.env=
dd.service=airflow-dag-processor dd.span_id=0 dd.trace_id=0 dd.version=
loc=manager.py:179
2026-06-05T17:41:34.403258Z [info ] Not time to refresh bundle dags-folder
[airflow.dag_processing.manager.DagFileProcessorManager] dd.env=
dd.service=airflow-dag-processor dd.span_id=0 dd.trace_id=0 dd.version=
loc=manager.py:585
2026-06-05T17:41:34.685971Z [info ] DAG bundles loaded: dags-folder
[airflow.dag_processing.bundles.manager.DagBundlesManager] dd.env=
dd.service=airflow-dag-processor dd.span_id=0 dd.trace_id=0 dd.version=
loc=manager.py:179
2026-06-05T17:41:36.232343Z [info ] DAG bundles loaded: dags-folder
[airflow.dag_p
```
Any help on fixing this would be greatly appreciated. If I am missing any
particular piece of information, please let me know.
GitHub link: https://github.com/apache/airflow/discussions/68098
----
This is an automatically sent email for [email protected].
To unsubscribe, please send an email to: [email protected]
