shreyaskj-0710 commented on issue #53712:
URL: https://github.com/apache/airflow/issues/53712#issuecomment-3182812229
Hi @pedro-cf ,
I tried to reproduce the issue, but it doesn’t seem to occur.
As suggested by you :
1] Use dynamic DAG generation (such as dynamically created DAGs from a DB
table).
-> Used the below code for that
# dags/dynamic_from_db_dags.py
from airflow import DAG
from airflow.operators.python import PythonOperator
from airflow.utils.db import provide_session
from airflow.models import DagModel
from datetime import datetime
import time
def long_running_task(dag_id, **kwargs):
print(f"Starting task for {dag_id} at {datetime.now()}")
time.sleep(3 * 60) # sleep for 3 minutes
print(f"Finished task for {dag_id} at {datetime.now()}")
# Function to fetch DAGs from DagModel table
@provide_session
def fetch_dags(session=None):
# Fetch all DAGs from the DagModel table
dag_models = session.query(DagModel).all()
dag_configs = []
for dm in dag_models:
dag_configs.append({
"dag_name": dm.dag_id,
"schedule": "*/10 * * * *"
})
return dag_configs
# Fetch DAG configurations dynamically
dag_configs = fetch_dags()
# Generate DAGs dynamically
for config in dag_configs:
dag_id = config["dag_name"]
schedule = config["schedule"]
dag = DAG(
dag_id=dag_id,
schedule_interval=schedule,
start_date=datetime(2025, 8, 13),
catchup=False,
)
with dag:
PythonOperator(
task_id="long_task",
python_callable=long_running_task,
op_kwargs={"dag_id": dag_id},
)
globals()[dag_id] = dag
2] Trigger DAG runs so that they enter the running state.
-> have triggered them
In the airflow UI page have selected 'Running' filter and it displayed all
dags correctly.
Please find the screenshot below :
<img width="1853" height="968" alt="Image"
src="https://github.com/user-attachments/assets/5f73b3fc-0eb7-4b26-b1d1-621d8ce8ecef";
/>
Please let me know if there’s anything else I can try to reproduce the issue
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
