[
https://issues.apache.org/jira/browse/AIRFLOW-1641?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16213027#comment-16213027
]
Nick Bauer commented on AIRFLOW-1641:
-------------------------------------
yes, airflow.cfg:
dagbag_import_timeout = 30
30 is the default.
or via environment variables:
export AIRFLOW__CORE_DAGBAG_IMPORT_TIMEOUT=30
Take a look at the log for your queued task that is stuck. It did not occur to
me that a queued task would even have a log, but once I looked at it, it was
easy to identify if it the issue was due to a timeout.
Even so, the task should fail instead of stay "queued", or retry depending on
your settings, possibly email if that is setup.
> Task gets stuck in queued state
> -------------------------------
>
> Key: AIRFLOW-1641
> URL: https://issues.apache.org/jira/browse/AIRFLOW-1641
> Project: Apache Airflow
> Issue Type: Bug
> Components: scheduler
> Affects Versions: 1.8.0
> Environment: Linux
> Reporter: Mas
> Labels: queued, scheduler, stuck, task
>
> Hello,
> I have one dag with ~20 tasks.
> The dags runs daily and some tasks can sometime last for hours, depending on
> the processed data behind.
> There are some interactions with AWS and a remote DB.
> I only use LocalExecutor.
> What this issue is about, is the fact that sometime (randomly, and without
> any clear reason) one of the tasks (here also, it is random) gets stuck in
> "queued" state and never starts running.
> The manual workaround is to restart the task manually by clearing it.
> Does anyone have ideas about the issue behind, and how to avoid it for the
> future?
> Thanks in advance for your help.
> PS: other people are facing the same behaviour:
> [link|https://stackoverflow.com/questions/45853013/airflow-tasks-get-stuck-at-queued-status-and-never-gets-running]
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
