karthik-raparthi commented on issue #10790:
URL: https://github.com/apache/airflow/issues/10790#issuecomment-1254506118
Hello,
We are facing a similar issue but it looks likes for me a combination of
EFS(Throughput-Provisioned(25MiB/s) + Worker not sending back an exception to
Scheduler that unable to read the dag file and **task struck in queue state
forever.**
**Scheduler Log**
`[^[[34m2022-09-16 22:38:31,848^[[0m] {^[[34mscheduler_job.py:^[[0m572}
ERROR^[[0m - Executor reports task instance <TaskInstance: dagName.TaskName
scheduled__2022-09-16T21:30:00+00:00 [queued]> finished (failed) although the
task says its queued. (Info: None) Was the task killed externally?^[[0m`
**Worker Log**
`File
"/mnt/airflow/airflow2/lib64/python3.8/site-packages/slack_sdk/__init__.py",
line 44, in <module>\n from .web import WebClient # noqa\n File
"/mnt/airflow/airflow2/lib64/python3.8/site-packages/slack_sdk/web/__init__.py",
line 3, in <module>\n from .client import WebClient # noqa\n File
"/mnt/airflow/airflow2/lib64/python3.8/site-packages/slack_sdk/web/client.py",
line 8, in <module>\n from slack_sdk.models.views import View\n File
"<frozen importlib._bootstrap>", line 991, in _find_and_load\n File "<frozen
importlib._bootstrap>", line 971, in _find_and_load_unlocked\n File "<frozen
importlib._bootstrap>", line 914, in _find_spec\n File "<frozen
importlib._bootstrap_external>", line 1342, in find_spec\n File "<frozen
importlib._bootstrap_external>", line 1314, in _get_spec\n File "<frozen
importlib._bootstrap_external>", line 1458, in find_spec\n File "<frozen
importlib._bootstrap_external>", line 101, in _path_isfile\n File "<frozen
importlib._bootstrap
_external>", line 93, in _path_is_mode_type\n File "<frozen
importlib._bootstrap_external>", line 87, in _path_stat\n File
"/mnt/airflow/airflow2/lib64/python3.8/site-packages/airflow/utils/timeout.py",
line 37, in handle_timeout\n raise
AirflowTaskTimeout(self.error_message)\nairflow.exceptions.AirflowTaskTimeout:
DagBag import timeout for /mnt/airflow/config2/dags/DagName.py after
30.0s.\nPlease take a look at these docs to improve your DAG import time:\n*
https://airflow.apache.org/docs/apache-airflow/2.2.2/best-practices.html#top-level-python-code\n*
https://airflow.apache.org/docs/apache-airflow/2.2.2/best-practices.html#reducing-dag-complexity,
PID: 13809\nTraceback (most recent call last):\n File
"/mnt/airflow/airflow2/bin/airflow", line 8, in <module>\n
sys.exit(main())\n File
"/mnt/airflow/airflow2/lib64/python3.8/site-packages/airflow/__main__.py", line
48, in main\n args.func(args)\n File
"/mnt/airflow/airflow2/lib64/python3.8/site-packages/airflow/cli/cli_p
arser.py", line 48, in command\n return func(*args, **kwargs)\n File
"/mnt/airflow/airflow2/lib64/python3.8/site-packages/airflow/utils/cli.py",
line 92, in wrapper\n return f(*args, **kwargs)\n File
"/mnt/airflow/airflow2/lib64/python3.8/site-packages/airflow/cli/commands/task_command.py",
line 276, in task_run\n dag = get_dag(args.subdir, args.dag_id)\n File
"/mnt/airflow/airflow2/lib64/python3.8/site-packages/airflow/utils/cli.py",
line 192, in get_dag\n raise
AirflowException(\nairflow.exceptions.AirflowException: Dag \'DagName\' could
not be found; either it does not exist or it failed to parse.\n'
[2022-09-16 22:38:12,536: ERROR/ForkPoolWorker-16] execute_command
encountered a CalledProcessError`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
