noelmcloughlin edited a comment on issue #16025:
URL: https://github.com/apache/airflow/issues/16025#issuecomment-894139568
How can I fix this? I'm using Airflow 2.1.2 on CentOS7 in a federated
topology and all jobs failing.
**Root cause is fact TASK args includes wrong `--subdir` valuer, passing
value of Scheduler's ${DAGS_HOME} instead of Workers ${DAG_HOME}.** Scheduler
and Worker are different host/domains so ${DAG_HOME} cannot be identical. In
`dagbag.py` using `settings.DAGS_FOLDER` would fix the problem I think:
`airflow/models/dagbag.py:122 dag_folder = dag_folder or
settings.DAGS_FOLDER`
**Error**
```
Aug 05 18:49:16 myfqdn airflow[4905]: [2021-08-05 18:49:16,782:
INFO/ForkPoolWorker-8] Executing co
mmand in Celery: ['airflow', 'tasks', 'run', 'touch_file_mytest', 'runme',
'2021-08-05T18:49:15.368167+00:00', '--local', '--poo
l', 'default_pool', '--subdir',
'/home/_airflowservice@mydomain/dags/mytest/example.py']
Aug 05 18:49:16 myfqdn airflow[4905]: [2021-08-05 18:49:16,898:
DEBUG/ForkPoolWorker-8] Calling callbacks: [<function default_action_log at
0x7f7cabef4c80>]
Aug 05 18:49:17 myfqdn airflow[4905]: [2021-08-05 18:49:17,143:
DEBUG/ForkPoolWorker-8] Setting up DB connection pool (PID 5128)
Aug 05 18:49:17 myfqdn airflow[4905]: [2021-08-05 18:49:17,146:
DEBUG/ForkPoolWorker-8] settings.prepare_engine_args(): Using NullPool
Aug 05 18:49:17 myfqdn airflow[4905]: [2021-08-05 18:49:17,148:
INFO/ForkPoolWorker-8] Filling up the DagBag from
/home/_airflowservice@mydomain/dags/mytest/example.py
Aug 05 18:49:17 myfqdn airflow[4905]: [2021-08-05 18:49:17,148:
DEBUG/ForkPoolWorker-8] Calling callbacks: []
Aug 05 18:49:17 myfqdn airflow[4905]: [2021-08-05 18:49:17,148:
ERROR/ForkPoolWorker-8] Failed to execute task dag_id could not be found:
touch_file_mytest. Either the dag did not exist or it failed to parse..
```
**CAUSE: Wrong `--subdir` value**
$ airflow tasks test touch_file_mytest runme
2021-08-06T09:03:16.244034+00:00 --subdir
/home/_airflowservice@**schedulerdomain**/dags/mytest/example.py
```
[2021-08-06 09:16:45,824] {dagbag.py:501} INFO - Filling up the DagBag from
/home/_airflowservice@schedulerdomain/dags/mytest/example.py
Traceback (most recent call last):
File "/home/_airflowservice@localdomain/.local/bin/airflow", line 8, in
<module> sys.exit(main())
File
"/home/_airflowservice@localdomain/.local/lib/python3.6/site-packages/airflow/__main__.py",
line 40, in main args.func(args)
File
"/home/_airflowservice@localdomain/.local/lib/python3.6/site-packages/airflow/cli/cli_parser.py",
line 48, in command return func(*args, **kwargs)
File
"/home/_airflowservice@localdomain/.local/lib/python3.6/site-packages/airflow/utils/cli.py",
line 91, in wrapper return f(*args, **kwargs)
File
"/home/_airflowservice@localdomain/.local/lib/python3.6/site-packages/airflow/cli/commands/task_command.py",
line 380, in task_test
dag = dag or get_dag(args.subdir, args.dag_id)
File
"/home/_airflowservice@localdomain/.local/lib/python3.6/site-packages/airflow/utils/cli.py",
line 193, in get_dag
'parse.'.format(dag_id) airflow.exceptions.AirflowException: dag_id
could not be found: touch_file_mytest. Either the dag did not exist or it
failed to parse.
```
**FIX: DROP INCORRECT `--subdir` arg **
$ airflow tasks test touch_file_mytest runme 2021-08-06T09:03:16.244034+00:00
```
[2021-08-06 09:15:34,234] {dagbag.py:501} INFO - Filling up the DagBag from
/home/_airflowservice@localdomain/dags
[2021-08-06 09:15:34,434] {taskinstance.py:896} INFO - Dependencies all met
for <TaskInstance: touch_file_mytest.runme 2021-08-06T09:03:16.244034+00:00
[failed]>
[2021-08-06 09:15:34,531] {taskinstance.py:896} INFO - Dependencies all met
for <TaskInstance: touch_file_mytest.runme 2021-08-06T09:03:16.244034+00:00
[failed]>
[2021-08-06 09:15:34,531] {taskinstance.py:1087} INFO
---------------------------------------------------------------------------------
[2021-08-06 09:15:34,531] {taskinstance.py:1088} INFO - Starting attempt 2
of 1
[2021-08-06 09:15:34,532] {taskinstance.py:1089} INFO
---------------------------------------------------------------------------------
[2021-08-06 09:15:34,532] {taskinstance.py:1107} INFO - Executing
<Task(BashOperator): runme> on 2021-08-06T09:03:16.244034+00:00
[2021-08-06 09:15:35,461] {taskinstance.py:1302} INFO - Exporting the
following env vars:
AIRFLOW_CTX_DAG_OWNER=airflow
AIRFLOW_CTX_DAG_ID=touch_file_mytest
AIRFLOW_CTX_TASK_ID=runme
AIRFLOW_CTX_EXECUTION_DATE=2021-08-06T09:03:16.244034+00:00
AIRFLOW_CTX_DAG_RUN_ID=manual__2021-08-06T09:03:16.244034+00:00
[2021-08-06 09:15:35,462] {subprocess.py:52} INFO - Tmp dir root location:
/tmp
[2021-08-06 09:15:35,463] {subprocess.py:63} INFO - Running command:
['bash', '-c', 'touch /tmp/airflow_mytest']
[2021-08-06 09:15:35,470] {subprocess.py:74} INFO - Output:
[2021-08-06 09:15:35,473] {subprocess.py:82} INFO - Command exited with
return code 0
[2021-08-06 09:15:35,781] {taskinstance.py:1211} INFO - Marking task as
SUCCESS. dag_id=touch_file_mytest, task_id=runme,
execution_date=20210806T090316, start_date=, end_date=20210806T091535
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
