nicnguyen3103 opened a new issue, #36885:
URL: https://github.com/apache/airflow/issues/36885
### Apache Airflow version
2.8.0
### If "Other Airflow 2 version" selected, which one?
_No response_
### What happened?
Using PostgresSQL backend, and an empty DagFolder, running command `airflow
dag-processor` return the following error
```
2024-01-19T13:11:23.238+0700] {settings.py:61} INFO - Configured default
timezone Timezone('UTC')
[2024-01-19T13:11:23.265+0700] {dag_processor_job_runner.py:60} INFO -
Starting the Dag Processor Job
[2024-01-19T13:11:23.283+0700] {dag_processor_job_runner.py:64} ERROR -
Exception when executing DagProcessorJob
Traceback (most recent call last):
File
"/Users/nic/.virtualenvs/airflow-migration/lib/python3.10/site-packages/airflow/jobs/dag_processor_job_runner.py",
line 62, in _execute
self.processor.start()
File
"/Users/nic/.virtualenvs/airflow-migration/lib/python3.10/site-packages/airflow/dag_processing/manager.py",
line 475, in start
return self._run_parsing_loop()
File
"/Users/nic/.virtualenvs/airflow-migration/lib/python3.10/site-packages/airflow/dag_processing/manager.py",
line 558, in _run_parsing_loop
ready = multiprocessing.connection.wait(self.waitables.keys(),
timeout=poll_time)
AttributeError: module 'multiprocessing' has no attribute 'connection'
Traceback (most recent call last):
File "/Users/nic/.virtualenvs/airflow-migration/bin/airflow", line 8, in
<module>
sys.exit(main())
File
"/Users/nic/.virtualenvs/airflow-migration/lib/python3.10/site-packages/airflow/__main__.py",
line 57, in main
args.func(args)
File
"/Users/nic/.virtualenvs/airflow-migration/lib/python3.10/site-packages/airflow/cli/cli_config.py",
line 49, in command
return func(*args, **kwargs)
File
"/Users/nic/.virtualenvs/airflow-migration/lib/python3.10/site-packages/airflow/utils/cli.py",
line 114, in wrapper
return f(*args, **kwargs)
File
"/Users/nic/.virtualenvs/airflow-migration/lib/python3.10/site-packages/airflow/utils/providers_configuration_loader.py",
line 55, in wrapped_function
return func(*args, **kwargs)
File
"/Users/nic/.virtualenvs/airflow-migration/lib/python3.10/site-packages/airflow/cli/commands/dag_processor_command.py",
line 65, in dag_processor
run_command_with_daemon_option(
File
"/Users/nic/.virtualenvs/airflow-migration/lib/python3.10/site-packages/airflow/cli/commands/daemon_utils.py",
line 86, in run_command_with_daemon_option
callback()
File
"/Users/nic/.virtualenvs/airflow-migration/lib/python3.10/site-packages/airflow/cli/commands/dag_processor_command.py",
line 68, in <lambda>
callback=lambda: run_job(job=job_runner.job,
execute_callable=job_runner._execute),
File
"/Users/nic/.virtualenvs/airflow-migration/lib/python3.10/site-packages/airflow/utils/session.py",
line 79, in wrapper
return func(*args, session=session, **kwargs)
File
"/Users/nic/.virtualenvs/airflow-migration/lib/python3.10/site-packages/airflow/jobs/job.py",
line 393, in run_job
return execute_job(job, execute_callable=execute_callable)
File
"/Users/nic/.virtualenvs/airflow-migration/lib/python3.10/site-packages/airflow/jobs/job.py",
line 422, in execute_job
ret = execute_callable()
File
"/Users/nic/.virtualenvs/airflow-migration/lib/python3.10/site-packages/airflow/jobs/dag_processor_job_runner.py",
line 62, in _execute
self.processor.start()
File
"/Users/nic/.virtualenvs/airflow-migration/lib/python3.10/site-packages/airflow/dag_processing/manager.py",
line 475, in start
return self._run_parsing_loop()
File
"/Users/nic/.virtualenvs/airflow-migration/lib/python3.10/site-packages/airflow/dag_processing/manager.py",
line 558, in _run_parsing_loop
ready = multiprocessing.connection.wait(self.waitables.keys(),
timeout=poll_time)
AttributeError: module 'multiprocessing' has no attribute 'connection'. Did
you mean: 'Condition'?
```
Adding a single dag inside the dag folder fix the issue:
```
[2024-01-19T13:16:47.159+0700] {settings.py:61} INFO - Configured default
timezone Timezone('UTC')
[2024-01-19T13:16:47.193+0700] {dag_processor_job_runner.py:60} INFO -
Starting the Dag Processor Job
```
### What you think should happen instead?
DagProcessor should be running regardless of dag folder empty or not
### How to reproduce
Running `airflow dag-processor` command with an empty dag folder with
PostgresSQL backend
### Operating System
MacOS
### Versions of Apache Airflow Providers
apache-airflow==2.8.0
apache-airflow-providers-common-sql==1.10.0
apache-airflow-providers-ftp==3.7.0
apache-airflow-providers-http==4.8.0
apache-airflow-providers-imap==3.5.0
apache-airflow-providers-sqlite==3.7.0
### Deployment
Official Apache Airflow Helm Chart
### Deployment details
_No response_
### Anything else?
_No response_
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of
Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
