GitHub user jgoedeke closed a discussion: Scheduler log directory created as
root
### Apache Airflow version
2.10.5
### If "Other Airflow 2 version" selected, which one?
_No response_
### What happened?
Every night at 00:00 a new directory is created in
`/opt/airflow/logs/scheduler/` once in a while this directory is owned by root
instead of the airflow user inside the docker container. This leads to an empty
Dag list on the webserver and airflow not working at all.
airflow-scheduler docker log shows many of:
````bash
airflow-scheduler-1 | Process DagFileProcessor10-Process:
airflow-scheduler-1 | Traceback (most recent call last):
airflow-scheduler-1 | File
"/usr/local/lib/python3.11/multiprocessing/process.py", line 314, in _bootstrap
airflow-scheduler-1 | self.run()
airflow-scheduler-1 | File
"/usr/local/lib/python3.11/multiprocessing/process.py", line 108, in run
airflow-scheduler-1 | self._target(*self._args, **self._kwargs)
airflow-scheduler-1 | File
"/home/airflow/.local/lib/python3.11/site-packages/airflow/dag_processing/processor.py",
line 174, in _run_file_processor
airflow-scheduler-1 | set_context(log, file_path)
airflow-scheduler-1 | File
"/home/airflow/.local/lib/python3.11/site-packages/airflow/utils/log/logging_mixin.py",
line 276, in set_context
airflow-scheduler-1 | flag = cast(FileTaskHandler,
handler).set_context(value)
airflow-scheduler-1 |
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
airflow-scheduler-1 | File
"/home/airflow/.local/lib/python3.11/site-packages/airflow/utils/log/file_processor_handler.py",
line 63, in set_context
airflow-scheduler-1 | local_loc = self._init_file(filename)
airflow-scheduler-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^
airflow-scheduler-1 | File
"/home/airflow/.local/lib/python3.11/site-packages/airflow/utils/log/file_processor_handler.py",
line 151, in _init_file
airflow-scheduler-1 | open(log_file_path, "a").close()
airflow-scheduler-1 | ^^^^^^^^^^^^^^^^^^^^^^^^
airflow-scheduler-1 | PermissionError: [Errno 13] Permission denied:
'/opt/airflow/logs/scheduler/2025-03-27/my_dag.py.log'
````
### What you think should happen instead?
The daily directory should be created always with the same ownership and in
case of a wrong permissions I would still expect airflow to show dags on the
webserver, but stop scheduling new tasks (at the end only the scheduler is
affected, right?).
### How to reproduce
It is hard to reproduce since this happens only once in while, maybe every two
weeks. It feels like a race condition or some other timing issue.
### Operating System
Debian GNU/Linux 12 (bookworm) | Docker version 28.0.4, build b8034c0
### Versions of Apache Airflow Providers
_No response_
### Deployment
Docker-Compose
### Deployment details
I use docker compose environment. For unattended-upgrades I start the container
as root user and use `gosu` in my entrypoint to set the correct user with:
`exec gosu airflow /entrypoint "$@"`. In the airflow Image I set the airflow
UID to 1000 with `RUN usermod -u 1000 airflow`. The logs directory is locally
mounted with
````yaml
volumes:
- /media/airflow-share/logs:/opt/airflow/logs
````
### Anything else?
_No response_
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [x] I agree to follow this project's [Code of
Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
GitHub link: https://github.com/apache/airflow/discussions/48561
----
This is an automatically sent email for [email protected].
To unsubscribe, please send an email to: [email protected]
