freget opened a new issue #12669:
URL: https://github.com/apache/airflow/issues/12669
<!--
Welcome to Apache Airflow! For a smooth issue process, try to answer the
following questions.
Don't worry if they're not all applicable; just try to include what you can
:-)
If you need to include code snippets or logs, please put them in fenced code
blocks. If they're super-long, please use the details tag like
<details><summary>super-long log</summary> lots of stuff </details>
Please delete these comment blocks before submitting the issue.
-->
**Apache Airflow version**: v2.0.0b2
**Kubernetes version (if you are using kubernetes)** (use `kubectl
version`): v1.16.13
**Environment**:
- **Cloud provider or hardware configuration**: Azure AKS
- **OS** (e.g. from /etc/os-release): official airflow image
**What happened**:
When configuring the worker to write logs to a PersistantVolumeClaim in
Kubernetes that does not support unix-style permissions (AzureFiles in my
case), writing the logs - and hence the whole job - fails.
```
Traceback (most recent call last):
File "/home/airflow/.local/bin/airflow", line 8, in <module>
sys.exit(main())
File
"/home/airflow/.local/lib/python3.8/site-packages/airflow/__main__.py", line
40, in main
args.func(args)
File
"/home/airflow/.local/lib/python3.8/site-packages/airflow/cli/cli_parser.py",
line 50, in command
return func(*args, **kwargs)
File
"/home/airflow/.local/lib/python3.8/site-packages/airflow/utils/cli.py", line
86, in wrapper
return f(*args, **kwargs)
File
"/home/airflow/.local/lib/python3.8/site-packages/airflow/cli/commands/task_command.py",
line 179, in task_run
ti.init_run_context(raw=args.raw)
File
"/home/airflow/.local/lib/python3.8/site-packages/airflow/models/taskinstance.py",
line 1922, in init_run_context
self._set_context(self)
File
"/home/airflow/.local/lib/python3.8/site-packages/airflow/utils/log/logging_mixin.py",
line 54, in _set_context
set_context(self.log, context)
File
"/home/airflow/.local/lib/python3.8/site-packages/airflow/utils/log/logging_mixin.py",
line 173, in set_context
handler.set_context(value)
File
"/home/airflow/.local/lib/python3.8/site-packages/airflow/utils/log/file_task_handler.py",
line 54, in set_context
local_loc = self._init_file(ti)
File
"/home/airflow/.local/lib/python3.8/site-packages/airflow/utils/log/file_task_handler.py",
line 256, in _init_file
os.chmod(full_path, 0o666)
PermissionError: [Errno 1] Operation not permitted:
'/opt/airflow/logs/dagname/jobname/2020-11-27T20:57:00+00:00/1.log'
```
**What you expected to happen**:
Airflow should recognize that permissions are not available on the volume or
allow disabling setting permissions for logfiles via config.
**How to reproduce it**:
Try setting the default log path to a volume not supporting permissions.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
