Echronix opened a new issue #18549:
URL: https://github.com/apache/airflow/issues/18549
### Apache Airflow version
2.1.2
### Operating System
Ubuntu Docker
### Versions of Apache Airflow Providers
apache-airflow-providers-amazon==2.0.0
apache-airflow-providers-celery==2.0.0
apache-airflow-providers-cncf-kubernetes==2.0.0
apache-airflow-providers-docker==2.0.0
apache-airflow-providers-elasticsearch==2.0.2
apache-airflow-providers-ftp==2.0.0
apache-airflow-providers-google==4.0.0
apache-airflow-providers-grpc==2.0.0
apache-airflow-providers-hashicorp==2.0.0
apache-airflow-providers-http==2.0.0
apache-airflow-providers-imap==2.0.0
apache-airflow-providers-microsoft-azure==3.0.0
apache-airflow-providers-mysql==2.0.0
apache-airflow-providers-neo4j==2.0.0
apache-airflow-providers-postgres==2.0.0
apache-airflow-providers-redis==2.0.0
apache-airflow-providers-sendgrid==2.0.0
apache-airflow-providers-sftp==2.0.0
apache-airflow-providers-slack==4.0.0
apache-airflow-providers-sqlite==2.0.0
apache-airflow-providers-ssh==2.0.0
### Deployment
Official Apache Airflow Helm Chart
### Deployment details
```
config:
core:
executor: 'CeleryExecutor'
logging:
remote_logging: 'True'
remote_base_log_folder:
'cloudwatch://arn:aws:logs:eu-central-1:<redacted>:log-group:airflow'
remote_log_conn_id: 'cloudwatch'
```
The log group is already created in Terraform.
### What happened
```
[2021-09-27 15:03:58,541: ERROR/ForkPoolWorker-15] Failed to execute task An
error occurred (AccessDeniedException) when calling the CreateLogGroup
operation: User:
arn:aws:sts::<redacted>:assumed-role/airflow-service-account-role/botocore-session-1632755037
is not authorized to perform: logs:CreateLogGroup on resource:
arn:aws:logs:eu-central-1:<redacted>:log-group:airflow:log-stream:.
Traceback (most recent call last):
File
"/home/airflow/.local/lib/python3.8/site-packages/airflow/executors/celery_executor.py",
line 117, in _execute_in_fork
args.func(args)
File
"/home/airflow/.local/lib/python3.8/site-packages/airflow/cli/cli_parser.py",
line 48, in command
return func(*args, **kwargs)
File
"/home/airflow/.local/lib/python3.8/site-packages/airflow/utils/cli.py", line
91, in wrapper
return f(*args, **kwargs)
File
"/home/airflow/.local/lib/python3.8/site-packages/airflow/cli/commands/task_command.py",
line 228, in task_run
ti.init_run_context(raw=args.raw)
File
"/home/airflow/.local/lib/python3.8/site-packages/airflow/models/taskinstance.py",
line 2031, in init_run_context
self._set_context(self)
File
"/home/airflow/.local/lib/python3.8/site-packages/airflow/utils/log/logging_mixin.py",
line 54, in _set_context
set_context(self.log, context)
File
"/home/airflow/.local/lib/python3.8/site-packages/airflow/utils/log/logging_mixin.py",
line 173, in set_context
handler.set_context(value)
File
"/home/airflow/.local/lib/python3.8/site-packages/airflow/providers/amazon/aws/log/cloudwatch_task_handler.py",
line 81, in set_context
self.handler = watchtower.CloudWatchLogHandler(
File
"/home/airflow/.local/lib/python3.8/site-packages/watchtower/__init__.py", line
148, in __init__
_idempotent_create(self.cwl_client, "create_log_group",
logGroupName=self.log_group)
File
"/home/airflow/.local/lib/python3.8/site-packages/watchtower/__init__.py", line
15, in _idempotent_create
method_callable(*args, **kwargs)
File
"/home/airflow/.local/lib/python3.8/site-packages/botocore/client.py", line
386, in _api_call
return self._make_api_call(operation_name, kwargs)
File
"/home/airflow/.local/lib/python3.8/site-packages/botocore/client.py", line
705, in _make_api_call
raise error_class(parsed_response, operation_name)
botocore.exceptions.ClientError: An error occurred (AccessDeniedException)
when calling the CreateLogGroup operation: User:
arn:aws:sts:<redacted>}:assumed-role/airflow-service-account-role/botocore-session-1632755037
is not authorized to perform: logs:CreateLogGroup on resource:
arn:aws:logs:eu-central-1:<redacted>:log-group:airflow:log-stream:
```
### What you expected to happen
I would expect Airflow to create a stream and write the logs, not to create
a log-group inside a log-group.
### How to reproduce
Run any dag with the chart config.
### Anything else
_No response_
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of
Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
