rjtshrm commented on issue #52501:
URL: https://github.com/apache/airflow/issues/52501#issuecomment-3210323075
Hi @ashb , I am still getting the errors with airflow 3.0.5 and cannot get
the logs to the cloudwatch. When even I trigger a new task, it could not find
the the connection id even though it is in Connections. On manually inspecting
to the task and api-server pod, I could not get any connections from python api.
```
/home/airflow/.local/lib/python3.11/site-packages/airflow/configuration.py:859
DeprecationWarning: The secret_key option in [webserver] has been moved to the
secret_key option in [api] - the old setting has been used, but please update
your config.
[2025-08-21T11:29:52.062+0000] {base_aws.py:632} WARNING - Unable to find
AWS Connection ID 'aws_connection', switching to empty.
[2025-08-21T11:29:52.064+0000] {base_aws.py:201} INFO - No connection ID
provided. Fallback on boto3 credential strategy (region_name='eu-central-1').
See:
https://boto3.amazonaws.com/v1/documentation/api/latest/guide/configuration.html
{"timestamp":"2025-08-21T11:29:53.585436Z","level":"info","event":"Executing
workload","workload":"ExecuteTask(token='eyJhbGciOiJIUzUxMiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiIwMTk4Y2M2Mi0wNmJlLTdkZDQtYmRlMi1kM2RhMWU3YWQzZTQiLCJhdWQiOiJ1cm46YWlyZmxvdy5hcGFjaGUub3JnOnRhc2siLCJuYmYiOjE3NTU3NzU2MzMsImV4cCI6MTc1NTc3NjIzMywiaWF0IjoxNzU1Nzc1NjMzfQ.cUHy6gJPU3xNvtO-r4Q8Hogf3jF3uPIu9cXMMBXxuPU_47aYF5Xuwk61ioMuuKMxDt3KxHi8009n6V41mqve5A',
ti=TaskInstance(id=UUID('0198cc62-06be-7dd4-bde2-d3da1e7ad3e4'),
task_id='emr-kafka_ml_dump', dag_id='integration_tests',
run_id='scheduled__2025-08-21T00:00:00+00:00', try_number=11, map_index=-1,
pool_slots=1, queue='default', priority_weight=3, executor_config=None,
parent_context_carrier={}, context_carrier={},
queued_dttm=datetime.datetime(2025, 8, 21, 10, 52, 16, 480084,
tzinfo=TzInfo(UTC))), dag_rel_path=PurePosixPath('integration_tests.py'),
bundle_info=BundleInfo(name='dags-folder', version=None),
log_path='dag_id=integration_tests/run_id=scheduled__2025-08
-21T00:00:00+00:00/task_id=emr-kafka_ml_dump/attempt=11.log',
type='ExecuteTask')","logger":"__main__"}
{"timestamp":"2025-08-21T11:29:57.072953Z","level":"info","event":"Connecting
to
server:","server":"http://airflow-api-server:8080/execution/","logger":"__main__"}
{"timestamp":"2025-08-21T11:29:57.169262Z","level":"info","event":"Secrets
backends loaded for
worker","count":1,"backend_classes":["EnvironmentVariablesBackend"],"logger":"supervisor"}
{"timestamp":"2025-08-21T11:29:57.264719Z","level":"warning","event":"Received
message after logging system
shutdown","category":"WatchtowerWarning","filename":"/home/airflow/.local/lib/python3.11/site-packages/watchtower/__init__.py","lineno":464,"logger":"py.warnings"}
```
```
>>> import airflow
/home/airflow/.local/lib/python3.11/site-packages/airflow/configuration.py:859
DeprecationWarning: The secret_key option in [webserver] has been moved to the
secret_key option in [api] - the old setting has been used, but please update
your config.
>>> airflow.__version__
'3.0.5'
>>> from airflow.sdk import Connection
>>> Connection.get("aws_connection")
2025-08-21 11:45:24 [debug ] Connection not found in any of the
configured Secrets Backends. Trying to retrieve from API server [task]
conn_id=aws_connection
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File
"/home/airflow/.local/lib/python3.11/site-packages/airflow/sdk/definitions/connection.py",
line 144, in get
return _get_connection(conn_id)
^^^^^^^^^^^^^^^^^^^^^^^^
File
"/home/airflow/.local/lib/python3.11/site-packages/airflow/sdk/execution_time/context.py",
line 158, in _get_connection
from airflow.sdk.execution_time.task_runner import SUPERVISOR_COMMS
ImportError: cannot import name 'SUPERVISOR_COMMS' from
'airflow.sdk.execution_time.task_runner'
(/home/airflow/.local/lib/python3.11/site-packages/airflow/sdk/execution_time/task_runner.py)
>>>
```
but if I display connection using airflow cli, it appears to be fine.
<img width="2618" height="232" alt="aws_connection"
src="https://github.com/user-attachments/assets/3c34f960-78c7-4498-a504-e03ec9f9537e";
/>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
