patgarz opened a new issue, #33091:
URL: https://github.com/apache/airflow/issues/33091
### Apache Airflow version
2.6.3
### What happened
When using `deferrable=True` as introduced in #31018 and with an AWS conn
that uses role assumption, the triggerer defers but then immediately fails and
exits with:
```txt
[2023-08-03, 18:58:12 EDT] {taskinstance.py:1824} ERROR - Task failed with
exception
Traceback (most recent call last):
File
"/app/.local/lib/python3.9/site-packages/airflow/providers/amazon/aws/sensors/s3.py",
line 176, in execute_complete
raise AirflowException(event["message"])
airflow.exceptions.AirflowException: object ReadOnlyCredentials can't be
used in 'await' expression
```
### What you think should happen instead
Task should be `deferred` while Triggerer waits and then resume once Sensor
configuration is satisfied.
### How to reproduce
Working: `aws_east` looks like: `aws:///?region_name=us-east-1`
Not working: `aws_east` looks like:
`aws:///?region_name=us-east-1&role_arn=arn%3Aaws%3Aiam%3A%3A123456789012%3Arole%2FRoleName`
```py
from airflow.decorators import dag
import pendulum
from airflow.providers.amazon.aws.sensors.s3 import S3KeySensor
@dag(
dag_id="Deferrable_Sensor_Test",
schedule=None,
start_date=pendulum.datetime(2022, 11, 15, tz="UTC"),
catchup=False,
)
def deferrable_sensor_test():
detect_zip= S3KeySensor(
task_id="detect",
aws_conn_id="aws_east",
bucket_name="bucket",
bucket_key="path/*.zip",
wildcard_match=True,
timeout=300,
soft_fail=True,
retries=0,
deferrable=True,
)
deferrable_sensor_test()
```
### Operating System
Ubuntu 20.04
### Versions of Apache Airflow Providers
apache-airflow==2.6.3
apache-airflow-providers-amazon==8.2.0
### Deployment
Other Docker-based deployment
### Deployment details
_No response_
### Anything else
I am unsure of and unfortunately unable to test other AWS conn configs other
than role assumption (for example using an access key ID & secret directly).
Full log for failed run (scrubbed for identifying information):
```txt
[2023-08-03, 18:25:56 EDT] {taskinstance.py:1103} INFO - Dependencies all
met for dep_context=non-requeueable deps ti=<TaskInstance: {TI_NAME}
manual__2023-08-03T22:25:55.844339+00:00 [queued]>
[2023-08-03, 18:25:56 EDT] {taskinstance.py:1103} INFO - Dependencies all
met for dep_context=requeueable deps ti=<TaskInstance: {TI_NAME}
manual__2023-08-03T22:25:55.844339+00:00 [queued]>
[2023-08-03, 18:25:56 EDT] {taskinstance.py:1308} INFO - Starting attempt 1
of 1
[2023-08-03, 18:25:56 EDT] {taskinstance.py:1327} INFO - Executing
<Task(S3KeySensor): {TASK}> on 2023-08-03 22:25:55.844339+00:00
[2023-08-03, 18:25:56 EDT] {standard_task_runner.py:57} INFO - Started
process 568 to run task
[2023-08-03, 18:25:56 EDT] {standard_task_runner.py:84} INFO - Running:
['airflow', 'tasks', 'run', '{DAG_ID}', '{TASK}',
'manual__2023-08-03T22:25:55.844339+00:00', '--job-id', '27750', '--raw',
'--subdir', 'DAGS_FOLDER/{DAG_FILEPATH}', '--cfg-path', '/tmp/tmp_aj8asfv']
[2023-08-03, 18:25:56 EDT] {standard_task_runner.py:85} INFO - Job 27750:
Subtask {TASK}
[2023-08-03, 18:25:56 EDT] {task_command.py:410} INFO - Running
<TaskInstance: {TI_NAME} manual__2023-08-03T22:25:55.844339+00:00 [running]> on
host {HOSTNAME}
[2023-08-03, 18:25:56 EDT] {taskinstance.py:1545} INFO - Exporting env vars:
AIRFLOW_CTX_DAG_OWNER='***' AIRFLOW_CTX_DAG_ID='{DAG_ID}'
AIRFLOW_CTX_TASK_ID='{TASK}'
AIRFLOW_CTX_EXECUTION_DATE='2023-08-03T22:25:55.844339+00:00'
AIRFLOW_CTX_TRY_NUMBER='1'
AIRFLOW_CTX_DAG_RUN_ID='manual__2023-08-03T22:25:55.844339+00:00'
[2023-08-03, 18:25:56 EDT] {s3.py:103} INFO - Poking for key : s3://***/*.zip
[2023-08-03, 18:25:56 EDT] {base.py:73} INFO - Using connection ID
'***_east' for task execution.
[2023-08-03, 18:25:57 EDT] {taskinstance.py:1415} INFO - Pausing task as
DEFERRED. dag_id={DAG_ID}, task_id={TASK}, execution_date=20230803T222555,
start_date=20230803T222556
[2023-08-03, 18:25:57 EDT] {local_task_job_runner.py:222} INFO - Task exited
with return code 100 (task deferral)
[2023-08-03, 18:25:58 EDT] {base.py:73} INFO - Using connection ID
'***_east' for task execution.
[2023-08-03, 18:25:58 EDT] {triggerer_job_runner.py:616} INFO - Trigger
{DAG_ID}/manual__2023-08-03T22:25:55.844339+00:00/{TASK}/-1/1 (ID 8) fired:
TriggerEvent<{'status': 'error', 'message': "object ReadOnlyCredentials can't
be used in 'await' expression"}>
[2023-08-03, 18:26:01 EDT] {taskinstance.py:1103} INFO - Dependencies all
met for dep_context=non-requeueable deps ti=<TaskInstance: {TI_NAME}
manual__2023-08-03T22:25:55.844339+00:00 [queued]>
[2023-08-03, 18:26:01 EDT] {taskinstance.py:1103} INFO - Dependencies all
met for dep_context=requeueable deps ti=<TaskInstance: {TI_NAME}
manual__2023-08-03T22:25:55.844339+00:00 [queued]>
[2023-08-03, 18:26:01 EDT] {taskinstance.py:1306} INFO - Resuming after
deferral
[2023-08-03, 18:26:01 EDT] {taskinstance.py:1327} INFO - Executing
<Task(S3KeySensor): {TASK}> on 2023-08-03 22:25:55.844339+00:00
[2023-08-03, 18:26:01 EDT] {standard_task_runner.py:57} INFO - Started
process 587 to run task
[2023-08-03, 18:26:01 EDT] {standard_task_runner.py:84} INFO - Running:
['airflow', 'tasks', 'run', '{DAG_ID}', '{TASK}',
'manual__2023-08-03T22:25:55.844339+00:00', '--job-id', '27751', '--raw',
'--subdir', 'DAGS_FOLDER/{DAG_FILEPATH}', '--cfg-path', '/tmp/tmp_pgylaxm']
[2023-08-03, 18:26:01 EDT] {standard_task_runner.py:85} INFO - Job 27751:
Subtask {TASK}
[2023-08-03, 18:26:01 EDT] {task_command.py:410} INFO - Running
<TaskInstance: {TI_NAME} manual__2023-08-03T22:25:55.844339+00:00 [running]> on
host {HOSTNAME}
[2023-08-03, 18:26:01 EDT] {taskinstance.py:1824} ERROR - Task failed with
exception
Traceback (most recent call last):
File
"/app/.local/lib/python3.9/site-packages/airflow/providers/amazon/aws/sensors/s3.py",
line 176, in execute_complete
raise AirflowException(event["message"])
airflow.exceptions.AirflowException: object ReadOnlyCredentials can't be
used in 'await' expression
[2023-08-03, 18:26:01 EDT] {taskinstance.py:1345} INFO - Marking task as
FAILED. dag_id={DAG_ID}, task_id={TASK}, execution_date=20230803T222555,
start_date=20230803T222556, end_date=20230803T222601
[2023-08-03, 18:26:01 EDT] {standard_task_runner.py:104} ERROR - Failed to
execute job 27751 for task {TASK} (object ReadOnlyCredentials can't be used in
'await' expression; 587)
[2023-08-03, 18:26:01 EDT] {local_task_job_runner.py:225} INFO - Task exited
with return code 1
[2023-08-03, 18:26:01 EDT] {taskinstance.py:2653} INFO - 0 downstream tasks
scheduled from follow-on schedule check
```
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of
Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
