cliebBS opened a new issue, #35932:
URL: https://github.com/apache/airflow/issues/35932
### Apache Airflow version
2.7.3
### What happened
When using the `SqsSensor` with `message_filtering="jsonpath"` and both
`message_filtering_config` and `message_filtering_match_values` set, the
following error is produced:
```
[2023-11-28, 18:24:44 UTC] {taskinstance.py:1937} ERROR - [findings_sync
manual__2023-11-27T16:12:36+00:00 notify_started_test_receive 18] Task failed
with exception
Traceback (most recent call last):
File
"/home/airflow/.local/lib/python3.11/site-packages/airflow/providers/amazon/aws/sensors/sqs.py",
line 144, in execute
super().execute(context=context)
File
"/home/airflow/.local/lib/python3.11/site-packages/airflow/sensors/base.py",
line 257, in execute
raise e
File
"/home/airflow/.local/lib/python3.11/site-packages/airflow/sensors/base.py",
line 239, in execute
poke_return = self.poke(context)
^^^^^^^^^^^^^^^^^^
File
"/home/airflow/.local/lib/python3.11/site-packages/airflow/providers/amazon/aws/sensors/sqs.py",
line 187, in poke
messages = process_response(
^^^^^^^^^^^^^^^^^
File
"/home/airflow/.local/lib/python3.11/site-packages/airflow/providers/amazon/aws/utils/sqs.py",
line 49, in process_response
messages = filter_messages(
^^^^^^^^^^^^^^^^
File
"/home/airflow/.local/lib/python3.11/site-packages/airflow/providers/amazon/aws/utils/sqs.py",
line 63, in filter_messages
return filter_messages_jsonpath(messages,
message_filtering_match_values, message_filtering_config)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File
"/home/airflow/.local/lib/python3.11/site-packages/airflow/providers/amazon/aws/utils/sqs.py",
line 87, in filter_messages_jsonpath
or any(result.value in message_filtering_match_values for result in
results)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File
"/home/airflow/.local/lib/python3.11/site-packages/airflow/providers/amazon/aws/utils/sqs.py",
line 87, in <genexpr>
or any(result.value in message_filtering_match_values for result in
results)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: unhashable type: 'dict'
```
### What you think should happen instead
The messages should be filtered using the provided JsonPath query and the
provided match values.
### How to reproduce
Sorry, all I have available is a FIFO queue, and the Amazon providers for
both SQS and SNS are unable to send messages to FIFO queues due to their lack
of support for the `message_deduplication_id` field, which FIFO queues require
to be set.
### Operating System
MacOS 14.1.1 and Amazon Linux 2
### Versions of Apache Airflow Providers
8.10.0
### Deployment
Official Apache Airflow Helm Chart
### Deployment details
_No response_
### Anything else
_No response_
### Are you willing to submit PR?
- [X] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of
Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]