ferruzzi commented on code in PR #24554:
URL: https://github.com/apache/airflow/pull/24554#discussion_r901893929
##########
airflow/providers/amazon/aws/sensors/sqs.py:
##########
@@ -136,15 +135,27 @@ def poke(self, context: 'Context'):
messages = self.filter_messages(messages)
num_messages = len(messages)
self.log.info("There are %d messages left after filtering",
num_messages)
+ return messages
+
+ def poke(self, context: 'Context'):
+ """
+ Check for message on subscribed queue and write to xcom the message
with key ``messages``
- if not num_messages:
+ :param context: the context object
+ :return: ``True`` if message is available or ``False``
+ """
+ sqs_conn = self.get_hook().get_conn()
Review Comment:
I know this is how it was done elsewhere in this file, and someone may
correct me if I'm wrong, but I think we're supposed to be moving towards the
following way of accessing the connection. If that's right, then could you
make that change while you are in here?
```suggestion
sqs_conn = self.get_hook().conn
```
##########
tests/system/providers/amazon/aws/example_sqs.py:
##########
@@ -66,6 +66,20 @@ def delete_queue(queue_url):
)
# [END howto_sensor_sqs]
+ # [START howto_sensor_sqs_batch]
+ # batch multiple messages from SQS.
+ # each SQS poll can retrieve no more than 10 messages
+ # due to requirements by AWS SQS
Review Comment:
```suggestion
# Retrieve multiple batches of messages from SQS.
# The SQS API only returns a maximum of 10 messages per poll.
```
##########
tests/system/providers/amazon/aws/example_sqs.py:
##########
@@ -66,6 +66,20 @@ def delete_queue(queue_url):
)
# [END howto_sensor_sqs]
+ # [START howto_sensor_sqs_batch]
+ # batch multiple messages from SQS.
+ # each SQS poll can retrieve no more than 10 messages
+ # due to requirements by AWS SQS
+ read_from_queue_in_batch = SqsBatchSensor(
+ task_id='read_from_queue_in_batch',
+ sqs_queue=create_queue,
+ # get maximum 10 messages each poll
Review Comment:
```suggestion
# Get maximum 10 messages each poll
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
