vincbeck commented on code in PR #54651:
URL: https://github.com/apache/airflow/pull/54651#discussion_r2301616750
##########
providers/apache/kafka/src/airflow/providers/apache/kafka/queues/kafka.py:
##########
@@ -35,44 +35,35 @@ class KafkaMessageQueueProvider(BaseMessageQueueProvider):
"""
Configuration for Apache Kafka integration with common-messaging.
- It uses the ``kafka://`` URI scheme for identifying Kafka queues.
+ [START kafka_message_queue_provider_description]
- **URI Format**:
-
- .. code-block:: text
-
- kafka://<broker>/<topic_list>
-
- Where:
-
- * ``broker``: Kafka brokers (hostname:port)
- * ``topic_list``: Comma-separated list of Kafka topics to consume messages
from
-
- **Examples**:
-
- .. code-block:: text
-
- kafka://localhost:9092/my_topic
-
- **Required kwargs**:
-
- * ``apply_function``: Function to process each Kafka message
-
- You can also provide ``topics`` directly in kwargs instead of in the URI.
+ * It uses the ``kafka`` as scheme for identifying Kafka queues.
+ * For parameter definitions take a look at
:class:`~airflow.providers.apache.kafka.triggers.await_message.AwaitMessageTrigger`.
.. code-block:: python
from airflow.providers.common.messaging.triggers.msg_queue import
MessageQueueTrigger
+ from airflow.sdk import Asset, AssetWatcher
+ # New scheme-based approach
Review Comment:
I do not think it is worth mentioning it
```suggestion
# New scheme-based approach
```
##########
providers/amazon/src/airflow/providers/amazon/aws/queues/sqs.py:
##########
@@ -38,7 +38,36 @@
class SqsMessageQueueProvider(BaseMessageQueueProvider):
- """Configuration for SQS integration with common-messaging."""
+ """
+ Configuration for SQS integration with common-messaging.
+
+ [START sqs_message_queue_provider_description]
+
+ * It uses the ``sqs`` as scheme for identifying SQS queues.
Review Comment:
```suggestion
* It uses ``sqs`` as scheme for identifying SQS queues.
```
##########
providers/apache/kafka/src/airflow/providers/apache/kafka/queues/kafka.py:
##########
@@ -35,44 +35,35 @@ class KafkaMessageQueueProvider(BaseMessageQueueProvider):
"""
Configuration for Apache Kafka integration with common-messaging.
- It uses the ``kafka://`` URI scheme for identifying Kafka queues.
+ [START kafka_message_queue_provider_description]
- **URI Format**:
-
- .. code-block:: text
-
- kafka://<broker>/<topic_list>
-
- Where:
-
- * ``broker``: Kafka brokers (hostname:port)
- * ``topic_list``: Comma-separated list of Kafka topics to consume messages
from
-
- **Examples**:
-
- .. code-block:: text
-
- kafka://localhost:9092/my_topic
-
- **Required kwargs**:
-
- * ``apply_function``: Function to process each Kafka message
-
- You can also provide ``topics`` directly in kwargs instead of in the URI.
+ * It uses the ``kafka`` as scheme for identifying Kafka queues.
Review Comment:
```suggestion
* It uses ``kafka`` as scheme for identifying Kafka queues.
```
##########
providers/amazon/src/airflow/providers/amazon/aws/queues/sqs.py:
##########
@@ -38,7 +38,36 @@
class SqsMessageQueueProvider(BaseMessageQueueProvider):
- """Configuration for SQS integration with common-messaging."""
+ """
+ Configuration for SQS integration with common-messaging.
+
+ [START sqs_message_queue_provider_description]
+
+ * It uses the ``sqs`` as scheme for identifying SQS queues.
+ * For parameter definitions take a look at
:class:`~airflow.providers.amazon.aws.triggers.sqs.SqsSensorTrigger`.
+
+ .. code-block:: python
+
+ from airflow.providers.common.messaging.triggers.msg_queue import
MessageQueueTrigger
+ from airflow.sdk import Asset, AssetWatcher
+
+ # New scheme-based approach
Review Comment:
I do not think it is worth mentioning it
```suggestion
```
##########
providers/redis/src/airflow/providers/redis/queues/redis.py:
##########
@@ -35,57 +33,30 @@ class
RedisPubSubMessageQueueProvider(BaseMessageQueueProvider):
"""
Configuration for Redis integration with common-messaging.
- It uses the ``redis+pubsub://`` URI scheme for identifying Redis queues.
+ [START redis_message_queue_provider_description]
- **URI Format**:
-
- .. code-block:: text
-
- redis+pubsub://<host>:<port>/<channel_list>
-
- Where:
-
- * ``host``: Redis server hostname
- * ``port``: Redis server port
- * ``channel_list``: Comma-separated list of Redis channels to subscribe to
-
- **Examples**:
-
- .. code-block:: text
-
- redis+pubsub://localhost:6379/my_channel
-
- You can also provide ``channels`` directly in kwargs instead of in the URI.
+ * It uses the ``redis+pubsub`` as scheme for identifying Redis queues.
Review Comment:
```suggestion
* It uses ``redis+pubsub`` as scheme for identifying Redis queues.
```
##########
providers/redis/src/airflow/providers/redis/queues/redis.py:
##########
@@ -35,57 +33,30 @@ class
RedisPubSubMessageQueueProvider(BaseMessageQueueProvider):
"""
Configuration for Redis integration with common-messaging.
- It uses the ``redis+pubsub://`` URI scheme for identifying Redis queues.
+ [START redis_message_queue_provider_description]
- **URI Format**:
-
- .. code-block:: text
-
- redis+pubsub://<host>:<port>/<channel_list>
-
- Where:
-
- * ``host``: Redis server hostname
- * ``port``: Redis server port
- * ``channel_list``: Comma-separated list of Redis channels to subscribe to
-
- **Examples**:
-
- .. code-block:: text
-
- redis+pubsub://localhost:6379/my_channel
-
- You can also provide ``channels`` directly in kwargs instead of in the URI.
+ * It uses the ``redis+pubsub`` as scheme for identifying Redis queues.
+ * For parameter definitions take a look at
:class:`~airflow.providers.redis.triggers.redis_await_message.AwaitMessageTrigger`.
.. code-block:: python
from airflow.providers.common.messaging.triggers.msg_queue import
MessageQueueTrigger
+ from airflow.sdk import Asset, AssetWatcher
+
+ # New scheme-based approach
Review Comment:
```suggestion
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
