vikramkoka commented on PR #46694:
URL: https://github.com/apache/airflow/pull/46694#issuecomment-2660125707
> Implementation wise, here is my thinking. I am starting by
`MessageQueueTrigger`.
>
> Given `msg_queue`, `MessageQueueTrigger` needs to figure which hook it
will use to poll/pop a message from the queue. Example: `if
msg_queue.starts_with("https://sqs.";): hook = SqsHook(...)`. Then we can use
the hook to retrieve the message. The hook will contain the logic for each
provider (AWS, Google, Kafka, ...). This means, this new provider will have a
dependency with all these providers. Do you think this is an issue? Did you
have something else in mind?
You are right Vincent. I did think about the "Composition vs. Inheritance"
approach tradeoff.
The composition style interface as defined here is easier for the DAG
author, but more maintenance for us.
I talked about this with Ash and Jed as well and because of the underlying
plumbing already present in Airflow for finding connections, et al, this seemed
reasonable as an approach in order to make the end-user experience better.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
