+1 On Thu, 13 Feb 2025 at 17:31, Constance Martineau <consta...@astronomer.io.invalid> wrote:
> +1 Excited to see this > > On Thu, Feb 13, 2025 at 11:01 AM Pierre Jeambrun <pierrejb...@gmail.com> > wrote: > > > +1 > > > > On Thu, Feb 13, 2025 at 4:36 PM Vincent Beck <vincb...@apache.org> > wrote: > > > > > Hi Vikram, > > > > > > Thanks for putting it together, I think this is great and will help > users > > > to use event driven scheduling in Airflow 3! > > > > > > On 2025/02/13 08:10:05 "Mehta, Shubham" wrote: > > > > +1 > > > > > > > > We should also consider extending this abstraction to Airflow > > publishing > > > (such as notifier use cases). Haven't thought deeply about it, but the > > > current design only includes a consumer hook without a producer hook, > > > suggesting an opportunity to simplify the publishing side as well. > > > > > > > > Thanks > > > > Shubham > > > > > > > > > > > > On 2025-02-12, 1:30 PM, "Vikram Koka" <vik...@astronomer.io.inva > > > <mailto:vik...@astronomer.io.inva>LID> wrote: > > > > > > > > > > > > CAUTION: This email originated from outside of the organization. Do > not > > > click links or open attachments unless you can confirm the sender and > > know > > > the content is safe. > > > > > > > > > > > > > > > > > > > > > > > > > > > > AVERTISSEMENT: Ce courrier électronique provient d’un expéditeur > > > externe. Ne cliquez sur aucun lien et n’ouvrez aucune pièce jointe si > > vous > > > ne pouvez pas confirmer l’identité de l’expéditeur et si vous n’êtes > pas > > > certain que le contenu ne présente aucun risque. > > > > > > > > > > > > > > > > > > > > > > > > > > > > Hey everyone, > > > > > > > > > > > > I created the following (very early) draft PR > > > > <https://github.com/apache/airflow/pull/46694> < > > > https://github.com/apache/airflow/pull/46694>> to introduce and > > > socialize > > > > the concept of a "Common Message Queue" abstraction, similar to the > > > "Common > > > > SQL" and "Common IO" abstractions already present in Airflow. > > > > > > > > > > > > This "Common Message Queue" abstraction will be a provider package > > > similar > > > > to the other "Common" packages and is intended to be a simple > > abstraction > > > > over Apache Kafka, Amazon SQS, Google PubSub, and others messaging > > > > frameworks. > > > > > > > > > > > > One of the key initial goals is to provide a simple DAG writing > > > abstraction > > > > for integrating Event Driven Scheduling coming with Airflow 3 with > > > > streaming systems such as Kafka. This proposal was crafted in > > > collaboration > > > > with Vincent Beck, who has been diligently working through AIP-82 as > > part > > > > of Airflow 3. > > > > > > > > > > > > At this stage, this is very much a work in progress draft and is > > intended > > > > to solicit feedback, rather than in any way a completed incarnation. > > > > > > > > > > > > Please feel free to comment on the PR directly or reach out to me > over > > > > email or slack. > > > > > > > > > > > > Best regards, > > > > Vikram > > > > > > > > > > > > -- > > > > > > > > > > > > Vikram Koka > > > > Chief Strategy Officer > > > > Email: vik...@astronomer.io <mailto:vik...@astronomer.io> > > > > > > > > > > > > > > > > > > > > <https://www.astronomer.io/> <https://www.astronomer.io/>> > > > > > > > > > > > > > > > > > > > > > > --------------------------------------------------------------------- > > > To unsubscribe, e-mail: dev-unsubscr...@airflow.apache.org > > > For additional commands, e-mail: dev-h...@airflow.apache.org > > > > > > > > >
