Re: [DISCUSS] DRAFT AIP-67 Multi-tenant deployment of Airflow components

2024-04-05 Thread Mehta, Shubham
@Gabe - Thanks for chiming in on this important discussion. I hope other users who are looking for a multi-tenant deployment do the same. As Niko mentioned, the proposal is not to have a separate database per tenant, but rather having isolation with other means, while giving the benefit of

Re: [DISCUSS] DRAFT AIP-67 Multi-tenant deployment of Airflow components

2024-04-05 Thread Oliveira, Niko
Thanks for the reply Gabe! I'm glad you're interested in the topic and weighing in for your company. > @Niko – If I misstated your intent, I hope you will clarify that for me. > Thanks! The intent (DB isolation being critical) was there, but the approach you attributed to me is not actually

Re: [DISCUSS] DRAFT AIP-67 Multi-tenant deployment of Airflow components

2024-04-05 Thread Gabe Schenz
The outcome that Niko brings up is that any tenants should not be able to interact with metadata that is from other tenants within the Airflow environment. If we focus on that as an outcome, what are the various options which would support that goal? 1. A separate db per tenant, as Niko

Re: [VOTE] Release Airflow 2.9.0 from 2.9.0rc2

2024-04-05 Thread Jed Cunningham
+1 (binding) Checked reproducibility, signatures, checksums, licences. Used it with the helm chart with a few different configs. All looks good!

Re: [DISCUSS] Proposal for adding Telemetry via Scarf

2024-04-05 Thread Michał Modras
My 2 cents: it must be possible to opt-out, preferably it should be possible to deploy Airflow instances without bundling the telemetry library dependencies. Other than that I don't mind it being e.g. optional provider. śr., 3 kwi 2024, 22:42 użytkownik Hussein Awala napisał: > > I'd like to

Re: [DISCUSS] Consider disabling self-hosted runners for commiter PRs

2024-04-05 Thread Wei Lee
+1 for this. I do not yet have enough chance to experience many job failures, but it won’t harm us to test them out. Plus, it saves some of the cost. Best, Wei > On Apr 5, 2024, at 11:36 PM, Jarek Potiuk wrote: > > Seeing no big "no's" - I will prepare and run the experiment - starting > some

Re: [DISCUSS] Consider disabling self-hosted runners for commiter PRs

2024-04-05 Thread Jarek Potiuk
Seeing no big "no's" - I will prepare and run the experiment - starting some time next week, after we get 2.9.0 out - I do not want to break anything there. In the meantime, preparatory PR to add "use self-hosted runners" label is out https://github.com/apache/airflow/pull/38779 On Fri, Apr 5,

Re: [DISCUSS] Consider disabling self-hosted runners for commiter PRs

2024-04-05 Thread Bishundeo, Rajeshwar
+1 with trying this out. I agree with keeping the canary builds self-hosted in order to validate the usage for the PRs. -- Rajesh From: Jarek Potiuk Reply-To: "dev@airflow.apache.org" Date: Friday, April 5, 2024 at 8:36 AM To: "dev@airflow.apache.org" Subject: RE: [EXTERNAL] [COURRIEL

Re: [DISCUSS] Consider disabling self-hosted runners for commiter PRs

2024-04-05 Thread Jarek Potiuk
Yeah. Valid concerns Hussein. And I am happy to share some more information on that. I did not want to put all of that in the original email, but I see that might be interesting for you and possibly others. I am closely following the numbers now. One of the reasons I am doing / proposing it now

Re: [DISCUSS] Consider disabling self-hosted runners for commiter PRs

2024-04-05 Thread Aritra Basu
I'm +0. Definitely don't see any issue with seeing the changes. -- Regards, Aritra Basu On Fri, Apr 5, 2024, 3:37 PM Amogh Desai wrote: > +1 I like the idea. > Looking forward to seeing the difference. > > Thanks & Regards, > Amogh Desai > > > On Fri, Apr 5, 2024 at 3:54 AM Ferruzzi, Dennis >

Re: [DISCUSS] Consider disabling self-hosted runners for commiter PRs

2024-04-05 Thread Hussein Awala
Although 900 runners seem like a lot, they are shared among the Apache organization's 2.2k repositories, of course only a few of them are active (let's say 50), and some of them use an external CI tool for big jobs (eg: Kafka uses Jenkins, Hudi uses Azure pipelines), but we have other very active

Re: [DISCUSS] Consider disabling self-hosted runners for commiter PRs

2024-04-05 Thread Amogh Desai
+1 I like the idea. Looking forward to seeing the difference. Thanks & Regards, Amogh Desai On Fri, Apr 5, 2024 at 3:54 AM Ferruzzi, Dennis wrote: > Interested in seeing the difference, +1 > > > - ferruzzi > > > > From: Oliveira, Niko > Sent: Thursday, April

Re: [VOTE] Release Airflow 2.9.0 from 2.9.0rc2

2024-04-05 Thread Amogh Desai
+1 non binding Tested a few example DAGs and tested to see if my changes work as expected. It looks good to me. Thanks & Regards, Amogh Desai On Fri, Apr 5, 2024 at 4:04 AM Jarek Potiuk wrote: > +1 (binding) - checked reproducibility, signatures, checksums, licences - > > all good. Installed