Hi Dennis, Thank you for your suggestion. We are going to add the Fabric Provider within the Microsoft Providers only, maintaining the same hierarchy as the existing ones like microsoft.azure, microsoft.mssql, microsoft.psrp, and microsoft.winrm. This will ensure a consistent and intuitive structure for users.
Best regards, Ambika Garg On Tue, 30 Jul 2024 at 21:28, Kaxil Naik <kaxiln...@gmail.com> wrote: > > Absolutely, will drop you a separate email about it. > > As you may understand, Microsoft is a huge organization. In that context, >> we are from the Microsoft Fabric group, which is a different group and >> product from Azure. We are committed to supporting and adding dashboards >> for Microsoft Fabric providers. > > > > I think Dennis had a question about that too: Should it be a separate > provider or part of MS Azure Provider? > >> We are committed to supporting and adding dashboards for Microsoft Fabric >> providers. > > > > On Tue, 30 Jul 2024 at 08:20, Abhishek Narain < > narain.abhis...@microsoft.com> wrote: > >> Thanks, everyone, for the input. >> Kaxil, >> As you may understand, Microsoft is a huge organization. In that context, >> we are from the Microsoft Fabric group, which is a different group and >> product from Azure. We are committed to supporting and adding dashboards >> for Microsoft Fabric providers. >> Also, are sponsorship opportunities open for the Airflow Summit? Can we >> connect offline? >> >> Regards, >> >> >> *Abhishek Narain *Principal Program Manager >> Data Integration >> narain.abhis...@microsoft.com >> >> [image: Microsoft Logo] >> >> >> <https://outlook.office.com/bookwithme/user/1e04a093bb8944209a5749e9b82a9...@microsoft.com?anonymous&ep=signature> >> >> Book time to meet with me >> <https://outlook.office.com/bookwithme/user/1e04a093bb8944209a5749e9b82a9...@microsoft.com?anonymous&ep=signature> >> >> >> ------------------------------ >> *From:* Kaxil Naik <kaxiln...@gmail.com> >> *Sent:* Monday, July 29, 2024 5:13 AM >> *To:* dev@airflow.apache.org <dev@airflow.apache.org> >> *Cc:* ambikagarg1...@gmail.com <ambikagarg1...@gmail.com>; Abhishek >> Narain <narain.abhis...@microsoft.com>; Blain David < >> david.bl...@infrabel.be> >> *Subject:* Re: [EXTERNAL] Re: [DISCUSS] Add the Microsoft Fabric >> Provider in Apache Airflow >> >> You don't often get email from kaxiln...@gmail.com. Learn why this is >> important <https://aka.ms/LearnAboutSenderIdentification> >> Airflow Summit Sponsorship is indeed a great and simple way to showcase >> "contributing back to the community.", maybe Infrabel is also interested in >> Sponsoring. On the System tests and Dashboard front, I would like to see it >> cover all the operators available in the existing MS Providers—there are >> four providers: Microsoft Azure [1], MSSQL [2], WinRM [3], and PSRP [4]. >> >> In the absence of activity from Microsoft, Wei from Astronomer spent a >> good few months (with >50 PRs [5]) adding Managed identity support for all >> Azure integrations, which is a best practice [6]. So, from the community's >> POV, what would help is a commitment but actions similar to those of the >> managed Airflow service providers/partners. >> >> Some good examples of System Dashboards: >> * https://astronomer.github.io/llm-dags-dashboard >> * https://storage.googleapis.com/providers-dashboard-html/dashboard.html >> * https://aws-mwaa.github.io/open-source/system-tests/dashboard.html >> * https://teradata.github.io/airflow/ >> >> >> [1] >> https://airflow.apache.org/docs/apache-airflow-providers-microsoft-azure >> [2] >> https://airflow.apache.org/docs/apache-airflow-providers-microsoft-mssql >> [3] >> https://airflow.apache.org/docs/apache-airflow-providers-microsoft-winrm >> [4] >> https://airflow.apache.org/docs/apache-airflow-providers-microsoft-psrp >> [5] >> https://github.com/apache/airflow/pulls?page=1&q=is%3Apr+label%3Aprovider%3Amicrosoft-azure+author%3ALee-W+is%3Aclosed >> [6] >> https://learn.microsoft.com/en-us/entra/identity/managed-identities-azure-resources/overview >> >> >> On Mon, 29 Jul 2024 at 08:00, Bienvenu Joffrey < >> joffrey.bienv...@infrabel.be> wrote: >> >> Good morning, >> >> I want to mention Infrabel's commitment to the support of a Fabric >> Provider / Fabric Operators in a Microsoft Provider: >> >> Last year, we started to work internally on integrations for PowerBI. >> David Blain's PR about MsGraph Hook< >> https://github.com/apache/airflow/pull/38111> is part of that effort. >> When we saw the work of Ambika on Github, we jumped in to give her code >> review and usage feedback. We are interacting with PowerBI from our on-prem >> Airflow and, since a few weeks, via Microsoft managed version on Fabric. >> Thus these Operators / this Provider is important for us, and Infrabel will >> open PR for fix and features as long as we are using them. >> >> We can also help with the dashboard: We already have tests Dags for most >> of the Operators we use. Some of our teams monitor their Airflow via a >> PowerBI dashboard (direct query to Airflow's API). We are not authorised to >> host and publish a dashboard to the public ourselves. But we could send the >> status of our pipelines to a dashboard managed by Microsoft ? >> >> >> Joffrey Bienvenu >> >> >> >> ________________________________ >> From: Jarek Potiuk <ja...@potiuk.com> >> Sent: Saturday, July 27, 2024 23:48 >> To: Abhishek Narain <narain.abhis...@microsoft.com> >> Cc: dev@airflow.apache.org <dev@airflow.apache.org>; ambika garg < >> ambikagarg1...@gmail.com> >> Subject: Re: [EXTERNAL] Re: [DISCUSS] Add the Microsoft Fabric Provider >> in Apache Airflow >> >> EXTERNAL MAIL: Ne cliquez pas sur les liens ou n'ouvrez pas les pièces >> jointes si vous ne connaissez pas l'expéditeur et ne lui faites pas >> confiance. En cas de doute, envoyez cet email en pièce jointe à >> ab...@infrabel.be<mailto:ab...@infrabel.be>. >> >> So - plenty of ways to show that "you are serious about contributing >> back". >> The ball is on your court now. >> >> On Sat, Jul 27, 2024 at 11:43 PM Jarek Potiuk <ja...@potiuk.com> wrote: >> >> > And BTW. We are going to have a common talk by the stakeholders who have >> > their dashboards of ours at Airflow Summit >> https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fairflowsummit.org%2F&data=05%7C02%7Cjoffrey.bienvenu%40infrabel.be%7C6736591b4643427461b408dcae85fb9b%7Cb82bc314ab8e4d6fb18946f02e1f27f2%7C0%7C0%7C638577137785057712%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sdata=iMCLQfD1XLbDARi04PwSBsGlMSGk06Er00xD92RDnyg%3D&reserved=0 >> <https://airflowsummit.org/><https://airflowsummit.org/> in >> > September, explaining how having such a dashboard managed by the >> > stakeholders improves the quality of provider code. I guess it won't be >> > possible to get such a dashboard from Microsoft before, but at least >> > mentioning that Microsoft is working on a similar solution would be a >> good >> > point at the talk - it might give the users assurance that the >> stakeholders >> > are taking care of them. >> > >> > BTW. Airflow Summit also still looks for sponsors and while Amazon, >> Google >> > and Astronomer are "organizers" and Platinum sponsors of the Summit, the >> > other stakeholders are more than welcome to become sponsors as well. >> Would >> > be great to see Microsoft as a sponsor there. >> > >> > J. >> > >> > On Sat, Jul 27, 2024 at 11:35 PM Jarek Potiuk <ja...@potiuk.com> wrote: >> > >> >> Sure. We have everything documented, agreed and approved by voting of >> the >> >> community here. Generally I strongly advice subscribing to the devlist >> and >> >> following what's happening here. That is basically a prerequisite of >> "being >> >> part of the community". >> >> >> >> System tests are described here: >> >> >> >> >> >> >> https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fairflow%2Fblob%2Fmain%2Fcontributing-docs%2Ftesting%2Fsystem_tests.rst&data=05%7C02%7Cjoffrey.bienvenu%40infrabel.be%7C6736591b4643427461b408dcae85fb9b%7Cb82bc314ab8e4d6fb18946f02e1f27f2%7C0%7C0%7C638577137785066182%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sdata=sFf%2BHHiq07NSdJxTdAUyj9FoSMjhxHhRkfT2JkRGRv4%3D&reserved=0 >> <https://github.com/apache/airflow/blob/main/contributing-docs/testing/system_tests.rst> >> < >> https://github.com/apache/airflow/blob/main/contributing-docs/testing/system_tests.rst >> > >> >> >> >> They are the result of implementing the AIP-4 (Airflow Improvement >> >> Proposal) >> >> >> https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcwiki.apache.org%2Fconfluence%2Fpages%2Fviewpage.action%3FpageId%3D93324355&data=05%7C02%7Cjoffrey.bienvenu%40infrabel.be%7C6736591b4643427461b408dcae85fb9b%7Cb82bc314ab8e4d6fb18946f02e1f27f2%7C0%7C0%7C638577137785071902%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sdata=CIq7DPytqB0BXdoMkJfcF1kkM0cYvPZNysyfEfheKA8%3D&reserved=0 >> <https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=93324355> >> < >> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=93324355 >> >. >> >> - which had recently been completed with Google and Terraform >> publishing >> >> their dashboards (Amazon and Astronomer implemented them before) >> >> >> >> Everything about providers is described here: >> >> >> https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fairflow%2Fblob%2Fmain%2FPROVIDERS.rst&data=05%7C02%7Cjoffrey.bienvenu%40infrabel.be%7C6736591b4643427461b408dcae85fb9b%7Cb82bc314ab8e4d6fb18946f02e1f27f2%7C0%7C0%7C638577137785076121%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sdata=cBAImmGXX6T1liOdEfU9JhHKid4WrlXmpg6p7IuLRNw%3D&reserved=0 >> <https://github.com/apache/airflow/blob/main/PROVIDERS.rst>< >> https://github.com/apache/airflow/blob/main/PROVIDERS.rst> (+ links to >> >> follow) >> >> >> >> That includes "mixed governance" model I referred to earlier >> >> >> https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fairflow%2Fblob%2Fmain%2FPROVIDERS.rst%23mixed-governance-model-for-3rd-party-related-community-providers&data=05%7C02%7Cjoffrey.bienvenu%40infrabel.be%7C6736591b4643427461b408dcae85fb9b%7Cb82bc314ab8e4d6fb18946f02e1f27f2%7C0%7C0%7C638577137785080120%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sdata=nEWIlZn%2FtnYwEG8RM9aLuXCiH4cz%2FDhd9zMF5HhOHTY%3D&reserved=0 >> <https://github.com/apache/airflow/blob/main/PROVIDERS.rst#mixed-governance-model-for-3rd-party-related-community-providers> >> < >> https://github.com/apache/airflow/blob/main/PROVIDERS.rst#mixed-governance-model-for-3rd-party-related-community-providers >> > >> >> >> >> Also the available dashboards from Amazon, Google, Astronomer and >> >> Teradata are available here >> >> >> https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fairflow.apache.org%2Fecosystem%2F%23airflow-provider-system-test-dashboards&data=05%7C02%7Cjoffrey.bienvenu%40infrabel.be%7C6736591b4643427461b408dcae85fb9b%7Cb82bc314ab8e4d6fb18946f02e1f27f2%7C0%7C0%7C638577137785084320%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sdata=LHw%2FLl0jWuyCmad%2BSTvidS65tMuSE4WbeU%2Bh3gjM1HQ%3D&reserved=0 >> <https://airflow.apache.org/ecosystem/#airflow-provider-system-test-dashboards> >> < >> https://airflow.apache.org/ecosystem/#airflow-provider-system-test-dashboards >> > >> >> - we do not tell our stakeholders how to develop dashboards (because >> for >> >> each of them the dashboard can be implemented best by following their >> own >> >> "usual" ways. The expectation we have that the "system tests" (also >> known >> >> as example dags) are executed with the latest main against the "real" >> >> services and the result of this run is published so that the release >> >> manager releasing the providers can take a look before releasing the >> >> provider - maintaining, reacting to issues and fixing them is done >> >> within the community (by raising PRs and fixes) but should be driven >> by the >> >> stakeholder. >> >> >> >> There is also a discussion started by Dennis Ferruzzi from Amazon about >> >> exposing machine-readable way of exposing dashboard results so that >> they >> >> can be aggregated and used as a driver for our CI jobs >> >> >> https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.apache.org%2Fthread%2Frx59zjvf3m3nkhq44c93hpttjcn1s85s&data=05%7C02%7Cjoffrey.bienvenu%40infrabel.be%7C6736591b4643427461b408dcae85fb9b%7Cb82bc314ab8e4d6fb18946f02e1f27f2%7C0%7C0%7C638577137785088375%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sdata=QAtF3lhQq1w5FlZcNU0ki%2Fs0QVJqqfHqmShCXMO3njg%3D&reserved=0 >> <https://lists.apache.org/thread/rx59zjvf3m3nkhq44c93hpttjcn1s85s>< >> https://lists.apache.org/thread/rx59zjvf3m3nkhq44c93hpttjcn1s85s> - >> >> which might be acutally something we might seriously consider - >> especialy >> >> that we got another contribution from Amazon on our AWS CI >> infrastructure >> >> which we can utilise for this. This is one of the projects that a >> >> "newcomer" in this area (Microsoft I mean) might consider as a project >> to >> >> lead - it would be a great contribution to the whole community if >> standard >> >> for this reporting is proposed and shared between the stakeholders and >> we >> >> have one common way of reporting it - contributions and engineering and >> >> project management effort in this area are most welcome, that could be >> >> another way Microsoft could show that they indeed want to contribute >> back >> >> and engage with the community. >> >> >> >> J. >> >> >> >> >> >> >> >> >> >> >> >> >> >> On Sat, Jul 27, 2024 at 11:04 PM Abhishek Narain < >> >> narain.abhis...@microsoft.com> wrote: >> >> >> >>> Sorry for the lack of context. Could you share documentation around >> >>> setting up dashboards so that we can assess the work required and get >> back? >> >>> >> >>> Get Outlook for iOS < >> https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Faka.ms%2Fo0ukef&data=05%7C02%7Cjoffrey.bienvenu%40infrabel.be%7C6736591b4643427461b408dcae85fb9b%7Cb82bc314ab8e4d6fb18946f02e1f27f2%7C0%7C0%7C638577137785092558%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sdata=Ku%2BDWNmQ1iiCctgrkoxtlJwFFXO8%2BBjCClK0xQ06KZ4%3D&reserved=0 >> <https://aka.ms/o0ukef><https://aka.ms/o0ukef>> >> >>> ------------------------------ >> >>> *From:* Jarek Potiuk <ja...@potiuk.com> >> >>> *Sent:* Saturday, July 27, 2024 1:48 PM >> >>> *To:* dev@airflow.apache.org <dev@airflow.apache.org>; Abhishek >> Narain < >> >>> narain.abhis...@microsoft.com> >> >>> *Cc:* ambika garg <ambikagarg1...@gmail.com> >> >>> *Subject:* Re: [EXTERNAL] Re: [DISCUSS] Add the Microsoft Fabric >> >>> Provider in Apache Airflow >> >>> >> >>> You don't often get email from ja...@potiuk.com. Learn why this is >> >>> important <https://aka.ms/LearnAboutSenderIdentification> >> >>> Adding Abishek explicitly and repeating the message: >> >>> >> >>> Hello Abishek, >> >>> >> >>> Does it mean that you are going to build and expose the dashboard with >> >>> system tests running on Microsoft Azure provider - similarly as >> Amazon, >> >>> Astronomer, Google, Terraform did? >> >>> >> >>> I guess that request will have to wait for it to happen - as a way to >> >>> actually show that the "intention" meets the "actual work done". >> >>> >> >>> Do you agree it's a great prerequisite before adding a new provider? >> >>> (and BTW - the new provider should also be only included when the >> dashboard >> >>> and system tests for it is there). >> >>> >> >>> I think it's a pretty reasonable expectation and a very >> straightforward >> >>> way to "put your money where your mouth is". >> >>> >> >>> What do you think? >> >>> >> >>> J. >> >>> >> >>> On Sat, Jul 27, 2024 at 10:43 PM Jarek Potiuk <ja...@potiuk.com> >> wrote: >> >>> >> >>>> Hello Abishek, >> >>>> >> >>>> Does it mean that you are going to build and expose the dashboard >> with >> >>>> system tests running on Microsoft Azure provider - similarly as >> Amazon, >> >>>> Astronomer, Google, Terraform did? >> >>>> >> >>>> I guess that request will have to wait for it to happen - as a way to >> >>>> actually show that the "intention" meets the "actual work done". >> >>>> >> >>>> Do you agree it's a great prerequisite before adding a new provider? >> >>>> (and BTW - the new provider should also be only included when the >> dashboard >> >>>> and system tests for it is there). >> >>>> >> >>>> I think it's a pretty reasonable expectation and a very >> straightforward >> >>>> way to "put your money where your mouth is". >> >>>> >> >>>> What do you think? >> >>>> >> >>>> J. >> >>>> >> >>>> On Sat, Jul 27, 2024 at 2:23 PM Abhishek Narain >> >>>> <narain.abhis...@microsoft.com.invalid> wrote: >> >>>> >> >>>>> Thank you for your detailed response, Ambika. >> >>>>> >> >>>>> I would like to reaffirm Microsoft’s commitment to supporting the >> >>>>> Fabric Provider for Apache Airflow. Our team is already dedicated to >> >>>>> delivering a native Apache Airflow powered SaaS experience known as >> Data >> >>>>> Workflows in Microsoft Fabric >> >>>>> < >> https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fblog.fabric.microsoft.com%2Fen-us%2Fblog%2Fintroducing-data-workflows-in-microsoft-fabric%3Fft%3DAll&data=05%7C02%7Cjoffrey.bienvenu%40infrabel.be%7C6736591b4643427461b408dcae85fb9b%7Cb82bc314ab8e4d6fb18946f02e1f27f2%7C0%7C0%7C638577137785096539%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sdata=5tDz5GctWzkHMhugHR1TCXNmN12796i2Nu4%2Fh7iPOgI%3D&reserved=0 >> <https://blog.fabric.microsoft.com/en-us/blog/introducing-data-workflows-in-microsoft-fabric?ft=All> >> < >> https://blog.fabric.microsoft.com/en-us/blog/introducing-data-workflows-in-microsoft-fabric?ft=All >> >>, >> >>>>> which is currently in public preview. By adding a native provider, >> we will >> >>>>> not only be building this for the community but also using it >> natively to >> >>>>> better integrate Data Workflows to run other Fabric jobs. >> >>>>> >> >>>>> We are excited to collaborate with the community to ensure the >> success >> >>>>> and sustainability of the Microsoft Fabric provider. >> >>>>> >> >>>>> >> >>>>> Regards, >> >>>>> >> >>>>> *Abhishek Narain* >> >>>>> Principal Product Manager >> >>>>> Data Integration >> >>>>> narain.abhis...@microsoft.com >> >>>>> >> >>>>> >> >>>>> [image: Microsoft Logo] >> >>>>> >> >>>>> [image: Pz8hmF9cQjDg8MHAaiKxp0eiZJ4AAAAAElFTkSuQmCC] >> >>>>> < >> https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Foutlook.office.com%2Fbookwithme%2Fuser%2F1e04a093bb8944209a5749e9b82a96d1%40microsoft.com%3Fanonymous%26ep%3Dsignature&data=05%7C02%7Cjoffrey.bienvenu%40infrabel.be%7C6736591b4643427461b408dcae85fb9b%7Cb82bc314ab8e4d6fb18946f02e1f27f2%7C0%7C0%7C638577137785100628%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sdata=UH%2BXt579Ok9bW5cxhw1m1ZknK7yZavPX3xIXJ3TNOEw%3D&reserved=0 >> <https://outlook.office.com/bookwithme/user/1e04a093bb8944209a5749e9b82a9...@microsoft.com?anonymous&ep=signature> >> < >> https://outlook.office.com/bookwithme/user/1e04a093bb8944209a5749e9b82a9...@microsoft.com?anonymous&ep=signature >> >> >> >>>>> >> >>>>> Book time to meet with me >> >>>>> < >> https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Foutlook.office.com%2Fbookwithme%2Fuser%2F1e04a093bb8944209a5749e9b82a96d1%40microsoft.com%3Fanonymous%26ep%3Dsignature&data=05%7C02%7Cjoffrey.bienvenu%40infrabel.be%7C6736591b4643427461b408dcae85fb9b%7Cb82bc314ab8e4d6fb18946f02e1f27f2%7C0%7C0%7C638577137785104697%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sdata=FICaCVg%2Byv7iEv37YrPYh1u4yFbh4aJrqrtwAb33jQ0%3D&reserved=0 >> <https://outlook.office.com/bookwithme/user/1e04a093bb8944209a5749e9b82a9...@microsoft.com?anonymous&ep=signature> >> < >> https://outlook.office.com/bookwithme/user/1e04a093bb8944209a5749e9b82a9...@microsoft.com?anonymous&ep=signature >> >> >> >>>>> >> >>>>> >> >>>>> >> >>>>> ------------------------------ >> >>>>> *From:* ambika garg <ambikagarg1...@gmail.com> >> >>>>> *Sent:* Wednesday, July 24, 2024 8:21 PM >> >>>>> *To:* dev@airflow.apache.org <dev@airflow.apache.org>; Abhishek >> >>>>> Narain <narain.abhis...@microsoft.com> >> >>>>> *Subject:* [EXTERNAL] Re: [DISCUSS] Add the Microsoft Fabric >> Provider >> >>>>> in Apache Airflow >> >>>>> >> >>>>> Thank you for your thoughtful and detailed inquiry regarding the >> >>>>> maintainability and support for the proposed Microsoft Fabric >> provider for >> >>>>> Apache Airflow. I appreciate your commitment to transparency and >> >>>>> collaboration within the Apache Airflow community. Your concerns >> about >> >>>>> long-term support and the active involvement of stakeholders are >> entirely >> >>>>> valid. >> >>>>> >> >>>>> To address your questions directly: >> >>>>> >> >>>>> 1. *Affiliation with Microsoft:* I am working as a vendor at >> >>>>> Microsoft, and I am actively contributing to this proposal on >> behalf of the >> >>>>> company. I have also developed the private plugin for Microsoft >> Fabric >> >>>>> < >> https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fambika-garg%2Fapache-airflow-microsoft-fabric-plugin&data=05%7C02%7Cjoffrey.bienvenu%40infrabel.be%7C6736591b4643427461b408dcae85fb9b%7Cb82bc314ab8e4d6fb18946f02e1f27f2%7C0%7C0%7C638577137785108807%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sdata=Eye4Z%2FhPfBlhE5%2FeSrKprMaQ%2Fs2Vzkcuj8QJayX%2Bfow%3D&reserved=0 >> <https://github.com/ambika-garg/apache-airflow-microsoft-fabric-plugin>< >> https://github.com/ambika-garg/apache-airflow-microsoft-fabric-plugin>>, >> >>>>> that triggers run of on-demand fabric items. Previously, I have >> worked on Synapse >> >>>>> operator that triggers pipeline run >> >>>>> < >> https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fairflow%2Fpull%2F35091&data=05%7C02%7Cjoffrey.bienvenu%40infrabel.be%7C6736591b4643427461b408dcae85fb9b%7Cb82bc314ab8e4d6fb18946f02e1f27f2%7C0%7C0%7C638577137785112705%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sdata=wIzPjMbm2L0MSGKm9VUe3myxMNYUZk6xmfMP6UN%2ByS0%3D&reserved=0 >> <https://github.com/apache/airflow/pull/35091>< >> https://github.com/apache/airflow/pull/35091>> and my new PR that >> >>>>> integrates the Power BI operator for dataset refresh >> >>>>> < >> https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fairflow%2Fpull%2F40356&data=05%7C02%7Cjoffrey.bienvenu%40infrabel.be%7C6736591b4643427461b408dcae85fb9b%7Cb82bc314ab8e4d6fb18946f02e1f27f2%7C0%7C0%7C638577137785116543%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sdata=TMTzeF5TRbfbsVzmnwjpHmRR%2ByCYjTCR6cx%2FDtvhq4E%3D&reserved=0 >> <https://github.com/apache/airflow/pull/40356>< >> https://github.com/apache/airflow/pull/40356>> is in-progress. >> >>>>> However, I understand that the community's interest is not just in >> >>>>> individual contributors but in the sustained involvement and >> support from >> >>>>> the organization. >> >>>>> >> >>>>> 2. *Microsoft’s Commitment:* I can assure you that this proposal is >> >>>>> backed by a commitment from Microsoft to not only contribute the >> initial >> >>>>> code but also to engage actively in its long-term maintenance and >> >>>>> integration. >> >>>>> >> >>>>> To ensure the community has full visibility into Microsoft’s >> >>>>> commitment and future plans, I have reached out to my manager. I am >> tagging >> >>>>> @abnar...@microsoft.com <abnar...@microsoft.com> Abhishek Narain, >> who >> >>>>> can provide more detailed information about our roadmap, resources, >> and >> >>>>> long-term strategy for supporting this integration. >> >>>>> >> >>>>> Thank you again for your patience. We are eager to work alongside >> the >> >>>>> community to ensure the success and sustainability of the Microsoft >> Fabric >> >>>>> provider. >> >>>>> >> >>>>> Best regards, >> >>>>> Ambika Garg >> >>>>> >> >>>> >> >>
