[like] K Mallam, Sunil reacted to your message: ________________________________ From: Jarek Potiuk <ja...@potiuk.com> Sent: Friday, December 22, 2023 7:46:47 PM To: dev@airflow.apache.org <dev@airflow.apache.org> Cc: Tworkiewicz, Adam <adam.tworkiew...@teradata.com>; Mishra, Harishankar <harishankar.mis...@teradata.com>; Chinthanippu, Satish <satish.chinthani...@teradata.com> Subject: [EXTERNAL] Re: [Discuss] New Airflow Community Provider: Teradata
[CAUTION: External Email] Thanks a lot Satish, Mishra, Adam for all the work put in this. I clearly see that you took seriously our expectations for a new provider we set. By a (quick) look at the arguments, code, dependencies, code quality, complexity, and system test dashboard (yay!) I personally think it's quite a good idea to accept it :). Of course it would have to go through a detailed review (and for example one comment I have is that it should like changing `apache-airflow>=2.6.0` not `apache-airflow>=2.8.0` as it is currently (I see no particular reason why we should limit it to 2.8.0+) . I think Teradata is popular enough and has a lot of customers, and I think it would be good to have a community-managed provider - especially that you commit to maintaining it as well and you have a roadmap as well. I really appreciate the effort you - clearly - put in all that. But I would love to hear what others think as well. J. On Fri, Dec 22, 2023 at 8:27 PM K Mallam, Sunil <sunil.kmal...@teradata.com.invalid> wrote: > Hi, > > > > I’m Sunil Mallam, a Staff Product Manager with Teradata, owning the > Third-Party Connectors and Integrations. > > > > Here’s our proposal to be a Community Provider, as we see Airflow being a > key integration for Teradata’s cloud customers in terms of *orchestrating > data pipelines and ML workflows *- > > > > *Why Teradata wants to be a community provider?* > > - Teradata has over 700 customers (most of them are from Fortune 1000 > list) who run enterprise workloads. > - Being on *Airflow’s documentation index provides* *better visibility > for our user*s and they can easily find Teradata’s provider package, > without Teradata having to put additional effort into promoting it. > - Also, our customers have confidence in the quality and stability of > the provider package if it’s managed and maintained by Teradata, while > being validated/approved by Airflow. > > > > *How is it a positive for the Airflow project/community and why should it > be accepted?* > > - We have a dedicated team to manage and maintain the provider package. > - Teradata will take responsibility for - > - end-to-end testing efforts, running system tests periodically in > their environment, and making the status available to Airflow. > - being available and responsive to any communication from the > community members. > - To support our dedication toward contributions, Teradata is a verified > adapter with dbt > > <https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdocs.getdbt.com%2Fdocs%2Fsupported-data-platforms&data=05%7C02%7CSunil.KMallam%40teradata.com%7C4dacea3216cf414eef4d08dc0326ccda%7C9151cbaafc6b4f4889bb8c4a71982138%7C0%7C0%7C638388712405325118%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=mc7Nk9WDRBxduhgpF1owNti1mL8MG%2FNBSEi1BbW2kWM%3D&reserved=0<https://docs.getdbt.com/docs/supported-data-platforms>> > and we’re > diligent in matching up to its releases (major and minor). > > > > *What is developed as part of the initial implementation (phase 1) and > what’s next (phase 2)?* > > *Phase 1 * > > 1. Building a Hook, Base Operator, and Transfer Operator (Teradata to > Teradata). > 2. Creating DAGs for a basic use case (data movement). > 3. Creating Tests and match Test Coverage. > 4. Create supporting documentation for users. > 5. Create a dashboard with system tests status history. > > > > *Phase 2* > > 1. Building a Transfer Operator (Cloud to Teradata and Teradata to > Cloud). > 2. Add SSL Support > 3. Enhancements to match Airflow releases. > 4. Enhancements to support Teradata VantageCloud use production use > cases. > 5. Support specific production use cases for Teradata Customers. > > > > *Initial Implementation Links* > > Implementation: > https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FTeradata%2Fairflow%2Ftree%2Ftd_develop%2Fairflow%2Fproviders%2Fteradata&data=05%7C02%7CSunil.KMallam%40teradata.com%7C4dacea3216cf414eef4d08dc0326ccda%7C9151cbaafc6b4f4889bb8c4a71982138%7C0%7C0%7C638388712405481490%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=kTYJsI%2F0qDlZHmyeBEqSiIGkSn4FPgzJfrX3kK0Ew58%3D&reserved=0<https://github.com/Teradata/airflow/tree/td_develop/airflow/providers/teradata> > > Documentation: > https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FTeradata%2Fairflow%2Ftree%2Ftd_develop%2Fdocs%2Fapache-airflow-providers-teradata&data=05%7C02%7CSunil.KMallam%40teradata.com%7C4dacea3216cf414eef4d08dc0326ccda%7C9151cbaafc6b4f4889bb8c4a71982138%7C0%7C0%7C638388712405481490%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=stCZRtYf9MyUC8G%2FSkysWvANcFKs011%2FN9FUqvbm0vs%3D&reserved=0<https://github.com/Teradata/airflow/tree/td_develop/docs/apache-airflow-providers-teradata> > > Unit Tests: > https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FTeradata%2Fairflow%2Ftree%2Ftd_develop%2Ftests%2Fproviders%2Fteradata&data=05%7C02%7CSunil.KMallam%40teradata.com%7C4dacea3216cf414eef4d08dc0326ccda%7C9151cbaafc6b4f4889bb8c4a71982138%7C0%7C0%7C638388712405481490%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=QlkB528W6gsUYYECK9h4wPptFFvYa4In9sYfN4utyKg%3D&reserved=0<https://github.com/Teradata/airflow/tree/td_develop/tests/providers/teradata> > > System Tests: > https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FTeradata%2Fairflow%2Ftree%2Ftd_develop%2Ftests%2Fsystem%2Fproviders%2Fteradata&data=05%7C02%7CSunil.KMallam%40teradata.com%7C4dacea3216cf414eef4d08dc0326ccda%7C9151cbaafc6b4f4889bb8c4a71982138%7C0%7C0%7C638388712405481490%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=D1gwjbU%2BPnFH5m8ViJu8x4xnVQYyEwYIk45CDnULGvA%3D&reserved=0<https://github.com/Teradata/airflow/tree/td_develop/tests/system/providers/teradata> > > System Tests Dashboard: > https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fteradata.github.io%2Fairflow%2F&data=05%7C02%7CSunil.KMallam%40teradata.com%7C4dacea3216cf414eef4d08dc0326ccda%7C9151cbaafc6b4f4889bb8c4a71982138%7C0%7C0%7C638388712405481490%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=mRdmzkMu3Igbrqa%2BvutQY2XGdQF9QBrkaKvWdpdgemQ%3D&reserved=0<https://teradata.github.io/airflow/> > > > > > > Thanks, > > *Sunil K Mallam* > Staff Product Manager – 3rd Party Developer Tools > > [image: A black and orange logo Description automatically generated] > <https://www.teradata.com/> > > 17095 Via Del Campo Ct. > San Diego, CA 92127 > teradata.com <https://www.teradata.com/> > > This e-mail is from Teradata Corporation and may contain information that > is confidential or proprietary. If you are not the intended recipient, do > not read, copy, or distribute the e-mail or any attachments. Instead, > please notify the sender and delete the e-mail and any attachments. Thank > you. > > Please consider the environment before printing. > > > > > > >
