We understand, Jarek. We’ll look forward to your notification about the next wave with Teradata provider’s release.
Also, we’ll make sure we plan our enhancements and releases based on the standard release process going forward. Thanks, Sunil K Mallam Staff Product Manager – 3rd Party Developer Tools [A black and orange logo Description automatically generated]<https://www.teradata.com/> 17095 Via Del Campo Ct. San Diego, CA 92127 teradata.com<https://www.teradata.com/> This e-mail is from Teradata Corporation and may contain information that is confidential or proprietary. If you are not the intended recipient, do not read, copy, or distribute the e-mail or any attachments. Instead, please notify the sender and delete the e-mail and any attachments. Thank you. Please consider the environment before printing. From: Jarek Potiuk <ja...@potiuk.com> Date: Wednesday, 21 February 2024 at 5:09 PM To: dev@airflow.apache.org <dev@airflow.apache.org> Cc: Tworkiewicz, Adam <adam.tworkiew...@teradata.com>, Mishra, Harishankar <harishankar.mis...@teradata.com>, Chinthanippu, Satish <satish.chinthani...@teradata.com>, Kaxil Naik <ka...@astronomer.io> Subject: Re: [EXTERNAL] Re: [VOTE] [RESULT] New Airflow Community Provider: Teradata > I’ve asked Elad about this and maybe I’ll ask here too, is there a possibility of an ad-hoc release this week? IMHO. Unless there is a good reason for it (for the community) then we do not do that. This is part of the price you pay for being a "community" provider, that it has to fit in the regular processes. There has to be a very good reason to engage the release manager, and PMC members and people who are testing, so we do not take it lightly. If you had your own provider released, you'd be free to release it as you want, but - unless there is a very good reason other than marketing and PR of it being available, it simply asks for more work for the community for no particular reasons. This is precisely what we want to avoid - that adding a new provider does not add unnecessary burden on maintainers and the community. This is why we insist on tests, integration tests, and following "standard release process" when it is possible and when there is a particular need. What's the specific reason you want to release it "faster" than usual? Do you think it is worth the extra effort needed? J. On Wed, Feb 21, 2024 at 12:10 PM K Mallam, Sunil <sunil.kmal...@teradata.com.invalid> wrote: > Thanks for your response, Jarek. We’ll keep a watch on the system tests > dashboard and make sure we run system tests for every release. > > > > *Jared: They are released roughly every-two-weeks. Currently we are in the > middle of RC2 from the previous wave, so the next wave for the Teradata > provider is ~ 2 weeks from now +/- a few days.* > > - This means Teradata provider release should be in the last week of > February, probably Feb 29 or Mar 01. Correct? > > > > I’ve asked Elad about this and maybe I’ll ask here too, is there a > possibility of an ad-hoc release this week? > > > > > > Regards, > > *Sunil K Mallam* > Staff Product Manager – 3rd Party Developer Tools > > [image: A black and orange logo Description automatically generated] > <https://www.teradata.com/> > > 17095 Via Del Campo Ct. > San Diego, CA 92127 > teradata.com <https://www.teradata.com/> > > This e-mail is from Teradata Corporation and may contain information that > is confidential or proprietary. If you are not the intended recipient, do > not read, copy, or distribute the e-mail or any attachments. Instead, > please notify the sender and delete the e-mail and any attachments. Thank > you. > > Please consider the environment before printing. > > > > *From: *Jarek Potiuk <ja...@potiuk.com> > *Date: *Friday, 16 February 2024 at 4:49 PM > *To: *dev@airflow.apache.org <dev@airflow.apache.org> > *Cc: *Tworkiewicz, Adam <adam.tworkiew...@teradata.com>, Mishra, > Harishankar <harishankar.mis...@teradata.com>, Chinthanippu, Satish > <satish.chinthani...@teradata.com>, Kaxil Naik <kaxiln...@gmail.com> > *Subject: *[EXTERNAL] Re: [VOTE] [RESULT] New Airflow Community Provider: > Teradata > > [CAUTION: External Email] > > > > > > > > Since our PR is merged and we’re awaiting the release, I wanted to get > > some clarification on the following topics – > > > > 1. Is this the link > > < > https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fairflow.apache.org%2Fdocs%2Fapache-airflow%2F2.8.1%2Frelease_notes.html%23airflow-2-8-1-2024-01-19&data=05%7C02%7CSunil.KMallam%40teradata.com%7Cda9b434e046746ad52f108dc32d1b1b2%7C9151cbaafc6b4f4889bb8c4a71982138%7C0%7C0%7C638441123413503151%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sdata=W6r6W4zndKXSBGqEV6TLUKpZli0dz8HSc%2FfqBlWIHzs%3D&reserved=0<https://airflow.apache.org/docs/apache-airflow/2.8.1/release_notes.html#airflow-2-8-1-2024-01-19> > <https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fairflow.apache.org%2Fdocs%2Fapache-airflow%2F2.8.1%2Frelease_notes.html%23airflow-2-8-1-2024-01-19&data=05%7C02%7CSunil.KMallam%40teradata.com%7Cda9b434e046746ad52f108dc32d1b1b2%7C9151cbaafc6b4f4889bb8c4a71982138%7C0%7C0%7C638441123413511768%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sdata=5x700tpN8XP%2FVvB%2F7RaGbSk%2F6tfjX7OYJuN%2BXI3nmsE%3D&reserved=0<https://airflow.apache.org/docs/apache-airflow/2.8.1/release_notes.html#airflow-2-8-1-2024-01-19>> > > > > we should be following for new Airflow versions? Or, is there > anything else > > we should look out for? > > > > > Providers are released independently from Airflow. They are released > roughly every-two-weeks. Currently we are in the middle of RC2 from the > previous wave, so the next wave for the Teradata provider is ~ 2 weeks from > now +/- a few days. The next Airflow major release (2.9) will also have > `[teradata]` as optional extra. We are unlikely to merge the Teradata PR to > 2.8 branch for that. That's about it. Providers are pretty much independent > from Airflow, we have a bunch of "popular" providers that are baked in the > reference image, together with Airflow, but - at least for now -Teradata is > not on the list. Maybe in the future people will start heavily use > Teradata provider and we decide that it should be installed in the image - > but this is on a case-by-case level. I think the last time we changed the > list was by adding Snowflake some 6 months ago or so. > > > > > 1. Would Teradata be displayed in the Providers Index after the > > release? > > > > > Yes. It should be (barring manual mistake), it's part of the release > process to update this page > https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fairflow.apache.org%2Fdocs%2F&data=05%7C02%7CSunil.KMallam%40teradata.com%7Cda9b434e046746ad52f108dc32d1b1b2%7C9151cbaafc6b4f4889bb8c4a71982138%7C0%7C0%7C638441123413522023%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sdata=ykXQ0qgkPFER7JefDIiMyo3WCr0PJfZVqYH5k90Mqec%3D&reserved=0<https://airflow.apache.org/docs/> > <https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fairflow.apache.org%2Fdocs%2F&data=05%7C02%7CSunil.KMallam%40teradata.com%7Cda9b434e046746ad52f108dc32d1b1b2%7C9151cbaafc6b4f4889bb8c4a71982138%7C0%7C0%7C638441123413528364%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sdata=zf%2FYzy5OhOMMN6DpnGIZRPlKpSu9CvaP9N%2FQ4ASuv9g%3D&reserved=0<https://airflow.apache.org/docs/>>. > when new > providers are released. It's a bit manual part so sometimes it's delayed a > bit - but my suggestion is you pay attention and verify it after we release > it. > > > > 1. Would team Airflow create the PyPI package for Teradata? Or, should > > we do it? > > > > > Release manager of Airflow (currently Elad, I sometimes replace Elad) will > do it. > > Your involvement is twofold: > > * as discussed before - ideally - and it's also something you should be > interested in - we should have system test dashboard run by you (you: > > https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fairflow.apache.org%2Fecosystem%2F%23airflow-provider-system-test-dashboards&data=05%7C02%7CSunil.KMallam%40teradata.com%7Cda9b434e046746ad52f108dc32d1b1b2%7C9151cbaafc6b4f4889bb8c4a71982138%7C0%7C0%7C638441123413534565%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sdata=JppGiHGbZQdLNlHAmYha9vmPO1wyrvUvA7sGmbhmxG4%3D&reserved=0)<https://airflow.apache.org/ecosystem/#airflow-provider-system-test-dashboards> > <https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fairflow.apache.org%2Fecosystem%2F%23airflow-provider-system-test-dashboards&data=05%7C02%7CSunil.KMallam%40teradata.com%7Cda9b434e046746ad52f108dc32d1b1b2%7C9151cbaafc6b4f4889bb8c4a71982138%7C0%7C0%7C638441123413540242%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sdata=lBmHZyDKqhRdUi3uiSYHD9OUYMmPXrNTrllfzN3vEL4%3D&reserved=0<https://airflow.apache.org/ecosystem/#airflow-provider-system-test-dashboards>> > where release manager and you could check the System Tests status for > Teradata provider run from latest main. That can give us confidence that > what we release passes not only unit or integration tests - but also works > with actual Teradata service. It's not a HARD requirement for first or > second release, but this one is great for you as well - as you can also be > confident that what is in Airflow's main continues to work - and it is part > of the mixed-governance/testing contribution done by 3rd-parties. > > * when we prepare a new wave of providers, we always announce release > candidattes are ready and ask people to take it for a spin and verify that > the providers they contributed to - work as expected. This includes RC > candidates published in PyPi (by the release manager) and it is accompanied > by a Github Issue where contributors add their "yep, tested, the changes I > worked on work fine" or "hey I found a bug, here is a fix, so we should > remove the provider from the wave and get RC2 with the fix > included".Example issue from 4 days ago here: > > https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fairflow%2Fissues%2F37358&data=05%7C02%7CSunil.KMallam%40teradata.com%7Cda9b434e046746ad52f108dc32d1b1b2%7C9151cbaafc6b4f4889bb8c4a71982138%7C0%7C0%7C638441123413545577%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sdata=KH4Ji0tilIX5ydlgSvE6hqgQMMOOc2Xep1T5RD7VyIA%3D&reserved=0<https://github.com/apache/airflow/issues/37358> > <https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fairflow%2Fissues%2F37358&data=05%7C02%7CSunil.KMallam%40teradata.com%7Cda9b434e046746ad52f108dc32d1b1b2%7C9151cbaafc6b4f4889bb8c4a71982138%7C0%7C0%7C638441123413550231%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sdata=eLid4D0Wf4m4YZ96OpTV74s%2BSqW%2FlAZRDBQlk8EVasg%3D&reserved=0<https://github.com/apache/airflow/issues/37358>> > So this is where your > involvement is needed - definitely now, wheen you contributed it, but also > later when new changes are merged - especially if future changes are done > by someone from your team - that someone will be usually tagged in the > Issue and will get notified, but also it's good if you watch the > announcements of our on devlist and run any extra checks you might want to > run (and look at your own dashboard - similarly to what Amazon team > currently does). That helps us to catch issues at RC level, before it gets > into hands of the users. I think with the "1st" release of provider, it > will not be on the list (this list is changes from the previous version) - > so just pay attention and test it wihen we release RC1. > > > J. > > > > > > > > > > Thanks, > > > > *SM* > > > > > > > > *From: *K Mallam, Sunil <sunil.kmal...@teradata.com> > > *Date: *Monday, 22 January 2024 at 10:16 PM > > *To: *dev@airflow.apache.org <dev@airflow.apache.org> > > *Cc: *Tworkiewicz, Adam <adam.tworkiew...@teradata.com>, Mishra, > > Harishankar <harishankar.mis...@teradata.com>, Chinthanippu, Satish > > <satish.chinthani...@teradata.com>, Kaxil Naik <kaxiln...@gmail.com> > > *Subject: *[VOTE] [RESULT] New Airflow Community Provider: Teradata > > > > Hi Airflow Community, > > > > > > > > We've sent a proposal for Teradata being Airflow's new community > provider. > > After 72 hours of voting, we have 9 positive votes (4 binding and 5 > > non-binding). Here's a link > > < > https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.apache.org%2Fthread%2Fl08gf31p89dk8v9md0cdoyqgcglfxwgw&data=05%7C02%7CSunil.KMallam%40teradata.com%7Cda9b434e046746ad52f108dc32d1b1b2%7C9151cbaafc6b4f4889bb8c4a71982138%7C0%7C0%7C638441123413554623%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sdata=mbn5HVHENKjGMAY7M0rXWknf5izOXah1GVwl4PasfJ4%3D&reserved=0<https://lists.apache.org/thread/l08gf31p89dk8v9md0cdoyqgcglfxwgw> > <https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.apache.org%2Fthread%2Fl08gf31p89dk8v9md0cdoyqgcglfxwgw&data=05%7C02%7CSunil.KMallam%40teradata.com%7Cda9b434e046746ad52f108dc32d1b1b2%7C9151cbaafc6b4f4889bb8c4a71982138%7C0%7C0%7C638441123413559026%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sdata=mXol0WTJVN1pHOLEciQN6l7plyvS6dT6zGME%2FeJNZyo%3D&reserved=0<https://lists.apache.org/thread/l08gf31p89dk8v9md0cdoyqgcglfxwgw>>> > to the > > voting thread. > > > > > > > > Please review our PR < > https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fairflow%2Fpull%2F36953&data=05%7C02%7CSunil.KMallam%40teradata.com%7Cda9b434e046746ad52f108dc32d1b1b2%7C9151cbaafc6b4f4889bb8c4a71982138%7C0%7C0%7C638441123413563299%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sdata=sI5gcEvqgCbhQJ0md849Vz68Z4ZAyxmZldH5PqiL4w4%3D&reserved=0<https://github.com/apache/airflow/pull/36953> > <https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fairflow%2Fpull%2F36953&data=05%7C02%7CSunil.KMallam%40teradata.com%7Cda9b434e046746ad52f108dc32d1b1b2%7C9151cbaafc6b4f4889bb8c4a71982138%7C0%7C0%7C638441123413567538%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sdata=oze%2Br%2BcWY9ZcbdqCCyitJOXWxed1urmAgZGvIp1MXQc%3D&reserved=0<https://github.com/apache/airflow/pull/36953>>> > and > > let us know if there's anything else you'd need from us. > > > > > > > > > > > > Thanks, > > > > *Sunil K Mallam* > > Staff Product Manager – 3rd Party Developer Tools > > > > [image: A black and orange logo Description automatically generated] > > > > 17095 Via Del Campo Ct. > > San Diego, CA 92127 > > teradata.com <https://www.teradata.com/> > > > > This e-mail is from Teradata Corporation and may contain information that > > is confidential or proprietary. If you are not the intended recipient, do > > not read, copy, or distribute the e-mail or any attachments. Instead, > > please notify the sender and delete the e-mail and any attachments. Thank > > you. > > > > Please consider the environment before printing. > > > > > > > > > > >
