+1 as well, don't forget about the async counterparts as well in the hook ;-)
General (Internal Property) ________________________________ From: Parag Ekbote <[email protected]> Sent: Tuesday, November 3, 2026 02:12 To: [email protected] <[email protected]> Subject: [DISCUSS] New Community Provider: Hugging Face [You don't often get email from [email protected]. Learn why this is important at https://aka.ms/LearnAboutSenderIdentification ] EXTERNAL MAIL: Indien je de afzender van deze e-mail niet kent en deze niet vertrouwt, klik niet op een link of open geen bijlages. Bij twijfel, stuur deze e-mail als bijlage naar [email protected]<mailto:[email protected]>. Hi Airflow community, I'd like to propose introducing a new community provider for Hugging Face. Integration overview Hugging Face <https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fhuggingface.co%2F&data=05%7C02%7Cdavid.blain%40infrabel.be%7Cd58f4ed265d74587f72b08df39479e2f%7Cb82bc314ab8e4d6fb18946f02e1f27f2%7C0%7C0%7C639292651896862729%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=Ict4zTMBmiCURE63cK9QHoTYw185JlnNyRkZIqCAOkU%3D&reserved=0<https://huggingface.co/>> is an open-source AI platform and community that provides tools and infrastructure for building, sharing and deploying machine learning models, datasets and applications. Hugging Face Datasets is widely used in data engineering and machine learning workflows, and a first-class Airflow provider would make dataset orchestration, transformation, and publishing simpler. The provider initially focuses on Hugging Face Datasets, exposing a HuggingFaceHook and native operators for loading datasets with revision pinning, applying user-defined transformations and publishing datasets to the Hugging Face Hub. Future iterations may extend the provider to Hugging Face Models and Inference Endpoints. System tests The provider integrates with the live Hugging Face Hub. System tests are included in the PR and exercise dataset loading, transformation and publishing workflows using the airflow-hf <https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fhuggingface.co%2Fairflow-hf&data=05%7C02%7Cdavid.blain%40infrabel.be%7Cd58f4ed265d74587f72b08df39479e2f%7Cb82bc314ab8e4d6fb18946f02e1f27f2%7C0%7C0%7C639292651896876633%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=UjmjHX5ugIgjLwZJXRq4kBxoHLbWr2%2BVTPMJyBuWvIc%3D&reserved=0<https://huggingface.co/airflow-hf>> Hugging Face organization. Proposed stewards Parag Ekbote, @ParagEkbote <https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FParagEkbote&data=05%7C02%7Cdavid.blain%40infrabel.be%7Cd58f4ed265d74587f72b08df39479e2f%7Cb82bc314ab8e4d6fb18946f02e1f27f2%7C0%7C0%7C639292651896883162%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=a%2F9QlVqphMckSJyeBKyhkJfNyHTfIXct6z9zAyu5sWo%3D&reserved=0<https://github.com/ParagEkbote>>, < [email protected]> Quentin Lhoest, @lhoestq <https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Flhoestq&data=05%7C02%7Cdavid.blain%40infrabel.be%7Cd58f4ed265d74587f72b08df39479e2f%7Cb82bc314ab8e4d6fb18946f02e1f27f2%7C0%7C0%7C639292651896889012%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=pyCnGTAr0x4%2FD3F7H7Utnm08g%2B1IRxAN0s29n2bMgOM%3D&reserved=0<https://github.com/lhoestq>>, < [email protected]> Sponsoring committer(s) Kaxil Naik, @kaxil <https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fkaxil&data=05%7C02%7Cdavid.blain%40infrabel.be%7Cd58f4ed265d74587f72b08df39479e2f%7Cb82bc314ab8e4d6fb18946f02e1f27f2%7C0%7C0%7C639292651896894587%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=g3fovALx3UMCnlhP42ZBHPcQz1nZn7gcAsLWcNCIx3o%3D&reserved=0<https://github.com/kaxil>>, <[email protected]> Working implementation <https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fairflow%2Fpull%2F76317&data=05%7C02%7Cdavid.blain%40infrabel.be%7Cd58f4ed265d74587f72b08df39479e2f%7Cb82bc314ab8e4d6fb18946f02e1f27f2%7C0%7C0%7C639292651896900255%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=rUt3Z2gABvZjlcfaM9NInLl4PPUF6C4WslBLzykoVro%3D&reserved=0<https://github.com/apache/airflow/pull/76317>> Incubation commitment We commit to: - Maintaining the provider and responding to issues within a reasonable time. - Meeting the incubation health metrics within six months. - Participating in quarterly governance updates. Best regards, Parag Ekbote
