+1 love it > On Nov 17, 2026, at 12:03 PM, Vikram Koka via dev <[email protected]> > wrote: > > +1 > Seems like a great addition > > Vikram > > >> On Tue, Nov 17, 2026 at 4:18 AM Kaxil Naik <[email protected]> wrote: >> >> +1 Would love to have polaris support. >> >> >> On Tue, 17 Nov 2026 at 04:18, Xiaodong Deng via dev < >> [email protected]> >> wrote: >> >>> Hi Airflow community, >>> >>> We'd like to propose adding a new community provider for Apache Polaris: >>> apache-airflow-providers-apache-polaris. >>> >>> Integration overview >>> --------------------------- >>> Apache Polaris (https://polaris.apache.org/) is an open-source catalog >> for >>> Apache Iceberg. It implements the Iceberg REST catalog API and adds >>> centralized access control, credential vending and catalog-level >> policies. >>> Many teams running Iceberg already orchestrate their data platform with >>> Airflow. Today, managing Polaris from a Dag means hand-written REST calls >>> or shelling out to the Polaris CLI. >>> >>> The proposed provider would add: >>> >>> - A "polaris" connection type. It supports Polaris's internal OAuth >> tokens, >>> and it masks every token it fetches. >>> - Hooks for the Polaris Management API and the catalog APIs. They can >> also >>> hand ready-made catalog properties to pyiceberg or to Spark and other >>> engines. >>> - Operators for catalogs, namespaces, tables, principals, principal and >>> catalog roles, grants, and policies. Typical uses include provisioning a >>> team's catalog with its roles and grants, onboarding a pipeline >> principal, >>> and managing table lifecycle. >>> - A helper that turns Polaris table-maintenance policies (compaction, >>> snapshot expiry, orphan-file removal) into Iceberg Spark procedure calls, >>> so a Dag can execute the maintenance a policy defines. >>> >>> It is built on the official Apache Polaris Python SDK >>> (apache-polaris>=1.9.0). It needs Airflow 3.1.1+ and Python 3.11+, and >>> pyiceberg is an optional extra. It has been tested against Polaris server >>> 1.8 and 1.9, and installs cleanly against Airflow's constraints for every >>> 3.1.1–3.4.0 release. >>> >>> System tests >>> ----------------- >>> Polaris is open source and runs as a single container, so no external >> paid >>> service is involved. The PR adds: >>> >>> - a Breeze integration (breeze --integration polaris) that starts Polaris >>> 1.9.0 in CI, with integration tests against it; >>> - system-test example Dags for provisioning, tables and policies, which >> run >>> against the same server. >>> >>> All tests can run in Airflow's own CI, so no external dashboard is >> needed. >>> >>> Proposed stewards >>> ------------------------- >>> - Yufei Gu, "flyrain", [email protected] (Apache Polaris PMC/Committer) >>> - Yong Zheng, "MonkeyCanCode", [email protected] (Apache Polaris >>> PMC/Committer) >>> - Xiaodong Deng, "XD-DENG", [email protected] (Apache Airflow >>> PMC/Committer) >>> >>> Sponsoring committer >>> ----------------------------- >>> - Xiaodong Deng, "XD-DENG", [email protected] >>> >>> Working implementation >>> -------------------------------- >>> Draft PR: https://github.com/apache/airflow/pull/77188. It includes >> unit, >>> integration and system tests, and how-to docs for every operator. >>> >>> Incubation commitment >>> ------------------------------- >>> We commit to: >>> - maintaining the provider and responding to issues within a reasonable >>> time; >>> - meeting the incubation health metrics within 6 months; >>> - participating in quarterly governance updates. >>> >>> >>> Regards, >>> >>> XD >>> >>
--------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
