shahar1 commented on code in PR #60896: URL: https://github.com/apache/airflow/pull/60896#discussion_r2715878341
########## providers/MANAGING_PROVIDERS_LIFECYCLE.rst: ########## @@ -18,6 +18,36 @@ **The outline for this document in GitHub is available at top-right corner button (with 3-dots and 3 lines).** +Provider Governance Overview +============================ + +Before diving into the technical details of creating and managing providers, please familiarize +yourself with the governance framework that applies to all community providers: + +* `Provider Governance Framework <https://github.com/apache/airflow/blob/main/PROVIDERS.rst#provider-governance-framework>`_ + +Key governance concepts: + +* **Lifecycle stages**: All providers move through Incubation → Production → Attic/Deprecation Review Comment: ```suggestion * **Lifecycle stages**: All providers move through Incubation and Production, and may enter Attic/Deprecation when applicable ``` In the original phrasing the "Attic/Deprecation" phase may sound mandatory. ########## PROVIDERS.rst: ########## @@ -88,49 +88,233 @@ Airflow main branch to being decommissioned and removed from the main branch in `Managing provider's lifecycle <https://github.com/apache/airflow/blob/main/providers/MANAGING_PROVIDERS_LIFECYCLE.rst>`_ +Provider Governance Framework +----------------------------- + +This section describes the governance framework for community providers, including lifecycle stages, +stewardship model, and quantitative health metrics. + +Airflow's success is built on its extensive ecosystem of community-supported integrations—over 1,600 +hooks, operators, and sensors released as part of 90+ provider packages. These integrations are critical +for "ubiquitous orchestration." This governance framework establishes a scalable method to grow the +number of integrations while ensuring quality. All governance remains with the Airflow PMC. + +Provider Stewardship Model +^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Each provider or integration component must have designated **stewards** responsible for ensuring +the health criteria described below are met: + +* **Minimum stewards**: At least two unique individuals must serve as stewards for each provider +* **Role definition**: Stewards are subject matter experts for the integration (service, language, + or skill). Stewardship is a responsibility, not an additional authority or privilege +* **Committer sponsorship**: Each steward must be sponsored by an existing Airflow Committer who + ensures stewards fulfill their responsibilities. The sponsor is responsible for guiding the stewards + through the process of maintaining the provider, following best practices including regular dependency + updates, issue resolution, and ensuring the provider meets the health metrics described below. The sponsor + is NOT responsible for the codebase itself, which remains the responsibility of the stewards. Review Comment: ``` is NOT responsible for the codebase itself, which remains the responsibility of the stewards. ``` I feel a bit uncomfortable about the wording - if a committer merges a PR to the codebase, they should have some responsibility to ensure that the PR is technically ok (e.g., CI passes succesfully, no potential security concerns, etc.) - even if a steward approves it. I'd adopt the relevant definitions from the i18n "[code owner](https://github.com/apache/airflow/tree/main/airflow-core/src/airflow/ui/public/i18n#42-code-owner)". ########## PROVIDERS.rst: ########## @@ -88,49 +88,233 @@ Airflow main branch to being decommissioned and removed from the main branch in `Managing provider's lifecycle <https://github.com/apache/airflow/blob/main/providers/MANAGING_PROVIDERS_LIFECYCLE.rst>`_ +Provider Governance Framework +----------------------------- + +This section describes the governance framework for community providers, including lifecycle stages, +stewardship model, and quantitative health metrics. + +Airflow's success is built on its extensive ecosystem of community-supported integrations—over 1,600 +hooks, operators, and sensors released as part of 90+ provider packages. These integrations are critical +for "ubiquitous orchestration." This governance framework establishes a scalable method to grow the +number of integrations while ensuring quality. All governance remains with the Airflow PMC. + +Provider Stewardship Model +^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Each provider or integration component must have designated **stewards** responsible for ensuring +the health criteria described below are met: + +* **Minimum stewards**: At least two unique individuals must serve as stewards for each provider +* **Role definition**: Stewards are subject matter experts for the integration (service, language, + or skill). Stewardship is a responsibility, not an additional authority or privilege Review Comment: nit: the comment in paranthesis looks a bit out of context ```suggestion * **Role definition**: Stewards are subject matter experts for the integration. Stewardship is a responsibility, not an additional authority or privilege ``` ########## PROVIDERS.rst: ########## @@ -88,49 +88,233 @@ Airflow main branch to being decommissioned and removed from the main branch in `Managing provider's lifecycle <https://github.com/apache/airflow/blob/main/providers/MANAGING_PROVIDERS_LIFECYCLE.rst>`_ +Provider Governance Framework +----------------------------- + +This section describes the governance framework for community providers, including lifecycle stages, +stewardship model, and quantitative health metrics. + +Airflow's success is built on its extensive ecosystem of community-supported integrations—over 1,600 +hooks, operators, and sensors released as part of 90+ provider packages. These integrations are critical +for "ubiquitous orchestration." This governance framework establishes a scalable method to grow the +number of integrations while ensuring quality. All governance remains with the Airflow PMC. + +Provider Stewardship Model +^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Each provider or integration component must have designated **stewards** responsible for ensuring +the health criteria described below are met: + +* **Minimum stewards**: At least two unique individuals must serve as stewards for each provider +* **Role definition**: Stewards are subject matter experts for the integration (service, language, + or skill). Stewardship is a responsibility, not an additional authority or privilege +* **Committer sponsorship**: Each steward must be sponsored by an existing Airflow Committer who + ensures stewards fulfill their responsibilities. The sponsor is responsible for guiding the stewards + through the process of maintaining the provider, following best practices including regular dependency + updates, issue resolution, and ensuring the provider meets the health metrics described below. The sponsor + is NOT responsible for the codebase itself, which remains the responsibility of the stewards. +* **Accountability**: Ultimate accountability remains with the Airflow PMC and Committers +* **Transitions**: Neither sponsorship nor stewardship are roles in perpetuity—they can be + transitioned to others based on mutual agreement and PMC approval + +.. note:: + + The quantitative criteria described below are aspirational. The PMC will revisit these metrics + based on actual experience 6 months after they are established and published. + +Provider Lifecycle Stages +^^^^^^^^^^^^^^^^^^^^^^^^^ + +All providers follow a three-stage lifecycle: **Incubation**, **Production**, and **Attic/Deprecation**. +Additionally, providers may be designated as **Mature** under specific circumstances. + +**Incubation Stage** + +All new providers or integration modules (such as Notifiers, Message components, etc.) should start +in incubation unless specifically accelerated by the PMC. Incubation ensures that code, licenses, and +processes align with ASF guidelines and community principles such as community, collaborative +development, and openness. + +Requirements for incubation: + +* Working codebase to bootstrap credibility and attract contributions +* Visibility in the provider health dashboard +* Designated stewards with committer sponsorship + +Quantitative graduation criteria: + +.. list-table:: + :header-rows: 1 + :widths: 40 60 + + * - Metric + - Threshold + * - PRs submitted + - Minimum of 10 PRs in the last 6 months + * - PR review time + - PRs reviewed within 14 days + * - Issues reported + - Minimum of 15 unique issues filed in the last 6 months + * - Contributors + - At least 3 unique individuals making contributions (code or documentation) in the last 6 months + * - Issue resolution rate + - At least 50% of reported issues closed within 90 days + * - Security/release issues + - All release and security related issues closed within 60 days + * - Governance participation + - Demonstrated participation in project governance channels including quarterly updates + * - Quality standards + - Meet quality criteria for code, documentation, and tests as listed in the Contributing Guide + +**Production Stage** + +All modules in production are expected to be well managed with prompt resolution of issues and +support for a consistent release cadence (at least monthly, but typically every 2 weeks when changes +exist). Production providers must: + +* Stay consistent with the main branch +* Pass tests for main + all supported Airflow versions +* Follow Airflow support guidelines, staying current with main Airflow releases + +Exceptions can be granted based on a PMC/devlist vote (PMC members only having binding votes) for +valid and one-off criteria. + +Quantitative criteria to maintain production status: + +.. list-table:: + :header-rows: 1 + :widths: 40 60 + + * - Metric + - Threshold + * - PRs submitted + - Minimum of 10 PRs in the last 6 months + * - PR review time + - PRs reviewed within 14 days + * - Issues reported + - Minimum of 20 unique issues filed in the last 6 months + * - Contributors + - At least 5 unique individuals making contributions (code or documentation) in the last 6 months + * - Issue resolution rate + - At least 60% of reported issues closed within 90 days + * - Security/release issues + - All release and security related issues closed within 30 days + * - Feature releases + - At least 1 feature release every 6 months + * - User engagement + - Maintain support activity with response to questions within 2 weeks on average + * - Governance participation + - Demonstrated participation in project governance channels including quarterly updates + +**Attic / Deprecation Stage** + +Modules should be moved into the Attic when relevance wanes, typically measured by declining +activity. This commonly occurs when the integrated solution has faded in popularity and is replaced +by more modern alternatives. + +Movement to the Attic must be communicated on the dev list and voted on by the PMC. Exceptions can +be granted based on the vote. + +Quantitative criteria triggering attic consideration: + +.. list-table:: + :header-rows: 1 + :widths: 40 60 + + * - Metric + - Threshold + * - PRs submitted + - Fewer than 5 PRs in the last 6 months + * - PR review time + - PRs not being reviewed in more than a month + * - Issues reported + - Fewer than 10 unique issues filed in the last 6 months + * - Contributors + - Fewer than 3 unique individuals making contributions (code or documentation) in the last 6 months + * - Issue resolution rate + - Less than 30% of reported issues closed within 90 days + * - Security/release issues + - Release and security related issues not getting closed within 30 days + * - Feature releases + - No feature releases in the last 6 months + +Consequences of attic status: + +* Modules remain readable but do not receive active maintenance +* After a period of at least 6 months in the attic, modules can be chosen for removal with Review Comment: I think that it would be great if this state can also be also reversible - ideally we won't get there at all, but recovering from attic should be easier for all sides rather than going through the incubation process all over again ########## PROVIDERS.rst: ########## @@ -88,49 +88,233 @@ Airflow main branch to being decommissioned and removed from the main branch in `Managing provider's lifecycle <https://github.com/apache/airflow/blob/main/providers/MANAGING_PROVIDERS_LIFECYCLE.rst>`_ +Provider Governance Framework +----------------------------- + +This section describes the governance framework for community providers, including lifecycle stages, +stewardship model, and quantitative health metrics. + +Airflow's success is built on its extensive ecosystem of community-supported integrations—over 1,600 +hooks, operators, and sensors released as part of 90+ provider packages. These integrations are critical +for "ubiquitous orchestration." This governance framework establishes a scalable method to grow the +number of integrations while ensuring quality. All governance remains with the Airflow PMC. + +Provider Stewardship Model +^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Each provider or integration component must have designated **stewards** responsible for ensuring Review Comment: How is it going to work for existing providers? Are we going to nominate stewards ad-hoc, or is it going to be applied only for new providers? If it's only for new providers, it's better to mention that it applies only for providers created after the approval of AIP-95 on Dec. 10, 2025. Otherwise, we might need a further discussion of how existing providers will adapt - because it's not that clear from the AIP. ########## PROVIDERS.rst: ########## @@ -88,49 +88,233 @@ Airflow main branch to being decommissioned and removed from the main branch in `Managing provider's lifecycle <https://github.com/apache/airflow/blob/main/providers/MANAGING_PROVIDERS_LIFECYCLE.rst>`_ +Provider Governance Framework +----------------------------- + +This section describes the governance framework for community providers, including lifecycle stages, +stewardship model, and quantitative health metrics. + +Airflow's success is built on its extensive ecosystem of community-supported integrations—over 1,600 +hooks, operators, and sensors released as part of 90+ provider packages. These integrations are critical +for "ubiquitous orchestration." This governance framework establishes a scalable method to grow the +number of integrations while ensuring quality. All governance remains with the Airflow PMC. + +Provider Stewardship Model +^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Each provider or integration component must have designated **stewards** responsible for ensuring +the health criteria described below are met: + +* **Minimum stewards**: At least two unique individuals must serve as stewards for each provider +* **Role definition**: Stewards are subject matter experts for the integration (service, language, + or skill). Stewardship is a responsibility, not an additional authority or privilege +* **Committer sponsorship**: Each steward must be sponsored by an existing Airflow Committer who + ensures stewards fulfill their responsibilities. The sponsor is responsible for guiding the stewards + through the process of maintaining the provider, following best practices including regular dependency + updates, issue resolution, and ensuring the provider meets the health metrics described below. The sponsor + is NOT responsible for the codebase itself, which remains the responsibility of the stewards. +* **Accountability**: Ultimate accountability remains with the Airflow PMC and Committers Review Comment: This sentence is quite vague -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
