kaxil opened a new issue, #57018:
URL: https://github.com/apache/airflow/issues/57018
### Body
This issue tracks the migration of Airflow providers to use the centralized
compatibility layer in `common.compat` provider. The goal is to replace
version-specific conditional imports scattered across providers with a
standardized approach that works seamlessly across Airflow 2.x and 3.x.
The `common.compat` layer provides lazy imports that abstract away version
differences, eliminating the need for conditional logic like `if
AIRFLOW_V_3_0_PLUS:` in provider code. This improves maintainability and
reduces code duplication.
**Foundational Work**: See PR #56884 for the architectural changes that
established this pattern.
## Example PRs
Review these PRs to understand the migration pattern:
- **Simple provider**: #57007 - Migrate asana provider to `common.compat`
- **Provider with hooks/operators**: #56996 - Migrate airbyte provider to
`common.compat`
- **Provider with triggers**: #56999 - Migrate dbt.cloud provider to
`common.compat`
## Migration Steps
1. Add `apache-airflow-providers-common-compat>=1.7.4` to the provider's
`dependencies` array in `pyproject.toml`
- Also add it to the `dev` dependency-group
- Include comment: `# + TODO: bump to next version`
2. Replace imports from local `version_compat.py` with `from
airflow.providers.common.compat.sdk import ...`
- Common imports: `BaseOperator`, `BaseHook`, `Context`, `task`, etc.
3. Update test files similarly
4. Simplify or remove `version_compat.py`:
- Keep only version check constants (`AIRFLOW_V_3_0_PLUS`, etc.) if still
needed
- Remove conditional import logic that's now handled by `common.compat`
5. Run verification:
```bash
prek mypy-providers --directory providers/<your-provider>
```
6. Run provider tests to ensure everything works
## Current Status (Updated: 2025-10-22)
**Total providers: 97**
**Already Migrated ✅ (34 providers)**
- apache/beam
- apache/hive
- apache/livy
- apache/spark
- airbyte
- amazon
- apprise
- asana
- atlassian/jira
- common/compat
- common/io
- databricks
- dbt/cloud
- discord
- docker
- edge3
- fab
- ftp
- google
- microsoft/azure
- openlineage
- opsgenie
- pagerduty
- papermill
- sftp
- slack
- smtp
- snowflake
- standard
- tableau
- telegram
- teradata
- trino
- yandex
**Remaining to Migrate 📋 (45 providers)**
### apache/
- [ ] apache/cassandra (PR #57016 open)
- [ ] apache/drill (PR #57016 open)
- [ ] apache/druid
- [ ] apache/flink (PR #57016 open)
- [ ] apache/hdfs (PR #57016 open)
- [ ] apache/iceberg (PR #57016 open)
- [ ] apache/impala (PR #57016 open)
- [ ] apache/kafka (PR #57016 open)
- [ ] apache/kylin
- [ ] apache/pig (PR #57016 open)
- [ ] apache/pinot (PR #57016 open)
- [ ] apache/tinkerpop (PR #57016 open)
### common/
- [ ] common/messaging
- [ ] common/sql
### microsoft/
- [ ] microsoft/mssql
- [ ] microsoft/psrp (PR #56995 open)
- [ ] microsoft/winrm (PR #56995 open)
### Other Providers (A-L)
- [ ] alibaba (PR #57000 open)
- [ ] arangodb (PR #57001 open)
- [ ] celery
- [ ] cloudant
- [ ] cncf/kubernetes
- [ ] cohere
- [ ] datadog
- [ ] dingding
- [ ] elasticsearch (PR #57016 open)
- [ ] exasol
- [ ] facebook
- [ ] git (PR #57008 open)
- [ ] github (PR #57009 open)
- [ ] grpc
- [ ] hashicorp (PR #57015 open)
- [ ] http
- [ ] imap
- [ ] influxdb
- [ ] jdbc (PR #57012 open)
- [ ] jenkins (PR #57012 open)
- [ ] keycloak
### Other Providers (M-Z)
- [ ] mongo
- [ ] mysql
- [ ] neo4j
- [ ] odbc
- [ ] openai
- [ ] openfaas
- [ ] opensearch
- [ ] oracle
- [ ] pgvector
- [ ] pinecone
- [ ] postgres
- [ ] presto
- [ ] qdrant
- [ ] redis (PR #57006 open)
- [ ] salesforce
- [ ] samba
- [ ] segment
- [ ] sendgrid
- [ ] singularity
- [ ] sqlite
- [ ] ssh (PR #57004 open)
- [ ] vertica
- [ ] weaviate
- [ ] ydb
- [ ] zendesk (PR #56997 open)
## How to Contribute
1. **Pick a provider**: Choose an unchecked provider from the list above
that doesn't have an open PR
2. **Review examples**: Look at PRs #57007, #56996, or #56999 to understand
the pattern
3. **Make changes**: Follow the migration steps outlined above
4. **Verify**: Run `prek mypy-providers --directory
providers/<your-provider>` and provider tests
5. **Submit PR**:
- Use title format: "Migrate X provider to `common.compat`"
- Reference this issue in your PR description
- Check off your provider in this issue when you open the PR
## Guidelines
- Focus on one provider at a time to avoid merge conflicts
- Follow the patterns established in example PRs - consistency is key
- Ensure all tests pass before submitting your PR
- If you encounter a provider-specific edge case, ask for guidance in this
issue
- Update this issue by checking off your provider when you open a PR
- Use the PR title format: "Migrate X provider to `common.compat`" for easy
tracking
## Notes
- The `common.compat` layer provides version-agnostic imports that work
across Airflow 2.x and 3.x
- This eliminates the need for conditional imports based on
`AIRFLOW_V_3_0_PLUS` and similar version checks
- See PR #56884 for the foundational architectural work
- Some providers may not need migration if they don't use version-specific
imports
---
Let's work together to improve Airflow's maintainability and reduce
technical debt! 🚀
**Note**: Provider counts may change as PRs are merged.
### Committer
- [x] I acknowledge that I am a maintainer/committer of the Apache Airflow
project.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
