klesh opened a new issue, #9092: URL: https://github.com/apache/devlake/issues/9092
## Proposal: Deprecate the Python plugin framework and consolidate on Go ### Summary Propose to deprecate the Python plugin framework (`backend/python/`, a.k.a. PyDevLake) and remove it from the codebase after a 3-month deprecation window. Going forward, all plugins should be written in Go. This reduces maintenance overhead, simplifies the Docker image and CI, and focuses contributor effort on a single, well-supported runtime. ### Background The Python plugin framework was introduced in Feb 2023 (#3702, #4438) to lower the barrier for writing plugins. In practice it never saw adoption: - Only **one** plugin ever shipped on it: `azuredevops` (Python). - The `dbt` Python plugin was already removed on 2026-08-31 (#8970). - A fully-featured Go implementation, `azuredevops_go`, now exists and is actively developed (on-premises support landed 2026-08-16, #9014). ### Evidence: Python plugins are not being maintained | Plugin | Last meaningful commit | Note | | --- | --- | --- | | `azuredevops` (Python) | 2025-12-31 `feat: add environment_pattern` (#8671) | No feature/fix since; only deps refresh & runtime bumps | | `pydevlake` framework | 2025-12-31 (same as above) | Last real fix: #8957 on 2026-07-15 (missing composite PKs) | | `dbt` (Python) | removed 2026-08-31 (#8970) | Already gone | In the last ~12 months, the only commits touching `backend/python/` are dependency refreshes, Poetry/uv lock bumps, a Python 3.9→3.11 runtime bump, and a single primary-key fix. Meanwhile the Go side shipped 40+ plugins and continues to receive features weekly. ### Benefits 1. **One runtime to maintain.** Eliminate the parallel PyDevLake framework, its RPC bridge, `pydevlake` package, `uv`/Poetry tooling, and Python-specific test/build scripts. 2. **Simpler Docker build.** Today `backend/Dockerfile` is based on `python:3.11-slim-bookworm`, installs python3-dev/pip/uv, copies `python/`, and runs `build.sh`. After removal the image can be a slim Go-based image — smaller, fewer CVEs, faster builds. `Dockerfile.local`'s "empty python plugins dir" workaround also goes away. 3. **Simpler CI.** Two workflows (`.github/workflows/test.yml`, `test-e2e.yml`) and Make targets (`python-dep`, `python-unit-test`, `build-python`, `build-pydevlake`) can be dropped. 4. **Clearer contributor story.** New plugin authors get one path (Go) with 40+ reference implementations instead of choosing between two frameworks where one is unmaintained. 5. **No functional loss.** The only remaining Python plugin (`azuredevops`) is superseded by `azuredevops_go`, which already has broader feature coverage (Server/on-prem support). ### Plan & Timeline | Phase | Window | Actions | | --- | --- | --- | | **1. Warning** | T+0 (on issue merge) | Add deprecation notices to: `backend/python/README.md`, plugin `README.md`s, server startup log line, and `config-ui` plugin selection UI. Mark `azuredevops` (Python) as deprecated in the plugin registry; point users to `azuredevops_go`. | | **2. Freeze** | T+1 month | Stop accepting new features/fixes for Python plugins. Only critical security fixes land. Add a CI gate rejecting new files under `backend/python/plugins/`. | | **3. Migration assist** | T+1~T+3 months | Ship a migration guide + script to move any remaining `azuredevops` (Python) connections/scopes to `azuredevops_go`. Document data-layer equivalence. | | **4. Removal** | T+3 months | Delete `backend/python/`, PyDevLake, the Python Dockerfile stage, `make python-*` targets, and Python CI jobs. Remove `azuredevops` (Python) from the plugin registry. | Total deprecation window: **3 months** from the date this proposal is accepted. ### Risks & Mitigations - **Existing users of `azuredevops` (Python):** Mitigated by `azuredevops_go` parity + migration guide in Phase 3. The Go plugin already covers cloud + on-prem. - **External contributors with in-progress Python plugins:** The 3-month window + the warning phase gives them time to port. No private Python plugins are known to exist in the wild. - **Reverting later:** Not needed; Go covers all current and planned use cases. ### What we need - 👍 / 👎 from maintainers and community on deprecating Python. - Owners to confirm `azuredevops_go` feature parity with the Python `azuredevops` plugin (esp. CI/CD environment extraction). - Volunteers to write the Phase-3 migration guide. --- /cc @merico-dev/devlake-maintainers -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
