klesh commented on PR #9005: URL: https://github.com/apache/devlake/pull/9005#issuecomment-5167106219
> @klesh Thanks — that's very helpful context, and I fully agree with the direction. Let me reframe this PR, because I think its rationale holds independently of the Python plugins: > > **We are not using the Python plugin.** As you recommend, we will go with the Golang Azure DevOps plugin (`backend/plugins/azuredevops_go`). So `backend/python/plugins/azuredevops` / `pydevlake` are not the driver here — and to be explicit: **we don't need a migration path**. We are not running the Python plugin today, so there is no existing data or config on our side to move over. > > **Python remains a required runtime even without the Python plugins**, because of dbt: > > * `backend/python/requirements.txt` pins `dbt-mysql==1.7.0` / `dbt-postgres==1.7.19`, which the Go `dbt` plugin shells out to. > * `devops/docker/alpine-dbt/Dockerfile` installs dbt on top of `alpine:3.15` (Python 3.9). > * `devops/docker/lake-builder/Dockerfile` still builds against **python:3.9**, which is EOL (2025-10) and therefore no longer receives security fixes — exactly the attack-surface concern you raised. > * `grafana/scripts` has its own `requirements.txt`. > > So this PR is really "stop shipping an EOL Python interpreter in our images", not "invest in the Python plugin framework". If the Python plugins are decommissioned, this PR gets _smaller_, not obsolete — the `pyproject.toml`/`poetry.lock` hunks would simply drop out. > > **One question**, so I can align my follow-up work with your plans: is there an issue or a rough timeline for decommissioning the Python plugin framework (`backend/python/pydevlake`, the remote-plugin bridge in `backend/server/services/remote`)? I'd rather not open dependency PRs that touch code you intend to delete. If that is the direction, I'm happy to reduce this PR to the parts that survive the removal (Docker base images + `backend/python/requirements.txt` for dbt), or to help with the removal itself. > > Related, and probably interesting for the same "reduce attack surface" reason: `dbt-mysql` has had no release since 2024-04 and pins `dbt-core~=1.7`, freezing the whole dbt stack on the 1.7 line and capping the usable Python version. I have a change prepared that drops `dbt-mysql` and moves `dbt-postgres` to 1.11 (dbt-core 1.12). That's a user-facing breaking change, so I'd like your opinion before opening it — happy to file an issue for discussion first. Thanks for the detailed explanation—I really appreciate it! You're spot on. My plan is to deprecate the problematic `dbt` plugin first, which is already in progress ([#8970](https://github.com/apache/devlake/issues/8970)) and scheduled for removal by August 31, 2026. Following that, I plan to deprecate the Python subsystem over the subsequent 3 months. I got your point and I am merging the PR for the time being. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
