DoDiODev opened a new pull request, #8963:
URL: https://github.com/apache/devlake/pull/8963
<!--
Licensed to the Apache Software Foundation (ASF) under one or more
contributor license agreements. See the NOTICE file distributed with
this work for additional information regarding copyright ownership.
The ASF licenses this file to You under the Apache License, Version 2.0
(the "License"); you may not use this file except in compliance with
the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
### β οΈ Pre Checklist
> Please complete _ALL_ items in this checklist, and remove before submitting
- [ x] I have read through the [Contributing
Documentation](https://devlake.apache.org/community/).
- [ ] I have added relevant tests. --> only updates no additional features
- [ ] I have added relevant documentation. --> only updates no additional
features
- [ ] I will add labels to the PR, such as `pr-type/bug-fix`,
`pr-type/feature-development`, etc.
<!--
Thanks for submitting a pull request!
We appreciate you spending the time to work on these changes.
Please fill out as many sections below as possible.
-->
### Summary
# Dependency Modernization & Infrastructure Hardening
Comprehensive update of all runtime dependencies, container images, and
developer
tooling to current LTS/stable versions. Goal: bring every component to a
supported
version line, harden the Grafana deployment for production use, and reduce
technical debt.
> **Implementation note:** This modernization was planned and executed with
the
> assistance of GitHub Copilot (agent mode). Each wave was driven by a
structured
> plan and validated iteratively.
## Summary of Changes
### π₯οΈ Frontend (config-ui)
- **Node.js 18 β 24 LTS** (Krypton); all version pins reconciled across CI,
Dockerfile, Volta, and corepack
- **Yarn 3 β 4.17.0** (corepack-managed; lockfile migrated v6 β v10)
- **React 18 β 19**, antd 5 β 6, Vite 5 β 8 (rolldown), TypeScript 5 β 6
- **ESLint 8 β 10** flat-config migration (`eslint.config.mjs`); replaced
unmaintained `eslint-plugin-header` with `eslint-plugin-headers`
- Fixed 51 `tsc` errors introduced by the major bumps (React 19 refs/useRef,
react-markdown v10 API, cron-parser v5 API, missing `@types/lodash`)
- Dockerfile: `node:24-bookworm-slim`, nginx `1.31.2`
> **Reviewer focus:** `config-ui/eslint.config.mjs` (new flat config rules),
> `config-ui/tsconfig.json` (`ignoreDeprecations: "6.0"` for TS 6 compat),
> and the `useRef(undefined)` pattern changes across components.
### π Python (backend/python)
- **Python runtime 3.9 β 3.11** (all `pyproject.toml`, `Dockerfile.server`,
`.devcontainer`)
- **Pydantic v1 β v2** / SQLModel 0.0.8 β 0.0.38 (full API migration:
validators, model configs, serialization, schema generation)
- Library bumps: inflect β 7.5, pytest β 9.1, jsonpointer β 3.1, fire β 0.7
- **sqlglot 25 β 30** (Grafana SQL transpiler; isolated, no pydevlake impact)
- RPC wire format (camelCase aliases, `exclude_none`) verified byte-identical
β no breaking change for the GoβPython contract
> **Reviewer focus:** `backend/python/pydevlake/pydevlake/model.py`
(Pydantic v2
> migration core β `@field_validator`, `model_config`, `sa_type` usage),
> `message.py` (`__root__` β `RootModel`), `extractor.py` (`_is_optional()`
> annotation introspection).
### π Grafana
- **Grafana 11.6 β 12.4.5 β 13.0.2** (two consecutive major upgrades)
- Entrypoint hardened (`grafana/scripts/entrypoint.sh`):
- Volume permission self-heal (legacy uid 472 β gid 0 compatible)
- Deprecated Angular `grafana-piechart-panel` auto-removal
- Datasource bootstrap: admin password default, idempotent creation,
delete-retry for stale provisioned datasources
- Home dashboard path detection (bind-mount vs. baked-in image layout)
- Removed duplicate/broken dashboards (`ai-model-roi.json` β uid collision +
cross-datasource reference failures in Grafana 12+ Scenes engine)
- `pluginVersion` aligned to `"13.0.2"` across all 58 dashboards
- PostgreSQL datasource: `postgresVersion` 1400 β 1800, type changed to
`grafana-postgresql-datasource` (Grafana 13 rename), UID-based references
> **Reviewer focus:** `grafana/scripts/entrypoint.sh` (the main logic change
β
> datasource lifecycle, permission handling, home dashboard probe). Dashboard
> JSON changes are mechanical (`pluginVersion` string replacement).
### ποΈ Databases
- **PostgreSQL 14/17 β 18.1** (`docker-compose-dev-postgresql.yml`, e2e test
compose)
- **MySQL 8.0.26 β 8.4.10 LTS** (e2e test compose, temporal deployment)
- MySQL dev images pinned from floating `mysql:8` to `mysql:8.4.10`
> **Reviewer focus:** `docker-compose-dev-postgresql.yml` and
> `backend/test/e2e/remote/docker-compose.test.yml` β version bumps only.
> `devops/releases/*` artifacts intentionally left untouched (frozen
releases).
### π³ Container Images & Infra
- **oauth2-proxy** 7.4.0 β 7.15.3 (both dev compose files)
- **nginx-unprivileged** 1.30.3 β 1.31.2 (`config-ui/Dockerfile`)
- **swag CLI** unified to v1.16.6 across all Dockerfiles
- Grafana Dockerfile: removed `grafana-cli plugins install
grafana-piechart-panel`
(core pie chart used; Angular plugin dropped in Grafana 12)
> **Reviewer focus:** Minimal β image tag changes only. Verify no
> `platform: linux/x86_64` regressions on ARM builds.
## Not Included (planned follow-ups)
| Item | Scope | Risk |
|------|-------|------|
| libgit2 1.3 β 1.9 + git2go v33 β v37 | Native lib ABI change | High |
| mockery v2 β v3 | Generated code churn | Medium |
| Python 3.11 β 3.13 | Runtime bump (after Pydantic v2) | Medium |
| Drop dbt-mysql + dbt-postgres β 1.10 | User-facing removal | Medium |
| aws-sdk-go v1 β v2, lib/pq β pgx | Go module modernization | Future |
## Commit Structure
Changes are grouped by phase, each independently committed and buildable:
| Commit(s) | Scope |
|-----------|-------|
| Pin & align | Floating image tags pinned, swag CLI unified, low-risk npm
bumps |
| Grafana 12 | Image upgrade + entrypoint hardening + dashboard fixes |
| nginx bump | `nginx-unprivileged` 1.30 β 1.31 |
| Python 3.11 | Runtime-only bump (no library changes) + devcontainer
refresh |
| sqlglot 30 | Grafana SQL transpiler bump (isolated) |
| Pydantic v2 | Full pydevlake migration: validators, models, serialization |
| Grafana 13 | Image 13.0.2, pluginVersion, PG datasource config |
| PostgreSQL 18 | Image bump + DB-agnostic compose support |
| MySQL 8.4 | e2e/temporal image bump to 8.4.10 LTS |
## Validation
| Area | Method | Result |
|------|--------|--------|
| Frontend | `yarn install && yarn tsc && yarn lint && yarn build && yarn
test` | β
Green |
| Python | `poetry run pytest` (pydevlake: 10 tests) | β
Green |
| Grafana (MySQL) | Container start, `/api/health`, 58 dashboards load, 0
`level=error` | β
Green |
| Grafana (PG) | Same validation on PostgreSQL compose variant | β
Green |
| PG Migrations | Fresh DB, `FORCE_MIGRATION=true`, `/ready`=200 | β
Green |
| Backend Go | `make build`, unit tests, E2E (AE plugin) | β
Green |
### Does this close any open issues?
no
### Screenshots
Include any relevant screenshots here.
### Other Information
Any other information that is important to this PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
