DoDiODev opened a new pull request, #9352:
URL: https://github.com/apache/devlake/pull/9352

   ### ⚠️ Pre Checklist
   
   > Please complete _ALL_ items in this checklist, and remove before submitting
   
   - [x] I have read through the [Contributing 
Documentation](https://devlake.apache.org/community/).
   - [x] I have added relevant tests. (existing suites adapted; no new behavior)
   - [x] I have added relevant documentation. (N/A — no user-facing change)
   - [x] I will add labels to the PR, such as `pr-type/bug-fix`, 
`pr-type/feature-development`, etc.
     (I can't set labels myself; suggested: `pr-type/refactor`, 
`component/plugins`)
   
   ### Summary
   
   Moves the Python plugin framework from Pydantic 1 to Pydantic 2, as agreed in
   #9092. Pydantic 1 does not support Python 3.14, so this is the first step
   before the Python runtime (3.11, EOL 2027-10) can be upgraded.
   
   | Package | Before | After |
   |---|---|---|
   | pydantic | 1.10.27 | 2.14.3 |
   | sqlmodel | 0.0.8 | 0.0.38 |
   | SQLAlchemy | 1.4.41 | 2.0.57 |
   
   Ported: `pydevlake`, `azuredevops`, `glpi`, `redmine`, `zephyr` and the e2e
   `fakeplugin`. The three Poetry locks only change the Pydantic stack (plus its
   new dependencies `pydantic-core`, `annotated-types`, `typing-inspection`;
   `sqlalchemy2-stubs` is dropped).
   
   **pydevlake**
   
   - Pydantic 2 APIs: `model_config`/`ConfigDict`, `field_validator`,
     `model_fields`, `model_dump_json`, `RootModel`, `Literal` instead of
     `Field(const=True)`.
   - `Optional[...]` fields get an explicit `= None` default (in Pydantic 2 they
     are required otherwise). This accounts for most of the diff in the domain
     layer and plugin models/migrations.
   - `sa_column=Column(...)` → `sa_type=...` (+ `sa_column_kwargs` for the
     `created_at`/`updated_at` defaults), so columns are not shared between
     inherited tables.
   - `ToolTable` sets `coerce_numbers_to_str=True` to keep the Pydantic 1
     behavior for APIs that return ids as numbers.
   - `autoextract` uses `model_validate`: SQLModel table models skip validation
     in `__init__`, so raw strings would no longer be coerced to
     `datetime`/enums.
   - `RawModel` keeps its column types (`data`/`input` binary, `url` TEXT), so
     existing raw tables stay compatible. Two tests wrote `str` into `data`;
     they now encode to bytes like the collector does (SQLAlchemy 2 rejects
     `str` for binary columns).
   - `message.py`: `SerializeAsAny` for `RemoteScope.data`, otherwise Pydantic 2
     serializes the base type and drops plugin-specific scope fields.
   - `model_info.py`: reads `json_schema_extra` / annotations instead of
     `field.field_info.extra` / `field.required`.
   - Test harness: expected and actual models are re-validated before comparing,
     so both sides have the same field types.
   
   **Plugins**
   
   - `glpi`: GLPI expands dropdowns to `'Name (id)'`; these values are reduced 
to
     the id before validation of the integer columns. The names are still read
     from the raw data for enrichment, so the output does not change.
   - `redmine`, `zephyr`: `class Config` → `model_config`.
   - `fakeplugin`: ids as strings, `Optional` defaults.
   
   **Dependabot**
   
   - The `pydantic` major-version ignore is removed.
   - The `sqlmodel` ignore stays: 0.0.48 rejects naive datetimes (pydevlake uses
     `datetime.now()`/`utcnow()`) and pulls in SQLAlchemy 2.1. That is a 
separate
     follow-up.
   
   ### Does this close any open issues?
   
   Part of #9092 (Pydantic step; Python runtime upgrade follows separately).
   
   ### Screenshots
   
   N/A
   
   ### Other Information
   
   **Tests**
   
   - `cd backend/python && sh ./run_tests.sh` — passed:
     azuredevops 8 passed / 1 skipped, glpi 72 passed / 1 skipped,
     redmine 31 passed / 1 skipped, zephyr 26 passed, pydevlake 13 passed.
   - `make unit-test` and `make e2e-test` (MySQL and PostgreSQL, includes the
     Python `fakeplugin`) — passed in the fork CI: [run 
42621392320](https://github.com/DoDiODev/devlake/actions/runs/42621392320) 
(16/16 jobs)
   
   **Not in this PR**
   
   - Python runtime 3.11 → 3.13/3.14 (next PR).
   - sqlmodel ≥ 0.0.48 / SQLAlchemy 2.1 (timezone-aware datetimes).
   - Pinning the uv installer in `backend/Dockerfile` and the lake-builder 
image.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to