potiuk commented on PR #67649: URL: https://github.com/apache/airflow/pull/67649#issuecomment-4569025679
The two failing test jobs (`Airflow CTL tests / airflow-ctl:P3.10 tests` and `Task SDK tests / task-sdk:P3.10 tests`) both fail with the same Pydantic v2 `ValidationError: Field required [type=missing]` pattern, across multiple generated models — `ConnectionResult`, `ConnectionResponse`, `DagVersionResponse`, `DagRun`, … The root cause is a behavior change in `datamodel-code-generator` between **0.33.0 and 0.57.0**: nullable fields no longer get an explicit `= None` default. With the old generator we got ```python host: Annotated[str | None, Field(title="Host")] = None # 0.33.0 — optional, defaults to None ``` …and with the new one we get ```python host: Annotated[str | None, Field(title="Host")] # 0.57.0 — required, must be passed (even as None) ``` Pydantic v2 treats `str | None` without an explicit default as **required**, not optional — so every call site that constructs e.g. `ConnectionResult(conn_id="x", conn_type="y")` without spelling out `host=None, schema=None, login=None, …` now fails validation. The regenerated datamodels in this PR are exactly what the new generator produces; the test failures are downstream of that semantic shift, not a bug in the datamodel files themselves. Possible paths forward: 1. **Pin `datamodel-code-generator` below the version that dropped the default** (likely the cleanest short-term fix — a `<X.Y.Z` upper bound keeps dependabot from re-proposing this specific bump until the upstream behavior is addressed). 2. **Look for an opt-in flag on `datamodel-codegen`** that restores explicit `= None` defaults for nullable fields and add it to the prek hook entries in `.pre-commit-config.yaml` (`generate-tasksdk-datamodels`, `generate-airflowctl-datamodels`). If such a flag exists this is the lowest-friction long-term fix. 3. **Update every call site and test fixture** to pass all nullable fields explicitly. Functionally correct against the new contract but high churn across `task-sdk` and `airflow-ctl`, and bakes in a more verbose construction pattern everywhere. Happy to push a follow-up commit on either (1) or (2) if a preference is expressed. --- Drafted-by: Claude Code (Opus 4.7, 1M context); reviewed by @potiuk before posting -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
