henry3260 opened a new pull request, #74047:
URL: https://github.com/apache/airflow/pull/74047
## Why
`go-sdk` reached across the monorepo for both schemas it generates from —
`airflow-core`'s Dag
serialization schema and `task-sdk`'s supervisor wire-schema snapshot. Two
things follow from that:
- A generated file is only explainable from a directory the published Go
module does not contain, so
`go generate` and two Go tests only work inside the monorepo.
`messages_test.go` carried a skip for
exactly this.
- Every change to a Python schema becomes a change that has to carry
regenerated Go with it, from
whoever happened to be editing the Python — Go toolchain, network fetch of
go-jsonschema and all.
That is what `check-go-sdk-generated-drift` (#73963) asked of them.
`ts-sdk` and `java-sdk` already vendor their copies
(`ts-sdk/schema/dag-schema.json`,
`java-sdk/sdk/schema/`) and split the work into a cheap sync plus a
generation check. `go-sdk` was
the only SDK still reaching out. Raised by @jason810496 in review of #73963.
## What
Both schemas are vendored under `go-sdk/schema/` and the generators read the
copies. Regenerating
from them produces byte-identical output — no `.gen.go` changes in this PR.
That splits one question into two hooks:
- **`sync-go-sdk-schemas`** (new, `scripts/ci/prek/sync_go_sdk_schemas.py`)
— is the copy equal to
its original? Copying is mechanical, so it copies for you and fails, the
way
`sync-ts-sdk-dag-schema` does. Pure file comparison: no Go toolchain, no
network. Its failure
message names what to regenerate, and for the supervisor schema it also
names
`SupervisorSchemaVersion`, which has to move when `api_version` does.
- **`check-go-sdk-generated-drift`** — is the committed Go what the copy
generates? It now watches
the copies rather than the Python originals, since what to do about a new
schema construct (a
generator rule, an authoring exclusion) is a decision rather than a copy.
`normalize_test.go` and `messages_test.go` read the vendored copies, so the
latter's skip is gone:
the version check now runs in a standalone checkout too.
The copies are named `dag-schema.json` and `supervisor-schema.json`, not
`*.schema.json`, so the
`lint-json-schema` hook (`files: .*\.schema\.json$`) does not rewrite them
and break byte-equality
with their source — the same reason `ts-sdk` named its copy that way.
Left alone deliberately: `go-sdk/airflow/spec_test.go` still reads
`trigger_rule.py` and
`weight_rule.py` from `airflow-core`. That is a tripwire against Python
enums, not generated code,
and vendoring a `.py` into `go-sdk` is a separate decision.
## Verified locally
- `go generate ./airflow/... ./pkg/execution/genmodels/...` from the
vendored copies leaves every
`.gen.go` byte-identical.
- `go test ./internal/genspec/... ./pkg/execution/...` passes.
- Bumping `api_version` in `task-sdk`'s schema makes `sync-go-sdk-schemas`
refresh the copy and fail
with the three follow-up steps, and
`TestSupervisorSchemaVersionMatchesSnapshot` then fails until
the constant moves.
- `uv run --project scripts pytest
scripts/tests/ci/prek/test_sync_go_sdk_schemas.py
scripts/tests/ci/prek/test_check_go_sdk_generated_drift.py` — 22 passed.
- `prek run --from-ref main --stage pre-commit`, `prek run
sync-go-sdk-schemas --all-files`,
`prek run check-go-sdk-generated-drift --all-files`, ruff and mypy all
pass. The manual stage was
not run to completion locally; CI covers it.
Draft because the manual-stage checks have not been run end to end yet.
related: #73963, #73954
---
##### Was generative AI tooling used to co-author this PR?
- [X] Yes — Claude Code (Opus 5)
Generated-by: Claude Code (Opus 5) following [the
guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions)
🤖 Generated with [Claude Code](https://claude.com/claude-code)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
