nevzheng opened a new issue, #11985:
URL: https://github.com/apache/gravitino/issues/11985
### What would you like to be improved?
**Why this matters (adoption).** The OpenAPI description is the contract that
downstream, non-Java consumers rely on: SDK generators (Rust/TS/Go), catalog
connectors, mock servers, and contract tests all consume it to produce typed
bindings. The whole value of shipping an OpenAPI spec is that someone can
point
a generator at it and get working, typed clients with zero friction. Today
that
doesn't hold — pointing a standard generator (e.g. `progenitor` for Rust) at
the
spec fails immediately. That's an adoption concern: it raises the cost of
building on Gravitino outside the Java client, and a strictly-valid,
codegen-clean spec is what makes the polyglot ecosystem (Python client,
connectors, third-party SDKs) cheap to grow.
**Why it's slipping through.** The spec under `docs/open-api/` *is* linted
in CI
by the Gradle `:docs:build` task (`redocly lint
--extends=recommended-strict`,
see `docs/build.gradle.kts`), but the check is ineffective:
1. **The linter is pinned to a ~2-year-old version, buried in Gradle.**
`docs/build.gradle.kts` hard-codes `@redocly/[email protected]` inside an
`NpxTask`
string literal — invisible to Dependabot/Renovate, so it has silently
drifted. Same `--recommended-strict` flags, current Redocly:
| Redocly (`--recommended-strict`) | Result on `openapi.yaml` |
|---|---|
| `1.23.1` (current CI) | ✅ valid, 0 issues |
| `2.38.0` (latest) | ❌ **10 errors** |
(Redocly CLI is semver; `1.23.1 → 2.38.0` spans the **2.0** major —
~mid-2025
— plus ~2 years of minor releases. 2.0 is why `recommended-strict`
legitimately
tightened, e.g. the old `spec` rule is now `struct`.)
2. **The spec is validated but never consumed as a machine artifact** —
linted
and rendered (Redoc), never bundled or generated-from — so issues that
break
typed codegen pass even a current linter.
The result is a spec with issues invisible to humans (it renders fine) and to
the Java client (which shares DTOs, not the spec), but fatal to typed
consumers.
A current Redocly + a codegen pass surface:
- `statistics.yaml` — `Statistic.value` uses `type: "null"`, OpenAPI 3.1
syntax
invalid in the declared **3.0.3** (the one hard structural error).
- 5 examples that don't validate against their schema (policies, datatype ×2,
tables, lineage).
- 3 `not`/`required` entries referencing undefined properties
(lineage `DatasetEvent`/`JobEvent`).
- `nullable` used without `type` (a fileset field).
- Unused `KerberosAuth` security scheme.
- (codegen-only, not caught by lint) `default: null` ×4 and one bare
`format: int32` without `type` — these break progenitor/typify generation.
Note: the declaration is correct — this is genuinely a 3.0.3 document with a
single stray 3.1 token, not one that should be upgraded to 3.1.
### How should we improve?
A spectrum, from a minimal unblock to the full pipeline — pick the appetite.
Fixing the surfaced findings is cross-cutting (trackable as subtasks).
**Option A — minimal (unblock in place).** Bump `@redocly/cli` `1.23.1 →
2.x` in
`docs/build.gradle.kts` and fix the findings so `--recommended-strict`
passes.
Caveat: crosses a **major** — v2 needs Node `20.19+/22.12+/23+` (the build
pins
Node `21.6.1`, unsupported), moved to ESM, renamed the `spec` rule to
`struct`.
Leaves the version an untracked Gradle string — root cause unaddressed.
**Option B — visible, auto-tracked linting (non-blocking).** Add a dedicated
GitHub Actions workflow that lints `docs/open-api/**` on PRs, with tool
versions
declared where Dependabot/Renovate tracks them (action ref or
`package.json`),
not a buried Gradle string. Run **warn-only** first, promote to enforcing
once
the backlog is fixed, and **deprecate the Gradle `lintOpenAPI` task**. Two
complementary linters:
- **Redocly** — structural validation + bundling (already used here).
- **Spectral** — opinionated *governance* rulesets Redocly's built-ins don't
cover: OWASP API-security, documentation-completeness (require
descriptions,
`operationId`s), and house conventions (consistent error envelope, the
`application/vnd.gravitino.v1+json` media type, identifier/naming rules).
Custom rules are JSONPath-based, so Gravitino-specific conventions can be
enforced mechanically.
**Option C — full quality-gate pipeline (recommended target).** On top of B,
treat the spec as a shippable artifact:
- **Bundle** (`redocly bundle` → single `openapi.json`) as a release
artifact,
so consumers point one clean file at their generator instead of resolving
~30
cross-referenced files. Also the codegen input.
- **Codegen smoke test** with **both** generators — `openapi-generator`
(broad,
multi-language; the standard consumer signal) and `progenitor` (Rust,
stricter
typify; catches issues the others tolerate, e.g. the `default:null` /
bare-`format` cases above). Fail CI if either can't generate.
- **Breaking-change diffing** (`oasdiff`) vs the base branch, with inline PR
annotations. (Optic evaluated — same category as oasdiff, skipped as
redundant.)
Recommended target pipeline:
**Redocly + Spectral lint → bundle → openapi-generator + progenitor codegen →
oasdiff** — the standard GitHub-Actions API quality gate, versions
Dependabot-tracked.
Environment: spec `v1.3.0`, `docs/open-api/`, `openapi: 3.0.3`.
**TODO (future, out of scope for this issue):**
- [ ] Contract testing
- [ ] Preview docs
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
