nevzheng opened a new pull request, #11989:
URL: https://github.com/apache/gravitino/pull/11989
### What changes were proposed in this pull request?
Rewrites the three lineage event schemas (`RunEvent`, `DatasetEvent`,
`JobEvent`) in `docs/open-api/lineage.yaml` as flattened, **closed** object
schemas (`additionalProperties: false`), removes the shared `BaseEvent` `allOf`
base and the `not: { required: [...] }` blocks, and fixes the `eventType`
example (`START|RUNNING|COMPLETE|ABORT|FAIL|OTHER` → `START`).
### Why are the changes needed?
The event schemas were ported from OpenLineage using JSON Schema idioms that
do not mean the same thing under OpenAPI 3.0, leaving the `/lineage`
request-body `oneOf` unsound:
- `not: { required: [job, run] }` referenced properties the schemas never
declare (a strict-lint error), and it only rejects a payload carrying **both**
fields — a `DatasetEvent` carrying just `run` validated fine.
- Because the schemas were open, variants matched payloads containing other
variants' fields: the union was disambiguated by accident, not by design.
Verified with an ajv payload matrix against the old and new bundled specs:
| payload | old spec | new spec |
|---|---|---|
| valid `RunEvent` | ACCEPT `[RunEvent]` | ACCEPT `[RunEvent]` |
| valid `JobEvent` | ACCEPT `[JobEvent]` | ACCEPT `[JobEvent]` |
| valid `DatasetEvent` | ACCEPT `[DatasetEvent]` | ACCEPT `[DatasetEvent]` |
| superset: `dataset`+`run`+`job` | **ACCEPT `[RunEvent]`** — dataset
silently dropped | REJECT |
| mixed: `dataset`+`run` (no `job`) | **ACCEPT `[DatasetEvent]`** — run
state smuggled in | REJECT |
| unknown top-level field | ACCEPT | REJECT |
All well-formed events resolve identically and unambiguously (no
regression); malformed mixtures are rejected instead of misclassified. The last
row is the deliberate tradeoff of closing the schemas: unknown *top-level*
fields are rejected — consistent with OpenLineage's model, where **facets** are
the sanctioned extension point and remain open (`BaseFacet` keeps
`additionalProperties: true`).
Flattening is forced by OpenAPI 3.0: `additionalProperties: false` cannot be
combined with a shared `allOf` base (a closed subschema cannot see the base's
properties and would reject them). Each event therefore declares the common
fields (`eventTime`, `producer`, `schemaURL`) itself; a comment in the file
documents this so the duplication is not "cleaned up" back into a base.
Clears 4 of the remaining Redocly `recommended-strict` findings (3 ×
`no-required-schema-properties-undefined`, 1 × `no-invalid-schema-examples`).
Part of #11985
### Does this PR introduce _any_ user-facing change?
No change for well-formed OpenLineage events — all three shapes validate
exactly as before. Payloads mixing fields of two event types, previously
misclassified by spec-driven validators, are now rejected by the spec.
Spec/docs-only; no server behavior change.
### How was this patch tested?
- `redocly lint --extends=recommended-strict` — lineage findings cleared (10
→ 6 against `main`), no new findings.
- `redocly bundle` — bundles to valid `openapi.json`.
- ajv behavioral matrix (table above) run against the old and new bundles,
exercising the `/lineage` `oneOf` with valid, mixed, superset, and
unknown-field payloads.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
