chihsuan opened a new pull request, #10240:
URL: https://github.com/apache/ozone/pull/10240
## Summary
Documentation-only sync of `ReconApi.md` and the in-product Swagger yaml
against the actual Recon JAX-RS code under
`hadoop-ozone/recon/src/main/java/.../api/`. No implementation changes. Adds 16
missing endpoints, fixes 16 existing endpoints' parameters/response examples to
match the current Java `@JsonProperty` field names and HTTP status behavior,
and fixes pre-existing structural bugs in the yaml.
## Why
Both docs had drifted over multiple releases. The yaml had at least one
schema reference that didn't resolve (`ClusterStorageReport`), several routes
documented with parameters that no longer exist (`batchNum` on
`/containers/unhealthy*`), and ~16 routes that exist in Java with no
documentation. JSON field names in the README examples (`replicatedTotal`,
`entities`, `deletedkeyinfo`, etc.) did not match the actual `@JsonProperty`
names serialized by the response DTOs.
## Endpoint inventory (46 routes total)
| Method | Path | Status | Notes |
| --- | --- | --- | --- |
| GET | `/containers` | Updated | Documented existing `prevKey` / `limit` in
yaml |
| GET | `/containers/{id}/keys` | Updated | Example refreshed with `lastKey`
and `CompletePath` |
| GET | `/containers/{id}/replicaHistory` | Unchanged | |
| GET | `/containers/missing` | Updated | Marked `@Deprecated` (use
`/containers/unhealthy/MISSING`) |
| GET | `/containers/unhealthy` | Updated | `batchNum` removed; documented
`maxContainerId` / `minContainerId` |
| GET | `/containers/unhealthy/{state}` | Updated | Same parameter rename |
| GET | `/containers/unhealthy/export` | Added | List export jobs |
| POST | `/containers/unhealthy/export` | Added | Submit async CSV export
job; 400/429 documented |
| GET | `/containers/unhealthy/export/{jobId}` | Added | Job status |
| GET | `/containers/unhealthy/export/{jobId}/download` | Added | TAR
download; 404/409/429 documented |
| DELETE | `/containers/unhealthy/export/{jobId}` | Added | Cancel job |
| GET | `/containers/deleted` | Updated | Section added in README (was
yaml-only) |
| GET | `/containers/mismatch` | Unchanged | |
| GET | `/containers/mismatch/deleted` | Unchanged | |
| GET | `/datanodes` | Updated | Example now includes `opState`, `version`,
`setupTime`, `revision`, `buildDate`, `layoutVersion`, `networkLocation`,
`openContainers` |
| PUT | `/datanodes/remove` | Updated | Corrected to `datanodesResponseMap`
wrapper with three outcome categories |
| GET | `/datanodes/decommission/info` | Added | README section (yaml
already had it) |
| GET | `/datanodes/decommission/info/datanode` | Added | README section
(yaml already had it) |
| GET | `/clusterState` | Updated | Example now includes
`missingContainers`, `openContainers`, `deletedContainers`,
`keysPendingDeletion`, `deletedDirs`, `scmServiceId`, `omServiceId`, full
`storageReport` |
| GET | `/volumes` | Unchanged | |
| GET | `/buckets` | Unchanged | |
| GET | `/keys/open` | Updated | Example uses real field names (`lastKey`,
`replicatedDataSize`, `fso`, `nonFSO`, `status`); documented `startPrefix`,
`includeFso`, `includeNonFso` |
| GET | `/keys/open/summary` | Updated | README section added |
| GET | `/keys/open/mpu/summary` | Added | Both yaml + README;
`totalDataSize` (not `totalUnreplicatedDataSize`) is intentional |
| GET | `/keys/deletePending` | Updated | Example uses `deletedKeyInfo`
(camelCase) and full `replicatedDataSize`/`unreplicatedDataSize` names;
`startPrefix` documented |
| GET | `/keys/deletePending/dirs` | Updated | Example uses `deletedDirInfo`
(`KeyEntityInfo` shape, not `RepeatedOmKeyInfo`) |
| GET | `/keys/deletePending/summary` | Updated | README section added |
| GET | `/keys/deletePending/dirs/summary` | Updated | README section added |
| GET | `/keys/listKeys` | Added | Both yaml + README; documented 400/503
responses |
| GET | `/blocks/deletePending` | Unchanged | |
| GET | `/namespace/summary` | Unchanged | |
| GET | `/namespace/usage` | Unchanged | |
| GET | `/namespace/quota` | Unchanged | |
| GET | `/namespace/dist` | Unchanged | |
| GET | `/pipelines` | Unchanged | |
| GET | `/task/status` | Unchanged | |
| GET | `/utilization/fileCount` | Unchanged | |
| GET | `/utilization/containerCount` | Unchanged | |
| GET | `/metrics/{api}` | Updated | Generalized from yaml's hardcoded
`/metrics/query` to match Java `@PathParam("api")` |
| GET | `/storageDistribution` | Added | Backported from `apache/ozone-site`
and re-verified against Java |
| GET | `/storageDistribution/download` | Added | Documented 200/202/500
responses |
| GET | `/pendingDeletion` | Added | Tri-modal response based on `component`
(`scm` / `om` / `dn`); 400/204 documented |
| GET | `/heatmap/readaccess` | Added | 404 documented when feature is
disabled |
| GET | `/heatmap/healthCheck` | Added | |
| GET | `/features/disabledFeatures` | Added | |
| GET | `/triggerdbsync/om` | Added | |
**Totals:** 16 added, 16 updated, 14 unchanged.
## Yaml structural fixes (separate commit)
- `ClusterStorageReport` was indentation-nested inside
`DataNodeStorageReport.properties`, breaking the top-level `$ref` referenced by
`ClusterState.storageReport`. De-nested.
- `info.version` was missing (required by OpenAPI 3.0).
- Three stray-quote bugs (`isKey": false`) in the
`MetadataDiskUsage.subPaths` example.
- `OpenKeys.startPrefix` schema type was `integer`; corrected to `string`
(matches Java `@QueryParam` signature).
## Validation performed
- **Static yaml validation:** `swagger-cli validate` passes (was failing on
master with `Token "ClusterStorageReport" does not exist`).
- **$ref completeness:** 52 schemas defined, 52 referenced; no broken refs,
no unused schemas.
- **Route coverage:** 46 Java routes (from `@Path` annotations), 46 yaml
`paths` entries, 46 README `### METHOD /api/v1/...` headings. All three sets
match exactly. Cross-check script in the dev-support follow-up below.
- **Live cluster spot check:** built the dist (`mvn -Pdist -DskipTests
-DskipShade -DskipRecon clean install`, ~4 min), brought up `compose/ozone`
with `OZONE_REPLICATION_FACTOR=3`, loaded a couple of keys + an open S3 MPU
upload, then `curl`'d 9 of the changed/new endpoints and diffed top-level JSON
keys against the schemas. 7 endpoints clean; 2 surfaced schema gaps that were
fixed in the last two commits before push:
- `UnhealthyContainerMetadata` was missing `firstKey`, `lastKey`,
`replicaMismatchCount` (added to `UnhealthyContainersResponse` in code; absent
in schema).
- `DatanodesSummary.datanodes[]` was missing `openContainers` (present on
`DatanodeMetadata` but absent in schema).
| Endpoint | HTTP | Result |
| --- | --- | --- |
| `/containers/unhealthy?limit=5` | 200 | ⚠️ → fixed (added
`firstKey`/`lastKey`/`replicaMismatchCount`) |
| `/storageDistribution` | 200 | ✅ matches |
| `/pendingDeletion?component=om` | 200 | ✅ |
| `/pendingDeletion?component=scm` | 200 | ✅ |
| `/pendingDeletion?component=dn` | 202 | ✅ |
| `/keys/open?...&includeNonFso=true&includeFso=true` | 200 | ✅ |
| `/datanodes` | 200 | ⚠️ → fixed (added `openContainers`) |
| `/clusterState` | 200 | ✅ all 15 keys match |
| `/features/disabledFeatures` | 200 | ✅ |
| `/keys/open/mpu/summary` | 200 | ✅ |
| `/keys/listKeys?startPrefix=/` | 400 | ✅ schema documents 400 for
shallow prefix |
| `/keys/listKeys?startPrefix=/vol1/buc1` | 200 | ✅ matches
`ListKeysResponse` |
## Out of scope / follow-up
- **`ReconApi.zh.md`** — the Chinese translation is already two endpoints
behind English and uses the same stale schemas. Will file as a follow-up
sub-task of HDDS-14643 once English is settled, so a bilingual reviewer can
audit the translation in one pass.
- **`apache/ozone-site` static yaml** — a sibling PR will mirror this PR's
changes into `ozone-site/static/recon-api.yaml`. The ozone-site yaml is
currently ahead in some places (it already has `/storageDistribution` and
`/pendingDeletion`) and behind in others, so a manual diff pass is required.
- **CI guard** — optional follow-up: add a CI check that extracts `@Path`
annotations from Recon and asserts every Java route appears in the yaml. Would
prevent future drift.
## Test plan
- [x] `swagger-cli validate
hadoop-hdds/docs/themes/ozonedoc/static/swagger-resources/recon-api.yaml`
- [x] Cross-check Java routes vs yaml paths vs README headings (46/46/46)
- [x] Cross-check `$ref` resolution (52/52)
- [x] Local Recon spot check (results above; 2 gaps surfaced and fixed)
- [ ] Swagger UI smoke test in browser (manual)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
