jerryshao opened a new issue, #12521:
URL: https://github.com/apache/gravitino/issues/12521
### Describe the feature
Add query parameters to the list jobs REST API (`GET
/metalakes/{metalake}/jobs/runs`) to filter jobs by queued/started/finished
time and sort the results — e.g. "jobs created in the last 24 hours."
### Motivation
Callers currently have to fetch the entire job list and filter/sort
client-side to answer time-scoped questions like "what ran in the last 24
hours," which doesn't scale as job history grows.
### Describe the solution
- New query parameters on `listJobs`: `queuedAfter`, `startedAfter`,
`finishedAfter` (ISO-8601 instant strings, inclusive lower bounds, AND-combined
— a job missing the relevant timestamp is excluded by that filter), `sortBy`
(`queuedAt`|`startedAt`|`finishedAt`, default `queuedAt`), `sortOrder`
(`asc`|`desc`, default `desc`).
- Implemented entirely in `JobOperations` (server REST layer):
filtering/sorting is applied in-memory after the existing
`jobOperationDispatcher.listJobs(...)` fetch and authorization filter — no
dispatcher/manager/storage/SQL changes. Jobs missing a sort field always sort
last, regardless of direction.
- Query params are validated up front (before the dispatcher fetch), so
invalid input fails fast with 400; `sortBy`/`sortOrder` are matched exactly
against the OpenAPI-documented casing.
- OpenAPI spec (`docs/open-api/jobs.yaml`) updated with the five new
parameters.
- Unit tests in `TestJobOperations` covering filter/sort combinations,
boundary inclusivity, null handling, and validation errors.
### Additional context
Builds on `queuedAt`/`startedAt` (#12508). Scoped to the REST layer only —
no SQL-level pushdown (`queuedAt` isn't a queryable column; it's derived from a
JSON-serialized `audit_info` blob) or Java/Python client changes. The full
unfiltered job list is still fetched before filtering in memory, which is fine
at current scale but could be revisited if job history volume grows.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
