awsomesud347 opened a new pull request, #12130:
URL: https://github.com/apache/seatunnel/pull/12130
### Purpose of this pull request
Closes #12123 (sub-task of #12126).
**Which REST API hits `getJobStatusData()`** — deliverable 1 of the issue:
it is `GET /metrics`
and `GET /openmetrics`, via `MetricsServlet` ->
`NodeExtension.getMetricFamilySamples()` ->
`JobMetricExports.collect()` -> `CoordinatorService.getJobCountMetrics()`.
That runs once per
Prometheus scrape on the master and needs only nine counters, so
retained-history size shows up
directly as `/metrics` latency. The job-listing endpoints do not reach
`getJobStatusData()` at
all — they are served by `JobInfoService` reading the IMaps directly.
Two changes, both in the "avoidable server-side scanning" category the issue
asks about:
1. `getJobCountMetrics()` now calls a new
`JobHistoryService.getJobStatusCounts()`, which folds
statuses directly instead of building a `JobStatusData` per retained job
only to read one enum
field from each. `getJobStatusData()` is unchanged and still serves the
CLI (`seatunnel.sh -l`),
whose all-results output cost is unavoidable.
2. `JobInfoService.getJobsByStateJson(state, start, rows)` applies the page
**before** the per-job
metrics and DAG point lookups. Previously `?page=1&rows=10` against N
retained jobs cost roughly
`1 scan + 2N point lookups + N metric aggregations` and then discarded
all but 10 — `page`/`rows`
bounded the response body, not the work. Now they bound both.
Retention/TTL is deliberately untouched; that belongs to #10767.
Benchmarks quantifying both paths are still in progress and will be added as
a follow-up comment;
this is opened as a draft until then.
### Does this PR introduce _any_ user-facing change?
No. Response shapes, parameter names and error messages are unchanged. The
unpaged
`getJobsByStateJson(String)` overload is kept so the legacy text-command
REST path behaves exactly
as before, and `PageBaseServlet.writeJsonWithPagination` is untouched so
`/running-jobs` is
unaffected.
### How was this patch tested?
- Added `JobHistoryServiceTest#testJobStatusCountsAgreeWithJobStatusData`,
asserting the new count
method matches `getJobStatusData()` folded by status so the two cannot
drift.
- `mvnw test -pl seatunnel-engine/seatunnel-engine-server` — 459 tests, 0
failures, 0 errors.
- `mvnw test` across engine common/core/serializer/client — 81 tests, 0
failures, 0 errors.
- `spotless:apply` and `-DskipTests verify` clean.
Note: `JobHistoryServiceTest` is `@DisabledOnOs(OS.WINDOWS)`, so the new
case was not executed on
my machine and runs for the first time in CI.
### Check list
* [x] If any new Jar binary package adding in your PR — none added.
* [x] If necessary, please update the documentation — no user-facing change,
so no doc update.
* [x] If necessary, please update `incompatible-changes.md` — no
incompatibility.
* [x] If you are contributing the connector code — not a connector change.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
