DanielLeens opened a new issue, #12123: URL: https://github.com/apache/seatunnel/issues/12123
## Description This is a scoped scalability task for the all-jobs history listing. It is not a regression report. Verified at `dev` commit `97d461bc0773399d632fd078735736ecd44f5f0b`: - `JobHistoryService.getJobStatusData()` (`JobHistoryService.java:155`) materializes `finishedJobStateImap.values().stream()` (`:189`) to build the all-jobs listing. - Single-job lookup `getJobDetailState` (`:211`) is a direct map `get`, so not every history query scans. - Writes carry a TTL from `history-job-expire-minutes` (`:260`, `:264`, `:270`, `:344`), so the cost scales with the number of finished jobs retained in the configured window, not with all jobs ever completed. Related: #10767 (open discussion on differentiated retention of historical runtime information in IMap / StateStore). ## Expected outcome - Identify the REST API and call frequency that hit `getJobStatusData()`. - Distinguish unavoidable all-results output cost from avoidable server-side scanning for filtered or paged queries, and add paging/filtering where the scan is avoidable. - Coordinate retention changes with #10767 rather than duplicating them. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
