[
https://issues.apache.org/jira/browse/FLINK-41286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18148175#comment-18148175
]
Sumin Joo commented on FLINK-41286:
-----------------------------------
Hi [~chenzihao] , I tested FLIP-585 on release-2.4 (5dc31f929a2) with a local
standalone cluster.
1. Job list appears early: passed (FILE and ROCKSDB).
Time until /jobs/overview lists all 59 jobs: FILE EAGER 365 s vs LAZY 3.0 s;
ROCKSDB EAGER 3.0 s vs LAZY 2.3 s (one run per case).
2. Opening a not-yet-downloaded job: does not behave as documented for jobs
that have an application ID (FILE storage).
*GET /jobs/<id>/vertices/<vid>* right after start returns 404 within 3-19 ms
instead of waiting; the same request succeeds only after the background task
finishes the job (about 46-89 s here).
The same archives copied to <dir>/<jobId> (no application ID) wait 50-58 s and
return 200, as the FLIP describes.
Cause:
AbstractHistoryServerHandler#loadResource (lines 339-354) asks only the job
HistoryServerArchiveFetcher (needLazyLoadIndividually,
waitLazyFetchArchiveFinished).
Jobs with an application ID are scheduled by
HistoryServerApplicationArchiveFetcher, which keeps the task in its own map, so
the handler finds nothing to wait for or to prioritize.
3. ROCKSDB: the same JUnit scenario gives the same result as FILE.
Could you confirm whether jobs with an application ID are meant to get the
wait/prioritize behavior in LAZY mode?
Thank you!
> Release Testing: Verify FLIP-585: Introduce Lazy Loading for HistoryServer
> --------------------------------------------------------------------------
>
> Key: FLINK-41286
> URL: https://issues.apache.org/jira/browse/FLINK-41286
> Project: Flink
> Issue Type: Sub-task
> Reporter: chenzihao
> Assignee: Sumin Joo
> Priority: Blocker
> Labels: release-testing
> Fix For: 2.4.0
>
>
> Options:
> - \{{historyserver.archive.load.mode}} = EAGER (default) | LAZY
> Steps:
> 1. Prepare a large number of archives (e.g a complex topology or multiple
> failovers).
> 2. Start with LAZY; verify the job list appears quickly, before all downloads
> finish.
> 3. Open a not-yet-downloaded job; verify it is fetched on demand and renders
> correctly.
> 4. Test LAZY with both FILE and ROCKSDB storage types.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)