[ 
https://issues.apache.org/jira/browse/FLINK-41286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18148175#comment-18148175
 ] 

Sumin Joo commented on FLINK-41286:
-----------------------------------

Hi [~chenzihao] , I tested FLIP-585 on release-2.4 (5dc31f929a2) with a local 
standalone cluster.  

 

1. Job list appears early: passed (FILE and ROCKSDB). 

Time until /jobs/overview lists all 59 jobs: FILE EAGER 365 s vs LAZY 3.0 s; 
ROCKSDB EAGER 3.0 s vs LAZY 2.3 s (one run per case).

 

2. Opening a not-yet-downloaded job: does not behave as documented for jobs 
that have an application ID (FILE storage). 

*GET /jobs/<id>/vertices/<vid>* right after start returns 404 within 3-19 ms 
instead of waiting; the same request succeeds only after the background task 
finishes the job (about 46-89 s here).

The same archives copied to <dir>/<jobId> (no application ID) wait 50-58 s and 
return 200, as the FLIP describes.

Cause:

AbstractHistoryServerHandler#loadResource (lines 339-354) asks only the job 
HistoryServerArchiveFetcher (needLazyLoadIndividually, 
waitLazyFetchArchiveFinished).

Jobs with an application ID are scheduled by 
HistoryServerApplicationArchiveFetcher, which keeps the task in its own map, so 
the handler finds nothing to wait for or to prioritize. 

 

3. ROCKSDB: the same JUnit scenario gives the same result as FILE. 

 

Could you confirm whether jobs with an application ID are meant to get the 
wait/prioritize behavior in LAZY mode?


Thank you!

> Release Testing: Verify FLIP-585: Introduce Lazy Loading for HistoryServer
> --------------------------------------------------------------------------
>
>                 Key: FLINK-41286
>                 URL: https://issues.apache.org/jira/browse/FLINK-41286
>             Project: Flink
>          Issue Type: Sub-task
>            Reporter: chenzihao
>            Assignee: Sumin Joo
>            Priority: Blocker
>              Labels: release-testing
>             Fix For: 2.4.0
>
>
> Options:
> - \{{historyserver.archive.load.mode}} = EAGER (default) | LAZY
> Steps:
> 1. Prepare a large number of archives (e.g a complex topology or multiple 
> failovers).
> 2. Start with LAZY; verify the job list appears quickly, before all downloads 
> finish.
> 3. Open a not-yet-downloaded job; verify it is fetched on demand and renders 
> correctly.
> 4. Test LAZY with both FILE and ROCKSDB storage types.



--
This message was sent by Atlassian Jira
(v10.3.26#10030026)

Reply via email to