Hi Zihao, Thanks for starting this discussion. The lazy loading approach addresses real pain points we've seen with HistoryServer in production, especially the slow startup and delayed job visibility when handling a large number of archived jobs.
A few thoughts: - Keeping EAGER as the default is a good call for backward compatibility. - The idea of making jobs visible after overview parsing, rather than waiting for all details to load, is sensible and should meaningfully improve the user experience. - The two-thread-pool design with priority separation looks reasonable. The benchmark numbers look promising. Looking forward to the community's feedback and further discussion. Thanks for working on this! Best regards, Verne Deng On 2026/05/09 03:43:22 zihao chen wrote: > Hi all, > > I’d like to start a discussion on FLIP-XXX: > *Introduce Lazy Loading for HistoryServer*. > > This FLIP proposes improving the HistoryServer > to address *slow startup*, *delayed job visibility* > when handling large numbers of archived jobs. > > [Proposal] > *Lazy loading mode* for on-demand job archive fetching > > [Compatibility] > Full backward compatibility (EAGER as default) > > The detailed design is described in the > FLIP document: > > https://docs.google.com/document/d/11YOivaOMOAoiG-zc6jjLoiUQqAwKoKmYgsgPtuZYo88/edit?usp=sharing > > This FLIP is split from the earlier discussion [1]. > > Looking forward to your feedback. > > [1] https://lists.apache.org/thread/6thlq9c5twyvzmcw7q24nm4q0rcbz5qp > > > Best regards, > > Zihao Chen >
