Hi Zihao, Thanks for initiating this FLIP. I think this proposal could be very helpful for improving the performance and overall user experience of accessing archived jobs in Flink.
>From the current proposal, lazy loading and a pluggable local storage backend seem like two relatively independent pieces of functionality. Would it make sense to split them into two separate FLIPs? Best, Feng On Mon, Mar 23, 2026 at 8:02 PM zihao chen <[email protected]> wrote: > Hi all, > > I’d like to start a discussion on *FLIP-XXX: Enhancing HistoryServer with > Lazy Loading and Pluggable Storage*. > > This FLIP proposes improving the HistoryServer to address *slow > startup*, *delayed > job visibility*, and *excessive small files* when handling large numbers of > archived jobs. > > The proposal introduces: > > - *Lazy loading mode* for on-demand job archive fetching > - Optional *RocksDB-based storage* to reduce small files > > Compatibility: Full backward compatibility (EAGER + FILE as default) > > The detailed designs are described in the FLIP document: > > > https://docs.google.com/document/d/1FQn-HQSIdhio3ImeOrbS9VfB6JqRTrRV4ZEZDTf3K2o/edit?usp=sharing > > Looking forward to your feedback. > > > Best regards, > > Zihao Chen >
