gl commented on issue #18446: URL: https://github.com/apache/druid/issues/18446#issuecomment-4886506214
We have a similar need with large historicals (~12k segments on PVC) where rolling restarts serialise on the eager load (minutes per pod) and lazyLoadOnStart=true shifts the cost to a first-query latency spike. The hot/cold split (loadEagerlyForPeriod) would help when the hot window matches query patterns, but there's a third useful strategy that's complementary rather than competing: load lazily then warm in the background: the historical becomes ready immediately (mmap only) and then warms column metadata on a low-priority pool ahead of queries, so the first query isn't penalised and cold/recent-unqueried segments still get warmed proactively. It would fit naturally as a fourth startupCacheLoadStrategy option and avoids the need to define "hot" (which is workload-dependent and can be wrong for low-traffic datasources where recent does not equal queried). Would be great to see this included alongside the hot/cold proposal. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
