mcvsubbu commented on pull request #6250: URL: https://github.com/apache/incubator-pinot/pull/6250#issuecomment-725006959
> Can we separate the unrelated changes to another PR? > > IMO, the lazy load should be implemented as following: > > * Load the metadata, but keep the index in deep store > * When segment gets queried, load index on demand > > The current way of downloading all the segments, uncompress and load them at query time won't really work. @kishoreg wdyt? I agree. Another variant (if we want to escape the download part of latency) is to download it during state transition, but not load it in memory. This variant can be useful in cases where an occasional query may cause some page churns but otherwise the working set returns to (say) most recent data. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
