clintropolis commented on issue #6988: [Improvement] historical fast restart by lazy load columns metadata(20X faster) URL: https://github.com/apache/incubator-druid/pull/6988#issuecomment-460871946 Interesting idea, I too have felt the pain of multi-day rollouts for tiers of densely loaded historical servers. I've only scanned changes so far, I'll try to do a full review later. In the mean time, if possible could you run query benchmarks before this patch and after this patch with `lazyLoadOnStart = false`, as a sanity check to make sure that there are not any odd performance effects of introducing the memoized suppliers? I don't really _expect_ any noticeable effect, but it would make buy in to this change easier. (No worries if you don't have a chance to get to it, i'll try to do this myself once I get around to full review). I would also be interested in the performance cost for queries that _do_ have to eat the deserialization when `lazyLoadOnStart = true`, and whether it opens up scenarios where the memoizing supplier becomes a sort of choke point for the processing pool if the cost is high.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
