clintropolis commented on issue #14796: URL: https://github.com/apache/druid/issues/14796#issuecomment-1673821303
Seems reasonable to make sure the new lookup is good before dropping the old one, though I'm not entirely certain how to check this.. what do you have in mind? It would be nice to make this more "proposal-ish" and include what changes are actually needed once you have a design. As for ideas (in case you need any) i guess we could make sure the `LookupExtractorFactory` started successfully, or maybe does it need a new method added to the interface like an `isInitialized` or something else? Also, if the lookup implementation doesn't load everything in heap like the built-in ones do, how do we know its in a good state in that case? With a new method I suppose those lookups could just do whatever made sense to make sure they can do lookup stuff. >Step 3: Delay the Drop notice execution until AddNotice loads the lookup on the current node, and make sure that we have one latest lookup loaded successfully and good to drop the previous one. This can be done by starting the schedule executor thread here https://github.com/apache/druid/blob/master/server/src/main/java/org/apache/druid/query/lookup/LookupReferencesManager.java#L675 that can execute it after Delay D, N times and it can get the latest stateRef from LookupReferenceManager and make sure that latest Ref cache is loaded successfully (Also make sure that LookupExtractorFactoryContainer that we are trying to remove is not same as current stateRef container). Will we need to split up the 'drop because i got a new version' from the 'drop because we no longer want this lookup loaded anywhere' cases? Or how do you envision that working? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
