pranavbhole commented on issue #14796: URL: https://github.com/apache/druid/issues/14796#issuecomment-1673957032
Thank you @clintropolis, Currently, Load Notice Handler tries to instantiate and start the new LookupExtractorFactoryContainer and start the extractor factory. Start is non blocking call, it just start the scheduler today and does not wait for the actual cache to load. Idea it to change the LoadNotice and wait for actual first cache to load using firstCacheLoadLatch in CacheScheduler, LoadNotice Handler: 1. If loadNotice is first time load then just go ahead and start container and do not wait for first cache load. 1. If it is Load Notice with update call, it will instantiate the new LookupExtractorFactoryContainer and wait for firstCacheLoadLatch (Need to expose the method at LookupExtractorFactory), We can not indefinitely to load the firstCache, we need new namespace config with default value would be 10 mins , `lookupLoadTimeout = 10` mins. 2. After cache load is successful, it will update the stateRef with new lookup state and with new container. Drop Notice Handler: 1. If drop notice is final lookup drop call then go ahead drop it directly 2. Once we receive the drop notice from update flow, we will peek into reference manager's stateRef and make sure that corresponding new container for given lookup is loaded (using wrapper method that check firstCacheLoadLatch in Cache Scheduler) 3. After new container is loaded successfully then we will go ahead remove the old container from lookupMap and clean up old container -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
