kfaraz commented on PR #17653: URL: https://github.com/apache/druid/pull/17653#issuecomment-2641793027
> @kfaraz this is listed under "pending items". What race conditions are you aware of? Thanks for calling this out, @gianm . There were some race conditions that I had identified but I have handled them now. I have updated the PR description accordingly. For the race conditions and corner cases, there are also relevant comments/javadocs in the code. - Races between two different transactions initiated by `IndexerSQLMetadataStorageCoordinator`. The intial design did not acquire a lock on the cache for the entirety of a transaction, but this has been fixed now. (see `SqlSegmentMetadataTransactionFactory`) - Race between polling from metadata store and writing to cache. The polling thread could try to remove a segment ID from cache if it was not found in the latest poll from metadata store. But this logic has been modified to remove only the segments which were last updated _before_ the poll started, so that we don't remove something which has just been added to cache (and thus is not included in the latest poll results). - Other minor stuff found during unit testing, which has already been handled. I will take another pass through the code, just to ensure that I haven't missed anything, adding comments where necessary. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
