ArafatKhan2198 commented on PR #9243: URL: https://github.com/apache/ozone/pull/9243#issuecomment-3579914381
Hey @sumitagrawl and @devmadhuu ! Following up on our last discussion, I've implemented the requested changes: - **Removed unnecessary locks and flags in ContainerKeyMapper and FileSizeCountTask** - Eliminated the isFlushingInProgress flags since workers block at read lock acquisition during flushes anyway. Also consolidated multiple locks into a single INITIALIZATION_LOCK that handles both table truncation and shared map clearing operations. - **Fixed shared map cleanup to prevent memory leaks** - Added proper reference counting so the last completing task clears the shared container count map. - **Verified no thread starvation scenarios** - Analyzed the lock acquisition patterns and confirmed that once the flush threshold is hit, all workers naturally queue for the write lock instead of acquiring new read locks, preventing writer starvation. - **Made locks task-local rather than global** - Each FileSizeCountTask instance now has its own `ReentrantReadWriteLock`, eliminating unnecessary cross-task synchronization since FSO and OBS write to different keys. - **Added parallelization to OmTableInsightTask simple tables** - Implemented parallel counting (5 iterators + 2 workers) for tables that only need entry counts. Tables with non-String keys (dTokenTable, s3SecretTable) fall back to sequential processing to avoid codec mismatches. I've tested the changes and they're working correctly. I've also prepared a brief documentation (attached) explaining how each task's parallelization works. Could you please review the code changes and the documentation? If you spot any potential issues with the logic or areas where the implementation could fail, let's discuss them. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
