devmadhuu opened a new pull request, #9009: URL: https://github.com/apache/ozone/pull/9009
## What changes were proposed in this pull request? Currently rebuilding of NSSummary data is based on multiple scenarios and being triggered across multiple code paths which might bring inconsistency. So to improve that, synchronized based approach was implemented so that one thread at a time should be running and building NSSummary, but now after staging and queue based implementation, this also should be updated to more simpler and consistent approach along with existing queue based tasks executions. In addition to above change, this PR change has also done few additional fixes and improvements: ``` - The reprocess task might terminate prematurely if shutdown is slow. executorService.shutdown() is called without awaitTermination. Potentially misleading success logs if shutdown is interrupted. P3 L Add executorService.awaitTermination with a reasonable timeout after shutdown(). - Slower reprocess execution due to thread starvation. Fixed-size thread pool (n=2) is smaller than the number of parallel tasks (n=3). Increased latency for full namespace reprocessing, delaying Recon's data availability. - When a thread is interrupted during Future.get(), the InterruptedException is caught, but the thread's interrupted status is not restored. ``` **Added few metrics:** ``` 1. reprocessDurationMs: A Timer to track the execution time of the reprocess method. 2. reprocessFailuresTotal: A Counter to track the number of failed reprocess attempts. 3. dbFlushSize: A Histogram to track the number of entries in each DB flush batch. 4. dbFlushFailure: A Counter for failed DB flushes. 5. executorActiveThreads: Number of active threads in NSSummary executor ``` ## What is the link to the Apache JIRA https://issues.apache.org/jira/browse/HDDS-13648 ## How was this patch tested? Tested manually using docker and existing junit and integration tests covering the change. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
