devmadhuu opened a new pull request, #9226:
URL: https://github.com/apache/ozone/pull/9226
## What changes were proposed in this pull request?
Based on attached `alloc_profile.html`, this PR change is to optimize
NSSummary Memory Allocations in Recon Task Processing.
The NSSummary task was experiencing performance bottlenecks when processing
OM DB events due to:
- Excessive object allocation and garbage collection pressure
- Redundant database reads for the same parent/directory entries
- Inefficient collection handling (creating new objects on every access)
**Performance Impact**
**Expected Improvements:**
1. 60-75% reduction in allocations during event processing
2. Significant DB read reduction - cache hit rate ~90%+ for hot parents
3. Lower GC pressure - fewer short-lived objects
4. Faster processing - in-place mutations vs object creation
**Trade-offs:**
1. Memory usage - 32K LRU cache (but small compared to benefits)
2. Code complexity - more helper methods
3. API changes - getChildDir() never returns null (breaking change)
**Why This Matters**
For large-scale Ozone deployments:
- High event volume - thousands of file operations per second
- Deep directory trees - same parent accessed repeatedly
- GC sensitivity - reducing allocations critical for latency
- Resource efficiency - less CPU/memory for background tasks
[alloc_profile.html](https://github.com/user-attachments/files/23249912/alloc_profile.html)
## What is the link to the Apache JIRA
Please create an issue in ASF JIRA before opening a pull request, and you
need to set the title of the pull
request which starts with the corresponding JIRA issue number. (e.g.
HDDS-XXXX. Fix a typo in YYY.)
(Please replace this section with the link to the Apache JIRA)
## How was this patch tested?
(Please explain how this patch was tested. Ex: unit tests, manual tests,
workflow run on the fork git repo.)
(If this patch involves UI changes, please attach a screenshot; otherwise,
remove this.)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
