rangareddy commented on issue #17292: URL: https://github.com/apache/hudi/issues/17292#issuecomment-5365816124
This issue was reviewed as part of the JIRA-migrated backlog triage (HUDI-8737). **Findings: routed to the performance validation pass; not decidable from code.** The body is a single line phrased as a question - "memory and resource usage - close readers that are cached for MDT?" - with no measurement, no OOM report and no named class, so there is nothing to confirm or refute by reading. What I can say is that there is a lifecycle hook to build on: `HoodieFileIndex.refresh()` calls `indicesSupport.foreach(idx => idx.invalidateCaches())`, so cached index state is at least reachable for cleanup. Whether cached metadata-table readers are actually retained beyond their useful life needs a heap profile on a real workload. **Possibly a duplicate:** #17378 (HUDI-9181), "cleaner and safer resource management around `hoodieMetadata` object, i.e. the reader for MT", is the same concern stated in different words and is also unspecified. Worth merging the two and attaching one concrete symptom - a retained-heap path or an OOM stack - to whichever survives. Keeping this open. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
