Akanksha-kedia opened a new pull request, #18964:
URL: https://github.com/apache/pinot/pull/18964
## Summary
Both `HnswVectorIndexCreator` and `MutableVectorIndex` open an `FSDirectory`
in their constructors and then immediately construct an `IndexWriter` over it.
If `IndexWriter` construction throws — e.g. due to a locked or corrupt index
directory, a codec initialization failure, or an `OutOfMemoryError` — the
already-opened `FSDirectory` is silently leaked: the `catch` block wraps and
rethrows the exception without ever calling `close()` on the directory.
On servers that repeatedly create or reload vector indexes under error
conditions, this accumulates unclosed file descriptors until the OS limit is
hit, causing new index creations to fail with "Too many open files".
### Root cause
```java
// Both files — same pattern
_indexDirectory = FSDirectory.open(indexFile.toPath()); // succeeds
_indexWriter = new IndexWriter(_indexDirectory, ...); // throws →
_indexDirectory leaks
```
### Fix
Use local variables so the directory is accessible in the `catch` block.
Close the directory (or writer, if construction succeeded) before rethrowing.
In `MutableVectorIndex`, also delete the temporary index directory so stale
`getTempDirectory()` subdirectories don't accumulate on failure.
```java
Directory indexDirectory = null;
IndexWriter indexWriter;
try {
indexDirectory = FSDirectory.open(indexFile.toPath());
indexWriter = new IndexWriter(indexDirectory, ...);
} catch (Exception e) {
if (indexDirectory != null) {
try { indexDirectory.close(); } catch (IOException ce) {
e.addSuppressed(ce); }
}
throw new RuntimeException("...", e);
}
_indexDirectory = indexDirectory;
_indexWriter = indexWriter;
```
## Files changed
- `HnswVectorIndexCreator.java` — offline segment generation path
- `MutableVectorIndex.java` — realtime ingestion path (also cleans up temp
dir on failure)
## Test plan
- [ ] Existing unit/integration tests pass
- [ ] `./mvnw checkstyle:check -pl pinot-segment-local` — 0 violations
cc @xiangfu0 @raghavyadav01 @Jackie-Jiang
🤖 Generated with [Claude Code](https://claude.com/claude-code)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
