PDGGK opened a new pull request, #9481:
URL: https://github.com/apache/paimon/pull/9481
### Purpose
`FileBackedDataset` opens the vector file and then does work that can fail:
```java
FileBackedDataset(File file, int dim, long totalCount, String phase, int
bufferSize)
throws IOException {
this.raf = new RandomAccessFile(file, "r");
this.channel = raf.getChannel();
...
this.recordSizeInBytes = checkedRecordSize(dim, bufferSize);
...
this.readBuf = ByteBuffer.allocateDirect(bufferSize);
```
`checkedRecordSize` throws `IllegalStateException` when a record does not
fit the read buffer, and `allocateDirect` can fail on its own. Both run after
the file is open.
Both call sites use try-with-resources:
```java
try (FileBackedDataset dataset = new FileBackedDataset(tempVectorFile, dim,
count, "pretrain")) {
```
but try-with-resources never receives an object whose constructor threw, so
its `close()` never runs and the handle is stranded.
The enclosing writer already guards its own file this way at `:112`, so this
brings the nested reader in line.
### Tests
`LuminaFileBackedDatasetCloseTest#testFailedConstructionReleasesTheFile`
runs 200 failed constructions with a dimension the buffer cannot hold, counting
entries in `/dev/fd` before and after. It skips itself where that directory is
not available.
Reverting the change turns it red: `[200 failed constructions must not
strand 200 descriptors]`.
A note on how the assertion got there, since the obvious versions do not
work. Asserting that the exception is raised passes either way. Letting the
descriptors run out does not work either — the `RandomAccessFile` finalizer
releases them under GC pressure, so twenty thousand leaked handles still never
exhaust the limit and the test stays green with the bug in place. Counting open
descriptors directly is what actually separates the two.
`mvn test -pl paimon-lumina` passes; spotless and checkstyle clean.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
