taoran92 opened a new issue, #10137:
URL: https://github.com/apache/paimon/issues/10137
### Search before asking
- [x] I searched in the issues and found nothing similar.
### Paimon version
master at `ad1e6c187817f23c2e20c15f6c59d5923fc5b6b0` (2.2-SNAPSHOT).
### Compute Engine
Standalone Java benchmark configuration loading in
`paimon-cluster-benchmark`; no running Flink job is required. The underlying
directory-stream behavior was reproduced on macOS with JDK 8.
### Minimal reproduce step
1. Create a temporary directory containing a `sinks` subdirectory. An empty
`sinks` directory is sufficient to exercise the directory enumeration.
2. Call `org.apache.paimon.benchmark.Sink.load(location)` repeatedly in the
same JVM.
3. Observe that the directory stream opened on each invocation is never
explicitly closed.
The relevant code is:
```java
Iterator<Path> it = Files.list(sinkLocation).iterator();
while (it.hasNext()) {
// Read and parse sink configuration files.
}
```
A separate JDK 8 probe of this exact API usage (`Files.list(...).iterator()`
followed by complete iteration) showed 20 additional open file descriptors
after 20 iterations. Explicitly closing the retained streams returned the
descriptor count to its baseline.
### What doesn't meet your expectations?
`Files.list` returns a resource-owning stream. Exhausting its iterator does
not close the underlying directory stream, and `Sink.load` discards the stream
reference without closing it. The directory resource is therefore left open
after normal completion, as well as when configuration-file reading or YAML
parsing throws an exception.
The stream should be closed on both normal and exceptional exits. Repeated
calls can accumulate directory resources. The current benchmark startup
normally loads sink configurations once, so this is a small resource-management
defect rather than evidence of widespread benchmark failures.
### Anything else?
Use try-with-resources around the `Stream<Path>` returned by `Files.list`,
keeping the entire existing iterator loop inside the block. This preserves
configuration-loading behavior and requires no new dependencies.
Affected file:
`paimon-benchmark/paimon-cluster-benchmark/src/main/java/org/apache/paimon/benchmark/Sink.java`.
### Are you willing to submit a PR?
- [x] I'm willing to submit a PR!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
