JackieTien97 opened a new pull request, #18788:
URL: https://github.com/apache/iotdb/pull/18788
## Description
A slow reader open or close currently holds the FileReaderManager monitor,
stalling reader lookup and reference operations for unrelated files. This
change moves reader I/O and lifecycle transitions to stable per-file entries,
so unrelated files can make progress while operations on the same file remain
serialized.
### Locking and lifecycle
- Keep separate closed/unclosed slots under each internal TsFileID entry and
a separate path namespace for external files. Hold the registry lock only for
admission, pinning, and reclamation; operation pins include threads waiting on
an entry, preventing replacement of an entry while a waiter still references it.
- Preserve resource-lock ordering and release the resource read lock when
reference registration fails or a release encounters a close failure. Retain a
failed-close handle for retry and do not serve it again; force close attempts
both slots.
- Serialize global test cleanup, block new admissions, and continue
accepting releases. Drain release pins again before atomically reclaiming
entries and reopening admission, restoring interruption only after cleanup.
Callers must stop reader use before global cleanup.
- Expose reader-map snapshots for tests and replace the metadata-cache
test's direct map mutation with a test-only reader setter.
### Validation
Validated locally with Amazon Corretto 17.0.5, Maven 3.9.9, and real cached
TsFile `2.4.1-260915-SNAPSHOT` artifacts (`20260915.102750-1`). Maven ran
offline with Develocity disabled in a fresh isolated worktree.
- Initial clean dependency-reactor test: FileReaderManagerConcurrencyTest
**15/15** and FileReaderManagerTest **1/1**, all passing.
- Final clean regression run: **99 tests, 0 failures, 0 errors, 1 skipped**
across 12 classes, including both FileReaderManager classes,
metadata/chunk/bloom-filter caches, TsFileResource, scan operators and
cancellation, resource management, device iteration, and inner compaction. The
skip is the existing `TimeSeriesMetadataCacheTest#testCachePlaceHolder`
annotated `@Ignore("Performance")`.
- `mvn clean test-compile -DskipTests`: **53 modules successful**.
- `mvn clean test-compile -P with-zh-locale -DskipTests`: **53 modules
successful**.
- `mvn spotless:apply -pl iotdb-core/datanode`, full-reactor `mvn
spotless:check`, and `git diff --check`: passed.
The 15 new regression tests cover slow open/close isolation, same-file
serialization, namespaces, constructor/close failures, reference rollback,
concurrent cleanup/releases, interruption, stable pinned entries, and
concurrent churn. Cleanup admission tests verify the actual WAITING state,
registry monitor, and pin/drain stack frame before releasing their gates.
Final test selection:
```sh
mvn clean test -pl iotdb-core/datanode -am \
-Dtest=FileReaderManagerConcurrencyTest,FileReaderManagerTest,TimeSeriesMetadataCacheTest,ChunkCacheTest,BloomFilterCacheTest,TsFileResourceTest,SeriesScanUtilCancellationTest,SeriesScanOperatorTest,AlignedSeriesScanOperatorTest,MultiTsFileDeviceIteratorTest,ReadChunkInnerCompactionTest,ResourceManagerTest
\
-Dsurefire.failIfNoSpecifiedTests=false -DfailIfNoTests=false
```
Distributed-cluster and object-storage end-to-end tests were not run.
<hr>
This PR has:
- [x] been self-reviewed for concurrent reader and lifecycle operations.
- [x] added comments explaining lock order and entry/pin lifetime.
- [x] added regression tests and adapted the existing metadata-cache test.
##### Key changed/added classes
`FileReaderManager`, `FileReaderManagerConcurrencyTest`,
`TimeSeriesMetadataCacheTest`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]