JackieTien97 opened a new pull request, #18788:
URL: https://github.com/apache/iotdb/pull/18788

   ## Description
   
   A slow reader open or close currently holds the FileReaderManager monitor, 
stalling reader lookup and reference operations for unrelated files. This 
change moves reader I/O and lifecycle transitions to stable per-file entries, 
so unrelated files can make progress while operations on the same file remain 
serialized.
   
   ### Locking and lifecycle
   
   - Keep separate closed/unclosed slots under each internal TsFileID entry and 
a separate path namespace for external files. Hold the registry lock only for 
admission, pinning, and reclamation; operation pins include threads waiting on 
an entry, preventing replacement of an entry while a waiter still references it.
   - Preserve resource-lock ordering and release the resource read lock when 
reference registration fails or a release encounters a close failure. Retain a 
failed-close handle for retry and do not serve it again; force close attempts 
both slots.
   - Serialize global test cleanup, block new admissions, and continue 
accepting releases. Drain release pins again before atomically reclaiming 
entries and reopening admission, restoring interruption only after cleanup. 
Callers must stop reader use before global cleanup.
   - Expose reader-map snapshots for tests and replace the metadata-cache 
test's direct map mutation with a test-only reader setter.
   
   ### Validation
   
   Validated locally with Amazon Corretto 17.0.5, Maven 3.9.9, and real cached 
TsFile `2.4.1-260915-SNAPSHOT` artifacts (`20260915.102750-1`). Maven ran 
offline with Develocity disabled in a fresh isolated worktree.
   
   - Initial clean dependency-reactor test: FileReaderManagerConcurrencyTest 
**15/15** and FileReaderManagerTest **1/1**, all passing.
   - Final clean regression run: **99 tests, 0 failures, 0 errors, 1 skipped** 
across 12 classes, including both FileReaderManager classes, 
metadata/chunk/bloom-filter caches, TsFileResource, scan operators and 
cancellation, resource management, device iteration, and inner compaction. The 
skip is the existing `TimeSeriesMetadataCacheTest#testCachePlaceHolder` 
annotated `@Ignore("Performance")`.
   - `mvn clean test-compile -DskipTests`: **53 modules successful**.
   - `mvn clean test-compile -P with-zh-locale -DskipTests`: **53 modules 
successful**.
   - `mvn spotless:apply -pl iotdb-core/datanode`, full-reactor `mvn 
spotless:check`, and `git diff --check`: passed.
   
   The 15 new regression tests cover slow open/close isolation, same-file 
serialization, namespaces, constructor/close failures, reference rollback, 
concurrent cleanup/releases, interruption, stable pinned entries, and 
concurrent churn. Cleanup admission tests verify the actual WAITING state, 
registry monitor, and pin/drain stack frame before releasing their gates.
   
   Final test selection:
   
   ```sh
   mvn clean test -pl iotdb-core/datanode -am \
     
-Dtest=FileReaderManagerConcurrencyTest,FileReaderManagerTest,TimeSeriesMetadataCacheTest,ChunkCacheTest,BloomFilterCacheTest,TsFileResourceTest,SeriesScanUtilCancellationTest,SeriesScanOperatorTest,AlignedSeriesScanOperatorTest,MultiTsFileDeviceIteratorTest,ReadChunkInnerCompactionTest,ResourceManagerTest
 \
     -Dsurefire.failIfNoSpecifiedTests=false -DfailIfNoTests=false
   ```
   
   Distributed-cluster and object-storage end-to-end tests were not run.
   
   <hr>
   
   This PR has:
   - [x] been self-reviewed for concurrent reader and lifecycle operations.
   - [x] added comments explaining lock order and entry/pin lifetime.
   - [x] added regression tests and adapted the existing metadata-cache test.
   
   ##### Key changed/added classes
   
   `FileReaderManager`, `FileReaderManagerConcurrencyTest`, 
`TimeSeriesMetadataCacheTest`.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to