Caideyipi opened a new pull request, #19031:
URL: https://github.com/apache/iotdb/pull/19031

   ## Description
   
   Concurrent LOAD snapshots can copy a progress file before its header has 
been flushed, causing 
`LoadTsFileSnapshotTest.testSnapshotTakenWhilePiecesAreAppliedStaysRestorable` 
to fail with an EOF error. Repair incomplete headers and reinitialize empty 
progress files on the next append. Bound tail scans by the captured file length 
so truncated entries larger than the read buffer are detected as well.
   
   Also copy staged files created between the data-file and progress-log 
directory scans. Previously, catch-up skipped a missing snapshot data file and 
could leave a progress log without the data it describes.
   
   Add deterministic regression coverage for header and entry truncation, large 
entries, and files appearing between directory scans. Coordinate the concurrent 
snapshot test with futures so writer errors propagate and file growth is 
observed independently of thread scheduling.
   
   ## Validation
   
   - `mvn spotless:apply -P with-integration-tests -pl iotdb-core/datanode`.
   - Clean DataNode dependency reactor build with build-output caches disabled; 
all 74 tests passed across `LoadTsFileProgressTest`, `LoadTsFileSnapshotTest`, 
`LoadTsFileManagerTest`, `LoadTsFileResumeTest`, `LoadTsFileCleanerTest`, and 
`LoadTsFileManagerSplitCoverageTest`.
   - The concurrent snapshot regression passed 30 consecutive JUnit runs 
against those freshly built classes.
   - `mvn clean test-compile -pl iotdb-core/datanode -am -P with-zh-locale 
-DskipTests` passed with build-output caches disabled. Both locale builds 
rebuilt all 28 required IoTDB reactor modules from source on JDK 17.
   - Confirmed the incomplete-header EOF, large torn-entry, and missing 
staged-file regressions against the preceding implementations before applying 
their fixes.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to