luoluoyuyu opened a new pull request, #19169:
URL: https://github.com/apache/iotdb/pull/19169

   ## Problem
   
   `IoTDBLoadStorageTsFileWithCompactionIT` (added in #19100) walks the **whole 
DataNode data
   directory** and requires every TsFile it finds to carry the table schema of 
table `t1`. The
   TsFile counting helpers used by the compaction assertions are built on the 
same scan.
   
   The DataNode data directory is shared by every database of the node, 
including the internal
   databases that store system records. Those are tree-model databases whose 
TsFiles legitimately
   carry an empty table schema map, so as soon as the node holds one of them 
the test fails with
   
   ```
   TsFile has no schema entry for table t1: {}
   ```
   
   This reproduces on a downstream distribution that records the password 
history of `root` in
   `root.__audit` on first login: the audit TsFile is flushed next to the test 
database and is
   picked up by the scan. The failure message did not name the file, so it 
looked like a schema
   registration bug rather than an over-broad scan.
   
   The same scan also makes the compaction assertions unsound:
   
   * `countTsFiles(...) >= 2` (precondition before compaction) can be satisfied 
by a single data
     file plus an unrelated TsFile;
   * `until(countTsFiles(...) == 1)` and `findSingleTsFile(...)` can never hold 
while an unrelated
     TsFile is present.
   
   ## Change
   
   * add `listDatabaseTsFiles(env)`, which lists only the TsFiles under
     `<dataPath>/sequence/load_storage_db` and 
`<dataPath>/unsequence/load_storage_db`;
   * route `assertAllTsFilesHaveTableSchema`, `countTsFiles` and 
`findSingleTsFile` through it;
   * keep every assertion unchanged (nothing is relaxed) and name the offending 
TsFile in the
     failure messages.
   
   ## Test plan
   
   * Reproduced the original failure before the change, on a build where the 
DataNode also stores a
     tree-model system database, and confirmed it is gone with the change while 
that database still
     exists (i.e. the fix is in the scan scope, not in a weakened assertion).
   * `mvn spotless:apply -pl integration-test -P with-integration-tests`
   * `mvn verify -DskipUTs -Dit.test=IoTDBLoadStorageTsFileWithCompactionIT 
-DfailIfNoTests=false -Dfailsafe.failIfNoSpecifiedTests=false -pl 
integration-test -P with-integration-tests`
     → `Tests run: 1, Failures: 0, Errors: 0`
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to