shuwenwei commented on code in PR #18814:
URL: https://github.com/apache/iotdb/pull/18814#discussion_r4236315111
##########
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/execution/fragment/QueryContext.java:
##########
@@ -145,7 +145,7 @@ public PatternTreeMap<ModEntry, ModsSerializer>
loadAllModificationsFromDisk(
}
protected boolean shouldSkipModification(ModEntry modification) {
- if (tables != null && modification instanceof TableDeletionEntry) {
+ if (isTableModel() && modification instanceof TableDeletionEntry) {
Review Comment:
Fixed in 36683baf596: `shouldSkipModification` only skips a
`TableDeletionEntry` when a table filter was collected (`tables != null`), so a
table-model context that never collected one (compaction, and also the table
`last_by` aggregation scan) keeps every modification instead of throwing. That
NPE is exactly what the CI failure in
`IoTDBDeletionTableIT.testLastByOverDeletedDeviceStillReturnsSingleNullRow`
turned out to be: the scan's catch-all wrapped it into a misleading "TsFile may
be corrupted" error, and the guard is the fix for it.
##########
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/tools/ImportWAL.java:
##########
@@ -949,17 +972,28 @@ static class WALReplayer implements WALReplayWorker {
new ReplayDecisionController(System.console()));
}
+ /**
+ * @param treeSession the session which replays tree model entries, it
carries no database
+ * @param tableSession the session which replays table model entries, null
when no table
+ * database is declared
+ * @param tableDatabaseName the target database resolved before replay;
null means no table
+ * database is declared, so the entries are replayed with the tree
model only
+ * @param replayDecisionPrompt the prompt which decides how to handle
entries that cannot be
+ * replayed automatically
+ */
WALReplayer(
final Session treeSession,
final Session tableSession,
final String tableDatabaseName,
final ReplayDecisionPrompt replayDecisionPrompt) {
this.treeSession = treeSession;
this.tableSession = tableSession;
+ this.isTableModel =
+ tableDatabaseName != null &&
PathUtils.isTableModelDatabase(tableDatabaseName);
Review Comment:
As you say, this is a pre-existing limitation of the tool rather than
something this PR introduces. A WAL entry does not carry a database (for table
data the database comes from the client session), so when several table-model
databases share one WAL node a single `-db/--database` is inherently
insufficient - and the same is true for a WAL node shared by tree and table
regions, where one model per directory cannot describe the content. Solving it
needs per-region resolution (WAL checkpoint `memTableId -> regionId` plus a
`region -> database` mapping) or a database/model marker in the WAL entry,
which is beyond this PR's scope, so we are not attempting it here. What the PR
does is state the limitation where the operator sees it: the tool now declares
in its help header - and repeats whenever the target database of a WAL
directory cannot be determined - that WALs written with the
`ElasticStrategy`/`RoundRobinStrategy` WAL node allocation strategies cannot be
imported.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]