github-actions[bot] commented on code in PR #69729:
URL: https://github.com/apache/doris/pull/69729#discussion_r4399670194
##########
fe/fe-core/src/main/java/org/apache/doris/cloud/backup/CloudRestoreJob.java:
##########
@@ -438,7 +448,8 @@ public void createReplicas(Database db, OlapTable localTbl,
Partition restorePar
localTbl.getTimeSeriesCompactionEmptyRowsetsThreshold(),
localTbl.getTimeSeriesCompactionLevelThreshold(),
localTbl.disableAutoCompaction(),
localTbl.getRowStoreColumnsUniqueIds(rowStoreColumns),
-
localTbl.getInvertedIndexFileStorageFormat(),
+
restoredPartitionFormats.getOrDefault(restorePart.getId(),
Review Comment:
[P1] Preserve the backed-up format through restore commit
This changes only the initial create request. For the stated
SNII-backup-to-V2-table case (same schema version, rollout enabled), a
shared-nothing backup carries its format in the schema, without a top-level
format marker. `CloudSnapshotMgr::convert_rowsets` copies that backup metadata
and builds restored rowsets without setting either format override in
`RowsetWriterContext`. `commit_restore_job` then replaces the tablet metadata
and detaches the rowset schemas under the existing V2 `(index_id,
schema_version)` key. Once reloaded, both overrides are absent and `get_rowset`
returns V2 schemas for the SNII files, so `MATCH` still opens SNII files with
the V2 parser and can fail. Carry the source physical format into the committed
tablet and rowset metadata, and cover the download/commit/read path; the new
tests stop at `createTablets`.
##########
fe/fe-core/src/main/java/org/apache/doris/backup/RestoreJob.java:
##########
@@ -1558,6 +1561,27 @@ protected Partition resetPartitionForRestore(OlapTable
localTbl, OlapTable remot
return resetTabletForRestore(localTbl, remoteTbl, remotePart,
replicaAlloc);
}
+ // The inverted index storage format the files of a backed-up partition
were written in: the
+ // partition's own entry in the backed-up PartitionInfo, else the
backed-up table's default.
+ protected static TInvertedIndexFileStorageFormat
getBackedUpInvertedIndexFileStorageFormat(
+ OlapTable remoteTbl, long remotePartId) {
+ TInvertedIndexFileStorageFormat format =
+
remoteTbl.getPartitionInfo().getInvertedIndexFileStorageFormat(remotePartId);
+ return format != null ? format :
remoteTbl.getInvertedIndexFileStorageFormat();
+ }
+
+ // The restored files keep the format they were written in, which can
differ from the target table's
+ // default, and the tablets were created with it. Record it on the
partition the way a partition created
+ // while the partition-level format rollout is on has it, and only then:
without the rollout the
+ // format is per table and nothing reads a partition's entry.
+ protected static void
recordBackedUpInvertedIndexFileStorageFormat(PartitionInfo localPartitionInfo,
+ long restoredPartId, OlapTable remoteTbl, long remotePartId) {
+ if (Config.isCloudMode() &&
Config.enable_partition_inverted_index_storage_format_rollout) {
Review Comment:
[P2] Replay the restored format independently of the local rollout flag
The rollout is a mutable master-only config, so `ADMIN SET FRONTEND CONFIG`
can enable it on the master while followers remain false. After an SNII
partition is restored into a V2 table, the master records SNII, but `DOWNLOAD`
replay calls this helper and skips the entry on those followers. Enabling
rollout there later (including after promotion) cannot recover it: `FINISHED`
replay does not record it again, so `SHOW PARTITIONS` and subsequent
`CloudSchemaChangeJobV2` tablet creation use V2. Persist the master's recording
decision/value and replay it without consulting follower-local config, as
`replayAddPartition` already does.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
