This is an automated email from the ASF dual-hosted git repository.
w41ter pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/master by this push:
new b53b8c9c115 [fix](restore) Reset next version for remote table when
restore (#40118)
b53b8c9c115 is described below
commit b53b8c9c115485f3912914e347e2c8dd45dde632
Author: smallx <[email protected]>
AuthorDate: Fri Aug 30 10:24:42 2024 +0800
[fix](restore) Reset next version for remote table when restore (#40118)
We should reset next version to visible version + 1 for all partitions
of remote table, when restoring table that do not exist locally.
在高并发insert场景，CCR源端表的next version可能比visible
version大比较多，目标集群restore全量快照后，切换到增量binlog后，commit事务时使用的version（来自next
version）就可能比当前visible version大比较多。
此时，对于MoW表，就会出现publish
version不连续，增量binlog一直无法publish的问题。事务状态会一直是`COMMITTED`，并伴随类似ErrMsg `wait
for publishing partition 15027 version 1037597. self version: 1037627.
table 15025`。
---
fe/fe-core/src/main/java/org/apache/doris/backup/RestoreJob.java | 3 +++
fe/fe-core/src/main/java/org/apache/doris/catalog/OlapTable.java | 6 ++++++
2 files changed, 9 insertions(+)
diff --git a/fe/fe-core/src/main/java/org/apache/doris/backup/RestoreJob.java
b/fe/fe-core/src/main/java/org/apache/doris/backup/RestoreJob.java
index b7880302442..12a30a97277 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/backup/RestoreJob.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/backup/RestoreJob.java
@@ -771,6 +771,9 @@ public class RestoreJob extends AbstractJob implements
GsonPostProcessable {
return;
}
+ // reset next version to visible version + 1 for all
partitions
+ remoteOlapTbl.resetVersionForRestore();
+
// Reset properties to correct values.
remoteOlapTbl.resetPropertiesForRestore(reserveDynamicPartitionEnable,
reserveReplica,
replicaAlloc,
isBeingSynced);
diff --git a/fe/fe-core/src/main/java/org/apache/doris/catalog/OlapTable.java
b/fe/fe-core/src/main/java/org/apache/doris/catalog/OlapTable.java
index c4414e17d39..78d97e8e48b 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/catalog/OlapTable.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/catalog/OlapTable.java
@@ -653,6 +653,12 @@ public class OlapTable extends Table implements
MTMVRelatedTableIf, GsonPostProc
setColocateGroup(null);
}
+ public void resetVersionForRestore() {
+ for (Partition partition : idToPartition.values()) {
+ partition.setNextVersion(partition.getVisibleVersion() + 1);
+ }
+ }
+
public Status resetIdsForRestore(Env env, Database db, ReplicaAllocation
restoreReplicaAlloc,
boolean reserveReplica, String srcDbName) {
// ATTN: The meta of the restore may come from different clusters, so
the
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
