This is an automated email from the ASF dual-hosted git repository.
bobhan1 pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/master by this push:
new 08231e3a635 [fix](cloud) Fix rowset notification cleanup and update
cloud load defaults (#70218)
08231e3a635 is described below
commit 08231e3a6359082264c88e6fe2c8050e911e5236
Author: bobhan1 <[email protected]>
AuthorDate: Tue Nov 10 14:11:43 2026 +0800
[fix](cloud) Fix rowset notification cleanup and update cloud load defaults
(#70218)
### Background
After a cloud load transaction commits in Meta Service, FE can notify
the participating BEs of its visible versions. BE uses cached committed
rowsets or lightweight empty-rowset entries to update local tablet
metadata, with normal metadata synchronization as the fallback.
### What problem does this PR solve?
Issue Number: None
Related PR: #59754 (introduced the notification path), #70209 (fixes the
unrelated Parquet unit-test build failure and is now included in master)
PR Type: Bugfix
Problem Summary: Completed notification tasks retain their registered
signatures, including when the BE feature is disabled. Consumed
empty-rowset markers also remain cached because removal only covers the
separate non-empty-rowset map. Release signatures on every callback exit
and represent both rowset kinds in one cache so removal and expiration
share the same lifecycle. Enable the existing FE and BE notification
settings by default, and default
cloud_mow_sync_rowsets_when_load_txn_begin to false so cloud MoW loads
skip forced synchronization when they begin. Remove the six
now-redundant FE/BE overrides from the Cloud p0/p1 pipeline configs so
those pipelines exercise the new defaults with unchanged effective
settings.
### Details
- **Frequency:** Every accepted notification leaves its signature
behind; every consumed empty entry remains until expiration.
- **Root cause:** The callback does not pair task registration with
removal, and the split empty/non-empty caches have different removal
behavior.
- **Why existing tests missed it:** They cover insertion and expiration
but do not check removal of an empty entry or resubmission after a
notification callback completes.
- **Fix:** Add a Defer guard before the BE enable check. Store empty
rowsets as null metadata in the existing committed-rowset cache, reuse
insertion and expiration, and avoid a redundant lookup during cleanup.
- **Other paths and branches:** The guard covers enabled notifications,
disabled notifications and cache misses. Non-empty and empty committed
rowsets now share cleanup; the independent MoW delete-bitmap cache is
unchanged. This change targets master.
- **Behavior change:** `enable_notify_be_after_load_txn_commit` and
`enable_cloud_make_rs_visible_on_be` now default to `true`;
`cloud_mow_sync_rowsets_when_load_txn_begin` now defaults to `false`.
All three settings retain their existing dynamic mutability. The cloud
rowset-builder test fixture now restores the incoming setting rather
than hard-coding `true` at teardown.
- **Concurrency and lifecycle:** Cache operations retain the existing
shared mutex and acquire no new nested locks. Signature cleanup runs
after tablet processing has released its locks. Refreshed entries remain
protected from older expiration records.
- **Compatibility and data correctness:** RPC fields and persistent
formats are unchanged. Cloud MoW loads continue to use a local flush
snapshot. Skipping initialization synchronization enlarges the existing
snapshot freshness window; the partial-update validation and
publish-alignment algorithms are unchanged. The rolling-upgrade caveat
is described below.
- **Observability and cost:** Move per-rowset insertion logging outside
the cache lock and lower it to DEBUG as notifications become enabled by
default. Existing notification and expiration logs retain
transaction/tablet identifiers.
### Testing
- The implementation commits were rebased onto master
`87b462e4f58104140f098f5f4a8b5d58bec70149`. The BE/FE runs below tested
implementation head `17d42f6ef184c7f851920830ad3ce4861fe679aa`. Final
head `4ba6d0db1480dd018a2e9353cffbb6137fe1c65d` adds only the
pipeline-configuration cleanup.
- On the rebased head, `./run-be-ut.sh --run
--filter='CloudCommittedRSMgrTest.*:EnabledAndDisabled/CloudCommittedRowsetTaskTest.*:TaskWorkerPoolTest.*:CloudTabletApplyVisiblePendingTest.*:CloudGroupRowsetBuilderWriterTest.*:*DeltaWriterCancelIntegrationTest.*'
-j100` passed all 46 ASAN tests with the requested defaults.
- On the same rebased head, `./run-fe-ut.sh --run
org.apache.doris.cloud.transaction.CloudGlobalTransactionMgrTest` passed
all 56 tests (0 failures, errors or skips); the Maven reactor completed
successfully.
- The BE build compiled both `task_worker_pool_test.cpp` and
`vparquet_writer_test.cpp` successfully. The latter uses the #70209 fix
already merged into master; no temporary source patch was applied.
- Before the original cleanup fix, the empty-entry removal test and both
enabled/disabled signature-release tests failed; the expiration-refresh
test passed.
- `build-support/check-build-hygiene.sh` and `git diff --check` passed
after rebase. The unchanged PR patch previously passed clang-format
v16/check-format and FE fe-common Maven validate/Checkstyle.
- For the final pipeline cleanup, checked all six removed values against
the FE/BE defaults, compared the remaining parsed settings in all four
files, and ran `git diff --check`: all passed. BE/FE tests were not
rerun for this config-only commit because its effective pipeline values
are unchanged.
- clang-tidy was not rerun, as requested. A cloud end-to-end regression
and a standalone production BE build were not run. The snapshot-window
and mixed-version behavior below was assessed from code, without a
running-cluster reproduction.
### Notes for Reviewers
Local review of `4ba6d0db1480dd018a2e9353cffbb6137fe1c65d` found no
Blocker or Major issue. One non-blocking P2 remains for rolling
upgrades.
| Existing review finding | Current disposition |
| --- | --- |
|
[MF-001](https://github.com/apache/doris/pull/70218#discussion_r4476514236):
FE notification retries | False positive: this notification is not
inserted into `AgentTaskQueue`. |
|
[MF-002](https://github.com/apache/doris/pull/70218#discussion_r4476514368):
stale MoW snapshot | Documented behavior tradeoff of the existing mode;
the new-P1 classification is withdrawn under the intended load
semantics. |
|
[MF-003](https://github.com/apache/doris/pull/70218#discussion_r4476514518):
old-BE signature leak | P2 remains for a new-FE/old-BE rolling-upgrade
window. |
|
[MF-004](https://github.com/apache/doris/pull/70218#discussion_r4476514655):
protected test access | False positive: BE UT uses
`-fno-access-control`, and the tests compiled and passed. |
**Snapshot semantics:** Even with initialization synchronization
enabled, `sync_rowsets`, local snapshot construction and the later flush
are not atomic with other transactions' commits. A key absent from the
captured snapshot can already enter the new-key validation path and fail
under ERROR or for an omitted non-nullable column without a default.
Disabling the initial synchronization can enlarge the freshness window,
including while an earlier MS-committed rowset is still absent locally.
This PR deliberately makes that existing mode the default; master
already configures Cloud p0/p1 to use it. This is an explicit
behavior/cost tradeoff, not a claim that local snapshots always contain
the latest committed keys. For APPEND updates that reach publication,
delete-bitmap calculation synchronizes missing versions and realigns
conflicting rows from historical columns; flush-time default/null fill
alone does not establish final data loss.
**Rolling upgrades:** Enabling the new FE notification default while an
old BE supports this task but lacks the cleanup guard exposes the old BE
signature leak, including when its local notification setting is false.
That process retains the signatures until it is restarted/upgraded. Keep
FE notifications disabled until all receiving BEs include the cleanup
fix.
### Release note
Cloud committed-rowset notifications are enabled by default through
enable_notify_be_after_load_txn_commit on FE and
enable_cloud_make_rs_visible_on_be on BE. Completed notifications
release their task signatures, and consumed empty-rowset entries are
removed promptly. The BE setting
cloud_mow_sync_rowsets_when_load_txn_begin now defaults to false, so
cloud MoW loads skip forced rowset synchronization at initialization
unless it is explicitly enabled.
### Check List (For Author)
- Behavior changed: Yes (see Details).
- Does this need documentation: Yes; document all three default-value
changes in the configuration reference and release notes.
- Pre-submission status: Partially verified: the implementation passed
the targeted BE/FE tests and the final pipeline cleanup passed static
effective-configuration checks. Cloud end-to-end regression and a
standalone production BE build were not run; clang-tidy was excluded as
requested. The local review has no Blocker/Major finding, with the
rolling-upgrade P2 and snapshot behavior documented above.
### AI Disclosure
- Tool: Codex (GPT-6).
- Generated: Code, tests and this description.
- Human review: [@gavinchou previously approved implementation head
`17d42f6ef18`](https://github.com/apache/doris/pull/70218#pullrequestreview-5782142366);
GitHub dismissed that approval after the pipeline cleanup commit. The
final head has been reviewed locally.
---
be/src/agent/task_worker_pool.cpp | 1 +
be/src/cloud/cloud_committed_rs_mgr.cpp | 45 +++++++------------
be/src/cloud/cloud_committed_rs_mgr.h | 2 +-
be/src/common/config.cpp | 4 +-
be/test/agent/task_worker_pool_test.cpp | 38 ++++++++++++++++
be/test/cloud/cloud_committed_rs_mgr_test.cpp | 50 ++++++++++++++++++++++
.../cloud_group_rowset_builder_writer_test.cpp | 4 +-
.../main/java/org/apache/doris/common/Config.java | 2 +-
.../pipeline/cloud_p0/conf/be_custom.conf | 3 --
.../pipeline/cloud_p0/conf/fe_custom.conf | 1 -
.../pipeline/cloud_p1/conf/be_custom.conf | 3 --
.../pipeline/cloud_p1/conf/fe_custom.conf | 2 -
12 files changed, 111 insertions(+), 44 deletions(-)
diff --git a/be/src/agent/task_worker_pool.cpp
b/be/src/agent/task_worker_pool.cpp
index f3fafb794b4..1777bb758ff 100644
--- a/be/src/agent/task_worker_pool.cpp
+++ b/be/src/agent/task_worker_pool.cpp
@@ -2594,6 +2594,7 @@ void calc_delete_bitmap_callback(CloudStorageEngine&
engine, const TAgentTaskReq
void make_cloud_committed_rs_visible_callback(CloudStorageEngine& engine,
const TAgentTaskRequest& req) {
+ Defer remove_task = [&] { remove_task_info(req.task_type, req.signature);
};
if (!config::enable_cloud_make_rs_visible_on_be) {
return;
}
diff --git a/be/src/cloud/cloud_committed_rs_mgr.cpp
b/be/src/cloud/cloud_committed_rs_mgr.cpp
index 2ad56668194..58fd0d5d758 100644
--- a/be/src/cloud/cloud_committed_rs_mgr.cpp
+++ b/be/src/cloud/cloud_committed_rs_mgr.cpp
@@ -52,22 +52,21 @@ void CloudCommittedRSMgr::add_committed_rowset(int64_t
txn_id, int64_t tablet_id
.count() +
config::tablet_txn_info_min_expired_seconds;
expiration_time = std::max(txn_expiration_min, expiration_time);
- std::unique_lock<std::shared_mutex> wlock(_rwlock);
- TxnTabletKey key(txn_id, tablet_id);
- _committed_rs_map.insert_or_assign(key, CommittedRowsetValue(rowset_meta,
expiration_time));
- _expiration_map.emplace(expiration_time, key);
- LOG(INFO) << "add pending rowset, txn_id=" << txn_id << ", tablet_id=" <<
tablet_id
- << ", rowset_id=" << rowset_meta->rowset_id().to_string()
- << ", expiration_time=" << expiration_time;
+ {
+ std::unique_lock<std::shared_mutex> wlock(_rwlock);
+ TxnTabletKey key(txn_id, tablet_id);
+ _committed_rs_map.insert_or_assign(key,
CommittedRowsetValue(rowset_meta, expiration_time));
+ _expiration_map.emplace(expiration_time, key);
+ }
+ VLOG_DEBUG << "add pending rowset, txn_id=" << txn_id << ", tablet_id=" <<
tablet_id
+ << ", rowset_id=" << (rowset_meta ?
rowset_meta->rowset_id().to_string() : "empty")
+ << ", expiration_time=" << expiration_time;
}
Result<std::pair<RowsetMetaSharedPtr, int64_t>>
CloudCommittedRSMgr::get_committed_rowset(
int64_t txn_id, int64_t tablet_id) {
std::shared_lock<std::shared_mutex> rlock(_rwlock);
TxnTabletKey key(txn_id, tablet_id);
- if (auto it = _empty_rowset_markers.find(key); it !=
_empty_rowset_markers.end()) {
- return std::make_pair(nullptr, it->second);
- }
auto iter = _committed_rs_map.find(key);
if (iter == _committed_rs_map.end()) {
return ResultError(Status::Error<ErrorCode::NOT_FOUND>(
@@ -89,8 +88,9 @@ void CloudCommittedRSMgr::remove_expired_committed_rowsets() {
while (!_expiration_map.empty()) {
auto iter = _expiration_map.begin();
- if (!_committed_rs_map.contains(iter->second) &&
- !_empty_rowset_markers.contains(iter->second)) {
+ // Consumed entries leave expiration records behind until this cleanup
pass.
+ auto it_rs = _committed_rs_map.find(iter->second);
+ if (it_rs == _committed_rs_map.end()) {
_expiration_map.erase(iter);
continue;
}
@@ -102,33 +102,18 @@ void
CloudCommittedRSMgr::remove_expired_committed_rowsets() {
auto key = iter->second;
_expiration_map.erase(iter);
- auto it_rs = _committed_rs_map.find(key);
- if (it_rs != _committed_rs_map.end() && it_rs->second.expiration_time
== expiration_time) {
+ // A refreshed entry can outlive this older expiration record.
+ if (it_rs->second.expiration_time == expiration_time) {
_committed_rs_map.erase(it_rs);
LOG(INFO) << "clean expired pending cloud rowset, txn_id=" <<
key.txn_id
<< ", tablet_id=" << key.tablet_id << ",
expiration_time=" << expiration_time;
}
- auto it_empty = _empty_rowset_markers.find(key);
- if (it_empty != _empty_rowset_markers.end() && it_empty->second ==
expiration_time) {
- _empty_rowset_markers.erase(it_empty);
- LOG(INFO) << "clean expired empty rowset marker, txn_id=" <<
key.txn_id
- << ", tablet_id=" << key.tablet_id << ",
expiration_time=" << expiration_time;
- }
}
}
void CloudCommittedRSMgr::mark_empty_rowset(int64_t txn_id, int64_t tablet_id,
int64_t txn_expiration) {
- int64_t txn_expiration_min =
-
duration_cast<std::chrono::seconds>(std::chrono::system_clock::now().time_since_epoch())
- .count() +
- config::tablet_txn_info_min_expired_seconds;
- txn_expiration = std::max(txn_expiration_min, txn_expiration);
-
- std::unique_lock<std::shared_mutex> wlock(_rwlock);
- TxnTabletKey txn_key(txn_id, tablet_id);
- _empty_rowset_markers.insert_or_assign(txn_key, txn_expiration);
- _expiration_map.emplace(txn_expiration, txn_key);
+ add_committed_rowset(txn_id, tablet_id, nullptr, txn_expiration);
}
void CloudCommittedRSMgr::_clean_thread_callback() {
diff --git a/be/src/cloud/cloud_committed_rs_mgr.h
b/be/src/cloud/cloud_committed_rs_mgr.h
index f61a01162a4..7ca2f78a12e 100644
--- a/be/src/cloud/cloud_committed_rs_mgr.h
+++ b/be/src/cloud/cloud_committed_rs_mgr.h
@@ -66,6 +66,7 @@ private:
};
struct CommittedRowsetValue {
+ // A null meta marks an empty rowset whose metadata was not written to
MS.
RowsetMetaSharedPtr rowset_meta;
int64_t expiration_time; // seconds since epoch
@@ -77,7 +78,6 @@ private:
std::map<TxnTabletKey, CommittedRowsetValue> _committed_rs_map;
// Multimap for efficient expiration cleanup: expiration_time -> <txn_id,
tablet_id>
std::multimap<int64_t, TxnTabletKey> _expiration_map;
- std::map<TxnTabletKey, int64_t /* expiration_time */>
_empty_rowset_markers;
std::shared_mutex _rwlock;
std::shared_ptr<Thread> _clean_thread;
CountDownLatch _stop_latch;
diff --git a/be/src/common/config.cpp b/be/src/common/config.cpp
index a204299f446..28066e9e4d8 100644
--- a/be/src/common/config.cpp
+++ b/be/src/common/config.cpp
@@ -2168,9 +2168,9 @@ DEFINE_mInt32(concurrency_stats_dump_interval_ms, "100");
DEFINE_Validator(concurrency_stats_dump_interval_ms,
[](const int32_t config) -> bool { return config >= 10; });
-DEFINE_mBool(cloud_mow_sync_rowsets_when_load_txn_begin, "true");
+DEFINE_mBool(cloud_mow_sync_rowsets_when_load_txn_begin, "false");
-DEFINE_mBool(enable_cloud_make_rs_visible_on_be, "false");
+DEFINE_mBool(enable_cloud_make_rs_visible_on_be, "true");
DEFINE_mBool(enable_cloud_random_segment_id, "false");
DEFINE_mInt32(file_handles_deplenish_frequency_times, "3");
diff --git a/be/test/agent/task_worker_pool_test.cpp
b/be/test/agent/task_worker_pool_test.cpp
index b8b2de0fc4b..8323ad1bd66 100644
--- a/be/test/agent/task_worker_pool_test.cpp
+++ b/be/test/agent/task_worker_pool_test.cpp
@@ -27,10 +27,13 @@
#include "agent/agent_server.h"
#include "cloud/cloud_storage_engine.h"
+#include "common/config.h"
#include "runtime/cluster_info.h"
#include "runtime/exec_env.h"
#include "storage/options.h"
#include "storage/storage_engine.h"
+#include "util/defer_op.h"
+#include "util/threadpool.h"
namespace doris {
@@ -106,6 +109,41 @@ TEST(TaskWorkerPoolTest, PreSubmitCallbackWithDedup) {
EXPECT_EQ(callback_count.load(), 1);
}
+class CloudCommittedRowsetTaskTest : public testing::TestWithParam<bool> {};
+
+TEST_P(CloudCommittedRowsetTaskTest, ReleasesSignatureAfterCallback) {
+ const bool original_enabled = config::enable_cloud_make_rs_visible_on_be;
+ Defer restore_config = [&] { config::enable_cloud_make_rs_visible_on_be =
original_enabled; };
+ config::enable_cloud_make_rs_visible_on_be = GetParam();
+
+ CloudStorageEngine engine(EngineOptions {});
+ std::atomic_int callback_count {0};
+ TaskWorkerPool workers("make_cloud_committed_rs_visible_test", 1,
[&](const auto& task) {
+ make_cloud_committed_rs_visible_callback(engine, task);
+ ++callback_count;
+ });
+
+ TAgentTaskRequest task;
+ task.__set_task_type(TTaskType::MAKE_CLOUD_COMMITTED_RS_VISIBLE);
+ task.__set_signature(GetParam() ? 5975401 : 5975400);
+ task.make_cloud_tmp_rs_visible_req.__set_txn_id(task.signature);
+ // A tablet absent from the local cache is skipped without fetching
metadata from MS.
+ task.make_cloud_tmp_rs_visible_req.__set_tablet_ids({59754});
+
+ ASSERT_TRUE(workers.submit_task(task).ok());
+ workers._thread_pool->wait();
+ ASSERT_EQ(callback_count.load(), 1);
+
+ // A completed notification must not suppress a later task with the same
signature,
+ // including when the feature was disabled before the first callback ran.
+ ASSERT_TRUE(workers.submit_task(task).ok());
+ workers._thread_pool->wait();
+ EXPECT_EQ(callback_count.load(), 2);
+}
+
+INSTANTIATE_TEST_SUITE_P(EnabledAndDisabled, CloudCommittedRowsetTaskTest,
+ testing::Values(true, false));
+
TEST(TaskWorkerPoolTest, PriorTaskWorkerPool) {
std::atomic_int normal_count {0};
std::atomic_int high_prior_count {0};
diff --git a/be/test/cloud/cloud_committed_rs_mgr_test.cpp
b/be/test/cloud/cloud_committed_rs_mgr_test.cpp
index 9df6597b8a6..791f168919e 100644
--- a/be/test/cloud/cloud_committed_rs_mgr_test.cpp
+++ b/be/test/cloud/cloud_committed_rs_mgr_test.cpp
@@ -25,6 +25,7 @@
#include "cloud/config.h"
#include "storage/rowset/rowset_meta.h"
+#include "util/defer_op.h"
namespace doris {
@@ -179,6 +180,55 @@ TEST_F(CloudCommittedRSMgrTest,
TestMarkAndCheckEmptyRowset) {
EXPECT_EQ(retrieved_expiration, txn_expiration);
}
+TEST_F(CloudCommittedRSMgrTest, TestRemoveEmptyRowset) {
+ const int64_t expiration_time = current_time_seconds() + 3600;
+ _mgr->mark_empty_rowset(1000, 2000, expiration_time);
+ _mgr->mark_empty_rowset(1000, 2001, expiration_time);
+ _mgr->mark_empty_rowset(1001, 2000, expiration_time);
+
+ _mgr->remove_committed_rowset(1000, 2000);
+ auto result = _mgr->get_committed_rowset(1000, 2000);
+ ASSERT_FALSE(result.has_value());
+ EXPECT_TRUE(result.error().is<ErrorCode::NOT_FOUND>());
+ EXPECT_TRUE(_mgr->get_committed_rowset(1000, 2001).has_value());
+ EXPECT_TRUE(_mgr->get_committed_rowset(1001, 2000).has_value());
+
+ // Repeated removal is harmless, and a consumed empty marker cannot mask a
new entry.
+ _mgr->remove_committed_rowset(1000, 2000);
+ auto rowset_meta = create_rowset_meta(2000, 1000);
+ _mgr->add_committed_rowset(1000, 2000, rowset_meta, expiration_time);
+ _mgr->remove_expired_committed_rowsets();
+ result = _mgr->get_committed_rowset(1000, 2000);
+ ASSERT_TRUE(result.has_value());
+ EXPECT_EQ(result->first, rowset_meta);
+}
+
+TEST_F(CloudCommittedRSMgrTest, TestRefreshExpiration) {
+ const auto original_min_expired_seconds =
config::tablet_txn_info_min_expired_seconds;
+ Defer restore_config = [&] {
+ config::tablet_txn_info_min_expired_seconds =
original_min_expired_seconds;
+ };
+ config::tablet_txn_info_min_expired_seconds = -100;
+ const int64_t now = current_time_seconds();
+
+ // An old expiration index entry must not remove either kind of refreshed
cache entry.
+ auto rowset_meta = create_rowset_meta(2000, 1000);
+ _mgr->add_committed_rowset(1000, 2000, rowset_meta, now - 10);
+ _mgr->mark_empty_rowset(1000, 2001, now - 10);
+ _mgr->add_committed_rowset(1000, 2000, rowset_meta, now + 3600);
+ _mgr->mark_empty_rowset(1000, 2001, now + 3600);
+ _mgr->remove_expired_committed_rowsets();
+
+ auto result = _mgr->get_committed_rowset(1000, 2000);
+ ASSERT_TRUE(result.has_value());
+ EXPECT_EQ(result->first, rowset_meta);
+ EXPECT_EQ(result->second, now + 3600);
+ result = _mgr->get_committed_rowset(1000, 2001);
+ ASSERT_TRUE(result.has_value());
+ EXPECT_EQ(result->first, nullptr);
+ EXPECT_EQ(result->second, now + 3600);
+}
+
TEST_F(CloudCommittedRSMgrTest, TestEmptyRowsetExpiration) {
// Save original config value
int32_t original_min_expired_seconds =
config::tablet_txn_info_min_expired_seconds;
diff --git a/be/test/olap/rowset/cloud_group_rowset_builder_writer_test.cpp
b/be/test/olap/rowset/cloud_group_rowset_builder_writer_test.cpp
index e53fcadff11..8ccb0bcae18 100644
--- a/be/test/olap/rowset/cloud_group_rowset_builder_writer_test.cpp
+++ b/be/test/olap/rowset/cloud_group_rowset_builder_writer_test.cpp
@@ -161,6 +161,7 @@ void add_initial_rowset(const CloudTabletSPtr& tablet,
int64_t version) {
class CloudGroupRowsetBuilderWriterTest : public testing::Test {
protected:
void SetUp() override {
+ _saved_sync_rowsets =
config::cloud_mow_sync_rowsets_when_load_txn_begin;
config::cloud_mow_sync_rowsets_when_load_txn_begin = false;
char buffer[MAX_PATH_LEN];
@@ -205,7 +206,7 @@ protected:
_changelog_tablet.reset();
_engine.reset();
EXPECT_TRUE(io::global_local_filesystem()->delete_directory(_storage_root_path).ok());
- config::cloud_mow_sync_rowsets_when_load_txn_begin = true;
+ config::cloud_mow_sync_rowsets_when_load_txn_begin =
_saved_sync_rowsets;
}
void create_remote_tablet_dir(int64_t tablet_id) {
@@ -336,6 +337,7 @@ protected:
0);
}
+ bool _saved_sync_rowsets = false;
std::unique_ptr<CloudStorageEngine> _engine;
CloudTabletSPtr _tablet;
CloudTabletSPtr _changelog_tablet;
diff --git a/fe/fe-common/src/main/java/org/apache/doris/common/Config.java
b/fe/fe-common/src/main/java/org/apache/doris/common/Config.java
index 9ab3ff27878..f79230306eb 100644
--- a/fe/fe-common/src/main/java/org/apache/doris/common/Config.java
+++ b/fe/fe-common/src/main/java/org/apache/doris/common/Config.java
@@ -3759,7 +3759,7 @@ public class Config extends ConfigBase {
public static boolean enable_tso_forward_when_counter_full = true;
@ConfField(mutable = true, masterOnly = true)
- public static boolean enable_notify_be_after_load_txn_commit = false;
+ public static boolean enable_notify_be_after_load_txn_commit = true;
// ATTN: DONOT add any config not related to cloud mode here
// ATTN: DONOT add any config not related to cloud mode here
diff --git a/regression-test/pipeline/cloud_p0/conf/be_custom.conf
b/regression-test/pipeline/cloud_p0/conf/be_custom.conf
index 86b396dce3e..c4aa0619d9f 100644
--- a/regression-test/pipeline/cloud_p0/conf/be_custom.conf
+++ b/regression-test/pipeline/cloud_p0/conf/be_custom.conf
@@ -67,7 +67,4 @@ python_env_mode=conda
python_conda_root_path=/opt/miniforge3
max_python_process_num=16
-enable_cloud_make_rs_visible_on_be=true
-cloud_mow_sync_rowsets_when_load_txn_begin=false
-
enable_cloud_random_segment_id=true
diff --git a/regression-test/pipeline/cloud_p0/conf/fe_custom.conf
b/regression-test/pipeline/cloud_p0/conf/fe_custom.conf
index 68012788a9e..9f916e1138f 100644
--- a/regression-test/pipeline/cloud_p0/conf/fe_custom.conf
+++ b/regression-test/pipeline/cloud_p0/conf/fe_custom.conf
@@ -48,7 +48,6 @@ enable_advance_next_id = true
check_table_lock_leaky = true
enable_outfile_to_local=true
-enable_notify_be_after_load_txn_commit=true
max_bucket_num_per_partition=512
enable_table_stream=true
enable_feature_binlog=true
diff --git a/regression-test/pipeline/cloud_p1/conf/be_custom.conf
b/regression-test/pipeline/cloud_p1/conf/be_custom.conf
index 973b34376b6..ae6363e3115 100644
--- a/regression-test/pipeline/cloud_p1/conf/be_custom.conf
+++ b/regression-test/pipeline/cloud_p1/conf/be_custom.conf
@@ -47,7 +47,4 @@ python_env_mode=conda
python_conda_root_path=/opt/miniconda3
max_python_process_num=16
-enable_cloud_make_rs_visible_on_be=true
-cloud_mow_sync_rowsets_when_load_txn_begin=false
-
enable_cloud_random_segment_id=true
diff --git a/regression-test/pipeline/cloud_p1/conf/fe_custom.conf
b/regression-test/pipeline/cloud_p1/conf/fe_custom.conf
index 157f3a07a09..b91a4ed6d38 100644
--- a/regression-test/pipeline/cloud_p1/conf/fe_custom.conf
+++ b/regression-test/pipeline/cloud_p1/conf/fe_custom.conf
@@ -37,5 +37,3 @@ enable_advance_next_id = true
arrow_flight_sql_port = 8081
enable_job_schedule_second_for_test = true
-
-enable_notify_be_after_load_txn_commit=true
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
