gavinchou commented on code in PR #62661:
URL: https://github.com/apache/doris/pull/62661#discussion_r3505034362
##########
regression-test/suites/load_p0/insert/test_insert_random_distribution_table.groovy:
##########
@@ -15,9 +15,15 @@
// specific language governing permissions and limitations
// under the License.
-suite("test_insert_random_distribution_table", "p0") {
+suite("test_insert_random_distribution_table", "p0,nonConcurrent") {
def tableName = "test_insert_random_distribution_table"
+ def enableAdaptiveRandomBucketConfig = sql """ ADMIN SHOW FRONTEND CONFIG
LIKE 'enable_adaptive_random_bucket_load'; """
+ String oldEnableAdaptiveRandomBucket =
enableAdaptiveRandomBucketConfig[0][1]
+ try {
+ // This case verifies the legacy batch-size based random tablet
rotation.
+ // Keep the suite nonConcurrent because this FE config is global.
+ sql """ ADMIN SET FRONTEND CONFIG
('enable_adaptive_random_bucket_load' = 'false') """
Review Comment:
This PR enables adaptive random bucket load by default, but this regression
explicitly disables the feature and therefore only preserves the legacy
random-bucket behavior. The new path spans FE assignment, thrift/protobuf
metadata, sender partition routing, receiver-side tablet selection/rotation,
cloud lazy writer init, and close/commit semantics. Could we add an
adaptive-enabled cloud load regression that keeps this config on and verifies
rows are routed/loaded correctly, ideally including rotation after memtable
flush or a runtime partition path?
##########
be/src/load/delta_writer/delta_writer.cpp:
##########
@@ -155,10 +161,22 @@ Status BaseDeltaWriter::init() {
return Status::OK();
}
-Status DeltaWriter::write(const Block* block, const DorisVector<uint32_t>&
row_idxs) {
+Status DeltaWriter::write(const Block* block, const DorisVector<uint32_t>&
row_idxs,
+ bool* memtable_flushed) {
+ if (memtable_flushed != nullptr) {
+ *memtable_flushed = false;
+ }
if (UNLIKELY(row_idxs.empty())) {
return Status::OK();
}
+ if (_req.enable_table_memtable_backpressure) {
Review Comment:
This table-level memtable backpressure does not appear to preserve the
existing high-priority load bypass. LoadChannelMgr skips the global memtable
limiter for high-priority loads because blocking here can cause RPC timeout,
but adaptive writes can still enter this new table-level wait before taking the
writer lock. Could we skip this wait, or make it bounded/non-blocking, when
_req.is_high_priority is true? The same policy should probably be applied to
CloudDeltaWriter and DeltaWriterV2 as well.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
