Caideyipi commented on code in PR #19059:
URL: https://github.com/apache/iotdb/pull/19059#discussion_r4421031502


##########
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/storageengine/StorageEngine.java:
##########
@@ -882,6 +883,11 @@ public TSStatus deleteDataRegion(DataRegionId regionId) {
     if (region != null) {
       LOGGER.info(StorageEngineMessages.REMOVING_DATA_REGION, regionId);
       region.markDeleted();
+      
PipeTsFileEpochProgressIndexKeeper.getInstance().clearProgressIndex(regionId.getId());

Review Comment:
   [P2] Prevent pending Pipe events from repopulating the deleted region
   
   `markDeleted()` stops storage-side work, but it neither deregisters the 
realtime Pipe sources nor drains their assigner. An insert already published to 
`PipeDataRegionAssigner` can therefore reach 
`PipeRealtimeDataRegionHybridSource.extractTabletInsertion()` after this clear 
and call `registerProgressIndex()` for the deleted region; that method 
recreates the region entry with `computeIfAbsent`. In particular, a downgraded 
tablet from a working TsFile can reintroduce the resource after deletion, while 
`forceCloseAllWorkingTsFileProcessors()` discards that processor without 
publishing its normal TsFile-close event. The entry then still depends on the 
later source `close()` for removal, so the new deletion cleanup does not 
guarantee that the deleted region's state stays cleared. Please coordinate the 
final clear with outstanding registrations, or reject registrations from the 
obsolete region lifecycle, and cover an event paused before registration while 
`deleteDataRegion()` completes
 . The added UTs only register before deletion, and the IT creates no Pipe, so 
none exercises this ordering.



##########
integration-test/src/test/java/org/apache/iotdb/db/it/IoTDBPipeProgressIndexCleanupIT.java:
##########
@@ -0,0 +1,85 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.iotdb.db.it;
+
+import org.apache.iotdb.it.env.EnvFactory;
+import org.apache.iotdb.it.framework.IoTDBTestRunner;
+import org.apache.iotdb.itbase.category.LocalStandaloneIT;
+
+import org.awaitility.Awaitility;
+import org.junit.AfterClass;
+import org.junit.Assert;
+import org.junit.BeforeClass;
+import org.junit.Test;
+import org.junit.experimental.categories.Category;
+import org.junit.runner.RunWith;
+
+import java.io.File;
+import java.nio.charset.StandardCharsets;
+import java.nio.file.Files;
+import java.sql.Connection;
+import java.sql.Statement;
+import java.util.concurrent.TimeUnit;
+
+@RunWith(IoTDBTestRunner.class)
+@Category({LocalStandaloneIT.class})
+public class IoTDBPipeProgressIndexCleanupIT {
+
+  private static final String DATABASE = "root.pipe_progress_index";
+  private static final String CLEANUP_LOG =
+      "Cleared pipe progress indexes of the deleted data region";

Review Comment:
   [P2] Make the cleanup assertion work with the Chinese locale
   
   With `-P with-zh-locale`, the DataNode emits the Chinese 
`StorageEngineMessages.LOG_CLEARED_PIPE_PROGRESS_INDEXES_OF_THE_DELETED_DATA_REGION_ARG_DC6E6D9F`
 value added in this PR. This hardcoded English substring can never match that 
log, so running this IT with both `with-integration-tests` and `with-zh-locale` 
always times out after 60 seconds even when cleanup succeeds. Derive the 
expected message from the selected `StorageEngineMessages` constant (accounting 
for the region-id placeholder), or use an assertion independent of the log 
language. Compiling both locales alone does not exercise this failure.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to