Caideyipi commented on code in PR #19059:
URL: https://github.com/apache/iotdb/pull/19059#discussion_r4421031502
##########
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/storageengine/StorageEngine.java:
##########
@@ -882,6 +883,11 @@ public TSStatus deleteDataRegion(DataRegionId regionId) {
if (region != null) {
LOGGER.info(StorageEngineMessages.REMOVING_DATA_REGION, regionId);
region.markDeleted();
+
PipeTsFileEpochProgressIndexKeeper.getInstance().clearProgressIndex(regionId.getId());
Review Comment:
[P2] Prevent pending Pipe events from repopulating the deleted region
`markDeleted()` stops storage-side work, but it neither deregisters the
realtime Pipe sources nor drains their assigner. An insert already published to
`PipeDataRegionAssigner` can therefore reach
`PipeRealtimeDataRegionHybridSource.extractTabletInsertion()` after this clear
and call `registerProgressIndex()` for the deleted region; that method
recreates the region entry with `computeIfAbsent`. In particular, a downgraded
tablet from a working TsFile can reintroduce the resource after deletion, while
`forceCloseAllWorkingTsFileProcessors()` discards that processor without
publishing its normal TsFile-close event. The entry then still depends on the
later source `close()` for removal, so the new deletion cleanup does not
guarantee that the deleted region's state stays cleared. Please coordinate the
final clear with outstanding registrations, or reject registrations from the
obsolete region lifecycle, and cover an event paused before registration while
`deleteDataRegion()` completes
. The added UTs only register before deletion, and the IT creates no Pipe, so
none exercises this ordering.
##########
integration-test/src/test/java/org/apache/iotdb/db/it/IoTDBPipeProgressIndexCleanupIT.java:
##########
@@ -0,0 +1,85 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.iotdb.db.it;
+
+import org.apache.iotdb.it.env.EnvFactory;
+import org.apache.iotdb.it.framework.IoTDBTestRunner;
+import org.apache.iotdb.itbase.category.LocalStandaloneIT;
+
+import org.awaitility.Awaitility;
+import org.junit.AfterClass;
+import org.junit.Assert;
+import org.junit.BeforeClass;
+import org.junit.Test;
+import org.junit.experimental.categories.Category;
+import org.junit.runner.RunWith;
+
+import java.io.File;
+import java.nio.charset.StandardCharsets;
+import java.nio.file.Files;
+import java.sql.Connection;
+import java.sql.Statement;
+import java.util.concurrent.TimeUnit;
+
+@RunWith(IoTDBTestRunner.class)
+@Category({LocalStandaloneIT.class})
+public class IoTDBPipeProgressIndexCleanupIT {
+
+ private static final String DATABASE = "root.pipe_progress_index";
+ private static final String CLEANUP_LOG =
+ "Cleared pipe progress indexes of the deleted data region";
Review Comment:
[P2] Make the cleanup assertion work with the Chinese locale
With `-P with-zh-locale`, the DataNode emits the Chinese
`StorageEngineMessages.LOG_CLEARED_PIPE_PROGRESS_INDEXES_OF_THE_DELETED_DATA_REGION_ARG_DC6E6D9F`
value added in this PR. This hardcoded English substring can never match that
log, so running this IT with both `with-integration-tests` and `with-zh-locale`
always times out after 60 seconds even when cleanup succeeds. Derive the
expected message from the selected `StorageEngineMessages` constant (accounting
for the region-id placeholder), or use an assertion independent of the log
language. Compiling both locales alone does not exercise this failure.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]