Pengzna commented on code in PR #17711:
URL: https://github.com/apache/iotdb/pull/17711#discussion_r3312149929
##########
iotdb-core/calc-commons/src/main/java/org/apache/iotdb/calc/utils/ObjectTypeUtils.java:
##########
@@ -109,7 +111,23 @@ public static Binary replaceRegionIdForObjectBinary(int
newRegionId, Binary orig
ObjectTypeUtils.parseObjectBinaryToSizeIObjectPathPair(originValue);
IObjectPath objectPath = pair.getRight();
try {
- IObjectPath newObjectPath = null;
+ final Path path = objectPath.getPath();
+ final int regionId = Integer.parseInt(path.getName(0).toString());
+ if (regionId == newRegionId) {
+ return originValue;
+ }
+
+ final IObjectPath newObjectPath;
+ if (objectPath instanceof PlainObjectPath) {
+ newObjectPath =
+ new PlainObjectPath(objectPath.toString().replaceFirst(regionId +
"", newRegionId + ""));
+ } else {
+ final String[] subPath = new String[path.getNameCount() - 1];
+ for (int i = 1; i < path.getNameCount(); i++) {
+ subPath[i - 1] = path.getName(i).toString();
+ }
+ newObjectPath = new Base32ObjectPath(Paths.get(newRegionId + "",
subPath));
Review Comment:
已修复，replaceRegionIdForObjectBinary() 现在从 Path segments 重建对象路径，只替换第 0 个
regionId segment，避免字符串 replaceFirst 误替换路径中其他数字。
##########
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/plan/planner/plan/node/write/RelationalInsertTabletNode.java:
##########
@@ -404,6 +405,11 @@ protected List<WritePlanNode>
doSplit(Map<TRegionReplicaSet, List<Integer>> spli
// Avoid using system arraycopy when there is no need to split
setRange(entry.getValue());
setDataRegionReplicaSet(entry.getKey());
+ for (int i = 0; i < columns.length; i++) {
+ if (dataTypes[i] == TSDataType.OBJECT) {
+ handleObjectValue(i, 0, times.length, entry, result);
Review Comment:
已修复，OBJECT no-split 处理和 redirect 构造都改为按 rowCount 遍历；补了 backing array 长于
rowCount 时不会生成额外 ObjectNode 的单测。
##########
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/pipe/sink/protocol/iotconsensusv2/payload/request/IoTConsensusV2ObjectFilePieceReq.java:
##########
@@ -0,0 +1,107 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.iotdb.db.pipe.sink.protocol.iotconsensusv2.payload.request;
+
+import org.apache.iotdb.common.rpc.thrift.TConsensusGroupId;
+import
org.apache.iotdb.commons.pipe.sink.payload.iotconsensusv2.request.IoTConsensusV2RequestType;
+import
org.apache.iotdb.commons.pipe.sink.payload.iotconsensusv2.request.IoTConsensusV2RequestVersion;
+import org.apache.iotdb.commons.queryengine.plan.planner.plan.node.PlanNode;
+import org.apache.iotdb.consensus.iotconsensusv2.thrift.TCommitId;
+import
org.apache.iotdb.consensus.iotconsensusv2.thrift.TIoTConsensusV2TransferReq;
+import org.apache.iotdb.db.queryengine.plan.planner.plan.node.write.ObjectNode;
+import org.apache.iotdb.db.storageengine.dataregion.wal.buffer.WALEntry;
+
+import java.nio.ByteBuffer;
+import java.util.Objects;
+
+public class IoTConsensusV2ObjectFilePieceReq extends
TIoTConsensusV2TransferReq {
+
+ private transient ObjectNode objectNode;
+
+ private IoTConsensusV2ObjectFilePieceReq() {
+ // Do nothing
+ }
+
+ public ObjectNode getObjectNode() {
+ return objectNode;
+ }
+
+ /////////////////////////////// Thrift ///////////////////////////////
+
+ public static IoTConsensusV2ObjectFilePieceReq toTIoTConsensusV2TransferReq(
+ final ObjectNode objectNode,
+ final TCommitId commitId,
+ final TConsensusGroupId consensusGroupId,
+ final int thisDataNodeId) {
+ final IoTConsensusV2ObjectFilePieceReq req = new
IoTConsensusV2ObjectFilePieceReq();
+
+ req.objectNode = objectNode;
+ req.commitId = commitId;
+ req.consensusGroupId = consensusGroupId;
+ req.dataNodeId = thisDataNodeId;
+ req.version = IoTConsensusV2RequestVersion.VERSION_1.getVersion();
+ req.type = IoTConsensusV2RequestType.TRANSFER_OBJECT_FILE_PIECE.getType();
+ req.body = objectNode.serialize();
Review Comment:
已修复，ObjectNode.serialize() 现在在 object/tmp 文件均读不到或短读时直接抛
SerializationRunTimeException，读取内容改为 readFully，避免提交零填充/截断内容；补了缺失文件 serialize
失败测试。
##########
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/plan/planner/plan/node/write/RelationalInsertTabletNode.java:
##########
@@ -455,6 +471,44 @@ private List<WritePlanNode> generateOneSplitList(
return result;
}
+ private void handleObjectValue(
+ int column,
+ int startRow,
+ int endRow,
+ Map.Entry<TRegionReplicaSet, List<Integer>> entry,
+ List<WritePlanNode> result) {
+ for (int row = startRow; row < endRow; row++) {
+ if (((Binary[]) columns[column])[row] == null) {
Review Comment:
已修复，handleObjectValue() 会先检查 null bitmap，跳过 marked row 后才读取 Binary；补了 stale
backing value + null bitmap 的覆盖。
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
