This is an automated email from the ASF dual-hosted git repository.
hxd pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/iotdb.git
The following commit(s) were added to refs/heads/master by this push:
new fd7e68f [IOTDB-1202] reduce the size of client library (#2795)
fd7e68f is described below
commit fd7e68f1ddbb0dfd44346caec0ed069d15da03db
Author: Xiangdong Huang <[email protected]>
AuthorDate: Mon Apr 5 11:14:47 2021 +0800
[IOTDB-1202] reduce the size of client library (#2795)
* seperate thrift module into 3 parts; extract Node Serialization from
Server module into Cluster module
* add instruction for the ordering of imports
* add snappy into service-rpc pom
* modify readme of the whole project
* add logback and slf4j in root/pom
---
README.md | 49 ++++--
README_ZH.md | 34 +++-
cluster/pom.xml | 15 ++
.../apache/iotdb/cluster/RemoteTsFileResource.java | 5 +-
.../org/apache/iotdb/cluster/log/HardState.java | 6 +-
.../iotdb/cluster/log/logtypes/AddNodeLog.java | 6 +-
.../iotdb/cluster/log/logtypes/RemoveNodeLog.java | 6 +-
.../log/snapshot/PullSnapshotTaskDescriptor.java | 6 +-
.../iotdb/cluster/partition/slot/SlotManager.java | 6 +-
.../cluster/partition/slot/SlotPartitionTable.java | 5 +-
.../iotdb/cluster/utils/NodeSerializeUtils.java | 90 ++++++++++
.../iotdb/cluster/utils/SerializeUtilTest.java | 13 ++
code-coverage/pom.xml | 6 +
docs/Development/ContributeGuide.md | 11 ++
grafana/pom.xml | 4 +
jdbc/pom.xml | 5 -
pom.xml | 18 ++
server/pom.xml | 5 +
.../org/apache/iotdb/db/utils/SerializeUtils.java | 62 -------
.../apache/iotdb/db/utils/SerializeUtilsTest.java | 13 --
service-rpc/pom.xml | 11 ++
session/pom.xml | 9 +-
thrift-cluster/README.md | 22 +++
thrift-cluster/pom.xml | 67 ++++++++
thrift-cluster/rpc-changelist.md | 181 +++++++++++++++++++++
.../src/main/thrift/cluster.thrift | 0
thrift-sync/README.md | 22 +++
thrift-sync/pom.xml | 67 ++++++++
thrift-sync/rpc-changelist.md | 181 +++++++++++++++++++++
.../src/main/thrift/sync.thrift | 0
thrift/README.md | 22 +++
31 files changed, 821 insertions(+), 126 deletions(-)
diff --git a/README.md b/README.md
index 3cec9c8..fe3a3de 100644
--- a/README.md
+++ b/README.md
@@ -102,20 +102,33 @@ Here in the Quick Start, we give a brief introduction of
using source code to in
## Build from source
-Skip this paragraph if you are using Windows. As we use Thrift for our RPC
module (communication and
+### Prepare Thrift compiler
+
+Skip this chapter if you are using Windows.
+
+As we use Thrift for our RPC module (communication and
protocol definition), we involve Thrift during the compilation, so Thrift
compiler 0.13.0 (or
higher) is required to generate Thrift Java code. Thrift officially provides
binary compiler for
-Windows, but unfortunately, they do not provide that for Unix OSs. However, we
compiled a Unix
-compiler ourselves and put it onto GitHub, and with the help of a maven
plugin, it will be
-downloaded automatically during compilation. This compiler works fine with
gcc8 or later, Ubuntu
-MacOS, and CentOS, but previous versions and other OSs are not guaranteed.
Should you find your gcc
-version or OS does not support the precompiled compiler, please upgrade your
gcc version or follow the
-Thrift official instructions to compile the compiler yourself and rename it
into `{project_root}\thrift\target\tools\thrift_0.12.0_0.13.0_linux.exe`.
-If you have already installed a compatible Thrift compiler, you may add the
following parameter
+Windows, but unfortunately, they do not provide that for Unix OSs.
+
+If you have permission to install new softwares, use `apt install` or `yum
install` or `brew install`
+to install the Thrift compiler (If you already have installed the thrift
compiler, skip this step).
+Then, you may add the following parameter
when running Maven:
`-Dthrift.download-url=http://apache.org/licenses/LICENSE-2.0.txt
-Dthrift.exec.absolute.path=<YOUR LOCAL THRIFT BINARY FILE>`.
-If you want to download the Thrift compiler from another position, you may add
the following
-parameter: `-Dthrift.download-url=<THE REMOTE URL FOR DOWNLOADING>
-Dthrift.exec.absolute.path=<THE DOWNLOADED BINARY FILE NAME>`. Or you may
directly modify our root pom if you are skilled enough.
-Here is the Thrift official site: https://thrift.apache.org/
+
+If not, then you have to compile the thrift compiler, and it requires you
install a boost library first.
+Therefore, we compiled a Unix compiler ourselves and put it onto GitHub, and
with the help of a
+maven plugin, it will be downloaded automatically during compilation.
+This compiler works fine with gcc8 or later, Ubuntu MacOS, and CentOS, but
previous versions
+and other OSs are not guaranteed.
+
+If you can not download the thrift compiler automatically because of network
problem, you can download
+it yourself, and then either:
+rename your thrift file to
`{project_root}\thrift\target\tools\thrift_0.12.0_0.13.0_linux.exe`;
+or, add Maven commands:
+`-Dthrift.download-url=http://apache.org/licenses/LICENSE-2.0.txt
-Dthrift.exec.absolute.path=<YOUR LOCAL THRIFT BINARY FILE>`.
+
+### Compile IoTDB
You can download the source code from:
@@ -129,6 +142,11 @@ The default master branch is the dev branch, If you want
to use a released versi
git checkout release/x.x.x
```
+From v0.11.3 on, the tag name format is change to: vx.x.x:
+
+```
+git checkout vx.x.x
+```
Under the root path of iotdb:
@@ -138,10 +156,13 @@ Under the root path of iotdb:
Using `-P compile-cpp` for compiling cpp client (For more details, read
client-cpp's Readme file.)
-Then the binary version (including both server and cli) can be found at
**distribution/target/apache-iotdb-{project.version}-bin.zip**
-
-> NOTE: Directories "thrift/target/generated-sources/thrift" and
"antlr/target/generated-sources/antlr4" need to be added to sources roots to
avoid compilation errors in the IDE.
+Then the binary version (including both server and cli) can be found at
**distribution/target/apache-iotdb-{project.version}-all-bin.zip**
+NOTE: Directories `thrift/target/generated-sources/thrift`,
`thrift-sync/target/generated-sources/thrift`,
+`thrift-cluster/target/generated-sources/thrift`
+and `antlr/target/generated-sources/antlr4` need to be added to sources roots
to avoid compilation errors in the IDE.
+In IDEAJ, you just need to right click on the root project name and choose
"Maven->Reload Project" after
+you run `mvn package` successfully.
### Configurations
diff --git a/README_ZH.md b/README_ZH.md
index 42a3c7f..b27fc9d 100644
--- a/README_ZH.md
+++ b/README_ZH.md
@@ -100,13 +100,23 @@ IoTDB提供了三种安装方法，您可以参考以下建议，选择最适合
## 从源码构建
-如果您使用Windows，请跳过此段。我们使用Thrift作为RPC模块来提供客户端-服务器间的通信和协议支持，因此在编译阶段我们需要使用Thrift
0.13.0
-（或更高）编译器生成对应的Java代码。Thrift只提供了Windows下的二进制编译器，Unix下需要通过源码自行编译。但我们预先编译了一个Thrift编译器，并将其上传到了GitHub
-，借助一个Maven插件，在编译时可以自动将其下载。该预编译的Thrift编译器在gcc8，Ubuntu, CentOS,
MacOS下可以工作，但是在更低的gcc
-版本以及其他操作系统上尚未确认。如果您在编译时发现了Thrift编译器相关的问题，请升级您的gcc版本或者依照Thrift
-官方的指示自行编译编译器，并将编译器放置到目录`{project_root}\thrift\target\tools\thrift_0.12.0_0.13.0_linux.exe`。
-如果您已经安装了一个兼容的Thrift编译器，您可以在运行Maven时通过以下参数指定使用您的编译器：`-Dthrift.download-url=http://apache.org/licenses/LICENSE-2.0.txt
-Dthrift.exec.absolute.path=<YOUR LOCAL THRIFT BINARY FILE>`。
-您也可以使用以下Maven参数来更换Thrift编译器的下载地址：`-Dthrift.download-url=<THE REMOTE URL FOR
DOWNLOADING> -Dthrift.exec.absolute.path=<THE DOWNLOADED BINARY FILE NAME>`。
+### 关于准备Thrift编译器
+
+如果您使用Windows，请跳过此段。
+
+我们使用Thrift作为RPC模块来提供客户端-服务器间的通信和协议支持，因此在编译阶段我们需要使用Thrift 0.13.0
+（或更高）编译器生成对应的Java代码。 Thrift只提供了Windows下的二进制编译器，Unix下需要通过源码自行编译。
+
+如果你有安装权限，可以通过`apt install`, `yum install`, `brew
install`来安装thrift编译器，然后在下面的编译命令中
+都添加如下参数即可：`-Dthrift.download-url=http://apache.org/licenses/LICENSE-2.0.txt
-Dthrift.exec.absolute.path=<你的thrift可执行文件路径>`。
+
+
+同时我们预先编译了一个Thrift编译器，并将其上传到了GitHub ，借助一个Maven插件，在编译时可以自动将其下载。
+该预编译的Thrift编译器在gcc8，Ubuntu, CentOS, MacOS下可以工作，但是在更低的gcc
+版本以及其他操作系统上尚未确认。
+如果您发现因为网络问题总是提示下载不到thrift文件，那么您需要手动下载，并并将编译器放置到目录`{project_root}\thrift\target\tools\thrift_0.12.0_0.13.0_linux.exe`。
+如果您放到其他地方，就需要在运行maven的命令中添加：`-Dthrift.download-url=http://apache.org/licenses/LICENSE-2.0.txt
-Dthrift.exec.absolute.path=<你的thrift可执行文件路径>`。
+
如果您对Maven足够熟悉，您也可以直接修改我们的根pom文件来避免每次编译都使用上述参数。
Thrift官方网址为：https://thrift.apache.org/
@@ -122,15 +132,23 @@ git clone https://github.com/apache/iotdb.git
git checkout release/x.x.x
```
+从0.11.3开始，版本的标签风格改为vx.x.x：
+```
+git checkout vx.x.x
+```
+
在 iotdb 根目录下执行 maven 编译:
```
> mvn clean package -DskipTests
```
-执行完成之后，可以在**distribution/target/apache-iotdb-{project.version}-bin.zip**找到编译完成的二进制版本(包括服务器和客户端)
+通过添加 `-P compile-cpp` 可以进行c++客户端API的编译。
+
+执行完成之后，可以在**distribution/target/apache-iotdb-{project.version}-all-bin.zip**找到编译完成的二进制版本(包括服务器和客户端)
> 注意:"thrift/target/generated-sources/thrift" 和
> "antlr/target/generated-sources/antlr4" 目录需要添加到源代码根中，以免在 IDE 中产生编译错误。
+> IDEA的操作方法：在上述maven命令编译好后，右键项目名称，选择"Maven->Reload project"，即可。
### 配置
diff --git a/cluster/pom.xml b/cluster/pom.xml
index 9a15a0d..d211396 100644
--- a/cluster/pom.xml
+++ b/cluster/pom.xml
@@ -76,6 +76,21 @@
<artifactId>libthrift</artifactId>
</exclusion>
</exclusions>
+ </dependency>
+ <dependency>
+ <groupId>commons-io</groupId>
+ <artifactId>commons-io</artifactId>
+ </dependency>
+ <dependency>
+ <groupId>org.apache.iotdb</groupId>
+ <artifactId>iotdb-thrift-cluster</artifactId>
+ <version>0.12.0-SNAPSHOT</version>
+ <exclusions>
+ <exclusion>
+ <groupId>org.apache.thrift</groupId>
+ <artifactId>libthrift</artifactId>
+ </exclusion>
+ </exclusions>
<scope>compile</scope>
</dependency>
<dependency>
diff --git
a/cluster/src/main/java/org/apache/iotdb/cluster/RemoteTsFileResource.java
b/cluster/src/main/java/org/apache/iotdb/cluster/RemoteTsFileResource.java
index fdf492a..04fb776 100644
--- a/cluster/src/main/java/org/apache/iotdb/cluster/RemoteTsFileResource.java
+++ b/cluster/src/main/java/org/apache/iotdb/cluster/RemoteTsFileResource.java
@@ -20,6 +20,7 @@
package org.apache.iotdb.cluster;
import org.apache.iotdb.cluster.rpc.thrift.Node;
+import org.apache.iotdb.cluster.utils.NodeSerializeUtils;
import org.apache.iotdb.db.conf.IoTDBDescriptor;
import org.apache.iotdb.db.engine.storagegroup.TsFileResource;
import org.apache.iotdb.db.utils.SerializeUtils;
@@ -82,7 +83,7 @@ public class RemoteTsFileResource extends TsFileResource {
}
public void serialize(DataOutputStream dataOutputStream) {
- SerializeUtils.serialize(source, dataOutputStream);
+ NodeSerializeUtils.serialize(source, dataOutputStream);
try {
// the path here is only for the remote node to get a download link, so
it does not matter
// if it is absolute
@@ -102,7 +103,7 @@ public class RemoteTsFileResource extends TsFileResource {
public void deserialize(ByteBuffer buffer) {
source = new Node();
- SerializeUtils.deserialize(source, buffer);
+ NodeSerializeUtils.deserialize(source, buffer);
setFile(new File(SerializeUtils.deserializeString(buffer)));
timeIndex =
diff --git a/cluster/src/main/java/org/apache/iotdb/cluster/log/HardState.java
b/cluster/src/main/java/org/apache/iotdb/cluster/log/HardState.java
index 03e12fc..2e9e172 100644
--- a/cluster/src/main/java/org/apache/iotdb/cluster/log/HardState.java
+++ b/cluster/src/main/java/org/apache/iotdb/cluster/log/HardState.java
@@ -20,7 +20,7 @@
package org.apache.iotdb.cluster.log;
import org.apache.iotdb.cluster.rpc.thrift.Node;
-import org.apache.iotdb.db.utils.SerializeUtils;
+import org.apache.iotdb.cluster.utils.NodeSerializeUtils;
import org.apache.iotdb.tsfile.utils.ReadWriteIOUtils;
import org.apache.commons.lang3.builder.EqualsBuilder;
@@ -46,7 +46,7 @@ public class HardState {
int isNull = buffer.get();
if (isNull == 1) {
Node node = new Node();
- SerializeUtils.deserialize(node, buffer);
+ NodeSerializeUtils.deserialize(node, buffer);
res.setVoteFor(node);
} else {
res.setVoteFor(null);
@@ -63,7 +63,7 @@ public class HardState {
dataOutputStream.writeByte(0);
} else {
dataOutputStream.writeByte(1);
- SerializeUtils.serialize(voteFor, dataOutputStream);
+ NodeSerializeUtils.serialize(voteFor, dataOutputStream);
}
} catch (IOException e) {
// unreachable
diff --git
a/cluster/src/main/java/org/apache/iotdb/cluster/log/logtypes/AddNodeLog.java
b/cluster/src/main/java/org/apache/iotdb/cluster/log/logtypes/AddNodeLog.java
index 75bb8f5..ea41685 100644
---
a/cluster/src/main/java/org/apache/iotdb/cluster/log/logtypes/AddNodeLog.java
+++
b/cluster/src/main/java/org/apache/iotdb/cluster/log/logtypes/AddNodeLog.java
@@ -21,7 +21,7 @@ package org.apache.iotdb.cluster.log.logtypes;
import org.apache.iotdb.cluster.log.Log;
import org.apache.iotdb.cluster.rpc.thrift.Node;
-import org.apache.iotdb.db.utils.SerializeUtils;
+import org.apache.iotdb.cluster.utils.NodeSerializeUtils;
import java.io.ByteArrayOutputStream;
import java.io.DataOutputStream;
@@ -50,7 +50,7 @@ public class AddNodeLog extends Log {
dataOutputStream.writeLong(getCurrLogIndex());
dataOutputStream.writeLong(getCurrLogTerm());
- SerializeUtils.serialize(newNode, dataOutputStream);
+ NodeSerializeUtils.serialize(newNode, dataOutputStream);
} catch (IOException e) {
// ignored
}
@@ -67,7 +67,7 @@ public class AddNodeLog extends Log {
setCurrLogTerm(buffer.getLong());
newNode = new Node();
- SerializeUtils.deserialize(newNode, buffer);
+ NodeSerializeUtils.deserialize(newNode, buffer);
}
@Override
diff --git
a/cluster/src/main/java/org/apache/iotdb/cluster/log/logtypes/RemoveNodeLog.java
b/cluster/src/main/java/org/apache/iotdb/cluster/log/logtypes/RemoveNodeLog.java
index 19359c2..8a84023 100644
---
a/cluster/src/main/java/org/apache/iotdb/cluster/log/logtypes/RemoveNodeLog.java
+++
b/cluster/src/main/java/org/apache/iotdb/cluster/log/logtypes/RemoveNodeLog.java
@@ -21,7 +21,7 @@ package org.apache.iotdb.cluster.log.logtypes;
import org.apache.iotdb.cluster.log.Log;
import org.apache.iotdb.cluster.rpc.thrift.Node;
-import org.apache.iotdb.db.utils.SerializeUtils;
+import org.apache.iotdb.cluster.utils.NodeSerializeUtils;
import java.io.ByteArrayOutputStream;
import java.io.DataOutputStream;
@@ -41,7 +41,7 @@ public class RemoveNodeLog extends Log {
dataOutputStream.writeLong(getCurrLogIndex());
dataOutputStream.writeLong(getCurrLogTerm());
- SerializeUtils.serialize(removedNode, dataOutputStream);
+ NodeSerializeUtils.serialize(removedNode, dataOutputStream);
} catch (IOException e) {
// ignored
}
@@ -54,7 +54,7 @@ public class RemoveNodeLog extends Log {
setCurrLogTerm(buffer.getLong());
removedNode = new Node();
- SerializeUtils.deserialize(removedNode, buffer);
+ NodeSerializeUtils.deserialize(removedNode, buffer);
}
public Node getRemovedNode() {
diff --git
a/cluster/src/main/java/org/apache/iotdb/cluster/log/snapshot/PullSnapshotTaskDescriptor.java
b/cluster/src/main/java/org/apache/iotdb/cluster/log/snapshot/PullSnapshotTaskDescriptor.java
index b811f1f..a2216fd 100644
---
a/cluster/src/main/java/org/apache/iotdb/cluster/log/snapshot/PullSnapshotTaskDescriptor.java
+++
b/cluster/src/main/java/org/apache/iotdb/cluster/log/snapshot/PullSnapshotTaskDescriptor.java
@@ -21,7 +21,7 @@ package org.apache.iotdb.cluster.log.snapshot;
import org.apache.iotdb.cluster.partition.PartitionGroup;
import org.apache.iotdb.cluster.rpc.thrift.Node;
-import org.apache.iotdb.db.utils.SerializeUtils;
+import org.apache.iotdb.cluster.utils.NodeSerializeUtils;
import java.io.DataInputStream;
import java.io.DataOutputStream;
@@ -77,7 +77,7 @@ public class PullSnapshotTaskDescriptor {
dataOutputStream.writeInt(previousHolders.size());
for (Node previousHolder : previousHolders) {
- SerializeUtils.serialize(previousHolder, dataOutputStream);
+ NodeSerializeUtils.serialize(previousHolder, dataOutputStream);
}
dataOutputStream.writeBoolean(requireReadOnly);
@@ -94,7 +94,7 @@ public class PullSnapshotTaskDescriptor {
previousHolders = new PartitionGroup();
for (int i = 0; i < holderSize; i++) {
Node node = new Node();
- SerializeUtils.deserialize(node, dataInputStream);
+ NodeSerializeUtils.deserialize(node, dataInputStream);
previousHolders.add(node);
}
diff --git
a/cluster/src/main/java/org/apache/iotdb/cluster/partition/slot/SlotManager.java
b/cluster/src/main/java/org/apache/iotdb/cluster/partition/slot/SlotManager.java
index 37cc0f2..528119f 100644
---
a/cluster/src/main/java/org/apache/iotdb/cluster/partition/slot/SlotManager.java
+++
b/cluster/src/main/java/org/apache/iotdb/cluster/partition/slot/SlotManager.java
@@ -6,8 +6,8 @@ package org.apache.iotdb.cluster.partition.slot;
import org.apache.iotdb.cluster.config.ClusterDescriptor;
import org.apache.iotdb.cluster.rpc.thrift.Node;
+import org.apache.iotdb.cluster.utils.NodeSerializeUtils;
import org.apache.iotdb.db.exception.StorageEngineException;
-import org.apache.iotdb.db.utils.SerializeUtils;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
@@ -298,7 +298,7 @@ public class SlotManager {
private void serialize(DataOutputStream outputStream) throws IOException {
outputStream.writeInt(slotStatus.ordinal());
if (slotStatus == SlotStatus.PULLING || slotStatus ==
SlotStatus.PULLING_WRITABLE) {
- SerializeUtils.serialize(source, outputStream);
+ NodeSerializeUtils.serialize(source, outputStream);
} else if (slotStatus == SlotStatus.SENDING) {
outputStream.writeInt(snapshotReceivedCount);
}
@@ -310,7 +310,7 @@ public class SlotManager {
if (descriptor.slotStatus == SlotStatus.PULLING
|| descriptor.slotStatus == SlotStatus.PULLING_WRITABLE) {
descriptor.source = new Node();
- SerializeUtils.deserialize(descriptor.source, buffer);
+ NodeSerializeUtils.deserialize(descriptor.source, buffer);
} else if (descriptor.slotStatus == SlotStatus.SENDING) {
descriptor.snapshotReceivedCount = buffer.getInt();
}
diff --git
a/cluster/src/main/java/org/apache/iotdb/cluster/partition/slot/SlotPartitionTable.java
b/cluster/src/main/java/org/apache/iotdb/cluster/partition/slot/SlotPartitionTable.java
index 8c25a63..469e84d 100644
---
a/cluster/src/main/java/org/apache/iotdb/cluster/partition/slot/SlotPartitionTable.java
+++
b/cluster/src/main/java/org/apache/iotdb/cluster/partition/slot/SlotPartitionTable.java
@@ -12,6 +12,7 @@ import org.apache.iotdb.cluster.partition.PartitionGroup;
import org.apache.iotdb.cluster.partition.PartitionTable;
import org.apache.iotdb.cluster.partition.slot.SlotStrategy.DefaultStrategy;
import org.apache.iotdb.cluster.rpc.thrift.Node;
+import org.apache.iotdb.cluster.utils.NodeSerializeUtils;
import org.apache.iotdb.db.utils.SerializeUtils;
import org.slf4j.Logger;
@@ -310,7 +311,7 @@ public class SlotPartitionTable implements PartitionTable {
dataOutputStream.writeInt(totalSlotNumbers);
dataOutputStream.writeInt(nodeSlotMap.size());
for (Entry<Node, List<Integer>> entry : nodeSlotMap.entrySet()) {
- SerializeUtils.serialize(entry.getKey(), dataOutputStream);
+ NodeSerializeUtils.serialize(entry.getKey(), dataOutputStream);
SerializeUtils.serializeIntList(entry.getValue(), dataOutputStream);
}
@@ -341,7 +342,7 @@ public class SlotPartitionTable implements PartitionTable {
for (int i = 0; i < size; i++) {
Node node = new Node();
List<Integer> slots = new ArrayList<>();
- SerializeUtils.deserialize(node, buffer);
+ NodeSerializeUtils.deserialize(node, buffer);
SerializeUtils.deserializeIntList(slots, buffer);
nodeSlotMap.put(node, slots);
idNodeMap.put(node.getNodeIdentifier(), node);
diff --git
a/cluster/src/main/java/org/apache/iotdb/cluster/utils/NodeSerializeUtils.java
b/cluster/src/main/java/org/apache/iotdb/cluster/utils/NodeSerializeUtils.java
new file mode 100644
index 0000000..6e14cc3
--- /dev/null
+++
b/cluster/src/main/java/org/apache/iotdb/cluster/utils/NodeSerializeUtils.java
@@ -0,0 +1,90 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.iotdb.cluster.utils;
+
+import org.apache.iotdb.cluster.rpc.thrift.Node;
+
+import java.io.DataInputStream;
+import java.io.DataOutputStream;
+import java.io.IOException;
+import java.nio.ByteBuffer;
+
+public class NodeSerializeUtils {
+
+ public static void serialize(Node node, DataOutputStream dataOutputStream) {
+ try {
+ byte[] internalIpBytes = node.internalIp.getBytes();
+ dataOutputStream.writeInt(internalIpBytes.length);
+ dataOutputStream.write(internalIpBytes);
+ dataOutputStream.writeInt(node.metaPort);
+ dataOutputStream.writeInt(node.nodeIdentifier);
+ dataOutputStream.writeInt(node.dataPort);
+ dataOutputStream.writeInt(node.clientPort);
+ byte[] clientIpBytes = node.clientIp.getBytes();
+ dataOutputStream.writeInt(clientIpBytes.length);
+ dataOutputStream.write(clientIpBytes);
+ } catch (IOException e) {
+ // unreachable
+ }
+ }
+
+ public static void deserialize(Node node, ByteBuffer buffer) {
+ int internalIpLength = buffer.getInt();
+ byte[] internalIpBytes = new byte[internalIpLength];
+ buffer.get(internalIpBytes);
+ node.setInternalIp(new String(internalIpBytes));
+ node.setMetaPort(buffer.getInt());
+ node.setNodeIdentifier(buffer.getInt());
+ node.setDataPort(buffer.getInt());
+ node.setClientPort(buffer.getInt());
+ int clientIpLength = buffer.getInt();
+ byte[] clientIpBytes = new byte[clientIpLength];
+ buffer.get(clientIpBytes);
+ node.setClientIp(new String(clientIpBytes));
+ }
+
+ public static void deserialize(Node node, DataInputStream stream) throws
IOException {
+ int ipLength = stream.readInt();
+ byte[] ipBytes = new byte[ipLength];
+ int readIpSize = stream.read(ipBytes);
+ if (readIpSize != ipLength) {
+ throw new IOException(
+ String.format(
+ "No sufficient bytes read when deserializing the ip of a node:
%d/%d",
+ readIpSize, ipLength));
+ }
+ node.setInternalIp(new String(ipBytes));
+ node.setMetaPort(stream.readInt());
+ node.setNodeIdentifier(stream.readInt());
+ node.setDataPort(stream.readInt());
+ node.setClientPort(stream.readInt());
+
+ int clientIpLength = stream.readInt();
+ byte[] clientIpBytes = new byte[clientIpLength];
+ int readClientIpSize = stream.read(clientIpBytes);
+ if (readClientIpSize != clientIpLength) {
+ throw new IOException(
+ String.format(
+ "No sufficient bytes read when deserializing the clientIp of a
node: %d/%d",
+ readClientIpSize, clientIpLength));
+ }
+ node.setClientIp(new String(clientIpBytes));
+ }
+}
diff --git
a/cluster/src/test/java/org/apache/iotdb/cluster/utils/SerializeUtilTest.java
b/cluster/src/test/java/org/apache/iotdb/cluster/utils/SerializeUtilTest.java
index 8fabcb5..0de5cbb 100644
---
a/cluster/src/test/java/org/apache/iotdb/cluster/utils/SerializeUtilTest.java
+++
b/cluster/src/test/java/org/apache/iotdb/cluster/utils/SerializeUtilTest.java
@@ -34,6 +34,7 @@ import org.apache.iotdb.tsfile.file.metadata.enums.TSDataType;
import org.apache.iotdb.tsfile.file.metadata.enums.TSEncoding;
import org.apache.iotdb.tsfile.utils.Binary;
+import org.junit.Assert;
import org.junit.Test;
import java.io.ByteArrayOutputStream;
@@ -178,4 +179,16 @@ public class SerializeUtilTest {
Log parsed = LogParser.getINSTANCE().parse(serialized);
assertEquals(log, parsed);
}
+
+ @Test
+ public void serdesNodeTest() {
+ Node node = new Node("127.0.0.1", 6667, 1, 6535, 4678, "127.0.0.1");
+ ByteArrayOutputStream baos = new ByteArrayOutputStream();
+ DataOutputStream outputStream = new DataOutputStream(baos);
+ NodeSerializeUtils.serialize(node, outputStream);
+ ByteBuffer buffer = ByteBuffer.wrap(baos.toByteArray());
+ Node anotherNode = new Node("127.0.0.1", 6667, 1, 6535, 4678, "127.0.0.1");
+ NodeSerializeUtils.deserialize(anotherNode, buffer);
+ Assert.assertEquals(node, anotherNode);
+ }
}
diff --git a/code-coverage/pom.xml b/code-coverage/pom.xml
index 6964883..13a3ffd 100644
--- a/code-coverage/pom.xml
+++ b/code-coverage/pom.xml
@@ -67,6 +67,12 @@
<artifactId>jacoco-maven-plugin</artifactId>
<version>0.8.5</version>
<configuration>
+ <excludes>
+
<exclude>org/apache/iotdb/service/sync/thrift/*</exclude>
+
<exclude>org/apache/iotdb/service/rpc/thrift/*</exclude>
+
<exclude>org/apache/iotdb/cluster/rpc/thrift/*</exclude>
+ <exclude>org/apache/iotdb/db/qp/sql/*</exclude>
+ </excludes>
<rules>
<rule
implementation="org.jacoco.maven.RuleConfiguration">
<element>BUNDLE</element>
diff --git a/docs/Development/ContributeGuide.md
b/docs/Development/ContributeGuide.md
index 1f98bba..3dea6c0 100644
--- a/docs/Development/ContributeGuide.md
+++ b/docs/Development/ContributeGuide.md
@@ -96,6 +96,17 @@
plugin](https://github.com/diffplug/spotless/tree/main/plugin-maven) together wi
5. Install the [Save Actions
plugin](https://plugins.jetbrains.com/plugin/7642-save-actions) , and
enable the plugin, along with "Optimize imports" and "Reformat file"
6. In the "Save Actions" settings page, setup a "File Path Inclusion" for
`.*\.java`. Otherwise you will get unintended reformatting in other files you
edit.
+7. Fix the issues of reordering the import packages: in IDEA: choose:
Preferences | Editor | Code Style | Java | imports. At the tail of the panel,
there is "Import Layout", change it to:
+```shell
+ import org.apache.iotdb.*
+ <blank line>
+ import all other imports
+ <blank line>
+ import java.*
+ <blank line>
+ import static all other imports
+```
+
## Contributing code
diff --git a/grafana/pom.xml b/grafana/pom.xml
index ac5a8e9..a24ba06 100644
--- a/grafana/pom.xml
+++ b/grafana/pom.xml
@@ -59,6 +59,10 @@
</dependencyManagement>
<dependencies>
<dependency>
+ <groupId>com.google.code.gson</groupId>
+ <artifactId>gson</artifactId>
+ </dependency>
+ <dependency>
<groupId>org.apache.iotdb</groupId>
<artifactId>iotdb-jdbc</artifactId>
<version>${project.version}</version>
diff --git a/jdbc/pom.xml b/jdbc/pom.xml
index 41e1ffd..dd17752 100644
--- a/jdbc/pom.xml
+++ b/jdbc/pom.xml
@@ -38,11 +38,6 @@
<dependencies>
<dependency>
<groupId>org.apache.iotdb</groupId>
- <artifactId>tsfile</artifactId>
- <version>${project.version}</version>
- </dependency>
- <dependency>
- <groupId>org.apache.iotdb</groupId>
<artifactId>service-rpc</artifactId>
<version>${project.version}</version>
</dependency>
diff --git a/pom.xml b/pom.xml
index d2989b5..1c2b0f3 100644
--- a/pom.xml
+++ b/pom.xml
@@ -82,6 +82,8 @@
<module>tsfile</module>
<module>antlr</module>
<module>thrift</module>
+ <module>thrift-cluster</module>
+ <module>thrift-sync</module>
<module>service-rpc</module>
<module>jdbc</module>
<module>session</module>
@@ -489,6 +491,14 @@
</dependencyManagement>
<dependencies>
<dependency>
+ <groupId>ch.qos.logback</groupId>
+ <artifactId>logback-classic</artifactId>
+ </dependency>
+ <dependency>
+ <groupId>org.slf4j</groupId>
+ <artifactId>slf4j-api</artifactId>
+ </dependency>
+ <dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<scope>test</scope>
@@ -646,6 +656,8 @@
<!-- put all source folders not in src/main/java
here-->
<sourceDirectory>antlr/target/generated-sources/antlr4</sourceDirectory>
<sourceDirectory>thrift/target/generated-sources/thrift</sourceDirectory>
+
<sourceDirectory>thrift-sync/target/generated-sources/thrift</sourceDirectory>
+
<sourceDirectory>thrift-cluster/target/generated-sources/thrift</sourceDirectory>
<sourceDirectory>spark-iotdb-connector/src/main/scala</sourceDirectory>
<sourceDirectory>spark-tsfile/src/main/scala</sourceDirectory>
</sourceDirectories>
@@ -1201,6 +1213,12 @@
<artifactId>jacoco-maven-plugin</artifactId>
<version>0.8.5</version>
<configuration>
+ <excludes>
+
<exclude>org/apache/iotdb/service/sync/thrift/*</exclude>
+
<exclude>org/apache/iotdb/service/rpc/thrift/*</exclude>
+
<exclude>org/apache/iotdb/cluster/rpc/thrift/*</exclude>
+ <exclude>org/apache/iotdb/db/qp/sql/*</exclude>
+ </excludes>
<rules>
<rule
implementation="org.jacoco.maven.RuleConfiguration">
<element>BUNDLE</element>
diff --git a/server/pom.xml b/server/pom.xml
index a985d6a..dee952f 100644
--- a/server/pom.xml
+++ b/server/pom.xml
@@ -76,6 +76,11 @@
<scope>test</scope>
</dependency>
<dependency>
+ <groupId>org.apache.iotdb</groupId>
+ <artifactId>iotdb-thrift-sync</artifactId>
+ <version>${project.version}</version>
+ </dependency>
+ <dependency>
<groupId>org.apache.commons</groupId>
<artifactId>commons-collections4</artifactId>
</dependency>
diff --git a/server/src/main/java/org/apache/iotdb/db/utils/SerializeUtils.java
b/server/src/main/java/org/apache/iotdb/db/utils/SerializeUtils.java
index e4f3bda..cb881e0 100644
--- a/server/src/main/java/org/apache/iotdb/db/utils/SerializeUtils.java
+++ b/server/src/main/java/org/apache/iotdb/db/utils/SerializeUtils.java
@@ -19,7 +19,6 @@
package org.apache.iotdb.db.utils;
-import org.apache.iotdb.cluster.rpc.thrift.Node;
import org.apache.iotdb.tsfile.common.conf.TSFileConfig;
import org.apache.iotdb.tsfile.file.metadata.enums.TSDataType;
import org.apache.iotdb.tsfile.read.TimeValuePair;
@@ -30,7 +29,6 @@ import org.apache.iotdb.tsfile.utils.ReadWriteIOUtils;
import org.apache.iotdb.tsfile.utils.TsPrimitiveType;
import java.io.ByteArrayOutputStream;
-import java.io.DataInputStream;
import java.io.DataOutputStream;
import java.io.IOException;
import java.nio.ByteBuffer;
@@ -119,66 +117,6 @@ public class SerializeUtils {
}
}
- public static void serialize(Node node, DataOutputStream dataOutputStream) {
- try {
- byte[] internalIpBytes = node.internalIp.getBytes();
- dataOutputStream.writeInt(internalIpBytes.length);
- dataOutputStream.write(internalIpBytes);
- dataOutputStream.writeInt(node.metaPort);
- dataOutputStream.writeInt(node.nodeIdentifier);
- dataOutputStream.writeInt(node.dataPort);
- dataOutputStream.writeInt(node.clientPort);
- byte[] clientIpBytes = node.clientIp.getBytes();
- dataOutputStream.writeInt(clientIpBytes.length);
- dataOutputStream.write(clientIpBytes);
- } catch (IOException e) {
- // unreachable
- }
- }
-
- public static void deserialize(Node node, ByteBuffer buffer) {
- int internalIpLength = buffer.getInt();
- byte[] internalIpBytes = new byte[internalIpLength];
- buffer.get(internalIpBytes);
- node.setInternalIp(new String(internalIpBytes));
- node.setMetaPort(buffer.getInt());
- node.setNodeIdentifier(buffer.getInt());
- node.setDataPort(buffer.getInt());
- node.setClientPort(buffer.getInt());
- int clientIpLength = buffer.getInt();
- byte[] clientIpBytes = new byte[clientIpLength];
- buffer.get(clientIpBytes);
- node.setClientIp(new String(clientIpBytes));
- }
-
- public static void deserialize(Node node, DataInputStream stream) throws
IOException {
- int ipLength = stream.readInt();
- byte[] ipBytes = new byte[ipLength];
- int readIpSize = stream.read(ipBytes);
- if (readIpSize != ipLength) {
- throw new IOException(
- String.format(
- "No sufficient bytes read when deserializing the ip of a node:
%d/%d",
- readIpSize, ipLength));
- }
- node.setInternalIp(new String(ipBytes));
- node.setMetaPort(stream.readInt());
- node.setNodeIdentifier(stream.readInt());
- node.setDataPort(stream.readInt());
- node.setClientPort(stream.readInt());
-
- int clientIpLength = stream.readInt();
- byte[] clientIpBytes = new byte[clientIpLength];
- int readClientIpSize = stream.read(clientIpBytes);
- if (readClientIpSize != clientIpLength) {
- throw new IOException(
- String.format(
- "No sufficient bytes read when deserializing the clientIp of a
node: %d/%d",
- readClientIpSize, clientIpLength));
- }
- node.setClientIp(new String(clientIpBytes));
- }
-
public static void serializeBatchData(BatchData batchData, DataOutputStream
outputStream) {
try {
int length = batchData.length();
diff --git
a/server/src/test/java/org/apache/iotdb/db/utils/SerializeUtilsTest.java
b/server/src/test/java/org/apache/iotdb/db/utils/SerializeUtilsTest.java
index ba91d9f..5924589 100644
--- a/server/src/test/java/org/apache/iotdb/db/utils/SerializeUtilsTest.java
+++ b/server/src/test/java/org/apache/iotdb/db/utils/SerializeUtilsTest.java
@@ -18,7 +18,6 @@
*/
package org.apache.iotdb.db.utils;
-import org.apache.iotdb.cluster.rpc.thrift.Node;
import org.apache.iotdb.tsfile.file.metadata.enums.TSDataType;
import org.apache.iotdb.tsfile.read.TimeValuePair;
import org.apache.iotdb.tsfile.read.common.BatchData;
@@ -85,18 +84,6 @@ public class SerializeUtilsTest {
}
@Test
- public void serdesNodeTest() {
- Node node = new Node("127.0.0.1", 6667, 1, 6535, 4678, "127.0.0.1");
- ByteArrayOutputStream baos = new ByteArrayOutputStream();
- DataOutputStream outputStream = new DataOutputStream(baos);
- SerializeUtils.serialize(node, outputStream);
- ByteBuffer buffer = ByteBuffer.wrap(baos.toByteArray());
- Node anotherNode = new Node("127.0.0.1", 6667, 1, 6535, 4678, "127.0.0.1");
- SerializeUtils.deserialize(anotherNode, buffer);
- Assert.assertEquals(node, anotherNode);
- }
-
- @Test
public void serdesINT32BatchDataTest() {
BatchData batchData = new BatchData(TSDataType.INT32);
int ivalue = 0;
diff --git a/service-rpc/pom.xml b/service-rpc/pom.xml
index 83995c8..f517fdd 100644
--- a/service-rpc/pom.xml
+++ b/service-rpc/pom.xml
@@ -40,12 +40,23 @@
<groupId>org.apache.iotdb</groupId>
<artifactId>tsfile</artifactId>
<version>${project.version}</version>
+ <!-- we want to reduce the size of JDBC -->
+ <exclusions>
+ <exclusion>
+ <groupId>*</groupId>
+ <artifactId>*</artifactId>
+ </exclusion>
+ </exclusions>
</dependency>
<dependency>
<groupId>org.apache.iotdb</groupId>
<artifactId>iotdb-thrift</artifactId>
<version>${project.version}</version>
</dependency>
+ <dependency>
+ <groupId>org.xerial.snappy</groupId>
+ <artifactId>snappy-java</artifactId>
+ </dependency>
</dependencies>
<build>
<plugins>
diff --git a/session/pom.xml b/session/pom.xml
index 3e3bb71..e1330c5 100644
--- a/session/pom.xml
+++ b/session/pom.xml
@@ -73,11 +73,6 @@
</dependency>
<dependency>
<groupId>org.apache.iotdb</groupId>
- <artifactId>tsfile</artifactId>
- <version>${project.version}</version>
- </dependency>
- <dependency>
- <groupId>org.apache.iotdb</groupId>
<artifactId>iotdb-server</artifactId>
<version>${project.version}</version>
<type>test-jar</type>
@@ -95,6 +90,10 @@
<version>${project.version}</version>
<scope>test</scope>
</dependency>
+ <dependency>
+ <groupId>commons-io</groupId>
+ <artifactId>commons-io</artifactId>
+ </dependency>
</dependencies>
<profiles>
<profile>
diff --git a/thrift-cluster/README.md b/thrift-cluster/README.md
new file mode 100644
index 0000000..b83e1e1
--- /dev/null
+++ b/thrift-cluster/README.md
@@ -0,0 +1,22 @@
+<!--
+
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+
+-->
+
+This modules maintains all RPC interfaces among servers for the cluster module.
\ No newline at end of file
diff --git a/thrift-cluster/pom.xml b/thrift-cluster/pom.xml
new file mode 100644
index 0000000..f4c4eea
--- /dev/null
+++ b/thrift-cluster/pom.xml
@@ -0,0 +1,67 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!--
+
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+
+-->
+<project xmlns="http://maven.apache.org/POM/4.0.0";
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance";
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
http://maven.apache.org/xsd/maven-4.0.0.xsd";>
+ <modelVersion>4.0.0</modelVersion>
+ <parent>
+ <groupId>org.apache.iotdb</groupId>
+ <artifactId>iotdb-parent</artifactId>
+ <version>0.12.0-SNAPSHOT</version>
+ <relativePath>../pom.xml</relativePath>
+ </parent>
+ <artifactId>iotdb-thrift-cluster</artifactId>
+ <name>rpc-thrift-cluster</name>
+ <description>RPC (Thrift) framework among servers.</description>
+ <dependencies>
+ <dependency>
+ <groupId>org.apache.thrift</groupId>
+ <artifactId>libthrift</artifactId>
+ </dependency>
+ <dependency>
+ <groupId>org.apache.iotdb</groupId>
+ <artifactId>iotdb-thrift</artifactId>
+ <version>${project.version}</version>
+ </dependency>
+ </dependencies>
+ <build>
+ <plugins>
+ <plugin>
+ <groupId>org.codehaus.mojo</groupId>
+ <artifactId>build-helper-maven-plugin</artifactId>
+ <version>3.2.0</version>
+ <executions>
+ <execution>
+ <id>add-source</id>
+ <phase>generate-sources</phase>
+ <goals>
+ <goal>add-source</goal>
+ </goals>
+ <configuration>
+ <sources>
+
<source>${project.build.directory}/generated-sources/thrift</source>
+ </sources>
+ </configuration>
+ </execution>
+ </executions>
+ </plugin>
+ </plugins>
+ </build>
+</project>
diff --git a/thrift-cluster/rpc-changelist.md b/thrift-cluster/rpc-changelist.md
new file mode 100644
index 0000000..797c0b6
--- /dev/null
+++ b/thrift-cluster/rpc-changelist.md
@@ -0,0 +1,181 @@
+<!--
+
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+
+-->
+# 0.11.x(version-2) -> 0.12.x(version-1)
+
+Last Updated on 2021.01.19 by Xiangwei Wei.
+
+
+## 1. Delete Old
+
+| Latest Changes | Related Committers |
+| ---------------------------------- | ------------------ |
+
+
+## 2. Add New
+
+| Latest Changes | Related
Committers |
+| ------------------------------------------------------------ |
---------------------- |
+| Add timeout in TSFetchResultsReq and TSExecuteStatementReq | Xiangwei Wei |
+
+
+## 3. Update
+
+| Latest Changes | Related
Committers |
+| ------------------------------------------------------------ |
---------------------- |
+
+
+# 0.10.x (version-2) -> 0.11.x (version-3)
+
+Last Updated on 2020-10-27 by Xiangwei Wei.
+
+
+## 1. Delete Old
+
+| Latest Changes | Related Committers |
+| ---------------------------------- | ------------------ |
+| Remove TSBatchExecuteStatementResp | Tian Jiang |
+
+
+## 2. Add New
+
+| Latest Changes | Related
Committers |
+| ------------------------------------------------------------ |
---------------------- |
+| set the input/output as TFramedTransport | Tian Jiang |
+| Add timeout(optional) in TSFetchResultsReq and TSExecuteStatementReq |
Xiangwei Wei |
+
+
+## 3. Update
+
+| Latest Changes | Related
Committers |
+| ------------------------------------------------------------ |
---------------------- |
+| Add sub-status in TSStatus | Tian Jiang |
+| Change the result of executeBatchStatement as TSStatus | Tian Jiang |
+| Change TSDeleteDataReq, delete timestamp and add startTime and endTime |
Wei Shao |
+| Add zoneId in TSOpenSessionReq | Xiangwei Wei |
+
+
+# 0.9.x (version-1) -> 0.10.x (version-2)
+
+Last Updated on 2020-5-25 by Kaifeng Xue.
+
+
+## 1. Delete Old
+
+| Latest Changes | Related Committers |
+| ---------------------------------- | ------------------ |
+| Remove TS_SessionHandle,TSHandleIdentifier | Tian Jiang |
+| Remove TSStatus,TSExecuteInsertRowInBatchResp | Jialin Qiao|
+
+
+## 2. Add New
+
+| Latest Changes | Related
Committers |
+| ------------------------------------------------------------ |
---------------------------------- |
+| Add parameter sessionId in getTimeZone, getProperties, setStorageGroup,
createTimeseries... | Tian Jiang|
+| Add struct TSQueryNonAlignDataSet | Haonan Hou|
+| Add struct TSInsertTabletsReq | Jialin Qiao|
+| Add method insertTablets | Jialin Qiao|
+| Add method testInsertTablets | Xiangdong Huang |
+| add new field `inferType` in TSInsertRecordReq | Jialin Qiao |
+
+## 3. Update
+
+| Latest Changes | Related
Committers |
+| ------------------------------------------------------------ |
---------------------- |
+| Replace TS_SessionHandles with SessionIds, TSOperationHandle with queryIds
| Tian Jiang |
+| Add optional TSQueryNonAlignDataSet in TSExecuteStatementResp,
TSFetchResultsResp and required bool isAlign in TSFetchResultsReq | Haonan Hou |
+| Rename TSStatusType to TSStatus | Jialin Qiao |
+| Remove sessionId in TSExecuteBatchStatementResp | Jialin Qiao |
+| Rename insertRows to insertReords, insert to insertRecord, insertBatch to
insertTablet | Jialin Qiao |
+| Use TsDataType and binary rather than string in TSInsertInBatchReq and
TSInsertReq | Kaifeng Xue |
+
+
+
+# 0.8.x -> 0.9.x (version-1)
+
+Last Updated on 2019-10-27 by Lei Rui.
+
+
+## 1. Delete Old
+
+| Latest Changes | Related Committers |
+| ---------------------------------- | ------------------ |
+| Delete struct TSSetStorageGroupReq | Jialin Qiao |
+| Remove struct TSDataValue | Lei Rui |
+| Remove struct TSRowRecord | Lei Rui |
+| Remove optional string version in TSFetchMetadataResp | Genius_pig |
+| Remove optional set<string> childPaths, nodesList, storageGroups, devices in
TSFetchMetadataResp | Genius_pig |
+| Remove optional map<string, string> nodeTimeseriesNum in TSFetchMetadataResp
| Genius_pig |
+| Remove optional list<list<string>> timeseriesList in TSFetchMetadataResp |
Genius_pig |
+| Remove optinoal optional i32 timeseriesNum in TSFetchMetadataResp |
Genius_pig |
+| Remove optional i32 nodeLevel in TSFetchMetadataReq | Genius_pig |
+
+
+## 2. Add New
+
+| Latest Changes | Related
Committers |
+| ------------------------------------------------------------ |
---------------------------------- |
+| Add struct TSBatchInsertionReq | qiaojialin
|
+| Add method TSExecuteBatchStatementResp insertBatch(1:TSBatchInsertionReq
req) | qiaojialin |
+| Add Struct TSStatusType | Zesong Sun
|
+| Add TSCreateTimeseriesReq | Zesong Sun
|
+| Add method TSStatus setStorageGroup(1:string storageGroup) | Zesong Sun,
Jialin Qiao |
+| Add method TSStatus createTimeseries(1:TSCreateTimeseriesReq req) | Zesong
Sun |
+| Add struct TSInsertReq | qiaojialin
|
+| Add method TSRPCResp insertRow(1:TSInsertReq req) | qiaojialin
|
+| Add struct TSDeleteDataReq | Jack Tsai,
qiaojialin |
+| Add method TSStatus deleteData(1:TSDeleteDataReq req) | Jack Tsai,
Jialin Qiao, qiaojialin |
+| Add method TSStatus deleteTimeseries(1:list\<string> path) | qiaojialin
|
+| Add method TSStatus deleteStorageGroups(1:list\<string> storageGroup) | Yi
Tao |
+| Add Struct TSExecuteInsertRowInBatchResp | Kaifeng Xue |
+| Add method insertRowInBatch(1:TSInsertInBatchReq req); | Kaifeng Xue |
+| Add method testInsertRowInBatch(1:TSInsertInBatchReq req); | Kaifeng Xue |
+| Add method testInsertRow(1:TSInsertReq req); | Kaifeng Xue |
+| Add method testInsertBatch(1:TSBatchInsertionReq req); | Kaifeng Xue |
+| Add struct TSCreateMultiTimeseriesReq | qiaojialin |
+| Add method createMultiTimeseries(1:TSCreateMultiTimeseriesReq req); |
qiaojialin |
+
+
+## 3. Update
+
+| Latest Changes | Related
Committers |
+| ------------------------------------------------------------ |
---------------------- |
+| Add required string timestampPrecision in ServerProperties | 1160300922
|
+| Add optional list\<string\> dataTypeList in TSExecuteStatementResp | suyue
|
+| Update TSStatus to use TSStatusType, instead of using ~~TS_StatusCode,
errorCode and errorMessage~~ | Zesong Sun |
+| Rename item in enum TSProtocolVersion from ~~TSFILE_SERVICE_PROTOCOL_V1~~ to
IOTDB_SERVICE_PROTOCOL_V1 | qiaojialin |
+| Rename method name from ~~TSExecuteStatementResp
executeInsertion(1:TSInsertionReq req)~~ to TSExecuteStatementResp
insert(1:TSInsertionReq req) | qiaojialin |
+| Add required i32 compressor in TSCreateTimeseriesReq | Jialin Qiao
|
+| Add optional list\<string> nodesList, optional map\<string, string>
nodeTimeseriesNum in TSFetchMetadataResp | jack870131 |
+| Add optional i32 nodeLevel in TSFetchMetadataReq | jack870131,
Zesong Sun |
+| Change the following methods' returned type to be TSStatus: <br />TSStatus
closeSession(1:TSCloseSessionReq req), <br />TSStatus
cancelOperation(1:TSCancelOperationReq req), <br />TSStatus
closeOperation(1:TSCloseOperationReq req), <br />TSStatus
setTimeZone(1:TSSetTimeZoneReq req), <br />TSStatus setStorageGroup(1:string
storageGroup), <br />TSStatus createTimeseries(1:TSCreateTimeseriesReq req),
<br />TSStatus insertRow(1:TSInsertReq req), <br />TSStatus
deleteData(1:TSDeleteDataReq [...]
+| Change from ~~required string path~~ to required list\<string> paths in
TSDeleteDataReq | qiaojialin |
+| Add optional set\<string> devices in TSFetchMetadataResp | Zesong Sun
|
+| Rename some fields in TSFetchMetadataResp: ~~ColumnsList~~ to columnsList,
~~showTimeseriesList~~ to timeseriesList, ~~showStorageGroups~~ to
storageGroups | Zesong Sun |
+| Change struct TSQueryDataSet to eliminate row-wise rpc writing | Lei Rui
|
+| Add optional i32 timeseriesNum in TSFetchMetadataResp | Jack Tsai
|
+| Add required i64 queryId in TSHandleIdentifier | Yuan Tian |
+| Add optional set\<string> childPaths in TSFetchMetadataResp | Haonan Hou
|
+| Add optional string version in TSFetchMetadataResp | Genius_pig
|
+| Add required i64 statementId in TSExecuteStatementReq | Yuan Tian |
+| Add required binary time, required list<binary> valueList, required
list<binary> bitmapList and remove required binary values, required i32
rowCount in TSQueryDataSet| Yuan Tian |
+| Add optional i32 fetchSize in TSExecuteStatementReq,<br />Add optional
TSQueryDataSet in TSExecuteStatementResp| liutaohua |
+| Add optional map<string, string> props, optional map<string, string> tags,
optional map<string, string> attributes and optional string aliasPath in
TSCreateTimeseriesReq | Yuan Tian |
diff --git a/thrift/src/main/thrift/cluster.thrift
b/thrift-cluster/src/main/thrift/cluster.thrift
similarity index 100%
rename from thrift/src/main/thrift/cluster.thrift
rename to thrift-cluster/src/main/thrift/cluster.thrift
diff --git a/thrift-sync/README.md b/thrift-sync/README.md
new file mode 100644
index 0000000..94c1de6
--- /dev/null
+++ b/thrift-sync/README.md
@@ -0,0 +1,22 @@
+<!--
+
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+
+-->
+
+This modules maintains all RPC interfaces for data synchronization among
servers.
\ No newline at end of file
diff --git a/thrift-sync/pom.xml b/thrift-sync/pom.xml
new file mode 100644
index 0000000..111b3c9
--- /dev/null
+++ b/thrift-sync/pom.xml
@@ -0,0 +1,67 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!--
+
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+
+-->
+<project xmlns="http://maven.apache.org/POM/4.0.0";
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance";
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
http://maven.apache.org/xsd/maven-4.0.0.xsd";>
+ <modelVersion>4.0.0</modelVersion>
+ <parent>
+ <groupId>org.apache.iotdb</groupId>
+ <artifactId>iotdb-parent</artifactId>
+ <version>0.12.0-SNAPSHOT</version>
+ <relativePath>../pom.xml</relativePath>
+ </parent>
+ <artifactId>iotdb-thrift-sync</artifactId>
+ <name>rpc-thrift-sync</name>
+ <description>RPC (Thrift) framework among servers for data
synchronization.</description>
+ <dependencies>
+ <dependency>
+ <groupId>org.apache.thrift</groupId>
+ <artifactId>libthrift</artifactId>
+ </dependency>
+ <dependency>
+ <groupId>org.apache.iotdb</groupId>
+ <artifactId>iotdb-thrift</artifactId>
+ <version>${project.version}</version>
+ </dependency>
+ </dependencies>
+ <build>
+ <plugins>
+ <plugin>
+ <groupId>org.codehaus.mojo</groupId>
+ <artifactId>build-helper-maven-plugin</artifactId>
+ <version>3.2.0</version>
+ <executions>
+ <execution>
+ <id>add-source</id>
+ <phase>generate-sources</phase>
+ <goals>
+ <goal>add-source</goal>
+ </goals>
+ <configuration>
+ <sources>
+
<source>${project.build.directory}/generated-sources/thrift</source>
+ </sources>
+ </configuration>
+ </execution>
+ </executions>
+ </plugin>
+ </plugins>
+ </build>
+</project>
diff --git a/thrift-sync/rpc-changelist.md b/thrift-sync/rpc-changelist.md
new file mode 100644
index 0000000..797c0b6
--- /dev/null
+++ b/thrift-sync/rpc-changelist.md
@@ -0,0 +1,181 @@
+<!--
+
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+
+-->
+# 0.11.x(version-2) -> 0.12.x(version-1)
+
+Last Updated on 2021.01.19 by Xiangwei Wei.
+
+
+## 1. Delete Old
+
+| Latest Changes | Related Committers |
+| ---------------------------------- | ------------------ |
+
+
+## 2. Add New
+
+| Latest Changes | Related
Committers |
+| ------------------------------------------------------------ |
---------------------- |
+| Add timeout in TSFetchResultsReq and TSExecuteStatementReq | Xiangwei Wei |
+
+
+## 3. Update
+
+| Latest Changes | Related
Committers |
+| ------------------------------------------------------------ |
---------------------- |
+
+
+# 0.10.x (version-2) -> 0.11.x (version-3)
+
+Last Updated on 2020-10-27 by Xiangwei Wei.
+
+
+## 1. Delete Old
+
+| Latest Changes | Related Committers |
+| ---------------------------------- | ------------------ |
+| Remove TSBatchExecuteStatementResp | Tian Jiang |
+
+
+## 2. Add New
+
+| Latest Changes | Related
Committers |
+| ------------------------------------------------------------ |
---------------------- |
+| set the input/output as TFramedTransport | Tian Jiang |
+| Add timeout(optional) in TSFetchResultsReq and TSExecuteStatementReq |
Xiangwei Wei |
+
+
+## 3. Update
+
+| Latest Changes | Related
Committers |
+| ------------------------------------------------------------ |
---------------------- |
+| Add sub-status in TSStatus | Tian Jiang |
+| Change the result of executeBatchStatement as TSStatus | Tian Jiang |
+| Change TSDeleteDataReq, delete timestamp and add startTime and endTime |
Wei Shao |
+| Add zoneId in TSOpenSessionReq | Xiangwei Wei |
+
+
+# 0.9.x (version-1) -> 0.10.x (version-2)
+
+Last Updated on 2020-5-25 by Kaifeng Xue.
+
+
+## 1. Delete Old
+
+| Latest Changes | Related Committers |
+| ---------------------------------- | ------------------ |
+| Remove TS_SessionHandle,TSHandleIdentifier | Tian Jiang |
+| Remove TSStatus,TSExecuteInsertRowInBatchResp | Jialin Qiao|
+
+
+## 2. Add New
+
+| Latest Changes | Related
Committers |
+| ------------------------------------------------------------ |
---------------------------------- |
+| Add parameter sessionId in getTimeZone, getProperties, setStorageGroup,
createTimeseries... | Tian Jiang|
+| Add struct TSQueryNonAlignDataSet | Haonan Hou|
+| Add struct TSInsertTabletsReq | Jialin Qiao|
+| Add method insertTablets | Jialin Qiao|
+| Add method testInsertTablets | Xiangdong Huang |
+| add new field `inferType` in TSInsertRecordReq | Jialin Qiao |
+
+## 3. Update
+
+| Latest Changes | Related
Committers |
+| ------------------------------------------------------------ |
---------------------- |
+| Replace TS_SessionHandles with SessionIds, TSOperationHandle with queryIds
| Tian Jiang |
+| Add optional TSQueryNonAlignDataSet in TSExecuteStatementResp,
TSFetchResultsResp and required bool isAlign in TSFetchResultsReq | Haonan Hou |
+| Rename TSStatusType to TSStatus | Jialin Qiao |
+| Remove sessionId in TSExecuteBatchStatementResp | Jialin Qiao |
+| Rename insertRows to insertReords, insert to insertRecord, insertBatch to
insertTablet | Jialin Qiao |
+| Use TsDataType and binary rather than string in TSInsertInBatchReq and
TSInsertReq | Kaifeng Xue |
+
+
+
+# 0.8.x -> 0.9.x (version-1)
+
+Last Updated on 2019-10-27 by Lei Rui.
+
+
+## 1. Delete Old
+
+| Latest Changes | Related Committers |
+| ---------------------------------- | ------------------ |
+| Delete struct TSSetStorageGroupReq | Jialin Qiao |
+| Remove struct TSDataValue | Lei Rui |
+| Remove struct TSRowRecord | Lei Rui |
+| Remove optional string version in TSFetchMetadataResp | Genius_pig |
+| Remove optional set<string> childPaths, nodesList, storageGroups, devices in
TSFetchMetadataResp | Genius_pig |
+| Remove optional map<string, string> nodeTimeseriesNum in TSFetchMetadataResp
| Genius_pig |
+| Remove optional list<list<string>> timeseriesList in TSFetchMetadataResp |
Genius_pig |
+| Remove optinoal optional i32 timeseriesNum in TSFetchMetadataResp |
Genius_pig |
+| Remove optional i32 nodeLevel in TSFetchMetadataReq | Genius_pig |
+
+
+## 2. Add New
+
+| Latest Changes | Related
Committers |
+| ------------------------------------------------------------ |
---------------------------------- |
+| Add struct TSBatchInsertionReq | qiaojialin
|
+| Add method TSExecuteBatchStatementResp insertBatch(1:TSBatchInsertionReq
req) | qiaojialin |
+| Add Struct TSStatusType | Zesong Sun
|
+| Add TSCreateTimeseriesReq | Zesong Sun
|
+| Add method TSStatus setStorageGroup(1:string storageGroup) | Zesong Sun,
Jialin Qiao |
+| Add method TSStatus createTimeseries(1:TSCreateTimeseriesReq req) | Zesong
Sun |
+| Add struct TSInsertReq | qiaojialin
|
+| Add method TSRPCResp insertRow(1:TSInsertReq req) | qiaojialin
|
+| Add struct TSDeleteDataReq | Jack Tsai,
qiaojialin |
+| Add method TSStatus deleteData(1:TSDeleteDataReq req) | Jack Tsai,
Jialin Qiao, qiaojialin |
+| Add method TSStatus deleteTimeseries(1:list\<string> path) | qiaojialin
|
+| Add method TSStatus deleteStorageGroups(1:list\<string> storageGroup) | Yi
Tao |
+| Add Struct TSExecuteInsertRowInBatchResp | Kaifeng Xue |
+| Add method insertRowInBatch(1:TSInsertInBatchReq req); | Kaifeng Xue |
+| Add method testInsertRowInBatch(1:TSInsertInBatchReq req); | Kaifeng Xue |
+| Add method testInsertRow(1:TSInsertReq req); | Kaifeng Xue |
+| Add method testInsertBatch(1:TSBatchInsertionReq req); | Kaifeng Xue |
+| Add struct TSCreateMultiTimeseriesReq | qiaojialin |
+| Add method createMultiTimeseries(1:TSCreateMultiTimeseriesReq req); |
qiaojialin |
+
+
+## 3. Update
+
+| Latest Changes | Related
Committers |
+| ------------------------------------------------------------ |
---------------------- |
+| Add required string timestampPrecision in ServerProperties | 1160300922
|
+| Add optional list\<string\> dataTypeList in TSExecuteStatementResp | suyue
|
+| Update TSStatus to use TSStatusType, instead of using ~~TS_StatusCode,
errorCode and errorMessage~~ | Zesong Sun |
+| Rename item in enum TSProtocolVersion from ~~TSFILE_SERVICE_PROTOCOL_V1~~ to
IOTDB_SERVICE_PROTOCOL_V1 | qiaojialin |
+| Rename method name from ~~TSExecuteStatementResp
executeInsertion(1:TSInsertionReq req)~~ to TSExecuteStatementResp
insert(1:TSInsertionReq req) | qiaojialin |
+| Add required i32 compressor in TSCreateTimeseriesReq | Jialin Qiao
|
+| Add optional list\<string> nodesList, optional map\<string, string>
nodeTimeseriesNum in TSFetchMetadataResp | jack870131 |
+| Add optional i32 nodeLevel in TSFetchMetadataReq | jack870131,
Zesong Sun |
+| Change the following methods' returned type to be TSStatus: <br />TSStatus
closeSession(1:TSCloseSessionReq req), <br />TSStatus
cancelOperation(1:TSCancelOperationReq req), <br />TSStatus
closeOperation(1:TSCloseOperationReq req), <br />TSStatus
setTimeZone(1:TSSetTimeZoneReq req), <br />TSStatus setStorageGroup(1:string
storageGroup), <br />TSStatus createTimeseries(1:TSCreateTimeseriesReq req),
<br />TSStatus insertRow(1:TSInsertReq req), <br />TSStatus
deleteData(1:TSDeleteDataReq [...]
+| Change from ~~required string path~~ to required list\<string> paths in
TSDeleteDataReq | qiaojialin |
+| Add optional set\<string> devices in TSFetchMetadataResp | Zesong Sun
|
+| Rename some fields in TSFetchMetadataResp: ~~ColumnsList~~ to columnsList,
~~showTimeseriesList~~ to timeseriesList, ~~showStorageGroups~~ to
storageGroups | Zesong Sun |
+| Change struct TSQueryDataSet to eliminate row-wise rpc writing | Lei Rui
|
+| Add optional i32 timeseriesNum in TSFetchMetadataResp | Jack Tsai
|
+| Add required i64 queryId in TSHandleIdentifier | Yuan Tian |
+| Add optional set\<string> childPaths in TSFetchMetadataResp | Haonan Hou
|
+| Add optional string version in TSFetchMetadataResp | Genius_pig
|
+| Add required i64 statementId in TSExecuteStatementReq | Yuan Tian |
+| Add required binary time, required list<binary> valueList, required
list<binary> bitmapList and remove required binary values, required i32
rowCount in TSQueryDataSet| Yuan Tian |
+| Add optional i32 fetchSize in TSExecuteStatementReq,<br />Add optional
TSQueryDataSet in TSExecuteStatementResp| liutaohua |
+| Add optional map<string, string> props, optional map<string, string> tags,
optional map<string, string> attributes and optional string aliasPath in
TSCreateTimeseriesReq | Yuan Tian |
diff --git a/thrift/src/main/thrift/sync.thrift
b/thrift-sync/src/main/thrift/sync.thrift
similarity index 100%
rename from thrift/src/main/thrift/sync.thrift
rename to thrift-sync/src/main/thrift/sync.thrift
diff --git a/thrift/README.md b/thrift/README.md
new file mode 100644
index 0000000..1d105e8
--- /dev/null
+++ b/thrift/README.md
@@ -0,0 +1,22 @@
+<!--
+
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+
+-->
+
+This modules maintains all RPC interfaces between clients and the server.
\ No newline at end of file
