This is an automated email from the ASF dual-hosted git repository. yuyuankang pushed a commit to branch IT-doc in repository https://gitbox.apache.org/repos/asf/iotdb.git
commit 604448d045edb735c7a2003e1429b67306eadef9 Author: Ring-k <[email protected]> AuthorDate: Sun Jun 19 21:15:06 2022 +0800 update doc --- .../Integration-Test-refactoring-tutorial.md | 346 +++++++++++++------- .../Integration-Test-refactoring-tutorial.md | 352 +++++++++++++-------- 2 files changed, 455 insertions(+), 243 deletions(-) diff --git a/docs/UserGuide/Integration-Test/Integration-Test-refactoring-tutorial.md b/docs/UserGuide/Integration-Test/Integration-Test-refactoring-tutorial.md index 449ec27ff5..1d87dde33e 100644 --- a/docs/UserGuide/Integration-Test/Integration-Test-refactoring-tutorial.md +++ b/docs/UserGuide/Integration-Test/Integration-Test-refactoring-tutorial.md @@ -19,143 +19,257 @@ --> -# Integration Test refactoring tutorial +# Developer Document for Integration Test -- Step 0. Prerequisites +**Integration test** is one of the phases in software testing, when different software modules are put together and tested as a whole. Integration tests are for evaluating whether a system or component meets the target functional requirements. - - **IT Location has been moved**; all Integration Tests have been moved to the integration module. - - **Test case MUST label**; the test classification label is the junit's `category`, which determines the test environment or process in which the test case is tested. - - **Code related to the test environment MAY need to be refactored**; this determines whether the current test environment can correctly test the test case. The corresponding statement needs to be refactored. +## Apache IoTDB Integration Test Criteria +### The Environment of the integration test of Apache IoTDB -- Step 1. Labeling test cases - - Add the appropriate `Category` before the test case class or test case method, which can collect any desired test category labels. +There are three kinds of environments for Apache IoTDB integration test, correspondingly **local standalone, Cluster, and remote.** The integration test should be conducted on at least one of them. Details of the three kinds are as follows. - - The `Category` of the following three test categories are all real and effective, +1. Local standalone. It is set up for integration testing of IoTDB, the standalone version. Any change of the configurations of IoTDB would require updating the configuration files before starting the database. +2. Cluster. It is set up for integration testing of IoTDB, the distribution version (pseudo-distribution). Any change of the configurations of IoTDB would require updating the configuration files before starting the database. +3. Remote. It is set up for the integration testing of a remote IoTDB instance, which could be either a standalone instance or a node in a remote cluster. Any change of the configuration is restricted and is not allowed currently. - ```java - @Category({LocalStandaloneTest.class, ClusterTest.class, RemoteTest.class}) - public class IoTDBAliasIT { - ...... - } - - - @Category({LocalStandaloneTest.class, ClusterTest.class}) - public class IoTDBAlignByDeviceIT { - ...... - } - - - @Category({LocalStandaloneTest.class}) - public class IoTDBArithmeticIT { - ...... - } - ``` +Integration test developers need to specify at least one of the environments when writing the tests. Please check the details below. - - You can also add `Category` at the test method level. - ```java - @Category({LocalStandaloneTest.class}) - public class IoTDBExampleIT { - - // This case can ONLY test in environment of local. - @Test - public void theStandaloneCase() { - ...... - } - - // This case can test in environment of local, cluster and remote. - @Test - @Category({ClusterTest.class, RemoteTest.class}) - public void theAllEnvCase() { - ...... - } - } - ``` +### Black-Box Testing - - At present, all test cases must at least add the `Category` of the stand-alone test, namely `LocalStandaloneTest.class`. +Black-box testing is a software testing method that evaluates the functionality of a program without regard to its internal structure or how it works. Developers do not need to understand the internal logic of the application for testing. **Apache IoTDB integration tests are conducted as black-box tests. Any test interacting with the system through JDBC or Session API is considered a black-box test case.** Moreover, the validation of the output should also be implemented through the JDBC [...] +### Steps of an integration test -- Step 2. Environmental code refactoring +Generally, there are three steps to finish the integration test, (1) constructing the test class and annotating the environment, (2) housekeeping to prepare for the test and clean up the environment after the test, and (3) implementing the logic of the integration test. To test IoTDB not under the default configuration, the configuration should be changed before the test, which will be introduced in section 4. - - If the test case needs to be tested in the Cluster or Remote environment, the environment-related code MUST be refactored accordingly. If it is only tested in the LocalStandalone environment, modifications are only recommended. (Not all test cases can be tested in the Cluster or Remote environment because statements that are limited by some functions, such as local file operations, cannot be refactored.) +<img style="width:100%; max-width:800px; max-height:600px; margin-left:auto; margin-right:auto; display:block;" src="https://github.com/apache/iotdb-bin-resources/blob/main/integration-test/pic/step.svg";> - | | LocalStandalone | Cluster | Remote | - | :------------------------- | :-------------: | :---------: | :---------: | - | setUp and tearDown | Recommend | Must | Must | - | getConnection | Recommend | Must | Must | - | change config | Recommend | Must | Not support | - | Local file operation | Won't change | Not support | Not support | - | Local descriptor operation | Won't change | Not support | Not support | - | restart operation | Won't change | Not support | Not support | +#### 1. Integration Test Class (IT Class) and Annotations - +When writing new IT classes, the developers are encouraged to create the new ones in the [integration-test](https://github.com/apache/iotdb/tree/master/integration-test) module. Except for the classes serving the other test cases, the classes containing integration tests to evaluate the functionality of IoTDB should be named "function"+"IT". For example, the test for auto-registration metadata in IoTDB is named “<font color=green>IoTDBAutoCreateSchema</font><font color=red>IT</font>”. - - The `setUp` and `tearDown` methods must be refactored in the Cluster and Remote environment +- Category`` Annotation. **When creating new IT classes, the ```@Category``` should be introduced explicitly**, and the test environment should be specified by ```LocalStandaloneIT.class```, ```ClusterIT.class```, and ```RemoteIT.class```, which corresponds to the Local Standalone, Cluster and Remote environment respectively. **In general, ```LocalStandaloneIT.class``` and ```ClusterIT.class``` should both be included**. Only in the case when some functionalities are only supported in t [...] +- RunWith Annotation. The ```@RunWith(IoTDBTestRunner.class)``` annotation should be included in every IT class. - ```java - @Category({LocalStandaloneTest.class, ClusterTest.class, RemoteTest.class}) - public class IoTDBAliasIT { - - @BeforeClass - public static void setUp() throws Exception { - // EnvironmentUtils.closeStatMonitor(); // orginal setup code - // EnvironmentUtils.envSetUp(); // orginal setup code - EnvFactory.getEnv().initBeforeClass(); // new initBeforeClass code - - insertData(); - } - - @AfterClass - public static void tearDown() throws Exception { - // EnvironmentUtils.cleanEnv(); // orginal code - EnvFactory.getEnv().cleanAfterClass(); // new cleanAfterClass code + +```java +// Introduce annotations to IoTDBAliasIT.class. The environments include local standalone, cluster and remote. +@RunWith(IoTDBTestRunner.class) +@Category({LocalStandaloneIT.class, ClusterIT.class, RemoteIT.class}) +public class IoTDBAliasIT { + ... +} + +// Introduce annotations to IoTDBAlignByDeviceIT.class. The environments include local standalone and cluster. +@RunWith(IoTDBTestRunner.class) +@Category({LocalStandaloneIT.class, ClusterIT.class}) +public class IoTDBAlignByDeviceIT { + ... +} +``` + +#### 2. Housekeeping to Prepare for the Test and Clean up the Environment after the Test + +Preparations before the test include starting an IoTDB (single or cluster) instance and preparing data for the test. The logic should be implemented in the ```setUp()``` method, and the method should follow the annotation ```@BeforeClass``` or ```@Before```. +The former means that this method is the first method executed for the IT class and is executed only once. The latter indicates that ```setUp()``` will be executed before each test method in the IT class. + +- Please start IoTDB instance through the factor class, i.e., ```EnvFactory.getEnv().initBeforeClass()```. +- Data preparation for the test includes registering storage groups, registering time series, and writing time series data as required by the test. It is recommended to implement a separate method within the IT class to prepare the data, such as ```insertData()```. +Please try to take advantage of the ```executeBatch()``` in JDBC or ```insertRecords()``` and ```insertTablets()``` in Session API if multiple statements or operations are to be executed. + +```java +@BeforeClass +public static void setUp() throws Exception { + // start an IoTDB instance + EnvFactory.getEnv().initBeforeClass(); + ... // data preparation +} +``` + +After the test, please clean up the environment by shut down the connections that have not been closed. This logic should be implemented in the ```tearDown()``` method. The ```tearDown()``` method follows the annotation ```@AfterClass``` or ```@After```. The former means that this method is the last method to execute for the IT class and is executed only once. The latter indicates that ```tearDown()``` will be executed after each test method in the IT class. + +- If the IoTDB connection is declared as an instance variable and is not closed after the test, please explicitly close it in the ```tearDown()``` method. +- The cleaning up should be implemented through the factory class, i.e., ```EnvFactory.getEnv().cleanAfterClass()```. + + +```java +@AfterClass +public static void tearDown() throws Exception { + ... // close the connection + // clean up the environment + EnvFactory.getEnv().cleanAfterClass(); +} +``` + +#### 3. Implementing the logic of IT + +IT of Apache IoTDB should be implemented as black-box testing. Please name the method as "functionality"+"Test", e.g., "<font color=green>selectWithAlias</font><font color=red>Test</font>". The interaction should be implemented through JDBC or Session API. + +1、With JDBC + +When using the JDBC interface, it is recommended that the connection be established in a try statement. Connections established in this way do not need to be closed in the tearDown method explicitly. Connections need to be established through the factory class, i.e., ```EnvFactory.getEnv().getConnection()```. It is not necessary to specify the IP address or port number. The sample code is shown below. + +```java +@Test +public void someFunctionTest(){ + try (Connection connection = EnvFactory.getEnv().getConnection(); + Statement statement = connection.createStatement()) { + ... // execute the statements and test the correctness + } catch (Exception e) { + e.printStackTrace(); + Assert.fail(); + } +} +``` + +Please note that, +- **It is required to use ```executeQuery()``` to query the data from the database and get the ResultSet.** +- **For updating the database without any return value, it is required to use ```execute()``` method to interact with the database.** +The sample code is as follows. + +```java +@Test +public void exampleTest() throws Exception { + try (Connection connection = EnvFactory.getEnv().getConnection(); + Statement statement = connection.createStatement()) { + // use execute() to set the storage groups + statement.execute("set storage group to root.sg"); + // use executeQuery() query the storage groups + try (ResultSet resultSet = statement.executeQuery("show storage group")) { + if (resultSet.next()) { + String storageGroupPath = resultSet.getString("storage group"); + Assert.assertEquals("root.sg", storageGroupPath); + } else { + Assert.fail("This ResultSet is empty."); } } - ``` + } +} +``` - +2、With Session API - - The `getConnection` must be refactored in Cluster and Remote environments +Currently, it is not recommended to implement IT with Session API. + +3、Annotations of Environment for the Test Methods + +For test methods, developers can also specify a test environment with the annotation before the method. It is important to note that a case with additional test environment annotations will be tested not only in the specified environment, but also in the environment of the IT class to which the use case belongs. The sample code is as follows. + + +```java +@RunWith(IoTDBTestRunner.class) +@Category({LocalStandaloneIT.class}) +public class IoTDBExampleIT { + + // This case will only be tested in a local stand-alone test environment + @Test + public void theStandaloneCaseTest() { + ... + } + + // The use case will be tested in the local standalone environment, the cluster environment, and the remote test environment. + @Test + @Category({ClusterIT.class, RemoteIT.class}) + public void theAllEnvCaseTest() { + ... + } +} +``` + +#### 4. Change the configurations of IoTDB when testing + +Sometimes, the configurations of IoTDB need to be changed in order to test the functionalities under certain conditions. Because changing the configurations on a remote machine is troublesome, configuration modification is not allowed in the remote environment. However, it is allowed in the local standalone and cluster environment. Changes of the configuration files should be implemented in the ```setUp()``` method, before ```EnvFactory.getEnv().initBeforeClass()```, and should be imple [...] + + +```java +@RunWith(IoTDBTestRunner.class) +@Category({LocalStandaloneIT.class, ClusterIT.class}) +public class IoTDBAlignedSeriesQueryIT { + + protected static boolean enableSeqSpaceCompaction; + protected static boolean enableUnseqSpaceCompaction; + protected static boolean enableCrossSpaceCompaction; + + @BeforeClass + public static void setUp() throws Exception { + // get the default configurations + enableSeqSpaceCompaction = ConfigFactory.getConfig().isEnableSeqSpaceCompaction(); + enableUnseqSpaceCompaction = ConfigFactory.getConfig().isEnableUnseqSpaceCompaction(); + enableCrossSpaceCompaction = ConfigFactory.getConfig().isEnableCrossSpaceCompaction(); + // update configurations + ConfigFactory.getConfig().setEnableSeqSpaceCompaction(false); + ConfigFactory.getConfig().setEnableUnseqSpaceCompaction(false); + ConfigFactory.getConfig().setEnableCrossSpaceCompaction(false); + EnvFactory.getEnv().initBeforeClass(); + AlignedWriteUtil.insertData(); + } + + @AfterClass + public static void tearDown() throws Exception { + EnvFactory.getEnv().cleanAfterClass(); + // revert to the default configurations + ConfigFactory.getConfig().setEnableSeqSpaceCompaction(enableSeqSpaceCompaction); + ConfigFactory.getConfig().setEnableUnseqSpaceCompaction(enableUnseqSpaceCompaction); + ConfigFactory.getConfig().setEnableCrossSpaceCompaction(enableCrossSpaceCompaction); + } +} +``` + +### Commands for starting IT + +1、Execute IT in the cluster environment + +```shell script +mvn clean verify \ + -Dsession.test.skip=true \ + -Diotdb.test.skip=true \ + -Dcluster.test.skip=true \ + -Dtsfile.test.skip=true \ + -Dcommons.test.skip=true \ + -Dconfignode.test.skip=true \ + -Dconsensus.test.skip=true \ + -Djdbc.test.skip=true \ + -Dmetrics.test.skip=true \ + -pl integration-test -am -PClusterIT +``` +2、Execute IT in the local standalone environment + +```shell script +mvn clean verify \ + -Dsession.test.skip=true \ + -Diotdb.test.skip=true \ + -Dcluster.test.skip=true \ + -Dtsfile.test.skip=true \ + -Dcommons.test.skip=true \ + -Dconfignode.test.skip=true \ + -Dconsensus.test.skip=true \ + -pl integration-test -am +``` + +3、Execute IT in the remote environment + +```shell script +mvn clean verify -pl integration-test -am -PRemoteIT \ + -DRemoteIp=127.0.0.1 \ + -DRemotePort=6667 +``` + +## Q&A +### Ways to check the log after the CI failure +1、click *Details* of the corresponding test + +<img style="width:100%; max-width:800px; max-height:600px; margin-left:auto; margin-right:auto; display:block;" src="https://github.com/apache/iotdb-bin-resources/blob/main/integration-test/pic/details.png";> + +2、check and download the error log + +<img style="width:100%; max-width:800px; max-height:600px; margin-left:auto; margin-right:auto; display:block;" src="https://github.com/apache/iotdb-bin-resources/blob/main/integration-test/pic/download1.png";> + +You can also click the *summary* at the upper left and then check and download the error log. + +<img style="width:100%; max-width:800px; max-height:600px; margin-left:auto; margin-right:auto; display:block;" src="https://github.com/apache/iotdb-bin-resources/blob/main/integration-test/pic/download2.png";> - ```java - private static void insertData() throws ClassNotFoundException { - // Class.forName(Config.JDBC_DRIVER_NAME); // orginal connection code - // try (Connection connection = - // DriverManager.getConnection( - // Config.IOTDB_URL_PREFIX + "127.0.0.1:6667/", "root", "root"); - try (Connection connection = EnvFactory.getEnv().getConnection(); // new code - Statement statement = connection.createStatement()) { - - for (String sql : sqls) { - statement.execute(sql); - } - } catch (Exception e) { - e.printStackTrace(); - } - } - ``` - - - The method of changing the IoTDB configuration must be refactored in the Cluster environment. (As the remote environment cannot change the configuration remotely at present, the test cases that change the configuration will not support testing in the remote environment) - - In the Cluster environment, as the configuration cannot be changed dynamically, only the configuration changes before the environment init are effective. - - The refactoring has included most of the configuration changes, which can be changed through the method of `ConfigFactory.getConfig()`. - - ```java - @Category({LocalStandaloneTest.class, ClusterTest.class}) - public class IoTDBCompleteIT { - private int prevVirtualStorageGroupNum; - - @Before - public void setUp() { - prevVirtualStorageGroupNum = - IoTDBDescriptor.getInstance().getConfig().getVirtualStorageGroupNum(); - // IoTDBDescriptor.getInstance().getConfig().setVirtualStorageGroupNum(16); // orginal code - ConfigFactory.getConfig().setVirtualStorageGroupNum(16); // new code - EnvFactory.getEnv().initBeforeClass(); - } - ``` - - - If the configuration item has not been included in the method of `ConfigFactory.getConfig()`, it needs to be defined in the BaseConfig.java interface file and implemented in StandaloneEnvConfig.java and ClusterEnvConfig.java, respectively. This part is not very common. For specific, please refer to the realized part. diff --git a/docs/zh/UserGuide/Integration-Test/Integration-Test-refactoring-tutorial.md b/docs/zh/UserGuide/Integration-Test/Integration-Test-refactoring-tutorial.md index e137e4f268..1be2b134a5 100644 --- a/docs/zh/UserGuide/Integration-Test/Integration-Test-refactoring-tutorial.md +++ b/docs/zh/UserGuide/Integration-Test/Integration-Test-refactoring-tutorial.md @@ -19,145 +19,243 @@ --> -# IoTDB社区Integration Test改造说明 +# 集成测试开发者文档 -- 步骤0. 前提须知 - - **位置已移动**；所有的Integration Test已被移动至单独的integration模块。 - - **测试用例必须打分类标签**； `Category` 即测试分类标签，决定了该测试用例在哪套测试环境或流程中被测试。 - - **涉及测试环境的代码可能要重构**；决定了该测试用例是否能被当前测试环境正确测试，需要根据相应的环境重构相应的代码。 +**集成测试**是软件测试中的一个阶段。在该阶段中，各个软件模块被组合起来作为一个整体进行测试。进行集成测试是为了评估某系统或某组件是否符合指定的功能需求。 +## Apache IoTDB 集成测试规范 -- 步骤1. 测试用例打标签 +### Apache IoTDB 集成测试的环境 - - 在测试用例类或者测试用例方法前加上合适的`Category`，可以是任意期望的测试分类标签的集合。 +Apache IoTDB 集成测试的环境一共有3种，分别为**本地单机测试环境、本地集群测试环境和远程测试环境。** Apache IOTDB 的集群测试需要在其中的1种或多种环境下完成。对于这三类环境的说明如下： +1. 本地单机测试环境：该环境用于完成本地的 Apache IoTDB 单机版的集成测试。若需要变更该环境的具体配置，需要在 IoTDB 实例启动前替换相应的配置文件，再启动 IoTDB 并进行测试。 +2. 本地集群测试环境：该环境用于完成本地的 Apache IoTDB 分布式版（伪分布式）的集成测试。若需要变更该环境的具体配置，需要在 IoTDB 集群启动前替换相应的配置文件，再启动 IoTDB 集群并进行测试。 +3. 远程测试环境：该环境用于测试远程 Apache IoTDB 的功能，连接的 IoTDB 实例可能是一个单机版的实例，也可以是远程集群的某一个节点。远程测试环境的具体配置的修改受到限制，暂不支持在测试时修改。 +集成测试开发者在编写测试程序时需要指定这三种环境的1种或多种。具体指定方法见后文。 - - 真实样例，下面三个测试类的`Category`都是真实有效的， +### 黑盒测试 - ```java - @Category({LocalStandaloneTest.class, ClusterTest.class, RemoteTest.class}) - public class IoTDBAliasIT { - ...... - } - - - @Category({LocalStandaloneTest.class, ClusterTest.class}) - public class IoTDBAlignByDeviceIT { - ...... - } - - - @Category({LocalStandaloneTest.class}) - public class IoTDBArithmeticIT { - ...... - } - ``` - - - 甚至，你还可以在测试方法级别加`Category`。 - - ```java - @Category({LocalStandaloneTest.class}) - public class IoTDBExampleIT { - - // This case can ONLY test in environment of local. - @Test - public void theStandaloneCase() { - ...... - } - - // This case can test in environment of local, cluster and remote. - @Test - @Category({ClusterTest.class, RemoteTest.class}) - public void theAllEnvCase() { - ...... - } - } - ``` - - - 目前，所有测试用例至少要加上单机测试的测试分类，即`LocalStandaloneTest.class`。 +**黑盒测试** 是一种软件测试方法，它检验程序的功能，而不考虑其内部结构或工作方式。开发者不需要了解待测程序的内部逻辑即可完成测试。**Apache IoTDB 的集成测试以黑盒测试的方式进行。通过 JDBC 或 Session API 的接口实现测试输入的用例即为黑盒测试用例。** 因此，测试用例的输出验证也应该通过 JDBC 或 Session API 的返回结果实现。 +### 集成测试的步骤 +集成测试的步骤主要分为三步，即 (1) 构建测试类和标注测试环境、(2) 设置测试前的准备工作以及测试后的清理工作以及 (3) 实现集成测试逻辑。如果需要测试非默认环境下的 IoTDB，还需要修改 IoTDB 的配置，修改方法对应小结的第4部分。 -- 步骤2. 环境代码重构 +<img style="width:100%; max-width:800px; max-height:600px; margin-left:auto; margin-right:auto; display:block;" src="https://github.com/apache/iotdb-bin-resources/blob/main/integration-test/pic/step.svg";> - - 如果测试用例需要在Cluster或者Remote环境下被测试，那么必须对环境相关的代码作相应重构，如果是仅在LocalStandalone环境下测试，则只推荐修改。（不是所有的测试用例可以在Cluster或者Remote环境下被测试，因为受限于部分功能的语句比如本地文件操作，这些代码不能被重构。） +#### 1. 集成测试类和注解 - | | LocalStandalone | Cluster | Remote | - | :------------------------- | :-------------: | :---------: | :---------: | - | setUp and tearDown | Recommend | Must | Must | - | getConnection | Recommend | Must | Must | - | change config | Recommend | Must | Not support | - | Local file operation | Won't change | Not support | Not support | - | Local descriptor operation | Won't change | Not support | Not support | - | restart operation | Won't change | Not support | Not support | - - - - - `setUp` 和`tearDown` 方法内的重构，在Cluster和Remote环境下是必须更改的 - - ```java - @Category({LocalStandaloneTest.class, ClusterTest.class, RemoteTest.class}) - public class IoTDBAliasIT { - - @BeforeClass - public static void setUp() throws Exception { - // EnvironmentUtils.closeStatMonitor(); // orginal setup code - // EnvironmentUtils.envSetUp(); // orginal setup code - EnvFactory.getEnv().initBeforeClass(); // new initBeforeClass code - - insertData(); - } - - @AfterClass - public static void tearDown() throws Exception { - // EnvironmentUtils.cleanEnv(); // orginal code - EnvFactory.getEnv().cleanAfterClass(); // new cleanAfterClass code - } +构建的集成测试类时，开发者需要在 Apache IoTDB 的 [integration-test](https://github.com/apache/iotdb/tree/master/integration-test) 模块中创建测试类。类名应当能够精简准确地表述该集成测试的目的。除用于服务其他测试用例的类外，含集成测试用例用于测试 Apache IoTDB 功能的类，应当命名为“功能+IT”。例如，用于测试IoTDB自动注册元数据功能的集成测试命名为“<font color=green>IoTDBAutoCreateSchema</font><font color=red>IT</font>”。 + +- Category 注解：**在构建集成测试类时，需要显式地通过引入```@Category```注明测试环境** ，测试环境用```LocalStandaloneIT.class```、```ClusterIT.class``` 和 ```RemoteIT.class```来表示，分别与“Apache IoTDB 集成测试的环境”中的本地单机测试环境、本地集群测试环境和远程测试环境对应。标签内是测试环境的集合，可以包含多个元素，表示在多种环境下分别测试。**一般情况下，标签```LocalStandaloneIT.class``` 和 ```ClusterIT.class``` 是必须添加的。** 当某些功能仅支持单机版 IoTDB 时可以只保留```LocalStandaloneIT.class```。 +- RunWith 注解： 每一个集成测试类上都需要添加 ```@RunWith(IoTDBTestRunner.class)``` 标签。 + +```java +// 给 IoTDBAliasIT 测试类加标签，分别在本地单机测试环境、 +// 本地集群测试环境和远程测试环境完成测试。 +@RunWith(IoTDBTestRunner.class) +@Category({LocalStandaloneIT.class, ClusterIT.class, RemoteIT.class}) +public class IoTDBAliasIT { + ... +} + +// 给 IoTDBAlignByDeviceIT 测试类加标签，分别在本地单机 +// 测试环境和本地集群测试环境完成测试。 +@RunWith(IoTDBTestRunner.class) +@Category({LocalStandaloneIT.class, ClusterIT.class}) +public class IoTDBAlignByDeviceIT { + ... +} +``` + +#### 2. 设置测试前的准备工作以及测试后的清理工作 + +测试前的准备工作包括启动 IoTDB（单机或集群）实例和测试用的数据准备。这些逻辑在setUp方法内实现。其中setUp方法前需要添加```@BeforeClass``` 或 ```@Before``` 标签，前者表示该方法为当前集成测试执行的第 1 个方法，并且在集成测试运行时只执行 1 次，后者表示在运行当前集成测试的每 1 个测试方法前，该方法都会被执行 1 次。 +- IoTDB 实例启动通过调用工厂类来实现，即```EnvFactory.getEnv().initBeforeClass()```。 +- 测试用的数据准备包括按测试需要提前注册存储组、注册时间序列、写入时间序列数据等。建议在测试类内实现单独的方法来准备数据，如insertData()。若需要写入多条数据，请使用批量写入的接口（JDBC中的executeBatch接口，或Session API 中的 insertRecords、insertTablets 等接口）。 + +```java +@BeforeClass +public static void setUp() throws Exception { + // 启动 IoTDB 实例 + EnvFactory.getEnv().initBeforeClass(); + ... // 准备数据 +} +``` + +测试后需要清理相关的环境，其中需要断开还没有关闭的连接。这些逻辑在 tearDown 方法内实现。其中 tearDown 方法前需要添加```@AfterClass``` 或 ```@After``` 标签，前者表示该方法为当前集成测试执行的最后一个方法，并且在集成测试运行时只执行 1 次，后者表示在运行当前集成测试的每一个测试方法后，该方法都会被执行 1 次。 +- 如果 IoTDB 连接以测试类成员变量的形式声明，并且在测试后没有断开连接，则需要在 tearDown 方法内显式断开。 +- IoTDB 环境的清理通过调用工厂类来实现，即```EnvFactory.getEnv().cleanAfterClass()```。 + +```java +@AfterClass +public static void tearDown() throws Exception { + ... // 断开连接等 + // 清理 IoTDB 实例的环境 + EnvFactory.getEnv().cleanAfterClass(); +} +``` + +#### 3. 实现集成测试逻辑 + +Apache IoTDB 的集成测试以黑盒测试的方式进行，测试方法的名称为“测试的功能点+Test”，例如“<font color=green>selectWithAlias</font><font color=red>Test</font>”。测试通过 JDBC 或 Session API 的接口来完成。 + +1、使用JDBC接口 + +使用JDBC接口时，建议将连接建立在 try 语句内，以这种方式建立的连接无需在 tearDown 方法内关闭。连接需要通过工厂类来建立，即```EnvFactory.getEnv().getConnection()```，不要指定具体的 ip 地址或端口号。示例代码如下所示。 + +```java +@Test +public void someFunctionTest(){ + try (Connection connection = EnvFactory.getEnv().getConnection(); + Statement statement = connection.createStatement()) { + ... // 执行相应语句并做测试 + } catch (Exception e) { + e.printStackTrace(); + Assert.fail(); } - ``` - - +} +``` +注意： +- **查询操作必须使用```executeQuery()```方法，返回ResultSet；** 对于**更新数据库等无返回值的操作，必须使用```execute()```方法。** 示例代码如下。 - - `getConnection` 的重构，在Cluster和Remote环境下是必须更改 - - ```java - private static void insertData() throws ClassNotFoundException { - // Class.forName(Config.JDBC_DRIVER_NAME); // orginal connection code - // try (Connection connection = - // DriverManager.getConnection( - // Config.IOTDB_URL_PREFIX + "127.0.0.1:6667/", "root", "root"); - try (Connection connection = EnvFactory.getEnv().getConnection(); // new code - Statement statement = connection.createStatement()) { - - for (String sql : sqls) { - statement.execute(sql); - } - } catch (Exception e) { - e.printStackTrace(); - } +```java +@Test +public void exampleTest() throws Exception { + try (Connection connection = EnvFactory.getEnv().getConnection(); + Statement statement = connection.createStatement()) { + // 使用 execute() 方法设置存储组 + statement.execute("set storage group to root.sg"); + // 使用 executeQuery() 方法查询存储组 + try (ResultSet resultSet = statement.executeQuery("show storage group")) { + if (resultSet.next()) { + String storageGroupPath = resultSet.getString("storage group"); + Assert.assertEquals("root.sg", storageGroupPath); + } else { + Assert.fail("This ResultSet is empty."); } - ``` - - - - - 更改配置的方法，在Cluster环境下是必须重构的。（由于目前Remote环境无法远程更改配置，更改配置的测试用例将不支持Remote环境下测试） - - - 在Cluster环境下，由于无法动态更改配置，只有环境init前的配置更改才有效。 - - 重构已包含了大部分的配置更改，通过`ConfigFactory.getConfig()` 的方法可以进行链式更改。 - - ```java - @Category({LocalStandaloneTest.class, ClusterTest.class}) - public class IoTDBCompleteIT { - private int prevVirtualStorageGroupNum; - - @Before - public void setUp() { - prevVirtualStorageGroupNum = - IoTDBDescriptor.getInstance().getConfig().getVirtualStorageGroupNum(); - // IoTDBDescriptor.getInstance().getConfig().setVirtualStorageGroupNum(16); // orginal code - ConfigFactory.getConfig().setVirtualStorageGroupNum(16); // new code - EnvFactory.getEnv().initBeforeClass(); - } - ``` - - - 若配置项尚未在`ConfigFactory.getConfig()` 的方法中包含，需要在BaseConfig.java接口文件中定义，在StandaloneEnvConfig.java和ClusterEnvConfig.java中分别实现，这部分不是很常用，具体方法可以参考已实现的部分，目前暂不列出。 + } + } +} +``` + +2、使用 Session API + +目前暂不支持使用 Session API 来做集成测试。 + +3、测试方法的环境标签 +对于测试方法，开发者也可以指定特定的测试环境，只需要在对应的测试方法前注明环境即可。值得注意的是，有额外测试环境标注的用例，不但会在所指定的环境中进行测试，还会在该用例隶属的测试类所对应的环境中进行测试。示例代码如下。 + + +```java +@RunWith(IoTDBTestRunner.class) +@Category({LocalStandaloneIT.class}) +public class IoTDBExampleIT { + + // 该用例只会在本地单机测试环境中进行测试 + @Test + public void theStandaloneCaseTest() { + ... + } + + // 该用例会在本地单机测试环境、本地集群测试环境和远程测试环境中进行测试 + @Test + @Category({ClusterIT.class, RemoteIT.class}) + public void theAllEnvCaseTest() { + ... + } +} +``` + +#### 4. 测试中 IoTDB 配置参数的修改 + +有时，为了测试 IoTDB 在特定配置条件下的功能需要更改其配置。由于远程的机器配置无法修改，因此，需要更改配置的测试不支持远程测试环境，只支持本地单机测试环境和本地集群测试环境。配置文件的修改需要在setUp方法中实现，在```EnvFactory.getEnv().initBeforeClass()```之前执行，应当使用 ConfigFactory 提供的方法来实现。在 tearDown 方法内，需要将 IoTDB 的配置恢复到原默认设置，这一步在环境清理（```EnvFactory.getEnv().cleanAfterTest()```）后通过调用ConfigFactory提供的方法来执行。实例代码如下。 + +```java +@RunWith(IoTDBTestRunner.class) +@Category({LocalStandaloneIT.class, ClusterIT.class}) +public class IoTDBAlignedSeriesQueryIT { + + protected static boolean enableSeqSpaceCompaction; + protected static boolean enableUnseqSpaceCompaction; + protected static boolean enableCrossSpaceCompaction; + + @BeforeClass + public static void setUp() throws Exception { + // 获取默认配置 + enableSeqSpaceCompaction = ConfigFactory.getConfig().isEnableSeqSpaceCompaction(); + enableUnseqSpaceCompaction = ConfigFactory.getConfig().isEnableUnseqSpaceCompaction(); + enableCrossSpaceCompaction = ConfigFactory.getConfig().isEnableCrossSpaceCompaction(); + // 更新配置 + ConfigFactory.getConfig().setEnableSeqSpaceCompaction(false); + ConfigFactory.getConfig().setEnableUnseqSpaceCompaction(false); + ConfigFactory.getConfig().setEnableCrossSpaceCompaction(false); + EnvFactory.getEnv().initBeforeClass(); + AlignedWriteUtil.insertData(); + } + + @AfterClass + public static void tearDown() throws Exception { + EnvFactory.getEnv().cleanAfterClass(); + // 恢复为默认配置 + ConfigFactory.getConfig().setEnableSeqSpaceCompaction(enableSeqSpaceCompaction); + ConfigFactory.getConfig().setEnableUnseqSpaceCompaction(enableUnseqSpaceCompaction); + ConfigFactory.getConfig().setEnableCrossSpaceCompaction(enableCrossSpaceCompaction); + } +} +``` + +### 集成测试的启动命令 + +1、在本地集群测试环境下运行集成测试 + +```shell script +mvn clean verify \ + -Dsession.test.skip=true \ + -Diotdb.test.skip=true \ + -Dcluster.test.skip=true \ + -Dtsfile.test.skip=true \ + -Dcommons.test.skip=true \ + -Dconfignode.test.skip=true \ + -Dconsensus.test.skip=true \ + -Djdbc.test.skip=true \ + -Dmetrics.test.skip=true \ + -pl integration-test -am -PClusterIT +``` +2、在本地单机测试环境下运行集成测试 + +```shell script +mvn clean verify \ + -Dsession.test.skip=true \ + -Diotdb.test.skip=true \ + -Dcluster.test.skip=true \ + -Dtsfile.test.skip=true \ + -Dcommons.test.skip=true \ + -Dconfignode.test.skip=true \ + -Dconsensus.test.skip=true \ + -pl integration-test -am +``` + +3、在远程测试环境下运行集成测试 + +```shell script +mvn clean verify -pl integration-test -am -PRemoteIT \ + -DRemoteIp=127.0.0.1 \ + -DRemotePort=6667 +``` + +## Q&A +### CI 出错后查看日志的方法 +1、点击出错的测试对应的 Details + +<img style="width:100%; max-width:800px; max-height:600px; margin-left:auto; margin-right:auto; display:block;" src="https://github.com/apache/iotdb-bin-resources/blob/main/integration-test/pic/details.png";> + +2、查看和下载日志 + +<img style="width:100%; max-width:800px; max-height:600px; margin-left:auto; margin-right:auto; display:block;" src="https://github.com/apache/iotdb-bin-resources/blob/main/integration-test/pic/download1.png";> + +也可以点击左上角的 summary 然后查看和下载其他错误日志。 + +<img style="width:100%; max-width:800px; max-height:600px; margin-left:auto; margin-right:auto; display:block;" src="https://github.com/apache/iotdb-bin-resources/blob/main/integration-test/pic/download2.png";> +
![https://github.com/apache/iotdb-bin-resources/blob/main/integration-test/pic/step.svg"](https://github.com/apache/iotdb-bin-resources/blob/main/integration-test/pic/step.svg"){kind=link}
![https://github.com/apache/iotdb-bin-resources/blob/main/integration-test/pic/details.png"](https://github.com/apache/iotdb-bin-resources/blob/main/integration-test/pic/details.png"){kind=link}
![https://github.com/apache/iotdb-bin-resources/blob/main/integration-test/pic/download1.png"](https://github.com/apache/iotdb-bin-resources/blob/main/integration-test/pic/download1.png"){kind=link}
![https://github.com/apache/iotdb-bin-resources/blob/main/integration-test/pic/download2.png"](https://github.com/apache/iotdb-bin-resources/blob/main/integration-test/pic/download2.png"){kind=link}