This is an automated email from the ASF dual-hosted git repository. github-merge-queue[bot] pushed a commit to branch gh-readonly-queue/dev/pr-12882-7f87688505cfc915de0ed4e87a4eda6fcee9f2cf in repository https://gitbox.apache.org/repos/asf/seatunnel.git
commit 4f70bc55d1fe78b7cb2a32a7cddac95d4560bedc Author: zhengxiang378928908-code <[email protected]> AuthorDate: Mon Oct 26 15:25:09 2026 +0000 [Fix][Connector-V2][JDBC] Report the declared length of Dameng NVARCHAR and NVARCHAR2 (#12882) Co-authored-by: Claude Opus 5.5 <[email protected]> --- .../introduction/concepts/incompatible-changes.md | 7 + .../introduction/concepts/incompatible-changes.md | 7 + .../seatunnel/jdbc/catalog/dm/DamengCatalog.java | 47 +++++- .../jdbc/catalog/dm/DamengCatalogTest.java | 148 +++++++++++++++++++ .../connectors/seatunnel/jdbc/JdbcDmIT.java | 161 +++++++++++++++++++++ 5 files changed, 369 insertions(+), 1 deletion(-) diff --git a/docs/en/introduction/concepts/incompatible-changes.md b/docs/en/introduction/concepts/incompatible-changes.md index f8e3f2f95c..50d813457f 100644 --- a/docs/en/introduction/concepts/incompatible-changes.md +++ b/docs/en/introduction/concepts/incompatible-changes.md @@ -20,6 +20,13 @@ Such a value now causes a parse error that identifies the field and the expected The same strictness applies wherever the shared JSON-to-row conversion is used without ignoring parse errors: the `JsonPath` transform (column `dest_type`), the `Python` transform (declared output columns and their declared types) and the Assert sink's `equals_to` check for `ARRAY` and `MAP` values. These callers do not consult the JSON format's `ignore-parse-errors` option, so that switch does not apply to them: a value that does not match the declared `ARRAY` or `MAP` type still fails [...] +### JDBC Dameng: NVARCHAR and NVARCHAR2 report their declared length + +- **Schema change: the Dameng catalog reports `NVARCHAR(n)` and `NVARCHAR2(n)` with length n instead of the byte length** + - **Description**: `NVARCHAR(n)` and `NVARCHAR2(n)` hold n characters, but the catalog took the length from `ALL_TAB_COLUMNS.DATA_LENGTH`, which counts bytes: 2n on a GB18030 database and 4n on a UTF-8 database, whatever `LENGTH_IN_CHAR` is set to. `NVARCHAR(10)` was therefore reported as `NVARCHAR(20)` or `NVARCHAR(40)`. The length now comes from `SYS.SYSCOLUMNS.LENGTH$`, the declared length. + - **Impact**: The source type of these columns changes, for example from `NVARCHAR(20)` to `NVARCHAR(10)`, and the column length used for sink auto-create is halved (GB18030) or quartered (UTF-8), for example a MySQL sink gets `VARCHAR(40)` instead of `VARCHAR(80)` for `NVARCHAR(10)`. Each new column still holds every value of the source column, because the length is counted in characters. Tables that already exist downstream are not altered. `CHAR`, `VARCHAR`, `VARCHAR2` and `NCHAR` c [...] + - **Migration Guide**: No change is needed. Jobs that rely on the wider auto-created columns, for example to write longer values from another source into the same table, should create the sink table themselves. + ### File sink: XML applies `date_format`, `datetime_format` and `time_format` - **Behavior change: the XML writer of the file sinks formats DATE, TIME and TIMESTAMP values with the configured formats** diff --git a/docs/zh/introduction/concepts/incompatible-changes.md b/docs/zh/introduction/concepts/incompatible-changes.md index 9b9a4c3ec8..2c444d5b85 100644 --- a/docs/zh/introduction/concepts/incompatible-changes.md +++ b/docs/zh/introduction/concepts/incompatible-changes.md @@ -19,6 +19,13 @@ 同样的严格校验也适用于未忽略解析错误而使用共享 JSON 转行转换的地方：`JsonPath` 转换（列的 `dest_type`）、`Python` 转换（声明的输出列及其声明类型），以及 Assert 的 `equals_to` 对 `ARRAY`、`MAP` 值的比较。这些调用方不会读取 JSON 格式的 `ignore-parse-errors` 选项，因此该开关对它们不生效：与声明的 `ARRAY` 或 `MAP` 类型不匹配的值在那里仍会失败，而这些转换插件自身的错误处理行为保持不变。 +### JDBC 达梦：NVARCHAR 和 NVARCHAR2 按声明长度上报 + +- **模式变更：达梦 Catalog 现在按声明长度 n 上报 `NVARCHAR(n)` 和 `NVARCHAR2(n)`，不再使用字节长度** + - **描述**：`NVARCHAR(n)` 和 `NVARCHAR2(n)` 可容纳 n 个字符，但 Catalog 此前从 `ALL_TAB_COLUMNS.DATA_LENGTH` 读取长度，该字段是字节数：GB18030 库上是 2n，UTF-8 库上是 4n，与 `LENGTH_IN_CHAR` 的取值无关。因此 `NVARCHAR(10)` 会被上报为 `NVARCHAR(20)` 或 `NVARCHAR(40)`。现在长度取自 `SYS.SYSCOLUMNS.LENGTH$`，即声明长度。 + - **影响**：这类列的源类型会变化，例如从 `NVARCHAR(20)` 变为 `NVARCHAR(10)`；Sink 自动建表使用的列长度变为原来的 1/2（GB18030）或 1/4（UTF-8），例如 `NVARCHAR(10)` 写入 MySQL Sink 时建为 `VARCHAR(40)`，而不是 `VARCHAR(80)`。由于长度按字符计算，新列仍能容纳源列的所有值。下游已存在的表不会被修改。`CHAR`、`VARCHAR`、`VARCHAR2` 和 `NCHAR` 列不受影响。 + - **迁移指南**：无需修改。如果作业依赖原来较宽的自动建表列（例如还要从其他数据源向同一张表写入更长的值），请自行创建 Sink 表。 + ### File Sink：XML 应用 `date_format`、`datetime_format` 和 `time_format` - **行为变更：文件类 Sink 的 XML 写入器按配置的格式输出 DATE、TIME 和 TIMESTAMP** diff --git a/seatunnel-connectors-v2/connector-jdbc/src/main/java/org/apache/seatunnel/connectors/seatunnel/jdbc/catalog/dm/DamengCatalog.java b/seatunnel-connectors-v2/connector-jdbc/src/main/java/org/apache/seatunnel/connectors/seatunnel/jdbc/catalog/dm/DamengCatalog.java index 69dc9f154f..d4791bcfec 100644 --- a/seatunnel-connectors-v2/connector-jdbc/src/main/java/org/apache/seatunnel/connectors/seatunnel/jdbc/catalog/dm/DamengCatalog.java +++ b/seatunnel-connectors-v2/connector-jdbc/src/main/java/org/apache/seatunnel/connectors/seatunnel/jdbc/catalog/dm/DamengCatalog.java @@ -54,6 +54,7 @@ public class DamengCatalog extends AbstractJdbcCatalog { private static final String SELECT_COLUMNS_SQL = "SELECT COLUMNS.COLUMN_NAME, COLUMNS.DATA_TYPE, COLUMNS.DATA_LENGTH, COLUMNS.DATA_PRECISION, COLUMNS.DATA_SCALE " + ", COLUMNS.NULLABLE, COLUMNS.DATA_DEFAULT, COMMENTS.COMMENTS ," + + "COLUMNS.CHAR_LENGTH, SYSCOL.LENGTH$ AS DECLARED_LENGTH, " + "CASE \n" + " WHEN COLUMNS.DATA_TYPE IN ('CHAR', 'CHARACTER', 'VARCHAR', 'VARCHAR2', 'VARBINARY', 'BINARY') THEN COLUMNS.DATA_TYPE || '(' || COLUMNS.DATA_LENGTH || ')'\n" + " WHEN COLUMNS.DATA_TYPE IN ('NUMERIC', 'DECIMAL', 'NUMBER') AND COLUMNS.DATA_PRECISION IS NOT NULL AND COLUMNS.DATA_SCALE IS NOT NULL AND COLUMNS.DATA_PRECISION != 0 AND COLUMNS.DATA_SCALE != 0 THEN COLUMNS.DATA_TYPE || '(' || COLUMNS.DATA_PRECISION || ', ' || COLUMNS.DATA_SCALE || ')'\n" @@ -64,6 +65,15 @@ public class DamengCatalog extends AbstractJdbcCatalog { + "ON COLUMNS.OWNER = COMMENTS.SCHEMA_NAME " + "AND COLUMNS.TABLE_NAME = COMMENTS.TABLE_NAME " + "AND COLUMNS.COLUMN_NAME = COMMENTS.COLUMN_NAME " + // SYSCOLUMNS.LENGTH$ is the declared length. ALL_TAB_COLUMNS derives + // DATA_LENGTH from it as a byte count and leaves CHAR_LENGTH at 0 for + // NVARCHAR2, see nationalCharLength. + + "LEFT JOIN SYS.SYSOBJECTS SCH " + + "ON SCH.NAME = COLUMNS.OWNER AND SCH.TYPE$ = 'SCH' " + + "LEFT JOIN SYS.SYSOBJECTS TAB " + + "ON TAB.SCHID = SCH.ID AND TAB.NAME = COLUMNS.TABLE_NAME AND TAB.TYPE$ = 'SCHOBJ' " + + "LEFT JOIN SYS.SYSCOLUMNS SYSCOL " + + "ON SYSCOL.ID = TAB.ID AND SYSCOL.NAME = COLUMNS.COLUMN_NAME " + "WHERE COLUMNS.OWNER = '%s' " + "AND COLUMNS.TABLE_NAME = '%s' " + "ORDER BY COLUMNS.COLUMN_ID ASC"; @@ -163,7 +173,12 @@ public class DamengCatalog extends AbstractJdbcCatalog { protected Column buildColumn(ResultSet resultSet) throws SQLException { String columnName = resultSet.getString("COLUMN_NAME"); String typeName = resultSet.getString("DATA_TYPE"); - long columnLength = resultSet.getLong("DATA_LENGTH"); + long columnLength = + nationalCharLength( + typeName, + resultSet.getLong("DATA_LENGTH"), + resultSet.getLong("CHAR_LENGTH"), + resultSet.getLong("DECLARED_LENGTH")); long columnPrecision = resultSet.getLong("DATA_PRECISION"); int columnScale = resultSet.getInt("DATA_SCALE"); String columnComment = resultSet.getString("COMMENTS"); @@ -185,6 +200,36 @@ public class DamengCatalog extends AbstractJdbcCatalog { return DmdbTypeConverter.INSTANCE.convert(typeDefine); } + /** + * Length of a column in the unit the type converter expects: characters for {@code NVARCHAR} + * and {@code NVARCHAR2}, {@code DATA_LENGTH} for every other type. + * + * <p>{@code NVARCHAR(n)} and {@code NVARCHAR2(n)} always hold n characters. {@code + * ALL_TAB_COLUMNS.DATA_LENGTH} reports them in bytes: n * 2 on a GB18030 database and n * 4 on + * a UTF-8 database, whatever {@code LENGTH_IN_CHAR} is. {@code CHAR_LENGTH} is n for {@code + * NVARCHAR} but 0 for {@code NVARCHAR2}, which the view does not list. {@code + * SYS.SYSCOLUMNS.LENGTH$} is n for both, so it is used first. + * + * @param dataLength {@code ALL_TAB_COLUMNS.DATA_LENGTH} + * @param charLength {@code ALL_TAB_COLUMNS.CHAR_LENGTH}, 0 if unknown + * @param declaredLength {@code SYS.SYSCOLUMNS.LENGTH$}, 0 if unknown + */ + static long nationalCharLength( + String typeName, long dataLength, long charLength, long declaredLength) { + if (!DmdbTypeConverter.DM_NVARCHAR.equalsIgnoreCase(typeName) + && !DmdbTypeConverter.DM_NVARCHAR2.equalsIgnoreCase(typeName)) { + return dataLength; + } + if (declaredLength > 0) { + return declaredLength; + } + if (charLength > 0) { + return charLength; + } + // Unknown character length: keep the byte count, which oversizes but never truncates. + return dataLength; + } + @Override protected String getUrlFromDatabaseName(String databaseName) { return defaultUrl; diff --git a/seatunnel-connectors-v2/connector-jdbc/src/test/java/org/apache/seatunnel/connectors/seatunnel/jdbc/catalog/dm/DamengCatalogTest.java b/seatunnel-connectors-v2/connector-jdbc/src/test/java/org/apache/seatunnel/connectors/seatunnel/jdbc/catalog/dm/DamengCatalogTest.java new file mode 100644 index 0000000000..808015bbdf --- /dev/null +++ b/seatunnel-connectors-v2/connector-jdbc/src/test/java/org/apache/seatunnel/connectors/seatunnel/jdbc/catalog/dm/DamengCatalogTest.java @@ -0,0 +1,148 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.seatunnel.connectors.seatunnel.jdbc.catalog.dm; + +import org.apache.seatunnel.api.table.catalog.Column; +import org.apache.seatunnel.common.utils.JdbcUrlUtil; + +import org.junit.jupiter.api.Assertions; +import org.junit.jupiter.api.Test; +import org.junit.jupiter.params.ParameterizedTest; +import org.junit.jupiter.params.provider.CsvSource; +import org.mockito.Mockito; + +import java.sql.ResultSet; +import java.sql.SQLException; + +class DamengCatalogTest { + + /** + * Metadata measured on DM V8 (0x7000c) for columns declared with length 10 or 100, on databases + * initialized with each LENGTH_IN_CHAR / CHARSET combination. Every one of these columns + * accepts exactly the declared number of characters, single-byte or Chinese. + */ + @ParameterizedTest(name = "{0} {1} on LENGTH_IN_CHAR={2}, {3}") + @CsvSource({ + // type, declared, LENGTH_IN_CHAR, charset, DATA_LENGTH, CHAR_LENGTH, SYSCOLUMNS.LENGTH$ + "NVARCHAR, 10, 0, GB18030, 20, 10, 10", + "NVARCHAR2, 10, 0, GB18030, 20, 0, 10", + "NVARCHAR, 100, 0, GB18030, 200, 100, 100", + "NVARCHAR2, 100, 0, GB18030, 200, 0, 100", + "NVARCHAR, 10, 1, GB18030, 20, 10, 10", + "NVARCHAR2, 10, 1, GB18030, 20, 0, 10", + "NVARCHAR, 10, 0, UTF-8, 40, 10, 10", + "NVARCHAR2, 10, 0, UTF-8, 40, 0, 10", + "NVARCHAR, 100, 0, UTF-8, 400, 100, 100", + "NVARCHAR2, 100, 0, UTF-8, 400, 0, 100", + "NVARCHAR, 10, 1, UTF-8, 40, 10, 10", + "NVARCHAR2, 10, 1, UTF-8, 40, 0, 10", + }) + void nationalCharLengthIsTheDeclaredLength( + String type, + long declared, + int lengthInChar, + String charset, + long dataLength, + long charLength, + long sysColumnsLength) { + Assertions.assertEquals( + declared, + DamengCatalog.nationalCharLength(type, dataLength, charLength, sysColumnsLength)); + } + + @Test + void nationalCharLengthFallsBackWhenTheDeclaredLengthIsUnknown() { + // NVARCHAR still has CHAR_LENGTH. + Assertions.assertEquals(10, DamengCatalog.nationalCharLength("NVARCHAR", 20, 10, 0)); + // NVARCHAR2 has CHAR_LENGTH = 0: keep the byte count, which oversizes but never + // truncates, instead of guessing a bytes-per-character ratio. + Assertions.assertEquals(40, DamengCatalog.nationalCharLength("NVARCHAR2", 40, 0, 0)); + Assertions.assertEquals(10, DamengCatalog.nationalCharLength("nvarchar2", 20, 0, 10)); + } + + @ParameterizedTest(name = "{0}") + @CsvSource({ + // type, DATA_LENGTH, CHAR_LENGTH, SYSCOLUMNS.LENGTH$ + "CHAR, 10, 10, 10", + "CHARACTER, 10, 10, 10", + "VARCHAR, 10, 10, 10", + "VARCHAR2, 10, 10, 10", + "NCHAR, 10, 10, 10", + // VARCHAR(10 CHAR) on GB18030: kept as DATA_LENGTH, as before this change. + "VARCHAR, 20, 10, 10", + "VARBINARY, 8, 0, 8", + }) + void otherTypesKeepDataLength( + String type, long dataLength, long charLength, long sysColumnsLength) { + Assertions.assertEquals( + dataLength, + DamengCatalog.nationalCharLength(type, dataLength, charLength, sysColumnsLength)); + } + + @ParameterizedTest(name = "{0}({1}) with DATA_LENGTH {2}") + @CsvSource({ + "NVARCHAR, 10, 20, 10", + "NVARCHAR2, 10, 20, 0", + "NVARCHAR, 10, 40, 10", + "NVARCHAR2, 10, 40, 0", + }) + void buildColumnReportsTheDeclaredNationalCharLength( + String type, long declared, long dataLength, long charLength) throws SQLException { + Column column = catalog().buildColumn(resultSet(type, dataLength, charLength, declared)); + Assertions.assertEquals(type + "(" + declared + ")", column.getSourceType()); + Assertions.assertEquals(declared * 4, column.getColumnLength()); + } + + @Test + void buildColumnKeepsCharAndVarcharUnchanged() throws SQLException { + Column varchar = catalog().buildColumn(resultSet("VARCHAR", 10, 10, 10)); + Assertions.assertEquals("VARCHAR2(10)", varchar.getSourceType()); + Assertions.assertEquals(40, varchar.getColumnLength()); + + Column nchar = catalog().buildColumn(resultSet("NCHAR", 10, 10, 10)); + Assertions.assertEquals("NCHAR(10)", nchar.getSourceType()); + Assertions.assertEquals(40, nchar.getColumnLength()); + + Column charColumn = catalog().buildColumn(resultSet("CHAR", 10, 10, 10)); + Assertions.assertEquals("CHAR(10)", charColumn.getSourceType()); + Assertions.assertEquals(40, charColumn.getColumnLength()); + } + + private static DamengCatalog catalog() { + return new DamengCatalog( + "dameng", + "SYSDBA", + "SYSDBA", + JdbcUrlUtil.getUrlInfo("jdbc:dm://localhost:5236"), + "SYSDBA", + "dm.jdbc.driver.DmDriver"); + } + + private static ResultSet resultSet( + String type, long dataLength, long charLength, long declaredLength) + throws SQLException { + ResultSet resultSet = Mockito.mock(ResultSet.class); + Mockito.when(resultSet.getString("COLUMN_NAME")).thenReturn("C"); + Mockito.when(resultSet.getString("DATA_TYPE")).thenReturn(type); + Mockito.when(resultSet.getLong("DATA_LENGTH")).thenReturn(dataLength); + Mockito.when(resultSet.getLong("CHAR_LENGTH")).thenReturn(charLength); + Mockito.when(resultSet.getLong("DECLARED_LENGTH")).thenReturn(declaredLength); + Mockito.when(resultSet.getString("NULLABLE")).thenReturn("Y"); + return resultSet; + } +} diff --git a/seatunnel-e2e/seatunnel-connector-v2-e2e/connector-jdbc-e2e/connector-jdbc-e2e-part-5/src/test/java/org/apache/seatunnel/connectors/seatunnel/jdbc/JdbcDmIT.java b/seatunnel-e2e/seatunnel-connector-v2-e2e/connector-jdbc-e2e/connector-jdbc-e2e-part-5/src/test/java/org/apache/seatunnel/connectors/seatunnel/jdbc/JdbcDmIT.java index e38a59268c..1b6568a1aa 100644 --- a/seatunnel-e2e/seatunnel-connector-v2-e2e/connector-jdbc-e2e/connector-jdbc-e2e-part-5/src/test/java/org/apache/seatunnel/connectors/seatunnel/jdbc/JdbcDmIT.java +++ b/seatunnel-e2e/seatunnel-connector-v2-e2e/connector-jdbc-e2e/connector-jdbc-e2e-part-5/src/test/java/org/apache/seatunnel/connectors/seatunnel/jdbc/JdbcDmIT.java @@ -22,6 +22,7 @@ import org.apache.seatunnel.shade.com.google.common.collect.Lists; import org.apache.seatunnel.shade.org.apache.commons.lang3.tuple.Pair; import org.apache.seatunnel.api.table.catalog.CatalogTable; +import org.apache.seatunnel.api.table.catalog.Column; import org.apache.seatunnel.api.table.catalog.TablePath; import org.apache.seatunnel.api.table.type.BasicType; import org.apache.seatunnel.api.table.type.LocalTimeType; @@ -38,8 +39,11 @@ import org.testcontainers.containers.wait.strategy.Wait; import org.testcontainers.utility.DockerLoggerFactory; import java.math.BigDecimal; +import java.sql.Connection; import java.sql.Date; +import java.sql.Driver; import java.sql.ResultSet; +import java.sql.SQLException; import java.sql.Statement; import java.sql.Timestamp; import java.time.Duration; @@ -50,6 +54,7 @@ import java.util.ArrayList; import java.util.HashMap; import java.util.List; import java.util.Map; +import java.util.Properties; public class JdbcDmIT extends AbstractJdbcIT { @@ -314,6 +319,162 @@ public class JdbcDmIT extends AbstractJdbcIT { } } + /** + * Starts the image with a fresh database initialized with LENGTH_IN_CHAR=1 and CHARSET=1 + * (UTF-8), the opposite of the image's default database on both settings. + */ + private static final String INIT_LENGTH_IN_CHAR_UTF8 = + "cd /opt/dmdbms/bin" + + " && su dmdba -c \"./dminit PATH=/opt/dmdbms/st DB_NAME=ST INSTANCE_NAME=ST" + + " PORT_NUM=5236 LENGTH_IN_CHAR=1 CHARSET=1 PAGE_SIZE=8 CASE_SENSITIVE=0" + + " SYSDBA_PWD=SYSDBA001\"" + + " && exec su dmdba -c \"./dmserver path=/opt/dmdbms/st/ST/dm.ini -noconsole\""; + + @Test + public void testNationalCharLengthFromCatalog() throws Exception { + // The shared container runs the image's default database: LENGTH_IN_CHAR=0, GB18030. + String jdbcUrl = jdbcCase.getJdbcUrl().replace(HOST, dbServer.getHost()); + assertNationalCharLengths(jdbcUrl, DM_PASSWORD, 0, 0, true); + + try (GenericContainer<?> lengthInCharUtf8 = + new GenericContainer<>(DM_IMAGE) + .withNetwork(NETWORK) + .withCommand("bash", "-c", INIT_LENGTH_IN_CHAR_UTF8) + .withExposedPorts(DM_CONTAINER_PORT) + .waitingFor( + Wait.forLogMessage(".*SYSTEM IS READY.*", 1) + .withStartupTimeout(Duration.ofMinutes(5))) + .withLogConsumer( + new Slf4jLogConsumer(DockerLoggerFactory.getLogger(DM_IMAGE)))) { + lengthInCharUtf8.start(); + String url = + String.format( + "jdbc:dm://%s:%d", + lengthInCharUtf8.getHost(), + lengthInCharUtf8.getMappedPort(DM_CONTAINER_PORT)); + assertNationalCharLengths(url, "SYSDBA001", 1, 1, false); + } + } + + /** + * Checks that the catalog reports the declared length of NVARCHAR and NVARCHAR2, which is also + * the number of characters they hold, and leaves CHAR, VARCHAR and NCHAR as before. + */ + private void assertNationalCharLengths( + String jdbcUrl, + String password, + int lengthInChar, + int unicodeFlag, + boolean checkReadOnlyUser) + throws Exception { + String table = "E2E_DM_NATIONAL_CHAR"; + String tableInfo = DM_USERNAME + "." + table; + TablePath tablePath = TablePath.of(DM_DATABASE, DM_USERNAME, table); + String reader = "E2E_DM_READER"; + try (Connection conn = connect(jdbcUrl, DM_USERNAME, password); + Statement statement = conn.createStatement()) { + try (ResultSet rs = + statement.executeQuery( + "SELECT SF_GET_LENGTH_IN_CHAR(), SF_GET_UNICODE_FLAG() FROM DUAL")) { + Assertions.assertTrue(rs.next()); + Assertions.assertEquals(lengthInChar, rs.getInt(1)); + Assertions.assertEquals(unicodeFlag, rs.getInt(2)); + } + statement.execute( + "CREATE TABLE " + + tableInfo + + " (V_NVARCHAR NVARCHAR(10), V_NVARCHAR2 NVARCHAR2(10)," + + " V_NVARCHAR_100 NVARCHAR(100), V_NVARCHAR2_100 NVARCHAR2(100)," + + " V_NCHAR NCHAR(10), V_CHAR CHAR(10), V_VARCHAR VARCHAR(10)," + + " V_VARCHAR2 VARCHAR2(10))"); + try { + // NVARCHAR(10) and NVARCHAR2(10) hold 10 characters, multi-byte or not. + String tenChinese = repeat("\u8fbe", 10); + statement.execute( + "INSERT INTO " + + tableInfo + + " (V_NVARCHAR, V_NVARCHAR2) VALUES ('" + + tenChinese + + "', '" + + tenChinese + + "')"); + Assertions.assertThrows( + SQLException.class, + () -> + statement.execute( + "INSERT INTO " + + tableInfo + + " (V_NVARCHAR2) VALUES ('" + + repeat("a", 11) + + "')")); + + assertCatalogLengths(jdbcUrl, DM_USERNAME, password, tablePath); + if (checkReadOnlyUser) { + // A user that can only read the table must get the same lengths. + statement.execute("CREATE USER " + reader + " IDENTIFIED BY \"Reader12345\""); + statement.execute("GRANT SELECT ON " + tableInfo + " TO " + reader); + assertCatalogLengths(jdbcUrl, reader, "Reader12345", tablePath); + } + } finally { + statement.execute("DROP TABLE " + tableInfo); + if (checkReadOnlyUser) { + statement.execute("DROP USER IF EXISTS " + reader + " CASCADE"); + } + } + } + } + + private void assertCatalogLengths( + String jdbcUrl, String user, String password, TablePath tablePath) { + DamengCatalog catalog = + new DamengCatalog( + "dameng", + user, + password, + JdbcUrlUtil.getUrlInfo(jdbcUrl), + DM_USERNAME, + DRIVER_CLASS); + catalog.open(); + try { + CatalogTable catalogTable = catalog.getTable(tablePath); + Map<String, Pair<String, Long>> expected = new HashMap<>(); + expected.put("V_NVARCHAR", Pair.of("NVARCHAR(10)", 40L)); + expected.put("V_NVARCHAR2", Pair.of("NVARCHAR2(10)", 40L)); + expected.put("V_NVARCHAR_100", Pair.of("NVARCHAR(100)", 400L)); + expected.put("V_NVARCHAR2_100", Pair.of("NVARCHAR2(100)", 400L)); + expected.put("V_NCHAR", Pair.of("NCHAR(10)", 40L)); + expected.put("V_CHAR", Pair.of("CHAR(10)", 40L)); + expected.put("V_VARCHAR", Pair.of("VARCHAR2(10)", 40L)); + expected.put("V_VARCHAR2", Pair.of("VARCHAR2(10)", 40L)); + expected.forEach( + (name, type) -> { + Column column = catalogTable.getTableSchema().getColumn(name); + Assertions.assertEquals(type.getLeft(), column.getSourceType(), name); + Assertions.assertEquals(type.getRight(), column.getColumnLength(), name); + }); + } finally { + catalog.close(); + } + } + + private Connection connect(String jdbcUrl, String user, String password) throws Exception { + Driver driver = (Driver) loadDriverClass().newInstance(); + Properties props = new Properties(); + props.put("user", user); + props.put("password", password); + Connection conn = driver.connect(jdbcUrl, props); + conn.setAutoCommit(true); + return conn; + } + + private static String repeat(String value, int count) { + StringBuilder builder = new StringBuilder(); + for (int i = 0; i < count; i++) { + builder.append(value); + } + return builder.toString(); + } + protected String buildTableInfoWithSchema(String database, String schema, String table) { return buildTableInfoWithSchema(schema, table); }
