This is an automated email from the ASF dual-hosted git repository.
yuqi1129 pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/gravitino.git
The following commit(s) were added to refs/heads/main by this push:
new 99ecf448d8 [#12764] fix(doris): preserve defaults when adding columns
(#12767)
99ecf448d8 is described below
commit 99ecf448d8eb66436a83945da0a483cf98451efd
Author: StormSpirit <[email protected]>
AuthorDate: Tue Sep 22 17:43:40 2026 +0800
[#12764] fix(doris): preserve defaults when adding columns (#12767)
### What changes were proposed in this pull request?
- Preserve string defaults when adding Doris columns, including values
with consecutive quotes or backslashes.
- Use single-quoted literals with doubled apostrophes for Doris versions
before 4.0 when version-aware escaping is needed. Apply the additional
backslash-escaping layer for Doris 3.x, while retaining the
double-quoted literal form for Doris 4.x.
- Detect the Doris version with `SELECT Version FROM FRONTENDS()` first,
then fall back to `SHOW FRONTENDS` on older versions that do not support
the table-valued function. Run the lookup only when a string default
contains backslashes or consecutive double quotes.
- Preserve repeated single and double quotes when converting string
defaults from JDBC metadata; continue decoding backslash escape
sequences.
- Add unit and Docker IT coverage for SQL generation, metadata and
inserted-value round-trips across Doris 1.2.x, 3.0.6.2, and 4.0.6.
Include a limited-privilege AddColumn case for Doris 3.0.6.2.
### Why are the changes needed?
The Doris catalog accepts a default value through the public AddColumn
API but currently omits it from the generated column definition. A
nullable column can therefore be added while silently losing its
requested default, and Doris may reject a non-nullable column because
the generated ADD COLUMN statement has no default.
Doris versions also parse backslashes in ALTER ADD COLUMN defaults
differently. Keeping this compatibility handling in an ADD-specific
write path preserves the requested default without changing CREATE TABLE
or unrelated MODIFY COLUMN write serialization. This patch separately
normalizes the escaped string-default forms returned by Doris JDBC
metadata during `loadTable()`.
Fix: #12764
### Does this PR introduce _any_ user-facing change?
Yes. Doris ADD COLUMN operations now preserve supported literal
defaults, including strings with backslashes and consecutive double
quotes. For a string default containing either condition, the connector
resolves the FE version once per ALTER request by querying `SELECT
Version FROM FRONTENDS()` first, then falls back to `SHOW FRONTENDS` on
Doris versions that do not support the table-valued function. Doris
1.2.x accounts using this fallback need permission to run `SHOW
FRONTENDS`; on Doris 3.0.6.2, the tested limited-privilege account can
query the version through `FRONTENDS()`. Explicit `DEFAULT NULL` remains
distinct from an unset default at SQL generation time. No public API or
property key is added or removed.
### How was this patch tested?
- `./gradlew :catalogs:catalog-jdbc-doris:spotlessApply`
- `./gradlew :catalogs:catalog-jdbc-doris:test -PskipITs`
- `./gradlew :catalogs:catalog-jdbc-doris:test --tests
'org.apache.gravitino.catalog.doris.integration.test.CatalogDorisIT'
-PskipDockerTests=false -PdorisMultiVersionTest`
- `./gradlew :catalogs:catalog-jdbc-doris:test --tests
'org.apache.gravitino.catalog.doris.integration.test.CatalogDoris3xIT.testAddColumnPreservesDefaultValue'
--tests
'org.apache.gravitino.catalog.doris.integration.test.CatalogDoris3xIT.testAddColumnBackslashDefaultWithLimitedDorisPrivileges'
-PskipDockerTests=false -PdorisMultiVersionTest`
- `./gradlew :catalogs:catalog-jdbc-doris:test --tests
'org.apache.gravitino.catalog.doris.integration.test.CatalogDoris4xIT.testAddColumnPreservesDefaultValue'
-PskipDockerTests=false -PdorisMultiVersionTest`
All listed validations passed.
---------
Signed-off-by: jiangxt2 <[email protected]>
---
.../DorisColumnDefaultValueConverter.java | 85 +++++-
.../doris/operation/DorisTableOperations.java | 139 ++++++++--
.../TestDorisColumnDefaultValueConverter.java | 141 ++++++++++
.../doris/integration/test/CatalogDoris3xIT.java | 280 +++++++++++++++++++
.../doris/integration/test/CatalogDoris4xIT.java | 174 ++++++++++++
.../doris/integration/test/CatalogDorisIT.java | 198 ++++++++++++++
.../TestDorisTableOperationsSqlGeneration.java | 300 +++++++++++++++++++--
7 files changed, 1264 insertions(+), 53 deletions(-)
diff --git
a/catalogs/catalog-jdbc-doris/src/main/java/org/apache/gravitino/catalog/doris/converter/DorisColumnDefaultValueConverter.java
b/catalogs/catalog-jdbc-doris/src/main/java/org/apache/gravitino/catalog/doris/converter/DorisColumnDefaultValueConverter.java
index 2566f226cd..0d0c8d033d 100644
---
a/catalogs/catalog-jdbc-doris/src/main/java/org/apache/gravitino/catalog/doris/converter/DorisColumnDefaultValueConverter.java
+++
b/catalogs/catalog-jdbc-doris/src/main/java/org/apache/gravitino/catalog/doris/converter/DorisColumnDefaultValueConverter.java
@@ -36,12 +36,58 @@ import
org.apache.gravitino.catalog.jdbc.converter.JdbcColumnDefaultValueConvert
import org.apache.gravitino.catalog.jdbc.converter.JdbcTypeConverter;
import org.apache.gravitino.rel.expressions.Expression;
import org.apache.gravitino.rel.expressions.UnparsedExpression;
+import org.apache.gravitino.rel.expressions.literals.Literal;
import org.apache.gravitino.rel.expressions.literals.Literals;
import org.apache.gravitino.rel.types.Decimal;
+import org.apache.gravitino.rel.types.Type;
import org.apache.gravitino.rel.types.Types;
public class DorisColumnDefaultValueConverter extends
JdbcColumnDefaultValueConverter {
+ /**
+ * Converts an ADD COLUMN default value to Doris SQL.
+ *
+ * @param defaultValue the Gravitino default value
+ * @param doubleEscapeBackslashes whether to add the extra backslash
escaping required by Doris
+ * 3.x ALTER ADD COLUMN
+ * @return the Doris SQL representation
+ */
+ public String fromGravitinoForAddColumn(
+ Expression defaultValue, boolean doubleEscapeBackslashes) {
+ return fromGravitinoForAddColumn(defaultValue, doubleEscapeBackslashes,
false);
+ }
+
+ /**
+ * Converts an ADD COLUMN default value to Doris SQL with a selected
string-literal delimiter.
+ *
+ * @param defaultValue the Gravitino default value
+ * @param doubleEscapeBackslashes whether to add the extra backslash
escaping required by Doris
+ * 3.x ALTER ADD COLUMN
+ * @param useSingleQuoteDelimiter whether string literals should use single
quotes and doubled
+ * apostrophes, as required by the legacy Doris parser for adjacent
double quotes
+ * @return the Doris SQL representation
+ */
+ public String fromGravitinoForAddColumn(
+ Expression defaultValue, boolean doubleEscapeBackslashes, boolean
useSingleQuoteDelimiter) {
+ if (defaultValue instanceof Literal) {
+ Literal<?> literal = (Literal<?>) defaultValue;
+ if (literal.equals(Literals.NULL)) {
+ return super.fromGravitino(defaultValue);
+ }
+ // Doris 1.2 parses literal defaults as quoted strings and uses
MySQL-style escaping.
+ if (literal.dataType() instanceof Type.NumericType
+ || literal.dataType() instanceof Types.StringType
+ || literal.dataType() instanceof Types.VarCharType
+ || literal.dataType() instanceof Types.FixedCharType) {
+ return quoteDorisLiteral(
+ String.valueOf(literal.value()), doubleEscapeBackslashes,
useSingleQuoteDelimiter);
+ }
+ }
+ // Doris accepts the base converter's standard SQL syntax for date/time
literals and
+ // expressions.
+ return super.fromGravitino(defaultValue);
+ }
+
@Override
public Expression toGravitino(
JdbcTypeConverter.JdbcTypeBean columnType,
@@ -90,13 +136,46 @@ public class DorisColumnDefaultValueConverter extends
JdbcColumnDefaultValueConv
: Literals.timestampLiteral(
LocalDateTime.parse(columnDefaultValue, DATE_TIME_FORMATTER));
case JdbcTypeConverter.VARCHAR:
- return Literals.of(columnDefaultValue,
Types.VarCharType.of(columnType.getColumnSize()));
+ return Literals.of(
+ unescapeDorisLiteral(columnDefaultValue),
+ Types.VarCharType.of(columnType.getColumnSize()));
case CHAR:
- return Literals.of(columnDefaultValue,
Types.FixedCharType.of(columnType.getColumnSize()));
+ return Literals.of(
+ unescapeDorisLiteral(columnDefaultValue),
+ Types.FixedCharType.of(columnType.getColumnSize()));
case JdbcTypeConverter.TEXT:
- return Literals.stringLiteral(columnDefaultValue);
+ return
Literals.stringLiteral(unescapeDorisLiteral(columnDefaultValue));
default:
return UnparsedExpression.of(columnDefaultValue);
}
}
+
+ private static String quoteDorisLiteral(
+ String value, boolean doubleEscapeBackslashes, boolean
useSingleQuoteDelimiter) {
+ String escapedBackslash = doubleEscapeBackslashes ? "\\\\\\\\" : "\\\\";
+ String escaped = value.replace("\\", escapedBackslash);
+ if (useSingleQuoteDelimiter) {
+ escaped = escaped.replace("'", "''");
+ return "'" + escaped + "'";
+ }
+ escaped = escaped.replace("\"", "\\\"");
+ return "\"" + escaped + "\"";
+ }
+
+ private static String unescapeDorisLiteral(String value) {
+ StringBuilder result = new StringBuilder(value.length());
+ for (int i = 0; i < value.length(); i++) {
+ char current = value.charAt(i);
+ if (current == '\\' && i + 1 < value.length()) {
+ char next = value.charAt(i + 1);
+ if (next == '\\' || next == '\'' || next == '"') {
+ result.append(next);
+ i++;
+ continue;
+ }
+ }
+ result.append(current);
+ }
+ return result.toString();
+ }
}
diff --git
a/catalogs/catalog-jdbc-doris/src/main/java/org/apache/gravitino/catalog/doris/operation/DorisTableOperations.java
b/catalogs/catalog-jdbc-doris/src/main/java/org/apache/gravitino/catalog/doris/operation/DorisTableOperations.java
index 8c25e45f56..f544dfad0d 100644
---
a/catalogs/catalog-jdbc-doris/src/main/java/org/apache/gravitino/catalog/doris/operation/DorisTableOperations.java
+++
b/catalogs/catalog-jdbc-doris/src/main/java/org/apache/gravitino/catalog/doris/operation/DorisTableOperations.java
@@ -55,6 +55,7 @@ import org.apache.commons.lang3.ArrayUtils;
import org.apache.commons.lang3.BooleanUtils;
import org.apache.commons.lang3.StringUtils;
import org.apache.gravitino.StringIdentifier;
+import
org.apache.gravitino.catalog.doris.converter.DorisColumnDefaultValueConverter;
import org.apache.gravitino.catalog.doris.utils.DorisUtils;
import org.apache.gravitino.catalog.jdbc.JdbcColumn;
import org.apache.gravitino.catalog.jdbc.JdbcTable;
@@ -64,6 +65,7 @@ import org.apache.gravitino.exceptions.NoSuchColumnException;
import org.apache.gravitino.exceptions.NoSuchTableException;
import org.apache.gravitino.rel.Column;
import org.apache.gravitino.rel.TableChange;
+import org.apache.gravitino.rel.expressions.Expression;
import org.apache.gravitino.rel.expressions.distributions.Distribution;
import org.apache.gravitino.rel.expressions.distributions.Strategy;
import org.apache.gravitino.rel.expressions.literals.Literal;
@@ -264,47 +266,78 @@ public class DorisTableOperations extends
JdbcTableOperations {
if (!hasAutoIncrement) {
return;
}
+ String version = getDorisVersion("AUTO_INCREMENT compatibility check");
+ if (!isVersionAtLeast(version, 2, 1, 0)) {
+ throw new UnsupportedOperationException(
+ "AUTO_INCREMENT requires Doris 2.1.0 or later. Current server
version: " + version);
+ }
+ }
+
+ private String getDorisVersion(String purpose) {
Preconditions.checkState(dataSource != null, "dataSource is required for
version validation");
String version = null;
- // SELECT VERSION() returns the MySQL protocol version (e.g. "5.7.99"),
not the Doris version.
- // SHOW FRONTENDS returns the actual Doris version in the "Version" column
- // (e.g. "doris-3.0.6.2-rc01-910c4249c5").
- try (Connection connection = dataSource.getConnection();
- Statement stmt = connection.createStatement();
- ResultSet rs = stmt.executeQuery("SHOW FRONTENDS")) {
- ResultSetMetaData meta = rs.getMetaData();
- int versionCol = -1;
- for (int i = 1; i <= meta.getColumnCount(); i++) {
- if ("Version".equals(meta.getColumnLabel(i))) {
- versionCol = i;
- break;
+ SQLException frontendsQueryFailure = null;
+ try (Connection connection = dataSource.getConnection()) {
+ // FRONTENDS() is readable with the default information_schema SELECT
privilege on Doris
+ // 3.0.6.2, while SHOW FRONTENDS requires ADMIN/OPERATOR there. Older
Doris releases such as
+ // 1.2.x do not support the table-valued function, so retain SHOW
FRONTENDS as a fallback.
+ try (Statement stmt = connection.createStatement();
+ ResultSet rs = stmt.executeQuery("SELECT Version FROM FRONTENDS()"))
{
+ if (rs.next()) {
+ version = extractDorisVersion(rs.getString(1));
}
+ } catch (SQLException e) {
+ frontendsQueryFailure = e;
}
- if (rs.next() && versionCol > 0) {
- String versionStr = rs.getString(versionCol);
- // Extract X.Y.Z from "doris-X.Y.Z-suffix-commit" using regex for
robustness
- Matcher matcher = DORIS_VERSION_PATTERN.matcher(versionStr);
- if (matcher.find()) {
- version = matcher.group(1);
+
+ if (version == null) {
+ try (Statement stmt = connection.createStatement();
+ ResultSet rs = stmt.executeQuery("SHOW FRONTENDS")) {
+ ResultSetMetaData meta = rs.getMetaData();
+ int versionCol = -1;
+ for (int i = 1; i <= meta.getColumnCount(); i++) {
+ if ("Version".equalsIgnoreCase(meta.getColumnLabel(i))) {
+ versionCol = i;
+ break;
+ }
+ }
+ if (rs.next() && versionCol > 0) {
+ version = extractDorisVersion(rs.getString(versionCol));
+ }
+ } catch (SQLException e) {
+ if (frontendsQueryFailure != null) {
+ e.addSuppressed(frontendsQueryFailure);
+ }
+ throw e;
}
}
} catch (SQLException e) {
throw new UnsupportedOperationException(
- "Unable to determine Doris version for AUTO_INCREMENT compatibility
check. "
- + "Ensure the connection user has permission to execute SHOW
FRONTENDS "
- + "and the Doris FE is reachable.",
+ "Unable to determine Doris version for "
+ + purpose
+ + ". "
+ + "Ensure the connection user can query FRONTENDS() or, on older
Doris versions, "
+ + "has permission to execute SHOW FRONTENDS, and the Doris FE is
reachable.",
e);
}
if (version == null) {
throw new UnsupportedOperationException(
- "Unable to determine Doris version for AUTO_INCREMENT compatibility
check. "
- + "Ensure the connection user has permission to execute SHOW
FRONTENDS "
- + "and the Doris FE is reachable.");
+ "Unable to determine Doris version for "
+ + purpose
+ + ". "
+ + "Ensure the connection user can query FRONTENDS() or, on older
Doris versions, "
+ + "has permission to execute SHOW FRONTENDS, and the Doris FE is
reachable.");
}
- if (!isVersionAtLeast(version, 2, 1, 0)) {
- throw new UnsupportedOperationException(
- "AUTO_INCREMENT requires Doris 2.1.0 or later. Current server
version: " + version);
+ return version;
+ }
+
+ @Nullable
+ private static String extractDorisVersion(@Nullable String versionString) {
+ if (versionString == null) {
+ return null;
}
+ Matcher matcher = DORIS_VERSION_PATTERN.matcher(versionString);
+ return matcher.find() ? matcher.group(1) : null;
}
@VisibleForTesting
@@ -759,6 +792,14 @@ public class DorisTableOperations extends
JdbcTableOperations {
TableChange.UpdateComment updateComment = null;
List<TableChange.SetProperty> setProperties = new ArrayList<>();
List<String> alterSql = new ArrayList<>();
+ Optional<String> addColumnDorisVersion =
+ Arrays.stream(changes)
+ .filter(TableChange.AddColumn.class::isInstance)
+ .map(TableChange.AddColumn.class::cast)
+ .map(TableChange.AddColumn::getDefaultValue)
+
.anyMatch(DorisTableOperations::requiresVersionAwareAddColumnEscaping)
+ ? Optional.of(getDorisVersion("ADD COLUMN default literal
compatibility check"))
+ : Optional.empty();
for (int i = 0; i < changes.length; i++) {
TableChange change = changes[i];
if (change instanceof TableChange.UpdateComment) {
@@ -772,7 +813,7 @@ public class DorisTableOperations extends
JdbcTableOperations {
} else if (change instanceof TableChange.AddColumn) {
TableChange.AddColumn addColumn = (TableChange.AddColumn) change;
lazyLoadTable = getOrCreateTable(databaseName, tableName,
lazyLoadTable);
- alterSql.add(addColumnFieldDefinition(addColumn));
+ alterSql.add(addColumnFieldDefinition(addColumn,
addColumnDorisVersion));
} else if (change instanceof TableChange.RenameColumn) {
throw new IllegalArgumentException("Rename column is not supported
yet");
} else if (change instanceof TableChange.UpdateColumnType) {
@@ -910,7 +951,8 @@ public class DorisTableOperations extends
JdbcTableOperations {
"MODIFY COLUMN `%s` COMMENT '%s'", col, escapeSqlLiteral(newComment,
'\''));
}
- private String addColumnFieldDefinition(TableChange.AddColumn addColumn) {
+ private String addColumnFieldDefinition(
+ TableChange.AddColumn addColumn, Optional<String> dorisVersion) {
String dataType = typeConverter.fromGravitino(addColumn.getDataType());
if (addColumn.fieldName().length > 1) {
throw new UnsupportedOperationException("Doris does not support nested
column names.");
@@ -930,6 +972,14 @@ public class DorisTableOperations extends
JdbcTableOperations {
if (!addColumn.isNullable()) {
columnDefinition.append("NOT NULL ");
}
+
+ if (!DEFAULT_VALUE_NOT_SET.equals(addColumn.getDefaultValue())) {
+ columnDefinition
+ .append("DEFAULT ")
+ .append(serializeAddColumnDefaultValue(addColumn.getDefaultValue(),
dorisVersion))
+ .append(SPACE);
+ }
+
// Append comment if available
if (StringUtils.isNotEmpty(addColumn.getComment())) {
columnDefinition
@@ -956,6 +1006,37 @@ public class DorisTableOperations extends
JdbcTableOperations {
return columnDefinition.toString();
}
+ private static boolean requiresVersionAwareAddColumnEscaping(Expression
defaultValue) {
+ if (!(defaultValue instanceof Literal)) {
+ return false;
+ }
+ Object value = ((Literal<?>) defaultValue).value();
+ if (value == null) {
+ return false;
+ }
+ String stringValue = String.valueOf(value);
+ return stringValue.contains("\\") || stringValue.contains("\"\"");
+ }
+
+ private String serializeAddColumnDefaultValue(
+ Expression defaultValue, Optional<String> dorisVersion) {
+ Preconditions.checkState(
+ columnDefaultValueConverter instanceof
DorisColumnDefaultValueConverter,
+ "DorisColumnDefaultValueConverter is required for Doris ADD COLUMN");
+ DorisColumnDefaultValueConverter converter =
+ (DorisColumnDefaultValueConverter) columnDefaultValueConverter;
+ boolean requiresDoubleEscaping =
+ dorisVersion
+ .map(
+ version ->
+ isVersionAtLeast(version, 3, 0, 0) &&
!isVersionAtLeast(version, 4, 0, 0))
+ .orElse(false);
+ boolean useSingleQuoteDelimiter =
+ dorisVersion.map(version -> !isVersionAtLeast(version, 4, 0,
0)).orElse(false);
+ return converter.fromGravitinoForAddColumn(
+ defaultValue, requiresDoubleEscaping, useSingleQuoteDelimiter);
+ }
+
private String updateColumnPositionFieldDefinition(
TableChange.UpdateColumnPosition updateColumnPosition, JdbcTable
jdbcTable) {
if (updateColumnPosition.fieldName().length > 1) {
diff --git
a/catalogs/catalog-jdbc-doris/src/test/java/org/apache/gravitino/catalog/doris/converter/TestDorisColumnDefaultValueConverter.java
b/catalogs/catalog-jdbc-doris/src/test/java/org/apache/gravitino/catalog/doris/converter/TestDorisColumnDefaultValueConverter.java
new file mode 100644
index 0000000000..0f494cbc79
--- /dev/null
+++
b/catalogs/catalog-jdbc-doris/src/test/java/org/apache/gravitino/catalog/doris/converter/TestDorisColumnDefaultValueConverter.java
@@ -0,0 +1,141 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.gravitino.catalog.doris.converter;
+
+import static
org.apache.gravitino.rel.Column.DEFAULT_VALUE_OF_CURRENT_TIMESTAMP;
+
+import org.apache.gravitino.catalog.jdbc.converter.JdbcTypeConverter;
+import org.apache.gravitino.rel.expressions.literals.Literal;
+import org.apache.gravitino.rel.expressions.literals.Literals;
+import org.apache.gravitino.rel.types.Types;
+import org.junit.jupiter.api.Assertions;
+import org.junit.jupiter.api.Test;
+
+/** Tests for converting Doris column default values to SQL and Gravitino
expressions. */
+public class TestDorisColumnDefaultValueConverter {
+
+ private static final DorisColumnDefaultValueConverter CONVERTER =
+ new DorisColumnDefaultValueConverter();
+
+ @Test
+ public void testEscapeStringLikeDefaultValues() {
+ Assertions.assertEquals(
+ "\"owner's\\\\value\"",
+ CONVERTER.fromGravitinoForAddColumn(
+ Literals.of("owner's\\value", Types.VarCharType.of(255)), false));
+ Assertions.assertEquals(
+ "\"owner's\\\\value\"",
+ CONVERTER.fromGravitinoForAddColumn(
+ Literals.of("owner's\\value", Types.FixedCharType.of(32)), false));
+ Assertions.assertEquals(
+ "\"owner's\\\\value\"",
+
CONVERTER.fromGravitinoForAddColumn(Literals.stringLiteral("owner's\\value"),
false));
+ Assertions.assertEquals(
+ "\"owner's \\\"value\\\"\\\\path\"",
+ CONVERTER.fromGravitinoForAddColumn(
+ Literals.of("owner's \"value\"\\path", Types.VarCharType.of(255)),
false));
+ Assertions.assertEquals(
+ "\"\"",
+ CONVERTER.fromGravitinoForAddColumn(Literals.of("",
Types.VarCharType.of(255)), false));
+ Assertions.assertEquals(
+ "\" \"",
+ CONVERTER.fromGravitinoForAddColumn(Literals.of(" ",
Types.VarCharType.of(255)), false));
+ }
+
+ @Test
+ public void testNumericDefaultUsesLegacyCompatibleQuotedSyntax() {
+ Assertions.assertEquals(
+ "\"7\"",
CONVERTER.fromGravitinoForAddColumn(Literals.integerLiteral(7), false));
+ }
+
+ @Test
+ public void testDelegateNullAndExpressionDefaultValues() {
+ Assertions.assertEquals("NULL",
CONVERTER.fromGravitinoForAddColumn(Literals.NULL, false));
+ Assertions.assertEquals(
+ "CURRENT_TIMESTAMP",
+
CONVERTER.fromGravitinoForAddColumn(DEFAULT_VALUE_OF_CURRENT_TIMESTAMP, false));
+ }
+
+ @Test
+ public void testDoubleEscapeSequencesForDoris3Alter() {
+ Literal<?> defaultValue = Literals.of("owner's \"value\"\\path",
Types.VarCharType.of(255));
+ Assertions.assertEquals(
+ "\"owner's \\\"value\\\"\\\\\\\\path\"",
+ CONVERTER.fromGravitinoForAddColumn(defaultValue, true));
+ }
+
+ @Test
+ public void testSingleQuotedLegacyLiteralPreservesAdjacentDoubleQuotes() {
+ Literal<?> defaultValue =
+ Literals.of("owner's a\"\"b \"value\"\\path",
Types.VarCharType.of(255));
+
+ Assertions.assertEquals(
+ "'owner''s a\"\"b \"value\"\\\\path'",
+ CONVERTER.fromGravitinoForAddColumn(defaultValue, false, true));
+ Assertions.assertEquals(
+ "'owner''s a\"\"b \"value\"\\\\\\\\path'",
+ CONVERTER.fromGravitinoForAddColumn(defaultValue, true, true));
+ }
+
+ @Test
+ public void testRepeatedApostrophesRemainLiteralDuringWriteAndRead() {
+ Literal<?> defaultValue = Literals.of("owner''s",
Types.VarCharType.of(255));
+ JdbcTypeConverter.JdbcTypeBean varcharType =
+ new JdbcTypeConverter.JdbcTypeBean(JdbcTypeConverter.VARCHAR);
+ varcharType.setColumnSize(255);
+
+ Assertions.assertEquals(
+ "'owner''''s'", CONVERTER.fromGravitinoForAddColumn(defaultValue,
false, true));
+ Assertions.assertEquals(
+ defaultValue, CONVERTER.toGravitino(varcharType, "owner''s", false,
false));
+ }
+
+ @Test
+ public void testUnescapeStringLikeDefaultValues() {
+ JdbcTypeConverter.JdbcTypeBean varcharType =
+ new JdbcTypeConverter.JdbcTypeBean(JdbcTypeConverter.VARCHAR);
+ varcharType.setColumnSize(255);
+ JdbcTypeConverter.JdbcTypeBean charType =
+ new JdbcTypeConverter.JdbcTypeBean(DorisTypeConverter.CHAR);
+ charType.setColumnSize(32);
+
+ Assertions.assertEquals(
+ Literals.of("owner's\\value", Types.VarCharType.of(255)),
+ CONVERTER.toGravitino(varcharType, "owner's\\\\value", false, false));
+ Assertions.assertEquals(
+ Literals.of("owner's\\value", Types.FixedCharType.of(32)),
+ CONVERTER.toGravitino(charType, "owner's\\\\value", false, false));
+ Assertions.assertEquals(
+ Literals.stringLiteral("owner's\\value"),
+ CONVERTER.toGravitino(
+ new JdbcTypeConverter.JdbcTypeBean(JdbcTypeConverter.TEXT),
+ "owner's\\\\value",
+ false,
+ false));
+ Assertions.assertEquals(
+ Literals.of("owner's \"value\"\\path", Types.VarCharType.of(255)),
+ CONVERTER.toGravitino(varcharType, "owner's \\\"value\\\"\\\\path",
false, false));
+ Assertions.assertEquals(
+ Literals.of("owner's \"value\"\\path", Types.VarCharType.of(255)),
+ CONVERTER.toGravitino(varcharType, "owner's \"value\"\\path", false,
false));
+ Assertions.assertEquals(
+ Literals.of("owner's a\"\"b \"value\"\\path",
Types.VarCharType.of(255)),
+ CONVERTER.toGravitino(varcharType, "owner's a\"\"b \"value\"\\\\path",
false, false));
+ }
+}
diff --git
a/catalogs/catalog-jdbc-doris/src/test/java/org/apache/gravitino/catalog/doris/integration/test/CatalogDoris3xIT.java
b/catalogs/catalog-jdbc-doris/src/test/java/org/apache/gravitino/catalog/doris/integration/test/CatalogDoris3xIT.java
index fc5e250f90..a7c73db6e4 100644
---
a/catalogs/catalog-jdbc-doris/src/test/java/org/apache/gravitino/catalog/doris/integration/test/CatalogDoris3xIT.java
+++
b/catalogs/catalog-jdbc-doris/src/test/java/org/apache/gravitino/catalog/doris/integration/test/CatalogDoris3xIT.java
@@ -309,6 +309,286 @@ public class CatalogDoris3xIT extends BaseIT {
.untilAsserted(() -> assertEquals(0,
tc.loadTable(tid).index().length));
}
+ @Test
+ void testAddColumnPreservesDefaultValue() throws SQLException {
+ TableCatalog tc = catalog.asTableCatalog();
+ NameIdentifier tid =
+ NameIdentifier.of(
+ schemaName,
GravitinoITUtils.genRandomName("t_add_column_preserves_default"));
+ String defaultedColumnName = "defaulted_col";
+ String nullableDefaultColumnName = "nullable_default_col";
+ String requestedDefaultValue = "owner's a\"\"b \"value\"\\path";
+
+ tc.createTable(
+ tid,
+ basicColumns(),
+ tableComment,
+ Collections.emptyMap(),
+ Transforms.EMPTY_TRANSFORM,
+ hashDist(),
+ null,
+ null);
+
+ tc.alterTable(
+ tid,
+ TableChange.addColumn(
+ new String[] {defaultedColumnName},
+ Types.VarCharType.of(255),
+ "defaulted column",
+ TableChange.ColumnPosition.defaultPos(),
+ false,
+ false,
+ Literals.of(requestedDefaultValue, Types.VarCharType.of(255))));
+
+ String diagnosticQualifiedTableName = schemaName + "." + tid.name();
+ String[] rawDefaults = new String[3];
+ Awaitility.await()
+ .atMost(MAX_WAIT_IN_SECONDS, TimeUnit.SECONDS)
+ .pollInterval(WAIT_INTERVAL_IN_SECONDS, TimeUnit.SECONDS)
+ .untilAsserted(
+ () -> {
+ try (Connection connection =
+ DriverManager.getConnection(
+ jdbcUrl, DorisContainer.USER_NAME,
DorisContainer.PASSWORD);
+ Statement statement = connection.createStatement();
+ ResultSet columns =
+ connection
+ .getMetaData()
+ .getColumns(schemaName, null, tid.name(),
defaultedColumnName)) {
+ String jdbcColumnDefault = null;
+ while (columns.next()) {
+ if
(defaultedColumnName.equals(columns.getString("COLUMN_NAME"))) {
+ jdbcColumnDefault = columns.getString("COLUMN_DEF");
+ break;
+ }
+ }
+ assertNotNull(jdbcColumnDefault, "JDBC COLUMN_DEF was not
returned");
+ rawDefaults[0] = jdbcColumnDefault;
+
+ try (ResultSet showColumns =
+ statement.executeQuery(
+ "SHOW FULL COLUMNS FROM " +
diagnosticQualifiedTableName)) {
+ while (showColumns.next()) {
+ if
(defaultedColumnName.equals(showColumns.getString("Field"))) {
+ rawDefaults[1] = showColumns.getString("Default");
+ break;
+ }
+ }
+ }
+ assertNotNull(
+ rawDefaults[1], "SHOW FULL COLUMNS did not return the
string default");
+
+ String infoSchemaQuery =
+ String.format(
+ "SELECT COLUMN_DEFAULT FROM information_schema.columns
"
+ + "WHERE TABLE_SCHEMA = '%s' AND TABLE_NAME = '%s'
"
+ + "AND COLUMN_NAME = '%s'",
+ schemaName, tid.name(), defaultedColumnName);
+ try (ResultSet infoSchema =
statement.executeQuery(infoSchemaQuery)) {
+ assertTrue(infoSchema.next(), "information_schema.columns
row was not returned");
+ rawDefaults[2] = infoSchema.getString("COLUMN_DEFAULT");
+ }
+ }
+ });
+
+ String metadataDiagnostic =
+ String.format(
+ "JDBC COLUMN_DEF=[%s], SHOW FULL COLUMNS Default=[%s], "
+ + "information_schema.COLUMN_DEFAULT=[%s]",
+ rawDefaults[0], rawDefaults[1], rawDefaults[2]);
+ DorisContainer.LOG.info("Doris default metadata diagnostic: {}",
metadataDiagnostic);
+ String[] insertedDefaultValue = new String[1];
+ try (Connection connection =
+ DriverManager.getConnection(
+ jdbcUrl, DorisContainer.USER_NAME, DorisContainer.PASSWORD);
+ Statement statement = connection.createStatement()) {
+ statement.executeUpdate(
+ String.format(
+ "INSERT INTO %s (%s, %s) VALUES (102, 'data')",
+ diagnosticQualifiedTableName, colName1, colName2));
+ try (ResultSet resultSet =
+ statement.executeQuery(
+ String.format(
+ "SELECT %s FROM %s WHERE %s = 102",
+ defaultedColumnName, diagnosticQualifiedTableName,
colName1))) {
+ assertTrue(resultSet.next());
+ insertedDefaultValue[0] = resultSet.getString(1);
+ assertFalse(resultSet.next());
+ }
+ }
+ assertEquals(
+ requestedDefaultValue,
+ insertedDefaultValue[0],
+ metadataDiagnostic + "; inserted value=[" + insertedDefaultValue[0] +
"]");
+
+ Awaitility.await()
+ .atMost(MAX_WAIT_IN_SECONDS, TimeUnit.SECONDS)
+ .pollInterval(WAIT_INTERVAL_IN_SECONDS, TimeUnit.SECONDS)
+ .untilAsserted(
+ () -> {
+ Column addedColumn = findColumn(tc.loadTable(tid),
defaultedColumnName);
+ assertEquals(Types.VarCharType.of(255), addedColumn.dataType());
+ assertFalse(addedColumn.nullable());
+ assertEquals("defaulted column", addedColumn.comment());
+ assertEquals(
+ Literals.of(requestedDefaultValue,
Types.VarCharType.of(255)),
+ addedColumn.defaultValue(),
+ metadataDiagnostic + "; inserted value=[" +
insertedDefaultValue[0] + "]");
+ });
+
+ tc.alterTable(
+ tid,
+ TableChange.addColumn(
+ new String[] {nullableDefaultColumnName},
+ Types.IntegerType.get(),
+ "nullable default column",
+ TableChange.ColumnPosition.defaultPos(),
+ true,
+ false,
+ Literals.integerLiteral(9)));
+
+ Awaitility.await()
+ .atMost(MAX_WAIT_IN_SECONDS, TimeUnit.SECONDS)
+ .pollInterval(WAIT_INTERVAL_IN_SECONDS, TimeUnit.SECONDS)
+ .untilAsserted(
+ () -> {
+ Column addedColumn = findColumn(tc.loadTable(tid),
nullableDefaultColumnName);
+ assertEquals(Types.IntegerType.get(), addedColumn.dataType());
+ assertTrue(addedColumn.nullable());
+ assertEquals("nullable default column", addedColumn.comment());
+ assertEquals(Literals.integerLiteral(9),
addedColumn.defaultValue());
+ });
+
+ String qualifiedTableName = String.format("`%s`.`%s`", schemaName,
tid.name());
+ try (Connection connection =
+ DriverManager.getConnection(
+ jdbcUrl, DorisContainer.USER_NAME, DorisContainer.PASSWORD);
+ Statement statement = connection.createStatement()) {
+ statement.executeUpdate(
+ String.format(
+ "INSERT INTO %s (`%s`, `%s`) VALUES (101, 'data')",
+ qualifiedTableName, colName1, colName2));
+
+ try (ResultSet resultSet =
+ statement.executeQuery(
+ String.format(
+ "SELECT `%s`, `%s` FROM %s WHERE `%s` = 101",
+ defaultedColumnName, nullableDefaultColumnName,
qualifiedTableName, colName1))) {
+ assertTrue(resultSet.next());
+ assertEquals(requestedDefaultValue, resultSet.getString(1));
+ assertEquals(9, resultSet.getInt(2));
+ assertFalse(resultSet.wasNull());
+ assertFalse(resultSet.next());
+ }
+ }
+ }
+
+ @Test
+ void testAddColumnBackslashDefaultWithLimitedDorisPrivileges() throws
SQLException {
+ TableCatalog rootTableCatalog = catalog.asTableCatalog();
+ NameIdentifier tableIdentifier =
+ NameIdentifier.of(
+ schemaName,
GravitinoITUtils.genRandomName("t_add_column_limited_privileges"));
+ String defaultedColumnName = "defaulted_col";
+ String requestedDefaultValue = "owner's a\"\"b \"value\"\\path";
+ String userName =
GravitinoITUtils.genRandomName("doris_add_column_user").replace('-', '_');
+ String userPassword = userName + "_test_password";
+ String limitedCatalogName =
GravitinoITUtils.genRandomName("doris_limited_catalog");
+ Catalog limitedCatalog = null;
+ boolean userCreated = false;
+
+ rootTableCatalog.createTable(
+ tableIdentifier,
+ basicColumns(),
+ tableComment,
+ Collections.emptyMap(),
+ Transforms.EMPTY_TRANSFORM,
+ hashDist(),
+ null,
+ null);
+
+ try {
+ try (Connection connection =
+ DriverManager.getConnection(
+ jdbcUrl, DorisContainer.USER_NAME, DorisContainer.PASSWORD);
+ Statement statement = connection.createStatement()) {
+ statement.execute(
+ String.format("CREATE USER '%s'@'%%' IDENTIFIED BY '%s'",
userName, userPassword));
+ userCreated = true;
+ statement.execute(
+ String.format(
+ "GRANT SELECT_PRIV, ALTER_PRIV ON `%s`.* TO '%s'@'%%'",
schemaName, userName));
+ }
+
+ Map<String, String> limitedCatalogProperties = Maps.newHashMap();
+ limitedCatalogProperties.put(JdbcConfig.JDBC_URL.getKey(), jdbcUrl);
+ limitedCatalogProperties.put(JdbcConfig.JDBC_DRIVER.getKey(),
DRIVER_CLASS_NAME);
+ limitedCatalogProperties.put(JdbcConfig.USERNAME.getKey(), userName);
+ limitedCatalogProperties.put(JdbcConfig.PASSWORD.getKey(), userPassword);
+ limitedCatalog =
+ metalake.createCatalog(
+ limitedCatalogName,
+ Catalog.Type.RELATIONAL,
+ PROVIDER,
+ "Doris catalog with limited table privileges",
+ limitedCatalogProperties);
+
+ limitedCatalog
+ .asTableCatalog()
+ .alterTable(
+ tableIdentifier,
+ TableChange.addColumn(
+ new String[] {defaultedColumnName},
+ Types.VarCharType.of(255),
+ Literals.of(requestedDefaultValue,
Types.VarCharType.of(255))));
+
+ Awaitility.await()
+ .atMost(MAX_WAIT_IN_SECONDS, TimeUnit.SECONDS)
+ .pollInterval(WAIT_INTERVAL_IN_SECONDS, TimeUnit.SECONDS)
+ .untilAsserted(
+ () ->
+ assertEquals(
+ Literals.of(requestedDefaultValue,
Types.VarCharType.of(255)),
+ findColumn(rootTableCatalog.loadTable(tableIdentifier),
defaultedColumnName)
+ .defaultValue()));
+
+ String qualifiedTableName = String.format("`%s`.`%s`", schemaName,
tableIdentifier.name());
+ try (Connection connection =
+ DriverManager.getConnection(
+ jdbcUrl, DorisContainer.USER_NAME, DorisContainer.PASSWORD);
+ Statement statement = connection.createStatement()) {
+ statement.executeUpdate(
+ String.format(
+ "INSERT INTO %s (`%s`, `%s`) VALUES (301, 'limited')",
+ qualifiedTableName, colName1, colName2));
+ try (ResultSet resultSet =
+ statement.executeQuery(
+ String.format(
+ "SELECT `%s` FROM %s WHERE `%s` = 301",
+ defaultedColumnName, qualifiedTableName, colName1))) {
+ assertTrue(resultSet.next());
+ assertEquals(requestedDefaultValue, resultSet.getString(1));
+ assertFalse(resultSet.next());
+ }
+ }
+ } finally {
+ try {
+ if (limitedCatalog != null) {
+ metalake.dropCatalog(limitedCatalogName, true);
+ }
+ } finally {
+ if (userCreated) {
+ try (Connection connection =
+ DriverManager.getConnection(
+ jdbcUrl, DorisContainer.USER_NAME,
DorisContainer.PASSWORD);
+ Statement statement = connection.createStatement()) {
+ statement.execute(String.format("DROP USER '%s'@'%%'", userName));
+ }
+ }
+ }
+ }
+ }
+
@Test
void testDeleteMissingIndexIfExists() {
TableCatalog tc = catalog.asTableCatalog();
diff --git
a/catalogs/catalog-jdbc-doris/src/test/java/org/apache/gravitino/catalog/doris/integration/test/CatalogDoris4xIT.java
b/catalogs/catalog-jdbc-doris/src/test/java/org/apache/gravitino/catalog/doris/integration/test/CatalogDoris4xIT.java
index c54154e511..dff696f912 100644
---
a/catalogs/catalog-jdbc-doris/src/test/java/org/apache/gravitino/catalog/doris/integration/test/CatalogDoris4xIT.java
+++
b/catalogs/catalog-jdbc-doris/src/test/java/org/apache/gravitino/catalog/doris/integration/test/CatalogDoris4xIT.java
@@ -345,6 +345,180 @@ public class CatalogDoris4xIT extends BaseIT {
.untilAsserted(() -> assertEquals(0,
tc.loadTable(tid).index().length));
}
+ @Test
+ void testAddColumnPreservesDefaultValue() throws SQLException {
+ TableCatalog tc = catalog.asTableCatalog();
+ NameIdentifier tid =
+ NameIdentifier.of(
+ schemaName,
GravitinoITUtils.genRandomName("t_add_column_preserves_default"));
+ String defaultedColumnName = "defaulted_col";
+ String nullableDefaultColumnName = "nullable_default_col";
+ String requestedDefaultValue = "owner's a\"\"b \"value\"\\path";
+
+ tc.createTable(
+ tid,
+ basicColumns(),
+ tableComment,
+ Collections.emptyMap(),
+ Transforms.EMPTY_TRANSFORM,
+ hashDist(),
+ null,
+ null);
+
+ tc.alterTable(
+ tid,
+ TableChange.addColumn(
+ new String[] {defaultedColumnName},
+ Types.VarCharType.of(255),
+ "defaulted column",
+ TableChange.ColumnPosition.defaultPos(),
+ false,
+ false,
+ Literals.of(requestedDefaultValue, Types.VarCharType.of(255))));
+
+ String diagnosticQualifiedTableName = schemaName + "." + tid.name();
+ String[] rawDefaults = new String[3];
+ Awaitility.await()
+ .atMost(MAX_WAIT_IN_SECONDS, TimeUnit.SECONDS)
+ .pollInterval(WAIT_INTERVAL_IN_SECONDS, TimeUnit.SECONDS)
+ .untilAsserted(
+ () -> {
+ try (Connection connection =
+ DriverManager.getConnection(
+ jdbcUrl, DorisContainer.USER_NAME,
DorisContainer.PASSWORD);
+ Statement statement = connection.createStatement();
+ ResultSet columns =
+ connection
+ .getMetaData()
+ .getColumns(schemaName, null, tid.name(),
defaultedColumnName)) {
+ String jdbcColumnDefault = null;
+ while (columns.next()) {
+ if
(defaultedColumnName.equals(columns.getString("COLUMN_NAME"))) {
+ jdbcColumnDefault = columns.getString("COLUMN_DEF");
+ break;
+ }
+ }
+ assertNotNull(jdbcColumnDefault, "JDBC COLUMN_DEF was not
returned");
+ rawDefaults[0] = jdbcColumnDefault;
+
+ try (ResultSet showColumns =
+ statement.executeQuery(
+ "SHOW FULL COLUMNS FROM " +
diagnosticQualifiedTableName)) {
+ while (showColumns.next()) {
+ if
(defaultedColumnName.equals(showColumns.getString("Field"))) {
+ rawDefaults[1] = showColumns.getString("Default");
+ break;
+ }
+ }
+ }
+ assertNotNull(
+ rawDefaults[1], "SHOW FULL COLUMNS did not return the
string default");
+
+ String infoSchemaQuery =
+ String.format(
+ "SELECT COLUMN_DEFAULT FROM information_schema.columns
"
+ + "WHERE TABLE_SCHEMA = '%s' AND TABLE_NAME = '%s'
"
+ + "AND COLUMN_NAME = '%s'",
+ schemaName, tid.name(), defaultedColumnName);
+ try (ResultSet infoSchema =
statement.executeQuery(infoSchemaQuery)) {
+ assertTrue(infoSchema.next(), "information_schema.columns
row was not returned");
+ rawDefaults[2] = infoSchema.getString("COLUMN_DEFAULT");
+ }
+ }
+ });
+
+ String metadataDiagnostic =
+ String.format(
+ "JDBC COLUMN_DEF=[%s], SHOW FULL COLUMNS Default=[%s], "
+ + "information_schema.COLUMN_DEFAULT=[%s]",
+ rawDefaults[0], rawDefaults[1], rawDefaults[2]);
+ DorisContainer.LOG.info("Doris default metadata diagnostic: {}",
metadataDiagnostic);
+ String[] insertedDefaultValue = new String[1];
+ try (Connection connection =
+ DriverManager.getConnection(
+ jdbcUrl, DorisContainer.USER_NAME, DorisContainer.PASSWORD);
+ Statement statement = connection.createStatement()) {
+ statement.executeUpdate(
+ String.format(
+ "INSERT INTO %s (%s, %s) VALUES (102, 'data')",
+ diagnosticQualifiedTableName, colName1, colName2));
+ try (ResultSet resultSet =
+ statement.executeQuery(
+ String.format(
+ "SELECT %s FROM %s WHERE %s = 102",
+ defaultedColumnName, diagnosticQualifiedTableName,
colName1))) {
+ assertTrue(resultSet.next());
+ insertedDefaultValue[0] = resultSet.getString(1);
+ assertFalse(resultSet.next());
+ }
+ }
+ assertEquals(
+ requestedDefaultValue,
+ insertedDefaultValue[0],
+ metadataDiagnostic + "; inserted value=[" + insertedDefaultValue[0] +
"]");
+
+ Awaitility.await()
+ .atMost(MAX_WAIT_IN_SECONDS, TimeUnit.SECONDS)
+ .pollInterval(WAIT_INTERVAL_IN_SECONDS, TimeUnit.SECONDS)
+ .untilAsserted(
+ () -> {
+ Column addedColumn = findColumn(tc.loadTable(tid),
defaultedColumnName);
+ assertEquals(Types.VarCharType.of(255), addedColumn.dataType());
+ assertFalse(addedColumn.nullable());
+ assertEquals("defaulted column", addedColumn.comment());
+ assertEquals(
+ Literals.of(requestedDefaultValue,
Types.VarCharType.of(255)),
+ addedColumn.defaultValue(),
+ metadataDiagnostic + "; inserted value=[" +
insertedDefaultValue[0] + "]");
+ });
+
+ tc.alterTable(
+ tid,
+ TableChange.addColumn(
+ new String[] {nullableDefaultColumnName},
+ Types.IntegerType.get(),
+ "nullable default column",
+ TableChange.ColumnPosition.defaultPos(),
+ true,
+ false,
+ Literals.integerLiteral(9)));
+
+ Awaitility.await()
+ .atMost(MAX_WAIT_IN_SECONDS, TimeUnit.SECONDS)
+ .pollInterval(WAIT_INTERVAL_IN_SECONDS, TimeUnit.SECONDS)
+ .untilAsserted(
+ () -> {
+ Column addedColumn = findColumn(tc.loadTable(tid),
nullableDefaultColumnName);
+ assertEquals(Types.IntegerType.get(), addedColumn.dataType());
+ assertTrue(addedColumn.nullable());
+ assertEquals("nullable default column", addedColumn.comment());
+ assertEquals(Literals.integerLiteral(9),
addedColumn.defaultValue());
+ });
+
+ String qualifiedTableName = String.format("`%s`.`%s`", schemaName,
tid.name());
+ try (Connection connection =
+ DriverManager.getConnection(
+ jdbcUrl, DorisContainer.USER_NAME, DorisContainer.PASSWORD);
+ Statement statement = connection.createStatement()) {
+ statement.executeUpdate(
+ String.format(
+ "INSERT INTO %s (`%s`, `%s`) VALUES (101, 'data')",
+ qualifiedTableName, colName1, colName2));
+
+ try (ResultSet resultSet =
+ statement.executeQuery(
+ String.format(
+ "SELECT `%s`, `%s` FROM %s WHERE `%s` = 101",
+ defaultedColumnName, nullableDefaultColumnName,
qualifiedTableName, colName1))) {
+ assertTrue(resultSet.next());
+ assertEquals(requestedDefaultValue, resultSet.getString(1));
+ assertEquals(9, resultSet.getInt(2));
+ assertFalse(resultSet.wasNull());
+ assertFalse(resultSet.next());
+ }
+ }
+ }
+
@Test
void testDeleteMissingIndexIfExists() {
TableCatalog tc = catalog.asTableCatalog();
diff --git
a/catalogs/catalog-jdbc-doris/src/test/java/org/apache/gravitino/catalog/doris/integration/test/CatalogDorisIT.java
b/catalogs/catalog-jdbc-doris/src/test/java/org/apache/gravitino/catalog/doris/integration/test/CatalogDorisIT.java
index 8b9af56ca2..7dd97e9f5b 100644
---
a/catalogs/catalog-jdbc-doris/src/test/java/org/apache/gravitino/catalog/doris/integration/test/CatalogDorisIT.java
+++
b/catalogs/catalog-jdbc-doris/src/test/java/org/apache/gravitino/catalog/doris/integration/test/CatalogDorisIT.java
@@ -36,6 +36,8 @@ import java.sql.Connection;
import java.sql.DriverManager;
import java.sql.PreparedStatement;
import java.sql.ResultSet;
+import java.sql.SQLException;
+import java.sql.Statement;
import java.time.LocalDate;
import java.time.LocalDateTime;
import java.util.Arrays;
@@ -720,6 +722,195 @@ public class CatalogDorisIT extends BaseIT {
"true",
tableCatalog.loadTable(tableIdentifier).properties().get("in_memory")));
}
+ @Test
+ void testAddColumnPreservesDefaultValue() throws SQLException {
+ String defaultedColumnName = "defaulted_col";
+ String nullableDefaultColumnName = "nullable_default_col";
+ String requestedDefaultValue = "owner's a\"\"b \"value\"\\path";
+ NameIdentifier tableIdentifier =
+ NameIdentifier.of(
+ schemaName,
GravitinoITUtils.genRandomName("test_add_column_preserves_default"));
+ TableCatalog tableCatalog = catalog.asTableCatalog();
+ tableCatalog.createTable(
+ tableIdentifier,
+ createColumns(),
+ table_comment,
+ createTableProperties(),
+ Transforms.EMPTY_TRANSFORM,
+ createDistribution(),
+ null,
+ Indexes.EMPTY_INDEXES);
+
+ tableCatalog.alterTable(
+ tableIdentifier,
+ TableChange.addColumn(
+ new String[] {defaultedColumnName},
+ Types.VarCharType.of(255),
+ "defaulted column",
+ TableChange.ColumnPosition.defaultPos(),
+ false,
+ false,
+ Literals.of(requestedDefaultValue, Types.VarCharType.of(255))));
+
+ String diagnosticQualifiedTableName = schemaName + "." +
tableIdentifier.name();
+ String[] rawDefaults = new String[3];
+ Awaitility.await()
+ .atMost(MAX_WAIT_IN_SECONDS, TimeUnit.SECONDS)
+ .pollInterval(WAIT_INTERVAL_IN_SECONDS, TimeUnit.SECONDS)
+ .untilAsserted(
+ () -> {
+ try (Connection connection =
+ DriverManager.getConnection(
+ jdbcUrl, DorisContainer.USER_NAME,
DorisContainer.PASSWORD);
+ Statement statement = connection.createStatement();
+ ResultSet columns =
+ connection
+ .getMetaData()
+ .getColumns(
+ schemaName, null, tableIdentifier.name(),
defaultedColumnName)) {
+ String jdbcColumnDefault = null;
+ while (columns.next()) {
+ if
(defaultedColumnName.equals(columns.getString("COLUMN_NAME"))) {
+ jdbcColumnDefault = columns.getString("COLUMN_DEF");
+ break;
+ }
+ }
+ Assertions.assertNotNull(jdbcColumnDefault, "JDBC COLUMN_DEF
was not returned");
+ rawDefaults[0] = jdbcColumnDefault;
+
+ try (ResultSet showColumns =
+ statement.executeQuery(
+ "SHOW FULL COLUMNS FROM " +
diagnosticQualifiedTableName)) {
+ while (showColumns.next()) {
+ if
(defaultedColumnName.equals(showColumns.getString("Field"))) {
+ rawDefaults[1] = showColumns.getString("Default");
+ break;
+ }
+ }
+ }
+ Assertions.assertNotNull(
+ rawDefaults[1], "SHOW FULL COLUMNS did not return the
string default");
+
+ String infoSchemaQuery =
+ String.format(
+ "SELECT COLUMN_DEFAULT FROM information_schema.columns
"
+ + "WHERE TABLE_SCHEMA = '%s' AND TABLE_NAME = '%s'
"
+ + "AND COLUMN_NAME = '%s'",
+ schemaName, tableIdentifier.name(),
defaultedColumnName);
+ try (ResultSet infoSchema =
statement.executeQuery(infoSchemaQuery)) {
+ Assertions.assertTrue(
+ infoSchema.next(), "information_schema.columns row was
not returned");
+ rawDefaults[2] = infoSchema.getString("COLUMN_DEFAULT");
+ }
+ }
+ });
+
+ String metadataDiagnostic =
+ String.format(
+ "JDBC COLUMN_DEF=[%s], SHOW FULL COLUMNS Default=[%s], "
+ + "information_schema.COLUMN_DEFAULT=[%s]",
+ rawDefaults[0], rawDefaults[1], rawDefaults[2]);
+ DorisContainer.LOG.info("Doris default metadata diagnostic: {}",
metadataDiagnostic);
+ String[] insertedDefaultValue = new String[1];
+ try (Connection connection =
+ DriverManager.getConnection(
+ jdbcUrl, DorisContainer.USER_NAME, DorisContainer.PASSWORD);
+ Statement statement = connection.createStatement()) {
+ statement.executeUpdate(
+ String.format(
+ "INSERT INTO %s (%s, %s, %s, %s) " + "VALUES (102, 'data-1',
'data-2', '2024-01-01')",
+ diagnosticQualifiedTableName,
+ DORIS_COL_NAME1,
+ DORIS_COL_NAME2,
+ DORIS_COL_NAME3,
+ DORIS_COL_NAME4));
+ try (ResultSet resultSet =
+ statement.executeQuery(
+ String.format(
+ "SELECT %s FROM %s WHERE %s = 102",
+ defaultedColumnName, diagnosticQualifiedTableName,
DORIS_COL_NAME1))) {
+ Assertions.assertTrue(resultSet.next());
+ insertedDefaultValue[0] = resultSet.getString(1);
+ Assertions.assertFalse(resultSet.next());
+ }
+ }
+ Assertions.assertEquals(
+ requestedDefaultValue,
+ insertedDefaultValue[0],
+ metadataDiagnostic + "; inserted value=[" + insertedDefaultValue[0] +
"]");
+
+ Awaitility.await()
+ .atMost(MAX_WAIT_IN_SECONDS, TimeUnit.SECONDS)
+ .pollInterval(WAIT_INTERVAL_IN_SECONDS, TimeUnit.SECONDS)
+ .untilAsserted(
+ () -> {
+ Column addedColumn =
+ findColumn(tableCatalog.loadTable(tableIdentifier),
defaultedColumnName);
+ Assertions.assertEquals(Types.VarCharType.of(255),
addedColumn.dataType());
+ Assertions.assertFalse(addedColumn.nullable());
+ Assertions.assertEquals("defaulted column",
addedColumn.comment());
+ Assertions.assertEquals(
+ Literals.of(requestedDefaultValue,
Types.VarCharType.of(255)),
+ addedColumn.defaultValue(),
+ metadataDiagnostic + "; inserted value=[" +
insertedDefaultValue[0] + "]");
+ });
+
+ tableCatalog.alterTable(
+ tableIdentifier,
+ TableChange.addColumn(
+ new String[] {nullableDefaultColumnName},
+ Types.IntegerType.get(),
+ "nullable default column",
+ TableChange.ColumnPosition.defaultPos(),
+ true,
+ false,
+ Literals.integerLiteral(9)));
+
+ Awaitility.await()
+ .atMost(MAX_WAIT_IN_SECONDS, TimeUnit.SECONDS)
+ .pollInterval(WAIT_INTERVAL_IN_SECONDS, TimeUnit.SECONDS)
+ .untilAsserted(
+ () -> {
+ Column addedColumn =
+ findColumn(tableCatalog.loadTable(tableIdentifier),
nullableDefaultColumnName);
+ Assertions.assertEquals(Types.IntegerType.get(),
addedColumn.dataType());
+ Assertions.assertTrue(addedColumn.nullable());
+ Assertions.assertEquals("nullable default column",
addedColumn.comment());
+ Assertions.assertEquals(Literals.integerLiteral(9),
addedColumn.defaultValue());
+ });
+
+ String qualifiedTableName = String.format("`%s`.`%s`", schemaName,
tableIdentifier.name());
+ try (Connection connection =
+ DriverManager.getConnection(
+ jdbcUrl, DorisContainer.USER_NAME, DorisContainer.PASSWORD);
+ Statement statement = connection.createStatement()) {
+ statement.executeUpdate(
+ String.format(
+ "INSERT INTO %s (`%s`, `%s`, `%s`, `%s`) "
+ + "VALUES (101, 'data-1', 'data-2', '2024-01-01')",
+ qualifiedTableName,
+ DORIS_COL_NAME1,
+ DORIS_COL_NAME2,
+ DORIS_COL_NAME3,
+ DORIS_COL_NAME4));
+
+ try (ResultSet resultSet =
+ statement.executeQuery(
+ String.format(
+ "SELECT `%s`, `%s` FROM %s WHERE `%s` = 101",
+ defaultedColumnName,
+ nullableDefaultColumnName,
+ qualifiedTableName,
+ DORIS_COL_NAME1))) {
+ Assertions.assertTrue(resultSet.next());
+ Assertions.assertEquals(requestedDefaultValue, resultSet.getString(1));
+ Assertions.assertEquals(9, resultSet.getInt(2));
+ Assertions.assertFalse(resultSet.wasNull());
+ Assertions.assertFalse(resultSet.next());
+ }
+ }
+ }
+
@Test
void testDorisIndex() {
String tableName = GravitinoITUtils.genRandomName("test_add_index");
@@ -1355,4 +1546,11 @@ public class CatalogDorisIT extends BaseIT {
assertPartition(Partitions.list("p1", p1Values, Collections.emptyMap()),
partitions.get("p1"));
assertPartition(Partitions.list("p2", p2Values, Collections.emptyMap()),
partitions.get("p2"));
}
+
+ private Column findColumn(Table table, String columnName) {
+ return Arrays.stream(table.columns())
+ .filter(column -> column.name().equals(columnName))
+ .findFirst()
+ .orElseThrow(() -> new AssertionError("Column not found: " +
columnName));
+ }
}
diff --git
a/catalogs/catalog-jdbc-doris/src/test/java/org/apache/gravitino/catalog/doris/operation/TestDorisTableOperationsSqlGeneration.java
b/catalogs/catalog-jdbc-doris/src/test/java/org/apache/gravitino/catalog/doris/operation/TestDorisTableOperationsSqlGeneration.java
index 164e9bb6b1..cf8fe7f781 100644
---
a/catalogs/catalog-jdbc-doris/src/test/java/org/apache/gravitino/catalog/doris/operation/TestDorisTableOperationsSqlGeneration.java
+++
b/catalogs/catalog-jdbc-doris/src/test/java/org/apache/gravitino/catalog/doris/operation/TestDorisTableOperationsSqlGeneration.java
@@ -20,6 +20,8 @@ package org.apache.gravitino.catalog.doris.operation;
import static
org.apache.gravitino.catalog.doris.DorisTablePropertiesMetadata.REPLICATION_ALLOCATION;
import static
org.apache.gravitino.catalog.doris.DorisTablePropertiesMetadata.REPLICATION_FACTOR;
+import static org.apache.gravitino.rel.Column.DEFAULT_VALUE_NOT_SET;
+import static
org.apache.gravitino.rel.Column.DEFAULT_VALUE_OF_CURRENT_TIMESTAMP;
import java.sql.Connection;
import java.sql.PreparedStatement;
@@ -33,15 +35,16 @@ import java.util.LinkedHashMap;
import java.util.List;
import java.util.Map;
import javax.sql.DataSource;
+import
org.apache.gravitino.catalog.doris.converter.DorisColumnDefaultValueConverter;
import org.apache.gravitino.catalog.doris.converter.DorisTypeConverter;
import org.apache.gravitino.catalog.jdbc.JdbcColumn;
import org.apache.gravitino.catalog.jdbc.JdbcTable;
-import
org.apache.gravitino.catalog.jdbc.converter.JdbcColumnDefaultValueConverter;
import org.apache.gravitino.catalog.jdbc.converter.JdbcExceptionConverter;
import org.apache.gravitino.rel.TableChange;
import org.apache.gravitino.rel.expressions.NamedReference;
import org.apache.gravitino.rel.expressions.distributions.Distribution;
import org.apache.gravitino.rel.expressions.distributions.Distributions;
+import org.apache.gravitino.rel.expressions.literals.Literal;
import org.apache.gravitino.rel.expressions.literals.Literals;
import org.apache.gravitino.rel.expressions.transforms.Transforms;
import org.apache.gravitino.rel.indexes.Index;
@@ -58,26 +61,15 @@ public class TestDorisTableOperationsSqlGeneration {
JdbcTable.builder().withName("test_table").withIndexes(Indexes.EMPTY_INDEXES).build();
public TestableDorisTableOperations() {
+ this("doris-3.0.6.2-rc01-910c4249c5");
+ }
+
+ public TestableDorisTableOperations(String dorisVersion) {
super.exceptionMapper = new JdbcExceptionConverter();
super.typeConverter = new DorisTypeConverter();
- super.columnDefaultValueConverter = new
JdbcColumnDefaultValueConverter();
+ super.columnDefaultValueConverter = new
DorisColumnDefaultValueConverter();
try {
- // Set up a mock DataSource for validateAutoIncrementVersion
- // Uses SHOW FRONTENDS to get the actual Doris version (not MySQL
protocol version)
- DataSource mockDataSource = Mockito.mock(DataSource.class);
- Connection mockConnection = Mockito.mock(Connection.class);
- Statement mockStatement = Mockito.mock(Statement.class);
- ResultSet mockResultSet = Mockito.mock(ResultSet.class);
- ResultSetMetaData mockMetaData = Mockito.mock(ResultSetMetaData.class);
-
Mockito.when(mockDataSource.getConnection()).thenReturn(mockConnection);
-
Mockito.when(mockConnection.createStatement()).thenReturn(mockStatement);
- Mockito.when(mockStatement.executeQuery("SHOW
FRONTENDS")).thenReturn(mockResultSet);
- Mockito.when(mockResultSet.getMetaData()).thenReturn(mockMetaData);
- Mockito.when(mockMetaData.getColumnCount()).thenReturn(1);
- Mockito.when(mockMetaData.getColumnLabel(1)).thenReturn("Version");
- Mockito.when(mockResultSet.next()).thenReturn(true);
-
Mockito.when(mockResultSet.getString(1)).thenReturn("doris-3.0.6.2-rc01-910c4249c5");
- super.dataSource = mockDataSource;
+ super.dataSource = mockVersionDataSource(dorisVersion);
} catch (Exception e) {
throw new RuntimeException(e);
}
@@ -156,10 +148,10 @@ public class TestDorisTableOperationsSqlGeneration {
.appendNecessaryProperties(Mockito.anyMap());
String sql = mockOps.createTableSql(tableName, new JdbcColumn[] {col1},
distribution);
- JdbcColumnDefaultValueConverter converter = new
JdbcColumnDefaultValueConverter();
+ DorisColumnDefaultValueConverter converter = new
DorisColumnDefaultValueConverter();
Assertions.assertTrue(
sql.contains("DEFAULT " +
converter.fromGravitino(col1.defaultValue())),
- "Should contain DEFAULT '' but was: " + sql);
+ "Should contain an empty DEFAULT value but was: " + sql);
}
@Test
@@ -182,7 +174,7 @@ public class TestDorisTableOperationsSqlGeneration {
.appendNecessaryProperties(Mockito.anyMap());
String sql = mockOps.createTableSql(tableName, new JdbcColumn[] {col1},
distribution);
- JdbcColumnDefaultValueConverter converter = new
JdbcColumnDefaultValueConverter();
+ DorisColumnDefaultValueConverter converter = new
DorisColumnDefaultValueConverter();
Assertions.assertTrue(
sql.contains("DEFAULT " +
converter.fromGravitino(col1.defaultValue())),
"Should contain DEFAULT value but was: " + sql);
@@ -234,6 +226,258 @@ public class TestDorisTableOperationsSqlGeneration {
Assertions.assertTrue(alterSql.contains("COMMENT 'owner\\\\''s
\"comment\"; --'"), alterSql);
}
+ @Test
+ public void testAddColumnDefaultValuesInGeneratedSql() {
+ TestableDorisTableOperations ops = new TestableDorisTableOperations();
+
+ String numericDefaultSql =
+ ops.alterTableSql(
+ "test_table",
+ TableChange.addColumn(
+ new String[] {"col2"},
+ Types.IntegerType.get(),
+ "comment",
+ TableChange.ColumnPosition.after("col1"),
+ false,
+ false,
+ Literals.integerLiteral(7)));
+ String nullDefaultSql =
+ ops.alterTableSql(
+ "test_table",
+ TableChange.addColumn(
+ new String[] {"col2"},
+ Types.IntegerType.get(),
+ null,
+ TableChange.ColumnPosition.defaultPos(),
+ true,
+ false,
+ Literals.NULL));
+ String currentTimestampSql =
+ ops.alterTableSql(
+ "test_table",
+ TableChange.addColumn(
+ new String[] {"created_at"},
+ Types.TimestampType.withoutTimeZone(),
+ DEFAULT_VALUE_OF_CURRENT_TIMESTAMP));
+ String unsetDefaultSql =
+ ops.alterTableSql(
+ "test_table",
+ TableChange.addColumn(
+ new String[] {"col2"},
+ Types.IntegerType.get(),
+ null,
+ TableChange.ColumnPosition.defaultPos(),
+ true,
+ false,
+ DEFAULT_VALUE_NOT_SET));
+
+ Assertions.assertEquals(
+ "ALTER TABLE `test_table`\n"
+ + "ADD COLUMN `col2` int NOT NULL DEFAULT \"7\" COMMENT 'comment'
AFTER `col1`;",
+ numericDefaultSql);
+ Assertions.assertEquals(
+ "ALTER TABLE `test_table`\nADD COLUMN `col2` int DEFAULT NULL ;",
nullDefaultSql);
+ Assertions.assertEquals(
+ "ALTER TABLE `test_table`\n"
+ + "ADD COLUMN `created_at` datetime DEFAULT CURRENT_TIMESTAMP ;",
+ currentTimestampSql);
+ Assertions.assertEquals("ALTER TABLE `test_table`\nADD COLUMN `col2` int
;", unsetDefaultSql);
+ }
+
+ @Test
+ public void testAddColumnEscapesStringDefaultValue() {
+ TestableDorisTableOperations ops = new TestableDorisTableOperations();
+ Literal<?> defaultValue = Literals.of("owner's \"value\"\\path",
Types.VarCharType.of(255));
+
+ String sql =
+ ops.alterTableSql(
+ "test_table",
+ TableChange.addColumn(new String[] {"col2"},
Types.VarCharType.of(255), defaultValue));
+
+ Assertions.assertEquals(
+ "ALTER TABLE `test_table`\n"
+ + "ADD COLUMN `col2` varchar(255) DEFAULT "
+ + new DorisColumnDefaultValueConverter()
+ .fromGravitinoForAddColumn(defaultValue, true, true)
+ + " ;",
+ sql);
+ }
+
+ @Test
+ public void testAddColumnKeepsStandardBackslashEscapingOutsideDoris3() {
+ for (String version : new String[] {"doris-1.2.2-release",
"doris-4.0.6-release-a851eab4"}) {
+ TestableDorisTableOperations ops = new
TestableDorisTableOperations(version);
+ Literal<?> defaultValue = Literals.of("owner's \"value\"\\path",
Types.VarCharType.of(255));
+ String sql =
+ ops.alterTableSql(
+ "test_table",
+ TableChange.addColumn(
+ new String[] {"col2"}, Types.VarCharType.of(255),
defaultValue));
+
+ boolean useSingleQuoteDelimiter = version.startsWith("doris-1.2.");
+ Assertions.assertEquals(
+ "ALTER TABLE `test_table`\n"
+ + "ADD COLUMN `col2` varchar(255) DEFAULT "
+ + new DorisColumnDefaultValueConverter()
+ .fromGravitinoForAddColumn(defaultValue, false,
useSingleQuoteDelimiter)
+ + " ;",
+ sql);
+ }
+ }
+
+ @Test
+ public void testAddColumnDoesNotQueryVersionWithoutBackslashes() throws
Exception {
+ TestableDorisTableOperations ops = new TestableDorisTableOperations();
+ DataSource unavailableDataSource = Mockito.mock(DataSource.class);
+ Mockito.when(unavailableDataSource.getConnection())
+ .thenThrow(new SQLException("Version query should not run"));
+ ops.setDataSource(unavailableDataSource);
+
+ String sql =
+ ops.alterTableSql(
+ "test_table",
+ TableChange.addColumn(
+ new String[] {"col2"},
+ Types.VarCharType.of(255),
+ Literals.of("owner's value", Types.VarCharType.of(255))));
+
+ Assertions.assertTrue(sql.contains("DEFAULT \"owner's value\""), sql);
+ }
+
+ @Test
+ public void testAddColumnQueriesVersionForAdjacentDoubleQuotes() throws
Exception {
+ TestableDorisTableOperations ops = new TestableDorisTableOperations();
+ DataSource versionDataSource =
mockVersionDataSource("doris-1.2.7-1-Unknown");
+ ops.setDataSource(versionDataSource);
+
+ String sql =
+ ops.alterTableSql(
+ "test_table",
+ TableChange.addColumn(
+ new String[] {"col2"},
+ Types.VarCharType.of(255),
+ Literals.of("before\"\"after", Types.VarCharType.of(255))));
+
+ Assertions.assertTrue(sql.contains("DEFAULT 'before\"\"after'"), sql);
+ Mockito.verify(versionDataSource).getConnection();
+ }
+
+ @Test
+ public void testAddColumnsQueryVersionOncePerAlterRequest() throws Exception
{
+ TestableDorisTableOperations ops = new TestableDorisTableOperations();
+ DataSource versionDataSource =
mockVersionDataSource("doris-3.0.6.2-rc01-910c4249c5");
+ ops.setDataSource(versionDataSource);
+
+ ops.alterTableSql(
+ "test_table",
+ TableChange.addColumn(
+ new String[] {"col2"},
+ Types.VarCharType.of(255),
+ Literals.of("first\\value", Types.VarCharType.of(255))),
+ TableChange.addColumn(
+ new String[] {"col3"},
+ Types.VarCharType.of(255),
+ Literals.of("second\\value", Types.VarCharType.of(255))));
+
+ Mockito.verify(versionDataSource, Mockito.times(1)).getConnection();
+ }
+
+ @Test
+ public void
testAddColumnFallsBackToShowFrontendsWhenTableFunctionIsUnavailable()
+ throws Exception {
+ TestableDorisTableOperations ops = new TestableDorisTableOperations();
+ DataSource dataSource = Mockito.mock(DataSource.class);
+ Connection connection = Mockito.mock(Connection.class);
+ Statement statement = Mockito.mock(Statement.class);
+ ResultSet resultSet = Mockito.mock(ResultSet.class);
+ ResultSetMetaData metadata = Mockito.mock(ResultSetMetaData.class);
+ Mockito.when(dataSource.getConnection()).thenReturn(connection);
+ Mockito.when(connection.createStatement()).thenReturn(statement);
+ Mockito.when(statement.executeQuery("SELECT Version FROM FRONTENDS()"))
+ .thenThrow(new SQLException("FRONTENDS table-valued function is
unavailable"));
+ Mockito.when(statement.executeQuery("SHOW
FRONTENDS")).thenReturn(resultSet);
+ Mockito.when(resultSet.getMetaData()).thenReturn(metadata);
+ Mockito.when(metadata.getColumnCount()).thenReturn(1);
+ Mockito.when(metadata.getColumnLabel(1)).thenReturn("Version");
+ Mockito.when(resultSet.next()).thenReturn(true);
+ Mockito.when(resultSet.getString(1)).thenReturn("doris-1.2.7-1-Unknown");
+ ops.setDataSource(dataSource);
+
+ String sql =
+ ops.alterTableSql(
+ "test_table",
+ TableChange.addColumn(
+ new String[] {"col2"},
+ Types.VarCharType.of(255),
+ Literals.of("value\\path", Types.VarCharType.of(255))));
+
+ Assertions.assertTrue(
+ sql.contains(
+ new DorisColumnDefaultValueConverter()
+ .fromGravitinoForAddColumn(
+ Literals.of("value\\path", Types.VarCharType.of(255)),
false, true)),
+ sql);
+ Mockito.verify(statement).executeQuery("SELECT Version FROM FRONTENDS()");
+ Mockito.verify(statement).executeQuery("SHOW FRONTENDS");
+ }
+
+ @Test
+ public void testAddColumnFailsClosedWhenBothVersionQueriesFail() throws
Exception {
+ TestableDorisTableOperations ops = new TestableDorisTableOperations();
+ DataSource dataSource = Mockito.mock(DataSource.class);
+ Connection connection = Mockito.mock(Connection.class);
+ Statement statement = Mockito.mock(Statement.class);
+ SQLException frontendsFailure = new SQLException("FRONTENDS() denied");
+ SQLException showFailure = new SQLException("SHOW FRONTENDS denied");
+ Mockito.when(dataSource.getConnection()).thenReturn(connection);
+ Mockito.when(connection.createStatement()).thenReturn(statement);
+ Mockito.when(statement.executeQuery("SELECT Version FROM FRONTENDS()"))
+ .thenThrow(frontendsFailure);
+ Mockito.when(statement.executeQuery("SHOW
FRONTENDS")).thenThrow(showFailure);
+ ops.setDataSource(dataSource);
+
+ UnsupportedOperationException exception =
+ Assertions.assertThrows(
+ UnsupportedOperationException.class,
+ () ->
+ ops.alterTableSql(
+ "test_table",
+ TableChange.addColumn(
+ new String[] {"col2"},
+ Types.VarCharType.of(255),
+ Literals.of("value\\path",
Types.VarCharType.of(255)))));
+
+ Assertions.assertTrue(
+ exception.getMessage().contains("ADD COLUMN default literal
compatibility check"),
+ exception.getMessage());
+ Assertions.assertSame(showFailure, exception.getCause());
+ Assertions.assertArrayEquals(new Throwable[] {frontendsFailure},
showFailure.getSuppressed());
+ }
+
+ @Test
+ public void testAddColumnFailsClosedWhenVersionQueryFails() throws Exception
{
+ TestableDorisTableOperations ops = new TestableDorisTableOperations();
+ DataSource unavailableDataSource = Mockito.mock(DataSource.class);
+ Mockito.when(unavailableDataSource.getConnection())
+ .thenThrow(new SQLException("SHOW FRONTENDS denied"));
+ ops.setDataSource(unavailableDataSource);
+
+ UnsupportedOperationException exception =
+ Assertions.assertThrows(
+ UnsupportedOperationException.class,
+ () ->
+ ops.alterTableSql(
+ "test_table",
+ TableChange.addColumn(
+ new String[] {"col2"},
+ Types.VarCharType.of(255),
+ Literals.of("owner's\\value",
Types.VarCharType.of(255)))));
+
+ Assertions.assertTrue(
+ exception.getMessage().contains("ADD COLUMN default literal
compatibility check"),
+ exception.getMessage());
+ }
+
@Test
public void testCreateTableWithPrimaryKeyIndex() {
TestableDorisTableOperations ops = new TestableDorisTableOperations();
@@ -1011,4 +1255,18 @@ public class TestDorisTableOperationsSqlGeneration {
return dataSource;
}
+
+ private static DataSource mockVersionDataSource(String dorisVersion) throws
Exception {
+ // FRONTENDS() exposes the Doris version instead of the MySQL protocol
version.
+ DataSource dataSource = Mockito.mock(DataSource.class);
+ Connection connection = Mockito.mock(Connection.class);
+ Statement statement = Mockito.mock(Statement.class);
+ ResultSet resultSet = Mockito.mock(ResultSet.class);
+ Mockito.when(dataSource.getConnection()).thenReturn(connection);
+ Mockito.when(connection.createStatement()).thenReturn(statement);
+ Mockito.when(statement.executeQuery("SELECT Version FROM
FRONTENDS()")).thenReturn(resultSet);
+ Mockito.when(resultSet.next()).thenReturn(true);
+ Mockito.when(resultSet.getString(1)).thenReturn(dorisVersion);
+ return dataSource;
+ }
}
