This is an automated email from the ASF dual-hosted git repository.
strongduanmu pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/shardingsphere.git
The following commit(s) were added to refs/heads/master by this push:
new 499c89a807e Refactor SQLRewriterIT to test more sql rewrite case
(#38700)
499c89a807e is described below
commit 499c89a807e43b147ccf3f9ee77ace8ca086e39e
Author: Zhengqiang Duan <[email protected]>
AuthorDate: Mon May 18 12:15:28 2026 +0800
Refactor SQLRewriterIT to test more sql rewrite case (#38700)
---
AGENTS.md | 6 +-
.../test/it/rewriter/engine/SQLRewriterIT.java | 98 ++++++++---
.../mocker/DialectStorageUnitMetaDataMocker.java} | 29 ++--
.../type/MySQLStorageUnitMetaDataMocker.java | 110 ++++++++++++
.../SQLRewriteEngineTestParametersBuilder.java | 184 +++++++++++++++++++--
.../it/rewriter/entity/RewriteAssertionEntity.java | 3 +
.../it/rewriter/entity/RewriteInputEntity.java | 3 +
.../it/rewriter/entity/RewriteOutputEntity.java | 3 +
....engine.mocker.DialectStorageUnitMetaDataMocker | 18 ++
9 files changed, 403 insertions(+), 51 deletions(-)
diff --git a/AGENTS.md b/AGENTS.md
index b14093cf1fe..1be4830d797 100644
--- a/AGENTS.md
+++ b/AGENTS.md
@@ -32,6 +32,7 @@ This guide is written **for AI coding agents only**. Follow
it literally; improv
- **Test-Driven**: design for testability, ensure unit-test coverage, and keep
background unit tests under 60s to avoid job stalls.
- **Quality Assurance**: run static checks, formatting, and code reviews.
- **Checkstyle Gate**: do not hand off code with Checkstyle/Spotless
failures—run the relevant module check locally and fix before completion.
+- **Formatting Gate**: after code changes, format only with `./mvnw
spotless:apply -Pcheck -T1C`, then check style with `./mvnw checkstyle:check
-Pcheck -T1C`; do not use any other formatting method.
- **Continuous Verification**: rely on automated tests and integration
validation.
- **Test Naming Simplicity**: keep test names concise and scenario-focused
(avoid “ReturnsXXX”/overly wordy or AI-like phrasing); describe the scenario
directly.
- **Coverage Discipline**: follow the dedicated coverage & branch checklist
before coding when coverage targets are stated.
@@ -185,9 +186,10 @@ Always state which topology, registry, and engine versions
(e.g., MySQL 5.7 vs 8
- **Success recipe:** explain why the change exists, cite the affected
data-flow step, keep public APIs backward compatible, and record defaults/knobs
alongside code changes.
## Verification & Commands
-- Core commands: `./mvnw clean install -B -T1C -Pcheck` (full build), `./mvnw
test -pl <module>[-am]` (scoped unit tests), `./mvnw spotless:apply -Pcheck
[-pl <module>]` (format), `./mvnw -pl <module> -DskipITs -Dspotless.skip=true
-Dtest=ClassName test` (fast verification), `./mvnw -pl proxy -am -DskipTests
package` (proxy packaging/perf smoke).
+- Core commands: `./mvnw clean install -B -T1C -Pcheck` (full build), `./mvnw
test -pl <module>[-am]` (scoped unit tests), `./mvnw -pl <module> -DskipITs
-Dspotless.skip=true -Dtest=ClassName test` (fast verification), `./mvnw -pl
proxy -am -DskipTests package` (proxy packaging/perf smoke).
- Coverage: when tests change or targets demand it, run `./mvnw test
jacoco:check@jacoco-check -Pcoverage-check` or scoped `-pl <module> -am
-Djacoco.skip=false test jacoco:report`; pair with the Coverage & Branch
Checklist.
-- Style: `./mvnw checkstyle:check -Pcheck` (scoped with `-pl <module> -am
-Pcheck` when possible) unless told otherwise.
+- Format: after code changes, run `./mvnw spotless:apply -Pcheck -T1C`; do not
use any other formatting method.
+- Style: after code changes and formatting, run `./mvnw checkstyle:check
-Pcheck -T1C`.
- Scoped defaults: prefer module-scoped runs over whole-repo builds; include
`-Dsurefire.failIfNoSpecifiedTests=false` when targeting specific tests.
- Testing ground rules: JUnit 5 + Mockito, `ClassNameTest` naming,
Arrange–Act–Assert, mock external systems/time/network, reset static caches,
and reuse swappers/helpers for complex configs.
- API bans: if a user forbids a tool/assertion, add it to the plan, avoid it
during implementation, and cite verification searches (e.g., `rg assertEquals`)
in the final report.
diff --git
a/test/it/rewriter/src/test/java/org/apache/shardingsphere/test/it/rewriter/engine/SQLRewriterIT.java
b/test/it/rewriter/src/test/java/org/apache/shardingsphere/test/it/rewriter/engine/SQLRewriterIT.java
index 9b393c94e2c..bb7630ee164 100644
---
a/test/it/rewriter/src/test/java/org/apache/shardingsphere/test/it/rewriter/engine/SQLRewriterIT.java
+++
b/test/it/rewriter/src/test/java/org/apache/shardingsphere/test/it/rewriter/engine/SQLRewriterIT.java
@@ -19,6 +19,8 @@ package org.apache.shardingsphere.test.it.rewriter.engine;
import com.google.common.base.Preconditions;
import org.apache.shardingsphere.database.connector.core.DefaultDatabase;
+import
org.apache.shardingsphere.database.connector.core.jdbcurl.parser.ConnectionProperties;
+import
org.apache.shardingsphere.database.connector.core.spi.DatabaseTypedSPILoader;
import org.apache.shardingsphere.database.connector.core.type.DatabaseType;
import
org.apache.shardingsphere.database.connector.core.type.DatabaseTypeRegistry;
import org.apache.shardingsphere.infra.binder.context.aware.ParameterAware;
@@ -29,11 +31,15 @@ import
org.apache.shardingsphere.infra.binder.engine.SQLBindEngine;
import org.apache.shardingsphere.infra.config.database.DatabaseConfiguration;
import
org.apache.shardingsphere.infra.config.database.impl.DataSourceProvidedDatabaseConfiguration;
import org.apache.shardingsphere.infra.config.props.ConfigurationProperties;
+import org.apache.shardingsphere.infra.config.props.ConfigurationPropertyKey;
+import
org.apache.shardingsphere.infra.config.props.MetadataIdentifierCaseSensitivity;
import org.apache.shardingsphere.infra.hint.HintValueContext;
import org.apache.shardingsphere.infra.hint.SQLHintUtils;
+import org.apache.shardingsphere.infra.instance.ComputeNodeInstanceContext;
import org.apache.shardingsphere.infra.metadata.ShardingSphereMetaData;
import
org.apache.shardingsphere.infra.metadata.database.ShardingSphereDatabase;
import
org.apache.shardingsphere.infra.metadata.database.resource.ResourceMetaData;
+import
org.apache.shardingsphere.infra.metadata.database.resource.unit.StorageUnit;
import org.apache.shardingsphere.infra.metadata.database.rule.RuleMetaData;
import
org.apache.shardingsphere.infra.metadata.database.schema.model.ShardingSphereSchema;
import org.apache.shardingsphere.infra.parser.SQLParserEngine;
@@ -57,6 +63,7 @@ import
org.apache.shardingsphere.infra.yaml.config.swapper.rule.YamlRuleConfigur
import org.apache.shardingsphere.parser.rule.SQLParserRule;
import
org.apache.shardingsphere.parser.rule.builder.DefaultSQLParserRuleConfigurationBuilder;
import
org.apache.shardingsphere.sql.parser.statement.core.statement.SQLStatement;
+import
org.apache.shardingsphere.test.it.rewriter.engine.mocker.DialectStorageUnitMetaDataMocker;
import
org.apache.shardingsphere.test.it.rewriter.engine.parameter.SQLRewriteEngineTestParameters;
import
org.apache.shardingsphere.test.it.rewriter.engine.parameter.SQLRewriteEngineTestParametersBuilder;
import org.junit.jupiter.api.extension.ExtensionContext;
@@ -66,24 +73,33 @@ import org.junit.jupiter.params.provider.ArgumentsProvider;
import org.junit.jupiter.params.provider.ArgumentsSource;
import org.junit.jupiter.params.support.ParameterDeclarations;
+import javax.sql.DataSource;
import java.io.File;
import java.io.IOException;
import java.net.URL;
+import java.sql.Connection;
+import java.sql.DatabaseMetaData;
+import java.sql.SQLException;
import java.util.Collection;
import java.util.Collections;
+import java.util.LinkedHashMap;
+import java.util.Map;
+import java.util.Map.Entry;
import java.util.Objects;
import java.util.Properties;
import java.util.stream.Stream;
-import static org.hamcrest.Matchers.is;
import static org.hamcrest.MatcherAssert.assertThat;
+import static org.hamcrest.Matchers.is;
+import static org.mockito.Mockito.RETURNS_DEEP_STUBS;
import static org.mockito.Mockito.mock;
+import static org.mockito.Mockito.when;
public abstract class SQLRewriterIT {
@ParameterizedTest(name = "{0}")
@ArgumentsSource(TestCaseArgumentsProvider.class)
- void assertRewrite(final SQLRewriteEngineTestParameters testParams) throws
IOException {
+ void assertRewrite(final SQLRewriteEngineTestParameters testParams) throws
IOException, SQLException {
Collection<SQLRewriteUnit> actual = createSQLRewriteUnits(testParams);
assertThat(actual.size(), is(testParams.getOutputSQLs().size()));
int count = 0;
@@ -97,45 +113,76 @@ public abstract class SQLRewriterIT {
}
}
- private Collection<SQLRewriteUnit> createSQLRewriteUnits(final
SQLRewriteEngineTestParameters testParams) throws IOException {
+ private Collection<SQLRewriteUnit> createSQLRewriteUnits(final
SQLRewriteEngineTestParameters testParams) throws IOException, SQLException {
YamlRootConfiguration rootConfig = loadRootConfiguration(testParams);
- DatabaseConfiguration databaseConfig = new
DataSourceProvidedDatabaseConfiguration(
- new
YamlDataSourceConfigurationSwapper().swapToDataSources(rootConfig.getDataSources()),
new
YamlRuleConfigurationSwapperEngine().swapToRuleConfigurations(rootConfig.getRules()));
DatabaseType databaseType =
TypedSPILoader.getService(DatabaseType.class, testParams.getDatabaseType());
- ResourceMetaData resourceMetaData = new
ResourceMetaData(Collections.emptyMap(), databaseConfig.getStorageUnits());
- String databaseName = null == rootConfig.getDatabaseName() ?
DefaultDatabase.LOGIC_NAME : rootConfig.getDatabaseName();
- String schemaName = new
DatabaseTypeRegistry(databaseType).getDefaultSchemaName(databaseName);
String sql = SQLHintUtils.removeHint(testParams.getInputSQL());
SQLParserEngine sqlParserEngine = new SQLParserRule(new
DefaultSQLParserRuleConfigurationBuilder().build()).getSQLParserEngine(databaseType);
SQLStatement sqlStatement = sqlParserEngine.parse(sql, false);
- Collection<ShardingSphereRule> rules =
createDatabaseRules(databaseConfig, schemaName, sqlStatement, databaseType);
- ShardingSphereDatabase database =
- new ShardingSphereDatabase(databaseName, databaseType,
resourceMetaData, new RuleMetaData(rules), mockSchemas(schemaName), new
ConfigurationProperties(new Properties()));
+ ShardingSphereDatabase database = createDatabase(rootConfig,
sqlStatement, databaseType);
RuleMetaData globalRuleMetaData = new
RuleMetaData(GlobalRulesBuilder.buildRules(Collections.emptyList(),
Collections.emptyList(), new ConfigurationProperties(new Properties())));
- ConfigurationProperties props = new
ConfigurationProperties(rootConfig.getProps());
+ ConfigurationProperties props =
createConfigurationProperties(rootConfig);
ShardingSphereMetaData metaData = new
ShardingSphereMetaData(Collections.singleton(database), mock(),
globalRuleMetaData, props);
HintValueContext hintValueContext =
SQLHintUtils.extractHint(testParams.getInputSQL());
- SQLStatementContext sqlStatementContext = bind(testParams, metaData,
databaseName, hintValueContext, sqlStatement, sqlParserEngine);
+ SQLStatementContext sqlStatementContext = bind(testParams, metaData,
database.getName(), hintValueContext, sqlStatement, sqlParserEngine);
ConnectionContext connectionContext =
createConnectionContext(database.getName());
QueryContext queryContext = new QueryContext(sqlStatementContext, sql,
testParams.getInputParameters(), hintValueContext, connectionContext, metaData);
+ Collection<ShardingSphereRule> rules =
database.getRuleMetaData().getRules();
RouteContext routeContext = new SQLRouteEngine(rules,
props).route(queryContext, globalRuleMetaData, database);
- SQLRewriteEntry sqlRewriteEntry = new SQLRewriteEntry(database,
globalRuleMetaData, props);
- SQLRewriteResult sqlRewriteResult =
sqlRewriteEntry.rewrite(queryContext, routeContext);
+ SQLRewriteResult sqlRewriteResult = new SQLRewriteEntry(database,
globalRuleMetaData, props).rewrite(queryContext, routeContext);
+ return createSQLRewriteUnits(sqlRewriteResult);
+ }
+
+ private Collection<SQLRewriteUnit> createSQLRewriteUnits(final
SQLRewriteResult sqlRewriteResult) {
return sqlRewriteResult instanceof GenericSQLRewriteResult
? Collections.singleton(((GenericSQLRewriteResult)
sqlRewriteResult).getSqlRewriteUnit())
: (((RouteSQLRewriteResult)
sqlRewriteResult).getSqlRewriteUnits()).values();
}
- private YamlRootConfiguration loadRootConfiguration(final
SQLRewriteEngineTestParameters testParams) throws IOException {
- URL url =
Objects.requireNonNull(Thread.currentThread().getContextClassLoader().getResource(testParams.getRuleFile()),
- String.format("Can not find configuration file `%s`",
testParams.getRuleFile()));
+ protected final YamlRootConfiguration loadRootConfiguration(final
SQLRewriteEngineTestParameters testParams) throws IOException {
+ return loadRootConfiguration(testParams.getRuleFile());
+ }
+
+ protected final YamlRootConfiguration loadRootConfiguration(final String
ruleFile) throws IOException {
+ URL url =
Objects.requireNonNull(Thread.currentThread().getContextClassLoader().getResource(ruleFile),
String.format("Can not find configuration file `%s`", ruleFile));
return YamlEngine.unmarshal(new File(url.getFile()),
YamlRootConfiguration.class);
}
- private Collection<ShardingSphereRule> createDatabaseRules(final
DatabaseConfiguration databaseConfig, final String schemaName, final
SQLStatement sqlStatement, final DatabaseType databaseType) {
- Collection<ShardingSphereRule> result = DatabaseRulesBuilder.build(
- DefaultDatabase.LOGIC_NAME, databaseType, databaseConfig,
mock(), new ResourceMetaData(databaseConfig.getDataSources(),
databaseConfig.getStorageUnits()));
- mockDatabaseRules(result, schemaName, sqlStatement);
+ private ShardingSphereDatabase createDatabase(final YamlRootConfiguration
rootConfig, final SQLStatement sqlStatement, final DatabaseType databaseType)
throws SQLException {
+ DatabaseConfiguration databaseConfig = new
DataSourceProvidedDatabaseConfiguration(
+ new
YamlDataSourceConfigurationSwapper().swapToDataSources(rootConfig.getDataSources()),
new
YamlRuleConfigurationSwapperEngine().swapToRuleConfigurations(rootConfig.getRules()));
+ ResourceMetaData resourceMetaData = new
ResourceMetaData(Collections.emptyMap(), createStorageUnits(databaseConfig,
databaseType));
+ String databaseName = null == rootConfig.getDatabaseName() ?
DefaultDatabase.LOGIC_NAME : rootConfig.getDatabaseName();
+ String schemaName = new
DatabaseTypeRegistry(databaseType).getDefaultSchemaName(databaseName);
+ Collection<ShardingSphereRule> rules =
createDatabaseRules(databaseConfig, schemaName, sqlStatement, databaseType);
+ ConfigurationProperties props =
createConfigurationProperties(rootConfig);
+ RuleMetaData ruleMetaData = new RuleMetaData(rules);
+ Collection<ShardingSphereSchema> schemas = mockSchemas(schemaName);
+ return new ShardingSphereDatabase(databaseName, databaseType,
resourceMetaData, ruleMetaData, schemas, props);
+ }
+
+ private ConfigurationProperties createConfigurationProperties(final
YamlRootConfiguration rootConfig) {
+ Properties result = new Properties();
+ result.putAll(rootConfig.getProps());
+
result.putIfAbsent(ConfigurationPropertyKey.METADATA_IDENTIFIER_CASE_SENSITIVITY.getKey(),
MetadataIdentifierCaseSensitivity.INSENSITIVE.name());
+ return new ConfigurationProperties(result);
+ }
+
+ private Map<String, StorageUnit> createStorageUnits(final
DatabaseConfiguration databaseConfig, final DatabaseType databaseType) throws
SQLException {
+ Map<String, StorageUnit> result = new
LinkedHashMap<>(databaseConfig.getStorageUnits().size(), 1F);
+ for (Entry<String, StorageUnit> entry :
databaseConfig.getStorageUnits().entrySet()) {
+ StorageUnit storageUnit = mock(StorageUnit.class,
RETURNS_DEEP_STUBS);
+ DataSource dataSource = mock(DataSource.class);
+ when(storageUnit.getStorageType()).thenReturn(databaseType);
+ when(storageUnit.getDataSource()).thenReturn(dataSource);
+ when(storageUnit.getConnectionProperties()).thenReturn(new
ConnectionProperties("127.0.0.1", 3306, entry.getKey(), null, new
Properties()));
+ Connection connection = mock(Connection.class);
+ DatabaseMetaData databaseMetaData = mock(DatabaseMetaData.class);
+ when(dataSource.getConnection()).thenReturn(connection);
+ when(connection.getMetaData()).thenReturn(databaseMetaData);
+
DatabaseTypedSPILoader.findService(DialectStorageUnitMetaDataMocker.class,
databaseType).ifPresent(optional ->
optional.mockStorageUnitMetaData(connection, databaseMetaData));
+ result.put(entry.getKey(), storageUnit);
+ }
return result;
}
@@ -162,6 +209,13 @@ public abstract class SQLRewriterIT {
return result;
}
+ private Collection<ShardingSphereRule> createDatabaseRules(final
DatabaseConfiguration databaseConfig, final String schemaName, final
SQLStatement sqlStatement, final DatabaseType databaseType) {
+ Collection<ShardingSphereRule> result =
DatabaseRulesBuilder.build(DefaultDatabase.LOGIC_NAME, databaseType,
+ databaseConfig, mock(ComputeNodeInstanceContext.class), new
ResourceMetaData(databaseConfig.getDataSources(),
databaseConfig.getStorageUnits()));
+ mockDatabaseRules(result, schemaName, sqlStatement);
+ return result;
+ }
+
protected abstract Collection<ShardingSphereSchema> mockSchemas(String
schemaName);
protected abstract void mockDatabaseRules(Collection<ShardingSphereRule>
rules, String schemaName, SQLStatement sqlStatement);
diff --git
a/test/it/rewriter/src/test/java/org/apache/shardingsphere/test/it/rewriter/entity/RewriteInputEntity.java
b/test/it/rewriter/src/test/java/org/apache/shardingsphere/test/it/rewriter/engine/mocker/DialectStorageUnitMetaDataMocker.java
similarity index 59%
copy from
test/it/rewriter/src/test/java/org/apache/shardingsphere/test/it/rewriter/entity/RewriteInputEntity.java
copy to
test/it/rewriter/src/test/java/org/apache/shardingsphere/test/it/rewriter/engine/mocker/DialectStorageUnitMetaDataMocker.java
index cc9a4665c06..2b5652a1d55 100644
---
a/test/it/rewriter/src/test/java/org/apache/shardingsphere/test/it/rewriter/entity/RewriteInputEntity.java
+++
b/test/it/rewriter/src/test/java/org/apache/shardingsphere/test/it/rewriter/engine/mocker/DialectStorageUnitMetaDataMocker.java
@@ -15,26 +15,23 @@
* limitations under the License.
*/
-package org.apache.shardingsphere.test.it.rewriter.entity;
+package org.apache.shardingsphere.test.it.rewriter.engine.mocker;
-import lombok.Getter;
-import lombok.Setter;
+import org.apache.shardingsphere.database.connector.core.spi.DatabaseTypedSPI;
-import javax.xml.bind.annotation.XmlAccessType;
-import javax.xml.bind.annotation.XmlAccessorType;
-import javax.xml.bind.annotation.XmlAttribute;
+import java.sql.Connection;
+import java.sql.DatabaseMetaData;
/**
- * Rewrite input entity for JAXB.
+ * Dialect storage unit meta data mocker.
*/
-@XmlAccessorType(XmlAccessType.FIELD)
-@Getter
-@Setter
-public final class RewriteInputEntity {
+public interface DialectStorageUnitMetaDataMocker extends DatabaseTypedSPI {
- @XmlAttribute(required = true)
- private String sql;
-
- @XmlAttribute
- private String parameters;
+ /**
+ * Mock storage unit meta data.
+ *
+ * @param connection connection
+ * @param databaseMetaData database meta data
+ */
+ void mockStorageUnitMetaData(Connection connection, DatabaseMetaData
databaseMetaData);
}
diff --git
a/test/it/rewriter/src/test/java/org/apache/shardingsphere/test/it/rewriter/engine/mocker/type/MySQLStorageUnitMetaDataMocker.java
b/test/it/rewriter/src/test/java/org/apache/shardingsphere/test/it/rewriter/engine/mocker/type/MySQLStorageUnitMetaDataMocker.java
new file mode 100644
index 00000000000..75c8d401a18
--- /dev/null
+++
b/test/it/rewriter/src/test/java/org/apache/shardingsphere/test/it/rewriter/engine/mocker/type/MySQLStorageUnitMetaDataMocker.java
@@ -0,0 +1,110 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.shardingsphere.test.it.rewriter.engine.mocker.type;
+
+import lombok.SneakyThrows;
+import
org.apache.shardingsphere.test.it.rewriter.engine.mocker.DialectStorageUnitMetaDataMocker;
+
+import java.sql.Connection;
+import java.sql.DatabaseMetaData;
+import java.sql.PreparedStatement;
+import java.sql.ResultSet;
+import java.sql.SQLException;
+import java.sql.Types;
+import java.util.LinkedHashMap;
+import java.util.Map;
+import java.util.Map.Entry;
+import java.util.concurrent.atomic.AtomicReference;
+
+import static org.mockito.Mockito.mock;
+import static org.mockito.Mockito.when;
+
+/**
+ * MySQL storage unit meta data mocker.
+ */
+public final class MySQLStorageUnitMetaDataMocker implements
DialectStorageUnitMetaDataMocker {
+
+ @Override
+ @SneakyThrows(SQLException.class)
+ public void mockStorageUnitMetaData(final Connection connection, final
DatabaseMetaData databaseMetaData) {
+ mockIdentifierCaseRule(connection);
+ mockDataTypeInfo(databaseMetaData);
+ }
+
+ private void mockIdentifierCaseRule(final Connection connection) throws
SQLException {
+ PreparedStatement preparedStatement = mock(PreparedStatement.class);
+ ResultSet resultSet = mock(ResultSet.class);
+ when(connection.prepareStatement("SELECT
@@lower_case_table_names")).thenReturn(preparedStatement);
+ when(preparedStatement.executeQuery()).thenReturn(resultSet);
+ when(resultSet.next()).thenReturn(true);
+ when(resultSet.getInt(1)).thenReturn(0);
+ }
+
+ private void mockDataTypeInfo(final DatabaseMetaData databaseMetaData)
throws SQLException {
+ Map<String, Integer> dataTypes = new LinkedHashMap<>(30, 1F);
+ dataTypes.put("TINYINT", Types.TINYINT);
+ dataTypes.put("SMALLINT", Types.SMALLINT);
+ dataTypes.put("MEDIUMINT", Types.INTEGER);
+ dataTypes.put("INT", Types.INTEGER);
+ dataTypes.put("INTEGER", Types.INTEGER);
+ dataTypes.put("BIGINT", Types.BIGINT);
+ dataTypes.put("DECIMAL", Types.DECIMAL);
+ dataTypes.put("NUMERIC", Types.NUMERIC);
+ dataTypes.put("FLOAT", Types.FLOAT);
+ dataTypes.put("REAL", Types.REAL);
+ dataTypes.put("DOUBLE", Types.DOUBLE);
+ dataTypes.put("BIT", Types.BIT);
+ dataTypes.put("DATE", Types.DATE);
+ dataTypes.put("DATETIME", Types.TIMESTAMP);
+ dataTypes.put("TIMESTAMP", Types.TIMESTAMP);
+ dataTypes.put("TIME", Types.TIME);
+ dataTypes.put("CHAR", Types.CHAR);
+ dataTypes.put("VARCHAR", Types.VARCHAR);
+ dataTypes.put("BINARY", Types.BINARY);
+ dataTypes.put("VARBINARY", Types.VARBINARY);
+ dataTypes.put("TINYBLOB", Types.BLOB);
+ dataTypes.put("BLOB", Types.BLOB);
+ dataTypes.put("MEDIUMBLOB", Types.BLOB);
+ dataTypes.put("LONGBLOB", Types.BLOB);
+ dataTypes.put("TINYTEXT", Types.LONGVARCHAR);
+ dataTypes.put("TEXT", Types.LONGVARCHAR);
+ dataTypes.put("MEDIUMTEXT", Types.LONGVARCHAR);
+ dataTypes.put("LONGTEXT", Types.LONGVARCHAR);
+ dataTypes.put("ENUM", Types.VARCHAR);
+ dataTypes.put("SET", Types.VARCHAR);
+ ResultSet resultSet = mock(ResultSet.class);
+ AtomicReference<Entry<String, Integer>> current = new
AtomicReference<>();
+ when(resultSet.next()).thenAnswer(invocation -> {
+ if (dataTypes.isEmpty()) {
+ return false;
+ }
+ Entry<String, Integer> next =
dataTypes.entrySet().iterator().next();
+ dataTypes.remove(next.getKey());
+ current.set(next);
+ return true;
+ });
+ when(resultSet.getString("TYPE_NAME")).thenAnswer(invocation ->
current.get().getKey());
+ when(resultSet.getInt("DATA_TYPE")).thenAnswer(invocation ->
current.get().getValue());
+ when(databaseMetaData.getTypeInfo()).thenReturn(resultSet);
+ }
+
+ @Override
+ public String getDatabaseType() {
+ return "MySQL";
+ }
+}
diff --git
a/test/it/rewriter/src/test/java/org/apache/shardingsphere/test/it/rewriter/engine/parameter/SQLRewriteEngineTestParametersBuilder.java
b/test/it/rewriter/src/test/java/org/apache/shardingsphere/test/it/rewriter/engine/parameter/SQLRewriteEngineTestParametersBuilder.java
index c07db3f5edc..a866790d989 100644
---
a/test/it/rewriter/src/test/java/org/apache/shardingsphere/test/it/rewriter/engine/parameter/SQLRewriteEngineTestParametersBuilder.java
+++
b/test/it/rewriter/src/test/java/org/apache/shardingsphere/test/it/rewriter/engine/parameter/SQLRewriteEngineTestParametersBuilder.java
@@ -21,8 +21,9 @@ import com.google.common.base.Splitter;
import com.google.common.base.Strings;
import lombok.AccessLevel;
import lombok.NoArgsConstructor;
-import org.apache.commons.lang3.StringUtils;
+import lombok.SneakyThrows;
import org.apache.commons.lang3.math.NumberUtils;
+import org.apache.shardingsphere.infra.exception.ShardingSpherePreconditions;
import org.apache.shardingsphere.test.it.rewriter.engine.type.SQLExecuteType;
import
org.apache.shardingsphere.test.it.rewriter.entity.RewriteAssertionEntity;
import
org.apache.shardingsphere.test.it.rewriter.entity.RewriteAssertionsRootEntity;
@@ -30,6 +31,16 @@ import
org.apache.shardingsphere.test.it.rewriter.entity.RewriteOutputEntity;
import
org.apache.shardingsphere.test.it.rewriter.loader.RewriteAssertionsRootEntityLoader;
import java.io.File;
+import java.io.IOException;
+import java.math.BigDecimal;
+import java.net.URISyntaxException;
+import java.net.URL;
+import java.nio.file.Files;
+import java.nio.file.Paths;
+import java.sql.Date;
+import java.sql.Timestamp;
+import java.time.LocalDate;
+import java.time.LocalTime;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.Collection;
@@ -37,9 +48,12 @@ import java.util.Collections;
import java.util.LinkedHashMap;
import java.util.LinkedList;
import java.util.List;
+import java.util.Locale;
import java.util.Map;
import java.util.Map.Entry;
import java.util.Objects;
+import java.util.regex.Matcher;
+import java.util.regex.Pattern;
import java.util.stream.Collectors;
/**
@@ -48,6 +62,26 @@ import java.util.stream.Collectors;
@NoArgsConstructor(access = AccessLevel.PRIVATE)
public final class SQLRewriteEngineTestParametersBuilder {
+ private static final String COMMA_WITH_SPACE_SEPARATOR = ", ";
+
+ private static final String DATABASE_TYPE_PLACEHOLDER = "${databaseType}";
+
+ private static final String DATABASE_TYPE_LOWER_PLACEHOLDER =
"${databaseTypeLower}";
+
+ private static final String HEXTORAW_PARAMETER_PREFIX = "HEXTORAW:";
+
+ private static final String ESCAPED_COMMA = "{ESCAPE_COMMA}";
+
+ private static final String ESCAPED_COLON = "{ESCAPE_COLON}";
+
+ private static final Pattern IFNULL_PARAMETER_COLUMN_PATTERN =
Pattern.compile("IFNULL\\s*\\(\\s*([`\\w.]+)\\s*,\\s*\\?",
Pattern.CASE_INSENSITIVE);
+
+ private static final Pattern DATE_PARAMETER_PATTERN =
Pattern.compile("\\d{4}-\\d{2}-\\d{2}");
+
+ private static final Pattern DATETIME_PARAMETER_PATTERN =
Pattern.compile("\\d{4}-\\d{2}-\\d{2} \\d{2}:\\d{2}:\\d{2}(\\.\\d{1,6})?");
+
+ private static final Pattern TIME_PARAMETER_PATTERN =
Pattern.compile("\\d{2}:\\d{2}(:\\d{2}(\\.\\d{1,6})?)?");
+
/**
* Load test parameters.
*
@@ -97,16 +131,27 @@ public final class SQLRewriteEngineTestParametersBuilder {
private static Collection<SQLRewriteEngineTestParameters>
createTestParameters(final String type, final String fileName, final
RewriteAssertionsRootEntity rootAssertions) {
Collection<SQLRewriteEngineTestParameters> result = new LinkedList<>();
for (RewriteAssertionEntity each : rootAssertions.getAssertions()) {
+ String ruleFile = Strings.isNullOrEmpty(each.getYamlRule()) ?
rootAssertions.getYamlRule() : each.getYamlRule();
for (String databaseType :
getDatabaseTypes(each.getDatabaseTypes())) {
- // TODO support appendLiteralCases for exits cases and remove
duplicate cases
+ String actualRuleFile = resolveRuleFile(ruleFile,
databaseType);
+ boolean success = setSQLIfSQLFileNotNull(each);
+ if (success) {
+ appendLiteralCases(type, fileName, actualRuleFile, each,
databaseType, result);
+ continue;
+ }
SQLExecuteType sqlExecuteType = null ==
each.getInput().getParameters() || each.getInput().getParameters().isEmpty() ?
SQLExecuteType.LITERAL : SQLExecuteType.PLACEHOLDER;
- result.add(new SQLRewriteEngineTestParameters(type,
each.getId(), fileName, rootAssertions.getYamlRule(), each.getInput().getSql(),
- createParameters(each.getInput().getParameters()),
createOutputSQLs(each.getOutputs()),
createOutputGroupedParameters(each.getOutputs()), databaseType,
sqlExecuteType));
+ result.add(new SQLRewriteEngineTestParameters(type,
each.getId(), fileName, actualRuleFile, each.getInput().getSql(),
+ createParameters(each.getInput().getParameters(),
each.getInput().getSql()), createOutputSQLs(each.getOutputs()),
createOutputGroupedParameters(each.getOutputs()),
+ databaseType, sqlExecuteType));
}
}
return result;
}
+ private static String resolveRuleFile(final String ruleFile, final String
databaseType) {
+ return ruleFile.replace(DATABASE_TYPE_PLACEHOLDER,
databaseType).replace(DATABASE_TYPE_LOWER_PLACEHOLDER,
databaseType.toLowerCase(Locale.ENGLISH));
+ }
+
private static Collection<String> getDatabaseTypes(final String
databaseTypes) {
return Strings.isNullOrEmpty(databaseTypes) ? getAllDatabaseTypes() :
Splitter.on(',').trimResults().splitToList(databaseTypes);
}
@@ -115,18 +160,121 @@ public final class SQLRewriteEngineTestParametersBuilder
{
return Arrays.asList("MySQL", "PostgreSQL", "Oracle", "SQLServer",
"SQL92", "openGauss");
}
- private static List<Object> createParameters(final String inputParams) {
+ private static boolean setSQLIfSQLFileNotNull(final RewriteAssertionEntity
entity) {
+ boolean result = false;
+ if (null != entity.getInput().getSqlFile()) {
+ URL resource =
SQLRewriteEngineTestParametersBuilder.class.getClassLoader().getResource(entity.getInput().getSqlFile());
+ ShardingSpherePreconditions.checkNotNull(resource, () -> new
IllegalArgumentException(String.format("Resource '%s' is not found.",
entity.getInput().getSqlFile())));
+ entity.getInput().setSql(getSQLFromFilePath(resource));
+ result = true;
+ }
+ for (RewriteOutputEntity each : entity.getOutputs()) {
+ if (null != each.getSqlFile()) {
+
each.setSql(getSQLFromFilePath(Objects.requireNonNull(SQLRewriteEngineTestParametersBuilder.class.getClassLoader().getResource(each.getSqlFile()))));
+ }
+ }
+ return result;
+ }
+
+ private static void appendLiteralCases(final String type, final String
fileName, final String ruleFile, final RewriteAssertionEntity assertionEntity,
+ final String databaseType, final
Collection<SQLRewriteEngineTestParameters> result) {
+ if (null == assertionEntity.getInput().getParameters() ||
assertionEntity.getInput().getParameters().isEmpty()) {
+ result.add(new SQLRewriteEngineTestParameters(type,
assertionEntity.getId(), fileName, ruleFile,
assertionEntity.getInput().getSql(),
+
createParameters(assertionEntity.getInput().getParameters(), null),
createOutputSQLs(assertionEntity.getOutputs()),
createOutputGroupedParameters(assertionEntity.getOutputs()),
+ databaseType, SQLExecuteType.LITERAL));
+ } else {
+ result.add(new SQLRewriteEngineTestParameters(type,
assertionEntity.getId(), fileName, ruleFile,
assertionEntity.getInput().getSql(),
+
createParameters(assertionEntity.getInput().getParameters(),
assertionEntity.getInput().getSql()),
createOutputSQLs(assertionEntity.getOutputs()),
+
createOutputGroupedParameters(assertionEntity.getOutputs()), databaseType,
SQLExecuteType.PLACEHOLDER));
+ result.add(new SQLRewriteEngineTestParameters(type,
assertionEntity.getId(), fileName, ruleFile,
+ getLiteralSQL(assertionEntity.getInput().getSql(),
assertionEntity.getInput().getParameters()), Collections.emptyList(),
createLiteralOutputSQLs(assertionEntity.getOutputs()),
+
createLiteralOutputGroupedParameters(assertionEntity.getOutputs()),
databaseType, SQLExecuteType.LITERAL));
+ }
+ }
+
+ private static String getLiteralSQL(final String sql, final String
parameters) {
+ List<String> params =
+
Splitter.on(COMMA_WITH_SPACE_SEPARATOR).omitEmptyStrings().trimResults().splitToList(parameters).stream().map(SQLRewriteEngineTestParametersBuilder::createLiteralParameter)
+ .collect(Collectors.toList());
+ return params.isEmpty() ? sql : String.format(sql.replace("%",
"ÿ").replace("?", "%s"), params.toArray()).replace("ÿ", "%").replace("%%",
"%").replace("'%'", "'%%'");
+ }
+
+ private static String createLiteralParameter(final String value) {
+ String actualValue = replaceEscapedChars(value);
+ return actualValue.startsWith(HEXTORAW_PARAMETER_PREFIX) ?
String.format("HEXTORAW('%s')",
actualValue.substring(HEXTORAW_PARAMETER_PREFIX.length())) : "'" + actualValue
+ "'";
+ }
+
+ @SneakyThrows({IOException.class, URISyntaxException.class})
+ private static String getSQLFromFilePath(final URL fileURL) {
+ return
Files.readAllLines(Paths.get(fileURL.toURI())).stream().collect(Collectors.joining(System.lineSeparator()));
+ }
+
+ private static List<Object> createParameters(final String inputParams,
final String sql) {
if (null == inputParams) {
return Collections.emptyList();
}
- return
Splitter.on(",").trimResults().splitToList(inputParams).stream().map(SQLRewriteEngineTestParametersBuilder::createInputParameter).collect(Collectors.toList());
+ List<String> inputParamValues = splitParameters(inputParams);
+ List<String> ifNullColumnNames = getIfNullColumnNames(sql);
+ List<Object> result = new ArrayList<>(inputParamValues.size());
+ for (int i = 0; i < inputParamValues.size(); i++) {
+ result.add(createInputParameter(inputParamValues.get(i), i <
ifNullColumnNames.size() ? ifNullColumnNames.get(i) : null));
+ }
+ return result;
+ }
+
+ private static Object createInputParameter(final String inputParam, final
String ifNullColumnName) {
+ String actualInputParam = replaceEscapedChars(inputParam);
+ if ("NULL".equalsIgnoreCase(actualInputParam)) {
+ return null;
+ }
+ if (NumberUtils.isCreatable(actualInputParam) && null !=
ifNullColumnName) {
+ String actualColumnName = ifNullColumnName.toLowerCase();
+ if (actualColumnName.contains("_decimal") ||
actualColumnName.contains("_numeric")) {
+ return new BigDecimal(actualInputParam);
+ }
+ if (actualColumnName.contains("_float") ||
actualColumnName.contains("_real")) {
+ return Float.parseFloat(actualInputParam);
+ }
+ if (actualColumnName.contains("_double")) {
+ return Double.parseDouble(actualInputParam);
+ }
+ }
+ if (DATETIME_PARAMETER_PATTERN.matcher(actualInputParam).matches()) {
+ return Timestamp.valueOf(actualInputParam);
+ }
+ if (DATE_PARAMETER_PATTERN.matcher(actualInputParam).matches()) {
+ return Date.valueOf(LocalDate.parse(actualInputParam));
+ }
+ if (TIME_PARAMETER_PATTERN.matcher(actualInputParam).matches()) {
+ return LocalTime.parse(actualInputParam);
+ }
+ if (NumberUtils.isCreatable(actualInputParam) &&
actualInputParam.contains(".")) {
+ return new BigDecimal(actualInputParam);
+ }
+ if (NumberUtils.isCreatable(actualInputParam)) {
+ return NumberUtils.createNumber(actualInputParam);
+ }
+ return actualInputParam;
}
- private static Object createInputParameter(final String inputParam) {
- if (StringUtils.isNumeric(inputParam)) {
- return NumberUtils.createNumber(inputParam);
+ private static List<String> splitParameters(final String parameters) {
+ return null == parameters ? Collections.emptyList() :
Splitter.on(",").omitEmptyStrings().trimResults().splitToList(parameters);
+ }
+
+ private static List<String> getIfNullColumnNames(final String sql) {
+ if (null == sql) {
+ return Collections.emptyList();
}
- return "NULL".equals(inputParam) ? null : inputParam;
+ List<String> result = new LinkedList<>();
+ Matcher matcher = IFNULL_PARAMETER_COLUMN_PATTERN.matcher(sql);
+ while (matcher.find()) {
+ result.add(matcher.group(1).replace("`", ""));
+ }
+ return result;
+ }
+
+ private static String replaceEscapedChars(final String value) {
+ return value.replace(ESCAPED_COMMA, ",").replace(ESCAPED_COLON, ":");
}
private static List<String> createOutputSQLs(final
List<RewriteOutputEntity> outputs) {
@@ -137,11 +285,25 @@ public final class SQLRewriteEngineTestParametersBuilder {
return result;
}
+ private static List<String> createLiteralOutputSQLs(final
List<RewriteOutputEntity> outputs) {
+ List<String> result = new ArrayList<>(outputs.size());
+ for (RewriteOutputEntity each : outputs) {
+ result.add(getLiteralSQL(each.getSql(), each.getParameters()));
+ }
+ return result;
+ }
+
private static List<List<Object>> createOutputGroupedParameters(final
List<RewriteOutputEntity> outputs) {
List<List<Object>> result = new ArrayList<>(outputs.size());
for (RewriteOutputEntity each : outputs) {
- result.add(createParameters(each.getParameters()));
+ result.add(createParameters(each.getParameters(), null));
}
return result;
}
+
+ private static List<List<Object>>
createLiteralOutputGroupedParameters(final List<RewriteOutputEntity> outputs) {
+ List<List<Object>> result = new ArrayList<>(outputs.size());
+ outputs.forEach(each -> result.add(Collections.emptyList()));
+ return result;
+ }
}
diff --git
a/test/it/rewriter/src/test/java/org/apache/shardingsphere/test/it/rewriter/entity/RewriteAssertionEntity.java
b/test/it/rewriter/src/test/java/org/apache/shardingsphere/test/it/rewriter/entity/RewriteAssertionEntity.java
index 013f98fb49f..082c997aa60 100644
---
a/test/it/rewriter/src/test/java/org/apache/shardingsphere/test/it/rewriter/entity/RewriteAssertionEntity.java
+++
b/test/it/rewriter/src/test/java/org/apache/shardingsphere/test/it/rewriter/entity/RewriteAssertionEntity.java
@@ -37,6 +37,9 @@ public final class RewriteAssertionEntity {
@XmlAttribute(required = true)
private String id;
+ @XmlAttribute(name = "yaml-rule")
+ private String yamlRule;
+
@XmlElement(required = true)
private RewriteInputEntity input;
diff --git
a/test/it/rewriter/src/test/java/org/apache/shardingsphere/test/it/rewriter/entity/RewriteInputEntity.java
b/test/it/rewriter/src/test/java/org/apache/shardingsphere/test/it/rewriter/entity/RewriteInputEntity.java
index cc9a4665c06..17f561dbc19 100644
---
a/test/it/rewriter/src/test/java/org/apache/shardingsphere/test/it/rewriter/entity/RewriteInputEntity.java
+++
b/test/it/rewriter/src/test/java/org/apache/shardingsphere/test/it/rewriter/entity/RewriteInputEntity.java
@@ -35,6 +35,9 @@ public final class RewriteInputEntity {
@XmlAttribute(required = true)
private String sql;
+ @XmlAttribute(name = "sql-file")
+ private String sqlFile;
+
@XmlAttribute
private String parameters;
}
diff --git
a/test/it/rewriter/src/test/java/org/apache/shardingsphere/test/it/rewriter/entity/RewriteOutputEntity.java
b/test/it/rewriter/src/test/java/org/apache/shardingsphere/test/it/rewriter/entity/RewriteOutputEntity.java
index 015d91601a0..2b65b74e37b 100644
---
a/test/it/rewriter/src/test/java/org/apache/shardingsphere/test/it/rewriter/entity/RewriteOutputEntity.java
+++
b/test/it/rewriter/src/test/java/org/apache/shardingsphere/test/it/rewriter/entity/RewriteOutputEntity.java
@@ -35,6 +35,9 @@ public final class RewriteOutputEntity {
@XmlAttribute(required = true)
private String sql;
+ @XmlAttribute(name = "sql-file")
+ private String sqlFile;
+
@XmlAttribute
private String parameters;
}
diff --git
a/test/it/rewriter/src/test/resources/META-INF/services/org.apache.shardingsphere.test.it.rewriter.engine.mocker.DialectStorageUnitMetaDataMocker
b/test/it/rewriter/src/test/resources/META-INF/services/org.apache.shardingsphere.test.it.rewriter.engine.mocker.DialectStorageUnitMetaDataMocker
new file mode 100644
index 00000000000..8dab509ae67
--- /dev/null
+++
b/test/it/rewriter/src/test/resources/META-INF/services/org.apache.shardingsphere.test.it.rewriter.engine.mocker.DialectStorageUnitMetaDataMocker
@@ -0,0 +1,18 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+org.apache.shardingsphere.test.it.rewriter.engine.mocker.type.MySQLStorageUnitMetaDataMocker
