Julian Hyde created CALCITE-5529:
------------------------------------
Summary: Improve dialect tests
Key: CALCITE-5529
URL: https://issues.apache.org/jira/browse/CALCITE-5529
Project: Calcite
Issue Type: Bug
Reporter: Julian Hyde
The current dialect tests are quick and robust but not very thorough. In
{{RelToSqlConverterTest}} there are about 400 tests (374 SQL statements and 30
relational expressions) that test translation to 538 SQL strings in target
dialects (an average of 1.34 dialects per test). None of those SQL strings are
executed, so we don't know whether they return the correct results or are even
valid.
Improvements:
* Define a list of dialects that we wish to test (the "target dialects") and
provide a means to connect to a database for each of these (typically a driver
on the class path, a URL, user name and password);
* In each test, for each target dialect, generate a SQL string. The generation
should not throw.
* In each test, the query is first executed against Calcite to create a
sample. (For any test, this can be overridden to use a different reference
dialect. Calcite will use a recording, if present.)
* In each test, for each target dialect that has an *enabled* connection,
execute the generated SQL against that connection. If there is a reference
result for the test, check that the result equals the reference result.
* In each test, for each target dialect that has a *recording* connection,
execute the generated SQL against that connection. Record the outcome (error,
column names and types, result) in the file. Proceed as for an enabled
connection. (Check result against reference result, etc.)
* In each test, for each target dialect that has a *replaying* connection,
look up the outcome of the generated SQL. If the outcome is present, proceed as
for an enabled connection. (Check result against reference result, etc.)
* In each test, for each target dialect that has a *replaying+recording*
connection, look up the outcome of the generated SQL. If the outcome is
present, proceed as for an enabled connection. (Check result against reference
result, etc.) If the outcome is not present, proceed as for a recording
connection. (Execute the SQL, record the outcome, check against reference
result, etc.)
With these improvements, the test suite:
* Remains fast and robust (because recordings are used most of the time);
* Can be run offline (using recordings);
* Can be run by contributors who do not necessarily have access to all of the
supported databases;
* Has full coverage (runs all tests against all supported dialects);
* Can re-generate the 'golden' references. (Must be online, have the required
driver and connection, and takes more time.)
Design:
* A new {{class DialectTestConfig}} describes the dialects to be tested, their
names, connections, the connection state (enabled, recording, replaying,
replaying+recording), and recording file name.
* A mock JDBC driver mediates recording/replay.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
