[
https://issues.apache.org/jira/browse/CALCITE-5529?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated CALCITE-5529:
------------------------------------
Labels: pull-request-available (was: )
> Improve dialect tests
> ---------------------
>
> Key: CALCITE-5529
> URL: https://issues.apache.org/jira/browse/CALCITE-5529
> Project: Calcite
> Issue Type: Bug
> Reporter: Julian Hyde
> Assignee: Julian Hyde
> Priority: Major
> Labels: pull-request-available
>
> The current dialect tests are quick and robust but not very thorough. In
> {{RelToSqlConverterTest}} there are about 400 tests (374 SQL statements and
> 30 relational expressions) that test translation to 538 SQL strings in target
> dialects (an average of 1.34 dialects per test). To run the 371 test methods
> takes about 12 seconds elapsed, 80 CPU seconds on my MacBook Pro. None of
> those SQL strings are executed, so we don't know whether they return the
> correct results or are even valid.
> Improvements:
> * Define a list of dialects that we wish to test (the "target dialects") and
> provide a means to connect to a database for each of these (typically a
> driver on the class path, a URL, user name and password);
> * In each test, for each target dialect, generate a SQL string. The
> generation should not throw.
> * In each test, the query is first executed against Calcite to create a
> sample. (For any test, this can be overridden to use a different reference
> dialect. Calcite will use a recording, if present.)
> * In each test, for each target dialect that has an *enabled* connection,
> execute the generated SQL against that connection. If there is a reference
> result for the test, check that the result equals the reference result.
> * In each test, for each target dialect that has a *recording* connection,
> execute the generated SQL against that connection. Record the outcome (error,
> column names and types, result) in the file. Proceed as for an enabled
> connection. (Check result against reference result, etc.)
> * In each test, for each target dialect that has a *replaying* connection,
> look up the outcome of the generated SQL. If the outcome is present, proceed
> as for an enabled connection. (Check result against reference result, etc.)
> * In each test, for each target dialect that has a *replaying+recording*
> connection, look up the outcome of the generated SQL. If the outcome is
> present, proceed as for an enabled connection. (Check result against
> reference result, etc.) If the outcome is not present, proceed as for a
> recording connection. (Execute the SQL, record the outcome, check against
> reference result, etc.)
> With these improvements, the test suite:
> * Remains fast and robust (because recordings are used most of the time);
> * Can be run offline (using recordings);
> * Can be run by contributors who do not necessarily have access to all of
> the supported databases;
> * Has full coverage (runs all tests against all supported dialects);
> * Can re-generate the 'golden' references. (Must be online, have the
> required driver and connection, and takes more time.)
> Design:
> * A new {{class DialectTestConfig}} describes the dialects to be tested,
> their names, connections, the connection state (enabled, recording,
> replaying, replaying+recording), and recording file name.
> * A mock JDBC driver mediates recording/replay.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
