mvanschellebeeck commented on issue #4248: URL: https://github.com/apache/arrow-datafusion/issues/4248#issuecomment-1320762453
Thanks @xudong963. sqllogictest-rs looks an awesome starting point! Looks like DuckDB runs an extended version of SQLLgic tests: > For testing plain SQL we use an extended version of the SQL logic test suite, adopted from [SQLite](https://www.sqlite.org/sqllogictest/doc/trunk/about.wiki) ### But with: 1. A port of the tests from [sql_integration](https://github.com/apache/arrow-datafusion/tree/master/datafusion/core/tests/sql) 2. And the utilisation of the **huge** set existing SQLLite sqllogictests (https://www.sqlite.org/sqllogictest/dir?ci=tip&name=test) We could develop a pretty solid testing foundation with [sqllogictest-rs](https://github.com/risinglightdb/sqllogictest-rs) as the parser and runner. ### On Current Test Migration We could migrate a large chunk of tests with a clever regex that captures on the query and result variables and manually handle some hard-to-automate edge cases like [this one](https://github.com/apache/arrow-datafusion/blob/master/datafusion/core/tests/sql/create_drop.rs#L116) (which creates a few tables) and [this one](https://github.com/apache/arrow-datafusion/blob/master/datafusion/core/tests/sql/create_drop.rs#L482) (that expects a failure). ### On Forking vs Directly using sqllogictest-rs On using sqllogictest-rs vs forking it - using it directly seems like a good first step to get the test suite up and running. Though I can imagine some datafusion-specific queries (that @alamb noted) will eventually force us to move onto a fork in the future. As an example - Materialize created their own sqllogictest driver, [mz_sqllogictest](https://dev.materialize.com/api/rust/mz_sqllogictest/index.html) to deal with this. ### As a v0, how does this approach sound? 1. Add a `tests/sqllogictests` folder at the root of arrow-datafusion 2. Use `sqllogitest-rs` and `datafusion-cli` 3. Implement the sqllogictest-rs runner using `datafusion-cli` 4. Migrate some tests from [sql_integration](https://github.com/apache/arrow-datafusion/tree/master/datafusion/core/tests/sql) 5. Setup a simple CI workflow to run the tests -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
