Omega359 commented on issue #13470:
URL: https://github.com/apache/datafusion/issues/13470#issuecomment-2539395881
> > The result is that everything runs however a LOT of queries are skipped
for DF for various reasons.
>
> that's totally understandable! what about having them with `query error
msg` instead of skip, so that they act as "executable todo" and thus we don't
need to remember to find and enable tests when corresponding functionality is
fixed?
I was torn about this because it would mean that the expected query types
and results would then be lost.
> > Because of the format differences between the slt files in DF vs sqlite
a custom version of sqllogictest dependency is required.
>
> i know you mentioned this earlier ([#13470
(comment)](https://github.com/apache/datafusion/issues/13470#issuecomment-2518423885)),
but i unfortunately don't know what the difference is. How big it is? Can we
avoid custom dependency with eg a regexp replace? cc @gliga
The primary issue is that those slt files are in what I call 'valuewise'
ordering, meaning that results are in a single column. For example:
```
query II rowsort
SELECT ALL + col1 AS col0, col2 AS col2 FROM tab1
----
14
96
47
68
5
59
```
Every other use of slt's use what I call 'rowwise' where the results are in
proper columns. Both have their uses - valuewise better handles string with
leading/trailing spaces, rowwise is easier to parse.
The main change I made to the sltlogictest-rs dep is to add support for
`control resultmode valuewise` which you will see at the top of all the sqlite
test files to allow that lib to be able to compare the results correctly.
I also stripped out everything related to mysql and mssql from those files.
This is the series of regex I used to clean up the files:
```
find ./ -type f -name "*.test" -exec rename 's/\.test/\.slt/' {} \;
find ./ -type f -name "*.slt" -exec sd -f i 'hash-threshold 8\n'
'hash-threshold 8\ncontrol resultmode valuewise\n' {} \;
find ./ -type f -name "*.slt" -exec sd -f i 'onlyif
mysql.+\n.+\n.+\n.+\n.+\n.+\n.+\n.+\n.+\n.+\n.+\n.+\n\n' '' {} \;
find ./ -type f -name "*.slt" -exec sd -f i 'onlyif
mysql.+\n.+\n.+\n.+\n.+\n.+\n.+\n.+\n.+\n.+\n.+\n\n' '' {} \;
find ./ -type f -name "*.slt" -exec sd -f i 'onlyif
mysql.+\n.+\n.+\n.+\n.+\n.+\n.+\n.+\n.+\n.+\n\n' '' {} \;
find ./ -type f -name "*.slt" -exec sd -f i 'onlyif
mysql.+\n.+\n.+\n.+\n.+\n.+\n.+\n.+\n.+\n\n' '' {} \;
find ./ -type f -name "*.slt" -exec sd -f i 'onlyif
mysql.+\n.+\n.+\n.+\n.+\n.+\n.+\n.+\n\n' '' {} \;
find ./ -type f -name "*.slt" -exec sd -f i 'onlyif
mysql.+\n.+\n.+\n.+\n.+\n.+\n.+\n\n' '' {} \;
find ./ -type f -name "*.slt" -exec sd -f i 'onlyif
mysql.+\n.+\n.+\n.+\n.+\n.+\n\n' '' {} \;
find ./ -type f -name "*.slt" -exec sd -f i 'onlyif
mysql.+\n.+\n.+\n.+\n.+\n\n' '' {} \;
find ./ -type f -name "*.slt" -exec sd -f i 'onlyif mysql.+\n.+\n.+\n.+\n\n'
'' {} \;
find ./ -type f -name "*.slt" -exec sd -f i 'onlyif mysql.+\n.+\n.+\n\n' ''
{} \;
find ./ -type f -name "*.slt" -exec sd -f i 'skipif mysql.+\n' '' {} \;
find ./ -type f -name "*.slt" -exec sd -f i 'skipif postgresql # PostgreSQL
requires AS when renaming output columns\n' '' {} \;
find ./ -type f -name "*.slt" -exec sd -f i 'onlyif
mssql.+\n.+\n.+\n.+\n.+\n.+\n.+\n.+\n.+\n.+\n.+\n.+\n\n' '' {} \;
find ./ -type f -name "*.slt" -exec sd -f i 'onlyif
mssql.+\n.+\n.+\n.+\n.+\n.+\n.+\n.+\n.+\n.+\n.+\n\n' '' {} \;
find ./ -type f -name "*.slt" -exec sd -f i 'onlyif
mssql.+\n.+\n.+\n.+\n.+\n.+\n.+\n.+\n.+\n.+\n\n' '' {} \;
find ./ -type f -name "*.slt" -exec sd -f i 'onlyif
mssql.+\n.+\n.+\n.+\n.+\n.+\n.+\n.+\n.+\n\n' '' {} \;
find ./ -type f -name "*.slt" -exec sd -f i 'onlyif
mssql.+\n.+\n.+\n.+\n.+\n.+\n.+\n.+\n\n' '' {} \;
find ./ -type f -name "*.slt" -exec sd -f i 'onlyif
mssql.+\n.+\n.+\n.+\n.+\n.+\n.+\n\n' '' {} \;
find ./ -type f -name "*.slt" -exec sd -f i 'onlyif
mssql.+\n.+\n.+\n.+\n.+\n.+\n\n' '' {} \;
find ./ -type f -name "*.slt" -exec sd -f i 'onlyif
mssql.+\n.+\n.+\n.+\n.+\n\n' '' {} \;
find ./ -type f -name "*.slt" -exec sd -f i 'onlyif mssql.+\n.+\n.+\n.+\n\n'
'' {} \;
find ./ -type f -name "*.slt" -exec sd -f i 'onlyif mssql.+\n.+\n.+\n\n' ''
{} \;
find ./ -type f -name "*.slt" -exec sd -f i 'skipif mssql # not
compatible\n' '' {} \;
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org
For additional commands, e-mail: github-h...@datafusion.apache.org
