comphead commented on PR #3892:
URL:
https://github.com/apache/datafusion-comet/pull/3892#issuecomment-4227377490
what exactly the query that failed in spark? I checked DF corr and PGQL corr
works the same.
```
> CREATE TABLE test_corr_nan(x double, y double, grp string);
0 row(s) fetched.
Elapsed 0.025 seconds.
> INSERT INTO test_corr_nan VALUES (cast('NaN' as double), cast('NaN' as
double), 'both_nan'), (cast('NaN' as double), 1.0, 'nan_val'), (1.0, cast('NaN'
as double), 'val_nan'), (NULL, cast('NaN' as double), 'null_nan'), (cast('NaN'
as double), NULL, 'nan_null'), (NULL, NULL, 'both_null'), (NULL, 1.0,
'null_val'), (1.0, NULL, 'val_null');
+-------+
| count |
+-------+
| 8 |
+-------+
1 row(s) fetched.
Elapsed 0.016 seconds.
> SELECT grp, corr(x, y) FROM test_corr_nan GROUP BY grp ORDER BY grp;
+-----------+---------------------------------------+
| grp | corr(test_corr_nan.x,test_corr_nan.y) |
+-----------+---------------------------------------+
| both_nan | NaN |
| both_null | NULL |
| nan_null | NULL |
| nan_val | NULL |
| null_nan | NULL |
| null_val | NULL |
| val_nan | NULL |
| val_null | NULL |
+-----------+---------------------------------------+
8 row(s) fetched.
Elapsed 0.036 seconds.
```
PGSQL
```
CREATE TABLE test_corr_nan(x float, y float, grp varchar);
INSERT INTO test_corr_nan VALUES (
cast('NaN' as float), cast('NaN' as float), 'both_nan'), (
cast('NaN' as float), 1.0, 'nan_val'),
(1.0, cast('NaN' as float), 'val_nan'),
(NULL, cast('NaN' as float), 'null_nan'),
(cast('NaN' as float), NULL, 'nan_null'),
(NULL, NULL, 'both_null'), (NULL, 1.0, 'null_val'), (1.0, NULL, 'val_null');
SELECT grp, corr(x, y) FROM test_corr_nan GROUP BY grp ORDER BY grp;
grp | corr
-----------+------
both_nan | NaN
both_null |
nan_null |
nan_val |
null_nan |
null_val |
val_nan |
val_null |
(8 rows)
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
