raghavyadav01 commented on issue #27223:
URL: https://github.com/apache/datafusion/issues/27223#issuecomment-6776205242
Checked those. DuckDB, Polars and Spark all reject concat of a 2D and a 1D
list, so none of them defines this case. On NULL arguments they split: DuckDB
skips them (like our 1D today), Spark and Polars return NULL.
| Query | DuckDB 1.5.6 | Polars 2.0 | Spark 4.2.1 | Postgres 17 | DataFusion
main | With fix |
|---|---|---|---|---|---|---|
| `[1,2]` + NULL (1D) | `[1, 2]` | NULL | NULL | `{1,2}` | `[1, 2]` | `[1,
2]` |
| NULL + NULL (1D) | `[]` | NULL | NULL | NULL | NULL | NULL |
| `[[30,40]]` + `[1,2]` (2D + 1D) | error | error | error |
`{{30,40},{1,2}}` | `[[30, 40], [1, 2]]` | same |
| `[[NULL,NULL,60]]` + NULL | `[[NULL, NULL, 60]]`* | NULL | NULL |
`{{NULL,NULL,60}}` | `[[NULL, NULL, 60], NULL]` | `[[NULL, NULL, 60]]` |
| NULL (2D) + NULL (1D) | error | error | error | NULL | `[NULL]` or NULL by
batch | NULL |
\* NULL cast to 2D, since DuckDB rejects mixing.
The fix keeps 2D + 1D consistent with our existing 1D behavior and removes
the batch dependence. Switching to the Spark rule would change 1D too, so I'd
leave that for a separate issue.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
