tschwarzinger opened a new pull request, #27170:
URL: https://github.com/apache/datafusion/pull/27170
## Which issue does this PR close?
- No issue, I found it while trying to mangle some data with the
`datafusion-cli`.
## Rationale for this change
Queries like:
```sql
COPY (
SELECT id, ts, x FROM (
SELECT 'b' AS id, ts, x FROM union_const_src
UNION ALL
SELECT 'a' AS id, ts, x FROM union_const_src
) ORDER BY id, ts
) TO 'test_files/scratch/copy/union_const_order/out.parquet';
```
This can cause issue in the current version of datafusion as `SortExec` can
report wrong equivalence properties when `preserve_partitioning = false`.
Running this leads to DataFusion believing that the result is sorted even
though it isn't.
## What changes are included in this PR?
- Change computation of equivalence properties when `preserve_partitioning`
is false (and there are more than one input partitions
## What is the testing strategy for this PR?
- Unit test for this scenario
- SLT test for the original issue I encountered
Running the SLT tests without the fix causes the following diff:
```text
- 02)--SortPreservingMergeExec: [id@0 ASC NULLS LAST, ts@1 ASC NULLS LAST]
+ 02)--SortPreservingMergeExec: [ts@1 ASC NULLS LAST]
```
## Are there any user-facing changes?
- Correct sort orders being reported
## Disclaimer
Generated by an LLM, Simplified and Checked by me
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
