[ 
https://issues.apache.org/jira/browse/SPARK-61296?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Utkarsh Bajaj updated SPARK-61296:
----------------------------------
    Component/s: SQL
                     (was: Shuffle)

> Return CollectLimitExec gather results in partition order
> ---------------------------------------------------------
>
>                 Key: SPARK-61296
>                 URL: https://issues.apache.org/jira/browse/SPARK-61296
>             Project: Spark
>          Issue Type: Bug
>          Components: Spark Core, SQL
>    Affects Versions: 4.0.5
>            Reporter: Utkarsh Bajaj
>            Priority: Major
>
> CollectLimitExec moves all sorted partitions into one partition/task through 
> a shuffle, and then keeps the rows between OFFSET and LIMIT. The shuffle 
> hands back the partitions in the order they arrive, not in partition order. 
> So the rows can come back out of order, and OFFSET can skip the wrong rows 
> for order by queries.



--
This message was sent by Atlassian Jira
(v10.3.26#10030026)

---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to