[jira] [Commented] (FLINK-38817) Out of order data seen while running tpc-ds queries

[Zhu Zhu (Jira)]

    [ 
https://issues.apache.org/jira/browse/FLINK-38817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18052001#comment-18052001
 ] 

Zhu Zhu commented on FLINK-38817:
---------------------------------

That issue is unrelated.

`ForwardForUnspecifiedPartitioner` differs from `ForwardPartitioner`. 
`ForwardPartitioner` must not be converted to other partitioners, as doing so 
could cause data inconsistency. In contrast, `ForwardForUnspecifiedPartitioner` 
was introduced specifically to allow safe conversion to other 
partitioners—since it acts as a default forward partitioner without the same 
consistency risks.

> Out of order data seen while running tpc-ds queries
> ---------------------------------------------------
>
>                 Key: FLINK-38817
>                 URL: https://issues.apache.org/jira/browse/FLINK-38817
>             Project: Flink
>          Issue Type: Bug
>          Components: Runtime / Coordination
>    Affects Versions: 2.2.0
>            Reporter: Bonnie Varghese
>            Priority: Major
>         Attachments: screenshot-1.png
>
>
> All unspecified edges are converted to Rescale edges by default for dynamic 
> graphs. Related Jira - https://issues.apache.org/jira/browse/FLINK-25046
> While testing tpc-ds queries I observed that after a global operation the 
> order of the global operation is not preserved due to Rescale edges.
> For SQL batch to work correctly, we should keep Forward edges after a global 
> operation such as `SortLimit` or `Sort `to obtain data correctness and 
> avoiding out of order data.
> I have put my observations and experiments in this doc here:
> [https://docs.google.com/document/d/1TTj2ddlQTfDgtGb0ISmiKWt6R9U4RxJ59o6bULC1YtI/edit?usp=sharing]
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)