xuyangzhong commented on PR #27699: URL: https://github.com/apache/flink/pull/27699#issuecomment-3976322921
> @xuyangzhong could you say more about `Allow calc that contains non-deterministic to consume duplicate changes `please? Do I understanding correctly - I assume the idea is that if we create duplicate results then it is now deterministic, as opposed to one result replacing another. It would be useful to detail here or in the ira some simple example SQL that the delta join cannot currently handle but can now handle after this change. So this this issue could be found by searching. > > I have looked at the flip and jira and cannot see more detail around the deterministic issue. Does it relate to the Flip when it says "a Deduplicate node will be introduced after the StreamPhysicalDeltaJoin to convert the +U records back to -U and +U records.". The flip does not talk about this non-deterministic issue. Thanks for the reminder. I've added the purpose of this PR to the Jira description. Yes, the FLIP lacks detailed documentation on this specific aspect. It primarily outlines the changes to the public user API, but does not provide a detailed and comprehensive list of what is supported versus unsupported in the future. Initially, out of an abundance of caution, support for non-deterministic functions was disabled. As Delta Join continues to evolve, a series of related extension efforts are now being progressively rolled out. They can all be categorized as part of the future work. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
