wpc009 edited a comment on pull request #18475: URL: https://github.com/apache/flink/pull/18475#issuecomment-1024372868
> As far as I can tell, the case in the 3rd chart is not working in your version. If `Complete` action happens at the very very end (around `to here` arrow), there is no guarantee that it will "complete the newly created future". As you haven't established happens-before relationship between `Reset` and `Complete` actions, `Complete` might see an outdated, already completed, version of the `availableFuture`, failing to wake up the task thread. Can I understand this way? Let's say there is thread1 and thread2. thread 1 running the `getAvailableFuture` function, the thread2 handling the future's completion callback. Consider the following scenario. 1. just before thread2 reach the `availableFuture.complete(null)` line, thread1 finish reseting the `avaiableFuture` and calls `anyOf` on each input's future. 2.thread2 reach the `availableFuture.complete(null)` line, after resolve the `availableFuture` object, but before it calls the complete method. During that time, thread1 finish reseting and `anyOf` calls. In 1, thread1 reseting the `availableFuture` happens before. thread2 resolve varialbe and call the `complete` happens after. So, without the dirty cache inference ( can be avoided by making the `availableFuture` volatile), thread2 will get the fresh new future and complete it. Event if thread2 get the obsolete instance, falls to complete the new future. the thread1 can still make the state right in `anyOf` calls. Since, at the every begining, the input's future is already completed. In 2, thread2 holds an obsolete instance and try to complete it. Still, thread1 cat make the state right in `anyOf`. This is same with the dirty cache scenario. I would say without `volatile` `notifyCompletion` see a obsolete future could happen. But, before entry the `notifyCompletion`, the input's future is already completed. In the `anyOf` function, it will recheck whether the future of the input is completed, and thus triggered the following `availableFuture.complete()`. Stream will not halt. Anyway it's cheap to make the availableFuture `volatile`. In that way, the `Complete` will not able to see the obsolete future even in this scenario. I couldn't see there exists some point that this version will fail, the tests prove that I'm running the `MultipleInputBenchmark` test for hours. No stream hang-up is show up. <img width="297" alt="image" src="https://user-images.githubusercontent.com/2689362/151582292-01aacf62-2bff-4fb7-a160-ec40552e69cd.png";> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
![https://user-images.githubusercontent.com/2689362/151582292-01aacf62-2bff-4fb7-a160-ec40552e69cd.png"](https://user-images.githubusercontent.com/2689362/151582292-01aacf62-2bff-4fb7-a160-ec40552e69cd.png"){kind=link}