[GitHub] [flink] wpc009 commented on pull request #18475: [FLINK-25728] Protential memory leaks in StreamMultipleInputProcessor

wpc009 commented on pull request #18475: URL: https://github.com/apache/flink/pull/18475#issuecomment-1024817221 I still believe this is a working fix. And no more complex than yours. They are on the same level. If the decision is made, then all I can say is that it takes a lot efforts

[GitHub] [flink] wpc009 commented on pull request #18475: [FLINK-25728] Protential memory leaks in StreamMultipleInputProcessor

wpc009 commented on pull request #18475: URL: https://github.com/apache/flink/pull/18475#issuecomment-1024815295 If the `Complete` indicates the input future completion. (Thats a little different in my graph. In the graph it refers to the line the calls the `complete` on the result

[GitHub] [flink] wpc009 commented on pull request #18475: [FLINK-25728] Protential memory leaks in StreamMultipleInputProcessor

wpc009 commented on pull request #18475: URL: https://github.com/apache/flink/pull/18475#issuecomment-1024448903 I can make test for the scenario you described which my fix will fail. Just manually create a gap, resolve the reference first, then a long sleep, then try to complete. During

[GitHub] [flink] wpc009 commented on pull request #18475: [FLINK-25728] Protential memory leaks in StreamMultipleInputProcessor

wpc009 commented on pull request #18475: URL: https://github.com/apache/flink/pull/18475#issuecomment-102960 In your version, under the same circumtance, `anyAvailable.get()` may get the obsolete object either. `AtomicReference` can not guranteen the `happen-before` relationship you

[GitHub] [flink] wpc009 commented on pull request #18475: [FLINK-25728] Protential memory leaks in StreamMultipleInputProcessor

wpc009 commented on pull request #18475: URL: https://github.com/apache/flink/pull/18475#issuecomment-1024372868 > As far as I can tell, the case in the 3rd chart is not working in your version. If `Complete` action happens at the very very end (around `to here` arrow), there is no

[GitHub] [flink] wpc009 commented on pull request #18475: [FLINK-25728] Protential memory leaks in StreamMultipleInputProcessor

wpc009 commented on pull request #18475: URL: https://github.com/apache/flink/pull/18475#issuecomment-1024349502 BellSoft Liberica jdk8u292. https://user-images.githubusercontent.com/2689362/151578190-851c24c5-f7ba-4f4d-b1b7-03ea96e45fd6.png;> -- This is an automated message from

[GitHub] [flink] wpc009 commented on pull request #18475: [FLINK-25728] Protential memory leaks in StreamMultipleInputProcessor

wpc009 commented on pull request #18475: URL: https://github.com/apache/flink/pull/18475#issuecomment-1024191941 I made a graph to explain how this fix works. Even with the race condition between future's `notifyCompletion` and the `checkReusableAndReset`. ![flink-issue-timeline

[GitHub] [flink] wpc009 commented on pull request #18475: [FLINK-25728] Protential memory leaks in StreamMultipleInputProcessor

wpc009 commented on pull request #18475: URL: https://github.com/apache/flink/pull/18475#issuecomment-102405 > > @wpc009 . Could you check my PR if it's solving the memory leak? My proposal is quite a bit simpler. > > Regarding the tests, unfortunately I don't

[GitHub] [flink] wpc009 commented on pull request #18475: [FLINK-25728] Protential memory leaks in StreamMultipleInputProcessor

wpc009 commented on pull request #18475: URL: https://github.com/apache/flink/pull/18475#issuecomment-1024041597 > @wpc009 . Could you check my PR if it's solving the memory leak? My proposal is quite a bit simpler. > > Regarding the tests, unfortunately I don't see a way for

[GitHub] [flink] wpc009 commented on pull request #18475: [FLINK-25728] Protential memory leaks in StreamMultipleInputProcessor

wpc009 commented on pull request #18475: URL: https://github.com/apache/flink/pull/18475#issuecomment-1023958486 Here is some benchmark results (From my laptop) | Benchmark | Mode | Cnt | Score| Error | Units

[GitHub] [flink] wpc009 commented on pull request #18475: [FLINK-25728] Protential memory leaks in StreamMultipleInputProcessor

wpc009 commented on pull request #18475: URL: https://github.com/apache/flink/pull/18475#issuecomment-1023885086 I have a fix. Tested on the `flink-benchmark`. I'm preparing some end-to-end test. I can push it at the end of today. -- This is an automated message from the Apache Git

[GitHub] [flink] wpc009 commented on pull request #18475: [FLINK-25728] Protential memory leaks in StreamMultipleInputProcessor

wpc009 commented on pull request #18475: URL: https://github.com/apache/flink/pull/18475#issuecomment-1022813985 It seems like newer JDK release have already fixed the memory issue of `CompletableFuture.anyOf`. Saddly, this issue is still exists in jdk8. It's time to abandon this old

[GitHub] [flink] wpc009 commented on pull request #18475: [FLINK-25728] Protential memory leaks in StreamMultipleInputProcessor

wpc009 commented on pull request #18475: URL: https://github.com/apache/flink/pull/18475#issuecomment-1021816172 Maybe I should push to `release-1.14` branch? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [flink] wpc009 commented on pull request #18475: [FLINK-25728] Protential memory leaks in StreamMultipleInputProcessor

wpc009 commented on pull request #18475: URL: https://github.com/apache/flink/pull/18475#issuecomment-1021815826 The master branch can not pass the CI pipeline either. Check this out. https://dev.azure.com/wpc009/Flink/_build/results?buildId=5=results -- This is an automated

[GitHub] [flink] wpc009 commented on pull request #18475: [FLINK-25728] Protential memory leaks in StreamMultipleInputProcessor

wpc009 commented on pull request #18475: URL: https://github.com/apache/flink/pull/18475#issuecomment-1021431119 > I'm afraid that those "time outs" might be because of some bug in this code causing a deadlock. I've also tried [to run a benchmark

[GitHub] [flink] wpc009 commented on pull request #18475: [FLINK-25728] Protential memory leaks in StreamMultipleInputProcessor

wpc009 commented on pull request #18475: URL: https://github.com/apache/flink/pull/18475#issuecomment-1020772870 > ## CI report: > * [3ade971](https://github.com/apache/flink/commit/3ade9719b565beeacf7761ab71d5abb7ba62e62a) Azure:

[GitHub] [flink] wpc009 commented on pull request #18475: [FLINK-25728] Protential memory leaks in StreamMultipleInputProcessor

wpc009 commented on pull request #18475: URL: https://github.com/apache/flink/pull/18475#issuecomment-1020772870 > ## CI report: > * [3ade971](https://github.com/apache/flink/commit/3ade9719b565beeacf7761ab71d5abb7ba62e62a) Azure: