wsry opened a new pull request #9993: [FLINK-14498][runtime]Introduce NetworkBufferPool#isAvailable() for interacting with LocalBufferPool. URL: https://github.com/apache/flink/pull/9993 ## What is the purpose of the change If the LocalBufferPool can not request available buffer from NetworkBufferPool, it would wait for 2 seconds before trying to request again in a loop way. Therefore it would bring some delays in practice. To improve this interaction, we introduce NetworkBufferPool#isAvailable to return a future which would be monitored by LocalBufferPool. Then once there are available buffers in NetworkBufferPool, it would complete this future to notify LocalBufferPool immediately. At the same time, the blocking exclusive buffer requests to NetworkBufferPool will be also notified immediately when there are buffers recycled. ## Brief change log - A new interface isAvailable which returns a CompletableFuture is added to the NetworkBufferPool. The future is completed when there are free segments in the pool. - LocalBufferPool will monitor the future returned by isAvailable and will be notified to retry to request MemorySegment from NetworkBufferPool when the future is complete. - Network buffer pool will maintain the state of buffer availability. The future will be transited to uncompleted state when there are no available segments and will be completed once there are available segments. - Two ut cases are added to verify the change. ## Verifying this change Two new ut cases are added to verify the change. One is for the new added isAvailable interface of NetworkBufferPool to verify that the buffer availability is correctly maintained and the future callback is correctly processed. And the other is for the interaction between LocalBufferPool and NetworkBufferPool to verify that blocking requests of multi local buffer pools can be fulfilled by recycled segments to the global network buffer pool. ## Does this pull request potentially affect one of the following parts: - Dependencies (does it add or upgrade a dependency): (yes / **no**) - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: (yes / **no**) - The serializers: (yes / **no** / don't know) - The runtime per-record code paths (performance sensitive): (yes / **no** / don't know) - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Yarn/Mesos, ZooKeeper: (yes / **no** / don't know) - The S3 file system connector: (yes / **no** / don't know) ## Documentation - Does this pull request introduce a new feature? (yes / **no**) - If yes, how is the feature documented? (**not applicable** / docs / JavaDocs / not documented)
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
