xintongsong opened a new pull request #9339: [FLINK-13555][runtime] SlotPool fails batch slot requests immediately if they are unfulfillable. URL: https://github.com/apache/flink/pull/9339 ## What is the purpose of the change This PR make SlotPool to fail batch slot requests immediately if they are unfulfillable, which means they cannot be fulfilled by any slot (available or not) in the cluster. There are two kinds of request slots from RM failures: 1. RM does not have available slots. All slots are in use at the moment. But they might become available later when the currently running tasks finish. 2. The slot request requires too many resources that can not be fulfilled by any slot (available or not) in the cluster. The request is also not likely to be fulfilled later. For the 2nd kinds of failures, it doesn't make sense to wait for the timeout. SlotPool should fail the job immediately, with proper error messages describing the problem and suggesting the user to tune job or cluster configurations. ## Brief change log - 6af614c1647429d76c8514a67e7c5439e90a50e1, 8b5e9ff88e6052dc5845a9fcff684261c697899f: Cleanups for existing codes. - dbdace5ae68331bd1203525425b357639e334752: Remove `SlotManagerException`, replace with its base class `ResourceException`. - bb2c7c92fdbafe3644ab76041b300a23a6c87744: Introduce `UnfulfillableSlotRequestException` and fail pending batch slot requests on it. - dfe5ef072dadfc0e1510f1923911304c635e54af: Add test cases for failing unfulfillable batch slot requests. ## Verifying this change This change added tests and can be verified as follows: - Add `SlotPoolBatchSlotRequestTest#testPendingBatchSlotRequestFailsIfAllocationFailsUnfulfillable` that validates that pending batch slot requests fail when allocation fail with unfulfillable exception. - Add `SlotPoolBatchSlotRequestTest#testPendingBatchSlotRequestFailsIfRMRequestFailsUnfulfillable` that validates that pending batch slot requests fail when requesting slots from RM fail with unfulfillable exception. ## Does this pull request potentially affect one of the following parts: - Dependencies (does it add or upgrade a dependency): (no) - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: (no) - The serializers: (no) - The runtime per-record code paths (performance sensitive): (no) - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Yarn/Mesos, ZooKeeper: (yes) - The S3 file system connector: (no) ## Documentation - Does this pull request introduce a new feature? (no) - If yes, how is the feature documented? (not applicable)
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
