kl0u opened a new pull request #8052: [FLINK-11458][checkpointing, rest] Add TERMINATE/SUSPEND Job with Savepoint (FLIP-34) URL: https://github.com/apache/flink/pull/8052 ## What is the purpose of the change This is a first (almost complete) implementation of the FLIP-34 effort. FLIP-34 can be found here https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=103090212 and proposes the addition of the `suspend` and `drain` commands to the `CLI` and the `REST` API in general. With this addition, the user will be able to write to the CLI: `flink stop -s <savepoint_dir> JOB_ID` for suspending a job with a savepoint, and `flink stop -s <savepoint_dir> -d JOB_ID` from draining the pipeline. ## Brief change log Not all changes/additions can be described here. A list of the most important changes follows: * Adds the `CheckpointType.SYNC_SAVEPOINT` for synchronous savepoints * Adds the `SynchronousSavepointLatch` which blocks the checkpointing thread of the `Task` until also the `notifyCheckpointComplete` is successfully executed. This will guarantee that in the case of a synchronous checkpoint, the task will wait for the notify callback to be completed, before the task can resume execution. * Adds the `advanceToEndOfTime` and propagates it from the JM and the `CheckpointCoordinator` to the TM. * Adds the adequate REST commands and handlers to expose this functionality through the REST API. ## Verifying this change This change added multiple tests and ITCases. In addition, this addition was manually verified locally. ## Does this pull request potentially affect one of the following parts: - Dependencies (does it add or upgrade a dependency): (yes / **no**) - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: (**yes** / no) - The serializers: (yes / **no** / don't know) - The runtime per-record code paths (performance sensitive): (yes / **no** / don't know) - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Yarn/Mesos, ZooKeeper: (**yes** / no / don't know) (adds the new `suspend` and `terminate` commands. - The S3 file system connector: (yes / **no** / don't know) ## Documentation - Does this pull request introduce a new feature? (**yes** / no) - If yes, how is the feature documented? (not applicable / docs / JavaDocs / **not documented yet**) ## NOTE TO REVIEWERS These changes have not been tested yet with: - The `AsyncIO` operations. - The `FileMonitoringSource`. - Any other operator/function that does not strictly respect the contract of the exposed APIs, e.g. any sink that does asynchronous work in the `notifyOnCheckpointComplete` and does not wait for its completion.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
