[
https://issues.apache.org/jira/browse/FLINK-19325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17237698#comment-17237698
]
Arvid Heise commented on FLINK-19325:
-------------------------------------
Merged into master as 640d7224251cb31f5630bc475df8804714dc13b1.
> Optimize the consumed time for checkpoint completion
> ----------------------------------------------------
>
> Key: FLINK-19325
> URL: https://issues.apache.org/jira/browse/FLINK-19325
> Project: Flink
> Issue Type: Improvement
> Components: Runtime / Checkpointing
> Affects Versions: 1.12.0, 1.11.2
> Reporter: Congxian Qiu
> Assignee: Arvid Heise
> Priority: Major
> Labels: pull-request-available
> Fix For: 1.12.0
>
>
> Currently when completing a checkpoint, we'll write out the state handle out
> in {{MetadataV2V3SerializerBase.java#serializeStreamStateHandle}}
> {code:java}
> static void serializeStreamStateHandle(StreamStateHandle stateHandle,
> DataOutputStream dos) throws IOException {
> if (stateHandle == null) {
> dos.writeByte(NULL_HANDLE);
> } else if (stateHandle instanceof RelativeFileStateHandle) {
> dos.writeByte(RELATIVE_STREAM_STATE_HANDLE);
> RelativeFileStateHandle relativeFileStateHandle =
> (RelativeFileStateHandle) stateHandle;
> dos.writeUTF(relativeFileStateHandle.getRelativePath());
> dos.writeLong(relativeFileStateHandle.getStateSize());
> } else if (stateHandle instanceof FileStateHandle) {
> dos.writeByte(FILE_STREAM_STATE_HANDLE);
> FileStateHandle fileStateHandle = (FileStateHandle) stateHandle;
> dos.writeLong(stateHandle.getStateSize());
> dos.writeUTF(fileStateHandle.getFilePath().toString());
> } else if (stateHandle instanceof ByteStreamStateHandle) {
> dos.writeByte(BYTE_STREAM_STATE_HANDLE);
> ByteStreamStateHandle byteStreamStateHandle = (ByteStreamStateHandle)
> stateHandle;
> dos.writeUTF(byteStreamStateHandle.getHandleName());
> byte[] internalData = byteStreamStateHandle.getData();
> dos.writeInt(internalData.length);
> dos.write(byteStreamStateHandle.getData());
> } else {
> throw new IOException("Unknown implementation of StreamStateHandle: " +
> stateHandle.getClass());
> }
> dos.flush();
> }
> {code}
> We'll call {{dos.flush()}} after every state handle written out. But this may
> consume too much time and is not needed, because we'll close the outputstream
> after all things have been written out.
> I propose to remove the {{dos.flush()}} here to optimize the consumed time
> for checkpoint completion
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
