Samrat Deb created FLINK-39778:
----------------------------------
Summary: Recoverable writer silently loses the in-flight tail on
resume
Key: FLINK-39778
URL: https://issues.apache.org/jira/browse/FLINK-39778
Project: Flink
Issue Type: Sub-task
Components: Connectors / FileSystem
Reporter: Samrat Deb
Fix For: 2.4.0
persist() uploads the un-parted tail to a side object
<key>/.incomplete/<uploadId>/<uuid> and records incompleteObjectName in the
NativeS3Recoverable. But recover() rebuilds
the stream from parts() + numBytesInParts() only —
s3recoverable.incompleteObjectName() is never read:
return new NativeS3RecoverableFsDataOutputStream(
s3AccessHelper, s3recoverable.getObjectName(), s3recoverable.uploadId(),
localTmpDir, userDefinedMinPartSize,
s3recoverable.parts(), s3recoverable.numBytesInParts());
After resume from a checkpoint with a partial part, those bytes are silently
dropped — exactly-once degrades to "at-most-once minus tail."
cleanupRecoverableState() deletes
the side object, confirming the intent; recover() just forgot to read it back.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
