[
https://issues.apache.org/jira/browse/FLINK-39778?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated FLINK-39778:
-----------------------------------
Labels: pull-request-available (was: )
> Recoverable writer silently loses the in-flight tail on resume
> --------------------------------------------------------------
>
> Key: FLINK-39778
> URL: https://issues.apache.org/jira/browse/FLINK-39778
> Project: Flink
> Issue Type: Sub-task
> Components: Connectors / FileSystem
> Reporter: Samrat Deb
> Priority: Major
> Labels: pull-request-available
> Fix For: 2.4.0
>
>
> persist() uploads the un-parted tail to a side object
> <key>/.incomplete/<uploadId>/<uuid> and records incompleteObjectName in the
> NativeS3Recoverable. But recover() rebuilds
> the stream from parts() + numBytesInParts() only —
> s3recoverable.incompleteObjectName() is never read:
> return new NativeS3RecoverableFsDataOutputStream(
> s3AccessHelper, s3recoverable.getObjectName(), s3recoverable.uploadId(),
> localTmpDir, userDefinedMinPartSize,
> s3recoverable.parts(), s3recoverable.numBytesInParts());
> After resume from a checkpoint with a partial part, those bytes are silently
> dropped — exactly-once degrades to "at-most-once minus tail."
> cleanupRecoverableState() deletes
> the side object, confirming the intent; recover() just forgot to read it back.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
