rkhachatryan commented on a change in pull request #17774:
URL: https://github.com/apache/flink/pull/17774#discussion_r769532023
##########
File path:
flink-state-backends/flink-statebackend-rocksdb/src/main/java/org/apache/flink/contrib/streaming/state/snapshot/RocksIncrementalSnapshotStrategy.java
##########
@@ -97,12 +98,23 @@
/**
* Stores the materialized sstable files from all snapshots that build the
incremental history.
+ * Used to check whether {@link PlaceholderStreamStateHandle} can be sent
or the original {@link
+ * StreamStateHandle} must be used.
*/
- @Nonnull private final SortedMap<Long, Set<StateHandleID>>
materializedSstFiles;
+ @Nonnull private final SortedMap<Long, Set<StateHandleID>>
uploadedStateIDs;
+
+ /**
+ * Last uploaded but potentially not confirmed SST files. Used if {@link
#uploadedStateIDs}
+ * doesn't contain the corresponding {@link StateHandleID}.
+ */
+ @Nonnull private final Map<StateHandleID, StreamStateHandle>
lastUploadedSstFiles;
Review comment:
Do you mean that if multiple checkpoint fail contigiously, TM would
re-upload already uploaded files?
I think that's not true: once the files are uploaded, handles are added to
`lastUploadedSstFiles`.
On abortion, it is **not** cleared.
On checkpoint, JM confirmation is **not** required to re-use these
previously uploaded files.
So, `PreviousSnapshot.getUploaded` will return those pre-uploaded files,
regardless of confirmation or abortion.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
