klion26 commented on a change in pull request #7351: [FLINK-11008][State
Backends, Checkpointing]SpeedUp upload state files using multithread
URL: https://github.com/apache/flink/pull/7351#discussion_r248333307
##########
File path:
flink-state-backends/flink-statebackend-rocksdb/src/main/java/org/apache/flink/contrib/streaming/state/RocksDBKeyedStateBackend.java
##########
@@ -218,7 +218,10 @@
private final boolean enableIncrementalCheckpointing;
/** Thread number used to download from DFS when restore. */
- private final int restoringThreadNum;
+ private final int numberOfRestoringThreads;
+
+ /** Thread number used to upload to DFS when snapshot. */
+ private final int numberOfSnapshottingThreads;
Review comment:
@StefanRRichter In my opinion, there may be a scenario that we use
multithread to restore from the checkpoint but do not need to use multithread
to snapshot when incremental checkpoint.
In my company, we only enabled multithread restoring. we use only
RocksDBIncremental Checkpoint to snapshot and restore from the latest completed
checkpoint(we do not use savepoint to restore the job and do not use
HeapStateBackend in the product).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
