fredia commented on code in PR #24653:
URL: https://github.com/apache/flink/pull/24653#discussion_r1590765393
##########
flink-streaming-java/src/main/java/org/apache/flink/streaming/runtime/tasks/SubtaskCheckpointCoordinatorImpl.java:
##########
@@ -234,16 +230,15 @@ class SubtaskCheckpointCoordinatorImpl implements
SubtaskCheckpointCoordinator {
private static ChannelStateWriter openChannelStateWriter(
String taskName,
- CheckpointStorage checkpointStorage,
- Environment env,
- int maxSubtasksPerChannelStateFile) {
+ CheckpointStorageWorkerView checkpointStorageWorkerView,
+ Environment env) {
return new ChannelStateWriterImpl(
env.getJobVertexId(),
taskName,
env.getTaskInfo().getIndexOfThisSubtask(),
- checkpointStorage,
+ checkpointStorageWorkerView,
env.getChannelStateExecutorFactory(),
- maxSubtasksPerChannelStateFile);
+ 1 /* one logical channel state file/executor per subtask */);
Review Comment:
> If we hard code maxSubtasksPerChannelStateFile is 1, it means
execution.checkpointing.unaligned.max-subtasks-per-channel-state-file doesn't
take effect after this PR, right?
Right.
In fact, both merge mechanisms can work simultaneously. The `unified file
merging mechanism` allows different output streams to be written to a physical
file, and the current merging mechanism allows multiple subtasks to be written
to one output stream.
I will keep
`execution.checkpointing.unaligned.max-subtasks-per-channel-state-file` first.
- If
`execution.checkpointing.unaligned.max-subtasks-per-channel-state-file:1` and
`state.checkpoints.file-merging.enabled: true`, only the `unified file merging
mechanism` takes effect.
- if `execution.checkpointing.unaligned.max-subtasks-per-channel-state-file
> 1` and `state.checkpoints.file-merging.enabled: false`, only the current
mechanism takes effect.
- if `execution.checkpointing.unaligned.max-subtasks-per-channel-state-file
> 1` and `state.checkpoints.file-merging.enabled: true`, both two mechanism
take effect.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
