zhuzhurk commented on a change in pull request #10832:
[FLINK-14163][runtime]Enforce synchronous registration of
Execution#producedPartitions
URL: https://github.com/apache/flink/pull/10832#discussion_r365723610
##########
File path:
flink-runtime/src/main/java/org/apache/flink/runtime/executiongraph/Execution.java
##########
@@ -605,6 +606,26 @@ public void setInitialState(@Nullable
JobManagerTaskRestore taskRestore) {
});
}
+ /**
+ * Register producedPartitions to {@link ShuffleMaster}
+ *
+ * <p>HACK: Please notice that this method simulates asynchronous
registration in a synchronous way
+ * by making sure the returned {@link CompletableFuture} from {@link
ShuffleMaster#registerPartitionWithProducer}
+ * is done immediately.
+ *
+ * <p>{@link Execution#producedPartitions} are registered through an
asynchronous interface
+ * {@link ShuffleMaster#registerPartitionWithProducer} to {@link
ShuffleMaster}, however they are not always
+ * accessed through callbacks. So, it is possible that {@link
Execution#producedPartitions}
+ * have not been available yet when accessed (in {@link
Execution#deploy} for example).
+ *
+ * <p>Since the only implementation of {@link ShuffleMaster} is {@link
NettyShuffleMaster},
+ * which indeed registers producedPartition in a synchronous way, hence
this method enforces
+ * synchronous registration under an asynchronous interface for now.
+ *
+ * <p>If asynchronous registration is needed in the future, use
callbacks to access {@link Execution#producedPartitions}.
Review comment:
This is a temporary fix. So we'd better have a TODO here as a reminder for
follow up fixes.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
