zhuzhurk commented on a change in pull request #7255: [FLINK-10945] Use
InputDependencyConstraint to avoid resource dead…
URL: https://github.com/apache/flink/pull/7255#discussion_r241636524
##########
File path:
flink-runtime/src/main/java/org/apache/flink/runtime/executiongraph/Execution.java
##########
@@ -755,23 +775,10 @@ else if (numConsumers == 0) {
// TODO The current approach may send many
update messages even though the consuming
// task has already been deployed with all
necessary information. We have to check
// whether this is a problem and fix it, if it
is.
- CompletableFuture.supplyAsync(
- () -> {
- try {
- final ExecutionGraph
executionGraph = consumerVertex.getExecutionGraph();
-
consumerVertex.scheduleForExecution(
-
executionGraph.getSlotProvider(),
-
executionGraph.isQueuedSchedulingAllowed(),
-
LocationPreferenceConstraint.ANY, // there must be at least one known location
-
Collections.emptySet());
- } catch (Throwable t) {
- consumerVertex.fail(new
IllegalStateException("Could not schedule consumer " +
- "vertex
" + consumerVertex, t));
- }
-
- return null;
- },
- executor);
+ if
(consumerVertex.checkInputDependencyConstraints()) {
Review comment:
From my understanding, the TODO comment is related to the "consumerState ==
CREATED" section in scheduleOrUpdateConsumers, which invokes cachePartitionInfo
first and then schedules the vertex. The cachePartitionInfo action is needed to
avoid deployment race, at the cost of redundant partition infos to update to
task, which is the concern as described in the TODO comment.
So far the redundant partition it's not a big problem. But I think we can
optimize it later. One possible solution in my mind is to remove known
partition infos from the cache when creating InputChannelDeploymentDescriptor.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
