StephanEwen commented on a change in pull request #12122:
URL: https://github.com/apache/flink/pull/12122#discussion_r425118547
##########
File path:
flink-streaming-java/src/main/java/org/apache/flink/streaming/api/graph/StreamGraph.java
##########
@@ -266,7 +268,10 @@ public boolean isIterative() {
String operatorName) {
if (operatorFactory.isStreamSource()) {
- addNode(vertexID, slotSharingGroup, coLocationGroup,
SourceStreamTask.class, operatorFactory, operatorName);
+ Class<? extends AbstractInvokable> invokableClass =
Review comment:
This looks fragile to me. See the top-level comment about an idea for a
cleaner way of handling this.
##########
File path:
flink-streaming-java/src/main/java/org/apache/flink/streaming/api/graph/StreamingJobGraphGenerator.java
##########
@@ -432,6 +434,20 @@ private StreamConfig createJobVertex(
chainedNames.get(streamNodeId),
jobVertexId,
operatorIDPairs);
+ if (streamNode.getOperatorFactory() instanceof
CoordinatedOperatorFactory) {
+ OperatorCoordinator.Provider
coordinatorProvider =
+
((CoordinatedOperatorFactory<?>) streamNode
+
.getOperatorFactory())
+
.getCoordinatorProvider(
+
chainedNames.get(streamNodeId),
+
new OperatorID(hashes.get(streamNodeId)));
+ try {
+ jobVertex.addOperatorCoordinator(new
SerializedValue<>(coordinatorProvider));
+ } catch (IOException e) {
+ throw new
FlinkRuntimeException(String.format(
+ "Coordinator Provider
for node %s is not serializable.", chainedNames.get(streamNodeId)));
+ }
+ }
Review comment:
I think @TsReaper's comment here is correct.
The proposed solution is a bit hacky, but a clean solution would need some
major prerequisite cleanup of the whole StreamGraph, and
StreamingJobGraphGenerator, which would be a separate bigger effort.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
