fapaul commented on a change in pull request #19073:
URL: https://github.com/apache/flink/pull/19073#discussion_r827833523
##########
File path:
flink-streaming-java/src/main/java/org/apache/flink/streaming/runtime/translators/SinkTransformationTranslator.java
##########
@@ -268,6 +277,26 @@ private void expand() {
+ " has set uid for some operators.");
}
+ // Set the operator uid hashes to support stateful upgrades
without prior uids
+ if (hasWriterUidHash) {
+ if (subTransformation.getName().equals(WRITER_NAME)) {
Review comment:
I talked to @AHeise about this problem offline and we did not find a
ideal solution even if only considering the writer and committer. The only idea
was to pull all writer and committer configuration up to the point where we add
the writer/committer to the datastream and directly manipulate the
transformation. Unfortunately, this still implies matching the transformation
while looping through the newly added transformation because we now would need
to skip the already configured transformation.
Overall, we evaluated the problem again and only occurs as a problem iff
some of the sub topology (pre-writer, pre/post committer) includes a
transformation matching the name of either the writer, committer or global
committer. We do not expect that this problem is likely to occur very often.
##########
File path:
flink-streaming-java/src/main/java/org/apache/flink/streaming/runtime/translators/SinkTransformationTranslator.java
##########
@@ -268,6 +277,26 @@ private void expand() {
+ " has set uid for some operators.");
}
+ // Set the operator uid hashes to support stateful upgrades
without prior uids
+ if (hasWriterUidHash) {
+ if (subTransformation.getName().equals(WRITER_NAME)) {
Review comment:
I talked offline to @AHeise about this problem offline and we did not
find a ideal solution even if only considering the writer and committer. The
only idea was to pull all writer and committer configuration up to the point
where we add the writer/committer to the datastream and directly manipulate the
transformation. Unfortunately, this still implies matching the transformation
while looping through the newly added transformation because we now would need
to skip the already configured transformation.
Overall, we evaluated the problem again and only occurs as a problem iff
some of the sub topology (pre-writer, pre/post committer) includes a
transformation matching the name of either the writer, committer or global
committer. We do not expect that this problem is likely to occur very often.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
