Sanil15 commented on code in PR #26276:
URL: https://github.com/apache/beam/pull/26276#discussion_r1172940498
##########
runners/samza/src/main/java/org/apache/beam/runners/samza/translation/SamzaPipelineTranslator.java:
##########
@@ -107,6 +114,47 @@ public static void createConfig(
pipeline.traverseTopologically(visitor);
}
+ /**
+ * Builds a map from PTransform to its input and output PValues. The map is
serialized and stored
+ * in the job config.
+ */
+ public static Map<String, Map.Entry<String, String>> buildTransformIOMap(
Review Comment:
Moved it to JSON renderer, it still needs to be a separate scan of the
pipeline using SamzaPipelineVisitor instead of a generic Beam PipelineVisitor
which is used by the JSONRenderer reason being SamzaPipelineVisitor traverses
the pipeline differently (does not enter some composite transforms I if they
can be directly translated). The PValues Maps we need for MetricOp stuff at
runtime need to be populated using this traversal logic from
SamzaPipelineVisitor
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
