xinyuiscool commented on code in PR #24276:
URL: https://github.com/apache/beam/pull/24276#discussion_r1037674483
##########
runners/samza/src/main/java/org/apache/beam/runners/samza/SamzaRunner.java:
##########
@@ -140,9 +141,14 @@ public SamzaPipelineResult run(Pipeline pipeline) {
LOG.info("Beam pipeline JSON graph:\n{}", jsonGraph);
final Map<PValue, String> idMap = PViewToIdMapper.buildIdMap(pipeline);
+ final Map<String, String> multiParDoStateIdMap = new HashMap<>();
Review Comment:
The current impl of invoking the createConfig() to populate this map and
then have another rewrite() to remove some configs and add new configs is quite
awkward. The more elegant impl should be following the existing practice of
building the viewToId map, do a pre-scan of the pipeline to create the
immutable nonUniqueStateIds set. I expect the code looks like the following:
// DO a pre-scan of the pipeline to build this map, and it stays immutable
afterwards.
final Set<String> nonUniqueStateIds = StateIdParser.scan(pipeline);
// Create the store configs in one shot, no rewriting businsess
SamzaPipelineTranslator.createConfig(
pipeline, options, idMap, nonUniqueStateIds, configBuilder);
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
