kennknowles commented on code in PR #31450:
URL: https://github.com/apache/beam/pull/31450#discussion_r1621053411
##########
runners/google-cloud-dataflow-java/src/main/java/org/apache/beam/runners/dataflow/DataflowPipelineTranslator.java:
##########
@@ -592,6 +593,14 @@ public CompositeBehavior
enterCompositeTransform(TransformHierarchy.Node node) {
if (!node.isRootNode()) {
parents.addFirst(node);
}
+ PTransform<?, ?> transform = node.getTransform();
+ if (transform != null) {
Review Comment:
I don't think this can be null.
##########
runners/google-cloud-dataflow-java/src/main/java/org/apache/beam/runners/dataflow/DataflowPipelineTranslator.java:
##########
@@ -954,6 +963,41 @@ private <K, V> void redistributeByKeyHelper(
}
});
+ registerTransformTranslator(
+ RedistributeArbitrarily.class,
+ new TransformTranslator<RedistributeArbitrarily>() {
+ @Override
+ public void translate(RedistributeArbitrarily transform,
TranslationContext context) {
+ redistributeArbitrarilyHelper(transform, context);
+ }
+
+ private <T> void redistributeArbitrarilyHelper(
+ RedistributeArbitrarily<T> transform, TranslationContext
context) {
+ StepTranslationContext stepContext = context.addStep(transform,
"GroupByKey");
Review Comment:
This is a problem because the input is not necessarily keyed. I think the
change you made above should make the RedistributeByKey translator suffice
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscr...@beam.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
