mosche commented on code in PR #24009:
URL: https://github.com/apache/beam/pull/24009#discussion_r1024996169
##########
runners/spark/3/src/main/java/org/apache/beam/runners/spark/structuredstreaming/translation/batch/PipelineTranslatorBatch.java:
##########
@@ -81,27 +80,13 @@ public class PipelineTranslatorBatch extends
PipelineTranslator {
TRANSFORM_TRANSLATORS.put(
SplittableParDo.PrimitiveBoundedRead.class, new
ReadSourceTranslatorBatch<>());
-
- TRANSFORM_TRANSLATORS.put(
- View.CreatePCollectionView.class, new
CreatePCollectionViewTranslatorBatch<>());
- }
-
- public PipelineTranslatorBatch(SparkStructuredStreamingPipelineOptions
options) {
- translationContext = new TranslationContext(options);
}
- /** Returns a translator for the given node, if it is possible, otherwise
null. */
+ /** Returns a {@link TransformTranslator} for the given {@link PTransform}
if known. */
@Override
@Nullable
protected <InT extends PInput, OutT extends POutput, TransformT extends
PTransform<InT, OutT>>
- TransformTranslator<InT, OutT, TransformT> getTransformTranslator(
- @Nullable TransformT transform) {
- // Root of the graph is null
- if (transform == null) {
- return null;
- }
- TransformTranslator<InT, OutT, TransformT> translator =
- TRANSFORM_TRANSLATORS.get(transform.getClass());
- return translator != null && translator.canTranslate(transform) ?
translator : null;
Review Comment:
exactly 👍
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
