[
https://issues.apache.org/jira/browse/BEAM-3913?focusedWorklogId=83367&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-83367
]
ASF GitHub Bot logged work on BEAM-3913:
----------------------------------------
Author: ASF GitHub Bot
Created on: 22/Mar/18 22:24
Start Date: 22/Mar/18 22:24
Worklog Time Spent: 10m
Work Description: lukecwik commented on a change in pull request #4938:
[BEAM-3913] Allow Fusion to Continue with unknown PTransforms
URL: https://github.com/apache/beam/pull/4938#discussion_r176589982
##########
File path:
runners/core-construction-java/src/main/java/org/apache/beam/runners/core/construction/graph/GreedyPCollectionFusers.java
##########
@@ -276,16 +281,27 @@ private static boolean unknownTransformFusion(
@SuppressWarnings("unused") Environment environment,
@SuppressWarnings("unused") Collection<PCollectionNode>
stagePCollections,
@SuppressWarnings("unused") QueryablePipeline pipeline) {
- throw new IllegalArgumentException(
- String.format("Unknown URN %s",
transform.getTransform().getSpec().getUrn()));
+ LOG.debug(
+ "Unknown {} {} will not fuse into an existing {}",
+ PTransform.class.getSimpleName(),
+ transform.getTransform(),
+ ExecutableStage.class.getSimpleName(),
+ PTransform.class.getSimpleName());
+ return false;
}
+ // Things with unknown URNs either execute within their own stage or are
executed by the runner.
+ // In either case, assume the
Review comment:
comment is incomplete.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 83367)
Time Spent: 20m (was: 10m)
> Support custom runner primitives in Fusion
> ------------------------------------------
>
> Key: BEAM-3913
> URL: https://issues.apache.org/jira/browse/BEAM-3913
> Project: Beam
> Issue Type: Improvement
> Components: runner-core
> Reporter: Thomas Groh
> Assignee: Thomas Groh
> Priority: Major
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Effectively, register runner-provided transform URNs in the
> GreedyPCollectionFusers to understand runner-specific primitives, and don't
> ever fuse them.
>
> It's probably appropriate to continue to crash when encountering truly
> unknown URNs
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
