[
https://issues.apache.org/jira/browse/BEAM-11094?focusedWorklogId=506422&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-506422
]
ASF GitHub Bot logged work on BEAM-11094:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 29/Oct/20 21:28
Start Date: 29/Oct/20 21:28
Worklog Time Spent: 10m
Work Description: kennknowles commented on a change in pull request
#13118:
URL: https://github.com/apache/beam/pull/13118#discussion_r514578489
##########
File path:
sdks/java/core/src/main/java/org/apache/beam/sdk/runners/TransformHierarchy.java
##########
@@ -483,12 +402,12 @@ private void setOutput(POutput output) {
checkState(
this.outputs == null, "Tried to specify more than one output for
%s", getFullName());
checkNotNull(output, "Tried to set the output of %s to null",
getFullName());
- this.outputs = output.expand();
+ this.outputs = PValues.fullyExpand(output.expand());
Review comment:
I think this line and the similar one below below made it so a
`PCollectionView` is not in `outputs`, so topological traversal does not
eagerly process `PCollectionView` as a value, and PCollectionViews also do not
have a producer (which is correct for the Beam model). This seems to affect
only Dataflow and only because it has special logic which treats
PCollectionView as a value in its v1beta3 translation.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 506422)
Time Spent: 50m (was: 40m)
> Only work with fully expanded PCollections in TransformHierarchy and
> PTransformReplacements
> -------------------------------------------------------------------------------------------
>
> Key: BEAM-11094
> URL: https://issues.apache.org/jira/browse/BEAM-11094
> Project: Beam
> Issue Type: Improvement
> Components: sdk-java-core
> Reporter: Kenneth Knowles
> Assignee: Kenneth Knowles
> Priority: P2
> Time Spent: 50m
> Remaining Estimate: 0h
>
> While developing #13053 I discovered that some output replacements were not
> being re-wired correctly. The current types in output replacements would
> allow false misses when two PValues were not expanded to the same degree.
> Fully expanding all PValues to PCollections is always allowed, always
> correct, and eliminates this class of errors.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
