lostluck commented on code in PR #34641:
URL: https://github.com/apache/beam/pull/34641#discussion_r2047778848
##########
sdks/go/pkg/beam/runners/prism/internal/preprocess.go:
##########
@@ -492,6 +492,9 @@ func finalizeStage(stg *stage, comps *pipepb.Components,
pipelineFacts *fusionFa
}
stg.internalCols = internal
+ // Sort the keys of internal producers (from stageFacts.PcolProducers)
+ // to ensure deterministic order for stable tests.
+ sort.Strings(stg.internalCols)
Review Comment:
Good find! I thought I had everything deterministic already.
##########
sdks/go/pkg/beam/runners/prism/internal/handlerunner.go:
##########
@@ -88,8 +88,7 @@ func (h *runner) PrepareTransform(tid string, t
*pipepb.PTransform, comps *pipep
}
func (h *runner) handleFlatten(tid string, t *pipepb.PTransform, comps
*pipepb.Components) prepareResult {
- if !h.config.SDKFlatten {
- t.EnvironmentId = "" // force the flatten to be a
runner transform due to configuration.
+ if !h.config.SDKFlatten && !strings.HasPrefix(tid, "ft_") {
Review Comment:
I'll note that there's no user serviceable way to do these configurations at
the moment, and it really was a hard binary. It would be acceptable to remove
the SDKFlatten option in favour of just a single approach that biases to runner
flattens, but does the SDK flatten to get around these issues.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscr...@beam.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
