youngoli commented on code in PR #21979:
URL: https://github.com/apache/beam/pull/21979#discussion_r911540966
##########
sdks/go/test/integration/io/fhirio/fhirio_test.go:
##########
@@ -177,6 +178,22 @@ func extractResourcePathFrom(resourceLocationUrl string)
(string, error) {
return resourceLocationUrl[startIdx:endIdx], nil
}
+// Useful to prevent flaky results.
+func runWithBackoffRetries(t *testing.T, p *beam.Pipeline) error {
Review Comment:
I'm still hesitating with this one only because rerunning a pipeline on
Dataflow is _slow_. This isn't a problem on the Direct runner, but on Dataflow
rerunning a pipeline is around a 5 minute affair even for a really simple
pipeline.
So this function will work. In the sense that 5 minutes is definitely going
to be long enough to fix the flakiness. But waiting a guaranteed 30 seconds
before running the pipeline is probably still better overall.
By the way, I should probably mention that tests within a package in Go run
in sequence by default. So the reason I'm being very conscious of every
additional integration test in this file is because each one will make the
entire test suite take ~5 minutes longer to finish (on Dataflow at least).
Anyway, I'm still gonna merge this because this isn't bad enough to block
anything. But if you can think of any way to fix the flakiness without
rerunning the whole pipeline it would probably be much better for performance
on Dataflow.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
