[
https://issues.apache.org/jira/browse/BEAM-3408?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thomas Groh reassigned BEAM-3408:
---------------------------------
Assignee: (was: Thomas Groh)
> PackageUtil should retry check for whether file is already staged
> -----------------------------------------------------------------
>
> Key: BEAM-3408
> URL: https://issues.apache.org/jira/browse/BEAM-3408
> Project: Beam
> Issue Type: Bug
> Components: runner-dataflow
> Reporter: Henning Rohde
> Priority: Major
>
> The code to stage files retries the upload, but the existence (and size)
> check is attempted only once:
> https://github.com/apache/beam/blob/dc3a664e3b4b4f63500954aae25dce0cb5f8b314/runners/google-cloud-dataflow-java/src/main/java/org/apache/beam/runners/dataflow/util/PackageUtil.java#L193
> This causes post-commit tests to fail sometimes:
> https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow/org.apache.beam$beam-runners-google-cloud-dataflow-java/4632/testReport/junit/org.apache.beam.sdk.io/BoundedReadFromUnboundedSourceTest/testDedup/
> https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow/org.apache.beam$beam-runners-google-cloud-dataflow-java/4636/testReport/junit/org.apache.beam.sdk.transforms/ViewTest/testEmptyMultimapSideInput/
> We should retry this check as well.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
