ahmedabu98 opened a new issue, #24097: URL: https://github.com/apache/beam/issues/24097
### What happened? https://github.com/apache/beam/pull/4869 added a [random string](https://github.com/apache/beam/pull/4869/files#diff-b47580f7fcc36b9a014a20be2dd4dc3207ce1e17120358bbf32330d64097f32fR170) to the end of query job names to make this step resilient to retries. This query step creates a temp destination + table to output the query to. The next step would be to export data from this temp table to files, then delete the temp table+destination and read from those files. The problem was if a retry happened right after/during temp table+destination deletion, we would need to retry the query job. However, BQ would see the old successful query job name and return as so without actually querying and creating the temp table. The next step, export to files, would always fail because the table we are trying to export from does not exist. The solution introduced by https://github.com/apache/beam/pull/4869 was written over in a code refactor in https://github.com/apache/beam/pull/12082 and the random string was lost [here](https://github.com/apache/beam/commit/9af54351ddc31c607d1634382704c8a52aaa78f3#diff-471ca9a2a4c3366613d75c1d98736b7d88c0873d2abc126e01577c937c3759dbL145). This re-introduced the problem of having a deterministic query job name that was vulnerable to retries. ### Issue Priority Priority: 2 ### Issue Component Component: io-java-gcp -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
