vetapalem-pravallika commented on issue #21061: URL: https://github.com/apache/beam/issues/21061#issuecomment-4586186148
Thanks for the investigation, @Wellbek. I also looked into the current Beam codebase and traced the path mentioned in the original report. The 2022 stack trace references `WaitForTempTableLoadJobs`, which relied on `WaitForBQJobs`. However, both of these were removed in commit `ac37784821eaedd97c2e6e39441c22ebc6d97cd3` (PR #23012, "Perform job waits in finish_bundle to allow BQ streaming writes with large batch loads"). The commit message explicitly states: > remove WaitForBQJobs and perform waits at each step's finish_bundle In the current implementation, BigQuery job waits are performed directly within the relevant `finish_bundle()` methods. I also searched for the current source tree and couldn't find any remaining references to either `WaitForBQJobs` or `WaitForTempTableLoadJobs`. I also reviewed the current `bigquery_file_loads.py` implementation. The calls to `wait_for_bq_job()` receive job references returned from `_insert_load_job()` and `_insert_copy_job()`, and I did not find an obvious path where a string job ID would be passed instead of a `JobReference`. Combined with the reproduction results reported above on Beam 2.73.0, I was unable to find evidence that the original failure path still exists on current releases. Unless there is a reproducible case on a currently supported Beam version, this issue appears to be no longer reproducible and may have been resolved by subsequent refactoring. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
