ahmedabu98 opened a new issue, #23235: URL: https://github.com/apache/beam/issues/23235
### What would you like to happen? Java's BigQuery connector delegates waiting for jobs to a [PendingJobManager](https://github.com/apache/beam/blob/935eeadd45a6100c68243fc5b266b20d26164173/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BigQueryHelpers.java#L91). The benefit with this job manager is that jobs are waited for and their state is checked on in parallel. This way, we get [useful logs](https://github.com/apache/beam/blob/935eeadd45a6100c68243fc5b266b20d26164173/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BigQueryHelpers.java#L135) and can determine if a job is significantly slower than the rest. In contrast, the Python connector waits for jobs and checks on their state sequentially. The connector could benefit from having a job manager. ### Issue Priority Priority: 3 ### Issue Component Component: io-py-gcp -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
