damjad opened a new pull request, #40420: URL: https://github.com/apache/beam/pull/40420
Fixes #40419 ## Summary `InsertBatchofRowsCallable` calls `insert.execute().getInsertErrors()`. The `execute()` call goes through `RetryHttpRequestInitializer`, but the response body is parsed by `parseAs()` **after** `executeUnparsed()` returns — outside the scope of the retry handler. A `SocketTimeoutException` (or connection reset / broken pipe) thrown during chunked response body reading lands in the `catch (IOException e)` block where `getErrorInfo(e)` returns `null` (it's not a `GoogleJsonResponseException`), and the exception is immediately re-thrown as `RuntimeException` — bypassing both the configured `InsertRetryPolicy` and the outer `insertAll` retry loop, causing a Dataflow bundle retry for every occurrence. **Fix**: in the `errorInfo == null` branch, call `transientNetworkErrorReason()` to identify socket-level failures. When detected, return synthetic `InsertErrors` with a specific reason (`"socketTimeout"`, `"connectionReset"`, `"brokenPipe"`) so the outer `insertAll` loop routes the affected rows through the pipeline's configured `InsertRetryPolicy` — using the same `INSERT_BACKOFF_FACTORY` retry budget (max 5 retries) as other transient BQ errors. The reason is also recorded via `serviceCallMetric` for observability. ## Changes - `BigQueryServicesImpl.java`: add `transientNetworkErrorReason()` helper and synthetic error return path in `InsertBatchofRowsCallable.call()` - `BigQueryServicesImplTest.java`: 7 new tests covering reason classification and retry/exhaustion behaviour ## Test plan - [ ] `testTransientNetworkErrorReason_socketTimeout` - [ ] `testTransientNetworkErrorReason_wrappedSocketTimeout` - [ ] `testTransientNetworkErrorReason_connectionReset` - [ ] `testTransientNetworkErrorReason_brokenPipe` - [ ] `testTransientNetworkErrorReason_nonTransient` - [ ] `testInsertSocketTimeoutRetry` — verifies rows are retried via InsertRetryPolicy on first SocketTimeoutException - [ ] `testInsertSocketTimeoutExhausted` — verifies IOException is thrown when outer backoff is exhausted -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
