ahmedabu98 commented on code in PR #26503:
URL: https://github.com/apache/beam/pull/26503#discussion_r1183047262
##########
sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/StorageApiWriteUnshardedRecords.java:
##########
@@ -598,7 +606,42 @@ long flush(
streamName,
clientNumber,
retrieveErrorDetails(contexts));
+ failedContext.failureCount += 1;
+
+ // Maximum number of times we retry before we fail the work item.
+ if (failedContext.failureCount > 5) {
+ throw new RuntimeException("More than 5 attempts to call
AppendRows failed.");
+ }
+
+ // The following errors are known to be persistent, so always
fail the work item in
+ // this case.
+ Throwable error =
Preconditions.checkStateNotNull(failedContext.getError());
+ Status.Code statusCode = Status.fromThrowable(error).getCode();
+ if (statusCode.equals(Status.Code.OUT_OF_RANGE)
+ || statusCode.equals(Status.Code.ALREADY_EXISTS)) {
+ throw new RuntimeException(
+ "Append to stream "
+ + this.streamName
+ + " failed with invalid "
+ + "offset of "
+ + failedContext.offset);
Review Comment:
Would it be helpful for future debugging if we include the error's
stacktrace here?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
