[
https://issues.apache.org/jira/browse/BEAM-9269?focusedWorklogId=446948&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-446948
]
ASF GitHub Bot logged work on BEAM-9269:
----------------------------------------
Author: ASF GitHub Bot
Created on: 16/Jun/20 22:12
Start Date: 16/Jun/20 22:12
Worklog Time Spent: 10m
Work Description: allenpradeep commented on a change in pull request
#12011:
URL: https://github.com/apache/beam/pull/12011#discussion_r441171212
##########
File path:
sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/spanner/SpannerAccessor.java
##########
@@ -63,23 +65,19 @@ static SpannerAccessor create(SpannerConfig spannerConfig) {
ValueProvider<Duration> commitDeadline = spannerConfig.getCommitDeadline();
if (commitDeadline != null && commitDeadline.get().getMillis() > 0) {
- // In Spanner API version 1.21 or above, we can set the deadline / total
Timeout on an API
- // call using the following code:
- //
- // UnaryCallSettings.Builder commitSettings =
- // builder.getSpannerStubSettingsBuilder().commitSettings();
- // RetrySettings.Builder commitRetrySettings =
commitSettings.getRetrySettings().toBuilder()
- // commitSettings.setRetrySettings(
- // commitRetrySettings.setTotalTimeout(
- // Duration.ofMillis(getCommitDeadlineMillis().get()))
- // .build());
- //
- // However, at time of this commit, the Spanner API is at only at
v1.6.0, where the only
- // method to set a deadline is with GRPC Interceptors, so we have to use
that...
- SpannerInterceptorProvider interceptorProvider =
- SpannerInterceptorProvider.createDefault()
- .with(new
CommitDeadlineSettingInterceptor(commitDeadline.get()));
- builder.setInterceptorProvider(interceptorProvider);
+ // Set the GRPC deadline on the Commit API call.
+ UnaryCallSettings.Builder commitSettings =
+ builder.getSpannerStubSettingsBuilder().commitSettings();
+ RetrySettings.Builder commitRetrySettings =
commitSettings.getRetrySettings().toBuilder();
+ commitSettings.setRetrySettings(
+ commitRetrySettings
Review comment:
This patch sounds good. But i have a question. RetrySettings seems to
support retry with backoff. The following retry mechanism could be moved to
here as well.
https://github.com/apache/beam/blob/master/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/spanner/SpannerIO.java#L1485
Can that be done? Not as part of this patch. I could send out a PR as well
after testing. That would clean up some code in SpannerIO as well
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 446948)
Time Spent: 4.5h (was: 4h 20m)
> Set shorter Commit Deadline and handle with backoff/retry
> ---------------------------------------------------------
>
> Key: BEAM-9269
> URL: https://issues.apache.org/jira/browse/BEAM-9269
> Project: Beam
> Issue Type: Improvement
> Components: io-java-gcp
> Affects Versions: 2.16.0, 2.17.0, 2.18.0, 2.19.0
> Reporter: Niel Markwick
> Assignee: Niel Markwick
> Priority: P2
> Labels: google-cloud-spanner
> Fix For: 2.20.0
>
> Time Spent: 4.5h
> Remaining Estimate: 0h
>
> Default commit deadline in Spanner is 1hr, which can lead to a variety of
> issues including database overload and session expiry.
> Shorter deadline should be set with backoff/retry when deadline expires, so
> that the Spanner database does not become overloaded.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
