[jira] [Work logged] (BEAM-9269) Set shorter Commit Deadline and handle with backoff/retry

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/BEAM-9269?focusedWorklogId=386100=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-386100
 ]

ASF GitHub Bot logged work on BEAM-9269:


Author: ASF GitHub Bot
Created on: 12/Feb/20 17:45
Start Date: 12/Feb/20 17:45
Worklog Time Spent: 10m 
  Work Description: chamikaramj commented on pull request #10752: 
[BEAM-9269] Add commit deadline for Spanner writes.
URL: https://github.com/apache/beam/pull/10752
 
 
   
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 386100)
Time Spent: 4h  (was: 3h 50m)

> Set shorter Commit Deadline and handle with backoff/retry
> -
>
> Key: BEAM-9269
> URL: https://issues.apache.org/jira/browse/BEAM-9269
> Project: Beam
>  Issue Type: Improvement
>  Components: io-java-gcp
>Affects Versions: 2.16.0, 2.17.0, 2.18.0, 2.19.0
>Reporter: Niel Markwick
>Assignee: Niel Markwick
>Priority: Major
>  Labels: google-cloud-spanner
>  Time Spent: 4h
>  Remaining Estimate: 0h
>
> Default commit deadline in Spanner is 1hr, which can lead to a variety of 
> issues including database overload and session expiry.
> Shorter deadline should be set with backoff/retry when deadline expires, so 
> that the Spanner database does not become overloaded.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Work logged] (BEAM-9269) Set shorter Commit Deadline and handle with backoff/retry

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/BEAM-9269?focusedWorklogId=385485=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-385485
 ]

ASF GitHub Bot logged work on BEAM-9269:


Author: ASF GitHub Bot
Created on: 11/Feb/20 21:53
Start Date: 11/Feb/20 21:53
Worklog Time Spent: 10m 
  Work Description: nielm commented on pull request #10752: [BEAM-9269] Add 
commit deadline for Spanner writes.
URL: https://github.com/apache/beam/pull/10752#discussion_r377922130
 
 

 ##
 File path: 
sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/spanner/SpannerConfig.java
 ##
 @@ -136,13 +163,51 @@ public SpannerConfig withHost(ValueProvider 
host) {
 return toBuilder().setHost(host).build();
   }
 
+  public SpannerConfig withCommitDeadline(Duration commitDeadline) {
+return 
withCommitDeadline(ValueProvider.StaticValueProvider.of(commitDeadline));
+  }
+
+  public SpannerConfig withCommitDeadline(ValueProvider 
commitDeadline) {
+return toBuilder().setCommitDeadline(commitDeadline).build();
+  }
+
+  public SpannerConfig withMaxCumulativeBackoff(Duration maxCumulativeBackoff) 
{
+return 
withMaxCumulativeBackoff(ValueProvider.StaticValueProvider.of(maxCumulativeBackoff));
+  }
+
+  public SpannerConfig withMaxCumulativeBackoff(ValueProvider 
maxCumulativeBackoff) {
+return toBuilder().setMaxCumulativeBackoff(maxCumulativeBackoff).build();
+  }
+
   @VisibleForTesting
   SpannerConfig withServiceFactory(ServiceFactory 
serviceFactory) {
 return toBuilder().setServiceFactory(serviceFactory).build();
   }
 
   public SpannerAccessor connectToSpanner() {
 SpannerOptions.Builder builder = SpannerOptions.newBuilder();
+
+if (getCommitDeadline() != null && getCommitDeadline().get().getMillis() > 
0) {
+
+  // In Spanner API version 1.21 or above, we can set the deadline / total 
Timeout on an API
+  // call using the following code:
+  //
+  // UnaryCallSettings.Builder commitSettings =
+  // builder.getSpannerStubSettingsBuilder().commitSettings();
+  // RetrySettings.Builder commitRetrySettings = 
commitSettings.getRetrySettings().toBuilder()
+  // commitSettings.setRetrySettings(
+  // commitRetrySettings.setTotalTimeout(
+  // Duration.ofMillis(getCommitDeadlineMillis().get()))
+  // .build());
+  //
+  // However, at time of this commit, the Spanner API is at only at 
v1.6.0, where the only
+  // method to set a deadline is with GRPC Interceptors, so we have to use 
that...
+  SpannerInterceptorProvider interceptorProvider =
+  SpannerInterceptorProvider.createDefault()
+  .with(new 
CommitDeadlineSettingInterceptor(getCommitDeadline().get()));
 
 Review comment:
   > Just to confirm, this deadline will not cause Dataflow workitems to fail 
but just that request will be retried by SpannerIO within the same workitem
   
   Correct, it will backoff/retry up to a configurable time limit (default 15 
mins per workitem). 
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 385485)
Time Spent: 3h 50m  (was: 3h 40m)

> Set shorter Commit Deadline and handle with backoff/retry
> -
>
> Key: BEAM-9269
> URL: https://issues.apache.org/jira/browse/BEAM-9269
> Project: Beam
>  Issue Type: Improvement
>  Components: io-java-gcp
>Affects Versions: 2.16.0, 2.17.0, 2.18.0, 2.19.0
>Reporter: Niel Markwick
>Assignee: Niel Markwick
>Priority: Major
>  Labels: google-cloud-spanner
>  Time Spent: 3h 50m
>  Remaining Estimate: 0h
>
> Default commit deadline in Spanner is 1hr, which can lead to a variety of 
> issues including database overload and session expiry.
> Shorter deadline should be set with backoff/retry when deadline expires, so 
> that the Spanner database does not become overloaded.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Work logged] (BEAM-9269) Set shorter Commit Deadline and handle with backoff/retry

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/BEAM-9269?focusedWorklogId=385451=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-385451
 ]

ASF GitHub Bot logged work on BEAM-9269:


Author: ASF GitHub Bot
Created on: 11/Feb/20 21:05
Start Date: 11/Feb/20 21:05
Worklog Time Spent: 10m 
  Work Description: chamikaramj commented on issue #10752: [BEAM-9269] Add 
commit deadline for Spanner writes.
URL: https://github.com/apache/beam/pull/10752#issuecomment-584851158
 
 
   Run Java PreCommit
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 385451)
Time Spent: 3.5h  (was: 3h 20m)

> Set shorter Commit Deadline and handle with backoff/retry
> -
>
> Key: BEAM-9269
> URL: https://issues.apache.org/jira/browse/BEAM-9269
> Project: Beam
>  Issue Type: Improvement
>  Components: io-java-gcp
>Affects Versions: 2.16.0, 2.17.0, 2.18.0, 2.19.0
>Reporter: Niel Markwick
>Assignee: Niel Markwick
>Priority: Major
>  Labels: google-cloud-spanner
>  Time Spent: 3.5h
>  Remaining Estimate: 0h
>
> Default commit deadline in Spanner is 1hr, which can lead to a variety of 
> issues including database overload and session expiry.
> Shorter deadline should be set with backoff/retry when deadline expires, so 
> that the Spanner database does not become overloaded.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Work logged] (BEAM-9269) Set shorter Commit Deadline and handle with backoff/retry

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/BEAM-9269?focusedWorklogId=385452=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-385452
 ]

ASF GitHub Bot logged work on BEAM-9269:


Author: ASF GitHub Bot
Created on: 11/Feb/20 21:05
Start Date: 11/Feb/20 21:05
Worklog Time Spent: 10m 
  Work Description: chamikaramj commented on issue #10752: [BEAM-9269] Add 
commit deadline for Spanner writes.
URL: https://github.com/apache/beam/pull/10752#issuecomment-584851223
 
 
   Retest this please
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 385452)
Time Spent: 3h 40m  (was: 3.5h)

> Set shorter Commit Deadline and handle with backoff/retry
> -
>
> Key: BEAM-9269
> URL: https://issues.apache.org/jira/browse/BEAM-9269
> Project: Beam
>  Issue Type: Improvement
>  Components: io-java-gcp
>Affects Versions: 2.16.0, 2.17.0, 2.18.0, 2.19.0
>Reporter: Niel Markwick
>Assignee: Niel Markwick
>Priority: Major
>  Labels: google-cloud-spanner
>  Time Spent: 3h 40m
>  Remaining Estimate: 0h
>
> Default commit deadline in Spanner is 1hr, which can lead to a variety of 
> issues including database overload and session expiry.
> Shorter deadline should be set with backoff/retry when deadline expires, so 
> that the Spanner database does not become overloaded.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Work logged] (BEAM-9269) Set shorter Commit Deadline and handle with backoff/retry

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/BEAM-9269?focusedWorklogId=385398=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-385398
 ]

ASF GitHub Bot logged work on BEAM-9269:


Author: ASF GitHub Bot
Created on: 11/Feb/20 19:39
Start Date: 11/Feb/20 19:39
Worklog Time Spent: 10m 
  Work Description: chamikaramj commented on issue #10752: [BEAM-9269] Add 
commit deadline for Spanner writes.
URL: https://github.com/apache/beam/pull/10752#issuecomment-584813963
 
 
   Run Java PreCommit
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 385398)
Time Spent: 3h 10m  (was: 3h)

> Set shorter Commit Deadline and handle with backoff/retry
> -
>
> Key: BEAM-9269
> URL: https://issues.apache.org/jira/browse/BEAM-9269
> Project: Beam
>  Issue Type: Improvement
>  Components: io-java-gcp
>Affects Versions: 2.16.0, 2.17.0, 2.18.0, 2.19.0
>Reporter: Niel Markwick
>Assignee: Niel Markwick
>Priority: Major
>  Labels: google-cloud-spanner
>  Time Spent: 3h 10m
>  Remaining Estimate: 0h
>
> Default commit deadline in Spanner is 1hr, which can lead to a variety of 
> issues including database overload and session expiry.
> Shorter deadline should be set with backoff/retry when deadline expires, so 
> that the Spanner database does not become overloaded.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Work logged] (BEAM-9269) Set shorter Commit Deadline and handle with backoff/retry

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/BEAM-9269?focusedWorklogId=385399=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-385399
 ]

ASF GitHub Bot logged work on BEAM-9269:


Author: ASF GitHub Bot
Created on: 11/Feb/20 19:39
Start Date: 11/Feb/20 19:39
Worklog Time Spent: 10m 
  Work Description: chamikaramj commented on pull request #10752: 
[BEAM-9269] Add commit deadline for Spanner writes.
URL: https://github.com/apache/beam/pull/10752#discussion_r377855990
 
 

 ##
 File path: 
sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/spanner/SpannerConfig.java
 ##
 @@ -136,13 +163,51 @@ public SpannerConfig withHost(ValueProvider 
host) {
 return toBuilder().setHost(host).build();
   }
 
+  public SpannerConfig withCommitDeadline(Duration commitDeadline) {
+return 
withCommitDeadline(ValueProvider.StaticValueProvider.of(commitDeadline));
+  }
+
+  public SpannerConfig withCommitDeadline(ValueProvider 
commitDeadline) {
+return toBuilder().setCommitDeadline(commitDeadline).build();
+  }
+
+  public SpannerConfig withMaxCumulativeBackoff(Duration maxCumulativeBackoff) 
{
+return 
withMaxCumulativeBackoff(ValueProvider.StaticValueProvider.of(maxCumulativeBackoff));
+  }
+
+  public SpannerConfig withMaxCumulativeBackoff(ValueProvider 
maxCumulativeBackoff) {
+return toBuilder().setMaxCumulativeBackoff(maxCumulativeBackoff).build();
+  }
+
   @VisibleForTesting
   SpannerConfig withServiceFactory(ServiceFactory 
serviceFactory) {
 return toBuilder().setServiceFactory(serviceFactory).build();
   }
 
   public SpannerAccessor connectToSpanner() {
 SpannerOptions.Builder builder = SpannerOptions.newBuilder();
+
+if (getCommitDeadline() != null && getCommitDeadline().get().getMillis() > 
0) {
+
+  // In Spanner API version 1.21 or above, we can set the deadline / total 
Timeout on an API
+  // call using the following code:
+  //
+  // UnaryCallSettings.Builder commitSettings =
+  // builder.getSpannerStubSettingsBuilder().commitSettings();
+  // RetrySettings.Builder commitRetrySettings = 
commitSettings.getRetrySettings().toBuilder()
+  // commitSettings.setRetrySettings(
+  // commitRetrySettings.setTotalTimeout(
+  // Duration.ofMillis(getCommitDeadlineMillis().get()))
+  // .build());
+  //
+  // However, at time of this commit, the Spanner API is at only at 
v1.6.0, where the only
+  // method to set a deadline is with GRPC Interceptors, so we have to use 
that...
+  SpannerInterceptorProvider interceptorProvider =
+  SpannerInterceptorProvider.createDefault()
+  .with(new 
CommitDeadlineSettingInterceptor(getCommitDeadline().get()));
 
 Review comment:
   This seems to be the case. I'll merge after tests pass. Thanks folks.
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 385399)
Time Spent: 3h 20m  (was: 3h 10m)

> Set shorter Commit Deadline and handle with backoff/retry
> -
>
> Key: BEAM-9269
> URL: https://issues.apache.org/jira/browse/BEAM-9269
> Project: Beam
>  Issue Type: Improvement
>  Components: io-java-gcp
>Affects Versions: 2.16.0, 2.17.0, 2.18.0, 2.19.0
>Reporter: Niel Markwick
>Assignee: Niel Markwick
>Priority: Major
>  Labels: google-cloud-spanner
>  Time Spent: 3h 20m
>  Remaining Estimate: 0h
>
> Default commit deadline in Spanner is 1hr, which can lead to a variety of 
> issues including database overload and session expiry.
> Shorter deadline should be set with backoff/retry when deadline expires, so 
> that the Spanner database does not become overloaded.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Work logged] (BEAM-9269) Set shorter Commit Deadline and handle with backoff/retry

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/BEAM-9269?focusedWorklogId=385390=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-385390
 ]

ASF GitHub Bot logged work on BEAM-9269:


Author: ASF GitHub Bot
Created on: 11/Feb/20 19:27
Start Date: 11/Feb/20 19:27
Worklog Time Spent: 10m 
  Work Description: chamikaramj commented on pull request #10752: 
[BEAM-9269] Add commit deadline for Spanner writes.
URL: https://github.com/apache/beam/pull/10752#discussion_r377849449
 
 

 ##
 File path: 
sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/spanner/SpannerConfig.java
 ##
 @@ -136,13 +163,51 @@ public SpannerConfig withHost(ValueProvider 
host) {
 return toBuilder().setHost(host).build();
   }
 
+  public SpannerConfig withCommitDeadline(Duration commitDeadline) {
+return 
withCommitDeadline(ValueProvider.StaticValueProvider.of(commitDeadline));
+  }
+
+  public SpannerConfig withCommitDeadline(ValueProvider 
commitDeadline) {
+return toBuilder().setCommitDeadline(commitDeadline).build();
+  }
+
+  public SpannerConfig withMaxCumulativeBackoff(Duration maxCumulativeBackoff) 
{
+return 
withMaxCumulativeBackoff(ValueProvider.StaticValueProvider.of(maxCumulativeBackoff));
+  }
+
+  public SpannerConfig withMaxCumulativeBackoff(ValueProvider 
maxCumulativeBackoff) {
+return toBuilder().setMaxCumulativeBackoff(maxCumulativeBackoff).build();
+  }
+
   @VisibleForTesting
   SpannerConfig withServiceFactory(ServiceFactory 
serviceFactory) {
 return toBuilder().setServiceFactory(serviceFactory).build();
   }
 
   public SpannerAccessor connectToSpanner() {
 SpannerOptions.Builder builder = SpannerOptions.newBuilder();
+
+if (getCommitDeadline() != null && getCommitDeadline().get().getMillis() > 
0) {
+
+  // In Spanner API version 1.21 or above, we can set the deadline / total 
Timeout on an API
+  // call using the following code:
+  //
+  // UnaryCallSettings.Builder commitSettings =
+  // builder.getSpannerStubSettingsBuilder().commitSettings();
+  // RetrySettings.Builder commitRetrySettings = 
commitSettings.getRetrySettings().toBuilder()
+  // commitSettings.setRetrySettings(
+  // commitRetrySettings.setTotalTimeout(
+  // Duration.ofMillis(getCommitDeadlineMillis().get()))
+  // .build());
+  //
+  // However, at time of this commit, the Spanner API is at only at 
v1.6.0, where the only
+  // method to set a deadline is with GRPC Interceptors, so we have to use 
that...
+  SpannerInterceptorProvider interceptorProvider =
+  SpannerInterceptorProvider.createDefault()
+  .with(new 
CommitDeadlineSettingInterceptor(getCommitDeadline().get()));
 
 Review comment:
   Just to confirm, this deadline will not cause Dataflow workitems to fail but 
just that request will be retried by SpannerIO within the same workitem ? 
(otherwise jobs can significantly slow down or fail if we rely on workitem 
retries.)
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 385390)
Time Spent: 3h  (was: 2h 50m)

> Set shorter Commit Deadline and handle with backoff/retry
> -
>
> Key: BEAM-9269
> URL: https://issues.apache.org/jira/browse/BEAM-9269
> Project: Beam
>  Issue Type: Improvement
>  Components: io-java-gcp
>Affects Versions: 2.16.0, 2.17.0, 2.18.0, 2.19.0
>Reporter: Niel Markwick
>Assignee: Niel Markwick
>Priority: Major
>  Labels: google-cloud-spanner
>  Time Spent: 3h
>  Remaining Estimate: 0h
>
> Default commit deadline in Spanner is 1hr, which can lead to a variety of 
> issues including database overload and session expiry.
> Shorter deadline should be set with backoff/retry when deadline expires, so 
> that the Spanner database does not become overloaded.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Work logged] (BEAM-9269) Set shorter Commit Deadline and handle with backoff/retry

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/BEAM-9269?focusedWorklogId=385227=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-385227
 ]

ASF GitHub Bot logged work on BEAM-9269:


Author: ASF GitHub Bot
Created on: 11/Feb/20 15:16
Start Date: 11/Feb/20 15:16
Worklog Time Spent: 10m 
  Work Description: nielm commented on issue #10752: [BEAM-9269] Add commit 
deadline for Spanner writes.
URL: https://github.com/apache/beam/pull/10752#issuecomment-584685459
 
 
   Run Java PreCommit
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 385227)
Time Spent: 2h 50m  (was: 2h 40m)

> Set shorter Commit Deadline and handle with backoff/retry
> -
>
> Key: BEAM-9269
> URL: https://issues.apache.org/jira/browse/BEAM-9269
> Project: Beam
>  Issue Type: Improvement
>  Components: io-java-gcp
>Affects Versions: 2.16.0, 2.17.0, 2.18.0, 2.19.0
>Reporter: Niel Markwick
>Assignee: Niel Markwick
>Priority: Major
>  Labels: google-cloud-spanner
>  Time Spent: 2h 50m
>  Remaining Estimate: 0h
>
> Default commit deadline in Spanner is 1hr, which can lead to a variety of 
> issues including database overload and session expiry.
> Shorter deadline should be set with backoff/retry when deadline expires, so 
> that the Spanner database does not become overloaded.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Work logged] (BEAM-9269) Set shorter Commit Deadline and handle with backoff/retry

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/BEAM-9269?focusedWorklogId=385225=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-385225
 ]

ASF GitHub Bot logged work on BEAM-9269:


Author: ASF GitHub Bot
Created on: 11/Feb/20 15:16
Start Date: 11/Feb/20 15:16
Worklog Time Spent: 10m 
  Work Description: nielm commented on issue #10752: [BEAM-9269] Add commit 
deadline for Spanner writes.
URL: https://github.com/apache/beam/pull/10752#issuecomment-584595347
 
 
   Run Java PreCommit
   
   
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 385225)
Time Spent: 2.5h  (was: 2h 20m)

> Set shorter Commit Deadline and handle with backoff/retry
> -
>
> Key: BEAM-9269
> URL: https://issues.apache.org/jira/browse/BEAM-9269
> Project: Beam
>  Issue Type: Improvement
>  Components: io-java-gcp
>Affects Versions: 2.16.0, 2.17.0, 2.18.0, 2.19.0
>Reporter: Niel Markwick
>Assignee: Niel Markwick
>Priority: Major
>  Labels: google-cloud-spanner
>  Time Spent: 2.5h
>  Remaining Estimate: 0h
>
> Default commit deadline in Spanner is 1hr, which can lead to a variety of 
> issues including database overload and session expiry.
> Shorter deadline should be set with backoff/retry when deadline expires, so 
> that the Spanner database does not become overloaded.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Work logged] (BEAM-9269) Set shorter Commit Deadline and handle with backoff/retry

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/BEAM-9269?focusedWorklogId=385226=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-385226
 ]

ASF GitHub Bot logged work on BEAM-9269:


Author: ASF GitHub Bot
Created on: 11/Feb/20 15:16
Start Date: 11/Feb/20 15:16
Worklog Time Spent: 10m 
  Work Description: nielm commented on issue #10752: [BEAM-9269] Add commit 
deadline for Spanner writes.
URL: https://github.com/apache/beam/pull/10752#issuecomment-584621556
 
 
   Run Java PreCommit
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 385226)
Time Spent: 2h 40m  (was: 2.5h)

> Set shorter Commit Deadline and handle with backoff/retry
> -
>
> Key: BEAM-9269
> URL: https://issues.apache.org/jira/browse/BEAM-9269
> Project: Beam
>  Issue Type: Improvement
>  Components: io-java-gcp
>Affects Versions: 2.16.0, 2.17.0, 2.18.0, 2.19.0
>Reporter: Niel Markwick
>Assignee: Niel Markwick
>Priority: Major
>  Labels: google-cloud-spanner
>  Time Spent: 2h 40m
>  Remaining Estimate: 0h
>
> Default commit deadline in Spanner is 1hr, which can lead to a variety of 
> issues including database overload and session expiry.
> Shorter deadline should be set with backoff/retry when deadline expires, so 
> that the Spanner database does not become overloaded.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Work logged] (BEAM-9269) Set shorter Commit Deadline and handle with backoff/retry

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/BEAM-9269?focusedWorklogId=385121=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-385121
 ]

ASF GitHub Bot logged work on BEAM-9269:


Author: ASF GitHub Bot
Created on: 11/Feb/20 12:56
Start Date: 11/Feb/20 12:56
Worklog Time Spent: 10m 
  Work Description: nielm commented on issue #10752: [BEAM-9269] Add commit 
deadline for Spanner writes.
URL: https://github.com/apache/beam/pull/10752#issuecomment-584621556
 
 
   Run Java PreCommit
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 385121)
Time Spent: 2h 20m  (was: 2h 10m)

> Set shorter Commit Deadline and handle with backoff/retry
> -
>
> Key: BEAM-9269
> URL: https://issues.apache.org/jira/browse/BEAM-9269
> Project: Beam
>  Issue Type: Improvement
>  Components: io-java-gcp
>Affects Versions: 2.16.0, 2.17.0, 2.18.0, 2.19.0
>Reporter: Niel Markwick
>Assignee: Niel Markwick
>Priority: Major
>  Labels: google-cloud-spanner
>  Time Spent: 2h 20m
>  Remaining Estimate: 0h
>
> Default commit deadline in Spanner is 1hr, which can lead to a variety of 
> issues including database overload and session expiry.
> Shorter deadline should be set with backoff/retry when deadline expires, so 
> that the Spanner database does not become overloaded.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Work logged] (BEAM-9269) Set shorter Commit Deadline and handle with backoff/retry

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/BEAM-9269?focusedWorklogId=385091=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-385091
 ]

ASF GitHub Bot logged work on BEAM-9269:


Author: ASF GitHub Bot
Created on: 11/Feb/20 11:39
Start Date: 11/Feb/20 11:39
Worklog Time Spent: 10m 
  Work Description: nielm commented on issue #10752: [BEAM-9269] Add commit 
deadline for Spanner writes.
URL: https://github.com/apache/beam/pull/10752#issuecomment-584595347
 
 
   Run Java PreCommit
   
   
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 385091)
Time Spent: 2h 10m  (was: 2h)

> Set shorter Commit Deadline and handle with backoff/retry
> -
>
> Key: BEAM-9269
> URL: https://issues.apache.org/jira/browse/BEAM-9269
> Project: Beam
>  Issue Type: Improvement
>  Components: io-java-gcp
>Affects Versions: 2.16.0, 2.17.0, 2.18.0, 2.19.0
>Reporter: Niel Markwick
>Assignee: Niel Markwick
>Priority: Major
>  Labels: google-cloud-spanner
>  Time Spent: 2h 10m
>  Remaining Estimate: 0h
>
> Default commit deadline in Spanner is 1hr, which can lead to a variety of 
> issues including database overload and session expiry.
> Shorter deadline should be set with backoff/retry when deadline expires, so 
> that the Spanner database does not become overloaded.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Work logged] (BEAM-9269) Set shorter Commit Deadline and handle with backoff/retry

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/BEAM-9269?focusedWorklogId=385087=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-385087
 ]

ASF GitHub Bot logged work on BEAM-9269:


Author: ASF GitHub Bot
Created on: 11/Feb/20 11:18
Start Date: 11/Feb/20 11:18
Worklog Time Spent: 10m 
  Work Description: nielm commented on pull request #10752: [BEAM-9269] Add 
commit deadline for Spanner writes.
URL: https://github.com/apache/beam/pull/10752#discussion_r377574096
 
 

 ##
 File path: 
sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/spanner/SpannerIO.java
 ##
 @@ -1248,20 +1307,44 @@ public void processElement(ProcessContext c) {
 }
   }
 
-  private static class WriteToSpannerFn extends DoFn, 
Void> {
+  @VisibleForTesting
+  static class WriteToSpannerFn extends DoFn, Void> {
 
 private transient SpannerAccessor spannerAccessor;
 private final SpannerConfig spannerConfig;
 private final FailureMode failureMode;
-private final Counter mutationGroupBatchesCounter =
-Metrics.counter(WriteGrouped.class, "mutation_group_batches");
-private final Counter mutationGroupWriteSuccessCounter =
+
+@VisibleForTesting static Sleeper sleeper = Sleeper.DEFAULT;
+
+private final Counter mutationGroupBatchesReceived =
+Metrics.counter(WriteGrouped.class, "mutation_group_batches_received");
+private final Counter mutationGroupBatchesWriteSuccess =
+Metrics.counter(WriteGrouped.class, 
"mutation_group_batches_write_success");
+private final Counter mutationGroupBatchesWriteFail =
+Metrics.counter(WriteGrouped.class, 
"mutation_group_batches_write_fail");
+
+private final Counter mutationGroupsReceived =
+Metrics.counter(WriteGrouped.class, "mutation_groups_received");
+private final Counter mutationGroupsWriteSuccess =
 Metrics.counter(WriteGrouped.class, "mutation_groups_write_success");
-private final Counter mutationGroupWriteFailCounter =
+private final Counter mutationGroupsWriteFail =
 Metrics.counter(WriteGrouped.class, "mutation_groups_write_fail");
 
+private final Counter spannerWriteSuccess =
+Metrics.counter(WriteGrouped.class, "spanner_write_success");
+private final Counter spannerWriteFail =
+Metrics.counter(WriteGrouped.class, "spanner_write_fail");
+private final Counter spannerWriteTotalLatency =
 
 Review comment:
   Its not possible - the only things we can export are counters and gauges.
   
   User will have to manually calculate average latency.
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 385087)
Time Spent: 2h  (was: 1h 50m)

> Set shorter Commit Deadline and handle with backoff/retry
> -
>
> Key: BEAM-9269
> URL: https://issues.apache.org/jira/browse/BEAM-9269
> Project: Beam
>  Issue Type: Improvement
>  Components: io-java-gcp
>Affects Versions: 2.16.0, 2.17.0, 2.18.0, 2.19.0
>Reporter: Niel Markwick
>Assignee: Niel Markwick
>Priority: Major
>  Labels: google-cloud-spanner
>  Time Spent: 2h
>  Remaining Estimate: 0h
>
> Default commit deadline in Spanner is 1hr, which can lead to a variety of 
> issues including database overload and session expiry.
> Shorter deadline should be set with backoff/retry when deadline expires, so 
> that the Spanner database does not become overloaded.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Work logged] (BEAM-9269) Set shorter Commit Deadline and handle with backoff/retry

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/BEAM-9269?focusedWorklogId=385085=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-385085
 ]

ASF GitHub Bot logged work on BEAM-9269:


Author: ASF GitHub Bot
Created on: 11/Feb/20 11:14
Start Date: 11/Feb/20 11:14
Worklog Time Spent: 10m 
  Work Description: nielm commented on pull request #10752: [BEAM-9269] Add 
commit deadline for Spanner writes.
URL: https://github.com/apache/beam/pull/10752#discussion_r377571907
 
 

 ##
 File path: 
sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/spanner/SpannerConfig.java
 ##
 @@ -42,6 +50,10 @@
   private static final String USER_AGENT_PREFIX = "Apache_Beam_Java";
   // A default host name for batch traffic.
   private static final String DEFAULT_HOST = 
"https://batch-spanner.googleapis.com/;;
+  // Deadline for Commit API call.
+  private static final Duration DEFAULT_COMMIT_DEADLINE = 
Duration.standardSeconds(15);
 
 Review comment:
   Configurable using SpannerIO.Write.withCommitDeadline()
   as is DEFAULT_MAX_CUMULATIVE_BACKOFF with 
SpannerIO.Write.withMaxCumulativeBackoff() 
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 385085)
Time Spent: 1h 40m  (was: 1.5h)

> Set shorter Commit Deadline and handle with backoff/retry
> -
>
> Key: BEAM-9269
> URL: https://issues.apache.org/jira/browse/BEAM-9269
> Project: Beam
>  Issue Type: Improvement
>  Components: io-java-gcp
>Affects Versions: 2.16.0, 2.17.0, 2.18.0, 2.19.0
>Reporter: Niel Markwick
>Assignee: Niel Markwick
>Priority: Major
>  Labels: google-cloud-spanner
>  Time Spent: 1h 40m
>  Remaining Estimate: 0h
>
> Default commit deadline in Spanner is 1hr, which can lead to a variety of 
> issues including database overload and session expiry.
> Shorter deadline should be set with backoff/retry when deadline expires, so 
> that the Spanner database does not become overloaded.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Work logged] (BEAM-9269) Set shorter Commit Deadline and handle with backoff/retry

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/BEAM-9269?focusedWorklogId=385086=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-385086
 ]

ASF GitHub Bot logged work on BEAM-9269:


Author: ASF GitHub Bot
Created on: 11/Feb/20 11:14
Start Date: 11/Feb/20 11:14
Worklog Time Spent: 10m 
  Work Description: nielm commented on pull request #10752: [BEAM-9269] Add 
commit deadline for Spanner writes.
URL: https://github.com/apache/beam/pull/10752#discussion_r377572157
 
 

 ##
 File path: 
sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/spanner/SpannerIO.java
 ##
 @@ -1285,30 +1373,84 @@ public void processElement(ProcessContext c) throws 
Exception {
   boolean tryIndividual = false;
 
 Review comment:
   its gone.
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 385086)
Time Spent: 1h 50m  (was: 1h 40m)

> Set shorter Commit Deadline and handle with backoff/retry
> -
>
> Key: BEAM-9269
> URL: https://issues.apache.org/jira/browse/BEAM-9269
> Project: Beam
>  Issue Type: Improvement
>  Components: io-java-gcp
>Affects Versions: 2.16.0, 2.17.0, 2.18.0, 2.19.0
>Reporter: Niel Markwick
>Assignee: Niel Markwick
>Priority: Major
>  Labels: google-cloud-spanner
>  Time Spent: 1h 50m
>  Remaining Estimate: 0h
>
> Default commit deadline in Spanner is 1hr, which can lead to a variety of 
> issues including database overload and session expiry.
> Shorter deadline should be set with backoff/retry when deadline expires, so 
> that the Spanner database does not become overloaded.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Work logged] (BEAM-9269) Set shorter Commit Deadline and handle with backoff/retry

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/BEAM-9269?focusedWorklogId=384877=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-384877
 ]

ASF GitHub Bot logged work on BEAM-9269:


Author: ASF GitHub Bot
Created on: 11/Feb/20 00:36
Start Date: 11/Feb/20 00:36
Worklog Time Spent: 10m 
  Work Description: allenpradeep commented on pull request #10752: 
[BEAM-9269] Add commit deadline for Spanner writes.
URL: https://github.com/apache/beam/pull/10752#discussion_r377394855
 
 

 ##
 File path: 
sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/spanner/SpannerConfig.java
 ##
 @@ -42,6 +50,10 @@
   private static final String USER_AGENT_PREFIX = "Apache_Beam_Java";
   // A default host name for batch traffic.
   private static final String DEFAULT_HOST = 
"https://batch-spanner.googleapis.com/;;
+  // Deadline for Commit API call.
+  private static final Duration DEFAULT_COMMIT_DEADLINE = 
Duration.standardSeconds(15);
 
 Review comment:
   Can this be made configurable.
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 384877)
Time Spent: 1.5h  (was: 1h 20m)

> Set shorter Commit Deadline and handle with backoff/retry
> -
>
> Key: BEAM-9269
> URL: https://issues.apache.org/jira/browse/BEAM-9269
> Project: Beam
>  Issue Type: Improvement
>  Components: io-java-gcp
>Affects Versions: 2.16.0, 2.17.0, 2.18.0, 2.19.0
>Reporter: Niel Markwick
>Assignee: Niel Markwick
>Priority: Major
>  Labels: google-cloud-spanner
>  Time Spent: 1.5h
>  Remaining Estimate: 0h
>
> Default commit deadline in Spanner is 1hr, which can lead to a variety of 
> issues including database overload and session expiry.
> Shorter deadline should be set with backoff/retry when deadline expires, so 
> that the Spanner database does not become overloaded.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Work logged] (BEAM-9269) Set shorter Commit Deadline and handle with backoff/retry

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/BEAM-9269?focusedWorklogId=384876=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-384876
 ]

ASF GitHub Bot logged work on BEAM-9269:


Author: ASF GitHub Bot
Created on: 11/Feb/20 00:36
Start Date: 11/Feb/20 00:36
Worklog Time Spent: 10m 
  Work Description: allenpradeep commented on pull request #10752: 
[BEAM-9269] Add commit deadline for Spanner writes.
URL: https://github.com/apache/beam/pull/10752#discussion_r377395567
 
 

 ##
 File path: 
sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/spanner/SpannerIO.java
 ##
 @@ -1285,30 +1373,84 @@ public void processElement(ProcessContext c) throws 
Exception {
   boolean tryIndividual = false;
 
 Review comment:
   we can remove this if this is not being used.
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 384876)
Time Spent: 1h 20m  (was: 1h 10m)

> Set shorter Commit Deadline and handle with backoff/retry
> -
>
> Key: BEAM-9269
> URL: https://issues.apache.org/jira/browse/BEAM-9269
> Project: Beam
>  Issue Type: Improvement
>  Components: io-java-gcp
>Affects Versions: 2.16.0, 2.17.0, 2.18.0, 2.19.0
>Reporter: Niel Markwick
>Assignee: Niel Markwick
>Priority: Major
>  Labels: google-cloud-spanner
>  Time Spent: 1h 20m
>  Remaining Estimate: 0h
>
> Default commit deadline in Spanner is 1hr, which can lead to a variety of 
> issues including database overload and session expiry.
> Shorter deadline should be set with backoff/retry when deadline expires, so 
> that the Spanner database does not become overloaded.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Work logged] (BEAM-9269) Set shorter Commit Deadline and handle with backoff/retry

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/BEAM-9269?focusedWorklogId=384878=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-384878
 ]

ASF GitHub Bot logged work on BEAM-9269:


Author: ASF GitHub Bot
Created on: 11/Feb/20 00:36
Start Date: 11/Feb/20 00:36
Worklog Time Spent: 10m 
  Work Description: allenpradeep commented on pull request #10752: 
[BEAM-9269] Add commit deadline for Spanner writes.
URL: https://github.com/apache/beam/pull/10752#discussion_r377397609
 
 

 ##
 File path: 
sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/spanner/SpannerIO.java
 ##
 @@ -1248,20 +1307,44 @@ public void processElement(ProcessContext c) {
 }
   }
 
-  private static class WriteToSpannerFn extends DoFn, 
Void> {
+  @VisibleForTesting
+  static class WriteToSpannerFn extends DoFn, Void> {
 
 private transient SpannerAccessor spannerAccessor;
 private final SpannerConfig spannerConfig;
 private final FailureMode failureMode;
-private final Counter mutationGroupBatchesCounter =
-Metrics.counter(WriteGrouped.class, "mutation_group_batches");
-private final Counter mutationGroupWriteSuccessCounter =
+
+@VisibleForTesting static Sleeper sleeper = Sleeper.DEFAULT;
+
+private final Counter mutationGroupBatchesReceived =
+Metrics.counter(WriteGrouped.class, "mutation_group_batches_received");
+private final Counter mutationGroupBatchesWriteSuccess =
+Metrics.counter(WriteGrouped.class, 
"mutation_group_batches_write_success");
+private final Counter mutationGroupBatchesWriteFail =
+Metrics.counter(WriteGrouped.class, 
"mutation_group_batches_write_fail");
+
+private final Counter mutationGroupsReceived =
+Metrics.counter(WriteGrouped.class, "mutation_groups_received");
+private final Counter mutationGroupsWriteSuccess =
 Metrics.counter(WriteGrouped.class, "mutation_groups_write_success");
-private final Counter mutationGroupWriteFailCounter =
+private final Counter mutationGroupsWriteFail =
 Metrics.counter(WriteGrouped.class, "mutation_groups_write_fail");
 
+private final Counter spannerWriteSuccess =
+Metrics.counter(WriteGrouped.class, "spanner_write_success");
+private final Counter spannerWriteFail =
+Metrics.counter(WriteGrouped.class, "spanner_write_fail");
+private final Counter spannerWriteTotalLatency =
 
 Review comment:
   Would an average writeLatency(if its possible to calculate) be more 
actionable to the user rather than a total write latency? In the sense, if the 
write latency is high then the user can reduce the workers.
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 384878)

> Set shorter Commit Deadline and handle with backoff/retry
> -
>
> Key: BEAM-9269
> URL: https://issues.apache.org/jira/browse/BEAM-9269
> Project: Beam
>  Issue Type: Improvement
>  Components: io-java-gcp
>Affects Versions: 2.16.0, 2.17.0, 2.18.0, 2.19.0
>Reporter: Niel Markwick
>Assignee: Niel Markwick
>Priority: Major
>  Labels: google-cloud-spanner
>  Time Spent: 1.5h
>  Remaining Estimate: 0h
>
> Default commit deadline in Spanner is 1hr, which can lead to a variety of 
> issues including database overload and session expiry.
> Shorter deadline should be set with backoff/retry when deadline expires, so 
> that the Spanner database does not become overloaded.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Work logged] (BEAM-9269) Set shorter Commit Deadline and handle with backoff/retry

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/BEAM-9269?focusedWorklogId=384773=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-384773
 ]

ASF GitHub Bot logged work on BEAM-9269:


Author: ASF GitHub Bot
Created on: 10/Feb/20 22:04
Start Date: 10/Feb/20 22:04
Worklog Time Spent: 10m 
  Work Description: nithinsujir commented on issue #10752: [BEAM-9269] Add 
commit deadline for Spanner writes.
URL: https://github.com/apache/beam/pull/10752#issuecomment-584380391
 
 
   Looks ok to me.
   
   Allen,
   Can you take a look as well?
   
   
   On Mon, Feb 10, 2020 at 1:43 PM Chamikara Jayalath 
   wrote:
   
   > Thanks Niel.
   >
   > R: @nithinsujir  can you review ?
   >
   > —
   > You are receiving this because you were mentioned.
   > Reply to this email directly, view it on GitHub
   > 
,
   > or unsubscribe
   > 

   > .
   >
   
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 384773)
Time Spent: 1h 10m  (was: 1h)

> Set shorter Commit Deadline and handle with backoff/retry
> -
>
> Key: BEAM-9269
> URL: https://issues.apache.org/jira/browse/BEAM-9269
> Project: Beam
>  Issue Type: Improvement
>  Components: io-java-gcp
>Affects Versions: 2.16.0, 2.17.0, 2.18.0, 2.19.0
>Reporter: Niel Markwick
>Priority: Major
>  Labels: google-cloud-spanner
>  Time Spent: 1h 10m
>  Remaining Estimate: 0h
>
> Default commit deadline in Spanner is 1hr, which can lead to a variety of 
> issues including database overload and session expiry.
> Shorter deadline should be set with backoff/retry when deadline expires, so 
> that the Spanner database does not become overloaded.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Work logged] (BEAM-9269) Set shorter Commit Deadline and handle with backoff/retry

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/BEAM-9269?focusedWorklogId=384764=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-384764
 ]

ASF GitHub Bot logged work on BEAM-9269:


Author: ASF GitHub Bot
Created on: 10/Feb/20 21:43
Start Date: 10/Feb/20 21:43
Worklog Time Spent: 10m 
  Work Description: chamikaramj commented on issue #10752: [BEAM-9269] Add 
commit deadline for Spanner writes.
URL: https://github.com/apache/beam/pull/10752#issuecomment-584371801
 
 
   Thanks Niel.
   
   R: @nithinsujir can you review ?
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 384764)
Time Spent: 1h  (was: 50m)

> Set shorter Commit Deadline and handle with backoff/retry
> -
>
> Key: BEAM-9269
> URL: https://issues.apache.org/jira/browse/BEAM-9269
> Project: Beam
>  Issue Type: Improvement
>  Components: io-java-gcp
>Affects Versions: 2.16.0, 2.17.0, 2.18.0, 2.19.0
>Reporter: Niel Markwick
>Priority: Major
>  Labels: google-cloud-spanner
>  Time Spent: 1h
>  Remaining Estimate: 0h
>
> Default commit deadline in Spanner is 1hr, which can lead to a variety of 
> issues including database overload and session expiry.
> Shorter deadline should be set with backoff/retry when deadline expires, so 
> that the Spanner database does not become overloaded.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Work logged] (BEAM-9269) Set shorter Commit Deadline and handle with backoff/retry

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/BEAM-9269?focusedWorklogId=384379=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-384379
 ]

ASF GitHub Bot logged work on BEAM-9269:


Author: ASF GitHub Bot
Created on: 10/Feb/20 12:16
Start Date: 10/Feb/20 12:16
Worklog Time Spent: 10m 
  Work Description: nielm commented on issue #10752: [BEAM-9269] Add commit 
deadline for Spanner writes.
URL: https://github.com/apache/beam/pull/10752#issuecomment-584095300
 
 
   Run Java PreCommit
   
   
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 384379)
Time Spent: 50m  (was: 40m)

> Set shorter Commit Deadline and handle with backoff/retry
> -
>
> Key: BEAM-9269
> URL: https://issues.apache.org/jira/browse/BEAM-9269
> Project: Beam
>  Issue Type: Improvement
>  Components: io-java-gcp
>Affects Versions: 2.16.0, 2.17.0, 2.18.0, 2.19.0
>Reporter: Niel Markwick
>Priority: Major
>  Labels: google-cloud-spanner
>  Time Spent: 50m
>  Remaining Estimate: 0h
>
> Default commit deadline in Spanner is 1hr, which can lead to a variety of 
> issues including database overload and session expiry.
> Shorter deadline should be set with backoff/retry when deadline expires, so 
> that the Spanner database does not become overloaded.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Work logged] (BEAM-9269) Set shorter Commit Deadline and handle with backoff/retry

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/BEAM-9269?focusedWorklogId=384324=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-384324
 ]

ASF GitHub Bot logged work on BEAM-9269:


Author: ASF GitHub Bot
Created on: 10/Feb/20 10:24
Start Date: 10/Feb/20 10:24
Worklog Time Spent: 10m 
  Work Description: nielm commented on issue #10752: [BEAM-9269] Add commit 
deadline for Spanner writes.
URL: https://github.com/apache/beam/pull/10752#issuecomment-584053824
 
 
   Run Java PreCommit
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 384324)
Time Spent: 40m  (was: 0.5h)

> Set shorter Commit Deadline and handle with backoff/retry
> -
>
> Key: BEAM-9269
> URL: https://issues.apache.org/jira/browse/BEAM-9269
> Project: Beam
>  Issue Type: Improvement
>  Components: io-java-gcp
>Affects Versions: 2.16.0, 2.17.0, 2.18.0, 2.19.0
>Reporter: Niel Markwick
>Priority: Major
>  Labels: google-cloud-spanner
>  Time Spent: 40m
>  Remaining Estimate: 0h
>
> Default commit deadline in Spanner is 1hr, which can lead to a variety of 
> issues including database overload and session expiry.
> Shorter deadline should be set with backoff/retry when deadline expires, so 
> that the Spanner database does not become overloaded.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Work logged] (BEAM-9269) Set shorter Commit Deadline and handle with backoff/retry

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/BEAM-9269?focusedWorklogId=384323=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-384323
 ]

ASF GitHub Bot logged work on BEAM-9269:


Author: ASF GitHub Bot
Created on: 10/Feb/20 10:24
Start Date: 10/Feb/20 10:24
Worklog Time Spent: 10m 
  Work Description: nielm commented on issue #10752: [BEAM-9269] Add commit 
deadline for Spanner writes.
URL: https://github.com/apache/beam/pull/10752#issuecomment-583419055
 
 
   Run Java PreCommit
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 384323)
Time Spent: 0.5h  (was: 20m)

> Set shorter Commit Deadline and handle with backoff/retry
> -
>
> Key: BEAM-9269
> URL: https://issues.apache.org/jira/browse/BEAM-9269
> Project: Beam
>  Issue Type: Improvement
>  Components: io-java-gcp
>Affects Versions: 2.16.0, 2.17.0, 2.18.0, 2.19.0
>Reporter: Niel Markwick
>Priority: Major
>  Labels: google-cloud-spanner
>  Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> Default commit deadline in Spanner is 1hr, which can lead to a variety of 
> issues including database overload and session expiry.
> Shorter deadline should be set with backoff/retry when deadline expires, so 
> that the Spanner database does not become overloaded.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Work logged] (BEAM-9269) Set shorter Commit Deadline and handle with backoff/retry

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/BEAM-9269?focusedWorklogId=383618=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-383618
 ]

ASF GitHub Bot logged work on BEAM-9269:


Author: ASF GitHub Bot
Created on: 07/Feb/20 14:43
Start Date: 07/Feb/20 14:43
Worklog Time Spent: 10m 
  Work Description: nielm commented on issue #10752: [BEAM-9269] Add commit 
deadline for Spanner writes.
URL: https://github.com/apache/beam/pull/10752#issuecomment-583419055
 
 
   Run Java PreCommit
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 383618)
Time Spent: 20m  (was: 10m)

> Set shorter Commit Deadline and handle with backoff/retry
> -
>
> Key: BEAM-9269
> URL: https://issues.apache.org/jira/browse/BEAM-9269
> Project: Beam
>  Issue Type: Improvement
>  Components: io-java-gcp
>Affects Versions: 2.16.0, 2.17.0, 2.18.0, 2.19.0
>Reporter: Niel Markwick
>Priority: Major
>  Labels: google-cloud-spanner
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> Default commit deadline in Spanner is 1hr, which can lead to a variety of 
> issues including database overload and session expiry.
> Shorter deadline should be set with backoff/retry when deadline expires, so 
> that the Spanner database does not become overloaded.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Work logged] (BEAM-9269) Set shorter Commit Deadline and handle with backoff/retry

[ASF GitHub Bot (Jira)]

 [ 
https://issues.apache.org/jira/browse/BEAM-9269?focusedWorklogId=383616=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-383616
 ]

ASF GitHub Bot logged work on BEAM-9269:


Author: ASF GitHub Bot
Created on: 07/Feb/20 14:41
Start Date: 07/Feb/20 14:41
Worklog Time Spent: 10m 
  Work Description: nielm commented on issue #10752: [BEAM-9269] Add commit 
deadline for Spanner writes.
URL: https://github.com/apache/beam/pull/10752#issuecomment-581440773
 
 
   Run Java PreCommit
 

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 383616)
Remaining Estimate: 0h
Time Spent: 10m

> Set shorter Commit Deadline and handle with backoff/retry
> -
>
> Key: BEAM-9269
> URL: https://issues.apache.org/jira/browse/BEAM-9269
> Project: Beam
>  Issue Type: Improvement
>  Components: io-java-gcp
>Affects Versions: 2.16.0, 2.17.0, 2.18.0, 2.19.0
>Reporter: Niel Markwick
>Priority: Major
>  Labels: google-cloud-spanner
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> Default commit deadline in Spanner is 1hr, which can lead to a variety of 
> issues including database overload and session expiry.
> Shorter deadline should be set with backoff/retry when deadline expires, so 
> that the Spanner database does not become overloaded.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)