[
https://issues.apache.org/jira/browse/BEAM-2556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16101173#comment-16101173
]
ASF GitHub Bot commented on BEAM-2556:
--------------------------------------
GitHub user cph6 opened a pull request:
https://github.com/apache/beam/pull/3644
[BEAM-2556] Add client-side throttling.
The approach used is as described in
https://landing.google.com/sre/book/chapters/handling-overload.html#client-side-throttling-a7sYUg
. By backing off individual workers in response to high error rates, we
relieve
pressure on the Datastore service, increasing the chance that the workload
can
complete successfully. This matches the implementation in the Java SDK.
Also fix the timekeeping of the dynamic batching (seconds vs milliseconds).
R: @vikkyrk
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/cph6/beam datastore_adaptive_py
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/beam/pull/3644.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #3644
----
commit 0b0d90cb45d0dd94e9bdab5f2ae41f5458019883
Author: Colin Phipps <[email protected]>
Date: 2017-07-24T21:01:09Z
Add client-side throttling.
The approach used is as described in
https://landing.google.com/sre/book/chapters/handling-overload.html#client-side-throttling-a7sYUg
. By backing off individual workers in response to high error rates, we
relieve
pressure on the Datastore service, increasing the chance that the workload
can
complete successfully. This matches the implementation in the Java SDK.
----
> Client-side throttling for Datastore connector
> ----------------------------------------------
>
> Key: BEAM-2556
> URL: https://issues.apache.org/jira/browse/BEAM-2556
> Project: Beam
> Issue Type: Improvement
> Components: sdk-java-gcp
> Reporter: Colin Phipps
> Assignee: Colin Phipps
> Priority: Minor
> Labels: datastore
>
> The Datastore connector currently has exponential backoff on errors, which is
> good. But it does not do any other throttling of its write load in response
> to errors; once a request succeeds, it resumes writing as quickly as it can.
> Write loads will be more stable and more likely to compete if the client
> throttles itself in the event that it receives high rates of errors from the
> Datastore service; specifically
> https://landing.google.com/sre/book/chapters/handling-overload.html#client-side-throttling-a7sYUg
> is a technique that Google has had success with on other services.
> We (Datastore) have a patch in progress to add this behaviour to the
> connector.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
