Colin Phipps created BEAM-2556:
----------------------------------
Summary: Client-side throttling for Datastore connector
Key: BEAM-2556
URL: https://issues.apache.org/jira/browse/BEAM-2556
Project: Beam
Issue Type: Improvement
Components: sdk-java-gcp
Reporter: Colin Phipps
Assignee: Colin Phipps
Priority: Minor
The Datastore connector currently has exponential backoff on errors, which is
good. But it does not do any other throttling of its write load in response to
errors; once a request succeeds, it resumes writing as quickly as it can.
Write loads will be more stable and more likely to compete if the client
throttles itself in the event that it receives high rates of errors from the
Datastore service; specifically
https://landing.google.com/sre/book/chapters/handling-overload.html#client-side-throttling-a7sYUg
is a technique that Google has had success with on other services.
We (Datastore) have a patch in progress to add this behaviour to the connector.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
