Peng Du created SAMZA-1795:
------------------------------
Summary: Add retry mechanism to Samza Table
Key: SAMZA-1795
URL: https://issues.apache.org/jira/browse/SAMZA-1795
Project: Samza
Issue Type: Improvement
Reporter: Peng Du
Assignee: Peng Du
Currently, there is no built-in retry mechanism in either local/remote tables.
For local table, this is a less issue because local data access typically
should not fail unexpectedly. However, remote table access can fail for various
reasons, eg. network issue, quota exceeded, transient authorization issue,
response delay, etc. In these cases, it would be ideal if we have retry to
avoid container shutdown, which can result from table throwing exceptions in
these circumstances. This is possible because Samza currently does not have
retry for task process() either. Even when it does, having a table-level retry
can avoid unnecessarily redo the other operations in the process() when table
is the only failure.
Adding a common implementation simplifies and unifies all table
implementations. Few design considerations need to be addressed:
* should we add this to RemoteTable or create a composite table
* what retry policy should we support (Samza only has exponential backoff
right now)
* how to separate retriable vs non-retriable exceptions
* how would async/rate-limiting/caching work with retries
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
