[
https://issues.apache.org/jira/browse/SAMZA-1795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16597744#comment-16597744
]
ASF GitHub Bot commented on SAMZA-1795:
---------------------------------------
GitHub user pdu-mn1 opened a pull request:
https://github.com/apache/samza/pull/618
SAMZA-1795 table: add common retry for IO functions
Add common retry functionality to table IO functions for data stores
that do not have native retry support. We use failsafe as the retry
library.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/pdu-mn1/samza retry-support
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/samza/pull/618.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #618
----
commit ca959912d1c0f3a6c7a2752e81d260b501dd75a8
Author: Peng Du <pdu@...>
Date: 2018-08-28T19:29:25Z
table: add common retry for IO functions
Add common retry functionality to table IO functions for data stores
that do not have native retry support. We use failsafe as the retry
library.
----
> Add retry mechanism to Samza Table
> ----------------------------------
>
> Key: SAMZA-1795
> URL: https://issues.apache.org/jira/browse/SAMZA-1795
> Project: Samza
> Issue Type: Improvement
> Reporter: Peng Du
> Assignee: Peng Du
> Priority: Major
>
> Currently, there is no built-in retry mechanism in either local/remote
> tables. For local table, this is a less issue because local data access
> typically should not fail unexpectedly. However, remote table access can fail
> for various reasons, eg. network issue, quota exceeded, transient
> authorization issue, response delay, etc. In these cases, it would be ideal
> if we have retry to avoid container shutdown, which can result from table
> throwing exceptions in these circumstances. This is possible because Samza
> currently does not have retry for task process() either. Even when it does,
> having a table-level retry can avoid unnecessarily redo the other operations
> in the process() when table is the only failure.
> Adding a common implementation simplifies and unifies all table
> implementations. Few design considerations need to be addressed:
> * should we add this to RemoteTable or create a composite table
> * what retry policy should we support (Samza only has exponential backoff
> right now)
> * how to separate retriable vs non-retriable exceptions
> * how would async/rate-limiting/caching work with retries
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
