[
https://issues.apache.org/jira/browse/SENTRY-957?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hao Hao updated SENTRY-957:
---------------------------
Description:
Right now if there is an error condition in evaluating paths in
MetastoreCacheInitializer in one of the tasks, we throw a Runtime Exception (in
Sentry-888). Changed the behavior to be:
* Retry the failed tasks x times. Each retry has a y millis wait duration. x
and y are based upon the user config
(SENTRY_HDFS_SYNC_METASTORE_CACHE_RETRY_MAX_NUM and
SENTRY_HDFS_SYNC_METASTORE_CACHE_RETRY_WAIT_DURAION_IN_MILLIS).
* After retry failure, throw exception or sync incomplete paths update based on
user config (SENTRY_HDFS_SYNC_METASTORE_CACHE_FAIL_ON_PARTIAL_UPDATE). The
default values is fail on partial update, which will throw runtime exception.
{noformat}
for (Future<CallResult> result : results) {
CallResult callResult = result.get();
// Fail the HMS startup if tasks are not all successful and
// fail on partial updates flag is set in the config.
if (callResult.getSuccessStatus() == false && failOnRetry) {
throw new RuntimeException(callResult.getFailure());
}
}
{noformat}
was:
Right now if there is an error condition in evaluating paths in
MetastoreCacheInitializer in one of the tasks, we throw a Runtime Exception (in
Sentry-888). Changed the behavior to be It looks like it would be best to log
an error and continue; as a plugin should not cause the service to come down?
On the other hand, doing so might result in incomplete sync which might make it
hard to get to the root cause.
{noformat}
for (Future<CallResult> result : results) {
CallResult callResult = result.get();
if (callResult.failure != null) {
throw new RuntimeException(callResult.failure);
}
}
{noformat}
> Exceptions in MetastoreCacheInitializer should probably not prevent HMS from
> starting up
> ----------------------------------------------------------------------------------------
>
> Key: SENTRY-957
> URL: https://issues.apache.org/jira/browse/SENTRY-957
> Project: Sentry
> Issue Type: Bug
> Components: Hdfs Plugin
> Affects Versions: 1.7.0
> Reporter: Sravya Tirukkovalur
> Assignee: Hao Hao
> Fix For: 1.7.0
>
> Attachments: SENTRY-957.001.patch, SENTRY-957.002.patch,
> SENTRY-957.003.patch
>
>
> Right now if there is an error condition in evaluating paths in
> MetastoreCacheInitializer in one of the tasks, we throw a Runtime Exception
> (in Sentry-888). Changed the behavior to be:
> * Retry the failed tasks x times. Each retry has a y millis wait duration. x
> and y are based upon the user config
> (SENTRY_HDFS_SYNC_METASTORE_CACHE_RETRY_MAX_NUM and
> SENTRY_HDFS_SYNC_METASTORE_CACHE_RETRY_WAIT_DURAION_IN_MILLIS).
> * After retry failure, throw exception or sync incomplete paths update based
> on user config (SENTRY_HDFS_SYNC_METASTORE_CACHE_FAIL_ON_PARTIAL_UPDATE). The
> default values is fail on partial update, which will throw runtime exception.
> {noformat}
> for (Future<CallResult> result : results) {
> CallResult callResult = result.get();
> // Fail the HMS startup if tasks are not all successful and
> // fail on partial updates flag is set in the config.
> if (callResult.getSuccessStatus() == false && failOnRetry) {
> throw new RuntimeException(callResult.getFailure());
> }
> }
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
