> On June 4, 2017, 3:53 a.m., Alexander Kolbasov wrote: > > sentry-hdfs/sentry-hdfs-namenode-plugin/src/main/java/org/apache/sentry/hdfs/SentryAuthorizationInfo.java > > Lines 39 (patched) > > <https://reviews.apache.org/r/59566/diff/4/?file=1739146#file1739146line39> > > > > The only changes in this file are comments and extra logging - right?
Yes. - Na ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/59566/#review176876 ----------------------------------------------------------- On June 1, 2017, 10:51 p.m., Na Li wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/59566/ > ----------------------------------------------------------- > > (Updated June 1, 2017, 10:51 p.m.) > > > Review request for sentry, Alexander Kolbasov, kalyan kumar kalvagadda, > Sergio Pena, and Vamsee Yarlagadda. > > > Bugs: SENTRY-1784 > https://issues.apache.org/jira/browse/SENTRY-1784 > > > Repository: sentry > > > Description > ------- > > HDFS starts requesting changeID = 0 instead of 1. DBUpdateForwarder returns > full update if the request changeID <= 0. After first full update, the > request changeID = 1, so only delta update is sent unless clean up removes > changes that are not sent to HDFS. This fixes both issues in this Jira. > > 1) What happens when NN just starts > NN sends requestedId = 0 to sentry server. Sentry server sends full update > back. The latest changeID for perm and path are returned to NN. NN saves the > latest changeID for perm and path. > NN initial changeID for permission and path is -1. The requestedId is NN > changeID + 1. So initial requestedId from NN to sentry server is 0. > > 2) What happens once NN gets a full snapshot > Once NN gets a full snapshot, it creats new UpdateableAuthzPermissions for > full perm update; it creates new UpdateableAuthzPaths for full path update; > > It saves the latest changeID for perm and path (referred as > latestChangeID_path and latestChangeID_perm). Next interval, NN sends > requestedId (latestChangeID_path + 1, latestChangeID_perm + 1) to Sentry > server. Since the lowest latestChangeID is 0, this requestedId >= 1. > > If Sentry server has changes equal or newer than requestedId, it sends back > delta changes to NN. Otherwise, it sends back empty list to NN. The > exceptions are: 1) the requestedId does not exist in SentryStore (maybe it is > cleaned up) or 2) the returned delta change list from SentryStore is empty > (maybe the change table is corrupted). If exception happens, full snapshot is > sent back to NN. > > 3) What happens when a delta is received from HMS. > NN updates the privilege and path based on the delta changes. It update save > the latest changeID for perm and path. > > > Diffs > ----- > > > sentry-hdfs/sentry-hdfs-common/src/main/java/org/apache/sentry/hdfs/ServiceConstants.java > 0741ebc > > sentry-hdfs/sentry-hdfs-common/src/main/java/org/apache/sentry/hdfs/UpdateableAuthzPaths.java > ad7f8c9 > > sentry-hdfs/sentry-hdfs-namenode-plugin/src/main/java/org/apache/sentry/hdfs/SentryAuthorizationInfo.java > 90ba721 > > sentry-hdfs/sentry-hdfs-namenode-plugin/src/main/java/org/apache/sentry/hdfs/SentryUpdater.java > 34caa0e > > sentry-hdfs/sentry-hdfs-namenode-plugin/src/main/java/org/apache/sentry/hdfs/UpdateableAuthzPermissions.java > 431c7fe > > sentry-hdfs/sentry-hdfs-service/src/main/java/org/apache/sentry/hdfs/DBUpdateForwarder.java > b8542b3 > > sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/service/thrift/SentryService.java > 9beb07b > > > Diff: https://reviews.apache.org/r/59566/diff/4/ > > > Testing > ------- > > TestHDFSIntegrationEnd2End > > > Thanks, > > Na Li > >
