> On July 1, 2016, 12:52 a.m., Hao Hao wrote: > > sentry-binding/sentry-binding-hive/src/main/java/org/apache/sentry/binding/metastore/SentryMetastorePostEventListener.java, > > line 66 > > <https://reviews.apache.org/r/49397/diff/3/?file=1433750#file1433750line66> > > > > What if the notification log fails, but the DDL operations succeeds as > > discussed in HIVE-13966?
Yes, Hive-13966 is needed for both Sentry's listener as well as DBNotificationListener. There is no way to workaround that without it just from with in the Listener. > On July 1, 2016, 12:52 a.m., Hao Hao wrote: > > sentry-binding/sentry-binding-hive/src/main/java/org/apache/sentry/binding/metastore/SentryMetastorePostEventListener.java, > > line 281 > > <https://reviews.apache.org/r/49397/diff/3/?file=1433750#file1433750line281> > > > > Why would we care about the variation here? As we may just need > > <hiveObj, location> mapping? We need it because we need to handle unmanaged rename versus managed rename differently, mainly due to the fact that underlying HDFS files are changed differently in each of the scanarios. Locations do not change in the case of unmanaged table rename versus it does change for a managed table rename. On a related note, it would be better to capture the command, as it is not the same as an addPath + a deletePath. Apart from adding, deleting the deltas, we do additional processing for the rename case: For example: drop old partition locations. - Sravya ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/49397/#review140285 ----------------------------------------------------------- On June 29, 2016, 11:54 p.m., Sravya Tirukkovalur wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/49397/ > ----------------------------------------------------------- > > (Updated June 29, 2016, 11:54 p.m.) > > > Review request for sentry, Anne Yu, Colin McCabe, Hao Hao, and Nachiket > Vaidya. > > > Bugs: Sentry-1329 > https://issues.apache.org/jira/browse/Sentry-1329 > > > Repository: sentry > > > Description > ------- > > This patch adapts SentryMetaStorePostEventListener to write HMS notification > logs. Also, > 1. Implements the SentryJSONMessageFactory to add custom information in the > notification log entry message, which includes > 1.1. Implementing Message class for each message type > 1.2. Implementing a deserializer > 2. Implements JSONAlterPartitionMessage and JSONAlterTableMessage to work > around the issue in Hive 1.1.0. > 3. Added test TestHMSNotificationLogWithSentryListener to verify that the > notification log entries are written properly and are backward compatible for > clients using DbNotificationListener > > Change-Id: I680beb6db4e534bb0a9e6ee042ea0d4f33f0943f > > update > > Change-Id: I6ce8846a801b3effc6a851ab3ce6f866b4a0c23e > > > Diffs > ----- > > sentry-binding/sentry-binding-hive/pom.xml > 07aaae3bac34582fe8222ff166a52c1e208b223a > > sentry-binding/sentry-binding-hive/src/main/java/org/apache/sentry/binding/metastore/SentryMetastorePostEventListener.java > d12ac151ddbf35d612fee3f869d602d6cdf54aa8 > > sentry-binding/sentry-binding-hive/src/main/java/org/apache/sentry/binding/metastore/messaging/json/JSONAlterPartitionMessage.java > PRE-CREATION > > sentry-binding/sentry-binding-hive/src/main/java/org/apache/sentry/binding/metastore/messaging/json/JSONAlterTableMessage.java > PRE-CREATION > > sentry-binding/sentry-binding-hive/src/main/java/org/apache/sentry/binding/metastore/messaging/json/SentryJSONAddPartitionMessage.java > PRE-CREATION > > sentry-binding/sentry-binding-hive/src/main/java/org/apache/sentry/binding/metastore/messaging/json/SentryJSONAlterPartitionMessage.java > PRE-CREATION > > sentry-binding/sentry-binding-hive/src/main/java/org/apache/sentry/binding/metastore/messaging/json/SentryJSONAlterTableMessage.java > PRE-CREATION > > sentry-binding/sentry-binding-hive/src/main/java/org/apache/sentry/binding/metastore/messaging/json/SentryJSONCreateDatabaseMessage.java > PRE-CREATION > > sentry-binding/sentry-binding-hive/src/main/java/org/apache/sentry/binding/metastore/messaging/json/SentryJSONCreateTableMessage.java > PRE-CREATION > > sentry-binding/sentry-binding-hive/src/main/java/org/apache/sentry/binding/metastore/messaging/json/SentryJSONDropDatabaseMessage.java > PRE-CREATION > > sentry-binding/sentry-binding-hive/src/main/java/org/apache/sentry/binding/metastore/messaging/json/SentryJSONDropPartitionMessage.java > PRE-CREATION > > sentry-binding/sentry-binding-hive/src/main/java/org/apache/sentry/binding/metastore/messaging/json/SentryJSONDropTableMessage.java > PRE-CREATION > > sentry-binding/sentry-binding-hive/src/main/java/org/apache/sentry/binding/metastore/messaging/json/SentryJSONMessageDeserializer.java > PRE-CREATION > > sentry-binding/sentry-binding-hive/src/main/java/org/apache/sentry/binding/metastore/messaging/json/SentryJSONMessageFactory.java > PRE-CREATION > > sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/dbprovider/TestDbPrivilegeCleanupOnDrop.java > 767bcbe02e0d511f52bc869e7b6a1ee1e6584a5a > > sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/hive/hiveserver/HiveServerFactory.java > 847da45dcc884d7d573b10a18d3122292754c8ad > > sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/metastore/TestHMSNotificationLogWithSentryListener.java > PRE-CREATION > > Diff: https://reviews.apache.org/r/49397/diff/ > > > Testing > ------- > > Added new tests > > > Thanks, > > Sravya Tirukkovalur > >
