> On Sept. 20, 2017, 9:41 p.m., Alexander Kolbasov wrote: > > sentry-binding/sentry-binding-hive/src/main/java/org/apache/sentry/binding/metastore/SentryMetastorePostEventListener.java > > Lines 22 (patched) > > <https://reviews.apache.org/r/62411/diff/1/?file=1829827#file1829827line22> > > > > This class isn't used in 2.0 so there is no need to do anything special > > with it.
It's not used on Hive 1.x because sentry HA either. I'm thinking on doing some cleanup about those clases (including the tests) that are not used anymore related to sentry HA. - Sergio ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/62411/#review185776 ----------------------------------------------------------- On Sept. 19, 2017, 3:10 p.m., Sergio Pena wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/62411/ > ----------------------------------------------------------- > > (Updated Sept. 19, 2017, 3:10 p.m.) > > > Review request for sentry, Alexander Kolbasov, Colm O hEigeartaigh, and > kalyan kumar kalvagadda. > > > Bugs: sentry-1958 > https://issues.apache.org/jira/browse/sentry-1958 > > > Repository: sentry > > > Description > ------- > > This patch bumps the Hive version of hive-authz1 to Hive 2.0. Moving to > authz2 has been a little complicated, so doing an incremental patch was > preffered. > Also, we're taking advantage of the unit tests nad e2e tests that already > exist on Sentry. There are tests that are not on the authz2, so this is why I > preffered to look into bumping the Hive 2.0 version first. > > The following issues with Hive 2.0 were found and addressed on Sentry: > - Hive 2.0.1 has a bug that Sentry cannot workaround. > - Hive 2.1.1 and higher has a different bug that Sentry cannot workaround. > - Hive CBO has a bug where ReadIdentity partitions do not have parents > causing Sentry to request extra privileges that the user might not have > CBO is disabled on the Sentry tests and it must be disabled on production > as well. > - HIVE-11145: Remove OFFLINE and NO_DROP from tables and partitions > Removed tests that use the protection mode operations as Hive do not > support them any more. > - HIVE-10453: HS2 leaking open file descriptors when using UDFs > Hive 2.0 clears all functions after a session is closed causing other users > who want to execute such > function to fail because they lost the function scope and they do not have > permissions to create functions > - HIVE-12320: hive.metastore.disallow.incompatible.col.type.changes should be > true by default > Sentry had some issues on the tests due to this Hive change. > - HIVE-10307: Support to use number literals in partition column > Hive 2.0 added an extra validation when using ALTER TABLE ... PARTITION > (spec) that throws an error if > spec is not a partition column. > - The HS2 webui fails to start when concurrency mode is enabled. The Sentry > tests are now configured to put > Hive in test mode so that the webui is not initialized. > - There are some column names that cannot be used as they are reservered by > Hive, i.e 'date' column name fails > in some Sentry tests. > - Hive 2.0 switched to log4j2 causing some Sentry tests to fail. > > Important changes on Sentry: > - Hive 2.0 has an authz1 bug with the use of SentryMetastoreFilterHook class. > This class is replaced automatically by Hive with a default one that uses > authz2. To make minimal changes on Sentry, a new class is created > (SentryHiveAuthorizerImpl) that only deals with metastore filtering. Also, > the class SentryHiveAuthorizerFactory is set by the > HiveAuthzBindingSessionHook automatically when a HS2 session is opened. > However, this new authorizer must be set manually on the hive-site.xml so > that other clients who use the HMS Client can use this new filter class (info > is mentioned on the SentryHiveAuthorizerFactory class. > > There will be follow-up patches to fix the following: > - Remove SentryMetastoreFilterHook and improve the SentryHiveAuthorizerImpl > to avoid the overhead of converting a list of hive privileges to a list of > strings and viceversa. > - Merge authz1 and authz2 dependencies. > - Review current authz2 and merge code into the new changes. > > > Diffs > ----- > > dev-support/test-patch.py ac91b590c93884911d27928ac60108dcaa5d81ed > pom.xml 1479f5fa3895833a1efa63d0f4a7c3db72c7cde9 > sentry-binding/sentry-binding-hive-follower-v2/pom.xml > fa7e9282592bf5fe15a35921b7173c81b45a1f1f > sentry-binding/sentry-binding-hive-follower/pom.xml > e69519cc63d5c89db6098fa169089b34dbaadae8 > > sentry-binding/sentry-binding-hive-follower/src/main/java/org/apache/sentry/binding/metastore/messaging/json/SentryJSONAlterPartitionMessage.java > 1e636c94afc99678658bdafe74fdd7aff7e12d92 > > sentry-binding/sentry-binding-hive-follower/src/main/java/org/apache/sentry/binding/metastore/messaging/json/SentryJSONDropPartitionMessage.java > d3ebf603f2d81b3fed7ec0d33031446185b80cb5 > > sentry-binding/sentry-binding-hive-follower/src/main/java/org/apache/sentry/binding/metastore/messaging/json/SentryJSONMessageDeserializer.java > cc0bbec830436241ef69fc3dc9a727bf105ae6e7 > > sentry-binding/sentry-binding-hive-follower/src/main/java/org/apache/sentry/binding/metastore/messaging/json/SentryJSONMessageFactory.java > efdf8b8a5b4eeca08e2a4ad39c95ab86e7855a76 > sentry-binding/sentry-binding-hive-v2/pom.xml > 5f5cbf3c15df324d741e5eca7b951e4aab5bd0d3 > sentry-binding/sentry-binding-hive/pom.xml > 92147e18b0560bd266bfc82d65b9bc70af3df627 > > sentry-binding/sentry-binding-hive/src/main/java/org/apache/hadoop/hive/ql/exec/SentryGrantRevokeTask.java > 97dbd2a503bf9d61c18f6f4fcaa494cbd429f07d > > sentry-binding/sentry-binding-hive/src/main/java/org/apache/sentry/binding/hive/HiveAuthzBindingSessionHook.java > a3aa0b0351deb7b41dec3e5bc7b0e62727e3cb21 > > sentry-binding/sentry-binding-hive/src/main/java/org/apache/sentry/binding/hive/authz/HiveAuthzPrivilegesMap.java > 9f3d42d3787fbceb62e9fd5e22395c17010867f4 > > sentry-binding/sentry-binding-hive/src/main/java/org/apache/sentry/binding/hive/authz/SentryHiveAuthorizerFactory.java > PRE-CREATION > > sentry-binding/sentry-binding-hive/src/main/java/org/apache/sentry/binding/hive/authz/SentryHiveAuthorizerImpl.java > PRE-CREATION > > sentry-binding/sentry-binding-hive/src/main/java/org/apache/sentry/binding/metastore/AuthorizingObjectStore.java > d20da81efcbc796456c114654df0261964c2cfc1 > > sentry-binding/sentry-binding-hive/src/main/java/org/apache/sentry/binding/metastore/AuthorizingObjectStoreBase.java > 9e066e12f1b3539e8787878723eec153dfd68d32 > > sentry-binding/sentry-binding-hive/src/main/java/org/apache/sentry/binding/metastore/MetastoreAuthzBindingBase.java > 6df4885cea3868b451ed14a5e354dd45794cbb3f > > sentry-binding/sentry-binding-hive/src/main/java/org/apache/sentry/binding/metastore/SentryMetastorePostEventListener.java > 3ec2eedab99e7ec66e2dc69b9c8af8eb482cac08 > > sentry-binding/sentry-binding-hive/src/main/java/org/apache/sentry/binding/metastore/SentryMetastorePostEventListenerBase.java > 5b9274e337d1f8a05e2b01384cb3a2e95ad07293 > > sentry-binding/sentry-binding-hive/src/main/java/org/apache/sentry/binding/metastore/SentryMetastorePostEventListenerNotificationLog.java > 58470d6ade3467ef7b03428272d61dc0dd1accea > > sentry-binding/sentry-binding-hive/src/test/java/org/apache/sentry/binding/hive/TestSentryHiveAuthorizationTaskFactory.java > aed218ec0308405c32e6a070451ef38c4c67bb27 > > sentry-binding/sentry-binding-hive/src/test/java/org/apache/sentry/binding/hive/TestURI.java > aa3de64a404b2a8e1dbe245a9cd2298c41745fc3 > sentry-binding/sentry-binding-hive/src/test/resources/log4j2.properties > PRE-CREATION > sentry-provider/sentry-provider-db/pom.xml > 6b7d3c07a0cdf43ff925c22d58c290a3a9affe8b > > sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/thrift/SentryPolicyStoreProcessor.java > cd854006f2b0e73a8a1bcb3c87a4df01bd30cc14 > sentry-tests/sentry-tests-hive/pom.xml > 51801dc8bdcd449f9acaa5658855a4227c47c56e > > sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/dbprovider/TestDbDDLAuditLog.java > e105f003b79b0ad9bec58b24d3e7fd87576e7ad7 > > sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/hdfs/TestHDFSIntegrationAdvanced.java > 2073d85f27349c933bd936f87dc754c6fb30a729 > > sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/hdfs/TestHDFSIntegrationBase.java > 718b02cbd9df9a551bd4de1768f93a914cad7b32 > > sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/hdfs/TestHDFSIntegrationEnd2End.java > d4bc97d2f59efee00034036ebc716ab69cf2116b > > sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/hive/AbstractTestWithStaticConfiguration.java > 0a39f5954de5867b4e64e9be3529219654d31378 > > sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/hive/TestMetadataObjectRetrieval.java > fb0ef19a52375126d21a4c787dd31a1deacfc232 > > sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/hive/TestOperationsPart1.java > 3a4da5059463459af43fd04ed3b34b6468848e7e > > sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/hive/TestPrivilegesAtFunctionScope.java > 249d3bcfe8b1f9f2a1256280cf9b80527cef1e4c > > sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/hive/hiveserver/HiveServerFactory.java > f3f58f64f66ea693ab99d2dd5a9b0eb298ae7d0f > > sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/metastore/TestDBNotificationListenerInBuiltDeserializer.java > e9b3a43debb6d47818c54f6a1f646f13668e35f2 > > sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/metastore/TestSentryListenerSentryDeserializer.java > f43b316928a6354674e824d04b8e7c4a9e5bec00 > sentry-tests/sentry-tests-hive/src/test/resources/log4j2.properties > PRE-CREATION > sentry-tests/sentry-tests-kafka/pom.xml > 56a3ef10a9071929776cb7211bdb8ead921deace > sentry-tests/sentry-tests-solr/pom.xml > c70476808688c80e1723d5e65e3b8cf6d1b64250 > sentry-tests/sentry-tests-sqoop/pom.xml > 9e7cee95be06c68b0eaa456a80d4120576ec977b > > > Diff: https://reviews.apache.org/r/62411/diff/1/ > > > Testing > ------- > > > Thanks, > > Sergio Pena > >
