Re: Review Request 66373: SENTRY-2194: Upgrade Sentry hadoop-version dependency to 2.7.5
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/66373/#review200297 --- Ship it! Ship It! - kalyan kumar kalvagadda On April 2, 2018, 6:38 p.m., Arjun Mishra wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/66373/ > --- > > (Updated April 2, 2018, 6:38 p.m.) > > > Review request for sentry, Alexander Kolbasov, Brian Towles, kalyan kumar > kalvagadda, Na Li, Steve Moist, Sergio Pena, Vadim Spector, and Xinran Tinney. > > > Repository: sentry > > > Description > --- > > Sentry clients use Configuration class defined in the hadoop-common code base > to parse or read configuration files. Hadoop community had made improvements > particularly to enhance security. The change introduces a new boolean > attribute restrictParser. Setting restrictParser to true will > > Limit XML parsing to conform with feature > "http://apache.org/xml/features/disallow-doctype-decl; > This is a security feature explained here - > https://www.owasp.org/index.php/XML_External_Entity_(XXE)_Prevention_Cheat_Sheet > boolean restrictSystemProps is set to true > Will prevent system properties from being read > set XML inclusion (XInclude) to false > prevent merging of xml documents > This change is currently included in hadoop-version 2.7.5. There is a new > implementation of addResources method to allow the setting of restrictParser > boolean. Sentry is currently using hadoop-version 2.7.2. Bumping this version > up and making appropriate changes will allow Sentry to take advantage of this > feature > > > Diffs > - > > pom.xml 61e0f9700 > > sentry-binding/sentry-binding-hbase-indexer/src/main/java/org/apache/sentry/binding/hbaseindexer/authz/HBaseIndexerAuthzBinding.java > d919fe702 > > sentry-binding/sentry-binding-hbase-indexer/src/main/java/org/apache/sentry/binding/hbaseindexer/conf/HBaseIndexerAuthzConf.java > cfbd37bf1 > > sentry-binding/sentry-binding-hive-conf/src/main/java/org/apache/sentry/binding/hive/conf/HiveAuthzConf.java > 8d5286cd3 > > sentry-binding/sentry-binding-kafka/src/main/java/org/apache/sentry/kafka/conf/KafkaAuthConf.java > 6ca621022 > > sentry-binding/sentry-binding-solr/src/main/java/org/apache/sentry/binding/solr/conf/SolrAuthzConf.java > 0883e70fe > > sentry-binding/sentry-binding-sqoop/src/main/java/org/apache/sentry/sqoop/conf/SqoopAuthConf.java > 7836871f6 > > sentry-hdfs/sentry-hdfs-namenode-plugin/src/main/java/org/apache/sentry/hdfs/SentryINodeAttributesProvider.java > cf96df47b > > sentry-provider/sentry-provider-common/src/main/java/org/apache/sentry/provider/common/HadoopGroupMappingService.java > 00b5cf608 > > sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/generic/tools/PermissionsMigrationToolCommon.java > ed28b735c > > sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/generic/tools/SentryConfigToolIndexer.java > c2341d322 > > sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/generic/tools/SentryConfigToolSolr.java > 5649f43fa > > sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/generic/tools/SentryShellGeneric.java > 907e1462c > > sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/tools/SentryShellHive.java > 729a51865 > > sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/service/thrift/SentryService.java > bf5d85b03 > > sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/tools/CreateSentryTestScaleData.java > b234c85c5 > sentry-tools/src/main/java/org/apache/sentry/shell/SentryCli.java 8b68d0d06 > > > Diff: https://reviews.apache.org/r/66373/diff/2/ > > > Testing > --- > > $ mvn -f sentry-binding/pom.xml test > $ mvn -f sentry-provider/pom.xml test > $ mvn -f sentry-hdfs/pom.xml test > > > Thanks, > > Arjun Mishra > >
Re: Review Request 66373: SENTRY-2194: Upgrade Sentry hadoop-version dependency to 2.7.5
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/66373/ --- (Updated April 2, 2018, 6:38 p.m.) Review request for sentry, Alexander Kolbasov, Brian Towles, kalyan kumar kalvagadda, Na Li, Steve Moist, Sergio Pena, Vadim Spector, and Xinran Tinney. Changes --- Updated description Repository: sentry Description (updated) --- Sentry clients use Configuration class defined in the hadoop-common code base to parse or read configuration files. Hadoop community had made improvements particularly to enhance security. The change introduces a new boolean attribute restrictParser. Setting restrictParser to true will Limit XML parsing to conform with feature "http://apache.org/xml/features/disallow-doctype-decl; This is a security feature explained here - https://www.owasp.org/index.php/XML_External_Entity_(XXE)_Prevention_Cheat_Sheet boolean restrictSystemProps is set to true Will prevent system properties from being read set XML inclusion (XInclude) to false prevent merging of xml documents This change is currently included in hadoop-version 2.7.5. There is a new implementation of addResources method to allow the setting of restrictParser boolean. Sentry is currently using hadoop-version 2.7.2. Bumping this version up and making appropriate changes will allow Sentry to take advantage of this feature Diffs - pom.xml 61e0f9700 sentry-binding/sentry-binding-hbase-indexer/src/main/java/org/apache/sentry/binding/hbaseindexer/authz/HBaseIndexerAuthzBinding.java d919fe702 sentry-binding/sentry-binding-hbase-indexer/src/main/java/org/apache/sentry/binding/hbaseindexer/conf/HBaseIndexerAuthzConf.java cfbd37bf1 sentry-binding/sentry-binding-hive-conf/src/main/java/org/apache/sentry/binding/hive/conf/HiveAuthzConf.java 8d5286cd3 sentry-binding/sentry-binding-kafka/src/main/java/org/apache/sentry/kafka/conf/KafkaAuthConf.java 6ca621022 sentry-binding/sentry-binding-solr/src/main/java/org/apache/sentry/binding/solr/conf/SolrAuthzConf.java 0883e70fe sentry-binding/sentry-binding-sqoop/src/main/java/org/apache/sentry/sqoop/conf/SqoopAuthConf.java 7836871f6 sentry-hdfs/sentry-hdfs-namenode-plugin/src/main/java/org/apache/sentry/hdfs/SentryINodeAttributesProvider.java cf96df47b sentry-provider/sentry-provider-common/src/main/java/org/apache/sentry/provider/common/HadoopGroupMappingService.java 00b5cf608 sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/generic/tools/PermissionsMigrationToolCommon.java ed28b735c sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/generic/tools/SentryConfigToolIndexer.java c2341d322 sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/generic/tools/SentryConfigToolSolr.java 5649f43fa sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/generic/tools/SentryShellGeneric.java 907e1462c sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/tools/SentryShellHive.java 729a51865 sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/service/thrift/SentryService.java bf5d85b03 sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/tools/CreateSentryTestScaleData.java b234c85c5 sentry-tools/src/main/java/org/apache/sentry/shell/SentryCli.java 8b68d0d06 Diff: https://reviews.apache.org/r/66373/diff/2/ Testing --- $ mvn -f sentry-binding/pom.xml test $ mvn -f sentry-provider/pom.xml test $ mvn -f sentry-hdfs/pom.xml test Thanks, Arjun Mishra
Re: Review Request 66373: SENTRY-2194: Upgrade Sentry hadoop-version dependency to 2.7.5
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/66373/ --- (Updated April 2, 2018, 3:31 p.m.) Review request for sentry, Alexander Kolbasov, Brian Towles, kalyan kumar kalvagadda, Na Li, Steve Moist, Sergio Pena, Vadim Spector, and Xinran Tinney. Changes --- Updated the commit message Repository: sentry Description --- Hadoop 2.7.5 version has many new features and bug fixes that Sentry can take advantage of. Currently Sentry is using 2.7.2 hadoop-version. We should bump this to 2.7.5 Diffs (updated) - pom.xml 61e0f9700 sentry-binding/sentry-binding-hbase-indexer/src/main/java/org/apache/sentry/binding/hbaseindexer/authz/HBaseIndexerAuthzBinding.java d919fe702 sentry-binding/sentry-binding-hbase-indexer/src/main/java/org/apache/sentry/binding/hbaseindexer/conf/HBaseIndexerAuthzConf.java cfbd37bf1 sentry-binding/sentry-binding-hive-conf/src/main/java/org/apache/sentry/binding/hive/conf/HiveAuthzConf.java 8d5286cd3 sentry-binding/sentry-binding-kafka/src/main/java/org/apache/sentry/kafka/conf/KafkaAuthConf.java 6ca621022 sentry-binding/sentry-binding-solr/src/main/java/org/apache/sentry/binding/solr/conf/SolrAuthzConf.java 0883e70fe sentry-binding/sentry-binding-sqoop/src/main/java/org/apache/sentry/sqoop/conf/SqoopAuthConf.java 7836871f6 sentry-hdfs/sentry-hdfs-namenode-plugin/src/main/java/org/apache/sentry/hdfs/SentryINodeAttributesProvider.java cf96df47b sentry-provider/sentry-provider-common/src/main/java/org/apache/sentry/provider/common/HadoopGroupMappingService.java 00b5cf608 sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/generic/tools/PermissionsMigrationToolCommon.java ed28b735c sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/generic/tools/SentryConfigToolIndexer.java c2341d322 sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/generic/tools/SentryConfigToolSolr.java 5649f43fa sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/generic/tools/SentryShellGeneric.java 907e1462c sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/tools/SentryShellHive.java 729a51865 sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/service/thrift/SentryService.java bf5d85b03 sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/tools/CreateSentryTestScaleData.java b234c85c5 sentry-tools/src/main/java/org/apache/sentry/shell/SentryCli.java 8b68d0d06 Diff: https://reviews.apache.org/r/66373/diff/2/ Changes: https://reviews.apache.org/r/66373/diff/1-2/ Testing --- $ mvn -f sentry-binding/pom.xml test $ mvn -f sentry-provider/pom.xml test $ mvn -f sentry-hdfs/pom.xml test Thanks, Arjun Mishra
Re: Review Request 66373: SENTRY-2194: Upgrade Sentry hadoop-version dependency to 2.7.5
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/66373/ --- (Updated April 2, 2018, 3:25 p.m.) Review request for sentry, Alexander Kolbasov, Brian Towles, kalyan kumar kalvagadda, Na Li, Steve Moist, Sergio Pena, Vadim Spector, and Xinran Tinney. Changes --- Added tests done Repository: sentry Description --- Hadoop 2.7.5 version has many new features and bug fixes that Sentry can take advantage of. Currently Sentry is using 2.7.2 hadoop-version. We should bump this to 2.7.5 Diffs - pom.xml 61e0f9700 sentry-binding/sentry-binding-hbase-indexer/src/main/java/org/apache/sentry/binding/hbaseindexer/authz/HBaseIndexerAuthzBinding.java d919fe702 sentry-binding/sentry-binding-hbase-indexer/src/main/java/org/apache/sentry/binding/hbaseindexer/conf/HBaseIndexerAuthzConf.java cfbd37bf1 sentry-binding/sentry-binding-hive-conf/src/main/java/org/apache/sentry/binding/hive/conf/HiveAuthzConf.java 8d5286cd3 sentry-binding/sentry-binding-kafka/src/main/java/org/apache/sentry/kafka/conf/KafkaAuthConf.java 6ca621022 sentry-binding/sentry-binding-solr/src/main/java/org/apache/sentry/binding/solr/conf/SolrAuthzConf.java 0883e70fe sentry-binding/sentry-binding-sqoop/src/main/java/org/apache/sentry/sqoop/conf/SqoopAuthConf.java 7836871f6 sentry-hdfs/sentry-hdfs-namenode-plugin/src/main/java/org/apache/sentry/hdfs/SentryINodeAttributesProvider.java cf96df47b sentry-provider/sentry-provider-common/src/main/java/org/apache/sentry/provider/common/HadoopGroupMappingService.java 00b5cf608 sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/generic/tools/PermissionsMigrationToolCommon.java ed28b735c sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/generic/tools/SentryConfigToolIndexer.java c2341d322 sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/generic/tools/SentryConfigToolSolr.java 5649f43fa sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/generic/tools/SentryShellGeneric.java 907e1462c sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/tools/SentryShellHive.java 729a51865 sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/service/thrift/SentryService.java bf5d85b03 sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/tools/CreateSentryTestScaleData.java b234c85c5 sentry-tools/src/main/java/org/apache/sentry/shell/SentryCli.java 8b68d0d06 Diff: https://reviews.apache.org/r/66373/diff/1/ Testing (updated) --- $ mvn -f sentry-binding/pom.xml test $ mvn -f sentry-provider/pom.xml test $ mvn -f sentry-hdfs/pom.xml test Thanks, Arjun Mishra
Re: Review Request 66373: SENTRY-2194: Upgrade Sentry hadoop-version dependency to 2.7.5 to take advantage of security vulnerability fix
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/66373/ --- (Updated April 2, 2018, 1:59 p.m.) Review request for sentry, Alexander Kolbasov, Brian Towles, kalyan kumar kalvagadda, Na Li, Steve Moist, Sergio Pena, Vadim Spector, and Xinran Tinney. Changes --- Updated description Repository: sentry Description (updated) --- Hadoop 2.7.5 version has many new features and bug fixes that Sentry can take advantage of. Currently Sentry is using 2.7.2 hadoop-version. We should bump this to 2.7.5 Diffs - pom.xml 61e0f9700 sentry-binding/sentry-binding-hbase-indexer/src/main/java/org/apache/sentry/binding/hbaseindexer/authz/HBaseIndexerAuthzBinding.java d919fe702 sentry-binding/sentry-binding-hbase-indexer/src/main/java/org/apache/sentry/binding/hbaseindexer/conf/HBaseIndexerAuthzConf.java cfbd37bf1 sentry-binding/sentry-binding-hive-conf/src/main/java/org/apache/sentry/binding/hive/conf/HiveAuthzConf.java 8d5286cd3 sentry-binding/sentry-binding-kafka/src/main/java/org/apache/sentry/kafka/conf/KafkaAuthConf.java 6ca621022 sentry-binding/sentry-binding-solr/src/main/java/org/apache/sentry/binding/solr/conf/SolrAuthzConf.java 0883e70fe sentry-binding/sentry-binding-sqoop/src/main/java/org/apache/sentry/sqoop/conf/SqoopAuthConf.java 7836871f6 sentry-hdfs/sentry-hdfs-namenode-plugin/src/main/java/org/apache/sentry/hdfs/SentryINodeAttributesProvider.java cf96df47b sentry-provider/sentry-provider-common/src/main/java/org/apache/sentry/provider/common/HadoopGroupMappingService.java 00b5cf608 sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/generic/tools/PermissionsMigrationToolCommon.java ed28b735c sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/generic/tools/SentryConfigToolIndexer.java c2341d322 sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/generic/tools/SentryConfigToolSolr.java 5649f43fa sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/generic/tools/SentryShellGeneric.java 907e1462c sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/tools/SentryShellHive.java 729a51865 sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/service/thrift/SentryService.java bf5d85b03 sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/tools/CreateSentryTestScaleData.java b234c85c5 sentry-tools/src/main/java/org/apache/sentry/shell/SentryCli.java 8b68d0d06 Diff: https://reviews.apache.org/r/66373/diff/1/ Testing --- Thanks, Arjun Mishra
Re: Review Request 66373: SENTRY-2194: Upgrade Sentry hadoop-version dependency to 2.7.5 to take advantage of security vulnerability fix
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/66373/#review200248 --- Ship it! Ship It! - Na Li On March 30, 2018, 7:05 a.m., Arjun Mishra wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/66373/ > --- > > (Updated March 30, 2018, 7:05 a.m.) > > > Review request for sentry, Alexander Kolbasov, Brian Towles, kalyan kumar > kalvagadda, Na Li, Steve Moist, Sergio Pena, Vadim Spector, and Xinran Tinney. > > > Repository: sentry > > > Description > --- > > MapReduce team had discovered a security vulnerability in parsing Map > Reduce-Job History Server configuration. That private files owned by a user > running the cluster, could be exposed. This has been applied to versions - > 2.7.5, 2.8.3, 2.9.0, or 3.0.0. Since Sentry uses hadoop-common Configuration > class to parse xml files, this change can be accommodated by our produce as > well. Sentry upstream is currently using 2.7.2 hadoop.version and we should > bump up this version to 2.7.5 to take advantage of this feature. > > The hadoop change involves adding a new boolean attribute restrictParser. > Setting restrictParser to true will > > Limit XML parsing to conform with feature > "http://apache.org/xml/features/disallow-doctype-decl; > This is a security feature explained here - > https://www.owasp.org/index.php/XML_External_Entity_(XXE)_Prevention_Cheat_Sheet > boolean restrictSystemProps is set to true > Will prevent system properties from being read > set XML inclusion (XInclude) to false > prevent merging of xml documents > With this change on hadoop side, only default resources, and hadoop-site.xml > have this feature turned off, so they will be read without restricted > parsing. Sentry is not listed as a default resource and would therefore have > to explicitly have this property set to true. > > > Diffs > - > > pom.xml 61e0f9700 > > sentry-binding/sentry-binding-hbase-indexer/src/main/java/org/apache/sentry/binding/hbaseindexer/authz/HBaseIndexerAuthzBinding.java > d919fe702 > > sentry-binding/sentry-binding-hbase-indexer/src/main/java/org/apache/sentry/binding/hbaseindexer/conf/HBaseIndexerAuthzConf.java > cfbd37bf1 > > sentry-binding/sentry-binding-hive-conf/src/main/java/org/apache/sentry/binding/hive/conf/HiveAuthzConf.java > 8d5286cd3 > > sentry-binding/sentry-binding-kafka/src/main/java/org/apache/sentry/kafka/conf/KafkaAuthConf.java > 6ca621022 > > sentry-binding/sentry-binding-solr/src/main/java/org/apache/sentry/binding/solr/conf/SolrAuthzConf.java > 0883e70fe > > sentry-binding/sentry-binding-sqoop/src/main/java/org/apache/sentry/sqoop/conf/SqoopAuthConf.java > 7836871f6 > > sentry-hdfs/sentry-hdfs-namenode-plugin/src/main/java/org/apache/sentry/hdfs/SentryINodeAttributesProvider.java > cf96df47b > > sentry-provider/sentry-provider-common/src/main/java/org/apache/sentry/provider/common/HadoopGroupMappingService.java > 00b5cf608 > > sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/generic/tools/PermissionsMigrationToolCommon.java > ed28b735c > > sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/generic/tools/SentryConfigToolIndexer.java > c2341d322 > > sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/generic/tools/SentryConfigToolSolr.java > 5649f43fa > > sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/generic/tools/SentryShellGeneric.java > 907e1462c > > sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/tools/SentryShellHive.java > 729a51865 > > sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/service/thrift/SentryService.java > bf5d85b03 > > sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/tools/CreateSentryTestScaleData.java > b234c85c5 > sentry-tools/src/main/java/org/apache/sentry/shell/SentryCli.java 8b68d0d06 > > > Diff: https://reviews.apache.org/r/66373/diff/1/ > > > Testing > --- > > > Thanks, > > Arjun Mishra > >
