----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/70676/#review224787 -----------------------------------------------------------
plugin-spark/src/main/scala/org/apache/ranger/authorization/spark/authorizer/RangerSparkPlugin.scala Lines 33 (patched) <https://reviews.apache.org/r/70676/#comment313583> RangerSparkPlugin.scala:33: value getInstance is not a member of object org.apache.ranger.authorization.hadoop.config.RangerConfiguration plugin-spark/src/main/scala/org/apache/ranger/authorization/spark/authorizer/RangerSparkPlugin.scala Lines 57 (patched) <https://reviews.apache.org/r/70676/#comment313584> RangerSparkPlugin.scala:57: value getInstance is not a member of object org.apache.ranger.authorization.hadoop.config.RangerConfiguration - bhavik patel On May 18, 2019, 3:28 p.m., Kent Yao wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/70676/ > ----------------------------------------------------------- > > (Updated May 18, 2019, 3:28 p.m.) > > > Review request for ranger and Don Bosco Durai. > > > Bugs: https://issues.apache.org/jira/browse/RANGER-2128 > > https://issues.apache.org/jira/browse/https://issues.apache.org/jira/browse/RANGER-2128 > > > Repository: ranger > > > Description > ------- > > Ranger Spark SQL Plugin for Spark SQL/Dataframe/Dataset > > > Diffs > ----- > > .gitignore c2def9d > agents-audit/pom.xml 4aaec09 > agents-common/pom.xml 4486af3 > agents-cred/pom.xml c622683 > agents-installer/pom.xml 20902cb > credentialbuilder/pom.xml 461dcd0 > embeddedwebserver/pom.xml a8c7cba > hbase-agent/pom.xml c044ef0 > hive-agent/pom.xml 0a091b2 > jisql/pom.xml 07fb08b > kms/pom.xml 043c67f > knox-agent/pom.xml 0c45c84 > plugin-atlas/pom.xml 87ff40d > plugin-elasticsearch/pom.xml b997310 > plugin-kafka/pom.xml 132baf7 > plugin-kms/pom.xml 1940f41 > plugin-kylin/pom.xml 69badfc > plugin-nifi-registry/pom.xml 3e99f04 > plugin-nifi/pom.xml c510d87 > plugin-solr/pom.xml 619cc9e > plugin-spark/.gitignore PRE-CREATION > plugin-spark/pom.xml PRE-CREATION > > plugin-spark/src/main/scala/org/apache/ranger/authorization/spark/authorizer/RangerSparkAccessRequest.scala > PRE-CREATION > > plugin-spark/src/main/scala/org/apache/ranger/authorization/spark/authorizer/RangerSparkAuditHandler.scala > PRE-CREATION > > plugin-spark/src/main/scala/org/apache/ranger/authorization/spark/authorizer/RangerSparkAuthorizer.scala > PRE-CREATION > > plugin-spark/src/main/scala/org/apache/ranger/authorization/spark/authorizer/RangerSparkPlugin.scala > PRE-CREATION > > plugin-spark/src/main/scala/org/apache/ranger/authorization/spark/authorizer/RangerSparkResource.scala > PRE-CREATION > > plugin-spark/src/main/scala/org/apache/ranger/authorization/spark/authorizer/RangerSparkSQLExtension.scala > PRE-CREATION > > plugin-spark/src/main/scala/org/apache/ranger/authorization/spark/authorizer/SparkAccessControlException.scala > PRE-CREATION > > plugin-spark/src/main/scala/org/apache/ranger/authorization/spark/authorizer/SparkAccessType.scala > PRE-CREATION > > plugin-spark/src/main/scala/org/apache/ranger/authorization/spark/authorizer/SparkObjectType.scala > PRE-CREATION > > plugin-spark/src/main/scala/org/apache/ranger/authorization/spark/authorizer/SparkOperationType.scala > PRE-CREATION > > plugin-spark/src/main/scala/org/apache/ranger/authorization/spark/authorizer/SparkPrivObjectActionType.scala > PRE-CREATION > > plugin-spark/src/main/scala/org/apache/ranger/authorization/spark/authorizer/SparkPrivilegeObject.scala > PRE-CREATION > > plugin-spark/src/main/scala/org/apache/ranger/authorization/spark/authorizer/SparkPrivilegeObjectType.scala > PRE-CREATION > > plugin-spark/src/main/scala/org/apache/ranger/authorization/spark/authorizer/authorizer.scala > PRE-CREATION > plugin-spark/src/main/scala/org/apache/spark/sql/AuthzUtils.scala > PRE-CREATION > > plugin-spark/src/main/scala/org/apache/spark/sql/catalyst/optimizer/RangerSparkAuthorizerExtension.scala > PRE-CREATION > > plugin-spark/src/main/scala/org/apache/spark/sql/catalyst/optimizer/RangerSparkMaskingExtension.scala > PRE-CREATION > > plugin-spark/src/main/scala/org/apache/spark/sql/catalyst/optimizer/RangerSparkOptimizer.scala > PRE-CREATION > > plugin-spark/src/main/scala/org/apache/spark/sql/catalyst/optimizer/RangerSparkRowFilterExtension.scala > PRE-CREATION > > plugin-spark/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/RangerSparkMasking.scala > PRE-CREATION > > plugin-spark/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/RangerSparkRowFilter.scala > PRE-CREATION > > plugin-spark/src/main/scala/org/apache/spark/sql/execution/RangerShowDatabasesCommand.scala > PRE-CREATION > > plugin-spark/src/main/scala/org/apache/spark/sql/execution/RangerShowTablesCommand.scala > PRE-CREATION > > plugin-spark/src/main/scala/org/apache/spark/sql/execution/RangerSparkPlanOmitStrategy.scala > PRE-CREATION > > plugin-spark/src/main/scala/org/apache/spark/sql/hive/PrivilegesBuilder.scala > PRE-CREATION > plugin-spark/src/test/resources/data/files/kv1.txt PRE-CREATION > plugin-spark/src/test/resources/log4j.properties PRE-CREATION > plugin-spark/src/test/resources/ranger-spark-security.xml PRE-CREATION > plugin-spark/src/test/resources/sparkSql_hive_jenkins.json PRE-CREATION > plugin-spark/src/test/resources/tpcds/q1.sql PRE-CREATION > plugin-spark/src/test/resources/tpcds/q10.sql PRE-CREATION > plugin-spark/src/test/resources/tpcds/q11.sql PRE-CREATION > plugin-spark/src/test/resources/tpcds/q12.sql PRE-CREATION > plugin-spark/src/test/resources/tpcds/q13.sql PRE-CREATION > plugin-spark/src/test/resources/tpcds/q14a.sql PRE-CREATION > plugin-spark/src/test/resources/tpcds/q14b.sql PRE-CREATION > plugin-spark/src/test/resources/tpcds/q15.sql PRE-CREATION > plugin-spark/src/test/resources/tpcds/q16.sql PRE-CREATION > plugin-spark/src/test/resources/tpcds/q17.sql PRE-CREATION > plugin-spark/src/test/resources/tpcds/q18.sql PRE-CREATION > plugin-spark/src/test/resources/tpcds/q19.sql PRE-CREATION > plugin-spark/src/test/resources/tpcds/q2.sql PRE-CREATION > plugin-spark/src/test/resources/tpcds/q20.sql PRE-CREATION > plugin-spark/src/test/resources/tpcds/q21.sql PRE-CREATION > plugin-spark/src/test/resources/tpcds/q22.sql PRE-CREATION > plugin-spark/src/test/resources/tpcds/q23a.sql PRE-CREATION > plugin-spark/src/test/resources/tpcds/q23b.sql PRE-CREATION > plugin-spark/src/test/resources/tpcds/q24a.sql PRE-CREATION > plugin-spark/src/test/resources/tpcds/q24b.sql PRE-CREATION > plugin-spark/src/test/resources/tpcds/q25.sql PRE-CREATION > plugin-spark/src/test/resources/tpcds/q26.sql PRE-CREATION > plugin-spark/src/test/resources/tpcds/q27.sql PRE-CREATION > plugin-spark/src/test/resources/tpcds/q28.sql PRE-CREATION > plugin-spark/src/test/resources/tpcds/q29.sql PRE-CREATION > plugin-spark/src/test/resources/tpcds/q3.sql PRE-CREATION > plugin-spark/src/test/resources/tpcds/q30.sql PRE-CREATION > plugin-spark/src/test/resources/tpcds/q31.sql PRE-CREATION > plugin-spark/src/test/resources/tpcds/q32.sql PRE-CREATION > plugin-spark/src/test/resources/tpcds/q33.sql PRE-CREATION > plugin-spark/src/test/resources/tpcds/q34.sql PRE-CREATION > plugin-spark/src/test/resources/tpcds/q35.sql PRE-CREATION > plugin-spark/src/test/resources/tpcds/q36.sql PRE-CREATION > plugin-spark/src/test/resources/tpcds/q37.sql PRE-CREATION > plugin-spark/src/test/resources/tpcds/q38.sql PRE-CREATION > plugin-spark/src/test/resources/tpcds/q39a.sql PRE-CREATION > plugin-spark/src/test/resources/tpcds/q39b.sql PRE-CREATION > plugin-spark/src/test/resources/tpcds/q4.sql PRE-CREATION > plugin-spark/src/test/resources/tpcds/q40.sql PRE-CREATION > plugin-spark/src/test/resources/tpcds/q41.sql PRE-CREATION > plugin-spark/src/test/resources/tpcds/q42.sql PRE-CREATION > plugin-spark/src/test/resources/tpcds/q43.sql PRE-CREATION > plugin-spark/src/test/resources/tpcds/q44.sql PRE-CREATION > plugin-spark/src/test/resources/tpcds/q45.sql PRE-CREATION > plugin-spark/src/test/resources/tpcds/q46.sql PRE-CREATION > plugin-spark/src/test/resources/tpcds/q47.sql PRE-CREATION > plugin-spark/src/test/resources/tpcds/q48.sql PRE-CREATION > plugin-spark/src/test/resources/tpcds/q49.sql PRE-CREATION > plugin-spark/src/test/resources/tpcds/q5.sql PRE-CREATION > plugin-spark/src/test/resources/tpcds/q50.sql PRE-CREATION > plugin-spark/src/test/resources/tpcds/q51.sql PRE-CREATION > plugin-spark/src/test/resources/tpcds/q52.sql PRE-CREATION > plugin-spark/src/test/resources/tpcds/q53.sql PRE-CREATION > plugin-spark/src/test/resources/tpcds/q54.sql PRE-CREATION > plugin-spark/src/test/resources/tpcds/q55.sql PRE-CREATION > plugin-spark/src/test/resources/tpcds/q56.sql PRE-CREATION > plugin-spark/src/test/resources/tpcds/q57.sql PRE-CREATION > plugin-spark/src/test/resources/tpcds/q58.sql PRE-CREATION > plugin-spark/src/test/resources/tpcds/q59.sql PRE-CREATION > plugin-spark/src/test/resources/tpcds/q6.sql PRE-CREATION > plugin-spark/src/test/resources/tpcds/q60.sql PRE-CREATION > plugin-spark/src/test/resources/tpcds/q61.sql PRE-CREATION > plugin-spark/src/test/resources/tpcds/q62.sql PRE-CREATION > plugin-spark/src/test/resources/tpcds/q63.sql PRE-CREATION > plugin-spark/src/test/resources/tpcds/q64.sql PRE-CREATION > plugin-spark/src/test/resources/tpcds/q65.sql PRE-CREATION > plugin-spark/src/test/resources/tpcds/q66.sql PRE-CREATION > plugin-spark/src/test/resources/tpcds/q67.sql PRE-CREATION > plugin-spark/src/test/resources/tpcds/q68.sql PRE-CREATION > plugin-spark/src/test/resources/tpcds/q69.sql PRE-CREATION > plugin-spark/src/test/resources/tpcds/q7.sql PRE-CREATION > plugin-spark/src/test/resources/tpcds/q70.sql PRE-CREATION > plugin-spark/src/test/resources/tpcds/q71.sql PRE-CREATION > plugin-spark/src/test/resources/tpcds/q72.sql PRE-CREATION > plugin-spark/src/test/resources/tpcds/q73.sql PRE-CREATION > plugin-spark/src/test/resources/tpcds/q74.sql PRE-CREATION > plugin-spark/src/test/resources/tpcds/q75.sql PRE-CREATION > plugin-spark/src/test/resources/tpcds/q76.sql PRE-CREATION > plugin-spark/src/test/resources/tpcds/q77.sql PRE-CREATION > plugin-spark/src/test/resources/tpcds/q78.sql PRE-CREATION > plugin-spark/src/test/resources/tpcds/q79.sql PRE-CREATION > plugin-spark/src/test/resources/tpcds/q8.sql PRE-CREATION > plugin-spark/src/test/resources/tpcds/q80.sql PRE-CREATION > plugin-spark/src/test/resources/tpcds/q81.sql PRE-CREATION > plugin-spark/src/test/resources/tpcds/q82.sql PRE-CREATION > plugin-spark/src/test/resources/tpcds/q83.sql PRE-CREATION > plugin-spark/src/test/resources/tpcds/q84.sql PRE-CREATION > plugin-spark/src/test/resources/tpcds/q85.sql PRE-CREATION > plugin-spark/src/test/resources/tpcds/q86.sql PRE-CREATION > plugin-spark/src/test/resources/tpcds/q87.sql PRE-CREATION > plugin-spark/src/test/resources/tpcds/q88.sql PRE-CREATION > plugin-spark/src/test/resources/tpcds/q89.sql PRE-CREATION > plugin-spark/src/test/resources/tpcds/q9.sql PRE-CREATION > plugin-spark/src/test/resources/tpcds/q90.sql PRE-CREATION > plugin-spark/src/test/resources/tpcds/q91.sql PRE-CREATION > plugin-spark/src/test/resources/tpcds/q92.sql PRE-CREATION > plugin-spark/src/test/resources/tpcds/q93.sql PRE-CREATION > plugin-spark/src/test/resources/tpcds/q94.sql PRE-CREATION > plugin-spark/src/test/resources/tpcds/q95.sql PRE-CREATION > plugin-spark/src/test/resources/tpcds/q96.sql PRE-CREATION > plugin-spark/src/test/resources/tpcds/q97.sql PRE-CREATION > plugin-spark/src/test/resources/tpcds/q98.sql PRE-CREATION > plugin-spark/src/test/resources/tpcds/q99.sql PRE-CREATION > > plugin-spark/src/test/scala/org/apache/ranger/services/spark/RangerAdminClientImpl.scala > PRE-CREATION > > plugin-spark/src/test/scala/org/apache/ranger/services/spark/SparkRangerAuthorizerTest.scala > PRE-CREATION > plugin-spark/src/test/scala/org/apache/spark/sql/RangerSparkTestUtils.scala > PRE-CREATION > > plugin-spark/src/test/scala/org/apache/spark/sql/catalyst/optimizer/RangerSparkAuthorizerExtensionTest.scala > PRE-CREATION > > plugin-spark/src/test/scala/org/apache/spark/sql/catalyst/optimizer/RangerSparkMaskingExtensionTest.scala > PRE-CREATION > > plugin-spark/src/test/scala/org/apache/spark/sql/catalyst/optimizer/RangerSparkRowFilterExtensionTest.scala > PRE-CREATION > > plugin-spark/src/test/scala/org/apache/spark/sql/execution/RangerSparkPlanOmitStrategyTest.scala > PRE-CREATION > plugin-sqoop/pom.xml af00e97 > plugin-yarn/pom.xml f273569 > pom.xml 2a2540a > ranger-atlas-plugin-shim/pom.xml 86d6a26 > ranger-elasticsearch-plugin-shim/pom.xml 59a653b > ranger-examples/pom.xml fa4cb20 > ranger-examples/sampleapp/pom.xml add9546 > ranger-hbase-plugin-shim/pom.xml 9ed0aeb > ranger-hdfs-plugin-shim/pom.xml f45621c > ranger-hive-plugin-shim/pom.xml 26258d7 > ranger-kafka-plugin-shim/pom.xml 5a7353f > ranger-kms-plugin-shim/pom.xml 0b505f6 > ranger-knox-plugin-shim/pom.xml db342b7 > ranger-kylin-plugin-shim/pom.xml 4bb9f34 > ranger-plugin-classloader/pom.xml 0aa71c3 > ranger-solr-plugin-shim/pom.xml 46b7821 > ranger-sqoop-plugin-shim/pom.xml 86ab432 > src/main/assembly/plugin-spark.xml PRE-CREATION > ugsync/pom.xml c30d8ca > unixauthclient/pom.xml 4625925 > unixauthpam/pom.xml 0f2a459 > > > Diff: https://reviews.apache.org/r/70676/diff/1/ > > > Testing > ------- > > https://issues.apache.org/jira/browse/RANGER-2128 > > ## UTs > add Unit Tests > add TPCDS 99 queries as unit tests. > > Can be verified via: > > 1. cd plugin-spark > 2. mvn clean package -Pspark-2.3 or mvn clean package -Pspark-2.4 > > ## Test manually > > 1. add the the shaded jar which can be found plugin-spark/target after mvn > package to SPARK_HOME/jars > 2. add/create ranger-spark-security.xml etc as we do for hive, then mv all > the plugin configuration file to SPARK_HOME/jars > 3. cd $SPARK_HOME > 4. run bin/spark-sql --conf > spark.sql.extensions=org.apache.ranger.authorization.spark.authorizer.RangerSparkSQLExtension > > **No need to configure HiveAuthorizerFactory etc in hive-site.xml** > > > Thanks, > > Kent Yao > >
