[
https://issues.apache.org/jira/browse/RANGER-2128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16524935#comment-16524935
]
Kent Yao commented on RANGER-2128:
----------------------------------
[~bosco] spark-authorizer works for spark2.1 and later, this pull request is
compatible with *2.2 and later * for supporting spark.sql.extensions
configuration.
I had it tested with my ranger 0.5-based personal release version against
2.2.1, and works fine.
START
{code:java}
bin/spark-shell --proxy-user hzyaoqin --conf
spark.sql.warehouse.dir=/user/hzyaoqin/warehouse --conf
spark.sql.extensions=org.apache.ranger.authorization.spark.authorizer.RangerSparkSQLExtension
{code}
TEST
{code:java}
scala> sc.setLogLevel("info");
scala> sql("show tables").show
{code}
LOG Snippet
{code:java}
18/06/27 19:11:43 INFO RangerPluginClassLoaderUtil:
getFilesInDirectory('/home/hadoop/spark-2.2.1-bin-hadoop2.7/jars/ranger-hive-plugin-impl'):
adding
/home/hadoop/spark-2.2.1-bin-hadoop2.7/jars/ranger-hive-plugin-impl/ranger-plugins-cred-0.5-mammut416.jar
18/06/27 19:11:43 INFO RangerPluginClassLoaderUtil:
getFilesInDirectory('/home/hadoop/spark-2.2.1-bin-hadoop2.7/jars/ranger-hive-plugin-impl'):
adding
/home/hadoop/spark-2.2.1-bin-hadoop2.7/jars/ranger-hive-plugin-impl/jersey-client-1.14.jar
18/06/27 19:11:43 INFO RangerPluginClassLoaderUtil:
getFilesInDirectory('/home/hadoop/spark-2.2.1-bin-hadoop2.7/jars/ranger-hive-plugin-impl'):
adding
/home/hadoop/spark-2.2.1-bin-hadoop2.7/jars/ranger-hive-plugin-impl/eclipselink-2.5.2.jar
18/06/27 19:11:43 INFO RangerPluginClassLoaderUtil:
getFilesInDirectory('/home/hadoop/spark-2.2.1-bin-hadoop2.7/jars/ranger-hive-plugin-impl'):
adding
/home/hadoop/spark-2.2.1-bin-hadoop2.7/jars/ranger-hive-plugin-impl/gson-2.2.4.jar
18/06/27 19:11:43 INFO RangerPluginClassLoaderUtil:
getFilesInDirectory('/home/hadoop/spark-2.2.1-bin-hadoop2.7/jars/ranger-hive-plugin-impl'):
adding
/home/hadoop/spark-2.2.1-bin-hadoop2.7/jars/ranger-hive-plugin-impl/ranger-plugins-common-0.5-mammut416.jar
18/06/27 19:11:43 INFO RangerPluginClassLoaderUtil:
getFilesInDirectory('/home/hadoop/spark-2.2.1-bin-hadoop2.7/jars/ranger-hive-plugin-impl'):
adding
/home/hadoop/spark-2.2.1-bin-hadoop2.7/jars/ranger-hive-plugin-impl/ranger-hive-plugin-0.5-mammut416.jar
18/06/27 19:11:43 INFO RangerPluginClassLoaderUtil:
getFilesInDirectory('/home/hadoop/spark-2.2.1-bin-hadoop2.7/jars/ranger-hive-plugin-impl'):
adding
/home/hadoop/spark-2.2.1-bin-hadoop2.7/jars/ranger-hive-plugin-impl/httpcore-4.4.6.jar
18/06/27 19:11:43 INFO RangerPluginClassLoaderUtil:
getFilesInDirectory('/home/hadoop/spark-2.2.1-bin-hadoop2.7/jars/ranger-hive-plugin-impl'):
adding
/home/hadoop/spark-2.2.1-bin-hadoop2.7/jars/ranger-hive-plugin-impl/ranger-plugins-audit-0.5-mammut416.jar
18/06/27 19:11:43 INFO RangerPluginClassLoaderUtil:
getFilesInDirectory('/home/hadoop/spark-2.2.1-bin-hadoop2.7/jars/ranger-hive-plugin-impl'):
adding
/home/hadoop/spark-2.2.1-bin-hadoop2.7/jars/ranger-hive-plugin-impl/solr-solrj-5.5.5.jar
18/06/27 19:11:43 INFO RangerPluginClassLoaderUtil:
getFilesInDirectory('/home/hadoop/spark-2.2.1-bin-hadoop2.7/jars/ranger-hive-plugin-impl'):
adding
/home/hadoop/spark-2.2.1-bin-hadoop2.7/jars/ranger-hive-plugin-impl/noggit-0.6.jar
18/06/27 19:11:43 INFO RangerPluginClassLoaderUtil:
getFilesInDirectory('/home/hadoop/spark-2.2.1-bin-hadoop2.7/jars/ranger-hive-plugin-impl'):
adding
/home/hadoop/spark-2.2.1-bin-hadoop2.7/jars/ranger-hive-plugin-impl/httpclient-4.5.3.jar
18/06/27 19:11:43 INFO RangerPluginClassLoaderUtil:
getFilesInDirectory('/home/hadoop/spark-2.2.1-bin-hadoop2.7/jars/ranger-hive-plugin-impl'):
adding
/home/hadoop/spark-2.2.1-bin-hadoop2.7/jars/ranger-hive-plugin-impl/mysql-connector-java-5.1.39.jar
18/06/27 19:11:43 INFO RangerPluginClassLoaderUtil:
getFilesInDirectory('/home/hadoop/spark-2.2.1-bin-hadoop2.7/jars/ranger-hive-plugin-impl'):
adding
/home/hadoop/spark-2.2.1-bin-hadoop2.7/jars/ranger-hive-plugin-impl/httpmime-4.5.3.jar
18/06/27 19:11:43 INFO RangerPluginClassLoaderUtil:
getFilesInDirectory('/home/hadoop/spark-2.2.1-bin-hadoop2.7/jars/ranger-hive-plugin-impl'):
adding
/home/hadoop/spark-2.2.1-bin-hadoop2.7/jars/ranger-hive-plugin-impl/ranger_solrj-0.5-mammut416.jar
18/06/27 19:11:43 INFO RangerPluginClassLoaderUtil:
getFilesInDirectory('/home/hadoop/spark-2.2.1-bin-hadoop2.7/jars/ranger-hive-plugin-impl'):
adding
/home/hadoop/spark-2.2.1-bin-hadoop2.7/jars/ranger-hive-plugin-impl/javax.persistence-2.1.0.jar
18/06/27 19:11:43 INFO RangerPluginClassLoaderUtil:
getFilesInDirectory('/home/hadoop/spark-2.2.1-bin-hadoop2.7/jars/ranger-hive-plugin-impl'):
adding
/home/hadoop/spark-2.2.1-bin-hadoop2.7/jars/ranger-hive-plugin-impl/mybatis-3.2.8.jar
{code}
PS, I'd like to know the latest version which supports hive1.2.1 by
ranger-hive-plugin to build spark-plugin assembly
Thanks
> Implement SparkSQL plugin
> -------------------------
>
> Key: RANGER-2128
> URL: https://issues.apache.org/jira/browse/RANGER-2128
> Project: Ranger
> Issue Type: New Feature
> Components: plugins, Ranger
> Affects Versions: 1.1.0
> Reporter: t oo
> Assignee: Kent Yao
> Priority: Major
> Fix For: 1.1.0
>
>
> Implement SparkSQL plugin
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
