[
https://issues.apache.org/jira/browse/RANGER-2128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16544822#comment-16544822
]
Kent Yao commented on RANGER-2128:
----------------------------------
Emmm..weird, ClassCastException shouldn't occur if the hive client was
correctly replaced by reflecting. When I tried thrift server, I did not see
such an exception.
Although I don’t want to admit it, I have to say that the current
implementation of Spark thrift server is really a mess. To meet spark's
architecture, it was designed as an single spark application without multi
tenancy support.
Livy Server is multi tenant. It launches spark applications separately in
different JVMs, which is easy to work with our current work.
I guess that we may implement spark sql plugin step by step,
1. Resolve the hive client and jersey incompatibility
2. Implement plugin for spark client mode
3. Implement plugin for spark cluster mode
4. Separate metastore layer from PhysicalScan layer in Spark Thrift Server
5. Implement plugin for thrift server
By 1-3, I guess we can supply all real multi tenant servers for spark, for
spark thrift server, we may need to make some efforts to spark community to
support this such as step 4 and some related work.
> Implement SparkSQL plugin
> -------------------------
>
> Key: RANGER-2128
> URL: https://issues.apache.org/jira/browse/RANGER-2128
> Project: Ranger
> Issue Type: New Feature
> Components: plugins, Ranger
> Affects Versions: 1.1.0
> Reporter: t oo
> Assignee: Kent Yao
> Priority: Major
> Fix For: 2.0.0
>
> Attachments: support_ranger11.tgz
>
>
> Implement SparkSQL plugin
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
