[
https://issues.apache.org/jira/browse/SUBMARINE-834?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17348117#comment-17348117
]
Adesh Kumar Rao commented on SUBMARINE-834:
-------------------------------------------
[~Qin Yao] Let me know if you have any doubts/questions regarding the idea.
> Ranger support for Spark Datasources
> ------------------------------------
>
> Key: SUBMARINE-834
> URL: https://issues.apache.org/jira/browse/SUBMARINE-834
> Project: Apache Submarine
> Issue Type: New Feature
> Components: Security
> Reporter: Adesh Kumar Rao
> Priority: Major
>
> Currently, the spark-security module is tightly coupled with
> "HiveTableRelation" and "MetastoreRelation".
> This does not work for Hive transactional (ACID enabled) table. Since Hive
> has changed the way data/metadata is stored for transactional tables when
> compared to non-transactional tables. Therefore, Spark can not read Hive
> transactional tables directly.
> So even if security module can enforce security on such tables, spark can't
> actually read anything.
>
> Reading hive's transactional tables in spark, needs
> [Spark-Acid|https://github.com/qubole/spark-acid], implemented as Spark
> Datasource. Since security module is tightly coupled with "HiveTableRelation"
> and "MetastoreRelation", it does not provides authorization support for any
> datasource.
>
> The idea is to support Spark datasource authorization (which has
> db/table/column/partitions anologus, similar to hive).
>
> We can create generic interfaces for datasource, which each datasource can
> implement and then it can authorized using the existing codebase.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
