[
https://issues.apache.org/jira/browse/SENTRY-1964?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16181561#comment-16181561
]
Alexander Kolbasov commented on SENTRY-1964:
--------------------------------------------
The idea is pretty simple. From HDFS sync standpoint we have either DB or Table
permissions which usually map to a whole subdir under either db prefix or table
prefix. SO there is no need to ship all individual partition paths from HMS to
Sentry to Namenode since Namenode can just recursively apply ACLs for anything
under the DB or Table prefix.
This is a huge saving in both time and memory for setups with large number of
tables/partitions.
> Do not save and send partition to HDFS
> --------------------------------------
>
> Key: SENTRY-1964
> URL: https://issues.apache.org/jira/browse/SENTRY-1964
> Project: Sentry
> Issue Type: Improvement
> Components: Sentry
> Affects Versions: 2.0.0
> Reporter: Na Li
> Assignee: Na Li
> Priority: Critical
>
> Right now, sentry saves partition info from HMS and send it to HDFS. HDFS
> only needs database and table info, and does not need partition info for ACL
> unless the partion location is not sharing the same prefix of its table.
> The partition data amount is huge, and causes performance issue. We can
> optimize it by not saving and not sending partition info if it shares the
> same path of its table.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
