[
https://issues.apache.org/jira/browse/SENTRY-1916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16154626#comment-16154626
]
Vamsee Yarlagadda commented on SENTRY-1916:
-------------------------------------------
[~akolb] HDFS path prefixes is a property on HDFS side where HDFS can change
this setting and simply restarting NN and assumes all the relevant paths are
passed to it. If we put this check in HMSFollower and only get a subset of
snapshot and persist it, then every time this setting is changed on HDFS side,
we should also get a full snapshot from Hive with all the prefixes so this puts
more burden over time. Rather we can persist the complete data from HMS
(without filtering) and everytime the HDFS setting is changed, it only requires
sentry to refilter entries and pass it along to HDFS. Makes sense?
> Sentry should not store paths outside of the prefix
> ---------------------------------------------------
>
> Key: SENTRY-1916
> URL: https://issues.apache.org/jira/browse/SENTRY-1916
> Project: Sentry
> Issue Type: Bug
> Components: Sentry
> Affects Versions: 2.0.0
> Reporter: Alexander Kolbasov
> Assignee: Alexander Kolbasov
> Attachments: SENTRY-1916.01.patch
>
>
> Before Sentry 2.0 we were only sending paths which were inside Hive prefix to
> HDFS. With Sentry HA we changed that and store all paths. This significantly
> increases the amount of memory when there are many external tables.
> [~vamsee] [~spena] [[email protected]] [~hahao] [[email protected]]
> [~mcrocker] FYI
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
