[
https://issues.apache.org/jira/browse/SENTRY-872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15308437#comment-15308437
]
Sravya Tirukkovalur commented on SENTRY-872:
--------------------------------------------
Thanks for uploading the updated deisgn doc [~cmccabe]! Some comments:
1. In Section "HIVE7973: Hive Replication Support ", seems like there is some
text missing at the end.
2. In Section "Future work", "The HDFS Plugin Should Use Update Log IDs". In
current design, we apply deltas in the NN plugin. I do not believe we
necessarily buffer deltas in NN, as there is no reason. So we may want to
remove this section.
3. We might want to add a section about "Sentry passive" which follows active
versus "Sentry standby" which warms up only when it acquires leadership? I
think we are inclining towards a passive which can serve requests with minimal
downtime, that is acquiring leadership should not take too long. But might be
better if we state it explicitly, so that we evaluate the alternatives
thoroughly?
4. There are some slight alternatives we might want to consider in the path of
propagating HMS updates to Sentry and NN. In the proposed design, we will need
to replicate HMS <obj,path> information as well as delta changes of
it(add/delete <ob,path>) in Sentry db for the passive to follow. Other option
is for passive to directly talk to HMS to get these deltas. If the only
motivation for replicating this in sentry db is bringing passive upto speed, I
think the later approach is preferable as there is no real need to replicate
both info and deltas? But, other parameter to consider is around full update.
That is, when Sentry restarts in the later approach, we will have to trigger a
full update from HMS. But without a proper snapshot solution in HMS, this would
mean we will have to lock HMS writes for this period, which means HMS is not
available for writes for this period.
5. Would be useful to have a detailed protocol description especially around
what happens when different services restart, and what in memory state does
each service rely on.
Let me know what you think and we can update the doc accordingly. Thanks!
> Uber jira for HMS HA + Sentry HA with HDFS plugin improvements
> --------------------------------------------------------------
>
> Key: SENTRY-872
> URL: https://issues.apache.org/jira/browse/SENTRY-872
> Project: Sentry
> Issue Type: Improvement
> Components: Hdfs Plugin
> Affects Versions: 1.5.0
> Reporter: Sravya Tirukkovalur
> Assignee: Sravya Tirukkovalur
> Fix For: 1.8.0
>
> Attachments: SENTRY-872.0.patch, SENTRY-872.pdf, SENTRY-872_design.pdf
>
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
