[
https://issues.apache.org/jira/browse/SENTRY-1325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15328704#comment-15328704
]
Colin Patrick McCabe commented on SENTRY-1325:
----------------------------------------------
We had an offline discussion about this and generally agreed that storing the
entire HMS state as a single large JSON string won't work very well in SQL. In
general, SQL databases are not optimized for writing extremely large keys, and
we can't really put an upper bound on the size of the HMS state. There may be
thousands of tables, each with thousands of partitions.
One solution would be to simply store this data in a "traditional" SQL schema,
and forget about JSON. This would require SQL migrations when the Hive schema
changed, of course.
Another approach would be to split up the SQL data into chunks. For example,
perhaps we could have a SQL blob for each Hive table. This would be a
manageable amount of data to store in the database.
> Store HMSPaths in Sentry DB to allow fast failover
> --------------------------------------------------
>
> Key: SENTRY-1325
> URL: https://issues.apache.org/jira/browse/SENTRY-1325
> Project: Sentry
> Issue Type: Sub-task
> Reporter: Hao Hao
> Assignee: Hao Hao
> Attachments: SENTRY-1325.0.patch
>
>
> HMSPaths represents a full HMS update. It is a tree structure hierarchy.
> Serialize/Deserialize HMSPaths to Json to persist into Sentry DB.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
