[
https://issues.apache.org/jira/browse/HIVE-17990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16283516#comment-16283516
]
Peter Vary commented on HIVE-17990:
-----------------------------------
Hi [~alangates],
Thanks for working on this. I have taken a quick look at this patch, and have
some questions:
AFAIK the current MetaStore API is quite verbose - and because of the
limitations of Thrift - every object is serialized in the response as many
times as it is referenced in the result. This causes serious memory issues in
HS2 and/or HMS when running queries on tens of thousands of partitions.
If I understand the proposal correctly, this specific change adds even more
data to the response, and in most of the cases this data will be the same for
every partition in the request. This is true for the initial changes and even
more so for the eventual results.
I am totally positive for the Schema Registry integration, and if we could find
a way to not to increase the memory pressure, or we have plans for help this in
the future, I would be happy.
Also it would be nice to have unit tests for the new HMS API interface methods
in this jira, or one follow-up jira.
Please feel free to correct me if I am wrong in anything stated above.
Thanks,
Peter
> Add Thrift and DB storage for Schema Registry objects
> -----------------------------------------------------
>
> Key: HIVE-17990
> URL: https://issues.apache.org/jira/browse/HIVE-17990
> Project: Hive
> Issue Type: Sub-task
> Components: Standalone Metastore
> Reporter: Alan Gates
> Assignee: Alan Gates
> Attachments: Adding-Schema-Registry-to-Metastore.pdf
>
>
> This JIRA tracks changes to Thrift, RawStore, and DB scripts to support
> objects in the Schema Registry.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
