[
https://issues.apache.org/jira/browse/HIVE-20545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16644514#comment-16644514
]
anishek commented on HIVE-20545:
--------------------------------
Thanks [~vihangk1] for the details. I wanted to understand some more on the
thirft object before/after serialization, Since DBNotificationListener is on
the metastore, there should not be additional serialization of thrift objets
there, however as the json grows larger definitely there is additional overhead
of space on rdbms + in memory to keep them, which i think would be good to
reduce.
So the serialization i assume you are mentioning is the json serialization of
the java objects? There is another effort we are looking at via HIVE-20679 to
see if we can have zipped message in db to reduce the network transfer and
rdbms overhead of these messages.
Just out of curiosity these stats i assume will help impala to better plan the
query ? if yes how will the same work on target warehouse after replication ?
> Ability to exclude potentially large parameters in HMS Notifications
> --------------------------------------------------------------------
>
> Key: HIVE-20545
> URL: https://issues.apache.org/jira/browse/HIVE-20545
> Project: Hive
> Issue Type: Improvement
> Components: Metastore
> Affects Versions: 3.1.0, 4.0.0
> Reporter: Bharathkrishna Guruvayoor Murali
> Assignee: Bharathkrishna Guruvayoor Murali
> Priority: Major
> Attachments: HIVE-20545.1.patch, HIVE-20545.2.patch,
> HIVE-20545.3.branch-3.patch, HIVE-20545.3.patch, HIVE-20545.4.patch,
> HIVE-20545.6.patch, HIVE-20545.7.patch
>
>
> Clients can add large-sized parameters in Table/Partition objects. So we need
> to enable adding regex patterns through HiveConf to match parameters to be
> filtered from table and partition objects before serialization in HMS
> notifications.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
